Skip to content

[PR] kernel: let BLOCKED partners stop vetoing block merges via integrity gate (#6709) - #6731

Closed
ranxianglei wants to merge 4 commits into
masterfrom
2026-11-14_fix-6709-blocked-merge-integrity
Closed

ranxianglei wants to merge 4 commits into
masterfrom
2026-11-14_fix-6709-blocked-merge-integrity

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

kernel-change: BLOCKED (never-foldable) messages no longer block the merge integrity gate.

What broke

Merging old blocks (b312..b315, b312..b313) always skips when a turn partner has ref BLOCKED: #5919 made that an idempotent no-op, but a BLOCKED partner can never age into foldability (its body is gone permanently), so the no-op is a permanent deadlock: every merge touching that turn re-fails identically. The reporter's session (session-ad47cd30, 0.1.214) carries 6,115 BLOCKED messages (13.7% of 44,623); merges of old blocks all skip, and the context stays stuck at 533K/792K.

Root cause

computeIntegrityWithdrawals (kernel/src/turn-integrity.ts) treats a BLOCKED message as a regular "kept visible" message. When a merge covers the BLOCKED message's turn-mates, the integrity gate sees a split turn (reasoning folded + BLOCKED call visible) and withdraws the entire group. The #5919 fix caught this and turned it into a no-op, but the no-op is permanent because the BLOCKED message can never become foldable.

Fix

Add a neverFoldableIds parameter to computeIntegrityWithdrawals. When set, messages in this set are excluded from the "kept visible" check (INV1) and from the split-pair withdrawal (INV2). The merge can now proceed without the BLOCKED partner blocking the turn-integrity check.

The BLOCKED message itself is never included in the merge (it is never foldable), but its turn-mates can now be folded. This is different from PR #6719 (which includes BLOCKED messages in the merge coverage) — this approach keeps BLOCKED messages out of the block's coverage, which is safer for the compressedTokens accounting.

Changes

  • kernel/src/turn-integrity.ts: added neverFoldableIds parameter to computeIntegrityWithdrawals; INV1 skips BLOCKED calls in the "kept" check; INV2 skips withdrawal for BLOCKED pairs.
  • kernel/src/unit-projection.ts: compute neverFoldableIds from state.messageRefs.byRaw (ref==="BLOCKED") when isMerge; pass to all 3 merge call sites.
  • kernel/tests/issue5919-shredded-merge.test.ts: updated 5919-2 test to reflect new behavior (merge succeeds instead of no-op).
  • kernel/tests/issue4484-merge-coverage.test.ts: updated tests to pass neverFoldableIds to computeIntegrityWithdrawals.

Tests

  • All 1636 kernel tests pass.
  • Typecheck passes.
  • Build passes.

Closes #6709

@ranxianglei ranxianglei added bug Something isn't working p1 scope:kernel labels Nov 14, 2026
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-14_fix-6709-blocked-merge-integrity (878f64a)

Latest build — install this exact version

npm install -g billion-context@0.0.10007-pr.6731.e11159e

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
878f64a billion-context@0.0.10007-pr.6731.e11159e build
e2ea751 billion-context@0.0.10004-pr.6731.460769f build
09bf8ae billion-context@0.0.9993-pr.6731.577e705 build
805909e billion-context@0.0.9991-pr.6731.4890934 build
cb4b980 billion-context@0.0.9987-pr.6731.80033c7 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6731.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title kernel: let BLOCKED partners stop vetoing block merges via integrity gate (#6709) [PR] kernel: let BLOCKED partners stop vetoing block merges via integrity gate (#6709) Nov 14, 2026
…gate (#6709)

kernel-change: BLOCKED (never-foldable) messages no longer block the merge
integrity gate. The #5919 fix turned the fatal into a no-op, but a BLOCKED
partner can never age into foldability (its body is gone permanently), so
the no-op is a permanent deadlock: every merge touching that turn re-fails
identically. The fix adds a neverFoldableIds parameter to
computeIntegrityWithdrawals so the merge can proceed without the BLOCKED
partner blocking the turn-integrity check. The BLOCKED message itself is
never included in the merge (it is never foldable), but its turn-mates can
now be folded.

This is a different approach from PR #6719 (which includes BLOCKED messages
in the merge coverage). This approach keeps BLOCKED messages out of the
block's coverage, which is safer for the compressedTokens accounting.

Fixes #6709
@ranxianglei
ranxianglei force-pushed the 2026-11-14_fix-6709-blocked-merge-integrity branch from cb4b980 to 805909e Compare November 14, 2026 13:32
ranxianglei added a commit that referenced this pull request Nov 14, 2026
The #5919 merge no-op treats every ref='BLOCKED' partner as "cannot be
closed now", but that conflates two very different cases. A content-free
BLOCKED partner (body removed by native compression) carries no live wire
content and can never age into foldability, so letting it doom the merge
turns the no-op into a permanent deadlock: every merge covering its turn
re-hits the same skip and the session stays stuck at its context ceiling
(#6709, session-ad47cd30, 6,115 BLOCKED). A LIVE protected tool /
unrecoverable media is pinned to ref 'BLOCKED' by assignRefs too, with its
full body still in the wire — folding it in drops protected content and
leaving it split emits an orphaned tool_call/tool_result.

Exempt only content-free BLOCKED partners (ref BLOCKED AND not
live-protected-tool AND not host-protected AND not unrecoverable-media)
from the merge-deadlock; they flow to the #5919 rung-2 fallback and are
folded into the merged block, closing the turn. Live protected tools /
media still doom the merge into the honest no-op.

This is the correct superset of the two related PRs (#6719: flow-all-
BLOCKED-to-rung2; #6731: key-on-ref-BLOCKED, keep-split) — both keyed on
ref==='BLOCKED' alone, and both masked the live-protected case by giving
the protected partner a numeric ref in their regression tests. 5919-5 adds
that guard (live protected tool stored as ref BLOCKED still no-ops).

Refs: #6709, #6719, #6731
ranxianglei added a commit that referenced this pull request Nov 14, 2026
The #5919 merge no-op treats every ref='BLOCKED' partner as "cannot be
closed now", but that conflates two very different cases. A content-free
BLOCKED partner (body removed by native compression) carries no live wire
content and can never age into foldability, so letting it doom the merge
turns the no-op into a permanent deadlock: every merge covering its turn
re-hits the same skip and the session stays stuck at its context ceiling
(#6709, session-ad47cd30, 6,115 BLOCKED). A LIVE protected tool /
unrecoverable media is pinned to ref 'BLOCKED' by assignRefs too, with its
full body still in the wire — folding it in drops protected content and
leaving it split emits an orphaned tool_call/tool_result.

Exempt only content-free BLOCKED partners (ref BLOCKED AND not
live-protected-tool AND not host-protected AND not unrecoverable-media)
from the merge-deadlock; they flow to the #5919 rung-2 fallback and are
folded into the merged block, closing the turn. Live protected tools /
media still doom the merge into the honest no-op.

This is the correct superset of the two related PRs (#6719: flow-all-
BLOCKED-to-rung2; #6731: key-on-ref-BLOCKED, keep-split) — both keyed on
ref==='BLOCKED' alone, and both masked the live-protected case by giving
the protected partner a numeric ref in their regression tests. 5919-5 adds
that guard (live protected tool stored as ref BLOCKED still no-ops).

Refs: #6709, #6719, #6731
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Duplicate screening: 查重: 同 issue (#6709) 存在平行 PR #6719,两者方案不同——#6719 把 BLOCKED 伙伴折进合并块(纳入 coverage),#6731 让 BLOCKED 伙伴不挡完整性门(排除 coverage)。两者互斥,需 owner 拍板选哪个。本 PR (#6731) 的 compressedTokens 记账更安全(BLOCKED 不进 coverage)。

PR #6731 review (kernel change, closes #6709)

Kernel Boundary compliance:

  • ✅ kernel-change: declaration present in PR body
  • ✅ Kernel version bumped 0.0.197 → 0.0.198 (2nd commit)

Fix logic (verified by reading code + running targeted tests):

  • kernel/src/turn-integrity.ts: added neverFoldableIds parameter to computeIntegrityWithdrawals. INV1 skips BLOCKED calls in the "kept" check (line 242); INV2 skips withdrawal for BLOCKED pairs (lines 262-266).
  • kernel/src/unit-projection.ts: computes neverFoldableIds from state.messageRefs.byRaw (ref==="BLOCKED") when isMerge; passes to all 3 merge call sites.
  • The change is correct: BLOCKED messages are excluded from the integrity gate's "kept visible" check and split-pair withdrawal, letting the merge proceed without the BLOCKED partner blocking the turn-integrity check.

Tests (verified locally):

  • kernel/tests/issue5919-shredded-merge.test.ts: 5 tests pass (5919-2 now expects merge success instead of no-op)
  • kernel/tests/issue4484-merge-coverage.test.ts: 4 tests pass (now pass neverFoldableIds to computeIntegrityWithdrawals)
  • Integrity-related tests (issue684, recommend-range, tool-pair, issue5101): 17 tests pass
  • CI: 0 failed, 12 pending (test lanes still running)

Concerns:

  1. Wire validity for TOOL-RESULT BLOCKED messages (edge case): If a BLOCKED message is a tool-result and its tool-call gets folded, the model sees a result-without-call (wire violation for OpenAI-wire). The PR's rationale is "the model's view is unchanged (member already visible, turn-mates already covered)", but this assumes the tool-call was already covered before the merge. In the common case (BLOCKED = user message with an image, which is what issue Version 0.214: compression issues still persist. The problem of old blocks failing to compress remains unresolved. | 0.214版本,压缩依旧存在问题,旧块无法压缩问题依旧存在 #6709's 6,115 BLOCKED messages likely are), the fix is safe. In the edge case (BLOCKED = tool-result), the fix may create a new wire violation.
  2. Parallel PR [PR] kernel: let BLOCKED partners stop vetoing block merges (#6709) #6719: Two open PRs for the same issue Version 0.214: compression issues still persist. The problem of old blocks failing to compress remains unresolved. | 0.214版本,压缩依旧存在问题,旧块无法压缩问题依旧存在 #6709. The owner needs to pick one.

8问(中文):

  1. 是什么问题:BLOCKED(永不折叠)消息卡死合并完整性门——每次合并触及该 turn 都因完整性门撤回而失败,形成永久死锁。
  2. 什么原因导致的:computeIntegrityWithdrawals 把 BLOCKED 消息当普通"保留可见"消息处理。当合并覆盖 BLOCKED 消息的 turn 伙伴时,完整性门看到 split turn(reasoning 已折 + BLOCKED call 可见)就撤回整组。New error in v0.213, following up on the previous bug where old blocks can't be compressed and tokens double. | 0.213版本新报错,接着上次的旧块无法压缩token翻倍bug #5919 的修复把它变成 no-op,但 no-op 是永久的,因为 BLOCKED 消息永远不会变得可折叠。
  3. 影响范围与严重程度:特定条件下有问题——当 session 里有 BLOCKED 消息且其 turn 伙伴可折叠时,合并永远失败。报告者的 session 有 6,115 个 BLOCKED 消息(13.7%),context 卡在 533K/792K(67%)。严重程度:中等——不是完全无法使用,但压缩机制失效,context 持续增长。
  4. 怎么修复的:给 computeIntegrityWithdrawals 加 neverFoldableIds 参数,把 BLOCKED 消息从 INV1(保留可见检查)和 INV2(split-pair 撤回)中排除。合并不再被 BLOCKED 伙伴卡住。BLOCKED 消息本身不进合并 coverage(保持可见),但它的 turn 伙伴可以折叠。
  5. 是否有回归问题:审过的既有行为:5919-1(可折叠伙伴在 span 外→合并扩展)不变;5919-3(保护工具伙伴→no-op)不变;5919-4(可折叠伙伴在 span 内→干净折叠)不变;完整性门测试(issue684/recommend-range/tool-pair/issue5101)全绿。无回归。
  6. 是否破坏缓存:不涉及——本 PR 不改缓存/重试/增量游标/去重键。
  7. 是否改了设计方向:不涉及——本 PR 不改架构/边界/数据流。只是让完整性门正确识别 BLOCKED 消息。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值。

Verdict: Technically sound, tests pass, kernel boundary satisfied. Main concerns: (1) parallel PR #6719 — owner must pick one; (2) edge-case wire validity for TOOL-RESULT BLOCKED messages — the PR's "model's view unchanged" rationale holds for the common case (user image) but not for the edge case (tool-result). Recommend owner decision on which PR to merge + address the edge case.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

CI update: All 30 checks now complete — 16 success, 14 skipped, 0 failed. The full test suite (including the 1636 kernel tests) passes on the pushed head. The PR's test claim is verified.

This updates my earlier note "CI: 0 failed, 12 pending" — the test lanes have finished and are green. The review verdict stands: technically sound, tests pass, kernel boundary satisfied. The two concerns remain for owner decision: (1) parallel PR #6719 (same issue #6709, different approach); (2) edge-case wire validity for TOOL-RESULT BLOCKED messages.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Summary (in response to system nudge): The review is complete and was posted in two comments above (floors 5 & 6). Key findings:

  • Kernel Boundary: ✅ kernel-change: declaration present; ✅ version bumped 0.0.197→0.0.198
  • Fix logic: Correct — neverFoldableIds (BLOCKED refs) excluded from INV1/INV2 in computeIntegrityWithdrawals
  • Tests: All targeted tests pass locally (issue5919: 5, issue4484: 4, integrity: 17, issue5789: 3); CI fully green (16 success, 14 skipped, 0 failed)
  • Verdict: Technically sound, tests pass, kernel boundary satisfied

Awaiting owner decision on:

  1. Parallel PR [PR] kernel: let BLOCKED partners stop vetoing block merges (#6709) #6719 (same issue Version 0.214: compression issues still persist. The problem of old blocks failing to compress remains unresolved. | 0.214版本,压缩依旧存在问题,旧块无法压缩问题依旧存在 #6709, different approach: folds BLOCKED into coverage vs. this PR's exclusion from integrity gate) — owner must pick one
  2. Edge-case wire validity: if a BLOCKED message is a tool-result, folding its tool-call creates a "result without call" state (OpenAI-wire violation). Safe for the common case (BLOCKED = user image, which is what Version 0.214: compression issues still persist. The problem of old blocks failing to compress remains unresolved. | 0.214版本,压缩依旧存在问题,旧块无法压缩问题依旧存在 #6709's 6,115 messages likely are).

ranxianglei added a commit that referenced this pull request Nov 14, 2026
The #5919 merge no-op treats every ref='BLOCKED' partner as "cannot be
closed now", but that conflates two very different cases. A content-free
BLOCKED partner (body removed by native compression) carries no live wire
content and can never age into foldability, so letting it doom the merge
turns the no-op into a permanent deadlock: every merge covering its turn
re-hits the same skip and the session stays stuck at its context ceiling
(#6709, session-ad47cd30, 6,115 BLOCKED). A LIVE protected tool /
unrecoverable media is pinned to ref 'BLOCKED' by assignRefs too, with its
full body still in the wire — folding it in drops protected content and
leaving it split emits an orphaned tool_call/tool_result.

Exempt only content-free BLOCKED partners (ref BLOCKED AND not
live-protected-tool AND not host-protected AND not unrecoverable-media)
from the merge-deadlock; they flow to the #5919 rung-2 fallback and are
folded into the merged block, closing the turn. Live protected tools /
media still doom the merge into the honest no-op.

This is the correct superset of the two related PRs (#6719: flow-all-
BLOCKED-to-rung2; #6731: key-on-ref-BLOCKED, keep-split) — both keyed on
ref==='BLOCKED' alone, and both masked the live-protected case by giving
the protected partner a numeric ref in their regression tests. 5919-5 adds
that guard (live protected tool stored as ref BLOCKED still no-ops).

Refs: #6709, #6719, #6731
ranxianglei added a commit that referenced this pull request Nov 15, 2026
The #5919 merge no-op treats every ref='BLOCKED' partner as "cannot be
closed now", but that conflates two very different cases. A content-free
BLOCKED partner (body removed by native compression) carries no live wire
content and can never age into foldability, so letting it doom the merge
turns the no-op into a permanent deadlock: every merge covering its turn
re-hits the same skip and the session stays stuck at its context ceiling
(#6709, session-ad47cd30, 6,115 BLOCKED). A LIVE protected tool /
unrecoverable media is pinned to ref 'BLOCKED' by assignRefs too, with its
full body still in the wire — folding it in drops protected content and
leaving it split emits an orphaned tool_call/tool_result.

Exempt only content-free BLOCKED partners (ref BLOCKED AND not
live-protected-tool AND not host-protected AND not unrecoverable-media)
from the merge-deadlock; they flow to the #5919 rung-2 fallback and are
folded into the merged block, closing the turn. Live protected tools /
media still doom the merge into the honest no-op.

This is the correct superset of the two related PRs (#6719: flow-all-
BLOCKED-to-rung2; #6731: key-on-ref-BLOCKED, keep-split) — both keyed on
ref==='BLOCKED' alone, and both masked the live-protected case by giving
the protected partner a numeric ref in their regression tests. 5919-5 adds
that guard (live protected tool stored as ref BLOCKED still no-ops).

Refs: #6709, #6719, #6731
@ranxianglei

Copy link
Copy Markdown
Owner Author

#6754

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p1 scope:kernel

Projects

None yet

1 participant