Skip to content

[PR] kernel: Covered ranges no longer poison sibling compress batches (#4240) | kernel: 已压缩区间不再拖垮同批其他压缩区间(covered ranges no longer poison sibling compress batches)(#4240) - #4713

Closed
ranxianglei wants to merge 16 commits into
masterfrom
2026-10-22_fix-covered-range-batch-poisoning
Closed

ranxianglei wants to merge 16 commits into
masterfrom
2026-10-22_fix-covered-range-batch-poisoning

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

kernel-change: #4240 的缺陷位于 kernel 的 applyCompression 原子批次契约内部——一个「解析成功但实际折叠不了任何消息」的区间(其消息已全部被活跃块覆盖)过去只在 apply 阶段才被 livelock 守卫发现,而 #3939 的全有或全无契约会因此拒绝整个批次,连累本可以落地的兄弟区间。host 侧无法修复:分类/筛查/门控逻辑都在 kernel/src/compress.ts,且判决措辞被 host 的 src/dead-ranges.ts(#2638 死区间记忆)按正则解析,两侧必须一起动。需要 owner 批准(kernel 边界规则 1)。

Closes #4240

问题与根因(三层)

L1 机制缺陷:全日志宿主(opencode 等)重发的历史里,已被折叠消息的 ref 仍然可见,所以分类阶段把「整段已被覆盖」的区间判成 ok,一路通过所有门控(其文本甚至计入 minCompressRange 总和),直到 applySingleRange 的 livelock 守卫才抛错;而 #3939 的原子契约把任何 apply 期抛错升级为整批拒绝——issue 里 m01137..m01400 / m01401..m01550 / m01551..m01777 三个陈旧区间把第 4 个真正可压的新区间(m01778–m02006)一起杀死,零进展。

L2 诱因:模型按自己上一轮成功压缩的块边界(b25–b32 的切分形状)重发区间,没有先跑 acp_status 取当前可压范围。

L3 体验放大器:零进展 + 同样的错误反复出现 = 「虽然能跑但是总是报错」。

修法(机制层面)

在分类阶段用与 apply 侧同一个只读 projectUnit 判定(遵循 #3236/#3371「资格判定与执行共用单一投影」纪律:recommend 侧已经在用同一判定降级这类单元)提前筛出 no-new 类区间:

  • 这类区间逐区间 warn-skip,兄弟区间照常落地——零新增投影在任何执行顺序下都是无操作,跳过它不提交任何东西,不会破坏原子性保护的顺序依赖不变量;
  • 纯 no-op 批次仍然硬拒:新增门控分支(countedRanges===0 && noNewSpecs>0)+ minCompressRange===0 时的循环后安全网(否则该批次会以零折叠零错误的 SUCCESS 回执返回——livelock 守卫要防的假进展,policy pi#199);
  • 真正的 zone/split fatal 与摘要长度失败仍然整批拒绝(行为不变);
  • 判决措辞保留 active block(s) bN 命名(src/dead-ranges.ts 的正则依赖,[Bug][preflight] range selector re-nominates a fully-folded span every turn — permanent "no new compressible messages" rejection loop + nudge churn #2638);新路径用 intersectingBlockIds(span 内任一成员被覆盖即命名)取代仅看边界的 coveringBlockIds,跨骑/嵌套块与旧 livelock 消息一样被点名。

行为变更披露(old → new,Deliverables 第 4 条)

场景 旧行为 新行为
批次含 ≥1 个全覆盖/零新增区间 整批原子拒绝,0 折叠 该区间 warn-skip,兄弟区间落地
纯 no-op 批次(全部区间都零新增) 整批拒绝(livelock 消息) 仍整批硬拒,但由门控分支给出更完整判决(点名覆盖块 + refold 阻塞原因 + acp_status 指引)
真正的 zone/split fatal、摘要超长 整批拒绝 不变(仍整批拒绝)
minCompressRange 求和 全覆盖区间的文本计入总和 不再计入(它折叠不了任何东西,不应抬高门槛)
no-op 区间附带超长摘要 长度预检杀整批 不再校验(其摘要根本不会被存储)
minCompressRange=0 的纯 no-op 批次 整批拒绝(apply 期抛错) 仍硬拒(新安全网,防止静默零折叠成功回执)

测试

  • 新增 kernel/tests/issue4240-covered-range-batch.test.ts(7 例):混合批次救援(issue 原形:2 陈旧+1 新鲜 → 新鲜落地、2 条 skip warning)、纯 no-op 硬拒、no-op+保护区 fatal → 仍整批拒、超长摘要不再杀可救批次、restored-inline refold 原位更新、陈旧文本不抬高门槛(honest too-small)、minCompressRange=0 安全网;
  • 更新既有断言:refold.test.ts minCompressRange 用户文档写 tokens、内核按 chars 执行(4× 单位缺口);附 registryLookup 冲突即静默回落的备注 #402 测试(判决从 apply 期 livelock 消息移到门控级判决,仍含点名+恢复指引)、validation.test.ts 零新增用例(同上);
  • 本地定向全绿(rebased head 3034c57,基于 ac844fa):受影响 kernel 套件 93/93、master 新增 kernel 套件(seq-interval-coverage / issue4289 / 4292 / 4311 / 4348 / 4457 / parse-compress-input / seq-restamp)141/141、bili 回执/preflight 套件 61/61、typecheck clean。全量回归以 CI 为准。

备注

…ges (#4240)

A compress batch in which some ranges resolve but fold nothing - every
message already owned by active blocks - used to die at apply time, where
the atomic batch contract rejected the WHOLE batch ('Batch rejected before
folding any range'), killing sibling ranges that would have landed. On
full-log hosts those refs stay visible, so classification calls them 'ok'
and they sail through every gate until applySingleRange's livelock throw.

Screen that class at classification time with the same read-only
projectUnit judgment the apply lane throws by (#3236/#3371 discipline):
- such ranges warn-skip individually; sibling ranges land
- all-no-op batches still hard-reject (new gate branch plus a
  minCompressRange===0 safety net), naming covering blocks and refold
  blockers
- genuine zone/split fatals and summary-length failures still reject the
  whole batch
- covered-range text no longer inflates the minCompressRange sum, and an
  oversized summary on a no-op range no longer kills the batch (its
  summary is never stored)

The verdict keeps the 'active block(s) bN' naming that src/dead-ranges.ts
parses for its #2638 structural-dead memory: intersectingBlockIds (any
span member covered) replaces boundary-only coveringBlockIds in the new
path.

Tests: kernel/tests/issue4240-covered-range-batch.test.ts (7 cases: mixed
batch salvage, pure no-op reject, fatal+no-op atomicity, length-pre-check
exclusion, restored-inline refold, gate-count honesty, zero-gate safety
net); updated the #402 assertion in refold.test.ts and the zero-new
assertion in validation.test.ts to the gate-level verdict.
@ranxianglei ranxianglei changed the title kernel: 已压缩区间不再拖垮同批其他压缩区间(covered ranges no longer poison sibling compress batches)(#4240) [PR] kernel: Covered ranges no longer poison sibling compress batches (#4240) | kernel: 已压缩区间不再拖垮同批其他压缩区间(covered ranges no longer poison sibling compress batches)(#4240) Oct 25, 2026
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-22_fix-covered-range-batch-poisoning (8f2d1f9)

Latest build — install this exact version

npm install -g billion-context@0.0.5965-pr.4713.7c24a1f

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
8f2d1f9 billion-context@0.0.5965-pr.4713.7c24a1f build
7a8b847 billion-context@0.0.5761-pr.4713.08a90d1 build
cb30d24 billion-context@0.0.5757-pr.4713.289fbe6 build
3034c57 billion-context@0.0.5739-pr.4713.70cc091 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4713.tgz
npm install -g package

Install instructions are refreshed on each push.

ci-check added 2 commits October 25, 2026 22:43
…ing block is named (#4240)

noNewRangeDetail's empty-covering branch rendered '...turn-integrity carve).;
otherwise run acp_status...' — a period followed by a semicolon in the
model-facing sentence. Make the tail conditional on whether the merge hint
exists; the covered-block wording stays byte-identical.
The kernel bump commit updated kernel/package.json but not the kernel entry
in the root package-lock.json, so the CI lock-sync job (npm install
--package-lock-only + git diff --exit-code) failed and skipped every test
lane. Regenerated with the same command CI runs — one-line diff.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 —— 本 PR 即 #4240 的修复(正文 Closes #4240),与既有 issue/PR 无同机制重复项;#4240 线程 floor 4/7 是同一修复工作的分析过程,非独立条目。

标签: pr(保留)· bug · opencode · scope/kernel · p2(缺陷修复;事故宿主为 opencode 类全日志宿主;改动域在 kernel 折叠判定;有绕行、无数据丢失,定 p2)。

独立审核完成(未照抄 issue 侧结论,diff 逐段重读 + 本地定向复验)。按八问:

  1. 是什么问题:模型重发「整段已被现有块折过」的区间时,整个 compress 批次被原子拒绝——同一批里真正可压的新区间被连坐,零进展、每轮同样报错。
  2. 什么原因导致的:三层。机制层:全日志宿主重发历史里已折叠消息的 ref 仍可见,分类判 ok 一路过门(文本还计入 minCompressRange 总和),直到 apply 期 livelock 守卫才发现没东西可折;契约层:[Kernel Initiative] Transactional folding boundaries + wire byte fidelity (unified prefix cache fix #3938, kernel side) | [kernel 专项] 折叠边界事务化 + wire 字节保真(前缀缓存统一修复 #3938 内核侧) #3939 原子批次把任何 apply 期抛错升级为整批拒绝,一个陈旧区间毒化兄弟区间;诱因层:模型按记忆里的旧块边界重发,没先跑 acp_status。
  3. 影响范围与严重程度:任何全日志宿主(opencode 等)+ 长会话(块多到模型记不住边界)都会触发;每次 = 该轮压缩零进展 + 报错刷屏,会话本身可用(压 acp_status 列出的当前可压范围即可继续)。p2。
  4. 怎么修复的:分类阶段用与 apply 侧同一个只读 projectUnit 判定([Bug] recommend/apply 对同一区间的 turn-integrity 判定永久不一致:acp_status 反复推荐的区间被 apply 侧整段拒绝(0.1.196,Range would split N turn(s) at the protected-zone boundary) #3236 单投影纪律)提前筛出「零新增」区间,逐区间 warn-skip、兄弟照常落地;纯 no-op 批次仍硬拒(新门控分支 + minCompressRange=0 的循环后安全网,防静默零折叠成功回执,pi#199);zone/split fatal 与摘要超长仍整批拒。我独立验证了顺序无关性论证:覆盖剔除用的是循环前一次性采集的固定 preExistingCoverage 快照,兄弟折叠只可能增加覆盖,所以「分类期零新增 ⇒ 任何执行顺序下都零新增」成立——跳过不提交任何东西,不破坏原子性保护的不变量。fatal 类(zone/split)在筛查处直接放行,仍由 applySingleRange 原样抛错拒批。
  5. 是否有回归问题:逐项审了 applyCompression 触碰路径的既有行为——纯 no-op 批次硬拒 ✅ 保留(判决措辞升级为点名覆盖块+refold 阻塞原因+acp_status 指引);zone/split fatal、摘要超长 ✅ 不变;refold 原位更新(fix: isolate side requests (title-gen) from main session kernel state #398/decompress 被下一轮静默撤销:syncBlocks 无条件复活已失活块 + bili 同时删除内容缓存 + loop 路径把折叠视图喂给回退检索(还原原文再被重折,二次解压永久退化) #400)✅ 保留(restored-inline 块在混合批次里仍原位更新,refold 计入 blocksCreated 不会误触发安全网,测试钉住);host 死区间记忆([Bug][preflight] range selector re-nominates a fully-folded span every turn — permanent "no new compressible messages" rejection loop + nudge churn #2638)✅ 兼容——active block(s) bN 命名保留,parseStructuralEmptyVerdict 两条正则对新措辞两种变体都命中,intersectingBlockIds(span 成员级相交)取代仅看边界的命名后跨骑/嵌套块与旧 livelock 消息一样被点名;minCompressRange 用户文档写 tokens、内核按 chars 执行(4× 单位缺口);附 registryLookup 冲突即静默回落的备注 #402 的 too-small 判决点名阻塞块 ✅ 保留(okBlockedReasons 后缀);拒绝路径返回 preLoopState、仅 deadRefs 墓碑差异 ✅ 保持(compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362)。行为变更披露(与 PR 表格一致,代码核实无误):混合批次含全覆盖区间从整批拒→warn-skip 兄弟落地;全覆盖文本不再抬高 minCompressRange 门槛;no-op 区间的超长摘要不再校验(其摘要不会被存储);纯 no-op 批次判决从 apply 期 livelock 文本变为门控级更完整文本。一个 PR 未触及的既有边缘保持不变:边界扩展理论上可能让后序兄弟折叠覆盖掉已通过筛查的前序区间 → apply 期抛错 → 整批拒;新旧行为一致,未被改坏。
  6. 是否破坏缓存:不涉及——不改块内容、ref 分配或前缀缓存接缝,只改「谁落地、谁被拒」。
  7. 是否改了设计方向:不涉及——仍是原子批次 + 单投影判定,只是把一个失败类的发现点从 apply 期前移到分类期(该类顺序无关,跳过不破坏原子性保护的不变量)。
  8. 是否引入新配置:无。

测试有效性:新增 7 例是真测试且双向断言齐全(正例:新鲜区间落地/跳过点名;反例:doesNotMatch "Summary too long"、doesNotMatch "already compressed"、拒绝时 warnings==[] 不泄漏 skip warning、nextBlockId 精确断言钉住无空分配)。更新的两例断言与新行为一致。

我直接修掉的两个小问题(已 push 到 PR 分支,head 现为 cb30d24e4,两个独立标注 commit):

  • chore: 根 package-lock.json 缺 kernel 0.0.164 版本记录 → CI lock-sync 红、全部 test 车道被跳过。用 CI 同款命令重新生成,一行 diff;
  • kernel: noNewRangeDetail 在「无覆盖块」分支会渲染出 ...carve).; otherwise...(句号+分号连排)的模型可见措辞,改为条件尾句,有覆盖块的措辞逐字节不变。
    两者均已本地复验:typecheck clean、受影响 kernel 套件 37/37、压缩决策面 kernel 套件 181/181、host 回执/preflight 套件 41/41、verif-subset-guard 2/2 全绿(kernel/src/compress.ts 属 verif 守护清单,子语言守卫已过)。

判定:代码侧就绪,可合——等 CI 在新 head 上 test/red-signal/static-gates 车道转绿(guard 车道此前已全绿,lock-sync 应随本次修复转绿;全量回归以远程 CI 为准)。kernel 变更按边界规则 3 合并权在你(owner 批准 + 人工 review)。

ci-check added 2 commits October 25, 2026 23:02
…ment

#4316 PR3/3 (restampRefsToLedger) re-homes citation numbers once on every
full-mode cold load; goldens pinning exact ref numbers are born-red. Re-pin
the 7 storage-lane tests (engine-proofs x6 + orphan-recovery x1) modulo that
renumbering — zero production code touched.

- golden-storage-drive.ts: per-turn citation-normalized digest (shaCiteNorm)
  and state digest modulo stripOneTimeAlignment (stateDigestAligned); the
  deep-sort helper is hoisted into golden-canonical.ts for sharing.
- corrupt-index-child.ts: new stripOneTimeAlignment normalizer — strips the
  refLedgerAligned stamp and re-homes refs onto canonical raw-id order so a
  clean full-mode read and the legacy fallback load compare equal.
- storage-engine-proofs.test.ts: wire legs assert shaCiteNorm equality vs
  the legacy restart leg and the committed goldens; state legs assert
  stateDigestAligned; rollback leg now spells BILI_STORAGE_UNIFIED=legacy
  explicitly (#3818 flip: absent resolves to full, deleting the env would
  vacate the legacy path).
- storage-orphan-recovery.test.ts: pins the content-addressable
  byRaw->byRef round-trip instead of a counter-era m00003 literal.
- golden/storage-render/*.json regenerated with shaCiteNorm/stateDigestAligned.

Converges the master-red storage-lane family (#4599 plan A, owner directive).
lock-sync also checks kernel/package-lock.json, which still recorded
0.0.163 in both version fields after the bump. Regenerated with the same
command CI runs ((cd kernel && npm install --package-lock-only)); two-line
diff matching the CI-expected blob exactly.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

补充:lock-sync 实际查两个 lockfile——根 package-lock.json(上条已修)之外还查 kernel/package-lock.json,后者 bump 后仍记录 0.0.163(两处 version 字段),所以第二轮 CI 还是红。已用 CI 同款命令(cd kernel && npm install --package-lock-only)重新生成,两行 diff、与 CI 期望的 blob 完全一致,commit 7a8b8471c 已 push。等这轮 CI:guard 车道 + e2e 车道已全绿,主 CI(test/red-signal/static-gates)在跑中,结果出来我再报一行。

ranxianglei and others added 11 commits October 25, 2026 23:58
…dger parity masks

The #4316 one-time renumbering restamped messageRefs but left
state.tokenSnapshot keyed by the OLD refs, so post-restart bodies
carried stale per-message token counts (1.3K vs 1.4K) and the
full-mode-restart wire drifted off the legacy baseline (#4599).

persist.ts: two-pass remap (stage moves, delete old keys, then write
new keys) so entries whose new ref collides with another move's old
ref survive; fires only when superseded refs exist.

Aligned normalizer (corrupt-index-child.ts): the cacheLedger's
segment bookkeeping (be/bl/bs, unfinalized tail line), the derived
agg block, and per-line miss-attribution (prov/seam) are restart
experience, not content — no-restart and restart legs legitimately
differ there. Masked in the aligned view only; raw digests keep
full accounting. Goldens regenerated (stateDigest + aligned).

storage-engine-proofs 12/12, golden-storage-render 5/5 locally.
…persisted state (#4599)

The fork-survival test asserted immediately after the compress tool
response, but the fold lands in the live session state asynchronously
(#3573 replay semantics). Slow CI runners persisted the pre-fold state
('0 block(s) (compressions=0)' cold load) and the deepEqual diverged.
Bounded 5s wait on the in-memory fold; fast machines exit on the first
iteration. Product semantics unchanged.
The #4316 cold-load alignment reads the unified sidecar; its ingest of
just-flushed records completes asynchronously. Slow runners compared
the pre-ingest state and saw refLedgerAligned missing while the fold
itself had landed. Poll (bounded 5s) until the loaded state carries
the flag; fast machines exit on the first probe.
Git-for-Windows autocrlf checks the bundled .md posts out with CRLF;
the exact-match '---' fences then failed to parse and every post lost
its title/excerpt/date frontmatter (Windows CI: 'bundled meet-bili post
found' assertion). Normalize CRLF to LF inside parseFrontmatter when
the fence is CRLF-shaped; LF content is byte-identical as before.
Regression test with a CRLF fixture.
… storage mode (#4599)

The #4316 cold-load alignment sets refLedgerAligned only when the
unified store is available; config-clean machines (CI runners) run
legacy mode where it is an honest skip. The test asserted the flag
unconditionally, so every shard carrying this file was born-red on CI
while unified-config machines passed. Expectation now mirrors
store.replicasAvailable(); verified 49/49 in BOTH modes locally.
The ci-check lane's expectation change (c496cfb) exposed a real Windows
bug: stripInheritedProxy builds inlineEnv as a spread copy of process.env,
which keeps the original key casing (ComSpec) — Windows env vars are
case-insensitive but plain objects are not. planClientSpawn's exact
env.COMSPEC probe then missed it and silently degraded every wrapped spawn
to the bare-name 'cmd.exe' fallback (verified by repro on a real win32 box:
childEnv.COMSPEC=undefined while process.env.COMSPEC was set).

Fix: probe COMSPEC/ComSpec/comspec in planClientSpawn and planDshSpawn.
Verified on real Windows (Node 22): launcher-opencode-inline + fix-679 +
dsh-refresh 37/37; host tsc clean, 35 pass / 2 win32-skips.
…OMSPEC case-probe

c496cfb re-derived the win32 expectation from ambient process.env, which
still mismatches production: Node's process.env member access is
case-insensitive on Windows while the child env handed to the planner is a
plain object whose key keeps the OS casing (ComSpec), so the exact
env.COMSPEC probe fell through to the bare-name fallback. The assertion now
derives from the child's OWN captured env/options and probes
COMSPEC/ComSpec/comspec exactly like planClientSpawn does post-dfeea8a11 —
self-contained under any concurrent-test env churn in single-process CI
shards — and additionally pins windowsVerbatimArguments=true on the wrap.
@ranxianglei

Copy link
Copy Markdown
Owner Author

关闭本 PR:内核契约部分已被 PR #5253 覆盖并合并(kernel 0.0.171),无需重复合并。

具体对应:#4713 要修的「一个已覆盖区间在 apply 期被 livelock 守卫抛错、#3939 原子契约把整批拒掉、好区间陪葬」,#5253 已用另一套契约解决——已覆盖区间不再报错,改为幂等 no-op 成功收据(result.noop: true),收据直接点名块号并指引用 bN..bM 改写;纯保护区区间降级为警告;endId 越界钳到最高已发 ref;同批可用区间照常折叠。kernel/tests/refold.test.ts 与 validation.test.ts 的对应翻新也已随 #5253 进主干(kernel 1493/1493 绿)。

另外本分支还夹着与 #4240 无关的 host 文件(src/dsh-channel.ts、src/launcher.ts、src/persist.ts、src/web/blogfeed.ts、tests/golden-storage-render 等),即使想继续也需要大 rebase。如后续发现 #4713 里有 #5253 没覆盖的独立改动(目前对比未发现),欢迎以干净分支重开。感谢报告和首修!

Superseded by #5253 (merged).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

❌ [ACP] Compression FAILED: Batch rejected before folding any range — 3 of 4 requested range(s) failed during application: range m01137..m01400:

1 participant