Skip to content

[PR] kernel: root-cause fix for kernel-compress batch — truncated-input misdiagnosis (#6946) + compress-orphan boundary pin (#5455) - #6983

Open
ranxianglei wants to merge 5 commits into
masterfrom
2026-11-17_kernel-compress-rcagg
Open

ranxianglei wants to merge 5 commits into
masterfrom
2026-11-17_kernel-compress-rcagg

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Root-cause aggregation: kernel-compress (2 items)

This is the unified root-cause fix PR for the kernel-compress aggregation batch (mirror #6980 / GitHub #6979). Per the owner's ruling on both member issues (2026-11-16): "修复走聚合 issue 统一出 PR" — the fix ships through this single aggregation PR.

Fixes

What this PR does

#6946 (cherry-picked from #6970 with -x, authorship preserved): the compress content string (a JSON-array doc as a string) was tail-truncated on the wire and misdiagnosed as kind=content-not-array / no-valid-ranges instead of truncated. Three facets, all in kernel/src/parse-compress-input.ts, host zero-change:

  1. salvageContentEntries recovers complete depth-0 STRING entries (a partial trailing string is dropped, not the whole salvage).
  2. EOF gate in the string-content fallback: fence-stripped content starting with [ that looksTruncated → classify as truncated via finishSalvage, skipping the line-form ladder.
  3. Set diag.length + rawPrefix for truncated string content so the host argCorruption receipt fires.

Kernel version bump 0.0.198 → 0.0.199 (with both lockfiles synced, #3782).

#5455 (new, tests only — no kernel/src change, no extra version bump): stripOrphanedToolCalls (kernel/src/prune.ts:625) exempts every compress call by name, so it cannot distinguish a genuinely-pending call from one answered in the input whose result was covered away. The state is currently UNREACHABLE through the pipeline (no producer), but the exemption means it would ship an unanswered compress tool_call if it ever became reachable. This PR pins the boundary with tests only (zero behavior change):

  • B (prune side): fixes the misleading comment in carrier-advertise-validate.test.ts that claimed stripOrphanedToolCalls "drops any call whose result left the view" (untrue for compress calls, which are exempt). Adds a tripwire test: an answered compress call whose result is covered still survives the orphan strip, contrasted with an ordinary answered call whose result is covered (stripped).
  • C' (apply side): pins the pair-atomicity invariant at the source — applyCompression never emits a block with a tool-result lacking its adjacent tool-call. This is the invariant that keeps the kernel: distinguish answered compress orphans from genuinely pending calls #5455 state unreachable.

Note on #6970

#6970 is the standalone PR for #6946. Its core fix is cherry-picked here (with -x). Per the owner's ruling that the fix ships through the aggregation PR, #6970 is superseded by this PR and should be closed after this merges.

Testing

  • Targeted kernel suites green: parse-compress-input, prune-orphan-mcp, carrier-advertise-validate, tool-pair-integrity (124 tests, 0 fail).
  • Kernel typecheck clean.
  • Full-suite verdict = remote CI test lanes on the pushed head.

kernel-change

kernel-change: #6946 truncated-input misdiagnosis fix (parse-compress-input.ts, owner-approved "go ahead 内核允许改") + #5455 boundary pin (tests only, no src change). Both member issues carry owner approval for the kernel change.

ci-check added 3 commits November 17, 2026 04:22
…ed, not content-not-array (#6946)

The vLLM-style gateway stringifies the compress content array, so content
arrives as a JSON string of the array. When that string is cut mid-stream
the prefix is a clean JSON-array start followed by EOF. The line-form
fallback misread the JSON prefix as headerless prose and reported
content-not-array / no-valid-ranges, steering the model away from a format
that was legal all along.

Three facets in parse-compress-input.ts:
1. salvageContentEntries recovers complete depth-0 STRING entries (not just
   {...} objects); a partial trailing string is dropped, never guessed.
2. An EOF gate in the string-content fallback: when the fence-stripped
   content starts with '[' and looksTruncated (unbalanced brackets /
   unterminated string = mid-stream cutoff), classify as truncated via
   finishSalvage instead of the line-form ladder. Balanced-but-malformed
   input keeps today's path (no regression).
3. Record diag.length + rawPrefix for a truncated stringified content array
   so the host argCorruption receipt fires instead of the misleading
   array-wrap hint.

kernel-change: fix input-parse misclassification of truncated stringified
content arrays; the host argCorruption receipt already exists, no host
change needed.

(cherry picked from commit 4ba234d)
The kernel/package.json version bump (0.0.198 -> 0.0.199, cherry-picked from
#6970) must be mirrored in BOTH the root package-lock.json's acp-kernel entry
and the kernel's standalone package-lock.json (#3782), or the lock-sync gate
fails. Version-only sync, no other churn.
#5455: stripOrphanedToolCalls (prune.ts:625) exempts every compress call by
name, so it cannot distinguish a genuinely-pending call from one that was
answered in the input but whose result was covered away. The state is
currently UNREACHABLE through the pipeline (no producer), but the exemption
means it would ship an unanswered compress tool_call if it ever became
reachable. No behavior change — this pins the boundary with tests only:

B (prune side): fix the misleading comment in carrier-advertise-validate.test.ts
that claimed stripOrphanedToolCalls 'drops any call whose result left the view'
(untrue for compress calls, which are exempt). Add a tripwire test showing an
answered compress call whose result is covered still survives the orphan strip,
contrasted with an ordinary answered call whose result is covered (stripped).

C' (apply side): pin the pair-atomicity invariant at the source — applyCompression
never emits a block with a tool-result lacking its adjacent tool-call. This is
the invariant that keeps the #5455 state unreachable.

No kernel/src change, no version bump (tests only).
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-17_kernel-compress-rcagg (904d8a5)

Latest build — install this exact version

npm install -g billion-context@0.0.11007-pr.6983.2386d4e

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
904d8a5 billion-context@0.0.11007-pr.6983.2386d4e build
e8ed712 billion-context@0.0.10499-pr.6983.b008e11 build
49f19da billion-context@0.0.10497-pr.6983.3a4281e build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6983.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title kernel: root-cause fix for kernel-compress batch — truncated-input misdiagnosis (#6946) + compress-orphan boundary pin (#5455) [PR] kernel: root-cause fix for kernel-compress batch — truncated-input misdiagnosis (#6946) + compress-orphan boundary pin (#5455) Nov 16, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — 本 PR 即 owner 裁决(2026-11-16 "修复走聚合 issue 统一出 PR")下的 kernel-compress 聚合唯一修复 PR(#5455 + #6946 合一);独立 PR #6970 被本 PR 取代(cherry-pick -x 保留作者署名),合入后应关闭。

CI 状态(已实时核实):head 49f19da 的 CI 唯一失败是 architecture-docs 门禁,它短路了整个 run —— 全部 test 矩阵 lane 被 skipped,全量测试一轮都没跑。已验证这是继承性红,不是本 PR 引入:

已执行:git merge origin/ci-fixes 进 PR 分支并推送,新 head e8ed712(merge 干净,只带入 reference/architecture.md 一行 + tests/issue2929-codex-fold-double-miss.test.ts),CI 重跑中。PR base 与 master 齐平(0 behind)。

代码审核(#6946 三面修复 + #5455 边界钉)进行中,完整八问结论随后补上。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核结论:代码与行为核实通过,可合入(kernel 变更属 human-only 审核档,合并由 owner 执行;#6970 被本 PR 取代,合入后请关闭)。

查重: 不重复——本 PR 即 owner 裁决下的 kernel-compress 聚合唯一修复 PR;#6970 为其被取代的独立分支。

我独立验证了什么(非照抄 PR 描述):

  1. A/B 行为对照(master 旧解析器 vs PR 新解析器,7 种输入形状逐一对跑):
输入形状 旧(master) 新(PR) 判定
对象数组截断在第一个对象 content-not-array, 0 ranges truncated, 0 ranges, length=144 误诊→正诊 ✓
对象数组截断在第二个对象 truncated, 1 range, 无 length truncated, 1 range, length=110 +argCorruption 收据 ✓
字符串数组截断在第二个字符串 no-valid-ranges, 0 ranges truncated, 1 range(首条完整字符串被救出) 可用性改善 ✓
字符串数组截断在第一个字符串 no-valid-ranges, 0 ranges truncated, 0 ranges, length=111 误诊→正诊 ✓
完整字符串数组 ok, 2 ranges ok, 2 ranges, 无 length 不变 ✓
[] 空数组 content-not-array content-not-array 不变(EOF 门不触发)✓
纯散文 content-not-array content-not-array 不变 ✓
  1. 定向 kernel 套件本地跑绿:parse-compress-input + prune-orphan-mcp + carrier-advertise-validate + tool-pair-integrity = 124 tests, 0 fail。
  2. CI:旧 head 49f19da 唯一红是 architecture-docs(继承性 master 红,已核实 master 同 lane 同错 + 4 个 codex 命中率问题分析 #2929 shard 红)→ 已 merge ci-fixes 哨兵推送新 head e8ed712 → 全绿:38 success / 14 skipped(门控 e2e)/ 0 fail,kernel-guard、lock-sync、version-guard 全过。
  3. kernel 边界合规:kernel-change: 声明在案;版本 0.0.198→0.0.199 且两个 lockfile 同步;cherry-pick 带 -x 标记、作者署名(ci-check)保留、内容与 [PR] kernel: classify tail-truncated stringified content arrays as truncated, not content-not-array (#6946) #6970 原 commit 逐字节一致;host 零改动(收据文本 src/stream.ts 的 argCorruption 分支本就存在,PR 只是把更多 case 路由进去)。

结构化八问

  1. 是什么问题:两个独立缺陷——(a) [Bug][compress] Tail-truncated tool-call content is misdiagnosed as kind=content-not-array; no salvage path recognizes truncated JSON-array input | 尾部截断的 compress 入参被误诊为 content-not-array,现有 salvage 均不识别截断的 JSON 数组 #6946:compress 的 content 字符串(JSON 数组文档)在 wire 上被尾部截断时,被误诊为 content-not-array/no-valid-ranges,收据里的 array-wrap 提示把模型引离一个本来就合法的格式;(b) kernel: distinguish answered compress orphans from genuinely pending calls #5455:stripOrphanedToolCalls 按名字豁免所有 compress 调用,分不清「真 pending」与「已应答但结果被覆盖」——该态当前不可达,但豁免意味着一旦可达就会发出未应答的 compress tool_call,且边界无任何测试钉住。
  2. 什么原因导致的:(a) salvageContentEntries 只救对象条目、救不了字符串条目,且解析失败后的 EOF 路径无条件走 line-form 阶梯,把 JSON 前缀当无头散文读 → 误分类;截断的字符串 content 又不记 diag.length → host 的 argCorruption 收据不触发。(b) prune 层的 compress 豁免是设计如此(保护真 pending 调用),compress 对原子性因此只靠 apply 侧 pair widening + integrity gate 单腿支撑,该不变量在源头没有测试。
  3. 影响范围与严重程度:(a) 触发条件 = wire 截断 compress 参数(vLLM 类网关 stringify content 数组的场景);严重度 = 调用被拒 + 错误诊断误导模型换格式——体验劣化,非数据丢失、非会话不可用;新代码在「第二条字符串被截」场景还多救出首条完整 range(0→1,可用性改善)。(b) 零行为变化(纯测试),该态经 owner 与外部贡献者独立穷举确认当前无生产者(planner / seq-interval / fork-recovery 三路径全排除)。
  4. 怎么修复的:(a) 三面全在 kernel/src/parse-compress-input.ts:① salvage 增救完整 depth-0 字符串条目(尾部残串丢弃,不猜);② EOF 门——fence 剥除后以 [ 开头且 looksTruncated(括号失衡/字符串未闭合,即 mid-stream 截断签名)→ 走 finishSalvage 判 truncated,跳过 line-form 阶梯;门只在截断签名上触发,平衡但畸形的文档保持原路径;③ 截断字符串 content 记 diag.length+rawPrefix → host argCorruption 收据触发。(b) B:修正 carrier-advertise-validate.test.ts 里对 compress 调用不成立的注释 + tripwire 测试(已应答 compress 调用结果被覆盖仍存活,对照普通调用被剥);C':在源头钉不变量——applyCompression 永不产出「含 tool-result 却缺相邻 tool-call」的块。
  5. 是否有回归问题:无。审过的既有行为逐项:严格 JSON 解析路径(完整数组)不变;[]/散文/平衡畸形文档的 line-form 阶梯路径不变(EOF 门要求截断签名才触发);单引号归一化重试、track: plugin-mode proxy-side emergency truncation (last-resort backstop, deferred from #467) #470 单对象 salvage、flat 单 range 与 stringified-entry 两个 fallback 均不触碰;对象条目 salvage 行为不变(字符串条目是纯增量);kind=truncated 对 salvaged 输入的判定是既有语义。高危模式逐项过:无阈值/时序漂移;收据文本字节不变(只改路由);fail-fast 未变吞错。出现持续压缩失败,无法自动恢复 #847 失败账本 parseKey 对受影响 case 从 parse:content-not-array 变 parse:truncated——会话内计数器,非持久化,无影响。
  6. 是否破坏缓存:不涉及——不触碰缓存、重试、增量游标、去重键。
  7. 是否改了设计方向:不涉及——kernel 输入解析层修复,host 零改动,两种压缩模式共用该解析路径、均受益,无架构转向。
  8. 是否引入新配置:不涉及——无新 env/config/默认值。kernel 版本 0.0.198→0.0.199 是 kernel/src 变更的 CI 强制纪律(lockfile 已同步),不是新配置面。

一处小观察(非阻塞):PR 正文用 Fixes #5455,合入时 GitHub 会自动关单;而 owner 在 #5455 里说「本单保留不关,修复验证后统一关闭」——若「验证」指 CI 绿则已满足,若指人工验证则合入前可把 Fixes 改为普通引用,由 owner 定。

# Conflicts:
#	kernel/package-lock.json
#	kernel/package.json
#	package-lock.json
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

冲突已解决并推送,merge 状态已实时核实为干净(mergeable: true,GitHub API 单次直查,非缓存)。

本次做了什么:

  1. 接冲突:PR 分支落后 master 135 个提交。按「同一机器人处理冲突与红项」的分工,执行 git merge origin/master(master head 6453feeeb842)进 PR 分支——选 merge 而非 rebase,因为分支 head 本身是 merge commit(此前接 ci-fixes 的产物),rebase 会搅乱已审核的历史。
  2. 解决 3 处冲突,全部是版本号:kernel/package.json、kernel/package-lock.json、package-lock.json。冲突根因:master 侧已把 kernel 版本走到 0.0.200(0.0.199 被 [PR] compress.reasoning.tail: drop reasoning of any closed round in the protected window (#7144 follow-up) #7252 的 compress receipt 修复占用,见 master 提交 b805e43acb8「resolve the 0.0.199 collision」),与本 PR 的 0.0.198→0.0.199 撞号。
  3. kernel 版本 0.0.200 → 0.0.201(披露:PR 正文写的是 0.0.198→0.0.199,合并后相对新 base 的实际 bump 是 0.0.200→0.0.201)。原因:本 PR 的 kernel/src/parse-compress-input.ts 改动相对新 base 仍是未 bump 的 kernel 源码变更,kernel-guard CI 强制「kernel/src 变更必须 bump 版本」,不 bump 必然红。两个 lockfile 同步到 0.0.201。按本车道规则未改 PR 正文,在此披露。
  4. 推送最终 head 904d8a5e98c(无 [ework:ci-held] 标记——这是本分支的一次全量 CI 通过;推送前已 fetch 确认远端 head 未动、无并发工作)。

本地定向验证(合并后全过):

  • lock-sync 等价检查:root + kernel 各跑 npm install --package-lock-only 后 git diff --exit-code,零漂移;
  • kernel 全套件:1655 tests,0 fail(含本 PR 的 4 个定向套件);
  • kernel typecheck 干净;root typecheck 干净;root build 成功。

CI 状态:新 head 的 CI 正在跑(镜像 check 缓存仍停在 11-16 的旧 head,暂无法实时确认绿/红;若红,按 CI 协调流程先查 #4599 再处置)。mergeable_state 目前为 blocked——与 11-16 全绿时一致,是 kernel 变更 human-only 审核档的 branch protection 阻塞,不是冲突或 CI 问题。


结构化八问

  1. 是什么问题:PR 分支与 master 冲突(dirty)——master 在 PR 之后合入了 135 个提交,其中 kernel 版本撞号(两边各自 bump 了 0.0.199)。
  2. 什么原因导致的:master 侧 [PR] compress.reasoning.tail: drop reasoning of any closed round in the protected window (#7144 follow-up) #7252 的 compress receipt 修复先占了 kernel 0.0.199 并推进到 0.0.200,本 PR 的 0.0.198→0.0.199 bump 与之撞号;三个版本文件因此产生内容冲突。
  3. 影响范围与严重程度:仅本 PR 的合并状态,不影响任何运行中实例;不解决则 PR 无法进入 owner 审核/合并。
  4. 怎么修复的:merge master 进 PR 分支,冲突统一解为「master 内容 + kernel 版本再 bump 到 0.0.201」(kernel/src 变更相对新 base 仍需独立 bump,CI 强制),lockfile 同步,普通 push(非 force)。
  5. 是否有回归问题:无。合并后分支相对 master 的 diff 逐文件核对过,恰好是 PR 原有 8 个文件(4 测试 + 1 src + 3 版本文件),无任何 master 侧语义被改动;kernel 全套件 1655/1655 绿;高危模式逐项过:无阈值/时序/输出格式漂移,无 fail-fast 变吞错。
  6. 是否破坏缓存:不涉及——纯合并与版本 bump,不触碰缓存/重试/游标/去重键。
  7. 是否改了设计方向:不涉及。
  8. 是否引入新配置:不涉及。唯一数值变化是 kernel 版本 0.0.201(相对合并前 PR 的 0.0.199),属 CI 强制的版本纪律,非配置面。

收工判定:冲突已清、本地验证全过、CI 在新 head 上运行中;剩余路径是 CI 绿 + owner 对 kernel 变更的 human-only 批准与合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

ranxianglei pushed a commit that referenced this pull request Nov 19, 2026
…ed, not content-not-array (#6946)

Rebased onto master (kernel 0.0.200). The salvage fix (parse-compress-input.ts + test) is byte-identical to the original; the kernel version is bumped 0.0.200 -> 0.0.201 to satisfy the version-discipline gate (kernel/src changed).

This PR is superseded by #6983 (the rcagg aggregation carries the same fix as a faithful cherry-pick). Close after #6983 merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant