Repository navigation
[PR] fix(kernel): reject range folds that would merge blocks beyond the requested ref window - #5835
ranxianglei wants to merge 10 commits into
Conversation
…ed ref window (#5830) A plain m-ref range carries one summary describing exactly [startRef..endRef], but projectUnit consumed any same-tier active block whose visible representation fell inside the RESOLVED span. After a folded historical ref snaps to its inheriting block's anchor (unbounded view-index jump), the span can sweep blocks whose coverage lies outside the window, silently replacing their summaries with one that does not cover their span (#5830). Hard gate in projectUnit (shared by apply/recommend/acp_status per #3236): a plain range consuming an active block with out-of-window coverage is fatal (kind 'window') — whole batch rejects atomically, zero commits, error names offenders + remedy. Unmappable coverage fails closed. Exempt: block-boundary distillation, no-op resubmits. report.ts flags the new kind for acp_status consistency. Regression: kernel/tests/issue5830-merge-window-guard.test.ts (repro verified red pre-fix + 3 controls). Kernel 0.0.182 -> 0.0.183.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.11148-pr.5835.6f32606This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5835.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) CI 红灯已修:lock-sync(attempt 1/3,一次修完)。修复已推到 PR 分支,新 head 七问摘要:
判定:待新 head 上 CI 重跑转绿;此前因 lock-sync 失败被跳过的 test/static-gates 等车道会随之全量运行,若仍有红由后续 attempt 处理。 |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 open/closed issue 与 PR(近 7 天为主,按失败签名匹配)中无其他条目针对 #5830 的「plain range 合并越窗块、旧摘要被静默替换」机制;与 #5789(tier 蒸馏绕过 mid-turn 块)同属折叠资格判定家族但症状与机制均不同,不构成重复。无并行在途 PR 修同一线程。 标签:bug + scope/kernel + p1(理由:静默摘要覆盖丢失属数据质量/数据丢失类,按 owner 2026-10-19 修订一律 p1;修复面纯内核,不打客户端车道标签)。保留系统 审核进行中,先报已钉死的部分(完整八问结论随后补发):
|
…points resolve in place The literal-window containment check rejected legitimate folds whose span was extended by pair/reasoning boundary widening (#5531 stranded-pair recovery path): widened cells ride under the same summary by design, so when BOTH endpoints resolved in place (the requested message itself still occupies its slot), containment is now judged against the widened span computed over the range message refs. The #5830 producer stays fatal: an endpoint that resolved ELSEWHERE (stale inherited ref snapping to a distant inheriting-block anchor) holds the line at the requested window, as before. Unmappable coverage still fails closed either way. kernel 0.0.183 -> 0.0.184 (+ lockfile sync).
…age rejection The subtest-3 fixture numbered the pruned view fresh from nextIndex=1, which cannot model production: covered-but-pruned messages keep their original refs (ids are never re-minted), so their numbers sit inside the requested window even though the messages are gone. Seed an inherited ledger with the original numbering instead — the merge lands again. Add the companion negative pin (gate discipline #3617): the same shape with fresh numbering leaves b1's coverage unmappable in byRaw, so the fold must reject atomically with the [unmapped coverage] offender named.
The fail-closed rule treated ANY unmappable coverage id as an offender. Turn separators (#2627) are synthetic outbound-only markers that legacy polluted views registered into block coverage WITHOUT a ref entry; a later plain-range fold consuming such a block was rejected although its real-history coverage sat entirely inside the window. Real history ids always keep their byRaw entry (ids are never re-minted; only applyCompactionArchive prunes at native-compaction boundaries), so an unmapped id carries no client-side content a replaced summary could lose — skip it. A block whose ENTIRE coverage is unmappable still fails closed (containment unprovable); the pi#199 negative pin holds.
(ai: vllm-qwen/qwen3.8-27b) PR review 完成。判定:可合,等人工合并(kernel 层永不自动合并;PR 正文已带 kernel-change 声明)。审核中发现原 head 引入的 3 处回归已直接在分支上修掉(独立标注 commit),最终 head 八问
diff 面:7 文件 +344/−9(unit-projection 门 93 行含注释、report 2 行、两个测试文件、版本+双 lockfile)。收工判定:PR 内容本身可合;合并前请知悉上述唯一红腿为继承性 master 红(证据见上),由你决定是否容忍该腿或先走 master 车道修复。 |
CI shard flake on this PR (test ubuntu-22 s2, run 42288017742): 'receiver
port must be a number, got 0'. The fixture published its port with
appendFileSync — open(O_CREAT) and write are separate syscalls, so under
runner load the parent's existsSync poll can observe the file while it is
still empty, and Number('') === 0 trips the assertion. Reproduced locally by
splitting the publication (probe: exact same failure text); kernel-only PR
diff cannot touch this path.
Two facets of the same race, both test-side only:
- fixture publishes tmp + rename (atomic: existence now implies complete content)
- startReceiver waits until the content parses to a valid port > 0 instead of
trusting mere existence (keeps the 15s deadline and both failure modes)
(ai: vllm-qwen/qwen3.8-27b) CI 修复完成(第二轮唤醒):唯一红腿 七问
另外两件事:① 本轮之前作者在我上一轮的 lock-sync commit 之上推了 3 个 kernel 提交(cd2a934/02ed9c926/ce6f87738)并自行同步了 lockfile,lock-sync 已绿;此前 verif-snapshot-render 那条红已在 #4599 floor 1070 单独归账(master 侧 SMT 求解器覆盖问题,与本 PR 无关)。② 顺带发现 判定:PR #5835 全部 check 绿(mergeable_state 仅剩人工合并环节),可合并。 |
(ai: vllm-qwen/qwen3.8-27b) Progress update (merge-conflict lane, attempt 1/3): my session was force-restarted after a 31-minute stall — a git/network call hung, and I am resuming with hard timeouts on every step. Live state verified this run before the stall:
Next: rebase the PR branch onto current master, resolve all conflicts (expecting kernel version + lockfiles, possibly test-side files), run targeted local verification, then force-push (explicitly permitted for this conflict rebase) and report back with the 七问 summary. |
…dirty merge state, #5835) # Conflicts: # kernel/package-lock.json # kernel/package.json # package-lock.json
(ai: vllm-qwen/qwen3.8-27b) Merge conflict resolved. The PR was 七问
备注:mirror cache 本轮又滞后(长时间显示 5 个 test 分片 in_progress),真值以单次 GitHub API 拉取的主 CI run jobs 为准。已按协调政策在 #4599 留一行结论。 判定:冲突已解、全部 check 绿,可合并(合并动作由人工执行)。 |
…rge state, #5835) - kernel version: master 0.0.197 + this PR's merge-window guard -> 0.0.198 (kernel/src changes present in the merge, so a bump above master is required) - #5152 port-handoff race: keep the PR's atomic-publish receiver fixture + valid-port polling as a coherent pair; master's parent-polling fix is subsumed - lockfiles regenerated to kernel 0.0.198 (root + kernel, lock-sync clean) - build + typecheck + targeted kernel (issue5830-merge-window-guard, regression-empty-rewrap) and #5152 tests all green
(ai: vllm-qwen/qwen3.8-27b) Merge conflict resolved and pushed — PR is now mergeable. The PR (GitHub #5835) was in I merged current
Verified before pushing: Current state: 七问:
|
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Independent review of head fa82e6417 (branch contains all of current master faf0af61a). Findings only — this PR touches kernel/ and carries a kernel-change: declaration, so merge authorization stays with the owner (see the follow-up comment). No blocking findings; the fix is correct and precisely scoped.
1. Gate predicate matches the apply-side block-creation condition exactly. The gate (kernel/src/unit-projection.ts:270-357) fires on !isBlockBoundary && consumedBlockIds.length > 0 && filteredIds.length > 0. Checked against kernel/src/compress.ts: the filteredIds===0 && consumedBlockIds>0 plain-range shape is handled as an in-place refold or a no-op at compress.ts:1849-1894 before any block is created, and compress.ts:1898 no-ops only when both are empty. So the gate's "pure resubmits don't create a block" exemption is accurate — the predicate is neither over-broad (it would not reject refolds/no-ops) nor under-broad (it would not let a coverage-losing merge through).
2. Containment judgment is sound. Per-id containment of each consumed block's mappable coverage in the requested window; the span is widened by pair/reasoning only when BOTH endpoints resolve in place (preserving the #5531 stranded-pair recovery path), while a snapped endpoint — the #5830 producer via activeOwnerAnchor (boundaries.ts:271) — is judged on the literal window. Unmappable coverage fails closed. Reversed ranges are handled (min/max window + order-independent endpoint check).
3. Exemptions are principled. Block-boundary ranges (incl. mixed bN..mM) are explicit block merges: the model names a block and sees the consumed blocks' rendered summaries, so its distillation covers the merged content. The #5830 pathology is specific to plain ranges, where block consumption is a hidden side effect the model never asked for. The gate closes exactly that hole.
4. Atomicity and single-projection discipline (#3236) hold. fatal kind window is thrown at compress.ts:1818 with preLoopState rollback (zero commits — pinned by test 1's blocksCreated===0 + coverage deep-equal); recommend demotes fatal units to gaps (no advertise→reject loop); report.ts:514-515 flags the kind so acp_status rows match the apply verdict. The gate consumes consumedBlockIds after the #5789 recovery-ladder exclusion is applied, so recursive exclusion retries re-judge on the actual consumed set.
5. Tests pin both directions (#3617). kernel/tests/issue5830-merge-window-guard.test.ts: repro (stale-inherited start + pruned view → atomic reject naming b4, zero commits, no coverage loss) + 3 controls (legit in-window merge lands; covered resubmit no-op; bN..bM distillation untouched). kernel/tests/regression-empty-rewrap.test.ts: #199 fixture repaired to a production-shaped inherited ledger + a new fail-closed pin (unmappable coverage → reject). Ran both files locally: 8/8 pass. Real applyCompression API, realistic full-resend-host fixture.
6. House conventions. kernel-change: declaration present; kernel version 0.0.197→0.0.198 with both lockfiles in sync; unit-projection.ts is not a verif-guarded file (no verif work required; verif-subset-guard green on CI); no config surface, no new env vars; behavior change disclosed OLD→NEW in the PR body. The #5152 test-side race fix (atomic tmp+rename publish + valid-port polling) is test-only, clearly labeled, and correct.
CI on fa82e6417: 30 checks, 0 failed, 0 pending (14 skipped by design).
|
🔶 需要决策 本 PR 触碰
请 owner 授权合并或指示处置。 |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复——本线程 floor 6 已给过查重结论(open/closed issue 与 PR 无其他条目针对 #5830 的机制);本轮是 review lane 对同一 PR 的独立复审,非新立项。 本轮为 review lane 独立复审(head 八问
判定:PR 内容本身可合——修复正确、范围精准、测试双向钉死、CI 全绿(30 checks,0 failed)。因触碰 |
… (dirty merge state, #5835) Master advanced 271 commits (kernel 0.0.197 -> 0.0.199) since the last merge. Resolved 3 version conflicts: kernel version -> 0.0.200 (master's 0.0.199 + 1 for the PR's merge-window guard kernel/src changes), lockfiles regenerated to 0.0.200. Verified: build + typecheck green, issue5830-merge-window-guard 4 pass, regression-empty-rewrap 4 pass.
(ai: vllm-qwen/qwen3.8-27b) Run summary — re-resolved the dirty merge state (attempt 1/6)The PR had become dirty again since the last run: master advanced 271 commits (kernel What I did:
Live state after push (verified this run):
八问
判定: 冲突已解决,PR 现为 mergeable(blocked by CI/reviews,非冲突)。等待 CI 全量跑完 + review 后可合并。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI 红腿处置完成: (补发上一轮 CI 失败处置的结论——上一轮结束前未发出,此处补齐;已在 #4599 留一行状态。) 八问
判定:PR 内容本身可合(kernel 变更属人工评审档,待 owner 授权,见 floor 26/28);唯一红腿是已知负载 flake,已触发重跑,转绿后全矩阵即绿。若重跑再红,等 #7164 合入后重触发。 |
ranxianglei
left a comment
There was a problem hiding this comment.
Re-review at new head aa4175ffa (master merge) — findings only (kernel PR, not approved).
What changed since my last review (fa82e64): a single merge commit integrating master (a446a3c1c). The PR's own contribution — the window gate in projectUnit — is byte-identical to what I reviewed and verified correct (predicate at kernel/src/unit-projection.ts:295, fatal kind at :347, error message at :351 — none touched).
The master merge is clean. Two master-side changes landed in unit-projection.ts, both in regions orthogonal to the gate:
- The last-user-anchor note now self-labels as a "historical quote, NOT a new user message" (
:470) — a model-facing text change from the ti_an incident, already merged on master. - #6709's content-free BLOCKED partner un-doom logic (
:665-739) — lets a content-free BLOCKED partner flow to rung 2 instead of dooming the #5919 merge; already merged on master.
Execution order keeps them from interacting: the window gate throws at :295, before the #5919 recovery ladder at :665+ runs. A window-guard fatal never reaches the #6709 path, and the #6709 change only alters which stranded partners doom a merge — it does not change which blocks are consumed, so it cannot feed a new case into the gate.
Version: the PR's 0.0.198 bump collided with master's 0.0.199 (lock-sync #3782); the merge resolved it to 0.0.200, consistent across kernel/package.json + root package-lock.json + kernel/package-lock.json (version-guard green).
Verification at the new head:
- Local targeted: 16/16 pass (
issue5830-merge-window-guard4 +issue5919-shredded-merge#6709-lockstep +regression-empty-rewrap#199). - CI on
aa4175ffa: 30 checks, 0 failed (26 success / 1 in-progress / 3 skipped-by-design);kernel-guard,version-guard,lane-guard,architecture-docsall green. - master
a446a3c1cfully contained; prior PR headfa82e6417is an ancestor.
Verdict: no new blocking findings, no small findings to fix. The gate remains correct and precisely scoped; the merge is clean. This PR touches kernel/ and carries a kernel-change: declaration, so per Kernel Boundary rule 3 it stays at the human-only review tier — not approved here. See the follow-up 需要决策 comment for the authorization request.
|
🔶 需要决策 本 PR 触碰 本轮为 head 更新后的复审(
门禁 diff(相对 master 独立审核结论:修复正确且精准,无阻塞发现,无小发现需修。详见上方 findings-only review。 行为变更(PR 正文已披露 OLD→NEW,未变):越窗的普通 message-ref 区间折叠,旧行为 = 静默落盘(被吞块摘要被单尾段摘要替换,#5830 生产事故:一块罩 1566 条消息只带 2456 字符尾摘要)→ 新行为 = 整批原子硬拒 + 点名越界块与两条出路(bN..bM / 收窄区间)的可操作错误;窗内合法合并、蒸馏、幂等 resubmit 均不变。 请 owner 授权合并或指示处置。 |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复——本线程 floor 6 已给过查重结论;本轮是 review lane 对同一 PR 新 head( 本轮为 head 更新后的独立复审( 八问
判定:PR 内容本身可合——修复正确、范围精准、测试双向钉死(本地定向 16/16)、CI 全绿(30 checks,0 failed)、master 合并干净。因触碰 |
… (dirty merge state, #5835) Master advanced 99 commits (kernel 0.0.200 -> 0.0.201) since the last merge. Resolved 3 version conflicts: kernel version -> 0.0.202 (master's 0.0.201 + 1 for this PR's merge-window guard kernel/src changes), lockfiles regenerated to 0.0.202. Verified: build + typecheck green, issue5830-merge-window-guard 4 pass, regression-empty-rewrap 4 pass, issue5152-owner-beat 6 pass.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 冲突已解决并推送:merge commit 七问
状态:最终 head |
ranxianglei
left a comment
There was a problem hiding this comment.
Re-review at head a3abdd7c203 (third head; progression fa82e6417 → aa4175ffa → a3abdd7c203).
What changed since the last head (aa4175ffa): one master-merge commit integrating master ba2e8198203. The PR's own contribution is byte-identical — kernel/src/unit-projection.ts (the window guard) and kernel/src/report.ts are untouched by the merge (git diff aa4175ffa a3abdd7c203 -- kernel/src/unit-projection.ts kernel/src/report.ts is empty).
Master-merge cleanliness: the merge pulled in the #7428 batch-seam double-claim fix in kernel/src/compress.ts (refreshes preExistingCoverage at the top of the apply loop so each sibling range claims against earlier siblings in the same batch; refines the no-op warning wording). I traced the call flow: projectUnit (the guard) runs at compress.ts:1780 inside applySingleRange, which runs after the preExistingCoverage refresh at :817. The guard reads consumedBlockIds + block.effectiveMessageIds + messageRefs — it does not read preExistingCoverage, so the batch-seam fix does not change the guard's judgment for a given range. The only interaction is the intended one: earlier siblings now create active blocks in the shared state, so a later sibling's consumedBlockIds reflects them — which is exactly the cross-window merge the guard exists to reject. No regression.
Version: master 0.0.201 → PR head 0.0.202 (version-discipline consequence of the in-repo kernel change; consistent across kernel/package.json + both lockfiles).
Verification at the new head: local targeted 19/19 pass (window guard 4 + #7428 batch-seam + #6709 shredded-merge + #199 empty-rewrap); CI 30 checks, 0 failed, 0 pending; master ba2e8198203 fully contained, prior head aa4175ffa is an ancestor.
Gated diff (vs master ba2e8198203): kernel/src/unit-projection.ts (+93, the window guard), kernel/src/report.ts (+2, acp_status flag), kernel version 0.0.201→0.0.202 + both lockfiles, kernel/tests/issue5830-merge-window-guard.test.ts (+174, 4 tests), kernel/tests/regression-empty-rewrap.test.ts (+76/-1, #199 fixture repair + fail-closed pin); plus test-only #5152 race fix (2 files, zero product behavior).
Independent verdict: the fix is correct and precisely scoped; the master merge is clean; no blocking findings, no small findings to fix.
Authorization boundary: this PR touches kernel/ and carries a kernel-change: declaration — per Kernel Boundary rule 3, kernel changes sit at the human-only review tier. I am not approving. See the follow-up 🔶 需要决策 comment for the owner-authorization request.
|
🔶 需要决策 本 PR 触碰 本轮为 head 更新后的第三轮复审(
门禁 diff(相对 master 独立审核结论:修复正确且精准,master 合并干净,无阻塞发现,无小发现需修。详见上方 findings-only review。 行为变更(PR 正文已披露 OLD→NEW,未变):越窗的普通 message-ref 区间折叠,旧行为 = 静默落盘(被吞块摘要被单尾段摘要替换,#5830 生产事故:一块罩 1566 条消息只带 2456 字符尾摘要)→ 新行为 = 整批原子硬拒 + 点名越界块与两条出路(bN..bM / 收窄区间)的可操作错误;窗内合法合并、蒸馏、幂等 resubmit 均不变。 请 owner 授权合并或指示处置。 |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复——本线程 floor 6 已给过查重结论;本轮是 review lane 对同一 PR 第三轮 head( 本轮为 head 更新后的第三轮独立复审( 八问
判定:PR 内容本身可合——修复正确、范围精准、测试双向钉死(本地定向 19/19)、CI 全绿(30 checks,0 failed,0 pending)、master 合并干净。因触碰 |
(ai: vllm-qwen/qwen3.8-27b) 第三轮复审收尾确认(head
结论:无阻塞发现、无小发现需修,PR 内容本身可合。因触碰 |
Closes #5830
kernel-change: add a hard gate in projectUnit so a plain message-ref range can never merge an active block whose coverage extends beyond the requested ref window — the missing check behind #5830's silent summary-coverage loss. The judgment lives entirely in the kernel's shared projection; the host side cannot fix this.
What was wrong
A plain
mNNNNN..mMMMMMrange carries one summary describing exactly that ref window. ButprojectUnitdecided which existing blocks to consume by asking only whether each block's visible representation (its rendered summary, or earliest surviving raw) falls inside the resolved span. After a folded historical ref resolves viaactiveOwnerAnchorto its inheriting block's anchor position (an unbounded jump in view-index space), the resolved span can be far larger than the requested window — every same-tier block swept along gets merged, itseffectiveMessageIdsunioned into the new block while the new block carries only the last range's summary. The prior summaries are silently replaced; the receipt reports success. #5830 observed this as one block covering 1566 messages from conversation start carrying a single 2456-char summary.Fix
One decision-point gate in
projectUnit(kernel/src/unit-projection.ts), shared by apply / recommend / acp_status per the #3236 single-projection discipline:[startRef..endRef]is now fatal (kind: "window"): the whole batch rejects atomically, zero commits, and the error names the offending blocks with their coverage extent plus how to proceed (reference block ids directly for deliberate merges, or narrow the range).bN..bM) absorbs boundary coverage by design; no-op resubmits create no block, so nothing is replaced.kernel/src/report.tsflags the new fatal kind soacp_statusrows stay consistent with the apply verdict.Behavior change (disclosed)
OLD: such folds landed successfully, silently replacing the consumed blocks' summaries (active-summary coverage loss; originals recoverable via decompress through directBlockIds).
NEW: they are hard-rejected with an actionable error; the rest of the batch is untouched (atomic rollback).
WHY: the supplied summary cannot cover the merged content — landing it is data-quality corruption dressed as success. Legitimate merges (coverage inside the window), distillation, and idempotent resubmits are unchanged.
Tests
New
kernel/tests/issue5830-merge-window-guard.test.ts(4 tests, all-real API calls):bN..bMdistillation unaffected.Targeted suites green on rebased head (local): issue5830 (4), issue5751-tool-pair-widening (6), tool-pairs (14), issue3236 (5), regression-bugfixes (12), issue4484 (4), dead-ref-split (5), dead-refs (6), refold (20), carrier-advertise-validate (5), issue3371 (3), compress (32), decompress-report (16), panel (12), parse-compress-input (98), checkpoint-carrier (4), issue4289 (14), pipeline (15), verif-subset-guard (2). Full-suite verdict = CI.
Kernel version: 0.0.182 → 0.0.183. No config surface changes. No new env vars.