Repository navigation
[PR] fix: re-anchor fold coverage mutated by host tool-result pruners via byte-verified sentinel shape (#5395) - #5401
ranxianglei wants to merge 1 commit into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8937-pr.5401.533fe60This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5401.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 标签: 八问
验证证据(独立产出,未采信 issue 侧结论):
与在途单关系:#5398(#3849 的 absent-retire 有界出口)正交互补——本 PR 让「被宿主改写的 id」重新可认领,#5398 让「真消失的 id」有界退出,合并顺序无依赖。 判定:ready-to-merge。身份认领逻辑按 §7.4 属人工 review 层(message-ref/id logic),以上为我的独立审核结论,合并由 owner 决定。 |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: ready to merge (approve-equivalent). Posted as a comment-review — GitHub refuses to record a formal approval on a self-authored PR. Independent end-to-end read of the full diff (6 files, +518/−13). No kernel/ touch, no config-surface change, no wire-shape change. Findings below are my own trace, not a restatement of the issue thread.
Correctness (verified against surrounding code, not just the hunks):
verifyPruneShape(src/fold-reconcile.ts) is an exact decomposition: head+marker+tail where head/tail are byte-exact prefix/suffix of the stored original, non-empty deleted middle enforced byoriginal.length >= headLen + tail.length + 1(which also kills overlapping prefix/suffix false-matches), trying every marker occurrence (the double-prune case). Byte-level identity ⇒ mis-attribution is structurally excluded.- Runs LAST in
planReconciliation(after every exact pass + the #3641 shape pass), so it can't steal a claim a stronger channel made — pinned by the "stable toolCallId ⇒ byTool=1, byPrune=0" control test. - k-th discipline holds: backbone order; one oldId→≤1 candidate (break); one candidate→≤1 oldId (
claimedCandidates). The shared-head/tail test pins "one candidate serves at most one old id." - #5142 interaction clean:
missingSetexcludesexpectedAbsentids at construction (!expectedAbsent?.has(id)), so legitimately-retired ids cannot be re-anchored here. - Zero steady-state storage reads: originals are pulled only when a marker-bearing resend exists, and only for superset-filtered missing ids (cap 64); no marker on the wire ⇒ index untouched (contract pinned by a test).
- Reasoned in BOTH compression modes: the pass lives in the shared planner/core with no pluginAgent-dependent branch, so plugin and proxy lanes behave identically.
Regression inventory (existing behaviors the diff touches, each judged):
- All 5 early returns + the final return of
reconcileFoldCoveragecarry the newbyPrune(additive field complete; no partial result shape). FoldReconcileResult.byPruneandConflictKind.host-content-mutationare purely additive; the existingdeepEqualpin intests/fold-reconcile.test.tswas updated to match.- Drift log lines gain one optional clause (inert when
byPrune == 0); the conflicts banner gains a first-match-wins advice tier (mixed ledgers keep their existing tiers verbatim). - The re-anchor only repoints block coverage onto the client's own resent id — bytes stay client-verbatim ⇒ prefix-cache stable; the one-time unfold→summary transition is the intended fix and is disclosed old→new in the PR body. No fail-fast→swallow, no threshold/timing/output-format drift, no cache-invalidation-strategy change.
Tests: genuine regression repro (re-minted toolCallId + pruned bytes ⇒ byPrune=1, converges in one pass) plus #3617 both-directions gates (foreign-original / no-marker / growth / role-mismatch / storage-absent each assert byPrune=0 + honest unmatched), cross-pass convergence (the bug was NON-convergence; the second pass is a clean resend with no ledger spam), warn-mode parity, k-th discipline, and the zero-read contract. Assertions check specific claim mappings / counts / state transitions — not tautologies.
CI: green on head fa6bcd08b (30 checks, 0 failed, 0 pending).
One non-blocking note (pre-existing, NOT introduced by this PR, benign): the #3641 shape pass claims ids after the plan.unmatched freeze but does not drop them from plan.unmatched (this new pass does, at src/fold-reconcile.ts:1657). It is harmless because any shape claim sets byShape > 0, which both forces shapeExplained and zeroes totalDrift, so shape-claimed ids never reach the destroy verdict. Left as-is deliberately (changing it would alter already-tested #3641 behavior); flagged for completeness only.
Merge stays human per house rules.
(ai: vllm-qwen/qwen3.8-27b) Review complete — verdict ready to merge. Independent end-to-end read of the full diff (6 files, +518/−13): no |
…byte-verified sentinel shape (#5395) Host compression plugins (dsh desktop's tool-result-pruner) splice a versioned sentinel between preserved head and tail of long folded tool results. The middle deletion breaks every identity channel at once (norm/ twin equality, wordbag/simhash distance, positional fence); when the host also re-mints tool-call ids across projections (#2396 class) the covered id goes PERMANENTLY unmatched — re-reported every turn, riding unfolded and feeding #2438/#5223 escalation (reported: 614 ids x 156 turns/day). A: pruner-shape pass — LAST channel in planReconciliation. Claims only on positive evidence: resent bytes decompose as head + sentinel + tail with head/tail byte-matching the STORED original (unified index), non-empty deleted middle (overlap guard). No toolCallId equality gate (re-minting is exactly the permanent-drift condition). Steady state pays zero storage reads (marker substring scan first; rows pulled only when a sentinel is on the wire). Store unavailable -> honest unmatched. B: each verified claim lands one 'host-content-mutation' conflict-ledger event with its own advice tier — the second compressor becomes visible on banner/stats instead of staying log-only. Self-limiting: repair mode re-covers the pruned ids, next pass sees them present. Behavior change (intended, the bug fix): pruned covered ids were previously permanently unmatched (drift warn every turn forever); they now re-anchor in ONE pass and converge to clean resend. Additive FoldReconcileResult.byPrune field; additive ConflictKind member; no config surface, no kernel/, no wire shape change. Runs identically in plugin and proxy lanes.
fa6bcd0 to
288f2e2
Compare
(ai: vllm-qwen/qwen3.8-27b) Both blockers cleared — PR is conflict-free and fully green at head 1. Merge conflict (previous wake, resolved this session). Rebased 2. Red CI on the rebased head (this wake). Live-verified first: 1 of 30 checks red — Live state now (verified this run): all 48 check runs on 结构化八问(中文)
判定:冲突已解、CI 全绿、mergeable=true,PR 处于可合并状态,等待人工合并。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Confirmation close-out (full detailed summary already posted above, comment id 70437):
Verdict: both blockers cleared, PR is conflict-free and fully green — ready for human merge. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Re-verified live this run — no action needed: the reported |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 七问(本轮 CI 处置摘要;前两轮完整记录见上方 comment id 70437)
|
查重
与 #3849(永久 unmatched 残差 riding wire)同症状家族但机制不同:#3849 的残差来自 wire/dialect 切换后所有身份通道同时失明;本 issue 的残差来自宿主自己的 tool-result pruner 逐字改写了 bili 已折叠消息的正文(head+MARKER+tail 拼接),内容哈希/simhash/wordbag 全部失配,且消息仍在、或然清扫也不触发。两单已互链;#3849 的 absent-retire 兜底走 PR #5398,本 PR 修的是本 issue 的根因(把这类变异变成可认领的)。无其他重复在途 PR。
是什么问题
v0.1.210 plugin lane(dsh desktop 自带
tool-result-pruner压缩插件)会把 >8KB 的工具结果原地改成前 4KB + "\n\n[... tool result middle pruned ...]\n\n" + 后 1KB。bili 的消息身份是内容哈希——正文一变,已折叠块覆盖的 id 就永远对不上任何锚点,每个回合都报一次resent history drifted: N covered id(s) missing with no anchor match,永不收敛(报告者实测 614 ids × 156 次/天)。修法(A+B,均为 host 侧,不碰 kernel/)
A — 字节验证的 pruner-shape 重锚通道(
src/fold-reconcile.ts):reconcile 规划器新增最后一个认领通道。仅当线上出现带已知 sentinel marker 的重发文本时才激活(稳态零存储读取,一次 indexOf/消息);此时只拉取「缺失 covered id ∩ 候选匹配(role/contentType 一致且变短)」的原文(上限 64),逐字节验证newText = original[0..head] + MARKER + original[-tail..](非空中间删除 + 重叠守卫)才认领。这是最强证据等级:重发字节字面分解为已知拼接形状且头尾与存储原文逐字相等,误绑概率≈0。故意不做 toolCallId 相等门——宿主跨投影重铸 id(#2396 类)正是永久漂移成立的条件之一。marker 表内置常量(当前仅 dsh pruner 一个),不开配置面(配置面 owner-gated;新宿主接入走 PR)。B — 冲突台账可见性(
src/conflict-watch.ts):新增 ConflictKindhost-content-mutation,byPrune>0 时记一条(字节级实证的第二压缩器证据),面板冲突 banner/advice 首次能显示「双压缩器」状态而非只有日志。自限:repair 模式重锚后下一 pass byPrune 归零。行为变更披露(旧→新)
byPrune: number(additive)host-content-mutation(additive)无配置面变更、无 kernel/ 变更、无 wire 形状变更、无默认值变更。
两种压缩模式
通道在 planReconciliation 核心内运行,plugin lane 与 proxy lane 共用同一 reconcile 路径,行为一致;测试含 proxy(pluginAgent undefined)等价用例。
验证
npm run typecheckclean(rebased head)Review 提示
身份认领逻辑按 §7.4 属人工 review 层(message-ref/id logic)。与 master #5142(expected-rebase accounting,同文件热区)已 rebase 合并:两者正交——#5142 让「合法退役的 id」退出 missing,本 PR 让「被宿主改写的 id」重新进入认领,参数各占一位(expectedAbsent, storedById)。
Closes #5395