Repository navigation
[PR] fix(fold-reconcile): a surviving direct-ID majority is partial loss, not total drift (#5715) - #5737
Conversation
…not total drift (#5715) A session where a strict majority of hard-covered ids still ride verbatim on the wire (field case: 1162/1171 direct-present with ten interior turns missing) was classified totalDrift because the verdict only saw claims==0 + unmatched>0 and never counted the surviving direct matches. After FOLD_DRIFT_ESCALATE_PASSES that falsely latched foldEpochConfirmPending and paused compression on a healthy DSH session (preflight 502, zero compressible ranges). Fix (lane-agnostic): compute a strict direct-ID majority over the covered set (separators / expected rebases / reasoning cores excluded); void the terminal epoch-confirm verdict on it and auto-clear any stale pending latch. totalDrift itself is untouched - the #3641 drift streak + frozen witness keep a genuine mid-history edit reading as honest loss (#4050/#2695), so only the terminal verdict is gated, never the classification. Genuine wholesale replacement has ~0 directly-present covered ids, so the gate cannot mask a real collapse; a negative control pins that detection stays intact. Behavior change: (a) healthy-majority partial interior loss no longer latches epoch-confirm / pauses compression; (b) at the mass bar such a session now logs one info note instead of a repeated drifted warn (sub-mass-bar single edits unchanged).
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8782-pr.5737.b639881This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5737.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复 —— 本线程(修复 #5715)仅此一个在途修复 PR(#5737),无并列第二 PR。 这是我自己已完成的 PR(正文带 代码审核(独立重读,非照抄 issue 结论)
测试有效性(本地定向实测)
CI 状态 —— 红,但与本 PR 无关(继承性 master 红)
结构化八问
收工判定:代码审核通过、测试有牙、CI 红系继承性(已 merge ci-fixes 收敛)。等新头 |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复(同症状家族,但与既有票的车道/机制不同):
标签:bug / zcode / scope/wiring / p1(缺陷类;ZCode 纯代理车道;请求管道会话绑定门;fork 子会话 413 不可用+缓存命中归零,按 p1 口径①「会话恢复失败类」定级)。 审核结论(已对 master 源码逐项核验):
修复方案(F1+F3):
接下来推分支开 PR,链接回帖。 |
(ai: vllm-qwen/qwen3.8-27b) **PR 已开:https://github.com/ranxianglei/billion-context/pull/5748**(分支 按八问:
行为变更披露(有意):① identified 客户端 fork 会话的首请求从「fresh start(零 refs/blocks/lineage)」变为「继承父 refs+fully-present blocks+lineage」(与 resume 同待遇);② 残留(本票清单跟踪,不在本 PR):场景 B(incoming 比父链长但内容断掉)需要升级到 0.1.212 后由 describeResumeMiss 给出分叉 cell 号才能定位——OP 升级后重做 B 场景贴一下即可;F4(preflight fail-fast 止血)与 F5(摘要器降级折叠)属另一失败域,待本 PR 落地后单独处理。 |
(ai: vllm-qwen/qwen3.8-27b) 本轮红 CI 处理结论:无需改代码,rerun 后全绿 本轮唯一红项 =
七问:
|
…plained episodes Review finding on the #5715 PR: the new one-per-episode info note fired in ADDITION to the existing benign notes whenever an explained mass episode (shape churn / head truncation / tail rewind) also happened to keep a strict direct-ID majority of covered ids — e.g. a host dropping the oldest ten turns of a thirty-turn covered span logs BOTH the #2695 truncation note and the #5715 'partial interior loss' note, the latter mislabeling a head cut as interior loss (verified by fixture replay: two info lines per episode where master logs its original surface). Gate the note on !shapeExplained && !truncationExplained && !tailTruncationExplained so explained episodes fall through to their pre-existing log behavior byte-for-byte; the unexplained majority-partial-loss case (the actual bug) is untouched. Regression test pins the no-double-report shape.
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE / 可合(GitHub 禁止作者自批,按舰队惯例以 COMMENT 态评审提交;合并由 owner 人工执行)— the fix names the producer and fixes it at the decision point: the classifier's totalDrift verdict itself is deliberately left untouched (preserving #3641's drift-streak / frozen-witness coupling), but the escalation path now consults the one dimension the old code ignored — how many covered ids still ride verbatim on the wire — and a latched epoch-confirm gets an auto-clear condition. Independent verification performed:
- Direction (①): the root cause was that escalation keyed only on
plan.claims.size===0 && realUnmatched.length>0and never counted surviving direct matches, so a 1162/1171 survivorship read as total drift. The newhardDirectMajoritypredicate (presentHard >= MIN_RESYNC_RUN && 2*presentHard > coveredHard, denominator = covered ids minus separators / expected rebases / reasoning) encodes exactly the issue's thesis. Replay of the #5715 shape (30-turn covered span, ten interior turns gone, claims/sim both empty): master escalates after 3 passes and latchesfoldEpochConfirmPending=true; this head stays clean and logs one info note per episode. - Regression pass (②) over the touched path's pre-existing behaviors: the three explainer families (#2695 head truncation / #3724 tail rewind / shape churn) each keep their own one-per-episode note and their original log surface — byte-for-byte parity with master verified by replay (see the spot fix below); wholesale replacement (presentHard≈0) still escalates — pinned by the negative-control test, so genuine epoch drift is not masked; the drift-streak accumulation, frozen-witness logic, and preflight read site are untouched; no wire bytes, stamps, or storage formats change.
- Tests (④): the new doctrine cases have teeth — the escalation case fails if the gate is inverted (proven by replaying the same fixture against master's
src/fold-reconcile.ts: it escalates and latches), and the regression test I added pins the no-double-report log shape. Full-suite verdict = CI lanes on this head: green (17 success / 13 platform-skipped / 0 failed atc39dffb6d). - Diff cleanliness (⑤/⑤b): fix commit is 2 files (+48/-1 src, +66 test); the ci-fixes merge carries the sanctioned dsh-pin realignment (#4599 convergence work, labeled as such in its commit). Naming, comments, and style match the surrounding module.
Behavior-change disclosure (old → new):
- Partial interior loss where a strict majority of covered ids still rides verbatim: before — after 3 passes the session latched epoch-confirm and preflight paused compression permanently (the #5715 bug); after — classified as healthy substrate, no latch, no pause. This is the intended fix.
- Unexplained mass-regime majority (≥10 non-reasoning unmatched, majority surviving, no explainer): before — repeated per-pass warn lines; after — one info note per episode. Log-surface only, no decision change beyond the first bullet.
- Explained episodes (truncation / tail rewind / shape churn): unchanged — identical log surface to master.
已顺手修:#5715 的 info note 原本也会在「已有解释」的回合上叠加输出(宿主裁掉覆盖段最老的十轮时,同轮既打 #2695 截断 note 又打一条把头部裁剪误标为 "partial interior loss" 的新 note——用 fixture 回放对照 master 原日志面证实了双行);给新分支加上三个解释器谓词门控,已解释回合落回原有日志行为,并补了一条钉住「不双报」形状的回归测试。+ commit c39dffb
结构化八问
- 是什么问题:折叠覆盖段里绝大多数消息仍逐字在网线上、只有少量中间回合缺失时,reconcile 层把它误判成「整体漂移」,连续 3 轮后锁死 epoch-confirm,该会话的压缩从此被 preflight 永久暂停(fail-fast 502、0 compressible ranges)。
- 什么原因导致的:升级判据只看
claims==0 && unmatched>0,从不数仍在网线上逐字存活的 id——99% 存活被读成 100% 漂移;且锁死后的持久化 pending 状态没有清除条件(旧 reset 只在 prevStreak≠0 时触发)。 - 影响范围与严重程度:任何插件车道会话,只要宿主丢掉少数中间回合而保留压倒性多数(DSH 报告,Windows,bili 0.1.212)即触发 → 该会话压缩能力永久不可用、需人工介入 = p1 可用性缺陷。
- 怎么修复的:新增单一
hardDirectMajority谓词(逐字存活的 covered id ≥3 且严格过半),用在三处:为真时不升级、为真时自动清除已锁死的 epoch-confirm、为真时每回合一条 info note 代替升级。totalDrift本身有意不动,保住 #3641 的诚实丢失语义。 - 是否有回归问题:审过触碰路径的既有行为——三个解释器家族(#2695/#3724/形状抖动)的 note 与日志面与 master 逐字节一致(回放验证);全量替换仍会升级(负控制测试钉住);漂移计数/冻结见证/preflight 读取点未动;无 wire/时序/阈值漂移。
- 是否破坏缓存:不涉及——不改 wire 字节、戳、存储格式,仅分类决策与日志行。
- 是否改了设计方向:不涉及——同一分类器架构,升级前多查一个证据维度;kernel/ 未触碰。
- 是否引入新配置:不涉及。
判定:可合。
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) PR #5737 审核完成:APPROVE / 可合。
|
…ge only (#5715) A field report on an independent 0.1.213/Linux-DSH session confirmed the same mechanism and surfaced a residual class the first cut missed: ids from a successfully REWOUND inactive compression block (#4050 view-deactivated lineage, re-fed into `covered` via recoveryCandidateIds) legitimately leave the wire after a rewind. Counting those absent inactive recovery candidates in the denominator dilutes a healthy active direct-ID majority below the bar and re-latches epoch-confirm on a live session. Compute hardDirectMajority over ACTIVE coverage only (the coveredIdsOf caliber), not the full `covered` set. totalDrift is still left untouched — the #3641 drift streak + frozen witness keep a genuine mid-history edit reading as honest loss (zombie accrual); only the terminal verdict is gated. The strict > half tie-breaker is kept: a repeated exactly-half loss still escalates, which is safer than inferring health from "not obviously lost". Adds three doctrine tests: absent inactive recovery candidates cannot dilute a healthy active majority; reasoning-only survivors do not mask a hard text loss; exactly-half present is not a clear majority (strict boundary).
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE / 可合(GitHub 禁止作者自批,按舰队惯例以 COMMENT 态评审提交;合并由 owner 人工执行)
Round-2 review at head 3298cd8ff7, re-done end-to-end after the master sync and the new commit 3298cd8ff7 ("run the surviving-majority test on active coverage only").
What changed since round 1 (c39dffb6d3)
5a91fd9b89— merged current master into the branch. Its contribution to this PR's own files is exactly ONE line: a doc-path comment fixup insrc/fold-reconcile.ts:35(MESSAGE-IDENTITY.md→docs/message-identity.md, from the docs centralization). No conflict-resolution surface insrc/fold-reconcile.tsor its tests; my round-1 log-surface gate survived intact (src/fold-reconcile.ts:2465-2467).3298cd8ff7— narrows thehardDirectMajoritybase from the fullcoveredset (active ∪ #4050 recovery lineage) to ACTIVE coverage only (coveredIdsOfcaliber, frozen into a separate never-mutatedactiveCovered;coveredkeeps its mutable role for the claim ladder and the early-return reset). Triggered by the independent field report on #5715 (0.1.213/Linux-DSH): ids from successfully rewound inactive blocks legitimately leave the wire, and counting their absence in the denominator dilutes a healthy active majority below the bar and re-latches epoch-confirm on a live session.
Direction check: the commit names the producer. Recovery candidates are "recoverable-but-gone" (#4050 view-deactivated lineage re-fed via recoveryCandidateIds) — their post-rewind absence is legitimate history, not substrate destruction, so they were a category error in the majority denominator. Narrowing the base only moves the ratio toward health when the live substrate is healthy; it cannot mask a real collapse because wholesale replacement has ~0 directly-present ACTIVE ids (negative-control test pins escalation). totalDrift stays untouched (#3641 drift-streak/frozen-witness zombie accrual preserved); the strict >half tie-breaker is kept with an explicit boundary test.
Regression audit (pre-existing behaviors on touched paths)
- Escalation gate (
src/fold-reconcile.ts:2410), stale-latch auto-clear (:2209), and the mass-bar log branch (:2465) all consume the narrower base — intended, disclosed below. - Early-return reset (
covered.size === 0) still uses the FULL set ✓ unchanged; claim ladder, per-block coverage, anchor bookkeeping untouched. - A session whose substrate is entirely inactive lineage can never benefit from the gate (empty active base ⇒ no majority) — conservative direction, consistent with the doctrine "under-detecting a real collapse is worse than one confirmable prompt".
- Behavior change old→new (disclosed per §4 deliverable 4): the majority base changes from active∪inactive-recovery to active-only; consequently sessions with a healthy active direct-ID majority plus large absent rewind residuals no longer latch/pause (they could before), and the info note's
${presentHard}/${coveredHard}now reports the active-caliber ratio. No config surface, wire shape, persistence format, or kernel change.
Tests: three new doctrine tests. Teeth verified empirically: replaying the new test file against the PREVIOUS head's source fails exactly one test ("absent inactive recovery candidates cannot dilute a healthy active majority", 15/16), passes at this head (16/16) — the narrowing is pinned. The other two pin pre-existing rules (reasoning excluded from both numerator and denominator; strict >half boundary) and pass on both heads. Local at this head: doctrine 16/16, all 26 fold-reconcile-surface test files 314/314, npm run typecheck clean. CI at 3298cd8ff7: at posting: 6 success / 11 skipped / 0 failed / 13 still in flight (test shards + static-gates) after ~20 min of monitoring — no failure observed in any completed lane.
One record-only observation (no action owed): the PR body predates 3298cd8ff7 and still describes the majority as computed "over the covered set"; the authoritative up-to-date description is in the commit message and the author's follow-up comment on #5715. Noted for the merge record only.
结构化八问:
- 是什么问题:长 DSH 会话的活跃直接 id 多数健康,但成功 rewind 后失活的压缩块(#4050 谱系)其 id 合法离开 wire;首版把这些「可恢复但已离场」的残差算进多数分母,把健康的活跃多数稀释到阈值以下 → 误判 total drift → epoch-confirm latch → preflight 永久暂停压缩(现场:0.1.213/Linux,foldDriftStreak=70、0 compressible ranges)。
- 什么原因导致的:多数基数取错口径——本应只用 active 口径(
coveredIdsOf),却用了 active∪inactive-recovery 全量;inactive recovery 候选只增分母不增分子。 - 影响范围与严重程度:仅「活跃直接 ID 多数健康、且失活残差多到足以翻转多数判定」的长会话;此前会被误 latch 而暂停压缩(可用性缺陷类),触发条件比首版 bug 窄得多。
- 怎么修复的:基数收窄为 active-only(冻结成独立对象,不动原
covered的可变用途);仍只在终局判决处 gate(!hardDirectMajority)+ 残留 latch 自动清除 + mass-bar info note;totalDrift/streak/frozen witness/zombie 机制一律不动;严格 >half 平局裁决保留并有边界测试钉住。 - 是否有回归问题:审过既有行为——early-return reset 仍用全量
covered;claim ladder / per-block coverage / anchor 记账未动;真·整体替换负控仍会 escalate;首版日志面门控在 master sync 后原样存活(src/fold-reconcile.ts:2465-2467)。行为变更(已披露):基数口径 active∪inactive → active-only,「活跃健康 + 大量失活残差」类会话不再被 latch。 - 是否破坏缓存:不涉及——无缓存、重试、增量游标、去重键改动。
- 是否改了设计方向:没有——与首版同一机制(只 gate 终局判决、不放宽 totalDrift),仅修正基数口径;架构/边界/数据流未转向。
- 是否引入了新的配置:没有。
判定:可合(合并由 owner 人工执行)。
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Round-2 review of PR #5737 at new head
|
Problem (#5715)
A DSH session where a strict majority of hard-covered ids still ride verbatim on the wire (field case: 1162/1171 direct-present with ten interior turns missing) was misclassified as
totalDrift. The verdict insrc/fold-reconcile.tsonly sawclaims==0 && unmatched>0and never counted the surviving direct matches. AfterFOLD_DRIFT_ESCALATE_PASSES(3) that falsely latchedfoldEpochConfirmPending, and preflight then paused compression on a healthy session —preflight fail-fast 502 … 0 compressible range(s)— permanently until a manual reset.Interior loss (neither a head cut nor a tail rewind) defeats every existing explainer (
truncationExplained,tailTruncationExplained,shapeExplained, …), so none of them suppressed the verdict.Fix (lane-agnostic)
In
reconcileFoldCoveragecompute a strict direct-ID majority over the covered set (separators / expected rebases / reasoning cores excluded — same denominators as the per-block coverage):and use it in three places:
!hardDirectMajority. The latch and its compression pause no longer fire on a healthy-majority session.foldEpochConfirmPendingremains with a zero drift streak (the reporter's second site), clear it immediately. This fires even when this pass still carries residual loss (a partial restore keepstotalDrifttrue, so the normal!totalDriftreset would never reach it).drifted/mutation/deletionwarn. Sub-mass-bar single edits keep their exact original per-pass warn.Why not relax
totalDriftitself?Deliberately left untouched. The #3641 drift streak + frozen witness are what keep a genuine mid-history edit reading as honest loss (zombie accrual) rather than sanctioned truncation (#4050/#2695). Relaxing the classification breaks that coupling; only the terminal verdict needs gating.
Why this can't mask a real collapse
Genuine wholesale replacement / host-native compaction has ~0 directly-present covered ids, so
hardDirectMajorityis false and detection proceeds unchanged. A negative-control test pins that.Behavior change (flagged per §4 deliverable 4)
epoch-confirm/ pauses compression. Old: anyclaims==0+ ≥10 unmatched interior loss could latch after 3 passes and pause forever. New: suppressed while a strict direct-ID majority survives.Tests
New block in
tests/fold-anchor-doctrine.test.ts:Verified locally (targeted): typecheck clean; fold-reconcile + doctrine + positional trio 108/108; 15-file drift/epoch surface set 129/129. Full-suite verdict from CI.
Closes #5715