Repository navigation
[PR] fix: escalate structural digest drift to alarm + envelope freeze, load the fuller side (#4323) - #4332
Conversation
…d the fuller side (#4323) Multi-writer shared sessions dir (#394) ping-pongs the JSON envelope and the SQLite index into permanent divergence; every drift-latched cold load then served the stale envelope unconditionally, putting active fold blocks back into context (~240K tokens, #4323 incident). - shadow.ts: count consecutive sampled drifts; at DRIFT_ESCALATE_STREAK (3) log ONE loud WARN naming both sides' active block counts and freeze THIS process's JSON envelope writes for the session; a clean sample re-admits. - persist.ts: all three save routes go index-only for frozen sessions (same path as writerOff); loadSync's drift latch serves the unified store when it strictly carries MORE active blocks than the envelope (tie or unreadable other side keeps the historical envelope preference). - runtime-status.ts: additive frozenEnvelopeSessions on the status wire. - tests/issue4323-drift-escalation.test.ts: escalation threshold both directions, re-admission, latched-load selection (index-richer / envelope-richer / tie / unlatched negative gate), wire field.
f567967 to
5bfcc0f
Compare
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.5145-pr.4332.aeec752This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr4332.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 本 PR 实现 #4323 中 owner 指定的两个代码修复(漂移升级+冻结、latched 冷加载取更全一侧),仓库内无针对同一缺陷的并行修复 PR。审核中发现的 CI 红项(LIVE_FILES 绊线、mysql-contract)是 master 侧既有缺陷,已由 #4333/#4334/#4326 跟踪,与本 PR 不同机制——未把任何 CI 修复合进本 PR。 标签: bug / scope/storage / p1(缓存泄露类存储层缺陷:折叠覆盖在冷加载丢失;沿用原 issue #4323 分类)。 审核结论(独立审核,非复读 issue 结论)① 方向(原 issue): #4323 的根因链(owner floor 4 钉死):多实例互写 → digest 漂移永不收敛 → drift latch 下无条件 serve 旧 envelope → 丢 N 个 active fold → ~240K 已折叠 token 回灌上下文。本 PR 与 owner 指定的两条修法一一对应,无偏离无越界:①连续 3 次采样漂移不收敛 → 一条响亮的 WARN(点名两侧 active block 数)+ 冻结本进程对该会话的 envelope 写入,直到干净样本再准入;②latched 冷加载从「无条件 envelope」改为「仅当 index 严格更多 active blocks 时取 index」。运营层(单实例纪律)与 #4177 第二实例检测被正确地留在 PR 之外。方向成立。 ② 回归审查(既有行为清点): 逐项读了改动前代码并判定:
③ 代码: 命名达意(driftStreaks/frozenEnvelopeIds/envelopeFrozen/activeBlockCount/countActiveBlocks);状态为 per-process 内存(与 ingestCounts 同纪律)、刻意不持久化且注释说明了理由; ④ 测试有效性: 6 个测试全是真断言、有牙:A 逐步钉 verifyDrift=1/2/3 配 frozen=false/false/true(阈值每步正+负门),修复后再准入,并钉「恰好 1 条升级告警 + 恰好 3 条逐样本 error 行」;B1–B4 覆盖选择矩阵全四象限(index 多→index / envelope 多→envelope / 平手→envelope / 无 latch→unified-first 负向门)。B1 在旧代码(永远 serve envelope)上必失败——真回归钉。小缺口:对侧不可读(activeBlockCount→null)分支落到历史行为,由 B2/B3 间接钉住,可接受。 ⑤ diff 整洁度: 4 文件 +490/-3,每行都与目的相关,无空白噪音、无锁文件/生成物。已 rebase 到当前 master(aa198fe):唯一冲突文件 runtime-status.ts 共 4 处,全是「master #4175 加的 durability 三字段 vs 本 PR 加的 frozenEnvelopeSessions 字段」落在同一位置,按双方并集解(纯增量字段)。rebase 后 typecheck clean,新套件 6/6 + 邻近 storage 套件 21/21 本地绿(全量回归归 CI)。新 head: 5bfcc0f(PR 描述已附 rebase note)。 ⑤b 代码整洁硬标准: 全项过——命名/死代码/重复/错误处理上下文/魔法数字(DRIFT_ESCALATE_STREAK 具名常量)/注释讲为什么/风格一致/零新增依赖。 行为变更披露(old → new)
prefix-cache/retry/watermark/dedup-key 语义均未触碰;无新配置面(仅内部常量)。 八问
CI 状态与合并判定
|
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.751-master.3221859 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #4323
Problem
Multiple bili instances sharing one sessions dir (#394 anti-pattern) ping-pong the JSON envelope and the SQLite index into permanent divergence (
[diverged: state]never converging). Every restart/eviction cold load hit the drift latch and served the STALE envelope unconditionally — measured incident: envelope 12 active blocks vs index 16, ~240K already-folded tokens silently returned to context (#4323; both recurrences same mechanism per owner analysis).Root cause confirmed by owner on #4323; this PR implements the two owner-specified code fixes. The ops layer (single-instance discipline, zombie-lane cleanup) stays with the operator; #4177 second-instance detection already lands post-0.1.204.
Changes
① Drift escalation + envelope-write freeze (src/storage/shadow.ts)
DRIFT_ESCALATE_STREAK = 3: consecutive sampled digest drifts are counted per session. At 3 in a row (the structural-divergence signature — one-off external churn heals on watermark clear) ONE loud WARN names both sides' active block counts and points at the multi-writer anti-pattern (无单实例守卫:多个 bili 代理进程并存共写 sessions 目录,伴随孤儿 ref、有注册表无 transcript、计数自相矛盾的污染簇 #394) and second-instance detection ([PR] storage: durability trio — open-time quick_check + online VACUUM INTO backup + second-process lockfile (#4175) #4177).② Drift-latched cold load takes the fuller fold coverage (src/persist.ts)
Observability (src/storage/runtime-status.ts)
frozenEnvelopeSessionscount on the /__bili/status + overview storage wire.Behavior changes (old → new)
No prefix-cache / retry / watermark / dedup-key semantics touched. No config surface added (internal constant only). No kernel/ or pi-subagents/ changes.
Verification
Review note (rebase, 2026-10-23)
Rebased onto current master (aa198fe): head f567967 → 5bfcc0f. Single conflicted file
src/storage/runtime-status.ts(4 hunks) — master's #4175 durability-trio fields (integrity/secondProcess/backup) and this PR's additivefrozenEnvelopeSessionslanded in the same four spots; resolved as the union of both (purely additive). Post-rebase: typecheck clean; targeted suites green locally (issue4323 6/6, storage neighbors 21/21). The redtestshards +mysql-contracton the rebased head are pre-existing master-side defects since #4177 merged — tracked in #4333 (fix: #4334 in flight) and #4326-B respectively, NOT caused by this PR; no CI fixes were bundled here.