Skip to content

[PR] fix(kernel): post-restart compression no longer silently folds wrong messages — slots freed during ID reconciliation are | fix(kernel): 重启后压缩不再静默折叠错消息——编号对账时腾出的座位保留给原主(#5196) - #5360

Merged
ranxianglei merged 6 commits into
masterfrom
fix/5196-restamp-slot-retirement
Oct 31, 2026

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

这是啥问题?(#5196)

这是啥问题?(#5196)

重启后第一次压缩可能把别人的消息静默折叠进去:模型拿着重启前看到的编号发 compress,HTTP 200 成功、块建好了,但块盖住的不是摘要里点名的那条消息,而是顶了号的另一条。0.1.209 有完整可复现 fixture(issue 里)。

触发条件:会话发生过重启冷载、且冷载触发了「编号对账重排」(#4316 PR3/3 设计:把编号对齐成库行序,防 churn 再漂移)。

根本原因(代码级)

  1. kernel/src/refs.ts restampRefsToLedger:每个消息的目标位=行序+1。同一趟里 A 先 stamp 落座后,它腾出的旧号对后面的 B「可见」——B 直接把 A 重启前的编号顶过去,而 superseded[那个号] 还记着 A。
  2. src/stream.ts supersededRefRejection(live-first):byRef[ref] !== undefined 时直接返回该号——编号活着就永远不查 superseded 映射。于是所有指向 A 的旧引用被解析到 B 头上。
  3. 结果:A 的压缩请求折叠了 B 的内容,A 的摘要文案盖在 B 的正文上。

历史脉络:#4807(修 #4797 死锁)明确接受了这个残余窗口;kernel liveness prune 曾被回滚(#5199 / af1f92a),stream.ts 注释里的「prune 会消掉歧义」是回滚时漏改的陈话。#5049 当时的 no-code-change 结论因此失效。

怎么修的(slot-retirement,#5196 评审方案 D)

对账中,每个挪窝消息腾出的座位当场保留(retired):目标位落在已保留座位上的消息不再顶班,改领游标处的全新号——走既有 displaced/rehomed 机制,零新拒绝面、零新等待、不可能死锁。由此任何新发/新落的编号都不可能同时指向两个身份,live-first 在内核产生的数据上构造性安全。

判定依据

  • kernel targeted:seq-restamp(含 3 条新回归 + 改写后的 S13b)/ccr-restamp/dead-refs/dead-ref-split/issue3939 = 23/23
  • host targeted:seq-ledger ×2 / issue4316-seq-bounds / issue4797-superseded-deadlock = 13/13;e2e-trae-model-restamp = 1/1
  • kernel tsc + root tsc clean;unused-exports gate clean(6 waived)
  • issue 里的 fixture 形状被逐字钉成回归测试(含「二次冷载纯 no-op」持久性钉)

kernel-change: #5196 静默错折叠修复必须落在内核重排函数本身(restampRefsToLedger 的座位退役语义),无宿主侧替代实现;顺带刷新 stream.ts 一段被 #5199 回滚后遗留的失真注释。kernel 0.0.173→0.0.174(patch,语义向后兼容:仅新增「腾出座位保留」行为,消除双身份歧义)。

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: fix/5196-restamp-slot-retirement (2daa8cb)

Latest build — install this exact version

npm install -g billion-context@0.0.7568-pr.5360.ee4a96a

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
2daa8cb billion-context@0.0.7568-pr.5360.ee4a96a build
9cd0960 billion-context@0.0.7561-pr.5360.1a67624 build
8475295 billion-context@0.0.7557-pr.5360.9d3fd52 build
2f3cd35 billion-context@0.0.7554-pr.5360.8c38abb build
2c805ff billion-context@0.0.7551-pr.5360.7e83cd6 build
58c9043 billion-context@0.0.7548-pr.5360.cc059b6 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5360.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix(kernel): 重启后压缩不再静默折叠错消息——编号对账时腾出的座位保留给原主(#5196) [PR] fix(kernel): post-restart compression no longer silently folds wrong messages — slots freed during ID reconciliation are | fix(kernel): 重启后压缩不再静默折叠错消息——编号对账时腾出的座位保留给原主(#5196) Oct 31, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收到。这是 [PR] 镜像单,对应 PR #5360(分支 fix/5196-restamp-slot-retirement,head 58c9043c)。

查重: 不重复。 检索了 open/closed issue 与在途 PR:本 PR 是 #5196(重启冷载编号对账腾座顶号 → 压缩折叠错消息)的唯一在途实现;相关历史条目 #4807/#4797(死锁修复)、#5199(liveness prune 回滚)、#5049(旧 no-code-change 结论)均已被本 PR 正文显式覆盖/取代,无并列重复 PR。

初审状态:CI 上 kernel-guard 红,其余已完成的静态门禁绿(test 分片因 gate 依赖跳过中)。正在拉分支核对代码与 kernel-guard 失败原因,完整八问结论随后补上。

@ranxianglei

Copy link
Copy Markdown
Owner Author

kernel-guard 红因:PR body 漏了内核人审门禁要求的 kernel-change: 声明(动了 kernel/ 源码必声明)。已补进 body 末尾——#5196 的修复只能落在内核重排函数本身,无宿主侧替代实现;0.0.173→0.0.174 为 patch,语义向后兼容。其余 24 项 checks 本就已绿。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

标签: bug + scope/kernel + p2(缺陷类;压缩引擎编号账本域;触发条件苛刻、命中后静默折错消息但可 decompress 挽回,能用有绕行 → p2)。

门禁状态先说清:你已把 kernel-change: 声明补进 PR body(空提交 2c805ff 触发重跑),这本身就是批准动作——按授权卡点政策我原本要另开申请 issue,你直接补完即闭环,不再立项。CI 在新 head 上重跑中(kernel-guard 应转绿,test 车道随之解除阻塞)。

独立审核(不采信 PR 结论,逐项重验)

方向关(原 issue):GitHub #5196 的根因分析成立,我对 master 代码独立复核过完整链条:①restampRefsToLedger(kernel/src/refs.ts)同趟内 A 先 stamp 落座后,腾出的旧号对后面的 B「可见」,B 直接顶号,而 superseded[该号] 仍记 A——一号双身份;②宿主 supersededRefRejection(src/stream.ts)live-first(byRef[ref] !== undefined 即返回,不查 superseded),旧引用全部解析到 B。issue 的分层判断正确:根因在内核产生双身份数据,宿主 live-first 只是放大器(#4807 接受的合理取舍,本 PR 未动)。修复打在源头(疏)而非在宿主拦截/披露症状(堵)——方案 D 选对了。

回归清单(触碰路径的既有行为,逐项过):

  • displaced 结算阶段与 master 字节级一致——包括刻意不清 byRef[held] 的语义(held 可能已被本趟新主人 stamp,S13c 依赖这一点,删了就毁掉刚落座的主人);S13c 测试未改、仍绿。✓
  • 两阶段顺序是承重墙:leftovers 必须在 displaced 之后处理。一条消息可同时出现在两个列表(自己的目标位被退役 + 自己的座位被他人 stamp 顶掉),displaced 阶段先停车,parked 集合让 leftovers 阶段去重跳过——实现顺序正确,我手工推演了该场景确认不会双重处理。✓
  • leftovers 阶段 delete byRef[held] 的安全前提成立:进入 leftovers 而未进 displaced 的消息,其座位不是任何其他行的 seq+1,不可能被任何人 stamp 过,清自己座位安全。三个场景(issue 形状/腾退链/互换)逐一手工推演通过。✓
  • 游标安全:退役座位全部是原有在用号,< 初始游标(highestUsedIndex+1);allocateFreeRef 只发 ≥ 游标的新号 → 新号永不与任何在位号/退役号相撞;零新拒绝面、零等待,构造性无死锁。✓
  • parked 状态的持久性:refLedgerAligned 跨冷载持久(src/state-hydration.ts:41 水合、src/persist.ts:1779-80 为 true 时早退)→ 二次冷载纯 no-op,有专门测试钉住;即使 flag 丢失,对 parked 布局重跑一趟也是干净归座或以更新号再停,无振荡、无双身份。✓
  • 有意行为变更(已在 PR body 披露,我认可披露充分):①互换(S13b)不再是双方原子换位——先者落座、后者停车领新号,消灭双身份的必然代价;②编号空间出现稀疏空档(展示层标签,无功能成本)。
  • 修复前已产生的歧义数据:[PR] fix(#4797): root-cause fix for superseded live-key collision — restamp pruning + semantics-preserving guard translation | fix(#4797): superseded 活键碰撞根治——restamp 剪枝 + 守卫保义翻译 + 一次性标志修复 (kernel 0.0.166) #4807 live-first 取舍保留不变;src/stream.ts 仅注释刷新(替换被回滚 prune 方案留下的失真表述),零行为面。✓
  • 版本纪律:kernel/src 变更 + 0.0.173→0.0.174,bump 与源变更匹配。✓

本地定向验证(PR head 实跑,非转述):kernel 五个定向套件 seq-restamp/ccr-restamp/dead-refs/dead-ref-split/issue3939 = 63/63 绿;kernel tsc --noEmit clean。全量回归交给 CI。

测试有效性:真测试。三条新回归钉的是具体终态布局而非计数——issue 形状测试断言「B 不得领 m7」+「m7 永久无活主」+ superseded 映射 + rehomed 数;腾退链测试防过度退役(未被腾出的座位仍可复用,恰好一个停车);持久性测试钉「二次冷载纯 no-op」。S13b 改写后断言新车契约(停车号=游标处、m00002 永久空)。这些回归在 master 上会红(issue 形状那条就是 bug 本身),断言有牙。

顺手小修(独立 commit 2f3cd35ae,已推 PR 分支):review 发现 leftovers 元组的 from 字段只写不读(结算一律以 byRaw[rawId] 为准,它反映后续可能的顶占)——死数据,简化为 string[],零行为变化,改后 seq-restamp 7/7 + tsc clean。

diff 整洁度:7 个文件每一行都与目的相关(内核机制 + 测试 + 注释刷新 + CHANGELOG 条目 + 版本 bump + lockfile 连带),干净可合。

结构化八问

  1. 是什么问题:重启后第一次压缩可能把别人的消息静默折叠进去——模型拿重启前看到的编号发 compress,HTTP 200 成功、块建好了,但块盖住的不是摘要点名的那条,而是顶了号的另一条(0.1.209 真实事故,hash 验证过的 fixture 已归档)。
  2. 什么原因导致的:冷载编号对账时每条消息的目标座位=库行序+1;同一趟里 A 先落座后腾出的旧号对后面的 B「可见」,B 直接顶过去,而 superseded 映射还记着 A——一个编号同时指向两个身份;宿主解析器又是「活号优先」,于是所有指向 A 的旧引用被解析到 B。根因在内核对账函数,宿主 live-first 是放大器(其取舍本身合理,未动)。
  3. 影响范围与严重程度:触发需三条件叠加——会话重启冷载 + 冷载触发对账重排 + 某消息目标位恰好落在同趟更早腾出的号上(换号/级联场景)。命中则静默折错消息,用户看到块覆盖不对才察觉;decompress 可取回原文,无永久丢失。p2。
  4. 怎么修复的:slot retirement——对账中每个挪窝消息腾出的座位当场保留;目标位落在保留座位上的消息不顶班,改经既有 displaced/rehomed 机制领游标处全新号。从此任何新发/新落的编号不可能同时指向两个身份,live-first 对内核产生的数据结构性安全。代价仅编号空间稀疏空档。
  5. 是否有回归问题:逐项审过,未发现无意回归(见上面回归清单)。两处有意变更均已披露:互换非原子化(必然代价)、编号稀疏空档(展示层)。fail-fast/阈值/时序/输出格式均未漂移;修复前歧义数据的处理路径不变。
  6. 是否破坏缓存:不涉及。停车/稀疏是一次性结算,refLedgerAligned 之后抑制再次对账,编号此后稳定,前缀缓存稳定性与既有 restamp 行为同级,无重复漂移。
  7. 是否改了设计方向:无架构转向——仍在 After model switch / client-side history rewrite, all message identities (content hashes) are replaced: ACP numbers re-m | 切模型/客户端重写历史后,消息身份(内容哈希)全部更换:acp 编号重铸、折叠覆盖大规模重挂、死 ref、churn 区临时膨胀重折 #4316 第三层对账框架内,同一函数加「腾出座位退役」不变式;「一次稳定对账」不变式降级为「对齐者严格 seq+1 + 停车者持全新号」,函数文档已写明。
  8. 是否引入了新的配置:无——无新 env/config/CLI;kernel 0.0.173→0.0.174(patch)。

判定:代码审核通过,可合——等 CI 全绿后由你做人工评审合并(kernel 恒为人审门,我不代合)。PR 链接:#5360

ework-agent and others added 5 commits October 31, 2026 22:59
…o messages (#5196)

Cold-load ref reconciliation (maybeRestampToLedger, #4316 PR3/3) aligns refs
to ledger row order seq+1. When mover A stamped its seat in one pass, the
slot A vacated became immediately visible as free to a later mover B in the
same pass — B took over A's pre-restart number while superseded[thatRef]
still named A. The live-first resolver (src/stream.ts
supersededRefRejection) then re-targeted every stale citation of that
number at B: a compress() citing A's pre-restart ref folded B's contents
behind A's summary text, HTTP 200, block created (0.1.209 field fixture).

restampRefsToLedger now retires every seat a mover vacates into its own
seq+1; any mover whose target is a retired seat parks on a fresh number at/
after the cursor via the existing displaced/rehomed path instead. A retired
slot can never be targeted again within the pass (targets are unique per
row), and park numbers always start above every used index, so no freshly
stamped or parked ref can carry two identities — live-first becomes safe by
construction on kernel-produced data, with zero new rejection surface.
Pre-#5196 ambiguous data keeps the documented #4807 live-first trade.

- kernel 0.0.173 → 0.0.174
- tests: seq-restamp S13b rewritten to parking semantics (a swap's second
  side parks, superseded carries both old mappings), 3 new regressions:
  the exact issue fixture shape, a vacate chain proving untouched seats
  stay reusable, and cold-load persistence of the parked layout
- src/stream.ts: refresh the stale 'kernel prune' doc paragraph left over
  from the rolled-back liveness prune (#5199 / af1f92a)

Verified: kernel seq-restamp/ccr-restamp/dead-refs/dead-ref-split/
issue3939 23/23, host seq-ledger×2/issue4316/issue4797 13/13,
e2e-trae-model-restamp 1/1, kernel+root tsc clean, unused-exports gate
clean.
…5196)

Review nit on the slot-retirement diff: the leftovers entries stored the
mover's pre-pass ref but settlement always reads byRaw[rawId] (authoritative —
it reflects any later displacement), so the field was never read. Zero
behavior change; seq-restamp 7/7 + tsc clean.
The fixture runs restampRefsToLedger on the same drifted shape as the kernel collision suite. Under in-pass slot retirement f no longer takes e's vacated m6 — it parks on a fresh number (m8) — so T1's hardcoded m4..m6 span stopped reaching f (fold covered d,e only). Re-cite the advertised live range m4..m8 so the retired slots m6/m7 inside the span translate onto their current owners instead of refusing, pin the parking layout with an explicit precondition, and refresh the layout comments. Behavior under test unchanged: advertised live refs fold, old cites translate.
…refs (#5345 fallout)

Master has been red since #5345 (guided pack editor): every PR's test shards
fail both #1024 catalog checks. (a) en cfg.q_pack_ai_p1 lost its {need}
placeholder vs zh — restored, text aligned. (b) the base-selector option
labels are referenced through a pair array (t(pair[1])), which the literal
t("key") scanner cannot see, so all four cfg.q_pack_base_* keys report as
dead. collectRefs now also accepts dot-qualified identifiers quoted verbatim
in page/client/report sources (catalog keys are namespaced; plain quoted
words like "billion-context" lack a dot and are unaffected).

tests/web-i18n.test.ts 10/10 locally; unblocks #5196's matrix.
@ranxianglei
ranxianglei force-pushed the fix/5196-restamp-slot-retirement branch from 8475295 to 9cd0960 Compare October 31, 2026 15:00
@ranxianglei

Copy link
Copy Markdown
Owner Author

本轮红因与处置(与 #5196 本身无关):master 自 #5345(pack 编辑器)起 i18n 目录校验恒红——每个新 PR 的 3 片测试连带挂。本 PR 已 rebase 到最新 master(含 #5356/#5147/#5345)并把最小修复带上:①英文 cfg.q_pack_ai_p1 补回 {need} 占位符;②死 key 扫描器接受「键名数组」间接引用(t(pair[1]) 形式的下拉选项),否则 4 个合法文案被误判死亡。另:本分支上 ci-check 的两笔配套改动(leftovers 死字段清理、#4797 测试套对齐 slot-retirement 布局)随 rebase 一并保留。web-i18n 10/10 + kernel seq-restamp 7/7 + host seq 家族 12/12 本地绿。

@ranxianglei
ranxianglei merged commit 05624b1 into master Oct 31, 2026
24 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.970-master.05624b1 (rolling master tag).

npm install -g billion-context@0.0.970-master.05624b1  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Nov 19, 2026
Cherry-picked from #5233 (qiangu2020) kernel core, rebased onto current
master. The deadRefs tombstone infrastructure already landed via #5360;
this adds the missing ref floor:
- highestIssuedIndex: byRef orphans whose byRaw owner is gone still count
- assignRefs: cursor clamped to max(requested, historicalHighWater+1)
- ledger stamp accepted only above the floor and outside tombstones
- allocateFreeRef skips tombstoned indices
- both restamp paths seed nextIndex from the unified floor

Prevents lifetime ref reuse after pruning (the #5485 defect).

Cherry-picked-from: 3e137c4 (qiangu2020/2026-10-30_ref-safety)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant