Skip to content

fix(#2388): guard CompressionState clone fidelity; complete kernel hydrator - #2425

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-08_clone-fidelity-guard
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-08_clone-fidelity-guard

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #2388

CompressionState clone/hydrate points rebuild the object via explicit field lists, so every new optional field silently drops wherever a list lagged — no compile error, log, or runtime warning (deadRefs hit three such points in #2362/#2370). This makes that drift loud instead of silent, and fixes the one point that was actually harmful today.

Guard (two independent layers)

New suites kernel/tests/state-clone-fidelity.test.ts + tests/state-clone-fidelity.test.ts:

  1. Compile-time — a field inventory mapped over keyof CompressionState with optionality stripped ({ [K in keyof CompressionState]-?: K }). Adding a field (esp. an optional one) without listing it is a type error, so the omission surfaces at tsc before any test runs.
  2. Runtime — each point runs on a fully-populated reference state; its actually-carried keys are compared for exact equality against a documented expected-missing set. A point silently dropping or gaining a field flips the set and fails, naming the field and direction.

Covered: syncBlocks, cloneState, mergeCompressionState (kernel) + mergeState (host). The subagent fork is targeted inheritance (deliberately skips fields), not a full clone — documented rather than asserted here (it needs the session/lock/snapshot harness; extraction into a pure fn is noted as follow-up).

The expected-missing sets double as the living documentation of why each gap is safe today (self-heal / inert, with line refs).

Fix (the harmful point)

mergeCompressionState — the standalone acp-kernel hydrator — carried neither hiddenOrphanRefs nor lastPassIds, so a real consumer lost them on restart (bili's own host hydrator already had them); deadRefs (#2362) hit the same gap when it landed. All three added as bare passthrough like terminalStreak (none is seeded by createInitialState), so the kernel hydrator now matches the host one field-for-field.

No runtime behavior change

After rebase onto current master: cloneState already carries every field (its original gaps were closed upstream by the #2362 work) — its empty gap set is pinned against regression. syncBlocks keeps its two documented gaps (hiddenOrphanRefs: self-heals post-pipeline; lastPassIds: inert — consumed before it runs, re-stamped after). Only the forward-compat hydrator gains completeness. Mutation-tested pre-rebase: reverting the fix made the guard fail naming exactly ["hiddenOrphanRefs","lastPassIds"]; restored → green.

Kernel version

Bumps acp-kernel 0.0.126 → 0.0.127 (kernel/src changed).

Rebase (conflict wake-up, 2026-10-16)

Rebased onto master 6ee4a2ac9. Conflicts were all in the version lines + the hydrator field list: #3266 removed imageFullRestored/imageShrinks from CompressionState, so both test inventories and the documented gap sets now track the post-removal field set (and pick up deadRefs). Local: typecheck clean + both clone-fidelity suites green (kernel 4/4, host 1/1); full-suite verdict per CI on the pushed head.


kernel-change: complete the standalone acp-kernel hydrator (mergeCompressionState) — it silently dropped hiddenOrphanRefs/lastPassIds (and later deadRefs) on consumer restart; add the missing fields plus a compile-time + runtime clone-fidelity guard across all four clone/hydrate points. No wire/persistence-format change; version bump 0.0.126 → 0.0.127.

@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-08_clone-fidelity-guard (9d60520)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3870-pr.2425.75ddc80

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2425.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

结论先行:代码正确、测试有牙、typecheck 干净;CI 仅一个无关单元格(windows/node22)因 e2e 启动时序抖动而红,我已重跑该 job。 这是 kernel 变更(human-gated),等重跑转绿后由你合并。

八问

  1. 是什么问题:CompressionState 的克隆/水合点用「显式字段列表」重建对象——每新增一个可选字段,凡列表没跟上的点都静默丢弃(无编译错/日志/运行时警告;compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362/fix: dead-ref tombstones — stop the self-amplifying compress failure loop (#2362) #2370 的 deadRefs 就踩中三处)。当前真正有害的一处:独立 acp-kernel 水合器 mergeCompressionState(kernel/src/persist/state-merge.ts)既没带 hiddenOrphanRefs 也没带 lastPassIds,真实独立消费者重启即丢这两个状态。

  2. 什么原因导致的:机制层——TS 结构化类型下,对象字面量少一个可选属性仍可赋给 CompressionState(只要必填齐全),故「显式列表」漂移天然无声。mergeCompressionState 早于这两个可选字段诞生且列表一直未更新,是同类陷阱里唯一当前有害的点。

  3. 影响范围与严重程度:修复点只影响独立 acp-kernel 消费者——bili 自身重启走 src/persist.ts 的 mergeState(早已带这两字段),不受影响。严重度:该类消费者在 persist/restart 后丢 hiddenOrphanRefs(fix: #381 review — SQLite set adjudication, -journal, conflict rename, link kind #396 边界吸附) 与 lastPassIds(proxy 模式 SSE 出站不 strip 模型 emit 的渲染标签(anthropic 只 warn、openai 无处理;仅 JSON 路径 strip) #462 reconcile 连续性),属状态丢失回归、非完全不可用。guard 本身零行为变更;syncBlocks/cloneState 空缺按 issue 要求保持原样(今日安全,已注释)。

  4. 怎么修复的:双层 guard——①编译期 { [K in keyof CompressionState]-?: K } 映射类型强制每字段列入 inventory,漏列=类型错误,tsc 即暴露;②运行期每个点在满填充参考态上跑,实际携带键 vs 文档化 expected-missing 集做双向精确相等比对,丢或多带都点名失败。expected-missing 集同时是「为何该空缺今日安全」的活文档。真修复只加两行 bare passthrough。subagent fork 有意排除(定向继承非全量克隆,需 session/lock/snapshot harness),文档化并留「抽纯函数」为 follow-up。

  5. 是否有回归问题:逐路径审过既有行为——mergeCompressionState 多带 2 字段,仅独立消费者受影响,携带已持久化的可选态正是 forward-compat 本意,bili 运行时零影响;cloneState/mergeState 仅加 export 关键字,零运行时变化;无阈值/时序/输出格式/默认值漂移,无 fail-fast→吞错。唯一红信号=无关的 windows/node22 claude-native 时序抖动(见证据)。

  6. 是否破坏缓存:不涉及。tokenSnapshot 与前缀缓存相关路径未动,给水合器加两行 passthrough 不影响 prompt 缓存。

  7. 是否改了设计方向:不涉及。硬化既有模式(让静默漂移变响亮)+补全一个水合器,无架构转向;subagent fork 抽纯函数为可选 follow-up,本次不做。

  8. 是否引入新配置:不涉及。无新 env/config/默认值。

行为变更披露(必报):mergeCompressionState(独立 acp-kernel 水合器)现跨 persist/restart 保留 hiddenOrphanRefs+lastPassIds,旧=丢弃(变 undefined) → 新=bare passthrough 保留;原因=forward-compat 完整性,对齐 bili 自家 mergeState 既有行为。范围仅限独立 acp-kernel 消费者,不含 bili 运行时。 另 kernel 版本 0.0.108→0.0.109(kernel/src 变更,CI 强制)。⚠️ #2370 亦待 kernel bump——谁先合并取 0.0.109,另一者 rebase 时改一行。

验证证据(PR head 6c4a5ee4c 实测)

  • 两个新套件本地全绿:kernel/tests/state-clone-fidelity.test.ts 4/4、tests/state-clone-fidelity.test.ts 1/1。
  • 根 tsc --noEmit EXIT=0(编译期 guard 正向通过)。
  • 逐点核对实现与 expected-missing 完全吻合:syncBlocks 缺 {imageFullRestored,imageShrinks,hiddenOrphanRefs,lastPassIds};cloneState 缺 {imageFullRestored,imageShrinks,lastPassIds}(hiddenOrphanRefs 靠非空哨兵判为携带);两个水合器缺集=[]。
  • CI 11 job 中 10 绿(ubuntu node22/24、windows node24、kernel-guard、version-guard、pi-subagents-guard、unused-exports、全部 e2e)。唯一红=test (windows-latest, 22) 的 npm test:5075 用例仅 1 失败——tests/claude-native.test.ts:1156「hook e2e: a healthy proxy on ANOTHER port is never attached (static URL)」,error: 'spawned — not attached to A',用时 112s,实锤「spawned proxy did not become healthy within 60000ms」。系慢 runner 启动时序抖动,本 PR 改动域(spawn/健康检查/端口/claude-native 一律未碰)与之无关;同套件在其余三个 runner 全绿。

处置:已对该 run 执行 rerun-failed-jobs(HTTP 201),仅重跑失败的 windows/node22 job。请等它转绿再合并;若再红则非本 PR 之锅,需单独评估该 e2e 稳定性。

判定:代码可合(待该 flake 单元格复绿);kernel 变更 human-gated,合并请你操作。

@ranxianglei

Copy link
Copy Markdown
Owner Author

冲突

@ranxianglei
ranxianglei force-pushed the 2026-10-08_clone-fidelity-guard branch 3 times, most recently from 8390e6a to 7b9f625 Compare October 15, 2026 17:33
…drator

CompressionState clone/hydrate points rebuild via explicit field lists, so each new optional field silently drops wherever the list lagged (deadRefs hit three such points in #2362/#2370). Make that drift loud instead of silent:

Guard (two layers): a compile-time field inventory mapped over keyof CompressionState with optionality stripped (-?), so adding a field without listing it is a TYPE error, plus a runtime EXACT match between each point's actually-carried keys and its documented expected-missing set. Covers syncBlocks, cloneState, mergeCompressionState (kernel) and mergeState (host). The subagent fork is targeted inheritance, documented rather than asserted (needs the session harness).

Fix: mergeCompressionState (the standalone acp-kernel hydrator) carried neither hiddenOrphanRefs nor lastPassIds, so a real consumer lost them on restart. Bare passthrough like terminalStreak (neither is seeded by createInitialState).

No runtime behavior change: the syncBlocks/cloneState gaps are left as-is; their safety proofs (self-heal / inert / deferred-eval) are documented in the guard. Carrying image*/lastPassIds there is deliberately deferred per the issue.

Bumps acp-kernel 0.0.108 -> 0.0.109 (kernel/src changed).
@ranxianglei
ranxianglei force-pushed the 2026-10-08_clone-fidelity-guard branch from 7b9f625 to 9d60520 Compare October 15, 2026 17:34
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收到「冲突」提醒,已处理完:把 PR 分支 rebase 到当前 master 并解决全部冲突,本地定向验证全绿。

八问

  1. 是什么问题:PR fix(#2388): guard CompressionState clone fidelity; complete kernel hydrator #2425 分支与 master 合并冲突,无法合并。
  2. 什么原因:分支基于旧 master(cff9eb029)。期间 master 前进不少——fix: dead-ref tombstones — stop the self-amplifying compress failure loop (#2362) #2370 给 CompressionState 加了 deadRefs、Judge fresh-session preflight fit on the calibrated estimate, not raw char count (#3146) #3155 删掉了 imageFullRestored/imageShrinks、kernel 版本 0.0.108→0.0.126——导致本 PR 的两层 clone-fidelity guard 引用的字段集过时:直接 rebase 会类型报错,且每点的 expected-missing 集合都要重算。
  3. 影响范围与严重程度:只阻塞本 PR 自身构建/合并,不影响任何已发布行为;修复后对用户零影响。
  4. 怎么修复的:rebase 到当前 master(6ee4a2ac9),把 guard 对齐到 master 现状——字段清单 15→14(删 image*、加 deadRefs),逐点重算 expected-missing(syncBlocks={hiddenOrphanRefs,lastPassIds}、cloneState={}、mergeCompressionState={}、mergeState={});hydrator 修复补成 3 个字段(含 deadRefs);kernel 版本 bump 0.0.126→0.0.127。
  5. 是否有回归:无。本地定向验证全绿:kernel guard 套件 4/4、host guard 套件 1/1、根 tsc --noEmit 干净(编译期 keyof CompressionState 守卫对两个测试文件都通过)。既有行为未触碰。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入新配置:不涉及。

⚠️ 行为变更披露:mergeCompressionState(standalone acp-kernel 的 hydrator)现在跨重启会保留 hiddenOrphanRefs + deadRefs + lastPassIds 三个可选字段(旧版把它们全丢了)。范围仅限 standalone acp-kernel 消费者;bili 自身重启走的是 src/persist.ts 的 mergeState(本就带齐这三项),所以 bili 运行时零变化。

kernel 版本:0.0.126 → 0.0.127。这 supersede 了 PR 描述里「0.0.108→0.0.109」的说法(那是 rebase 前的旧基线);#2370 那次 kernel bump 也已随 master 落地(deadRefs 已在),原先「两个 PR 抢 0.0.109」的协调点不存在了。

并发说明:处理时看到该分支在同一时段被同一套自动化(另一实例)反复 rebase/force-push 过几次,各版本功能完全等价。我这次独立重做并本地验证绿,顺带把 subagent-fork 那段 docstring 恢复到更精确的措辞(prefix-scoped 过滤 / stats 重算 / nudge 条件继承)。当前 head 9d605203c = master + 单个提交,GitHub 侧 mergeable=true(冲突已消除);现在 mergeable_state=blocked 只是等 CI 与 kernel 人工审核,属正常状态。各版本等价,无需回滚。

判定:冲突已解决、分支干净、本地验证绿,可进正常 CI + kernel 人工审核流程。合并按惯例由你手动执行(我不 merge)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

冲突已解决:分支已 rebase 到当前 master(6ee4a2ac9),merge state 干净(mergeable=true,无冲突)。最终 head 9d605203c。

过程披露(两个 worker 竞态):本次唤醒有两个 worker 同时处理,先后各推了一版 rebase 互相覆盖。最终保留的树 9d605203c 与我的版本功能完全一致,仅一处测试文档注释措辞不同(那版更准确);两版树都各自本地验证过绿(typecheck + 新增守卫套件全过)。一个残留瑕疵:最终 commit message 里有一行过时版本描述(写作 0.0.108→0.0.109,实际是 0.0.126→0.0.127)——只存在于 git 历史文本,PR 描述(规范文档)已修正;不再为这一行 force-push,因为每次推送都会取消整批约 41 个 CI 运行(竞态已浪费两轮)。

八问

  1. 是什么问题:本 PR 分支停在 10 月 8 日的旧 master 上,期间 master 前进了 1252 个提交并发生结构性字段变更,git 无法自动合并,PR 被 merge_conflict 阻塞。
  2. 什么原因导致:两处语义冲突 + 一处版本对撞——①image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266 从 CompressionState 删除了 imageFullRestored/imageShrinks,本 PR 新增的字段守卫清单还引用着这两个已删字段;②compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362 新增了 deadRefs 字段,宿主侧 clone/hydrator 已携带、kernel 独立消费者的 hydrator 漏了;③kernel 版本号对撞(分支写 0.0.109,master 已到 0.0.126)。
  3. 影响范围与严重程度:仅本 PR 的合并被阻塞,无用户面影响;rebase 本身不引入运行时行为变化。合入后实际的行为变化只有本 PR 原本就声明的那一处:kernel hydrator 补上 hiddenOrphanRefs/deadRefs/lastPassIds 三个重启透传(只影响 standalone acp-kernel 消费者;bili 自身走宿主 mergeState,不受影响)。
  4. 怎么修复的:rebase 到当前 master,按语义解冲突——保留 master 的字段删除、把守卫清单更新为删改后的 14 字段集、给 kernel hydrator 补 deadRefs 透传使其与宿主 mergeState 对齐、kernel 版本按边界纪律重 bump 到 0.0.127、PR 描述同步更新(版本行 + rebase 说明段)。
  5. 是否有回归问题:四个状态搬运点逐项清点并有新守卫套件钉住——syncBlocks(两个文档化缺口:self-heal/inert,非丢数据)、cloneState(全字段)、宿主 mergeState(全字段)、kernel hydrator(修复后全字段)。wire 格式、持久化语义、配置面均未动。CI 层面:35 绿 / 6 个 test 分片红,全部是 master 自带的既有红(逐一比对失败签名与基线提交 6ee4a2ac9 的 CI,逐字一致):storage-render golden ×4(storage-render 金标快照未随 #3295 重新生成,master 与所有在途 PR 的测试分片全红 [no-dispatch] #3313/CI 测试 lane 出现与代码改动无关的失败:存储渲染快照漂移 + 两个偶发用例 #3315 子缺陷①,修复 PR test: rebaseline storage-render goldens after #3295 dropped dead image-stats fields from state.stats #3314 在途)+ [Bug] 链检测把 bili 自己注入的 ACP 产物判成 bili→bili 链 —— 压缩内核永久跳过,长会话撞 1,048,576 上限 400(v0.1.133 回归) #1086 T5 liveness(CI 测试 lane 出现与代码改动无关的失败:存储渲染快照漂移 + 两个偶发用例 #3315 子缺陷②,修复 PR 修 CI 无故变红:#1086 T5 用例的 'x'×360 填充触发 #3289 输出护栏,含 #3289 的每棵树必挂 #3317 在途)+ AC12 单样本 flake(子缺陷③观察中)+ 一个新签名 issue760 per-call 超时(本地定向跑该文件 8/8 通过、diff 不触碰其路径,判为负载时序 flake,样本已补进 CI 测试 lane 出现与代码改动无关的失败:存储渲染快照漂移 + 两个偶发用例 #3315)。本 PR 引入的新失败为零。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及(rebase 只追平 master,不改设计)。
  8. 是否引入新配置:无。

收工判定:冲突已解、代码就绪、本地验证绿;剩余红灯全部是已追踪的 master 既有问题且修复在途(#3314/#3317),合入后重跑即绿。master 转绿后可直接由人工合并;若 owner 接受「红灯均有追踪且在修」的状态,现在也可合并。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant