Skip to content

[PR] test: golden drive refuses to run on a stale kernel dist (#3325 lesson) - #3330

Merged
ranxianglei merged 1 commit into
masterfrom
test/golden-kernel-dist-freshness-guard
Oct 24, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
test/golden-kernel-dist-freshness-guard

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Why

PR #3325 (closed) regenerated the storage-render goldens from a stale local workspace and would have made every fresh-build CI leg fail. Root cause of that mistake: the golden drive renders persisted-state digests from node_modules/acp-kernel/dist (a build product) while the bili side runs live TS via tsx. A stale kernel dist silently stamps removed fields into persisted state — post-#3266 it still produced metadata.effectiveConfig.imageCompression and state.stats.image{BytesSaved,TokensSaved,Shrunk} — so digests from a stale workspace can never be reproduced by a fresh build. I verified offline: fresh npm install --ignore-scripts + npm run build passes golden-storage-render 5/5 against master's committed goldens (node 22 and node 25), while the stale workspace diverges deterministically.

What

  • kernelDistStale(root) in tests/golden-storage-drive.ts: dist-entry mtime vs newest kernel/src (+ tsup.config.ts) source mtime.
  • driveWire() fails fast with an actionable error (npm run build) before touching session state — call-time, not import-time, so the guard's own unit test can import the module from a stale workspace.
  • tests/golden-kernel-dist-guard.test.ts: stale / fresh / missing-dist passthrough on synthetic trees.

Red-green proof

  • Stale dist (my workspace pre-build): Error: golden drive: node_modules/acp-kernel/dist is older than kernel/src (…/kernel/src/compress.ts) — … Rebuild first: npm run build
  • After npm run build: golden-storage-render 5/5 ✅ + guard tests 3/3 ✅ + golden-storage-write / storage-write-amplification / persist-guard-ownership 7/7 ✅

This kills the whole "master golden red" false-positive class for anyone (human or agent) working from a long-lived checkout.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: test/golden-kernel-dist-freshness-guard (bff4d6d)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3952-pr.3330.f4a3799

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3330.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — open/closed issues 与 PR 中均无针对「golden drive 在陈旧 kernel dist 下渲染」的既有条目(检索词: stale kernel dist / golden / #3325)。#3325 本身已关闭,本 PR 是其教训的机制化修复,方向正确。

初步印象: diff 很小(2 个测试文件,+116/-0),纯测试基础设施,不触碰 wire/config/kernel。正在做完整审核(读码 + 回归面清点 + CI 状态核对),稍后按八问回复结论。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核结论:可合(等你手动 merge)— 完整八问如下

1. 是什么问题:#3325 暴露的机制漏洞——golden 驱动从 node_modules/acp-kernel/dist(构建产物)渲染持久化状态摘要,而 bili 侧跑的是 tsx 活 TS;长期 checkout 里 dist 陈旧时会把已删除字段(#3266 后的 imageCompression/state.stats.image*)戳进摘要,导致陈旧工作区产出的 golden 永远无法被全新构建复现。本 PR 给 golden 驱动加了 fail-fast 防护。

2. 什么原因导致的:构建产物与活源码的双轨失步——dist 的 mtime 落后于 kernel/src 任何源文件时,渲染结果就不可复现;此前没有任何机制检测到这种失步,错误以「master golden 红」的假阳性形式在 CI 上爆发。

3. 影响范围与严重程度:影响所有在长期 checkout 上跑 golden 测试或重新生成 golden 的人/agent(人类和 agent 都中招);严重度=流程性风险(假红 CI、被污染的 golden 提交),非运行时用户故障——#3325 就是差点发生的实例。

4. 怎么修复的:mtime 比较(kernelDistStale:dist 入口 vs 最新 kernel/src+tsup.config.ts 源文件),在 driveWire() 调用时(非 import 时)fail-fast,报错直接给出最新源文件路径 + npm run build 指引。调用时判定是关键设计:守卫自身的单测才能从陈旧工作区 import 该模块。覆盖完整性已核:全部渲染/再生成路径都过 driveWire(含 #3325 肇事脚本 scripts/update-storage-render-goldens.ts)。

5. 是否有回归问题:无。纯测试基础设施,+116/-0,不触碰 wire/config/kernel/persistence。本地实测双向验证:

  • 绿路径:fresh build 后 guard 3/3 + golden-storage-render 5/5 ✅(CI 亦同);
  • 红路径:人工把 dist mtime 拨回 7 天后,import 不抛(call-time 设计成立)、driveWire 在碰任何 session state 之前抛出可操作错误、scratch 目录未被创建 ✅;
  • CI 不会误报:run-ci-tests.mjs 在套件前跑 pretest build(kernel+pi-subagents),CI 里 dist 恒新于 src ✅。
  • 已知边界(不拦):若只删除 kernel/src 文件而不重建,mtime 不变、守卫感知不到——罕见流程,下次正常构建自然修复。

6. 是否破坏缓存:不涉及(未触碰缓存/重试/增量游标/去重键)。

7. 是否改了设计方向:不改变架构;是在既有 gate 体系(kernel-guard、unused-exports 等)旁加一道测试基础设施纪律,方向与仓库惯例一致。

8. 是否引入了新的配置:不涉及(无 env/config/默认值)。

CI 状态说明(唯一红腿与本 PR 无关)

head 7e2c0494b 上 38 个 check 仅 1 红:test (ubuntu-latest, 22, 3) —— 失败点是 tests/lineage-reconcile-boot.test.ts:468 的 AC12(多跳孙会话资格门断言翻红),与本 PR 零关联(本 PR 只改两个 golden 测试文件,node --test 跨文件进程隔离;同一 head 本地单跑该文件 4/4 通过)。疑似分片 c=4 负载下的时序敏感 flake,已按「发现即立项」开 #3339(带来源标记)跟踪。不构成合并阻塞。

八问结论:可合,merge 交给你(系统规则禁止 agent merge)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

还红

The storage-render golden drive renders persisted-state digests from
node_modules/acp-kernel/dist (a build product) while bili runs live via
tsx. A stale kernel dist silently stamps removed fields (post-#3266 it
still produced metadata.effectiveConfig.imageCompression and
state.stats.image* counters), so a golden regenerated from that workspace
bakes digests no fresh build can reproduce — CI then fails every leg that
carries the golden file. That is exactly how PR #3325 went wrong: 'master
red' was verified and regenerated from a stale local workspace, while a
clean fresh-install replica passed 5/5 against the committed goldens.

- kernelDistStale(root): compares node_modules/acp-kernel/dist/index.js
  mtime against the newest kernel/src (and tsup.config.ts) source; pure,
  synthetic-tree testable.
- driveWire() now fails fast with the rebuild instruction
  ('npm run build') before touching any session state. Call-time, not
  import-time, so the guard's own unit tests can import the module from a
  stale workspace.
- tests/golden-kernel-dist-guard.test.ts pins both directions + the
  missing-dist passthrough.

Verified both ways locally: stale dist -> drive refuses with the message;
after npm run build -> golden-storage-render 5/5 against master's
committed goldens.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

「还红」已定位并处理:红不是本 PR 的。 根因经插桩实证,是既有测试 tests/lineage-reconcile-boot.test.ts AC12(#2807 引入)的时序竞态 flake,与本 PR diff(只动 golden drive 两个测试文件)无关。

机制:AC12 靠「直接改会话内存」模拟重启、再读盘验证,但落盘是 500ms debounce。CI 分片负载下,定时器若在改动前落地,改动前的旧状态就被写进磁盘、最终 boot 读到旧形状 → 孙会话被判定已完成采纳 → derivedBlocksAdopted 断言翻红。探针证据:失败 run 里孙会话的磁盘记录带着采纳门标记而 blocks 已被剥离——正是「strip 前后各抓一次快照」的竞态签名。本地单跑 4/4 全绿、CI 才红,也与此一致(负载拉长事件循环跨度)。

修复已在 master:#3345(每次模拟重启改动后加同步强制落盘 flushSync 屏障,把改后形状钉死在盘上,+12/-1 纯测试改动)。我在本 issue 独立排查时收敛到同一机制族与同一修法方向,互相印证。

我刚做的:

另发现一个实现侧问题(已单独立项,与本 flake 无关):#3351 —— 多跳 fork 中,若子会话首触时其注册父会话还没有折叠块、且未命中字节级强匹配,弱链接路径会无条件关闭采纳门并落盘——之后父会话再折叠出块,子会话也永远继承不到(#2807 原症状在多跳拓扑下的复发形态)。建议修法 = 只在采纳实际生效时关门(与 boot sweep 纪律对齐)。属行为变更,等你拍板;不在本 PR 范围内。

结构化八问:

  1. 是什么问题:随机 CI 红,来自既有测试的时序竞态,不是被审代码的缺陷。
  2. 什么原因导致的:测试模拟重启裸改内存不强制落盘,debounce 定时器相对改动的落点决定盘上形状;负载下改动前快照先落盘,最终 boot 读到旧状态。
  3. 影响范围与严重程度:仅该测试文件(AC9/AC10/AC11 共用构造,同族隐患已由 fix(tests): pin AC12 restart shape with a synchronous persist barrier #3345 一并钉死);生产零影响;严重度 = 随机 CI 红误导排障。
  4. 怎么修复的:每次模拟重启改动后同步强制落盘屏障(fix(tests): pin AC12 restart shape with a synchronous persist barrier #3345,已合入);本 PR rebase 到修复后的 master。
  5. 是否有回归问题:无——纯测试改动;实现侧 gate 语义逐项审过,无本次引入的回归(Forked child permanently loses block inheritance when its registered parent had no blocks at first contact (weak-link gate closes unconditionally) | 注册父会话在子会话首触时还没有折叠块,子会话将永久失去块继承(弱链接门无条件关闭) #3351 是既有的设计层问题,单列待决)。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:没有。
  8. 是否引入了新的配置:没有。

判定:可合(按规矩我不代合并)。等新 head CI 绿即可合入;#3351 等你拍板。

@rahul05ranjan rahul05ranjan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice guard — the #3325 postmortem reasoning checks out, and I verified the mechanics before commenting.

What I confirmed:

  • Premise is real: kernel/package.json exports . → ./dist/index.js, so the golden drive renders from the build product while the bili side runs live TS via tsx — a stale dist genuinely diverges persisted state with no other signal.
  • Guard mechanics: I extracted kernelDistStale (golden-storage-drive.ts:36) type-stripped into a standalone Node script and ran 10 adversarial cases: dist-newer/dist-older/missing-dist/missing-kernel-src, exact-equal mtimes (strict > so coarse filesystems don't false-positive), tsup.config.ts newer than all of src, a newer .md in src correctly ignored, .mjs correctly matched, nested subdirs walked. 10/10 pass. Couldn't run the repo suite itself on my box (no node_modules, installs not available there) — disclosing the method; your test file pins the same cases plus the error passthrough.
  • Placement is right: driveWire is the single choke point — golden-storage-render.test.ts and storage-engine-proofs.test.ts, plus scripts/update-storage-render-goldens.ts, all go through it, and __goldenGuard() (line 466) fires before SCRATCH_ROOT/session state is touched. Call-time arming instead of import-time is the correct call — the guard's own unit test imports this module from a possibly-stale workspace, and only actual drives must refuse.
  • CI can't false-positive: pretest rebuilds the kernel dist before npm test, so after checkout→build dist is always newer than src; the guard only bites on manual tsx runs from a long-lived workspace — exactly the #3325 failure mode.

One nit: the makeTree(distAtSrc) parameter name (golden-kernel-dist-guard.test.ts:11) reads as "dist at src" but what it means is "dist newer than src" (fresh). Cost me a double-take reading the tests.

Goldens that refuse to lie beat CI archaeology — good kill of this whole false-positive class.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.805-master.4e41d6f (rolling master tag).

npm install -g billion-context@0.0.805-master.4e41d6f  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants