Skip to content

test: rebaseline storage-render goldens after remove-absorb (#3309) - #3325

Closed
ranxianglei wants to merge 1 commit into
masterfrom
chore/golden-regen-absorb-removal
Closed

ranxianglei wants to merge 1 commit into
masterfrom
chore/golden-regen-absorb-removal

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Master is red on its own test matrix: golden-storage-render fails 4/5 on clean origin/master (a046d4e). Master push CI skips the test lanes (known hole), so it surfaced as nine red legs on #3318 and will block every PR that runs the matrix.

Root cause (field-level proof)

Dumped the drive's final view at the last green golden (4bf28c5) and on current master via driveWire(…, { dumpStateTo }):

  • only divergence: metadata.effectiveAbsorb — present in the old view, absent now. Removed by the remove-absorb merge (feat: remove the absorb distillation feature (#3293) #3309).
  • everything else byte-identical: blocks / messageRefs / nudge / tokenSnapshot / nextBlockId / blockContents digests match.

#3298 regenerated the goldens on a pre-#3309 tree; the batch merge order left them one field behind.

Change

node --import tsx scripts/update-storage-render-goldens.ts — 4 files, digest lines only (16 lines).

Verification

Master's golden-storage-render legs are red on the merged tree (proven on
clean origin/master a046d4e: 4/5 fail, chat/anthropic/responses/google all
drift). Field-level forensics via driveWire dumpStateTo at the last green
golden (4bf28c5) vs current master: the ONLY divergence is
metadata.effectiveAbsorb, removed by the remove-absorb merge (#3309).
blocks/messageRefs/nudge/tokenSnapshot/nextBlockId digests are byte-identical.

#3298 regenerated the goldens on a pre-#3309 tree, so the batch merge
(or #3298's own branch point) left the goldens one absorption field behind;
master push CI skips the test matrix (known hole), so the drift surfaced on
the next PRs to run it — #3318's nine red test legs are entirely this.

Regenerated via scripts/update-storage-render-goldens.ts; golden-storage-render
+ storage-write-amplification + persist-guard-ownership all green.
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: chore/golden-regen-absorb-removal (26f3fd2)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3915-pr.3325.305def5

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3325.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

FALSE ALARM — closing. This regen is wrong and master was never red on fresh builds.

Full forensics (reproduced offline in a clean clone, fresh npm install --ignore-scripts + npm run build):

  1. Master's committed goldens pass 5/5 on a faithful CI replica (fresh install + fresh build, node 22 and node 25 both). The 'master red' I reported was an artifact of my own long-lived workspace: the golden drive spawns the bili proxy from the ROOT dist/, and my local dist was stale — it still stamped metadata.effectiveConfig.imageCompression + state.stats.imageBytesSaved/imageTokensSaved/imagesShrunk (fields removed by image_full 无法被正确调用:同一张图每轮重复降采样(38 次)、restore 提示注入 38 次,模型只能猜 ref → 每轮一次失败+重发,缓存命中从 75% 掉到 28% #3266). Stale-dist runs therefore produce digests that fresh builds never produce, in BOTH directions of comparison.
  2. This PR's regen baked exactly those stale-dist digests — every fresh-build CI leg that runs the golden test (u22-2, u24-1, w22-1) fails against it.
  3. fix(opencode-native): proxy 不可用时超窗请求本地拒发,不再静默直连撞上游 400 #3318's nine red legs were from its pre-fix: teach code-mode (PTC/run_code) harnesses the ACP calling conventions + name the project in the system prompt (#3252) #3298-regen run window (Oct 15 19:07Z, before fix: teach code-mode (PTC/run_code) harnesses the ACP calling conventions + name the project in the system prompt (#3252) #3298's rebaseline merged); against current master its matrix should be green.

Lesson recorded: the golden drive trusts dist/ freshness — local verification of storage-render goldens MUST rebuild (npm run build) first. I'll follow up with a guard that fails the test loudly when dist is older than src.

Branch left for reference; no merge.

@ranxianglei
ranxianglei deleted the chore/golden-regen-absorb-removal branch October 16, 2026 02:13
ranxianglei added a commit that referenced this pull request Oct 16, 2026
The storage-render golden drive renders persisted-state digests from
node_modules/acp-kernel/dist (a build product) while bili runs live via
tsx. A stale kernel dist silently stamps removed fields (post-#3266 it
still produced metadata.effectiveConfig.imageCompression and
state.stats.image* counters), so a golden regenerated from that workspace
bakes digests no fresh build can reproduce — CI then fails every leg that
carries the golden file. That is exactly how PR #3325 went wrong: 'master
red' was verified and regenerated from a stale local workspace, while a
clean fresh-install replica passed 5/5 against the committed goldens.

- kernelDistStale(root): compares node_modules/acp-kernel/dist/index.js
  mtime against the newest kernel/src (and tsup.config.ts) source; pure,
  synthetic-tree testable.
- driveWire() now fails fast with the rebuild instruction
  ('npm run build') before touching any session state. Call-time, not
  import-time, so the guard's own unit tests can import the module from a
  stale workspace.
- tests/golden-kernel-dist-guard.test.ts pins both directions + the
  missing-dist passthrough.

Verified both ways locally: stale dist -> drive refuses with the message;
after npm run build -> golden-storage-render 5/5 against master's
committed goldens.
ranxianglei added a commit that referenced this pull request Oct 24, 2026
…shness-guard

test: golden drive refuses to run on a stale kernel dist (#3325 lesson)
ranxianglei added a commit that referenced this pull request Oct 24, 2026
…n guard fired on restored artifacts

The #4377 cache skip left kernel/dist with save-time mtimes while the
checkout rewrote kernel/src mtimes; kernelDistStale() compares mtimes, so
the first real cache-HIT PR (this one — no kernel changes) made the
golden-storage drives refuse to run on 5 legs. Touch the dist tree on
every HIT (metadata-only; bytes are already marker-verified).
ranxianglei pushed a commit that referenced this pull request Oct 24, 2026
…d cannot misfire on cache-restored test legs

The #4360 pretest cache restores kernel/dist byte-exact (marker-verified)
with the SAVED mtimes, while the freshly checked-out sources are newer.
On the merge ref (which carries bff4d6d's mtime guard), every leg whose
shard runs tests/golden-storage-render.test.ts or
tests/storage-engine-proofs.test.ts then fails its first chat-wire drive
with 'node_modules/acp-kernel/dist is older than kernel/src (#3325)' on
perfectly correct bytes — 6/8 shards red on PR #4497 head 2c56f42
(run 40087540317) and on master itself.

A marker-verified HIT proves the bytes are correct for this source hash,
so the timestamp is refreshed to now on that branch only: the guard keeps
its teeth on genuinely stale workspaces (they still miss and rebuild),
and no rebuild is skipped that wasn't already proven correct.
ranxianglei added a commit that referenced this pull request Oct 24, 2026
…n guard fired on restored artifacts

The #4377 cache skip left kernel/dist with save-time mtimes while the
checkout rewrote kernel/src mtimes; kernelDistStale() compares mtimes, so
the first real cache-HIT PR (this one — no kernel changes) made the
golden-storage drives refuse to run on 5 legs. Touch the dist tree on
every HIT (metadata-only; bytes are already marker-verified).
ranxianglei pushed a commit that referenced this pull request Oct 24, 2026
…d cannot misfire on cache-restored test legs

The #4360 pretest cache restores kernel/dist byte-exact (marker-verified)
with the SAVED mtimes, while the freshly checked-out sources are newer.
On the merge ref (which carries bff4d6d's mtime guard), every leg whose
shard runs tests/golden-storage-render.test.ts or
tests/storage-engine-proofs.test.ts then fails its first chat-wire drive
with 'node_modules/acp-kernel/dist is older than kernel/src (#3325)' on
perfectly correct bytes — 6/8 shards red on PR #4497 head 2c56f42
(run 40087540317) and on master itself.

A marker-verified HIT proves the bytes are correct for this source hash,
so the timestamp is refreshed to now on that branch only: the guard keeps
its teeth on genuinely stale workspaces (they still miss and rebuild),
and no rebuild is skipped that wasn't already proven correct.
ranxianglei pushed a commit that referenced this pull request Oct 24, 2026
…d cannot misfire on cache-restored test legs

The #4360 pretest cache restores kernel/dist byte-exact (marker-verified)
with the SAVED mtimes, while the freshly checked-out sources are newer.
On the merge ref (which carries bff4d6d's mtime guard), every leg whose
shard runs tests/golden-storage-render.test.ts or
tests/storage-engine-proofs.test.ts then fails its first chat-wire drive
with 'node_modules/acp-kernel/dist is older than kernel/src (#3325)' on
perfectly correct bytes — 6/8 shards red on PR #4497 head 2c56f42
(run 40087540317) and on master itself.

A marker-verified HIT proves the bytes are correct for this source hash,
so the timestamp is refreshed to now on that branch only: the guard keeps
its teeth on genuinely stale workspaces (they still miss and rebuild),
and no rebuild is skipped that wasn't already proven correct.
ranxianglei pushed a commit that referenced this pull request Oct 25, 2026
…, #4599 sentinel)

actions/cache restores a verified-fresh kernel/pi-subagents dist with SAVE-TIME
mtimes, older than the just-checked-out sources; tests/golden-storage-drive.ts'
#3325 staleness guard is a pure mtime oracle (newest kernel/src vs
node_modules/acp-kernel/dist/index.js), so every cache-hit leg reds every
golden drive ~4ms in ('dist is older than kernel/src'). The marker already
proved content freshness — restamp the verified dist tree's mtimes to now so
the two oracles agree. Content untouched; HIT still never rebuilds.
Regression test pins the restore-then-warm sequence end to end.
@github-actions github-actions Bot mentioned this pull request Oct 27, 2026
ranxianglei pushed a commit that referenced this pull request Nov 14, 2026
…d cannot misfire on cache-restored test legs

The #4360 pretest cache restores kernel/dist byte-exact (marker-verified)
with the SAVED mtimes, while the freshly checked-out sources are newer.
On the merge ref (which carries bff4d6d's mtime guard), every leg whose
shard runs tests/golden-storage-render.test.ts or
tests/storage-engine-proofs.test.ts then fails its first chat-wire drive
with 'node_modules/acp-kernel/dist is older than kernel/src (#3325)' on
perfectly correct bytes — 6/8 shards red on PR #4497 head 2c56f42
(run 40087540317) and on master itself.

A marker-verified HIT proves the bytes are correct for this source hash,
so the timestamp is refreshed to now on that branch only: the guard keeps
its teeth on genuinely stale workspaces (they still miss and rebuild),
and no rebuild is skipped that wasn't already proven correct.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant