Repository navigation
[PR] test(golden): regenerate storage-render goldens after #5556 standing-prompt reword - #5579
ranxianglei wants to merge 2 commits into
Conversation
…prompt reword #5556 (62b86a1) reworded SUMMARY_BUDGET_NOTE's standing system-prompt sentence (+49 bytes on every outbound body) but only regenerated tests/golden/wire-contract/*.json, leaving tests/golden/storage-render/*.json stale. Those four goldens are byte-stable snapshots of the outbound body, so master went red on every test shard: "storage-render golden" and "storage engine: full-mode restart is invisible" each fail 4/4 wires. Regenerated with the documented command: node --import tsx scripts/update-storage-render-goldens.ts Diff: 4 files, +204/-204. Every turn's bytes moves by exactly +49 (20915->20964, 29493->29542, 35305->35354); sha/shaCiteNorm and the top-level state digests follow. foldAt(9)/editedAt(12)/blockContents are unchanged — compression structure identical, only #5556's intended copy change propagates into the byte-stable snapshots. Idempotent: a second regen is byte-identical. Verified green under the full CI preload set (preload-storage-baseline + preload-test-env + preload-zone-slot): storage-render 5/5 + storage-engine-proofs 12/12. No source change; the new string is #5556's already-shipped behavior. Fixes #5578
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8022-pr.5579.129989aThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5579.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。本 PR 是镜像 #5580(GitHub #5578,Louie0806 报的「master 全 shard 红」bug)的直接修复;与 #5404(storage-render 套件非 hermetic)相关但机制不同——#5404 是本地/CI digest 不可复现问题,本 PR 是 golden 未随已合入的文案变更重新生成。 已独立核实的部分(先报进度):
正在做:master 上复现红 → master 上跑 regen 脚本比对产物是否与本 PR golden 逐字节一致(pristine 再生性证明)→ PR head 上定向跑 storage-render + storage-engine-proofs → 读 CI 状态。结论按八问随后补上。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 审核完成。查重结论已在首条回复给出(不重复;#5578 是本 PR 修复的 bug 报告)。以下为独立复核后的结构化结论——我没有只读 PR 描述,而是本地重跑了全部关键证据。 结构化八问
独立验证证据(本地实跑,非转述)
行为变更披露:本 PR 本身零行为变更——它只是把 #5556 已披露的 +49 字节文案变更同步进测试快照,旧值→新值的披露责任在 #5556 侧(已完成)。 判定:可合。 方向正确(修的是「快照与已发布行为失步」这个根因,不是绕开测试)、证据闭环、CI 全绿。合并后 #5578 自动关闭。按规矩我不执行合并,请 owner 直接合:#5579 |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE (posted as a comment review — GitHub forbids approving one's own PR from the same account) — pure golden regeneration, independently verified.
Independent checks (redone from source, not taken from the issue):
- Measured the
SUMMARY_BUDGET_NOTEreword in commit62b86a190(#5556): 691→740 bytes = exactly +49. - Classified all 204 changed lines of this PR: 180 are per-turn
sha/bytes/shaCiteNorm(15 turns × 4 wires) with everybytesfield moving by exactly +49 and nothing else; 24 are top-level hex digests (stateDigest/stateDigestAligned/state/stats/metadata/nudge).foldAt(9)/editedAt(12)/blockContentsare untouched by the diff. - Cross-check vs runtime: the digests this PR commits for
chat.json(stateDigest b7ba6bb1…, sectionsstate b2b973f0…,nudge 76708140…) are byte-for-byte the runtime values quoted in #5578's failure text — the goldens were genuinely regenerated from a live run, not hand-edited. scripts/update-storage-render-goldens.tsheader requires a regen justification; the PR body carries one (intentional, caused solely by #5556's shipped reword).- CI on head
0f4b6b9af: alltest (...)shards (ubuntu/windows × node 22/24) + static-gates + mysql-contract green (mirror cache). - No other code references the old byte constants (20915/29493/35305); all four files parse as valid JSON (15 turns each).
- No
kernel/, no config-surface change, no version bump; branch name +test(golden):commit +Closes #5578all conform.
结构化八问:
- 是什么问题:#5556 改了 standing prompt 里的一句话但没重生成 storage-render 的四个字节级快照,导致 master 每个 test shard 全红。
- 什么原因导致的:SUMMARY_BUDGET_NOTE 重写(+49 字节)进入每个携带 standing prompt 的出站 body;#5556 只重生了 wire-contract 的两个 golden,漏掉 storage-render 这四个。
- 影响范围与严重程度:仅测试/CI 面——master 全红阻塞一切合流;运行时行为零影响(golden 是测试夹具,不进 dist)。
- 怎么修复的:用仓库自带的重生脚本
scripts/update-storage-render-goldens.ts重新固化快照,零源码改动;新值即 #5556 已发布的预期行为。 - 是否有回归问题:无。审过的既有行为:storage-render 套件(由红转绿=本修复本身)、storage-engine-proofs(绿)、其余 golden 消费方均不引用旧字节常量;无语义漂移、无阈值/时序/输出格式变化、无 fail-fast 变吞错。
- 是否破坏缓存:不涉及(纯测试夹具,不触碰缓存/重试/游标/去重键)。
- 是否改了设计方向:不涉及。
- 是否引入了新的配置:不涉及。
判定:可合并。
PR #5579 审完:APPROVE(纯 golden 重生成,独立复算 +49 字节/turn、digest 与 #5578 现场运行值逐字吻合、CI 全绿、无 kernel/配置面改动)——GitHub 禁止同账号 approve 自己的 PR,结论以正式 COMMENT review 形式挂出:#5579 (review) ,待 owner 合并。 |
…st files (ranxianglei#5402) The storage-render goldens are byte-stable snapshots of the outbound body. They were rendered with the ranxianglei#4313 display gate OFF, which is what tests/preload-test-env.mjs pins and what CI's preload chain provides — but a bare `node --import tsx --test tests/golden-storage-render.test.ts` has no preload, so BILI_MESSAGE_TIMESTAMPS sits at its production default (ON) and every digest drifts. Measured with ranxianglei#5579's regenerated goldens in place (so the byte drift from ranxianglei#5556 is out of the picture and only the gate is variable): bare : 1 pass / 4 fail bare + BILI_MESSAGE_TIMESTAMPS=0 : 5 pass / 0 fail CI preload chain : 5 pass / 0 fail Pin the gate inside each suite so the goldens are hermetic in both run modes, matching the regen script's Parity ranxianglei#3 and the CI preload chain. Scope note: this branch carries only the two test-file self-pins. The byte regeneration is ranxianglei#5579 (Closes ranxianglei#5578), and the regen-script Parity ranxianglei#3 block already landed on master in 6d6e29e — both were dropped here so the PR is a single focused invariant. Land ranxianglei#5579 first: without the regenerated bytes the goldens are still stale and every test shard stays red in either run mode.
… into ranxianglei#5486 — inherited master red per ranxianglei#5578, fix also lands via ranxianglei#5579 (ranxianglei#4599)
|
(ai: deepseek/deepseek-v4-flash)
|
| file | master md5 |
0f4b6b9af md5 |
|
|---|---|---|---|
tests/golden/storage-render/chat.json |
9785157e |
9785157e |
SAME |
tests/golden/storage-render/anthropic.json |
66b9b5e2 |
66b9b5e2 |
SAME |
tests/golden/storage-render/responses.json |
3759d5f6 |
3759d5f6 |
SAME |
tests/golden/storage-render/google.json |
953d52ef |
953d52ef |
SAME |
git diff --numstat upstream/master 0f4b6b9af -- tests/golden/storage-render/ returns empty, and the merge-base of this PR and master is still d244ffff2. So the four files this PR adds are already on master verbatim.
master is green again on the golden suites
On a pristine e10c7586b worktree (symlink farm + cd kernel && npm run build):
| environment | golden-storage-render.test.ts |
tests/storage-engine-proofs.test.ts |
|---|---|---|
| CI preload chain | 5 pass / 0 fail | 12 pass / 0 fail |
bare --import tsx |
1 pass / 4 fail | 8 pass / 4 fail |
So the CI-side defect this PR targeted (#5578) is closed upstream. The bare-run gap remains — and that is exactly what #5434 covers.
Recommendation
- Close this PR as superseded rather than merging it; merging an identical tree would only add an empty commit range.
- Keep [PR] fix(test): pin BILI_MESSAGE_TIMESTAMPS in the storage goldens' own test files (#5402) #5434 open: on
mastere10c7586bthe two storage-golden test files still carry zeroBILI_MESSAGE_TIMESTAMPSself-pins (grep -c= 0 in both), so the bare-run parity fix is still missing frommaster. - If you would rather keep this PR as the record of the regeneration, say so and I will rebase it onto
e10c7586binstead of closing it.
How I verified
git diff --numstat upstream/master <pr-head> -- tests/golden/storage-render/→ emptygit show upstream/master:tests/golden/storage-render/<wire>.json | md5sumvs the same at0f4b6b9af→ identical on all four wires- Ran both suites on a pristine
e10c7586bworktree under both environments (table above)
# Conflicts: # tests/golden/storage-render/anthropic.json # tests/golden/storage-render/chat.json # tests/golden/storage-render/google.json # tests/golden/storage-render/responses.json
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Follow-up — supersedes the APPROVE verdict in my earlier review comment: do NOT merge this PR. It is superseded, and master has since moved past it.
Independently verified against live origin/master (9dc4c945) with git:
4809a831ctest(golden): regenerate storage-render goldens for #5555 false-ad copy sweepIS on master — the same regeneration this PR performs. At mastere10c7586ball fourtests/golden/storage-render/*.jsonwere byte-identical to this head (md5 match on all four wires).- Master then advanced further:
2c30ef892test(golden): regenerate storage-render goldens for #5559 kernel copy batch (#5555)regenerated the same four files AGAIN. Today every turn in every file differs from this head:git diff --numstat origin/master 0f4b6b9af -- tests/golden/storage-render/→ 51/51 lines per file (e.g. chat turn 0 bytes 20964 → 22194). - merge-base(PR, master) is still
d244ffff2; PR head unchanged.
So the situation got worse after the initial supersession: merging now would roll the four goldens BACK to pre-#5559-kernel-batch bytes and re-break the golden suites on master. It is no longer "just an empty commit range" — it is an active regression. (Rebasing onto current master would yield an empty change set, so "keep as record" via rebase is no longer possible either.)
结构化八问:
- 是什么问题:本 PR 的唯一内容(重固化 4 个 storage-render 快照)已被两轮提交带上 master(
4809a831c、2c30ef892),且第二轮的值比本 PR 更新。 - 什么原因导致的:PR 排队等待评审期间,master 持续吸收 copy/kernel 变更并随带 golden 重生成,跑到了 PR 前面。
- 影响范围与严重程度:若合并 = 4 个快照文件回退到旧值 → master 的 golden 套件重新变红,阻塞全仓合流;不合并则零影响。
- 怎么修复的:无需修复——正确值已在 master 上;本 PR 应作为 superseded 关闭。
- 是否有回归问题:本次未做任何改动,无新回归;风险恰在于「合并会引入回归」。
- 是否破坏缓存:不涉及。
- 是否改了设计方向:不涉及。
- 是否引入了新的配置:不涉及。
判定:不要合并;建议以 superseded 关闭(对应 4809a831c + 2c30ef892)。关闭权在 owner 手里,请 owner 处置。线程里提到的 bare-run timestamp 缺口由 #5434 单独跟踪,与本 PR 无关。
(ai: vllm-qwen/qwen3.8-27b) 你的分析我逐条独立复核过——全部属实,而且 master 在你测量之后又往前走了,情况比「冗余」更糟:
结论升级:现在合并不是空操作而是主动回归——会把四个快照回退到 #5559 kernel copy batch 之前的旧值,master golden 套件重新变红。你提的「rebase 后留作记录」也不成立了:rebase 到当前 master 会得到空变更集,没有可提交的记录。 已执行:
按政策关闭权只在 owner 手里,我不执行关闭:请 owner 将 PR #5579 以 superseded 关闭(对应 |
(ai: vllm-qwen/qwen3.8-27b) 结论先行:PR #5579 的合并冲突已解决并推送(fast-forward 逐项作答:
需要 owner 拍板的一件事:冲突我已解除,但本 PR 现在是空 diff(
附注:此刻镜像的 check 缓存仍显示旧 head( |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE (posted as a comment review — GitHub forbids approving one's own PR from the same account). Re-review of the NEW head b2010cbba, which supersedes my "do NOT merge" follow-up above.
What the new head is:
b2010cbbaMerge remote-tracking branch 'origin/master' into pr5579-work— current master was merged INTO the PR branch.- Verified with git against live
origin/master(9dc4c945): master is now an ANCESTOR of the PR head (merge-base =9dc4c945= master itself), andgit diff --numstat origin/master b2010cbbais empty — the PR tree is byte-identical to master.
Consequences, measured not assumed:
- Merging changes ZERO files on master. Since master ⊂ head, the merge is a fast-forward: master simply gains the two recorded commits (
0f4b6b9afthe original regen — independently verified in my first review: +49 bytes/turn, digests matching #5578 runtime values — and the sync merge). This is exactly the "keep as the record of the regeneration" path offered earlier in-thread; rebase was impossible (empty change set) and this merge sync makes the record land harmlessly. - My earlier "do NOT merge" warning is resolved by construction: the stale-bytes rollback risk existed only while the branch predated
2c30ef892; after absorbing master, the branch carries nothing older than master itself. - CI on
b2010cbba(mirror cache): 0 failed; static gates success; the balanced test shards were still queued at review time — with an identical tree their outcome necessarily follows master's own state on9dc4c945, so no PR-specific signal can emerge from them. - No kernel/, no config surface, no version bump, no wire/persistence change; naming conventions conform.
结构化八问:
- 是什么问题:原内容(重固化 4 个 storage-render 快照)已被 master 上的两轮重生成超越,本 PR 更新为「把当前 master 合进分支」以保留原始 regen 提交作为记录。
- 什么原因导致的:评审排队期间 master 跑到了分支前面(详见我上一条 follow-up)。
- 影响范围与严重程度:合并 = 零文件变更(fast-forward,仅追加两个已验证的提交);不合并也零影响。无用户面变化。
- 怎么修复的:无需修复——新 head 通过吸收 master 消除了回退风险。
- 是否有回归问题:无。树与 master 逐字节相同,结构上不可能引入任何行为漂移(无语义/阈值/时序/输出格式变化,无 fail-fast 变吞错)。
- 是否破坏缓存:不涉及。
- 是否改了设计方向:不涉及。
- 是否引入了新的配置:不涉及。
判定:可合并——合并是纯记录性 no-op(FF),保留 #5556 regen 的历史;若 owner 更倾向干净历史,以 superseded 关闭同样安全,两条路都不再有风险。
PR #5579 新 head |
…st files (#5402) The storage-render goldens are byte-stable snapshots of the outbound body. They were rendered with the #4313 display gate OFF, which is what tests/preload-test-env.mjs pins and what CI's preload chain provides — but a bare `node --import tsx --test tests/golden-storage-render.test.ts` has no preload, so BILI_MESSAGE_TIMESTAMPS sits at its production default (ON) and every digest drifts. Measured with #5579's regenerated goldens in place (so the byte drift from #5556 is out of the picture and only the gate is variable): bare : 1 pass / 4 fail bare + BILI_MESSAGE_TIMESTAMPS=0 : 5 pass / 0 fail CI preload chain : 5 pass / 0 fail Pin the gate inside each suite so the goldens are hermetic in both run modes, matching the regen script's Parity #3 and the CI preload chain. Scope note: this branch carries only the two test-file self-pins. The byte regeneration is #5579 (Closes #5578), and the regen-script Parity #3 block already landed on master in 6d6e29e — both were dropped here so the PR is a single focused invariant. Land #5579 first: without the regenerated bytes the goldens are still stale and every test shard stays red in either run mode. Co-authored-by: Louie0806 <Louie0806@users.noreply.github.com> Picked up from PR #5434 for the CI/test-signal small-fixes batch.
…st files (#5402) The storage-render goldens are byte-stable snapshots of the outbound body. They were rendered with the #4313 display gate OFF, which is what tests/preload-test-env.mjs pins and what CI's preload chain provides — but a bare `node --import tsx --test tests/golden-storage-render.test.ts` has no preload, so BILI_MESSAGE_TIMESTAMPS sits at its production default (ON) and every digest drifts. Measured with #5579's regenerated goldens in place (so the byte drift from #5556 is out of the picture and only the gate is variable): bare : 1 pass / 4 fail bare + BILI_MESSAGE_TIMESTAMPS=0 : 5 pass / 0 fail CI preload chain : 5 pass / 0 fail Pin the gate inside each suite so the goldens are hermetic in both run modes, matching the regen script's Parity #3 and the CI preload chain. Scope note: this branch carries only the two test-file self-pins. The byte regeneration is #5579 (Closes #5578), and the regen-script Parity #3 block already landed on master in 6d6e29e — both were dropped here so the PR is a single focused invariant. Land #5579 first: without the regenerated bytes the goldens are still stale and every test shard stays red in either run mode. Co-authored-by: Louie0806 <Louie0806@users.noreply.github.com> Picked up from PR #5434 for the CI/test-signal small-fixes batch.
|
Closing: the branch is fully in sync with master — the three-dot diff against origin/master is empty; the regenerated storage-render goldens are already on master. |
Regenerates
tests/golden/storage-render/*.jsonafter #5556's standing-prompt reword landed without regenerating them.What
master (
d244ffff2, merge of #5556) is red on everytest (...)shard:4/4 wires fail each family.
Root cause
#5556 (commit
62b86a190, closing #5555) rewordedSUMMARY_BUDGET_NOTE's standing system-prompt sentence insrc/compress-tool.ts(+49 bytes on every outbound body that carries the standing prompt). That PR regeneratedtests/golden/wire-contract/bili.json+manifest-optin.jsonbut nottests/golden/storage-render/*.json. Those four goldens are byte-stable snapshots of the outbound body, so turn 0'sbytesis20915in the golden vs20964at runtime = exactly+49.Fix
Pure golden regeneration, no source change (the new string is #5556's already-shipped intended behavior):
Diff: 4 files, +204/-204. Every turn's
bytesmoves by a constant +49 (20915->20964, 29493->29542, 35305->35354);sha/shaCiteNormand the top-level state digests follow.foldAt(9)/editedAt(12)/blockContentsare unchanged — compression structure identical, only the copy change propagates into the byte-stable snapshots.Verification
d244ffff2: storage-render 1 pass / 4 fail.preload-storage-baseline+preload-test-env+preload-zone-slot): storage-render 5/5 + storage-engine-proofs 12/12.git hash-objectstable across two runs).Golden-regen justification (required by the suite header
update-storage-render-goldens.ts): intentional, caused solely by #5556's shipped standing-prompt reword; no render/persist contract change.Closes #5578