Skip to content

[PR] test: unpin legacy envelope suites from the storage baseline - #6561

Merged
ranxianglei merged 2 commits into
ranxianglei:masterfrom
sx4im:fix-6547-legacy-envelope-suites-red
Nov 15, 2026
Merged

ranxianglei merged 2 commits into
ranxianglei:masterfrom
sx4im:fix-6547-legacy-envelope-suites-red

Conversation

@sx4im

@sx4im sx4im commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

After #6480 merged, two test suites went deterministically red on every shard, unrelated to any single PR diff.

notice-registry failed on ubuntu because the global baseline pins BILI_STORAGE_UNIFIED=legacy, and the suite asserts the sqlite-default notice is visible. That notice is only active when no storage mode is pinned, so the suite now clears the five storage knobs at load, same as sqlite-default-notice does.

persist-zstd and persist-metadata-fold-gate failed on windows because in full mode each store holds index.db open in the temp dir, and teardown only ran rmrf. Windows refuses to delete open files. Both suites now close every store they created before removing the dir.

Test-only change, no product code touched. Verified locally with the CI preload chain: notice-registry 5/0 (was 3/2), persist-zstd 17/0, persist-metadata-fold-gate 3/0, and sqlite-default-notice 8/0. The full-suite verdict is CI on this head.

Fixes #6547

@ranxianglei ranxianglei changed the title test: unpin legacy envelope suites from the storage baseline [PR] test: unpin legacy envelope suites from the storage baseline Nov 13, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Duplicate screening: no duplicate — GitHub PR search for #6547 returns this PR only. The sibling #6480 follow-ups (#6567, #6576) target the e2e lanes, a different symptom and different files; they do not overlap this one.

Review — I re-verified everything independently rather than taking the issue's analysis on faith:

Direction (original issue): #6547's diagnosis is sound and evidence-backed, and this PR implements exactly the two proposed fixes at the right layer — no symptom masking (it does not loosen the assertions, does not drop the notice from the expected list, and does not extend the rmrf retry ladder to paper over self-held handles — that ladder exists for external locks only, #1910/#2799/#2908).

Defect A (notice-registry): reproduced on pre-PR master (e1f31ad9d5, whose parent is the cited #6480 merge a007974683) with the CI preload chain: 3 pass / 2 fail, failing exactly visibleNotices: fresh install… and visibleNotices: dismissing one hides only that one. Mechanism confirmed in code: tests/preload-storage-baseline.mjs:19 pins BILI_STORAGE_UNIFIED=legacy, and sqliteDefaultNoticeActive in src/notices.ts returns false under any explicit non-sqlite selection — so the suite's own "default posture" assertion (which its header comment already declares as the intent) cannot hold under the inherited pin. On this head: 5/0. The five-knob clear matches tests/sqlite-default-notice.test.ts verbatim — the documented discipline from the preload file's own header (#4011 lesson).

Defect B (persist suites): verified the fd lifecycle directly — a full-mode SessionStore holds index.db, index.db-wal, index.db-shm open in the test dir (3 handles before, 0 after closeQuietly(), src/persist.ts:3965). Master's teardown ran rmrf with all three still open → deterministic EBUSY on Windows. On this head: persist-zstd 17/0, persist-metadata-fold-gate 3/0 (Linux can't reproduce the EBUSY itself, but the handle-release the fix depends on is verified).

Regression review (existing behavior of the touched files): all 5 notice-registry tests are consistent with the cleared-knob posture; the env mutation is module-level in a per-process node --test worker, so it cannot leak into other suites. In both persist suites the close happens in finally strictly after every assertion, each test owns a fresh mkdtemp dir, and closeQuietly cancels (not flushes) pending writes — no test depends on a post-assertion flush. No high-risk patterns: no semantic drift, no threshold/timing/output-format change, no fail-fast→swallow. The one deliberate behavior change — notice-registry's test environment moves from "inherits legacy pin" to "no storage pin (default posture)" — is the fix itself and is disclosed in the PR body.

Test validity: the changed assertions have teeth (red before, green after, reproduced by me). The gate's negative direction (pinned mode ⇒ notice hidden) stays covered by sqlite-default-notice.test.ts (8/0 here, unchanged) and by the baseline itself for the rest of the corpus.

Diff & code quality: 1 commit, 3 test files, +38/−2, no product code, no unrelated changes, no whitespace/lock churn; branch name and commit use only the GitHub number. Naming (stores, closeAll), comments (why + #6547 provenance), and style all meet the bar. The inline five-knob list duplicates sqlite-default-notice.test.ts, but that matches the repo's established per-suite convention and the issue's own proposed shape — not a defect.

Local verification summary (CI preload chain, this head): notice-registry 5/0 (master: 3/2), sqlite-default-notice 8/0, persist-metadata-fold-gate 3/0, persist-zstd 17/0; tsc --noEmit clean. All four numbers match your reported local runs exactly.

CI status (verified live just now): all guard/gate jobs green (kernel-guard, lane-guard, config-surface-guard, docs-sync, docs-coverage, lock-sync, version-guard, build-artifact, unused-exports, test-import-graph, architecture-docs, gates); gated e2e lanes skipped as expected for a test-only diff. The test matrix lanes (ubuntu 22/24 × 3, windows 22 × 5), static-gates, and mysql-contract are still queued — the full-suite verdict is pending. The branch is current (base = master e1f31ad9d5, no conflict).

Verdict: review passes — mergeable once the queued CI test lanes go green (the previously-red ubuntu shard 3 and windows shards 3/4/5 should turn green on this head).


结构化八问

  1. 是什么问题:[PR] storage: pure-SQLite default — JSON envelopes stop being written unless dual-write is requested (#6479) #6480(pure-sqlite 默认翻转)合入后 CI 测试矩阵出现两簇确定性红——notice-registry 在 ubuntu 分片红,persist-zstd / persist-metadata-fold-gate 在 windows 分片红,与任何单条 PR 的 diff 无关。
  2. 什么原因导致的:A——全局测试基线(tests/preload-storage-baseline.mjs:19)无条件钉 BILI_STORAGE_UNIFIED=legacy,而 [PR] storage: pure-SQLite default — JSON envelopes stop being written unless dual-write is requested (#6479) #6480 给 notice-registry 新加了「默认姿态下 sqlite-default 通知可见」的断言,套件自身没清继承来的钉(违反该文件页首 [PR] fix: make full-mode storage tests deterministic (#4004) #4011 纪律);B——[PR] storage: pure-SQLite default — JSON envelopes stop being written unless dual-write is requested (#6479) #6480 给两个 persist 套件加了 full 双写钉,每个 store 在临时目录打开 index.db(+wal/shm),teardown 只 rmrf 不关句柄,Windows 拒绝删除被打开的文件。
  3. 影响范围与严重程度:全部测试分片(ubuntu 22/24、windows 22),master CI 持续红,阻塞其他 PR 的合并信号;不触碰产品代码、无用户可见行为变化。属 CI/测试基础设施级故障,非用户面缺陷。
  4. 怎么修复的:A——套件加载时清空五个 storage knob(与 sqlite-default-notice.test.ts 既有纪律逐字一致),断言落在默认姿态上;B——把套件创建的所有 store 收集起来,finally 里先 closeQuietly() 再 rmrf。两处都是测试卫生修复,不放松任何断言、不扩 rmrf 重试阶梯(那阶梯只针对外部锁)。
  5. 是否有回归问题:审过三个套件的既有行为——notice-registry 5 个测试在「无钉」姿态下全部自洽(套件页首本就声明 defaults 意图),env 改动进程隔离不外泄;persist-zstd 17 个、fold-gate 3 个测试全绿,close 在断言之后、不改变断言语义(closeQuietly 只取消未落盘写、无测试依赖断言后 flush)。无高危模式(无语义漂移/阈值漂移/fail-fast 变吞错)。唯一行为变更:notice-registry 的测试环境从「继承 legacy 钉」变为「默认姿态」——即修复本身,PR 正文已披露。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及(五个 knob 均为既有 env 变量,只是清空、未新增)。

收工判定:审核通过、可合并——待 CI 测试矩阵(当前 queued)跑绿即可;master 上红的 ubuntu 分片与 windows 分片在本 head 应转绿。

@Louie0806

Copy link
Copy Markdown
Contributor

(ai: deepseek/deepseek-v4-flash)

Independently verified on a detached worktree at e1f31ad with the CI preload chain: your patch takes tests/notice-registry.test.ts from 3 pass / 2 fail to 5 pass / 0 fail, and leaves both persist suites at 17/0 and 3/0 on Linux. The two persist suites only redden on Windows, which matches your diagnosis.

One heads-up on merge ordering, since it affects you if this lands after PR #5178: I had independently prepared the same two fixes and they are currently carried as two commits on top of #5178 (9bc1671bb clearing the same five storage knobs in the notice-registry suite, 86ed00a02 closing every store before rmrf in both persist suites). A trial merge of your head into #5178's head conflicts in exactly two files, tests/persist-zstd.test.ts and tests/persist-metadata-fold-gate.test.ts, because we both add a stores array and a close loop at the same spots.

If this PR merges first, I will drop my two commits and take yours verbatim on #5178's next master sync, so there is no duplicated fix in master. If #5178 merges first, this PR can be closed as delivered, since the fixes are semantically equivalent. No action needed from you either way.

@Louie0806

Copy link
Copy Markdown
Contributor

(ai: deepseek/deepseek-v4-flash)

The single red check on this head is test (windows-latest, 22, 3), and it is not related to this patch.

That job's only failure is not ok 1415 - successive rotations in tests/logger-rotate.test.ts retain distinct payloads without overwriting: the retained generations decode to [0,1,2,3,4,5,6,8,9,10,11], i.e. record 7 is missing. tests/logger-rotate.test.ts is byte-identical between master head e1f31ad9d5b47500362699a9867819b861c7de96 and this PR head 8a082c24c (md5 25200cdd65c2a432530bc775fbc02569 on both), and this PR touches only the three suites named in its body. So the failure cannot be caused by the change under review.

Two details worth pinning so this does not get misread as a regression:

Net: the three suites this PR fixes are green in the log (notice-registry, persist-zstd, persist-metadata-fold-gate all pass), and the remaining red is a known shared-runner timing flake on an untouched byte-identical file. A re-run of that shard is the right next step; the patch itself is verified locally (notice-registry 3/2 to 5/0, both persist suites 17/0 and 3/0).

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Duplicate screening: no duplicate (established in the review above — #6547 maps to this PR only).

CI attribution — verified independently, not taken on faith:

Action taken: re-run of the failed job triggered (run 43458831689, job 132025271933). If it lands green, the matrix is fully green and the PR is mergeable. If it lands red with the same signature, I will post it in #4599 as a candidate deterministic Windows logger-rotation race — that would be a separate matter from this PR, and this PR's own changes would remain verified green either way.

Merge ordering with #5178: both orderings work, and I need nothing from either side —

The trial-merge conflict in the two persist files is expected (same fix, same spots) and resolves by whichever lands first. No duplicated fix can reach master under either ordering.


结构化八问

  1. 是什么问题:本 PR CI 唯一红项是 test (windows-latest, 22, 3),其唯一失败用例是 tests/logger-rotate.test.ts 的「successive rotations retain distinct payloads without overwriting」(记录 7 丢失)——与本 PR 的 diff 无关。
  2. 什么原因导致的:共享 Windows runner 负载下的日志轮转写丢失竞态(load-sensitive timing race)。该测试文件与 master 逐字节相同(两侧 md5 均为 25200cdd…),本 PR 只改 3 个测试文件;CI runner 按设计不重试具体断言失败(只重试 CI flake:test (windows-latest, 24) 单文件文件级失败:子测试零执行、异常被吞为 'test failed' #3029/Flaky Windows CI: issue1993-preflight-400 suite cancelled by early event-loop drain | Windows CI 偶发假红:preflight-400 测试进程提前退出致 5 例取消 #3665/windows-22-1 lane randomly red: during test cleanup, unlink index.db hit an EBUSY external lock, couldn't survive tmp-rm | windows-22-1 车道随机红:测试清理期 unlink index.db 遇 EBUSY 外部锁,熬不过 tmp-rm 的 9.6 秒重试梯子(#2799/#2908 族复发) #4347/CI: windows-latest node-22 shard 2 intermittently red on timing-sensitive e2e tests under load (rotating flakes: issue760, #4079, storage-full-flip #3818) #4161 这几类特定 flake 形状),所以一次性竞态直接把分片打红。
  3. 影响范围与严重程度:仅本 PR 的 CI 信号(41 项检查中 1 项红);无用户可见影响、无产品代码涉及。
  4. 怎么修复的:无代码改动——已触发失败分片重跑(run 43458831689);本地 3/3 绿(CI preload 链、本 head)+ 对方 pristine master 10/10 绿,佐证 runner 负载 flake。若重跑同签名再红,升级至 CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 按「Windows logger 轮转确定性竞态候选」处理。
  5. 是否有回归问题:不涉及(无代码改动);本 PR 修复的三个套件在 CI 日志中全绿。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

收工判定:审核已通过(上方 review);唯一红项为无关 flake,重跑已触发——重跑转绿即可合并。

@ranxianglei
ranxianglei merged commit fd60415 into ranxianglei:master Nov 15, 2026
48 checks passed
@ranxianglei

Copy link
Copy Markdown
Owner

Closes pair #6547 was auto-closed by the batch merge.

Superseded by #6765 (merged to master as fd60415).

Your commit c070df9 is in master verbatim — same content, same authorship, only the object ID changed (the batch used a real git merge, no rebase or cherry-pick rewrite). Merged under a different branch name, so GitHub does not flip this PR by itself.

Thanks for the fix — it is live in master from fd60415 onward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug][tests] #6480 makes two legacy-envelope suites red on every shard: notice-registry inherits the legacy pin, persist suites hold index.db past rmrf

3 participants