Repository navigation
[PR] fix(storage): persist a schema reader floor + unblock the notice-store migration - #6347
ranxianglei wants to merge 3 commits into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.11119-pr.6347.1a21188This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6347.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重: 不重复 — #5768 是本 PR cherry-pick #5761 一半的来源 PR(patch 逐字节一致、作者信息保留),不是并行工作;本 PR 合入后 #5768 被取代,owner 关闭败方。 方向关(①): 两个根因都真实存在且定位正确。#5761 根因是 hydrate 重入门用 回归审查(②): 无无意回归。触碰路径既有行为逐项核验:
代码审查(③): 两后端门等价性成立(sqlite: floor > version → 拒绝;mysql: current > version && floor > version,功能等价);原子性论证成立(floor ≤ 写入 build 的 stamp,故 floor > runningVersion 蕴含 stamp 高于 runningVersion);torn pair 会低报 floor,所以 stamp+floor 必须同事务——已实现。 测试有效性(④): 真测试、有牙,#3617 双向门——sqlite 6 条(不变量 floor≤version;迁移写 floor+stamp;拒绝臂精确错误字段;3 个通过臂:当前版本+已知 floor / 无 floor 行 / 加性未来 bump),mysql 5 条(typed driver fake;拒绝臂;通过臂含「无迁移时不重写 floor」;迁移臂断言 stamp+floor 在同一 begin..commit 窗口),notice-store 2 条回归测试在旧代码上会失败(预水合 dismissal 阻塞迁移;水合中 dismissal 被最终发布覆盖)。 diff 整洁度(⑤): 6 文件 +591/−35,每行都在范围内;无 CHANGELOG 条目——与近期 fix PR(如 #6326)惯例一致,不阻塞。 八问:
CI: 12 绿 + 16 跳过,0 失败;2 个 test shard 仍在跑(ubuntu-24 shard 1、ubuntu-22 shard 3)——全部 windows shard、mysql-contract 真驱动车道、static gates 已绿。 |
fba61ee to
ddb159d
Compare
(ai: vllm-qwen/qwen3.8-27b) 本次运行:解决 PR 与 master 的合并冲突并推送。 做了什么
八问
判定: 冲突已解决并推送,PR 恢复干净可合;CI 在新 head 上重跑中(0 失败)。本 PR 属持久化格式扩展 + 行为产品决策,不合 auto-merge 门,CI 全绿后需人工合并。 |
… legacy→kv migration (#5761) A dismissal landing before the cli-boot hydration lazily created the receipts cache, which made hydrateNoticeReceipts bail on `cache !== null` — skipping the one-time legacy JSON→kv migration for that process's lifetime (re-import churn on every boot) and hiding sibling lanes' kv rows until restart; a dismissal landing inside the hydrate's await gap was clobbered by the final cache assignment. Twin of the host-plugin-notice fix that landed in the #5757 line (commit 4d74613), ported onto master's API surface (unifiedKvActive/getStore; the kvStore() probe from that line is not merged yet): - hydrate unions into the existing lazy cache instead of bailing; a new hydrated latch (distinct from lazy-init) gates re-entry - the live-cache snapshot moves to just before the publish with no awaits in between, so mid-hydration dismissals survive the final assignment - publish only on success: a failed listKvAsync now leaves the cache null (legacy-file fallback stays live, next call retries) instead of clobbering it with an empty/partial map for the process lifetime Tests: two regression cases mirroring the #5757 shapes — both fail on the pre-fix code and pass with it; adjacent notice suites green. (cherry picked from commit cf85588)
…ntics (cherry picked from commit 0b2c949)
…s a store it cannot read (#5788) An older build opening a store a newer build migrated with a NON-ADDITIVE change would silently write rows it cannot read back (rolling upgrade / rollback). Persist, per store, the oldest schema version the writing build can still read (STORAGE_MIN_READER_VERSION / MYSQL_MIN_READER_VERSION, both 8 — v8 is the only non-additive migration, the identity backfill). The floor and the stamp move in ONE transaction (a torn pair would under-report the floor and let a too-old build in). On open, a build below the stored floor REFUSES TO START (same bucket as CANTOPEN/readonly — the data is newer, not corrupt; degrading would split-brain, #3923). A missing or malformed floor is unknown and passes, so existing stores never false-refuse. sqlite: kv_store(storage, schema_reader_floor) + user_version in one txn; gate in the from>=version no-op branch. mysql: bili_meta schema_reader_floor + schema_version in one txn; gate in init(). shadow.ts: dedicated refusal branches naming the real remediation (upgrade bili / restore a pre-upgrade backup) instead of the generic tail, whose "move index.db aside to force a fresh rebuild" would destroy the newer build's data. Tests: tests/storage-reader-floor.test.ts — sqlite hermetic (invariant, migration writes floor atomically, refuse arm, pass arms: current / missing row / additive future bump) + mysql hermetic (typed driver fake: refuse arm, pass arms, migration writes stamp+floor in one txn).
ddb159d to
16c0802
Compare
(ai: vllm-qwen/qwen3.8-27b) 本次运行:master 又前进了( 做了什么
八问
判定: 冲突已解决并推送,PR 恢复干净可合;git 层面确认 master 已是 PR head 的祖先。CI 在新 head 上重跑中(镜像缓存稍后收敛)。本 PR 属持久化格式扩展 + 行为产品决策,不合 auto-merge 门,CI 全绿后需人工合并。 |
|
(ai: vllm-qwen/qwen3.8-27b) Review verdict: APPROVE — clean, no blocking findings. (Posted as a comment: GitHub rejects a formal APPROVE review on a PR authored by the same account, "can't approve your own pull request" — this comment is the review. Both fixes are correct, well-tested, and the one behavior change is disclosed. #5761 (notice-store hydration race): The old #5788 (schema reader floor): Persists per-store the oldest schema version the writing build can still read ( Behavior change (disclosed in PR body): opening a store whose persisted reader floor is above this build's schema version now REFUSES TO START where it previously opened (and silently wrote). This is the intended fix — the old open-and-write was the data-corruption footgun. Pre-feature stores (no floor row) and additive-bump stores (floor ≤ this build) are unaffected. Tests are comprehensive: 八问(中文,供人工 reviewer)
判定:可合并(APPROVE)。 无 kernel/ 改动、无配置面变更、测试充分、行为变更已披露,无阻塞项。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Review of PR #6347 (mirror #6348) complete — verdict: APPROVE, no blocking findings. What I did:
Outcome: The PR is open and blocked awaiting human merge (merge is human-only per AGENTS.md). No small findings to fix on the spot. The 8-question checklist is in the review comment on the PR. |
Root-cause batch for the
[根因] storageaggregation (mirror #6267). One PR, two root causes, both in the storage / notice-persistence layer:#5761 — notice-store hydration race (cherry-picked from #5768)
src/notice-store.ts: a dismiss landing between process start and cli-boot hydration bailed the whole-process hydration (thecache !== nullgate), skipping the one-time legacy JSON→kv migration and hiding sibling lanes' kv rows in-process. The fix unions the hydrate into whatever lazy cache exists instead of bailing, with a distincthydratedlatch and a live-cache snapshot just before publish (publish only on success — a failed hydrate leaves the legacy-file fallback live and retries next call).Commits (cherry-picked with
-x, original authors preserved):13e7f93d26ci-check — the fix6ef363c9feework-review — comment alignment#5788 — schema reader floor
The schema stamp only says "how far a store was migrated" — nothing about whether an OLDER build can still read the result. An older build opening a newer-migrated store would no-op (sqlite) or re-run additive DDL (mysql) and then silently write rows it cannot read back.
The fix persists, per store, the oldest schema version the writing build can still read (
STORAGE_MIN_READER_VERSION/MYSQL_MIN_READER_VERSION, both 8 — v8 is the only non-additive migration, the identity backfill):kv_store(storage, schema_reader_floor)+user_versionin ONE transaction; gate in thefrom >= versionno-op branch.bili_meta schema_reader_floor+schema_versionin ONE transaction; gate ininit().A missing or malformed floor is UNKNOWN and passes, so existing (pre-feature) stores never false-refuse. The floor and the stamp move atomically (a torn pair would under-report the floor and let a too-old build in).
Behavior change (disclosed): opening a store whose persisted reader floor is above this build's schema version now REFUSES TO START where it previously opened (and silently wrote). This is the intended fix — the previous open-and-write was the data-corruption footgun. Pre-feature stores (no floor row) and additive-bump stores (floor ≤ this build) are unaffected.
Tests:
tests/storage-reader-floor.test.ts(sqlite hermetic: invariant, atomic floor write, refuse arm, pass arms; mysql hermetic via a typed driver fake: refuse arm, pass arms, stamp+floor in one txn).tests/notice-unified-store.test.tscarries the #5761 regression cases.Relationship to PR #5768
#5768 fixes #5761 alone. This batch PR cherry-picks its two commits (authorship preserved) and adds the #5788 reader floor. When this merges, #5768 is superseded — please close whichever loses. If #5768 merges first, I'll rebase this onto it.