Repository navigation
[PR] refactor(storage): extract StorageBackend contract behind UnifiedStore (#3388 Phase A) - #3432
ranxianglei wants to merge 3 commits into
Conversation
Phase A of #3388 (storage backends beyond SQLite: MySQL/Greenplum/extensible). Pulls UnifiedStore's synchronous instance-method surface into a StorageBackend interface (src/storage/backend.ts), makes UnifiedStore implement it, and widens export.ts renderSessionJsonlFromTables to accept the interface so the contract has a genuine second consumer across a module boundary. Pure refactor: zero behavior change, no signature or caller changes. The interface is intentionally synchronous (mirrors today's store) and widens to async in Phase C alongside the first networked backend. This is the seam that lets MySQL/Greenplum backends be added without touching consumers (persist/SessionStore/export/web-list/GC/runtime-status). Refs #3388
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4049-pr.3432.15f019dThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3432.tgz
npm install -g packageInstall instructions are refreshed on each push. |
reference/architecture.md had no entry for the unified storage layer (#2671): the whole src/storage/ directory was unlisted and the Persistence orientation bullet described only persist.ts + session.ts. Add the src/storage/ section (all 9 files incl. the StorageBackend contract from #3388 Phase A) and correct the Persistence bullet to point at the unified store + contract. Doc-only; no code or behavior change.
(ai: vllm-qwen/qwen3.8-27b) 收到,开始 review PR #3432(#3388 Phase A)。 查重: 不重复 —— 开着的 issue/PR 里没有其它条目做「StorageBackend 契约提取」;同区域的在途工作(#3419 并发覆盖、#3423 shadow digest 漂移、#3348 sqlite bootstrap 超时)机制与症状都不同,不构成重复。父 issue 只有 #3388。 接下来按规范走:方向审(对照 #3388)、回归清点、代码/测试/diff 审查,结论用八问体例回帖。 |
…not-throw contract Review follow-up on #3432 Phase A: the new contract documents close() as best-effort ("implementations must not throw into the caller") and every call site already defends (UnifiedShadow.close swallows; persist flushAll/ closeQuietly treat it as never-throwing; export degrades to the legacy renderer), but the SQLite impl only guarded the WAL checkpoint — the final db handle release could still throw. Same isolation pattern as the existing checkpoint guard: warn-log instead of propagate.
(ai: vllm-qwen/qwen3.8-27b) Review 结论(#3388 Phase A)独立审完(方向、回归、代码、测试、diff 五关),并已在 PR 分支补了一个小修 commit ① 方向审(对照 #3388):#3388 要的是 sqlite 之外的存储后端 + 扩展性。Phase A 选「操作级接缝而非 SQL 方言 shim」是对的—— ② 回归清点(逐项既有行为):
③ 代码审查发现 + 已修:契约文档说 ④ 测试有效性:PR 未新增测试——对纯类型提取可接受: ⑤/⑤b diff 与代码质量:4 文件 115 行,每行都服务 PR 目的,无空行/重排/lockfile/version 污染;命名达意、无死代码、注释讲为什么、风格与仓库一致、零新依赖。通过。 结构化八问
判定:可合 —— 方向成立、回归为零、验证齐备(tsc + 定向 148 项 + 干净 merge);CI 全量绿后即可由你合并。 |
|
Superseded by the re-scoped two-step plan the owner set on #3388: (1) redo the StorageBackend seam against current master — this PR's interface list predates schema v6/v7 (reapBlocks / archive / restore / gc / media hydration / incremental pointers are missing from the contract), (2) pluggable backends with sqlite itself as the default registered backend, MySQL as the first real one. Fresh PR will carry the full operation surface; closing this one rather than force-rebasing a contract that is incomplete by construction. |
What
Phase A of #3388 — storage backends beyond SQLite (MySQL / Greenplum / extensible), aimed at separating the proxy layer from the DB service so bili can run distributed.
This PR extracts
UnifiedStore's synchronous instance-method surface into a newStorageBackendinterface (src/storage/backend.ts), makesUnifiedStoreimplement it, and widensexport.tsrenderSessionJsonlFromTables(store: ...)fromUnifiedStoretoStorageBackendso the contract has a genuine second consumer across a module boundary (not a decorative type alias).Design decisions
ingestSessioncommit CAS refcount + authoritative-set replace + watermark atomically in one transaction. That atomicity cannot survive dialect translation, so each backend implements the same operations natively — SQLite keeps its proven SQL, MySQL/Greenplum get fresh implementations against the same contract.asyncin Phase C alongside the first networked backend (pg / mysql2 are Promise-based). Keeping Phase A purely synchronous means zero signature/caller changes — a true pure refactor.UnifiedStore.open(path)and the SQLite-specificengine/engineSourcegetters are deliberately left out of the interface — they are implementation details; a networked backend selects its target via the Phase-B config knob, not a file path.Behavior change
None. Pure refactor. No runtime path changed, no config added, no wire / persistence-format change.
UnifiedStoreis the same class now carrying animplementsclause;renderSessionJsonlFromTablesaccepts a strictly wider type while every existing call still passes aUnifiedStore.Verification
384bdf6f4, merge of 定时全量失败自动开 issue,转绿自动关 #3428). The three touched files were byte-identical across the 34-commit gap, so no conflicts.tsc --noEmit -p tsconfig.jsonclean. Theimplementsclause forces every method signature to match the interface.tests/*storage*.test.ts,tests/export-*.test.ts,tests/unified-acp-invariants.test.ts,tests/cli-export.test.ts) → 148 tests, 147 pass, 0 fail, 1 skip. Full-suite verdict deferred to CI on the pushed head.Scope / what's next (all tracked in #3388)
StorageBackendcontract +UnifiedStoreimplements it.storage.backendconfig knob (defaultsqlite) — owner-gated config surface, reported in-thread.Refs #3388