Repository navigation
[PR] storage: StorageBackend contract (step 1) — sqlite through the registry, mysql a registered stub (#3931) - #3932
Conversation
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复 — 这是 #3931 的唯一实现 PR;open+closed 里检索 标签: pr(既有) + scope/storage + p3 — 存储层抽象缝(step 1),零行为变更的架构铺路,非缺陷、不涉具体客户端,严重度取最低档。 状态: CI gate/version-guard 已绿,test 各 lane 还在排队;我已完成分支拉取,正在按规范做独立 review(原 issue 方向 → 回归清点 → 代码/测试/diff)。结论稍后按八问体例给出。 |
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4722-pr.3932.b3cbc12This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3932.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…#3932 review) resolveBackendName collapses unregistered names to sqlite before the boot notice consults it, so a typo'd selection (env or file) fell back silently while CONFIGURATION.md promised a notice; the unknown-name branch of backendSelectionNotice was dead production code. The notice now judges the raw winning tier via selectedBackendRaw (resolveBackendName builds on it, behavior-identical), and treats "" as unset so the mapper is total over its string domain. Behavior change: unknown names log ONE warn line at boot (previously silent); effective backend unchanged (sqlite).
…3932 review) The --jsonl path hardcoded new SqliteBackend(UnifiedStore.open(index)), so a deliberate mysql selection silently exported from sqlite instead of failing with the documented stub error that CONFIGURATION.md and #3931 promise. The selection is now enforced before the guarded open: registered-but-unshipped names go through openBackend and throw their documented line before any file access; typo'd names warn once and read sqlite. The guarded try/catch is untouched, so the corrupt-index legacy-tail degrade contract (#2690) on the sqlite path is byte-for-byte preserved.
(ai: vllm-qwen/qwen3.8-27b) Review complete — found 2 real defects in the selection surface, fixed directly on the PR branch and pushed (two labeled commits, additive, no rebase-replace). Independent review (re-derived from the diff, not read off the issue)
Defects found & fixed (pushed as labeled commits)
VerificationLocal targeted runs (full-suite verdict belongs to CI, per local-testing discipline): 15/15 green — storage-backend-knob 4/4 (incl. new row), storage-backend-contract 6/6, export-jsonl 5/5 (incl. new row); 结构化八问
判定:两个缺陷已就地修复并推送( |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) CI 终态跟进(head CI 主工作流:全绿。 19/20 通过 + 那个红是什么、为什么判为 flake:
另一处红(不属本 PR): 八问增量(相对上条回复):代码零新增变更;回归结论不变(Windows 分片 flake 经重跑证伪,不构成回归;既有行为审计清单见上条);缓存不涉及;设计方向不变;配置面不变( 判定更新:CI 门禁已全部转绿,本 PR 具备合并条件;合并动作本身留给人工。 |
…he registry, mysql a registered stub (#3931) - src/storage/backend.ts: operation-level async contract (~40 ops: ingest, reads, session_ext/kv, watermark/pointers, reap journal, archive/gc, checkpoints, summaries, diagnostics) — the seam is operations, not SQL dialects, so per-backend transaction semantics survive. SqliteBackend wraps the existing UnifiedStore (sync under the hood, Promise surface). - registry: openBackend()/listBackendNames(); 'mysql' validates but throws the documented step-2 error at open (mysql2 never loaded in step 1); resolveBackendName (env BILI_STORAGE_BACKEND > storage.backend > sqlite) + backendSelectionNotice keep non-sqlite boots on sqlite with ONE warn. - shadow.maybeOpen takes the file backend tier and logs the notice. - export.ts jsonl renderer consumes the contract (seam proof); guarded open wraps UnifiedStore in the adapter — behavior pinned by existing export tests unchanged. - tests: storage-backend-contract (runs against ANY backend via the registry — reused verbatim for mysql in step 2) + knob matrix. - docs: storage.backend seeds (process.yaml + env-only.yaml), regenerated CONFIGURATION en/zh; CHANGELOG. Zero schema change, zero behavior change for sqlite users (#3931).
…#3932 review) resolveBackendName collapses unregistered names to sqlite before the boot notice consults it, so a typo'd selection (env or file) fell back silently while CONFIGURATION.md promised a notice; the unknown-name branch of backendSelectionNotice was dead production code. The notice now judges the raw winning tier via selectedBackendRaw (resolveBackendName builds on it, behavior-identical), and treats "" as unset so the mapper is total over its string domain. Behavior change: unknown names log ONE warn line at boot (previously silent); effective backend unchanged (sqlite).
…3932 review) The --jsonl path hardcoded new SqliteBackend(UnifiedStore.open(index)), so a deliberate mysql selection silently exported from sqlite instead of failing with the documented stub error that CONFIGURATION.md and #3931 promise. The selection is now enforced before the guarded open: registered-but-unshipped names go through openBackend and throw their documented line before any file access; typo'd names warn once and read sqlite. The guarded try/catch is untouched, so the corrupt-index legacy-tail degrade contract (#2690) on the sqlite path is byte-for-byte preserved.
5e3a52c to
5d665fa
Compare
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 分支已更新到最新 master(接上条):上一条回复后 master 前进了 56 个 commit(包含触碰本 PR 重叠文件的变更: |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.667-master.45ca4b8 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Implements #3931 — step 1 of the #3388 pluggable-storage plan, scoped by the owner: land the abstraction seam, keep every test green, re-wire sqlite through it, leave MySQL for step 2.
What
StorageBackendcontract (src/storage/backend.ts) — operation-level and async: ~40 operations (ingest, message/block/ref reads, session_ext/kv, ingest watermark + incremental pointers, the Reaped blocks resurrect after restart in full mode: stale index row outlives the #2695 reap and poisons the JSON envelope #3841 reap journal, archive/restore/gc, checkpoints/replicas, summaries, refcount diagnostics, close). The seam is operations, not SQL dialects — a dialect shim cannot preserveingestSession's single-transaction CAS refcount discipline, so each backend implements the same operations natively. The interface is async because step-2 network backends are; the sqlite implementation stays synchronous under the hood with aPromise.resolvesurface (hot-path cost: one microtask).SqliteBackendwraps the existingUnifiedStore;openBackend("sqlite", …)resolves via the registry. Zero schema change (stays v7), zero migration, zero behavior change.mysqlis a registered stub — the name validates (configs won't bounce between "unknown" and "unimplemented" when step 2 lands), but opening it throws one documented line; a non-sqlite selection at lane boot logs a single notice and stays on sqlite (boots never crash);mysql2is never loaded.--jsonlexport renderer now consumes the contract end-to-end (its behavior is pinned by the existing export tests, unmodified);tests/storage-backend-contract.test.tsruns the same assertions against ANY backend instance via the registry — step 2 runs it unchanged against mysql.storage.backendfile key +BILI_STORAGE_BACKENDenv (defaultsqlite), documented via the yaml seeds → regenerated CONFIGURATION en/zh.Explicitly not here (step 2, #3388)
Verification
Stakes first (
storage-backend-contract+storage-backend-knob, red on the missing module before implementation). Green locally: contract 6/6, knob 3/3, export-jsonl 4/4 (one mechanical adaptation: the renderer is async now), storage-unified 14/14, archive-gc 4/4, reap-journal 2/2, golden-storage-render 5/5, storage-ext 5/5 — plus the full suite run attached to CI. tsc clean (therenderNudgeText4-arg errors on current master are stale-kernel-dist artifacts, not from this branch).Fixes #3931.