Repository navigation
Storage review fixes: authoritative refs/blocks, corrupt-index guard, BEGIN IMMEDIATE, restore 409 + CCR sidecar (#2690 review) - #2768
Conversation
…nt, corrupt-index guard, BEGIN IMMEDIATE, restore 409 + CCR sidecar Owner review (blocking findings, issuecomment-6108173553): 1. refs rows: DELETE-then-INSERT per ingest — a renumbered byRaw map (native compaction drops a front key) no longer trips the (session_id, ord) PK; ghost rows cleared, ords compact. 2. blocks rows: authoritative replacement — destroyed-substrate rows are deleted (CAS refcounts released); #2695 zombies cannot resurrect on later saves. 3. conversations table dropped before schema v1 ships (zero writers). 4. corrupt index.db: guarded open — warn + legacy-only, never bricks the proxy (3-strike self-disable stays as backstop). 5. BEGIN IMMEDIATE at top-level transactions (multi-process SQLITE_BUSY_SNAPSHOT, #394 topology). 6. restoreReplica: refuses live-id overwrite (409 via admin mapping); freezes the disk content-store envelope into the checkpoint and materializes the sidecar for the restored id (CCR originals survive the next live save — buildRecord drops forkContentStore). 7. export: listSessions/exportSession close one-shot handles (Windows EPERM regression); --jsonl degrades to legacy renderer on unreadable/encrypted index. 8. PLUGIN.md registration example updated to fork protocol v2. Tests: tests/storage-review-fixes.test.ts (4) + full storage family green locally (unified 13, replicas 4, corpus 9, export-jsonl 4, cli-export 2).
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3049-pr.2768.e0dd559This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2768.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 ## PR #2768 review — 可合(待新 head CI 全绿) 审原 issue(方向关):owner 对 #2690 的三遍 review 给出 7 项 blocking + 若干 pre-merge 纪律项。本 PR 逐条对应、scope 严格锁在新存储面(11 文件全部在 storage surface + docs + tests,无蔓延),且没有一项是治标:fix 1 比建议方案更强(整表替换而非 逐项核实(代码级):
回归审查(既有行为清单):knob-off 默认路径零触碰(maybeOpen 早退 null、不开句柄)✓;两种压缩模式的 wire/rebuild/preflight/nudge 完全未涉及(纯存储面)✓;unified lane 既有行为(digest reconcile、organic import、GC parity、加密 opt-out、3-strike、replica 往返)由 17 个既有存储测试覆盖,本地全绿 ✓。高危模式逐项过:无 fail-fast 变吞错(corrupt-guard 是显式 warn 降级而非静默吞)、无阈值/时序漂移、无输出格式变化。行为变更披露(旧→新+原因):① restore 到已存在 id:静默覆盖 → 409 拒绝(默认 fresh-id 路径不变);② corrupt index.db:进程启动即挂 → warn + legacy-only 降级(可用性修复);③ --jsonl(index 不可读):报错 → 降级 legacy-tail renderer。残留缺口(不拦):restore 的 exists-check 走 测试有效性:4 个新测试是真测试——我做了负向对照:把 小问题直接修(已在 PR 分支补两个独立标注 commit,push 至新 head
遗留待 owner 拍板(源自 #2690 review pre-merge 清单,本 PR 未触及): 结构化八问
判定:可合 —— 待新 head |
|
🔗 Heads-up from the #2771 thread: sibling PR #2772 ( |
…review-fixes # Conflicts: # src/storage/shadow.ts
…ranxianglei/billion-context into 2026-10-11_storage-review-fixes
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.279-master.da6315e # this exact build
npm install -g billion-context@master # always the newest merged build
|
Addresses every blocking finding from the owner review of #2690 (issuecomment-6108173553). Scope is strictly the new storage surface; the knob-off default path is untouched (cleared twice by independent review).
Blocking findings → fixes
(session_id, ord)PK conflict when the byRaw map renumbers (native compaction drops a front key) → 3-strike self-disableindex.dbbricks the proxy (unguardedUnifiedStore.open)[storage] unified sidecar FAILED to open (…) — staying legacy-only; delete or repair index.db to re-enable, shadow null, legacy keeps working; 3-strike backstop unchangedlistSessionsleaks one SQLite connection per call (re-breaks the Windows EPERM fix)try/finally store.closeQuietly()insrc/export.tsBEGIN IMMEDIATEat top level (better-sqlite3.transaction(fn).immediate(), node:sqliteBEGIN IMMEDIATE); savepoints unchanged for nestingrestoreReplicawith a caller-chosen id silently overwrites a live session; restore loses the content-store envelope → CCR can't fetch originals after rollbackalready exists→ HTTP 409;createReplicafreezes the disk content-store envelope into the checkpoint blob;restoreReplicamaterializes the sidecar for the restored id (survives the next live save, which dropsforkContentStore)Also from the review notes (non-blocking but cheap):
conversationstable dropped before schema v1 ships (zero writers).--jsonldegrades to the legacy-tail renderer on an unreadable index (guarded open) and is gated onBILI_ENCRYPTION_KEY(index.db holds cleartext).exportSessioncloses its one-shot handle.BILI_STORAGE_UNIFIEDrow documents the shadow/full disk expectation (≈2× unique-content bytes, no auto-VACUUM).Tests
New
tests/storage-review-fixes.test.ts(hermetic pins, same harness as storage-replicas):/already exists/) and the restored id round-trips CCR originals through the real load path (loadSync+loadContentStoreon a fresh handle)Full storage family green locally: unified 13, replicas 4, corpus 9, export-jsonl 4, cli-export 2; tsc clean.
prepared by an agent under the local git identity (ranxianglei), human merge required