Repository navigation
[PR] fix: auto-quarantine a corrupt index.db instead of degrading to legacy-only forever - #3965
ranxianglei wants to merge 3 commits into
Conversation
91ef989 to
89d8159
Compare
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4814-pr.3965.ad5885fThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3965.tgz
npm install -g packageInstall instructions are refreshed on each push. |
|
Rebased onto current master (docs-sync red was the pre-#3969 master state; the rebased head carries the regen). Zero conflicts; tsc clean; corruption-quarantine + storage-review-fixes + storage-unified suites green locally. |
89d8159 to
e3d2701
Compare
…no longer degrades permanently) The pre-existing #2771 review test asserted the OLD contract for corruption-class open failures: permanent legacy-only degrade until manual file deletion. #3959 deliberately changes that class to auto-quarantine + fresh reopen, so the expectation flips from replicasAvailable()==false to true, plus quarantine-file assertions. The original safety property (SessionStore construction must NOT throw / legacy load still works) is kept verbatim.
e3d2701 to
11fae9e
Compare
(ai: vllm-qwen/qwen3.8-27b) 冲突已解,分支已 rebase 到当前 master 并 force-push: 八问:
判定:冲突闭环,PR 回到可合并状态,等 CI 对新 head 跑绿即可合入。 |
CI auto-fix (issue #3965): two pre-existing pins asserted the OLD corrupt-index behavior (permanent legacy-only degrade) that this PR intentionally replaced with quarantine + fresh-open self-heal: - web-storage-banner (#3040): now asserts effective=full, degraded=false, live engine, and the .corrupt-<ts> quarantine artifact with the exact corrupt bytes preserved. - storage-engine-proofs (#2771): re-pointed the #2781 handle-release roundtrip to AFTER closeQuietly — under self-heal the store legitimately holds a live index.db handle until close, and Windows refuses to move a directory containing open handles (the old mid-test placement threw there, skipping cleanup and surfacing as a masked EBUSY in rmrf). Adds the forensics assertions (one quarantine copy, bytes verbatim).
(ai: vllm-qwen/qwen3.8-27b) CI auto-fix done (attempt 1/2): pushed 1158918 to the PR branch — two stale test pins that this PR's intentional behavior change broke are now realigned to the new contract. Both pass on CI on every lane they ran, including Windows. What was actually red: your rebase already fixed docs-sync. The remaining failures split into two groups:
八问
判定:lane 内可修项已全部修完并在 CI 验证(ubuntu-22-1、windows-22-1 两条此前红的 lane 现已全绿);PR 整体转绿取决于 master 侧的 #3769 flake 收敛(#4015 线程进行中)或红 lane 重跑运气。按 auto-fix lane 约定停在这里,不再消耗第 2 次尝试;合并仍归人。 |
Closes #3959
Family: #3788 (new sub-mechanism entry: corrupt index.db file — engine healthy)
Behavior change (old -> new)
OLD: when
index.dbfailed to open with a genuine corruption error (database disk image is malformed/file is not a database), the sidecar degraded to legacy-only on every boot until an operator manually deleted the file ("delete the file to reset the index"). Field report #3959: one corrupt db pinned the machine to legacy-only across repeated restarts.NEW: on a corruption-class open failure the dead file is QUARANTINED (renamed in place to
index.db.corrupt-<timestamp>, never deleted, kept for forensics) and a fresh index opens immediately. The legacy JSON session files remain the source of truth in both shadow and full mode (full keeps writing them), so sessions re-ingest as they are saved/loaded — nothing is lost. The warn log now names the quarantine path.Unchanged by design:
Mechanism
src/storage/driver.ts: new exportedisCorruptIndexError(err)— matches SQLITE_CORRUPT/SQLITE_NOTADB codes or their field messages only. Deliberately narrow (see doc comment).src/storage/shadow.ts:quarantineCorruptIndex()renames index.db + its -wal/-shm/-journal sidecars under the.corrupt-<ts>suffix (same pattern as persist.ts [bug] 会话状态写入缺 fsync:内核崩溃后留下 size 正常但 VDL=0 的全零状态文件,loadSync 永久抛错 → 502 acp-proxy failure 且不自愈(含 7 处 tmp+rename 写路径盘点) #3157; walkers match *.json so they ignore it). A stale WAL left behind cannot poison the fresh db (verified empirically: node:sqlite resets an orphaned unclean-exit WAL, integrity_check stays ok).maybeOpenretries the open once after quarantine; if even that fails it falls back to today's exact degrade path.No config surface change. No kernel change. No wire-shape change (log wording only).
Tests
New hermetic suite
tests/storage-corrupt-index-selfheal.test.ts(both directions pinned, #3617 discipline):Existing test updated (intentional, per the behavior change above):
tests/storage-review-fixes.test.ts"review 3" (#2771) pinned the OLD contract for corruption-class errors — permanent legacy-only degrade until manual delete. It now pins the self-heal contract (exactly one quarantined copy, fresh index present, sidecar available, no re-quarantine on the next process); its original safety assertions (SessionStore construction must not throw, legacy load still succeeds) are kept verbatim. The non-corruption degrade path stays pinned by storage-cantopen-hint + the negative cases above.Local targeted runs green on the rebased head: the new suite + storage-review-fixes + the whole storage-subsystem set (146 tests) + adjacent suites; typecheck clean. Full-suite verdict = CI on this head.
Rebased onto current master after the MySQL-backend step-3 merges (#3955/#3966) landed in
shadow.ts; conflict resolved by re-landing the four surgical changes on the new structure (no semantic interaction — the mysql async path does not touch the sqlite open branch).