Skip to content

test(storage): close index.db handles before rmrf — fix Windows EBUSY in storage-engine-proofs (#2772 follow-up) - #2781

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-11_engproof-windows-fix
Oct 11, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-11_engproof-windows-fix

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 11, 2026 •

Copy link
Copy Markdown
Owner

Problem

The test (windows-latest, 22) and test (windows-latest, 24) lanes have been red on master since #2772 landed (visible on the #2768 merge run: 4 failures, all in tests/storage-engine-proofs.test.ts). Master pushes skip the test job, so it went unnoticed.

Error: EBUSY: resource busy or locked, unlink 'C:\Users\runner~1\AppData\Local\Temp\bili-engproof-*\index.db'

Root cause

The four failing tests run under BILI_STORAGE_UNIFIED=shadow|full, so every SessionStore they construct opens an index.db handle via the unified shadow. The tests closed those stores with cancelAll() — which only cancels pending timers, it does not close the SQLite connection. The rmrf(tmpdir) cleanup retries EBUSY/EPERM for ~3s and then gives up. POSIX unlink tolerates open handles, which is why the Linux lanes never saw it — same discipline fixed for the storage suites in #2690/#2768 (closeQuietly).

Leak sites:

  • loadFolded() helper: its loader store is discarded right after loadSync, but kept the handle — this alone leaked in all four failing tests (lineage/import-guard/rollback/corrupt legs all call it under shadow|full).
  • End-of-test read-only stores in the import-guard, rollback, and corrupt-index legs.

Fix

  • loadFolded: cancelAll → closeQuietly
  • read-only end-of-test stores: cancelAll → closeQuietly
  • deliberately kept cancelAll at the corrupt-fallback read that is followed by store2.writeNow(...) (mid-test: the store must stay usable; its own closeQuietly already runs at test end)

Test-only change; no product code. Verified locally: storage-engine-proofs 10/10, tsc --noEmit clean.

Tests

  • node --import tsx --test tests/storage-engine-proofs.test.ts — 10/10
  • tsc --noEmit clean
  • windows-latest matrix (this PR)

Fixes the Windows test matrix regression introduced by #2772.


prepared by an agent under the local git identity (ranxianglei), human merge required


Update 2 — the deeper root cause: open() leaked the handle (product fix)

After the hygiene commit, the Windows matrix was down to ONE failure: the corrupt-index leg itself. Root cause is a product bug that #2771's fallback inherits: UnifiedStore.open created the Database handle, then threw on the first pragma/migrate statement (SqliteError: file is not a database) without closing the handle — better-sqlite3 only reads the header on first statement execution, so the constructor had succeeded. On Windows you cannot delete a file a live process holds open, which made the "delete the file to reset the index" recovery advice impossible; the leaked fd is also what kept EPERM-failing the tmpdir cleanup.

 follow-up)

The four engine-proof tests that run under BILI_STORAGE_UNIFIED=shadow|full
leaked their SessionStore shadow handles (cancelAll only cancels timers; it
does not close the SQLite connection), so the rmrf(tmpdir) cleanup retried
EBUSY for ~3s and then failed on windows-latest (Node 22/24). POSIX unlink
tolerates open handles, which is why Linux never saw it.

- loadFolded: cancelAll -> closeQuietly (the loader store is discarded right
  after loadSync; this was the leak shared by all four failing tests)
- end-of-test read-only stores in the import-guard/rollback/corrupt legs:
  cancelAll -> closeQuietly
- kept cancelAll at the corrupt-fallback read that is followed by writeNow
  (mid-test: the store must stay usable; its own closeQuietly follows at the
  end of the test)
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-11_engproof-windows-fix (54f21d3)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3054-pr.2781.d523988

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2781.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei
ranxianglei merged commit 4387bcf into master Oct 11, 2026
12 of 14 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.282-master.4387bcf (rolling master tag).

npm install -g billion-context@0.0.282-master.4387bcf  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

ranxianglei pushed a commit that referenced this pull request Oct 20, 2026
CI auto-fix (issue #3965): two pre-existing pins asserted the OLD
corrupt-index behavior (permanent legacy-only degrade) that this PR
intentionally replaced with quarantine + fresh-open self-heal:

- web-storage-banner (#3040): now asserts effective=full, degraded=false,
  live engine, and the .corrupt-<ts> quarantine artifact with the exact
  corrupt bytes preserved.
- storage-engine-proofs (#2771): re-pointed the #2781 handle-release
  roundtrip to AFTER closeQuietly — under self-heal the store legitimately
  holds a live index.db handle until close, and Windows refuses to move a
  directory containing open handles (the old mid-test placement threw
  there, skipping cleanup and surfacing as a masked EBUSY in rmrf).
  Adds the forensics assertions (one quarantine copy, bytes verbatim).
ranxianglei pushed a commit that referenced this pull request Oct 21, 2026
A NOTADB-class index.db (garbage header) makes the quick_check statement
throw a raw driver error instead of returning verdict rows. That throw
bypassed every db.close() in open(), so the handle created at
openSqliteDatabase stayed open until process exit while the store degraded
to legacy-only. On Windows the next rmrf(dir) then dies with
'EBUSY: resource busy or locked, unlink .../index.db' — exactly what took
out the #2771 (storage-engine-proofs) and #3040 (web-storage-banner)
cleanup on the windows CI shard of this PR. POSIX unlink ignores open
handles, so Linux shards stayed green.

Close the handle before rethrowing (classification unchanged: raw throw
still degrades to legacy-only, it is NOT quarantined — that would break
the #3040 degraded=true contract). Pinned by a new storage-durability
test: /proc/self/fd scan (Linux) + the #2781 rename dance (Windows teeth);
verified red pre-fix, green post-fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant