Skip to content

1.3.2: storage bloat remediation, doctor --fix cross-project repair, and patch bump - #18

Merged
LucasArmstrong merged 1 commit into
mainfrom
v1.3.2
Sep 29, 2026
Merged

LucasArmstrong merged 1 commit into
mainfrom
v1.3.2

Conversation

@LucasArmstrong

Copy link
Copy Markdown
Member

Summary

Release v1.3.2 resolves a critical LanceDB storage bloat issue where stale indices and unpruned version manifests accumulated on disk, growing directories to upwards of 6.4 GB (with 98%+ waste). Across workspace databases, disk usage was reduced from ~7.5 GB to ~19 MB (99.7% reclamation) while keeping 100% of visual states intact.

Key Changes

  • Storage Health Engine (src/core/storage-health.ts): Instant non-locking audit of data fragments, stale indices, old version manifests, and transaction logs.
  • doctor --fix: Adds automated local database compaction, stale index cleanup, and version manifest pruning to the doctor command.
  • doctor-global --fix & --scan <dir>: Added cross-project storage bloat audit and repair across all registered/discovered projects in a single sweep.
  • Index Deduplication: Updated src/core/storage.ts to check table.listIndices() before attempting to recreate scalar or vector indexes on startup.
  • LanceDB Compaction & Cleanup: Enhanced storage.optimize() with cleanupOlderThan and deleteUnverified: true to purge unreferenced historical files.
  • Background Periodic Compaction: Added periodic compaction every 6 eviction sweeps (~30 min) in src/core/eviction.ts and extended shutdown timeout to 5,000ms.
  • Size Limit: Lowered default MAX_LANCEDB_SIZE_MB from 1,000 MB to 500 MB.
  • Test Suite: Added tests/unit/storage-health.test.ts. All 80 test files (394 tests) pass with full matrix and CI validation.

@LucasArmstrong
LucasArmstrong merged commit 45adb37 into main Sep 29, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant