feat(storage): vectordb persistence + FTS5 opt-in + 24h search clamp

[aksOps]

Summary

Storage rebalance initiative to shrink hot-path disk footprint at 150 services × 7 days while keeping triage latency intact. Five logically distinct phases shipped together because they share files and only make sense as a unit.

• PR1 — FTS5 disable + reclaim + 24h search clamp. SQLite FTS5 (logs_fts + porter+unicode61 + bm25 + sync triggers) is now opt-in via LOG_FTS_ENABLED. Default off — saves 30–40% disk at scale because the FTS table doubles every indexed log body. New POST /api/admin/drop_fts reclaims the table + indexes (refused while the flag is on). search_logs MCP tool and /api/logs?q= clamped to last 24h via storage.ClampSearchWindowTo24h to bound the LIKE-fallback worst case.
• PR2 — vectordb snapshot wire format. Versioned VDB1 magic + uint32 version + CRC32-IEEE + gob payload. Magic/version mismatch or CRC failure rejects the file so the loader falls back cleanly to a full DB rebuild. Atomic write via tmp+sync+rename, mode 0o600, EXDEV fallback for cross-device data dirs. LogVector.Vec exported for gob; Index gains lastIndexedID watermark.
• PR3 — DB tail-replay. vectordb.ReplaySource + ReplayRow keep the package free of storage imports. Repository.LogsForVectorReplay is WHERE id > ? AND severity IN (...) ORDER BY id ASC, paged. Index.ReplayFromDB walks pages from LastIndexedID() so a snapshot plus tail replay reconstructs the post-snapshot delta with no double-indexing. main.go wires a vectorReplayAdapter to project storage.Log → vectordb.ReplayRow.
• PR4 — periodic snapshot loop + shutdown hook. Index.SnapshotLoop(ctx, path, interval) ticks periodically and fires one final write on ctx.Done so SIGTERM captures the maximum in-memory state. Shutdown step 3a (snapCancel + 5s wait on snapDone right after graphRAG.Stop()) lands the final snapshot before DB close.
• PR5 — observability + docs. Five new metrics: otelcontext_vectordb_snapshot_writes_total{result}, snapshot_duration_seconds, snapshot_size_bytes, snapshot_load_total{result}, replay_logs_total. Index.SetSnapshotObserver is the wiring-layer hook so vectordb stays free of telemetry imports. CLAUDE.md refreshed: storage architecture row for vectordb persistence, FTS5 reframed as opt-in, 24h clamp documented, three new env vars (VECTOR_INDEX_SNAPSHOT_PATH, VECTOR_INDEX_SNAPSHOT_INTERVAL, LOG_FTS_ENABLED).

Plan and design notes: docs/superpowers/specs/2026-04-30-storage-rebalance-plan.md.

Test plan

• go vet ./... — clean
• go test ./... — 516 pass / 27 packages
• go build . — 46 MB binary
• CI pipeline green (security stack + SonarCloud + lint)
• Smoke: start ./otelcontext with LOG_FTS_ENABLED=false (default) on an existing SQLite data dir; verify daemon starts, /api/logs?q=err returns 400 when window straddles >24h ago, vectordb snapshot file appears at data/vectordb.snapshot after 5m
• Smoke: kill -TERM the daemon, restart, verify 🔍 Vector index: loaded snapshot log + otelcontext_vectordb_snapshot_load_total{result="success"} increment
• Smoke: curl -X POST http://localhost:8080/api/admin/drop_fts while flag is off — returns 200 with reclaimed_bytes body; while flag is on — returns 405

🤖 Generated with Claude Code

[sonarqubecloud]

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.2% Duplication on New Code

See analysis details on SonarQube Cloud