Skip to content

perf(sidecar): skip re-embedding unchanged documents; fire-and-forget initial reindex - #19

Open
wu546526 wants to merge 1 commit into
chendpoc:mainfrom
wu546526:fix/reindex-fingerprint-skip
Open

wu546526 wants to merge 1 commit into
chendpoc:mainfrom
wu546526:fix/reindex-fingerprint-skip

Conversation

@wu546526

Copy link
Copy Markdown

Summary

session_start calls reindex on every attach, and chunkReindex re-embeds the entire memory file unconditionally. With a remote embedder (ollama bge-m3, 128 chunks) that measured 7.3-7.8s per round — the dominant cost of every session attach / new chat.

  • handleReindex now computes a content fingerprint (sha256 over sorted chunk ids + contents + embedder identity) and short-circuits on a match, returning skipped: "fingerprint_match"
  • Fingerprint is persisted in the existing meta table; embedder model/provider/dim changes still trigger a full rebuild (and clearChunksIfEmbeddingMismatch stays as-is)
  • The initial reindexScheduler.runNow() in bootstrapSidecar becomes fire-and-forget so session_start never blocks on a full re-embed (preflight already falls back to MEMORY.md while the index rebuilds)
  • Empty-document reindex never fingerprint-skips (it is the explicit clear path)

Measurements

before after
repeated reindex (same docs, 128 chunks, ollama bge-m3) 7.3-7.8s ~2ms
content changed 7.3-7.8s 7.3s (correctly rebuilds), then ~2ms
session startup attributable to pi-memory ~7-8s <1s

Test plan

  • New test/reindexFingerprint.test.ts (6 tests): fingerprint order-insensitivity, skip-on-match, rebuild-on-change, empty-set never skips, data still queryable after a skipped round.
  • npx vitest run: 203 passed (pre-existing scheduler/status failures unchanged — they depend on the local launchd/home environment and fail on a clean checkout too).
  • npx tsc --noEmit clean.

This came out of investigating slow session attach in a pi-web-ui deployment (raw-WS attach timing dropped 7.7s → 1.0s with this fix plus a per-cwd extension cache fix on the pi side).

… initial reindex

session_start calls reindex on every attach. With a remote embedder
(ollama bge-m3, 128 chunks) each call re-embedded the entire memory
file — 7.3-7.8s per round on this machine — because chunkReindex
embeds unconditionally and nothing tracks what was last indexed.

- Add a content fingerprint (sha256 of sorted ids+contents + embedder
  identity) persisted in the meta table; handleReindex short-circuits
  on a match and returns skipped: "fingerprint_match"
- Expose get/setLastReindexFingerprint on VecStore (db is private)
- Make the initial reindexScheduler.runNow() in bootstrapSidecar
  fire-and-forget: session_start must not block on a full re-embed;
  preflight already falls back to MEMORY.md while the index rebuilds
- Empty-document reindex never fingerprint-skips (explicit clear path)

Measured: repeated reindex ~7.3-7.8s -> ~2ms; session startup cost
attributable to pi-memory drops by ~7s per attach / new chat.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant