You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(core): import stores first and embeds later (deferred embedding, controlled repair --reembed) #1608
uteke import embeds every row while it imports (import_export.rs). With a local ONNX model that is CPU-heavy: a large file keeps the machine busy for the whole import, and a backend that is down or rate-limited stops the import at that row (the rows already imported stay; importing again skips them through the duplicate check). remember already has the opposite policy: it stores the row keyword-only and uteke repair --reembed adds the vector later.
Uteke Cloud does the same split: a write returns before the vector exists and an outbox worker embeds it afterwards.
Proposal (owner direction, 2026-10-30: store first, embed later; next update, not 0.22.0)
import stores the rows without a vector (keyword-searchable at once) and reports how many still need one.
Optional: an embed worker inside uteke-serve that drains the backlog in the background at a low priority, like the cloud outbox worker.
Open questions
Default or flag? A flag (import --defer-embed) keeps today's behaviour for everyone, including the LongMemEval harness, which imports with an embedder and must not change (benchmark invariant). Making it the default is a behaviour change for CORE POST /import.
Duplicate check without vectors. The cosine duplicate gate (>= 0.95) needs a vector. Without one, dedup falls back to an exact content hash (source_hash exists), and near-duplicates are only caught when the vector is added later. Decide whether that is acceptable.
Contradiction and edge wiring at import also read the vector; they would run when the row is embedded.
Backlog visibility:uteke stats / verify should show "N memories without a vector".
Server worker vs CLI only for the first step.
CORE impact: the response of POST /import and the meaning of "imported" (uteke-cloud contract issue first).
Related
#1606 (a store with no embedding backend imports keyword-only), #1592 (chunk vectors are re-embedded by repair --reembed), #1607 (CHANGELOG wording about embedding failures during import).
Activity