Repository navigation
fix(serve): adopt an existing index on POST /repos instead of force-rebuilding - #357
Merged
Merged
Conversation
…ebuilding Registering a repo through the running serve (codesearch index add, the git-hook path, or a script adding many repos) unconditionally ran the destructive force reindex. A repo that already had a valid .codesearch.db - for example one indexed directly before being registered - lost its index and had to re-embed every file, serialized behind CODESEARCH_INDEX_JOBS. With a batch of registrations that looked like no progress at all for a long time. The local 'codesearch index add' path already reuses an existing index; the serve path now matches it. Adopt when metadata.json records a model and the request is not changing it, and run the gated incremental refresh instead of the clear-and-rebuild. A new index (no recorded model) and an explicit model change still take the force path, because changing the model invalidates every stored vector. Tests: decision table for the adopt rule, plus an end-to-end add that seeds an unchanged tracked file and asserts its chunk survives (the old force path clears it).
flupkede
approved these changes
Oct 20, 2026
flupkede
left a comment
Owner
There was a problem hiding this comment.
Reviewed in depth (see session notes): adoption decision table correct, force path preserved for legacy/no-model/mismatch, tests pin both paths, changelog follows convention. Safe to merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Registering a repo through a running serve (
POST /repos, which is whatcodesearch index adddelegates to) unconditionally ran the destructive force reindex. If the repo already had a valid.codesearch.db— for example one built directly withcodesearch index <path>before it was registered — the registration cleared the vector store, FTS index and file metadata, then re-embedded every file.For a batch of registrations this is expensive and confusing: each add looks like it starts another background job, and the heavy-job gate (
CODESEARCH_INDEX_JOBS) serializes the rebuilds, so the visible result is a long period with no committed progress while already-indexed work is thrown away and redone. The localcodesearch index addpath already reuses an existing index; the serve path did not.Fix
POST /reposnow adopts a usable index already on disk instead of force-rebuilding it:metadata.jsonrecords a model and the request is not changing it. The background task runs the gated incremental refresh, so unchanged files stay in the index and only real changes are re-chunked/re-embedded.modeldiffers from the recorded one. A model change is the one case that genuinely invalidates every stored vector (dimensions and vector space change).The adoption decision is a small helper (
should_adopt_existing_index) next to the existingresolve_add_repo_modelprecedence rule.Tests
should_adopt_existing_index_precedence: decision table — no recorded model → build; recorded model with no/matching override → adopt; differing override → build.add_repo_handler_adopts_existing_index_instead_of_rebuilding: end-to-end fixture with a tracked unchanged file and one seeded chunk. The add returns 202 and the chunk/file survive the background refresh; the old force path clears the store first, so the assertion fails without the fix.add_repo_handler_*tests (brand-new repo, serve-default model, recorded model kept) still pass, as does the full QC gate.Notes
The add task still spawns one background task per registration (that is the "N jobs" in the log), but with adoption each task is an incremental refresh that normally returns "index is up to date" almost immediately rather than a full rebuild.
A separately observed "no committed progress" symptom on an old binary is the pre-v1.6.0 file-metadata persistence issue fixed by 241e641 ("persist file metadata per index batch, atomically"); this change addresses the registration path that discarded existing indexes in the first place.