Repository navigation
feat(search): collection rows join the page search as guests on the page scale - #107
Merged
Merged
Conversation
Rows live in a dedicated rowDocs map on the one SearchIndex; corpus statistics (df, vocab, avgLen) stay page-only, so page scores are computed exactly as before. rankRows() scores row documents with the same idf/bm25/FIELD_WEIGHT and coverage/phrase rules, runSearch merges the two lists into one ranked result (pages win ties) and exposes SearchHit.row with collection/row identity and a direct-open href.
allRows() reads every row uncapped (query() keeps its 500-row limit for the UI); the search index needs them all. whenCollectionsOpen() runs a callback as soon as the singleton store is open without ever opening it itself, so boot-time row indexing can attach to the first UI open without creating a database as a side effect.
Mdwiki.start() opens an existing collections DB eagerly and indexes every collection; without the file it attaches via whenCollectionsOpen. store.subscribe marks collections dirty and a trailing 250 ms timer coalesces bursts into one re-read per collection. The operation is always 'derive collection X from the current DB state', never a delta, so missed events, bursts and crashes converge; a vanished collection (404) removes its rows. collectionRowsForIndex maps allRows() via the plainCellText rule extracted from database-block.ts, so value-to-text exists once. test/helpers pins COLLECTIONS_DB to the temp space: boot time opening must never reach the live collections database.
The results list keys by row id for row hits, renders them as a link to the direct-open href with a collection-name chip and the snippet, reads the count line as 'X pages · Y rows' and keeps layer/facet counters on page hits only.
…he MCP search tool Review findings on the implementation: the search page printed result.total as the page count although total now includes row hits, so it double-counted; it now prints total minus rows as pages. The MCP search tool description gains the sentence about row hits, the row object and the page-only tag and folder queries, which the feature owed the machine surface.
…opied instance The walk copies space and database into a temp dir, starts the built server on a free port and proves over REST that a real value is searchable right after boot, that pages and rows share one ranked list, that create, edit and delete reach the index without a restart, and that a #tag query stays page-only. Every write stays below the temp dir, so the source instance is never touched.
… live instance Terminal capture of scripts/search-rows-walk.sh on 2026-11-15 against a copy of the live instance: the phone query returns one page and four row hits in one ranked list, the scratch row is searchable a second after its create, the edit replaces the old value in the index, the delete removes it and the #tag query returns no rows, all with one unchanged server process.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #84: the wiki search covers collection rows next to pages, live, without a restart.
Index. Rows are a separate document set inside the one
SearchIndex(rowDocs), built fromproperty: textlines with the same tokenize/stem/compound machinery as pages.rankRowsreuses the page idf, bm25 and field weights; df, vocab and avgLen stay page-only, so feeding rows never moves a page score (a golden test fingerprints page scores and order before and after).runSearchmerges pages and rows into one ranked list; a row hit has an empty path and arowobject with collectionId, collectionName, rowId, the matched property and anhrefthat opens the row directly in the table view.#tagandin:folderqueries stay page-only.Live wiring.
SpaceIndex.setCollectionRowsremembers its inputs and re-applies them on every rebuild, so page reloads cannot drop the rows.Mdwikiseeds rows at boot only when the collections file already exists (never creates the DB as a side effect) and otherwise attaches at the first open throughwhenCollectionsOpen. Store events mark collections dirty and coalesce into one re-index 250 ms after the last write;flushCollectionRowsruns the same converge-by-reread path, so a deleted collection loses its rows.UI and REST. The search page renders row hits in the same list with a
Collection · Propertylabel and links torow.href; the count line readsX pages · Y rows.GET /api/v1/searchand the MCPsearchtool pass the same objects through, so agents see row hits too; the MCP description now says so.Tests.
test/search-rows.test.tsadds 17 cases in three layers: row docs on the bare index (shape, href, idempotency, validation), a page-ranking regression fingerprint (scores, order, per_page and facets byte-identical before and after feeding rows), and the liveMdwikipath on a real temp SQLite file (boot seeding, attach-on-first-open without creating the DB, update and delete propagation, reload survival, burst coalescing under fake timers). 484 tests pass on this branch (main 459, plus 8 from the merged #83 branch, plus these 17),svelte-check0 errors, build green.Verification lever.
scripts/search-rows-walk.shreruns the acceptance against a copy of any instance: copy space and database into a temp dir, start the built server on a free port, search a real value, create/edit/delete a scratch row over REST and assert each change reaches the index without a restart and that#tagqueries return no rows. Every write stays below the temp dir.Evidence, measured 2026-11-15 against a copy of the live instance (7 collections, 153 rows):
The capture shows a real Rolodex phone value found at boot as one page plus four row hits in one ranked list, the scratch row searchable one second after its create, the edit replacing the old value in the index, the delete removing it, the
#tagquery returning no rows, and one unchanged server process across the whole walk.Two review fixes on top of the implementation: the search page count line printed
result.totalas pages although total now includes row hits (nowtotal - rows), and the MCP search description owed the row-hit sentence.