refactor(storage): isolate dialect search behind Bun capabilities - #1345
Open
mariusvniekerk wants to merge 6 commits into
Open
refactor(storage): isolate dialect search behind Bun capabilities#1345mariusvniekerk wants to merge 6 commits into
mariusvniekerk wants to merge 6 commits into
Conversation
roborev: Combined Review (
|
mariusvniekerk
force-pushed
the
t3code/bun-search-unification
branch
from
August 7, 2026 03:46
7bb6694 to
5fcc6ce
Compare
roborev: Combined Review (
|
mariusvniekerk
force-pushed
the
t3code/bun-search-unification
branch
from
August 7, 2026 14:10
5fcc6ce to
a27f90c
Compare
roborev: Combined Review (
|
mariusvniekerk
force-pushed
the
t3code/bun-search-unification
branch
from
August 7, 2026 16:34
a27f90c to
db3867f
Compare
roborev: Combined Review (
|
Session search needs engine-specific matching and ranking, but visibility, metadata hydration, pagination, and lifecycle consistency are common storage behavior. Keeping the complete read in each backend duplicated canonical rules and let raw queries escape the guarded Bun snapshot. Pass the operation-scoped Bun handle into narrow FTS capabilities so SQLite FTS5, PostgreSQL ILIKE, and DuckDB matching differ only at the SQL boundary while every public store resolves through one Bun hydration path.
Lexical content search needs dialect-specific matching, but source hydration, visibility, snippets, timestamps, conversation-unit ranges, ordering, and cursors are canonical behavior. Keeping those rules in three stores made parity fixes expensive and allowed each backend to drift. Route substring and regex matching through portable Bun queries, retain only SQLite FTS5 versus portable FTS candidate selection as a capability, and delete the superseded backend scanners. Capability limits now remain operation-scoped so the shared store can fetch its cursor sentinel without collapsing the public maximum page size.
The common search layer depends on candidate windows having the same visibility, ordering, and limit semantics on every adapter. Boundary lookahead, optional SQLite FTS, missing activity timestamps, and backend-specific FTS ordering could otherwise shorten pages or reorder identical results. Make the capability pagination contract explicit, preserve in-session substring search without a global index, use the canonical activity fallback and recency order, and reject unsupported FTS sources. Retain Unicode recent-edit matching on SQLite without returning to full edit-payload hydration.
Semantic and hybrid search need dialect-specific vector and lexical candidate generation, but visibility, unit resolution, scope filtering, fusion, hydration, redaction, and final ordering are canonical behavior. Keeping complete pipelines in SQLite and PostgreSQL duplicated those rules and made backend parity fragile. Route both modes through BunStore capabilities, retain only SQLite FTS5 versus PostgreSQL keyword candidate SQL at the adapter boundary, and preserve backend-specific unavailable guidance. Remove the superseded concrete enrichment and pagination paths so future search changes have one owner.
Shared search must preserve Unicode matching, stable pagination, semantic range provenance, and snapshot safety across every adapter. Review exposed cases where portable SQL or transaction boundaries could silently drop valid matches, retain stale vector anchors, or exhaust the PostgreSQL connection pool. Keep engine-specific candidate generation narrow while separating bounded session search from global FTS availability and ensuring canonical hydration owns final visibility and range behavior.
Shared search decisions should follow explicit adapter capabilities instead of backend-name checks, while DuckDB session search must use the mirror canonical message relationship even when optional source row IDs are absent.
mariusvniekerk
force-pushed
the
t3code/bun-search-unification
branch
from
August 9, 2026 03:46
db3867f to
e0be781
Compare
roborev: Combined Review (
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-text and vector candidate retrieval legitimately differs by engine, but hydration, visibility, filtering, deduplication, and public pagination should not. This layer narrows dialect-specific search to capability adapters and moves the observable search pipeline into the shared Bun store.
DuckDB tool-content search now uses canonical session and message-ordinal identity instead of disposable physical row IDs. Timestamp ordering and portable search choices are supplied through narrow adapter hooks, while engine-specific relevance ranks remain allowed where the engines expose genuinely different semantics. Stack 3 of 5.
generated by a clanker