Skip to content

feat(ios): add unified local search — private FTS5 index, Search screen, and Linux eval package - #62

Draft
obvious-autobuild[bot] wants to merge 11 commits into
obv/fl-next-historical-nutrition-r1from
obv/fl-l2-unified-search
Draft

obvious-autobuild[bot] wants to merge 11 commits into
obv/fl-next-historical-nutrition-r1from
obv/fl-l2-unified-search

Conversation

@obvious-autobuild

@obvious-autobuild obvious-autobuild Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

What changed and why

Unified local search for FridgeLuck, built so the search index never shares migration ownership with the app database:

  • Private FTS5 index (apps/ios/Platform/Search/): the index lives in its own SQLite file (fridgeluck-search-index.sqlite, created next to the app database). The shared migration list in DatabaseMigrations is untouched — the index stores its own schema version and rebuilds from sources when the file is missing, corrupt, or at an older schema version.
  • Source adapters: read-only adapters over IngredientRepository, inventory, RecipeRepository, and cooking-journal sources produce typed documents (ingredient / inventory / recipe / journal) with canonical string IDs. Inventory canonical IDs match InventoryActiveItem.id exactly.
  • Query pipeline: normalization and tokenization with a length cap, FTS prefix matching, deterministic bm25-based ranking with stable tie-breaking, and source-kind filters.
  • Lifecycle: epoch-driven lazy revalidation (a file-stat epoch of the source database — any write changes it and triggers a rebuild at the next bootstrap or search), restore-notification handling, and stale-hit cleanup on query.
  • Journal date tokens without touching UserDataRepository: the journal search adapter reads cooking_history joined to recipes through typed GRDB reads directly, so search date tokens never depend on the journal aggregate path. An earlier commit also fixed UserDataRepository.cookingJournal() silently falling back to the current date for cooked_at; per review, that out-of-scope change was reverted (commit bf3f2ba) — the fix belongs to a dedicated follow-up PR.
  • App integration: AppDependencies builds the shared SearchIndexService and keeps the restore observer alive for the process lifetime. A magnifying-glass entry on the home stack opens the new Search screen (Feature/Search/SearchScreen.swift), which debounces queries, filters by kind, and resolves tapped results against the live repositories into a detail preview.
  • Linux check package (apps/ios/Tools/unified-search-check/): copies production search and persistence sources (no Xcode needed) and runs document, engine, lifecycle, and evaluation tests. Retrieval evaluation uses a frozen corpus (Scripts/gen_corpus.py → 83 documents: 24 ingredients, 9 inventory records, 30 recipes, 20 journal entries; 839 queries) scored with recall@K, precision@K, and MRR. Expectations come from a construction-based oracle that re-implements the matching contract (unicode61 tokenization per indexed column, implicit-AND prefix terms, underscore-kept query tokens as FTS5 adjacency phrases) independently of the engine code path, so an engine regression shows up as a metric drop rather than a silent pass. The committed report is Reports/eval-report.json (corpus unified-search-eval-corpus-v2).

Review fixes (round 2)

  • Restored the searchIndexService / searchRestoreToken stored-property declarations in AppDependencies that a parallel-edit clobber had dropped (their initializers and uses were pushed but the declarations were missing — the root cause of the hosted-CI compile errors).
  • Resolved the SearchScreen type errors CI reported: explicit Hashable on SearchKindFilter, the empty-state call passes a String, and the filter-button content is disambiguated.
  • Reverted the out-of-scope UserDataRepository timestamp-decoding change and reworked the journal adapter to read cooking_history directly; SearchSources no longer takes a journal repository.
  • Grew the frozen evaluation corpus from 19 to 839 queries with oracle-computed expectations (the review's 500+ requirement); the oracle caught that the original hand-written expectations encoded top-K intuition and silently omitted journal-title matches, e.g. chicken also matches four "Chicken …" journal entries.

Tap-resolution fix (round 3)

Hosted job 114340457509 (run 38095507072, branch head e1e2c52) failed at SearchScreen.swift:154: the tap path called SearchIndexService.resolve, which did not exist on the service. Fixed branch-locally in 1e72cba (base branch and CI files untouched):

  • SearchIndexService now exposes resolve(_ hit:). It runs the epoch revalidation, then resolves the hit through the same CompositeSearchHitResolver the engine uses for ranking — so taps read live repositories and never cached index data — and refuses:
    • deleted records: the resolver returns nil; the dead document is repaired out of the index so it cannot be tapped again (a repeat tap still refuses);
    • stale hits: when the kind carries a checkable live revision (inventory, journal), a hit whose revision no longer matches the live record returns nil without guessing. The document stays in place and is healed by the next epoch-triggered rebuild — matching the engine's documented staleness contract. Ingredients and recipes have no checkable revision and rely on the rebuild, exactly as the engine documents.
  • Isolated tests (SearchResolveTests.swift) use the production resolver through the service — no stubs — with a constant test epoch so direct database writes do not trigger a rebuild and the index intentionally keeps stale/deleted documents while resolve itself must do the refusing. Coverage: valid taps on recipe / journal / inventory / ingredient hits resolve the canonical record; deleted records are refused and repaired (6 → 5 docs); stale journal (rating change) and stale inventory (last_updated_at change) hits are refused without mutating the index.

How I checked

  • swift test in the check package on Linux (Swift 6.4, x86_64-unknown-linux-gnu): 42/42 tests pass — 35/35 baseline before the fix, +7 new resolve tests, all green after (mirrors refreshed via Scripts/refresh.sh).
  • Independent reproduction of the frozen 839-query evaluation: two fresh evaluator runs (EVAL_REPORT_PATH set, --filter SearchEvalTests) wrote reports byte-identical to each other and to the committed Reports/eval-report.json — SHA-256 2e914784dce6071a4cd01fa1ae32781c57c0603da52c0ceabed42eabf41a57aa for all three. Metrics: mean recall@K, precision@K, and MRR all 1.0.
  • historical-nutrition-check on Linux: 12/12 pass after running its own Scripts/refresh.sh (its mirrors are untracked in a fresh checkout and must be refreshed first).
  • Hosted macOS CI on head 1e72cba: run 38103789437 — success; both jobs (build-and-test, dead-code-advisory) passed. This is the compile gate for the app target and includes the app-side tests.
  • Oracle self-checks: the generator asserts every corpus query is covered and that generated expectations are non-empty and discriminating; the earlier underscore-tokenizer divergence between oracle and SearchText.tokens was caught by the eval itself (cheddar_cheese) and fixed in the oracle.

What I could not check

  • No Xcode in this environment. Runtime behavior of the Search screen on device/simulator (debounce feel, sheet presentation, navigation, keyboard flow) is unverified; hosted CI compiles the app target and runs its tests on macOS, which is the compile gate.
  • apps/ios/Tools/barcode-check does not exist on this branch — it belongs to the barcode-intake branch (PR feat(ios): barcode intake — FLBarcode module, GTIN validation, source-pinned cache (milestone 1) #51) and cannot be run here. On this branch only unified-search-check and historical-nutrition-check are Linux-runnable.
  • A fresh checkout must run Scripts/refresh.sh per check package before swift test; the mirrored sources are untracked.

Handoff

  • Current state: green. Branch head 1e72cba on obv/fl-l2-unified-search; hosted run 38103789437 succeeded on both jobs. The previously failing run 38095507072 (job 114340457509, SearchScreen.swift:154 — SearchIndexService has no member resolve) was fixed by 1e72cba, and that fix's own CI run is the green one recorded above. No "compilation is fixed" claim predates that run.
  • Review-feedback disposition: ① app compile/wiring failures → fixed (root cause: parallel-edit clobber of AppDependencies stored properties; explicit Hashable + String empty state in SearchScreen); ② 500+ query evaluation → 839 frozen queries with a construction-based oracle and committed report (reproduced byte-identically this round); ③ out-of-scope timestamp change → reverted, journal adapter reads cooking_history directly; ④ incorporate snapshot parent 6ee55e7 → merged, history preserved; ⑤ missing tap-path resolution → SearchIndexService.resolve(_:) with deleted/stale refusal, isolated tests added.
  • Unrelated branch observation (not touched): FLFeatureLogic contains an inherited import CoreGraphics in FeatureLogic/Assistant/LiveAssistantPanelLayout.swift (from commit d48f385). CoreGraphics does not exist on Linux, so any Linux suite that path-depends on the apps/ios package cannot build here. This PR's check packages copy individual files instead of path-depending, so they are unaffected. Fixing that import belongs to whoever owns the Live Assistant feature.
  • To re-verify locally: cd apps/ios/Tools/unified-search-check && bash Scripts/refresh.sh && EVAL_REPORT_PATH=<worktree-internal path>/eval-report.json swift test (Swift 6.4 on this sandbox; avoid /tmp for the report path on shared sandboxes). Regenerate the corpus with python3 Scripts/gen_corpus.py (deterministic, byte-identical).
  • Open items: the UserDataRepository.cookingJournal() current-date fallback remains a real production bug now that the fix is out of this PR — it deserves its own small PR; restore-notification posting is still unwired (as before, by design).

Assumptions

  • The restore notification (FridgeLuckUserDataDidRestore) is observed but never posted by this module; a follow-up wires it to the backup/restore flow (PR feat(ios): add local backup archive codec, validator, and staged restore engine #61's restore currently replaces source records without posting it).
  • Search results resolve into a self-contained detail sheet; deeper navigation (recipe preview drawer, ingredient editor) is left for review since there is no dedicated recipe detail view to link to.
  • The search index file lives beside the app database in Application Support; deleting it is always safe (full rebuild on next search).

Human author: Sam Gu (sgu07966@gmail.com)

🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact

SamGu-NRX and others added 5 commits October 10, 2026 21:25
cookingJournal() read cooked_at with `as? Date`, which failed on the
stored value and silently fell back to the current date, corrupting
journal timestamps downstream (including search date tokens).

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
A private SQLite FTS5 index (own file, no shared migration ownership)
built from read-only adapters over ingredient, inventory, recipe, and
cooking-journal sources. Includes typed record kinds and canonical IDs,
query normalization/tokenization with a length cap, deterministic
bm25-based ranking, live hit resolution with stale-record cleanup,
epoch-driven lazy revalidation, and restore-notification handling.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…tion

A Linux SwiftPM package (no Xcode required) that copies production
search and persistence sources and runs document, engine, and lifecycle
tests plus a retrieval evaluation over a frozen 83-document, 19-query
corpus generated by its Scripts/gen_corpus.py, scoring recall@K,
precision@K, and MRR with an optional JSON report.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
AppDependencies builds the shared SearchIndexService beside the app
database, using a file-stat epoch provider and keeping the restore
observer alive for the process lifetime. A magnifying-glass entry on
the home stack opens the Search screen, which debounces queries,
filters by source kind, and resolves tapped results against the live
repositories into a detail preview.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ea5b450b-fb6a-4b6c-bd96-6ae18bc37d96

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

SamGu-NRX and others added 5 commits October 10, 2026 23:06
Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…ompile errors

Restores the searchIndexService and searchRestoreToken stored properties in
AppDependencies (a parallel edit had dropped the declarations while keeping
their initializers), resolves the SearchScreen type-inference errors flagged
by hosted CI (explicit Hashable on SearchKindFilter, String empty-state
message), and reverts the out-of-scope UserDataRepository timestamp-decoding
change. The journal search adapter now reads cooking_history joined to
recipes directly, so journal date tokens no longer depend on the reverted
repository path, and SearchSources drops its journal repository parameter.

Co-authored-by: obvious-autobuild[bot] <obvious-autobuild[bot]@users.noreply.github.com>
Replaces the 19-query hand-written corpus with 839 frozen queries whose
expectations come from a construction-based oracle: the oracle re-implements
the matching contract (unicode61 tokenization per indexed column, implicit-AND
prefix terms, underscore-kept query tokens as FTS5 adjacency phrases)
independently of the engine code path. Adds the committed evaluation report
for the frozen corpus (recall@K, precision@K, MRR all 1.0) and updates the
check-package fixtures and journal-adapter mirror to the current sources.

Co-authored-by: obvious-autobuild[bot] <obvious-autobuild[bot]@users.noreply.github.com>
…rload collision

The earlier wiring commit contained only the navigateToSearch references in
destination tracking and state reset; the stored property, the entry point,
and the destination never made it into the pushed file (a parallel-edit
clobber before commit). This adds the @State declaration, a trailing
magnifying-glass toolbar button on the home stack, an openSearch() action
consistent with the other navigation actions, and the SearchScreen
navigation destination.

Also replaces the index-path construction with plain concatenation: on
macOS, UniformTypeIdentifiers adds appendingPathComponent(_:conformingTo:)
to String, and that overload won resolution over the NSString one, failing
hosted CI with a missing-argument error (Linux never sees it).

Co-authored-by: obvious-autobuild[bot] <obvious-autobuild[bot]@users.noreply.github.com>
SearchScreen's tap path called SearchIndexService.resolve, which did not
exist (hosted run 38095507072, SearchScreen.swift:154: value of type
'SearchIndexService' has no member 'resolve'). The service already holds
the same CompositeSearchHitResolver the engine uses for ranking; this
exposes resolution through the service so taps read live repositories,
never cached index data.

resolve(_:) refuses deleted records (repairing the dead document out of
the index) and refuses stale hits when the kind carries a checkable live
revision (inventory, journal), leaving the document in place for the
next epoch-triggered rebuild to heal, matching the engine's documented
staleness contract.

Isolated tests cover: tap on valid recipe/journal/inventory/ingredient
hits resolves the canonical record; deleted records are refused and
repaired out of the index; stale journal and inventory hits are refused
without mutating the index. The 839-query evaluator reproduces the
committed report byte-identically across two fresh runs.

Linux: unified-search-check 42/42, historical-nutrition-check 12/12.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant