[obvious-autobuild]

What changed and why

The catalog's recommendation promise — "what can you cook from your kitchen" — was mostly unmet: against the shipped resources, an exact replay of the hydration-time identity bridge resolved only 112 of 329 vocabulary keys, so only 70 of 763 recipes (9.2%) were fully cookable from a perfect pantry. Two causes: the vocabulary's fdc_id references were absent from the shipped USDA catalog, and most remaining keys had no machine-resolvable id at all.

This change fixes the first cause completely and measures the result honestly:

1. Reach harness (evaluation/catalog-reach/reach_report.py): replays the identity bridge's exact rules — legacy-id path, FDC exact-name path, FDC alias path, first-writer-wins per (key, id) — against the shipped bundle, usda_ingredient_catalog.sqlite, and legacy data.json; reports resolvable keys, fully resolvable recipes overall and by rights class, per-cuisine reach, and a pantry replay (200 seeded pantries of 12–25 ingredients over a 1,000-item pool, seed 42). Baseline captured as reach-before.json/.md at the branch point.
2. 150 new canonical USDA rows (scripts/data/import_missing_fdc_rows.py): every fdc_id the vocabulary references but the canonical catalog lacked, sourced only from the public FDC bulk downloads (Foundation 2026-04-30, SR Legacy 2018-04, FNDDS 2024-10-31, CC0, no API key): the food row's own description as the display name, per-100 g macros from its food_nutrient entries, category labels driven by the vocabulary's own categories with FDC food-category fallback, USDA provenance in source_meta.verification_source. 116 ids closed the pre-merge gap; 34 more closed the same gap against PR data(bundle): correct rights labels from the release gate, attach legacy ingredient links, complete 51-key nutrition #168's resolved vocabulary (950 records total). Survey/FNDDS rows use legacy nutrient ids (208/203/205/204/291/269/307) in food_nutrient.csv, so the extractor falls back to them — without this, 79 of 116 rows would have shipped all-zero macros.
3. Shipped artifacts rebuilt: usda_ingredient_catalog.sqlite rebuilt from the merged canonical (macro freeze verified programmatically: zero existing rows changed); USDAIngredientNutritionStaticData.swift regenerated from the committed compact JSON, which also fixes pre-existing drift — the shipped Swift predated the compact JSON's own regeneration (matchedCount 15 → 49, older generatedAtUTC).
4. PR data(bundle): correct rights labels from the release gate, attach legacy ingredient links, complete 51-key nutrition #168 merged at head 02de2fb (merge commit d4dc4a3), which resolved 51 more vocabulary keys and is the other half of the final number.

Result: 283 of 329 keys resolvable; 655 of 763 recipes fully resolvable (85.9%); original-rights 373 of 461 (80.9%). Exact median pantry reach is 0 fully-resolvable recipes (p10 0, p90 1) and the at-most-2-missing median is 72 (p10 67, p90 91) — typical pantries buy toward a cuisine, not the whole catalog. 46 keys remain unresolvable (top blockers: garam_masala 26 recipes, scotch_bonnet 15, gochujang 9, thai_chilies 8, mirin 5, harissa 4): they carry neither fdc_id nor legacy_id in the current vocabulary, so only cited same-food legacy/FDC ids (PR #168's pending legacy-link work) can resolve them; this PR invents none.

Note on PR #172's claim of "124 resolved keys": my exact-rule replay measured 112 at the branch point and I could not reproduce 124; the final state (283) is measured, not inherited.

How I checked it

• Baseline replay at the branch point: python3 evaluation/catalog-reach/reach_report.py → reach-before.{json,md}: 112/329 keys, 70/763 (9.2%), original-rights 33/461.
• Importer dry run then apply with the schema's own validation on every merge: python3 scripts/data/import_missing_fdc_rows.py --fdc-csv-root /tmp/fdc/ext [--apply] → 116 then 34 records, validate_catalog PASS each time (950 records, 3,919 unique aliases, 14 required search terms present).
• Macro freeze: parsed diff of all 800 pre-existing records before vs after merge → 0 changed.
• Nutrient-id scheme verified per dataset from nutrient.csv (all three datasets carry both id schemes; food_nutrient rows for Survey foods use the legacy ids).
• Spot checks: salt 0 kcal, coconut oil 895 kcal, butternut squash 45 kcal, soy milk 3.55 g protein/100 g — all match USDA reference values.
• SQLite rebuild: usda_core.build_sqlite over the merged canonical, 950 records written.
• Static Swift: regenerated and diffed — diff vs shipped is exactly the pre-existing compact-JSON drift, regenerated in place.
• Post-merge reach: reach-after.{json,md} → 283/329 keys, 655/763 (85.9%), original-rights 373/461 (80.9%).
• python3 -m pytest data/recipes/tools/tests/ -q → 17 passed, 1 failed; the failure (test_bundle_compiles_byte_identically) is pre-existing — it also fails at the merge-base (1 failed / 13 passed there, 1 failed / 18 passed here) — because the test invokes build_vocab.py, which reads /tmp/canon/identities.json, an uncommitted scratch canon input; the compiled bundle data/recipes/build/recipes-v2.json itself is committed.
• git push of obv/burst8-fridgeluck-catalog-reach (two commits + the data(bundle): correct rights labels from the release gate, attach legacy ingredient links, complete 51-key nutrition #168 merge, Conventional Commits, no force-push, no base-branch writes).

What I could not check

• No Xcode here: the iOS app was not built. The sqlite and Swift static-data changes compile into the app only via hosted macOS CI on this PR, and the hydration-time XCTest (identity bridge, marker rehydration) runs only there. The bridge replay in this PR mirrors the Swift rules but is a Python reimplementation, not a substitute.
• The byte-identical bundle compile test could not pass anywhere in this environment (missing uncommitted build artifact, pre-existing).
• uv sync was unavailable; the pipeline ran under system Python 3.13 with pydantic, filelock, orjson installed directly.

Assumptions

• PR data(bundle): correct rights labels from the release gate, attach legacy ingredient links, complete 51-key nutrition #168 was merged at head 02de2fb — the latest head, with build-and-test and dead-code-advisory green (verified live twice this session). The task's 120-minute wait for a newer green head was cut short at ~70 minutes because that head was already the latest and green.
• Display names for FDC rows use title-cased USDA descriptions (mixed-case descriptions pass through unchanged); alt_names left empty; sprite metadata left at defaults — all three are editable through the existing review-batch flow without touching macros.
• Category labels prefer the vocabulary's own category per key (meat/fish/shellfish → protein, spice → herb_spice, grain → grain_legume), falling back to FDC food-category mapping, then other.
• The compact JSON (usda_ingredient_nutrition_compact.json) derives from the legacy 50-ingredient data.json via the FDC API and is unaffected by this change, so it was left untouched; its generator needs FDC_API_KEY, which this change does not require.
• Base branch obv/burst6-fridgeluck-catalog-recommend per the task; draft PR; no merge, no base push, no CI/deploy configuration touched.

Review response (r1) — independent review of 2026-10-15

Independent review (comment 6205279003) verified the data work and ranked four findings; all four are addressed at head a254845c:

1. HIGH — build-and-test red at the PR head, inherited from the base branch. Confirmed: run 38814953544 failed in ReverseScanService.swift/ReverseScanMealView.swift (ScoredRecipeID ?? Int64), files outside this PR's diff; the base branch fixed exactly this in 832c5921 + 38263783. Fixed by merging origin/obv/burst6-fridgeluck-catalog-recommend (38263783) into this branch (merge commit a069bb17, no conflicts — those commits touch only the two failing files plus two test/preview files). Both fix commits are now ancestors of the head.

   Round 2 (same finding): the merge exposed a second inherited compile break — run 38835536610 (2m47s, exit 65) failed in Tests/TasteStore/TasteEventStoreTests.swift with seven type errors: the cookedEvent helper declares -> TasteEvent while constructing a TasteEventRecord. Verified this file is byte-identical to the base head (git diff 38263783 HEAD is empty for it), i.e. the breakage rides both branches; trunk fixed exactly this in 9de29858 (fix(ios): cookedEvent helper returns the store record type), which the reviewer's base-green run picked up through that PR's merge ref. Fixed by cherry-picking 9de29858 onto this branch (one file, one line; head 6b8a3489). CI is green at 6b8a3489: run 38836100289 — build-and-test pass 6m16s, dead-code-advisory pass 3m7s.

2. MEDIUM — body misstated the remaining blockers and the baseline original-rights figure. Confirmed against the committed reports and fixed above: the after-state top blockers are garam_masala 26, scotch_bonnet 15, gochujang 9, thai_chilies 8, mirin 5, harissa 4 (the chicken_stock/ground_turmeric values previously quoted were the reach-before state), and the baseline original-rights figure is 33/461, not 38/461.

3. LOW — wrong cause given for the pre-existing test_bundle_compiles_byte_identically failure. Confirmed by rerunning the test (FileNotFoundError: /tmp/canon/identities.json) and by git ls-files (the build bundle is committed); corrected above. Pre-existing claim re-verified: same single failure at the merge-base.

4. LOW — kcal-drift validator failed open on exemptions missing kcal fields. Confirmed (abs(entry.get(field, nan) - value) is always False for NaN). Fixed in validate_bundle.py: a missing old_kcal/new_kcal now reports exemption missing <field> explicitly. Regression test test_fieldless_exemption_fails_closed fails before the fix (planted field-less exemption produced zero errors) and passes after; the full data/recipes/tools suite is 18 passed, 1 failed (the same pre-existing byte-identity test, unchanged).

Commands for the fix round: python3 -m pytest data/recipes/tools/tests/test_kcal_drift.py::test_fieldless_exemption_fails_closed -q (failed pre-fix, passed post-fix), python3 -m pytest data/recipes/tools/tests/ -q (18 passed, 1 pre-existing failure), git fetch origin obv/burst6-fridgeluck-catalog-recommend && git merge --no-ff origin/obv/burst6-fridgeluck-catalog-recommend, git merge-base --is-ancestor 832c5921 HEAD && git merge-base --is-ancestor 38263783 HEAD (both true); round 2: git show 9de29858 (trunk's fix, message names the same seven macOS type errors), git cherry-pick 9de29858476414fc426c4002262169c5849b4fec, git diff 9de29858 HEAD -- apps/ios/Tests/TasteStore/TasteEventStoreTests.swift (0 lines — tree matches trunk's fixed version), gh pr checks 175 (all pass).

🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: b9fcb4c3-296c-416b-aef8-674778b029e5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[obvious-autobuild]

Independent review (Obvious, Autobuild + Max)

Verdict: changes needed. The data work itself is sound — every load-bearing number I could reproduce, I reproduced exactly (both reach reports, the macro freeze, the bundle byte-identity, the sqlite rebuild) — but hosted CI is red at this head (inherited from the base branch, with the fix already on base), and the PR body misstates two measured results in ways that will mislead readers.

Findings (ranked by severity)

1. HIGH — build-and-test fails at this head; the breakage is inherited from the base branch and the fix already exists on base.
CI run 38814953544 (headSha aa3b5c0d, exit 65) fails with Swift compile errors: apps/ios/Capability/Core/Services/ReverseScanService.swift:14 (binary operator '??' cannot be applied to operands of type 'ScoredRecipeID' and 'Int64', plus :236-237) and apps/ios/Feature/Estimate/ReverseScanMealView.swift:91. None of these files are in this PR's 23-file diff. Evidence: the merge-base with obv/burst6-fridgeluck-catalog-recommend is 9f5a7017, where ReverseScanService.swift:12-16 still reads var id: Int64 { recipe.id ?? Int64(...) }; the base branch fixed exactly this in 832c5921 ("complete the pooled-identity switch in reverse scan", touches only the two failing files) and 38263783 ("import FLFeatureLogic in the catalog tests"), neither of which is an ancestor of this head. Base head 38263783 is green (run 38817702837, 9m2s, headSha verified). Smallest fix: merge origin/obv/burst6-fridgeluck-catalog-recommend (38263783) into obv/burst8-fridgeluck-catalog-reach and let CI re-run — those two commits touch only apps/ios, so no conflicts with this PR's data/eval files are expected.

2. MEDIUM — the body's "remaining unresolvable" blockers are the before-state values, not the after-state.
The body says the 46 remaining keys' top blockers are "chicken_stock 43 recipes, ground_turmeric 28, garam_masala 26, vegetable_stock 23, ground_ginger 16" — but this PR pins and resolves four of those five (FDC_PINS: chicken_stock 172884, vegetable_stock 171180, ground_turmeric 172231, ground_ginger 170926), and those values are exactly the reach-before.json blockers. The committed — and independently reproduced — reach-after.json top blockers are: garam_masala 26, scotch_bonnet 15, gochujang 9, thai_chilies 8, mirin 5, harissa 4. Related body error: the baseline original-rights figure "38/461" contradicts the committed reach-before.json/.md and my merge-base reproduction, which both say 33/461 (7.2%). Smallest fix: edit the body paragraph to the after-state blockers and 33/461. The headline numbers (283/329 keys, 655/763 = 85.9%, 373/461 original-rights) are correct.

3. LOW — the explanation of the pre-existing test failure is wrong about the cause.
The body says test_bundle_compiles_byte_identically fails because it "reads data/recipes/build/recipes-v2.json, a scratch build directory that is not committed" — that file is committed (this PR modifies it). The actual failure is FileNotFoundError: /tmp/canon/identities.json (build_vocab.py's uncommitted scratch canon input). The failure being pre-existing is true: at merge-base 9f5a7017 the suite is 1 failed / 13 passed with the same test; at this head, 1 failed / 17 passed. Smallest fix: a one-line body correction.

4. LOW — the new kcal-drift validator fails open when an exemption omits the kcal fields.
data/recipes/tools/validate_bundle.py:108-111: abs(entry.get(field, float("nan")) - value) > KCAL_MATCH_TOLERANCE — NaN comparisons are always False, so an exemption entry missing old_kcal/new_kcal passes the "entries must reproduce both kcal values" contract silently (a blank vocabulary_change still fails, and the committed changes file carries both fields). Smallest fix: before the comparison, if field not in entry: errors.append(f"{rid}: exemption missing {field}"); continue, plus a planted-violation test for a field-less entry.

What I verified (commands and results)

Environment: fresh worktree of the PR head (git fetch origin pull/175/head + git worktree add --detach /tmp/pr175-review), Python 3.13.14, pytest 9.0.3, pydantic/filelock/orjson installed per the body's note.

• Reach after, reproduced exactly. python3 evaluation/catalog-reach/reach_report.py --repo-root <head worktree> --out /tmp/reach-head.json → "resolvable keys 283/329, fully resolvable recipes 655/763 (85.9%)"; diff of pretty-printed output vs committed reach-after.json → identical, including the pantry replay (exact median 0, p10 0, p90 1; at-most-2-missing median 72, p10 67, p90 91; seed 42, 200 pantries).
• Reach before, reproduced at the merge-base — the fail-without-change evidence. Same script run in a 9f5a7017 worktree → "resolvable keys 112/329, fully resolvable recipes 70/763 (9.2%)", original-rights 33/461, pantry median 68 — matches the committed reach-before.json. The measurement is real and moves only because of this PR's data.
• Tests. python3 -m pytest data/recipes/tools/tests/ -q at head → 17 passed, 1 failed (the byte-identity test); at the merge-base → 1 failed, 13 passed (the same single test) → pre-existing, unchanged by this PR. The new test_kcal_drift.py passes and includes a seeded-violation negative control that runs the real validator (exit 1 with the recipe id named).
• Macro freeze, verified independently. Parsed scripts/data/catalog/usda_curated_ingredients.json at base (800 records) vs head (950): 0 pre-existing records changed, exactly 150 added (116 + 34 as the body describes).
• SQLite. 950 ingredient_catalog rows = 950 ingredients rows; display_name == ingredients.name for all 950; no orphan rows; name TEXT NOT NULL UNIQUE; 3,919 distinct nonempty aliases (matches the body). Spot values match USDA references and the body's claims: salt (173468) 0.0 kcal; Coconut oil (2710182) 895 kcal; butternut squash (169295) 45 kcal; Soy milk, unsweetened (2705405) protein 3.55 g/100 g; Hoisin sauce (2707441) 220 kcal; chicken stock (172884) 36 kcal.
• Bundle integrity. Shipped apps/ios/Resources/RecipeCatalog/recipes-v2.json is byte-identical to data/recipes/build/recipes-v2.json (2,611,873 bytes, sha256 d442a11f…3edcbc2, matching the final commit's recorded hash); generated_at is the pinned 2026-10-15T16:46:35Z. The bundle's 46 per_100g: null keys are exactly the 46 keys in unresolved-candidates.json (set-equal), and per-100g coverage is 283/329 as claimed.
• Static Swift regeneration is real. USDAIngredientNutritionStaticData.swift matches the committed compact JSON (matchedCount 49 = matched_count 49; generatedAtUTC 2026-02-11T05:07:06Z = generated_at_utc), i.e. the pre-existing drift fix mirrors its source faithfully.
• Bridge parity. Read RecipeCatalogIngredientBridge.rebuild and BundledDataLoaderUSDACatalog against reach_report.py: legacy_id-first precedence, fdc exact-name then alias resolution, alias lowercasing, unique ingredient names — the Python replay is faithful to the hydration rules it claims to replay.
• Fail-closed tooling. record_nutrition_changes.py raises on any >40% swing that traces to no resolved key on the recipe's own lines; build_vocab.py refuses to emit when the unresolved-candidates audit drifts from the vocabulary.
• CI provenance. gh pr checks 175 → build-and-test fail; gh run view 38814953544 --json headSha → aa3b5c0d (this head); gh run view 38817702837 → success at 38263783 (base head).

What I couldn't verify

• Claims that require the FDC bulk CSVs (not available here): the "79 of 116 rows would have shipped all-zero macros" figure and the byte-level CSV extraction itself. The importer reads correctly (dry-run default, schema-validated append-only merge, unique-name fallbacks, documented legacy nutrient-id fallback) and its output rows match USDA reference values where I spot-checked.
• The iOS build and tests (no Xcode here), blocked at this head by the inherited compile failure; the hydration-time XCTest path runs only on macOS CI.
• test_bundle_compiles_byte_identically passing anywhere (needs the uncommitted /tmp/canon scratch input; pre-existing, unchanged by this PR).
• I did not independently recompute per-recipe nutrition (the 164/500/99 coverage split, median +11.5 kcal, 193 recipes moved). The >40% moves are covered by the drift validator; its committed changes file reproduces the shipped bundle's kcal values (enforced by test_seeded_drift_violation_and_real_changes_file_consistency).

No scope complaints beyond the above: the 23-file diff is what the body says it is, the three carried #168 commits and the merge are the stated stacking design, and nothing touches CI or deploy configuration.