Skip to content

feat(data): rebuild the shipped catalog — corpus fixes, fail-closed claims, wave-2 packet ingestion (762→822 recipes) - #210

Merged
SamGu-NRX merged 5 commits into
alphafrom
obv/burst12-fridgeluck-catalog-rebuild
Oct 16, 2026
Merged

SamGu-NRX merged 5 commits into
alphafrom
obv/burst12-fridgeluck-catalog-rebuild

Conversation

@obvious-autobuild

@obvious-autobuild obvious-autobuild Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Rebuild the shipped catalog: corpus fixes, fail-closed diet claims, and the wave-2 packet ingestion

Applies tonight's reviewed recipe-fix packets (#195 amounts-a, #196 amounts-b, #197 rubric — 189 fixes), compiles diet claims fail-closed, and ingests the merged 2026-10 breakfast (#201) and pantry (#203) original batches. The shipped bundle goes from 762 to 822 recipes (rcp-000167..rcp-000994), every recipe rights-documented.

Branch: obv/burst12-fridgeluck-catalog-rebuild · base: alpha
Commits: d5ed6a4d (fixes + claims), 8a4840c9 (bundle/manifests/fixtures pass 1), 4c193f4d (wave-2 ingestion + pass 2)

What changed and why

  1. Corpus fixes (feat(data): real amounts for catalog recipes rcp-000377..rcp-000934 #195/feat(data): real amounts for catalog recipes rcp-000167..rcp-000376 (packet A) #196/feat(evaluation): recipe-quality rubric audit — 591-recipe sweep, 60-recipe hand-scored sample, 18 pin-aware fixes #197). data/recipes/tools/apply_corpus_fixes.py applied all 189 fixes (86 amounts-a, 85 amounts-b, 18 rubric) one-shot: every fix's before_sha256 matched its pinned corpus line in a dry run; nothing skipped. Ten fixes flagged changes_rights_pin (title/structure edits of FridgeLuck's own text) carry reproducible revision pins in data/recipes/evidence/rights.json (fl_revisions) — title, ingredient-name hash, and full-body hash must match the shipped record or the rights gate refuses.
  2. Fail-closed diet claims. compile_bundle.py now ships an authored true diet claim only when the ingredient-line classes prove it; unprovable claims ship null; a proven vegan claim requires proven vegetarian and dairy-free. Net effect: 7 claim flips true→null on 5 recipes (rcp-000179, -000190, -000199, -000207, -000532); no recipe gained a true claim. One unprovable authored claim was stripped at the corpus layer per the repository's strip_unproven_claims.py policy (fl-original-0341 vegetarian, the rubric fix added a banana line the composition gate can't resolve) and is recorded in stripped_claims. validate_bundle.py audits both rules.
  3. Wave-2 ingestion. data/recipe-bundle/ingest_packets_2026_10_wave2.py ingested both packets as fl-original-0769..0828 (existing originals end at -0768; the ingester refuses collisions): 60 ingested, 0 title-collision exclusions. Traceability map: data/recipes/evidence/ingest-2026-10-wave2-map.json. Both packet branches are pinned in evidence/rights.json at their commits (b4a2c13, 100984f) via pin_rights_evidence.py (+386 lines, no churn to existing pins); every ingested recipe cites its packet record; rights_evidence.py accepts the wave-2 tool name in citations.
  4. Vocabulary: unchanged. All 400 surface names in the new recipes already resolve through the committed name-mappings.json (400/400 mapped). A raw build_vocab.py run was tried and rejected: the committed vocabulary carries curated state (class assignments, identity-audit pins, portion averages, sources) the raw rebuild does not reproduce; the working tree keeps the committed files.
  5. Bundle + app copy + manifests. Recompiled (sha256 2bf71ab4…, 2,901,511 bytes, generated_at pinned 2026-10-16T11:00:00Z), mirrored byte-identically to apps/ios/Resources/RecipeCatalog/recipes-v2.json; both manifests re-pinned (sha, bytes, counts, coverage 165 computed / 543 estimated / 114 partial, provenance adds feat(data): add breakfast-2026-10 originals packet (30 RS v1 breakfast recipes) #201/feat(data): pantry-2026-10 originals — 30 pantry-first recipes lifting exact-cookable reach to every household #203 + the ingestion batch).
  6. Reports. Taste-lab golden fixture regenerated with the lab's own command (regenerate-fixtures --seed 7), replay exact. reach_report_v3.py rerun twice (tracked app-side SQLite both sides, restored to its tracked state). Full record: evaluation/recipe-quality/rebuild.md.

How I checked it (commands and results)

  • python3 -I data/recipes/tools/apply_corpus_fixes.py --dry-run → 189/189 matched, then applied; revision-gate test file 11/11 passed.
  • python3 -I data/recipes/tools/validate_corpus.py → VALIDATION GREEN on the 822-recipe corpus (rights, quarantine, release gates, dedupe, lint).
  • python3 data/recipes/tools/validate_bundle.py → PASS (453 informational null-claim warnings, 0 errors).
  • python3 -m pytest data/recipes/tools/tests/ --deselect data/recipes/tools/tests/test_pipeline.py::test_vocab_rebuild_deterministic → 48 passed, 1 failed; the failure is test_kcal_drift (stale new_kcal entries in data/recipe-bundle/nutrition-changes-2026-10.json), reproduced on the alpha tip in a clean worktree — pre-existing, not introduced here; regenerating that file is fix(data): correct calorie rows behind #192's ingredient re-pins (kcal audit) #205's nutrition-record rebuild follow-up.
  • The deselected test_vocab_rebuild_deterministic is operator-only (needs pinned /tmp FDC/canon downloads). With those inputs present it fails because the committed vocabulary carries curation the raw builder does not reproduce — and running it rewrites the curated vocabulary files mid-suite; files were restored from git. Also pre-existing by inspection: the builder never writes the classes/class_source fields the committed vocab carries.
  • python3 -I data/recipes/tools/compile_bundle.py ×3 → byte-identical (MD5 0c0df917…); app copy cmp → identical.
  • python3 -m taste_lab validate-fixtures → "All fixtures replay exactly"; pytest tests in the lab → 19 passed.
  • Corpus/rights/bundle set equality verified programmatically (822 = 822 = 822, band rcp-000167..rcp-000994, unique); both manifests pin the working bundle sha.
  • Reach, pass 2 (before = HEAD 762-recipe post-fix bundle): median near-cookable omnivore 107.0 → 186.5 (+79.5); vegetarian 57.0→110.5, gluten_free 46.5→95.0, dairy_free 48.5→86.0, vegan 23.5→31.5. Exact-cookable omnivore 1.5→9.0.

What I couldn't check

  • No Xcode here: the iOS app build/compile of the new catalog resource is not exercised locally; hosted macOS CI covers the Swift build. The resource is a data file byte-identical to the validated data/recipes/build output, and both manifests pin its sha.
  • data/recipe-bundle/nutrition-changes-2026-10.json is not regenerated (not an owned path; the kcal-drift test failure predates this branch and is owned by fix(data): correct calorie rows behind #192's ingredient re-pins (kcal audit) #205's follow-up).
  • Reach outputs land in /tmp (evaluation/catalog-reach is not an owned path on this task); the numbers above are the record.

Assumptions

  • The vocabulary is intentionally curated post-build_vocab, so an additive "rebuild" that strips curated state is wrong; with 400/400 new surfaces already mapped, leaving it untouched is the correct additive outcome.
  • The wave-2 ingester is a separate file because the existing ingest_packets_2026_10.py refuses any fl-original-* collision by design; I widened only the citation regex, not the collision guard.
  • Rights pins for the packets were regenerated from the packets' own committed evidence at their merge commits; no record content was hand-edited.

CI (hosted macOS)

  • Run 38905405934 on 4c193f4: failed — two tests, both fixed in 9dbea0d (see evaluation/recipe-quality/rebuild.md "CI round 1 fixes"):
    1. RecipeCatalogManifestIntegrityTests.testManifestLegacyLinkCountMatchesTheShippedBundle — the second-pass manifest patch wrote legacy_ingredient_links as a bare integer; restored the app manifest's record shape {keys_with_legacy_id: 49, resolved_in_map: 49, unlinked: 1, source} (recomputed from the legacy map: 49 resolved + 1 proposed, unchanged).
    2. TasteRankerTests.testGoldenFixturesReplayExactly (golden-hh01) — the taste-lab regen command writes ONE household per invocation (hh00/seed 7, hh01/seed 11, hh23/seed 13); the first pass regenerated only hh00, leaving hh01/hh23 with 762-recipe expectations while the rebuilt bundle's fixed amounts push rcp-000188/000443 past the 24 g protein threshold Swift keys on. Regenerated both with their own seeds; all three fixtures replay exactly on the 822 catalog, lab suite 19/19.
  • Run 38907720833 on 9dbea0d (this branch's head): success (16m57s) — build-and-test green.

Not merged

Per workspace policy this stays a draft and is not merged. When ready, the human merge command is:

gh pr merge <this-pr> --squash

Human author: Sam Gu (sgu07966@gmail.com)

🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact

ObviousApp and others added 3 commits October 16, 2026 12:45
…provable diet claims

Apply the three merged corpus-fix packets (#195 amounts-a 86, #196
amounts-b 85, #197 rubric 18) to the corpus via apply_corpus_fixes.py,
which refuses unless every fix's before_sha256 matches the pinned corpus
line, no two files fix one recipe, and the rights-pin revisions cover
exactly the changes_rights_pin fixes.

Ten fixes edit FridgeLuck-original text after the authoring record was
pinned at ingest. The recipes keep LicenseRef-FL-Original — the revision
is FridgeLuck's own edit of its own text — and the rights gate
(rights_evidence.py) accepts them only against a pinned revision in
evidence/rights.json (fl_revisions, derived by fl_revisions.py) that
describes the shipped record exactly: title, ingredient-name hash, and
full-body hash. Planted step edits, renamed lines, retitles, missing
revisions and wrong licences still refuse
(test_rights_revisions.py, 11 tests).

strip_unproven_claims.py removed one claim the composition gate cannot
prove after the fixes (fl-original-0341 vegetarian; the added banana
line is unresolved in the suite allergen table).

compile_bundle.py now fails diet claims closed: an authored true claim
survives only when the line composition classes prove it — unprovable
claims ship null, vegan true implies vegetarian and dairy_free true
(compile refuses otherwise; validate_bundle.py audits both).

Checks: validate_corpus GREEN; validate_bundle PASS (439 informational
null warnings); pytest data/recipes/tools/tests 48 passed, 1 pre-existing
kcal-drift failure (fails on alpha tip 7c4b30c too), 1 skip.

Co-authored-by: Sam Gu <sgu07966@gmail.com>
…fests

Recompile the bundle from the fixed corpus with fail-closed claims
(generated_at pinned 2026-10-16T09:30:00Z; byte-identical across 3 runs,
MD5 c70c21c742b345539cb1da553a39a261), mirror it byte-identically to the
app catalog, and regenerate both manifests against the actual file — they
had drifted apart, each pinning a different bundle sha.

Diet claims in the shipped bundle (true/false/null): vegetarian
327/333/102 -> 325/333/104, vegan 121/585/56 -> 120/585/57, gluten_free
336/314/112 -> 335/314/113, dairy_free 306/282/174 -> 303/282/177 — seven
true->null flips on five recipes, nothing gained a true claim.

Taste-lab golden fixture regenerated with the lab's own command
(regenerate-fixtures --seed 7) and replay-verified; reach_report_v3
rerun before/after on the tracked catalog sqlite; results recorded in
evaluation/recipe-quality/rebuild.md.

Checks: validate_corpus GREEN; validate_bundle PASS; data tests 48
passed, 1 skipped, 1 pre-existing kcal-drift failure (fails on the alpha
tip too); taste-lab tests 19 passed.

Co-authored-by: Sam Gu <sgu07966@gmail.com>
…catalog

Ingest the merged 2026-10 breakfast (#201) and pantry (#203) original
packets with data/recipe-bundle/ingest_packets_2026_10_wave2.py:
60 recipes as fl-original-0769..0828, 0 title-collision exclusions, with
a packet->corpus traceability map in data/recipes/evidence/ingest-2026-10-wave2-map.json.
rights_evidence.py accepts the wave-2 tool name in packet citations and
pin_rights_evidence.py pins both packet branches (b4a2c13, 100984f) in
evidence/rights.json (+386 lines, no churn to existing pins); every
ingested recipe cites its packet record.

The vocabulary is unchanged: all 400 surface names in the new recipes
already resolve through the committed name-mappings (400/400), and a raw
build_vocab run regresses curated state (classes, identity pins, portion
averages), so the working tree keeps the committed vocabulary files.

Recompile the bundle: 822 recipes, band rcp-000167..rcp-000994, sha256
2bf71ab4..., generated_at 2026-10-16T11:00:00Z, byte-identical across 3
runs (MD5 0c0df917...), mirrored byte-identically to the iOS catalog;
both manifests re-pinned (sha, bytes, counts, coverage, provenance adds
#201/#203 and the ingestion batch). Corpus-size pin 762->822. Taste-lab
golden fixture regenerated (replay exact; lab tests 19 passed). Reach
(second pass, before = HEAD 762-recipe post-fix bundle): median
near-cookable omnivore 107.0 -> 186.5. rebuild.md records both passes.

Co-authored-by: Sam Gu <sgu07966@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 17e87df2-a446-4d39-bf45-c232d43a189c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

SamGu-NRX and others added 2 commits October 16, 2026 13:25
…egacy-links record

Hosted-macOS CI on 4c193f4 (run 38905405934) failed two tests:

1. RecipeCatalogManifestIntegrityTests.testManifestLegacyLinkCountMatchesTheShippedBundle:
the second-pass manifest patch wrote legacy_ingredient_links as a bare
integer; the app manifest's shape is a record {keys_with_legacy_id,
resolved_in_map, unlinked, source}. Restored the record with the
recomputed values (49 keys carry a legacy_id; the legacy map has 49
resolved + 1 proposed, unchanged).

2. TasteRankerTests.testGoldenFixturesReplayExactly (golden-hh01): the
taste-lab regen command writes one household per invocation (--household,
one eval seed each: hh00/seed 7, hh01/seed 11, hh23/seed 13); the first
pass regenerated only hh00, so hh01/hh23 still carried 762-recipe
expectations while the shipped 822-recipe bundle gives
rcp-000188/000443 protein >= 24 g (estimated coverage) and the Swift
extractor emits nutrient:high_protein for them. Regenerated hh01
(--household 1 --seed 11) and hh23 (--household 23 --seed 13) on the 822
catalog; all three fixtures replay exactly and the lab suite passes 19/19.

rebuild.md records the round-1 CI fixes.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
@SamGu-NRX
SamGu-NRX marked this pull request as ready for review October 16, 2026 14:11
@SamGu-NRX
SamGu-NRX merged commit bb4dd45 into alpha Oct 16, 2026
3 checks passed
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T14:15:01.133299Z 15850ed Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@codemetrics-ai

Copy link
Copy Markdown

Note

Review skipped

Codemetrics is installed on this repository, but it hasn't been added to your team yet.

To enable AI code reviews for this repo, go to your dashboard and add FridgeLuck to your team's repositories.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants