[obvious-autobuild]

Human author: Sam Gu (sgu07966@gmail.com)

Taste Profile & Learning — SPEC (taste-residual-v1)

Draft spec + machine-checked contracts for household taste learning: a rule-anchored residual ranker — today's rule score, plus small bounded evidence-weighted feature adjustments — with exposure logging, a three-layer profile statement model (restriction / inferred / pinned) with first-class correction, and deterministic verification contracts. Addresses the same problem as the two failed full-learning attempts (FLTaste trailed the rule ranker by ~11% NDCG@10 in the reduced campaign) without repeating their failure mode: personalization starts at today's behavior and earns deviations slowly.

This PR changes only docs/spec/taste/ — no app, backend, CI, or deploy code.

What's here

• SPEC.md — normative math (§2), worked golden fixture (§3), statement model (§4), privacy/household (§5), verification (§6), alternatives (§7), scope (§8), adversarial review + independent-review response (§9), verification record (§10). Opens with the load-bearing assumption: bounded residuals must improve pairwise order in replay without moving top-10 composition more than a few percent, or the data is too thin to learn from at all.
• contracts/*.schema.json — four JSON Schema (draft-2020-12 subset) contracts; the event schema validates per-kind payload branches through allOf/if/then with additionalProperties: false.
• contracts/examples/ — 19 canonical examples: one per event kind (12), one exposure, five statement variants, and the golden fixture coriander-v1.json (three rated coriander cooks, an unrated cook + recipe_rated merge, a save, a skip, an active pin, and a featureless 60.0 ranking tie).
• contracts/check.py — stdlib-only checker: schema validation, canonical serialization of the committed bytes, calendar-valid whole-second UTC timestamps, 6-decimal float scans, exposure/slate-position cross-references, statement evidence gates, a required-example manifest, and a full recompute of the golden fixture's expected state (byte-compare at 6 decimals).
• contracts/gen_examples.py — regenerates every example; implements the model independently from the checker against the same constants.
• contracts/mutation_matrix.py — 52 adversarial mutations, each required to fail the checker, gated by copy and positive controls.

Why the second commit

An independent review of ef37a82 returned nine findings; all nine landed as contract changes with a corresponding mutation case (per-kind payload validation, loud failure on unconsumed evidence events, one GLOBAL_CLAMP over pins + learned sums with the raw contribution preserved, hedged/confident statement over-claim gates, one canonical-serialization wording, corrected schema/spec references, household-size cap removal, calendar-valid timestamps + document-wide float scans, and an exercised ranking tie-break). §9.1 of the SPEC maps each finding to its disposition and verification. Two further invariants came out of the fix round itself: a required-example manifest, and a canonical (at, id) input-log order for committed fixtures — which immediately caught an out-of-order regeneration of the fixture.

How I checked

• python3 docs/spec/taste/contracts/gen_examples.py — regenerates all 19 examples; fixture values: ingredient:101 alpha 0.495379 / beta 1.126520, contribution −0.546217; pin-bound applied contributions −8.0 with raw −8.646369 / −8.546217; ranking totals 60.0 / 60.0 / 59.075155 / 55.0 / 53.0.
• python3 docs/spec/taste/contracts/check.py — checked 19 examples … all examples valid and self-consistent, exit 0.
• python3 docs/spec/taste/contracts/mutation_matrix.py — 52 cases: every mutation caught, the size-4-household control validates, the untouched-copy positive control passes, exit 0.
• Reproduced the review's baseline failures on ef37a82 before fixing (wrong-kind payload fields, malformed timestamp, over-precision float, over-claiming hedged statement all passed the old checker; household size 4 was rejected; an unmergeable recipe_rated was silently ignored).
• git log / git status reviewed before commit: 15 files under docs/spec/taste/ only.

What I could not check

• No Swift/TypeScript implementation exists yet — this PR is spec + contracts only. The fixture's parity role (Swift and TypeScript must reproduce it byte-for-byte via decimal-fixed formatting) is what implementation PRs must satisfy.
• No Xcode/macOS here: hosted macOS CI is not expected to exercise these files beyond docs-level checks.
• Full draft-2020-12 conformance: check.py implements the subset the schemas use; a full validator was deliberately not added as a dependency.

Assumptions

• Feature keys (ingredient:101 = coriander, cuisine:indian) are illustrative fixture identifiers, not catalog IDs to resolve.
• The fixture's evaluation instant is its last event's timestamp, keeping the expected output deterministic.
• "Reduced campaign" NDCG figures are cited from recorded PR evaluation reports, not re-measured here.

🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: af9830d0-af1a-4621-b163-3ade4aa23e49

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[obvious-autobuild]

Independent review (Obvious, Autobuild + Max)

Verdict: changes needed — as a draft, this is close: the normative math verified correct against an independent reimplementation, the checker genuinely bites on every expected section, and CI is green. But §6's coverage claims outrun what the contracts actually check, and two normative-math inconsistencies (GLOBAL_CLAMP semantics, hedged-statement rule) would force implementation PRs to guess on parity-critical behavior. All fixable with small spec/checker edits; no existing code touched, no regression surface (adds files only).

Findings (ranked by severity)

1. Medium — payload is not kind-discriminated; SPEC.md:102 claims it is.
taste-event.schema.json:63 declares payload as a bare {"type": "object"} — no per-kind properties, no additionalProperties, no required payload fields. SPEC.md:102 says the schema covers "12 event kinds, discriminated by kind with per-kind payload properties". Demonstrated: putting the lechef_feedback payload (tag/note/cookEventId) into examples/events/recipe_saved.json → check.py exits 0; adding an unknown sneakyExtra field to a fixture-input cook payload → exits 0. (The schema's own "unknown fields are dropped on ingest" note defends unknown-field openness, not wrong-kind payloads being valid.)
Smallest fix: add per-kind payload subschemas (if/then or oneOf on kind) — note check.py's validator implements neither keyword, so schema and checker must change together — or drop the "per-kind payload properties" claim from SPEC.md:102 and say payload shapes are enforced in implementation PRs.

2. Medium — the reference model ignores every non-recipe_cooked evidence kind, while SPEC.md:100 says "Everything in §2–§4 is pinned by machine-checkable contracts".
check.py:145 and gen_examples.py:151 continue past all non-cook kinds. Demonstrated: adding a recipe_rated (rating 5, cookEventId = cook 1) at the last cook's timestamp to the fixture input → check.py exits 0, i.e. the rating's content is silently ignored, although SPEC.md:79 says such an event "updates the same posterior at strength 1.0 with its own timestamp". Also SPEC.md:123 claims "the checker's fixture uses this path" for the cookEventId merge — the fixture's three cooks carry embedded ratings; no recipe_rated event exists in it. Saves, skips, substitutions, lechef, unsaved reversal, exposure resolution (§2.4) and correction replay (§4) are schema-shaped only, never behavior-pinned.
Smallest fix: implement at least the recipe_rated-merge and one save/skip path in both scripts and extend the fixture, or scope SPEC.md:100 (and the PR body) to the cooked path; either way, make recompute raise on non-cook kinds so future fixtures can't silently skip them.

3. Medium — GLOBAL_CLAMP is specified two inconsistent ways, and the checks enforce neither effective semantics.
SPEC.md:41 defines it as "Max |Σ contributions| per recipe"; SPEC.md:61 clamps per feature — where it can never bind, since |d·c·CAP| ≤ CAP ≤ 4.0 < 8.0 (dead code). check.py:178 mirrors the dead per-feature clamp and then sums contributions unclamped; gen_examples.py:181 applies no clamp at all; SPEC.md:126's pinned-value disposition ("at most a couple of pins fit before the clamp saturates") presumes the per-recipe sum clamp exists. Until pins exist, 3+ features at class caps can sum to ±8–12 unclamped, so this is reachable.
Smallest fix: clamp the Σ in the totalScore formula (drop or annotate the per-feature clamp), implement the sum clamp in both scripts, and keep a fixture case where it binds.

4. Medium — the hedged-statement rule in §4 contradicts §9's disposition, and check.py implements the over-claiming variant.
SPEC.md:89: "hedged wording below 10 denominators". SPEC.md:125: "hedged template is reserved for numerator = denominator" (the anti-over-claim disposition). check.py:225 implements hedged = denom < 10 regardless of numerator. With numerator 2 / denominator 3 (ratio 0.67 ≥ 0.6) it renders "You seem to go off X: the last 2 recipes you cooked with it were rated 2 stars or below" — exactly the lying sentence §9 promises to prevent. The fixture hides this because its numerator equals its denominator (3/3), so both readings pass.
Smallest fix: pick one rule, reconcile §4/§9, align check.py:225 and gen_examples.py:317.

5. Medium — three inconsistent definitions of the canonical form the parity contract depends on.
golden-fixture.schema.json:5 promises comparison "byte-exact after canonical serialization: floats with exactly 6 fractional digits … no whitespace"; the committed fixture is json.dump(indent=2, sort_keys=True) where floats print as 60.0 (indented, 1-decimal); check.py:304-330 compares in-memory round(v, 6) structures via json.dumps with default separators. So "canonical 6-decimal floats" has three meanings (checker: round-to-6; schema: exactly-6-fractional-digits; file: neither). SPEC.md:131 requires implementations to reproduce the fixture "byte-for-byte" — of which form?
Smallest fix: pin one serialization in §6 (e.g. %.6f floats, sorted keys, fixed indent) and make check.py compare serialized strings, not parsed structures.

6. Low — contract/SPEC drift in enums, pointers, and decay constants.

• profile-statement.schema.json:20 includes layer "stated"; SPEC §4 and the PR body describe three layers (restriction/inferred/pinned).
• hiddenStatements.reason enum has 5 values (rejected, pinned-opposite extra) vs SPEC.md:89's three hidden reasons.
• taste-event.schema.json:21 points to "SPEC.md section 7" for the event catalogue — §7 is "Alternatives considered"; no section enumerates payloads-by-kind and emitting surfaces.
• profile-statement.schema.json points to "the template catalogue in SPEC section 4" — §4 defines no template catalogue.
• Ranking-time and staleness decay hardcode cook_rating's 60-day half-life (check.py:169, check.py:217) while §2.2 "decay each feature to the evaluation instant" and §4 "3 × class half-life" never say which half-life applies when a feature mixes evidence classes; the scripts silently pick one resolution.

7. Low — householdSize maximum 2 is unjustified.
exposure.schema.json:71. SPEC §5 explicitly contemplates multi-eater households; nothing in the repo constrains household size to 2 (no other reference found). A 3-person household's exposure log is contract-invalid. Drop the maximum or document the bound.

8. Low — timestamp pattern accepts a malformed form; canonical checks narrower than §6 states.
The at patterns in both schemas and UTC_RE (check.py:23) use (:?\.[0-9]{3})? — optional colon before .ddd; intended was the non-capturing (?:\.[0-9]{3})?. Demonstrated: mutedUntil: "2027-02-08T10:00:00:.789Z" on a statement example → exits 0. And the canonical-float check runs only on the golden fixture (check.py:304): ratio: 0.123456789 in an event example → exits 0, while SPEC.md:104 reads as enforcing it across examples.

9. Low — SPEC.md:124 "the golden fixture pins the tiebreak" doesn't hold. The fixture's events have distinct timestamps and no two recipes tie on totalScore, so neither the (at, id) event tiebreak nor the rank tiebreak is exercised. Add a tie or soften the claim.

What I verified (commands and results)

• Fresh clone, checked out head ef37a827a259de31c5d545330e1957bdc1570552 (matches PR head); read AGENTS.md, .obvious/skills index, PR template, ios-ci.yml.
• python3 docs/spec/taste/contracts/check.py → checked 17 examples … all examples valid and self-consistent, exit 0 — matches the body.
• python3 docs/spec/taste/contracts/gen_examples.py → prints alpha 0.750847 / beta 1.627927 / contribution −0.652252 / totals 62.240527, 60.347748, 60.0, 55.0; git status --porcelain clean afterward — regeneration is deterministic and byte-identical to the committed fixtures.
• Negative matrix on checker copies (12 mutations): corrupting any expected section (totalScore +0.5 → ranking mismatch; alpha +0.01 → featureState mismatch; numerator → profileStatements mismatch; hidden reason → hiddenStatements mismatch) exits 1, as does dropping a required field and householdSize: 4. The recompute comparison genuinely bites; the body's claimed negative test reproduces.
• Independent reimplementation of §2.2 written from the spec text only (not from the PR's code): all posteriors, contributions, and ranking totals match the fixture at 6 decimals; every pinned float sits ≥ 3.6e-3 from a 7th-decimal half boundary, supporting §9's "avoid exact-half cases" parity claim.
• Real jsonschema 4.26.0 draft-2020-12 validator (local $id registry): 4 schemas structurally valid, 17/17 examples valid — agrees with check.py. Portability note: standard validators resolve the sibling $refs against the https://fridgeluck.dev/... $ids and fail offline unless all four schemas are pre-registered by $id; check.py resolves $refs as local file paths. Fine for the checker; worth one sentence for Swift/TS implementers.
• Body citations: the ~11% NDCG@10 figure traces to PR FLTaste suite-B evaluation campaign + scorecard wiring (learning) #115's executed campaign (engine vs rule-port −11.15% [−12.18%, −10.08%], negative on all 6 seeds, activation gate NOT MET; trust-vector second) — accurate as cited. PR feat(evaluation): add taste-lab — taste-residual-v1 reference engine, simulator, and offline ranking evaluation #149's taste-lab replay (+0.0059 NDCG, ~4% top-10 churn) additionally supports §0's post-hoc evidence criterion. "25 files / 17 fixtures / one statement per layer" all match the tree. gh pr checks 146: build-and-test pass (7m24s), dead-code-advisory pass (3m1s) — consistent with "hosted macOS CI won't exercise these files".
• Diff touches only docs/spec/taste/ (25 files, all additions); no CI/deploy/workflow changes. Housekeeping: the PR body is missing the workspace-required Obvious footer (sibling feat(spec): recipe bundle-v2, ingredient identity, and taste-event contracts #148 also lacks it).

What I couldn't verify

• Swift/TypeScript parity — no implementation exists (the body discloses this); the golden fixture's parity role is untested by definition.
• The raw evaluation artifacts behind the −11.15% figure — I verified the recorded claim in PR FLTaste suite-B evaluation campaign + scorecard wiring (learning) #115's body, not the run data (packages/FLTaste/Eval/RESULTS.md lives on FLTaste suite-B evaluation campaign + scorecard wiring (learning) #115's branch, not alpha).
• Whether the stated layer and the two extra hidden-reason enum values are deliberate forward-compat for sibling burst5 specs — if so, a sentence in SPEC §4 resolves finding 6.
• Behavior of the 0.15 exposure-share floor at large feasible sets — unexercised by the fixture (all shares ≥ 0.25).