Skip to content

feat(lechef): bounded tool layer, fake phone/model, and 65-session scripted suite - #156

Draft
obvious-autobuild[bot] wants to merge 2 commits into
obv/burst5-fridgeluck-spec-lecheffrom
obv/burst5-fridgeluck-lechef-tools
Draft

obvious-autobuild[bot] wants to merge 2 commits into
obv/burst5-fridgeluck-spec-lecheffrom
obv/burst5-fridgeluck-lechef-tools

Conversation

@obvious-autobuild

@obvious-autobuild obvious-autobuild Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

What changed and why

Implements the LeChef tool layer for backend/gemini-agent (spec packet burst5-fridgeluck-lechef-tools, base branch obv/burst5-fridgeluck-spec-lechef), using that base branch's contract schemas (docs/spec/lechef/contracts/) as the wire-format authority.

  • src/lechef/types.ts + wire.ts — nine proposal kinds, strict payload shapes, ISO-8601 millisecond timestamps, canonical SHA-256 payload hashing (payloadHash), proposal/receipt parse/serialize. Proposal ids are content-derived (base36 clock + per-runtime counter): deterministic under the manual clock and unique within a runtime; two fresh runtimes under identical clocks can generate identical ids, so cross-restart uniqueness relies on relaunch carrying prior receipts — the phone's receipt dedupe is the safety net, and an equal id is always treated as a replay (never re-executed).
  • src/lechef/proposals.ts — the nine proposal tools. The phone is the authority: explicit approval is the default; device-auto requires a timer citing a fresh (unconsumed, current-revision) touch/voice device event. meal_log is never device-auto, even with a voice event. Rejections come back as submitted:false with an issues list — nothing throws across the model boundary.
  • src/lechef/reads.ts (+ foodSafety.ts, substitutions.ts, allergens.ts, fit.ts, cameraLint.ts, canon data) — six honest read tools. Food-safety answers are sourced checks (with a temperature only where the reviewed canon covers the product) or explicit unknowns. Substitution rechecks run fail-closed against the eater union; unknown composition blocks the swap. Camera looks lint the observation: the lint is a closed list of verdict phrases plus two construction classes (verb+verdict co-occurrence like "looks safe"/"is done", and temperature+safety-predicate co-occurrence) — deliberately narrow, not a general verdict detector; visual descriptions pass, verdict constructions are refused and never relayed to the model.
  • src/lechef/fakePhone.ts — device-side authority: proposals stage as pending and resolve on device approval/rejection, TTL (300 s) is checked at resolution time, receipts survive kill-and-relaunch (dedupe by proposalID — a resubmitted proposal returns the existing receipt unchanged; the executed receipt is the audit record and is never overwritten or re-executed), device events are consumed on use, and goOffline() marks the session offline (camera looks send zero frames).
  • src/lechef/suite/ + src/run/lechef-suite.ts — a scripted fake model, a 65-session matrix (5 jobs × 13 situations: baseline, offline, mic-denied, live-disconnect, app-kill-relaunch, empty-kitchen, allergen-conflict, partial-data, expired-proposal, stale-revision, voice-authorized-timer, duplicate-call, unknown-product), a runner, and a scorer with hard gates. The launcher prints the deterministic report.
  • src/__tests__/lechef/lechefTools.test.ts — 36 portable tests: wire hashing/TTL, the proposal flow (device-auto, explicit-only meal log, TTL expiry, relaunch and same-phone resubmission dedupe, consumed-event refusal), read honesty (order-independent reads, explicit unknowns), the reviewed allergen hard-block paths, camera-lint verdict constructions, offline camera behavior, and the committed suite gates.
  • README.md — a LeChef section documenting the runtime and the four verification commands.

The spec's §15 slice layout (src/agent/, src/contracts/, src/evaluation/, ≥155 scripts) was superseded by the base branch's contract re-cut; the tool layer lives at src/lechef/ with tests at src/__tests__/lechef/ per the task brief, and the suite runs 65 sessions covering the same five jobs and safety assertions.

Review fixes (commit 61926db)

An independent review of this PR found six issues; all are fixed or corrected. Red-first: 9 tests failed on the pre-fix code and pass after; each fix is also pinned by the committed suite.

  1. Allergen hard-block had no live coverage (high). The allergen-conflict sessions used out-of-vocabulary restriction spellings ("wheat-free"), so they exercised the fail-closed confirmation branch, not the reviewed hard-block; the scorer also passed on any lookupSubstitution call. Now the sessions use reviewed in-vocabulary eaters ("wheat", "sesame"), substitution-adaptation looks up soy sauce (reviewed compound fact: carries wheat — every table row is recheck-blocked), and the scorer requires the block to have fired: deleting or stubbing recheckSwapAllergens now fails substitution-adaptation:allergen-conflict (mutation-verified; the same mutation passed 65/65 before).
  2. The unauthorized-actions gate could not count unauthorized execution (medium). The old loop whitelisted every resolution value the runner could produce, so the count was structurally zero. It now cross-checks each resolution against its receipt: an executed receipt resolving as neither approved-by-user nor device-auto — and an approval resting on a non-executed receipt — counts as unauthorized. Unit tests cover both directions and the consistent zero.
  3. Resubmission overwrote the executed receipt (medium). The dedupe path returned a fresh rejected/already_resolved receipt, replacing the executed audit record; the relaunch test never resubmitted. Now the dedupe returns the existing receipt unchanged, and tests resubmit the same proposal wire on the same phone and across a relaunch (same receipt, recordedAt unchanged, no re-execution).
  4. Camera-lint claims exceeded its patterns (low-medium). The module header claimed narrow pattern coverage while the doc comment over-promised; "looks safe", "is done", "smells spoiled" and colour-based doneness text ("no longer pink") were not refused. Added a verb+verdict construction class and the colour phrase; the header now states the actual scope: closed phrase list plus two construction classes, deliberately narrow, visual descriptions pass.
  5. Offline / mic-denied / duplicate-call situations did not exercise their named behaviors (low). goOffline() was a no-op that only bumped a revision counter; mic-denied recorded nothing and dropped nothing; duplicate-call only duplicated a read. Now offline really takes the device offline (session snapshot reports it, camera look sends 0 frames, the read reports an explicit unknown, and the scorer asserts the unknown), mic-denied really drops the attempted voice device event (counted in the run record; the timer still executes by explicit approval), and duplicate-call resubmits the same proposal wire after approval for every job.
  6. PR-body inaccuracies (low). Test counts, the situation list (a "swap-reviewed" situation does not exist; unknown-product was missing), and the proposal-id uniqueness claim were corrected in this body: 334 tests across 21 files (298 pre-existing + 36 in the LeChef file), the 13 real situations are listed above, and id uniqueness is stated as per-runtime with the receipt dedupe as the cross-restart safety net.

How I checked it

All inside backend/gemini-agent unless noted; everything deterministic, zero network. Numbers are for head 61926db:

  • bun x tsc -p tsconfig.json --noEmit — clean.
  • bun test src/__tests__ — 334 pass, 0 fail across 21 files (298 pre-existing + 36 in the LeChef file).
  • bun run src/run/lechef-suite.ts — 65 sessions; 65/65 job successes, 0 unsafe answers, 0 unauthorized actions; gates unsafeAnswersZero=true unauthorizedActionsZero=true allSessionsRan=true.
  • python3 docs/spec/lechef/contracts/check.py (repo root) — 20 checks over 6 schemas and 8 fixtures, all valid.
  • Red-first evidence: before the fixes, 9 of the new tests failed (dedupe ×2, camera constructions ×2, offline ×2, unauthorized gate ×2, mic-denied ×1); all pass after.
  • Mutation check: stubbing recheckSwapAllergens to return [] fails substitution-adaptation:allergen-conflict (64/65, failure: "expected the allergen recheck to block the suggestion"); the identical mutation passed 65/65 on the pre-fix code.
  • Determinism: running the same script twice produces identical receipts and final state (asserted in the test file).

What I could not check

  • No Xcode/macOS here, but this PR touches only Bun/TypeScript, Python (unchanged), and docs — nothing native to build. The repo's macOS CI runs on the PR (green on the pre-fix commit; re-run queued on 61926db).
  • No live Gemini transport is involved; the fake model stands in for the model loop by design, so real-model prompt behavior is out of scope for this packet.

Assumptions

  • Base-branch contract schemas and examples are the wire authority (they differ from PR feat(lechef): shared LeChef tool layer — phone-authoritative proposals, read tools, eval harness (Part 1, M1) #102's original wire shape in timestamp format and kind set).
  • Suite launcher at src/run/lechef-suite.ts sits outside the strict src/lechef/+src/__tests__/lechef/ test scope to give the committed suite a stable entry point; everything else is inside those two directories.
  • An executed swap whose receipt carries an "unsupported" allergen-recheck verdict is scored as honest fail-closed information (the user explicitly approved the action), not as an unsafe answer; unsafe answers are model-utterance verdicts and unrefused verdict-shaped camera text.
  • Draft PR only; not merged.

🔗 Obvious Project · 🧵 Obvious Thread

…ripted suite

Implement the LeChef assistant tool runtime in backend/gemini-agent per
the base-branch contract schemas (docs/spec/lechef/contracts/):

- wire.ts/types.ts: nine proposal kinds, strict payload shapes, ISO-ms
  timestamps, canonical SHA-256 payload hashing, proposal/receipt
  parse/serialize; content-derived proposal ids (deterministic under
  the manual clock, unique across sessions)
- proposals.ts: nine proposal tools; the phone is the authority —
  explicit approval, or device-auto only for a timer citing a fresh
  touch/voice device event; meal_log is never device-auto; rejections
  return issues, never throw across the model boundary
- reads.ts: six honest read tools; food-safety answers are sourced
  checks or explicit unknowns; camera looks refuse verdict-shaped text
  (fail closed on lint findings); substitutions fail closed on unknown
  composition with eater-union allergen rechecks
- fakePhone.ts: staged approvals, 300 s TTL at resolution, receipt
  dedupe across kill-and-relaunch, one-shot device-event consumption
- suite/: scripted fake model, 65-session matrix (5 jobs x 13
  situations), runner, scorer; committed launcher src/run/lechef-suite.ts
- tests: 20 portable tests incl. contract-fixture round-trips porting
  contracts/check.py expectations

Verification: bun run check clean; bun test src/__tests__ 318 pass;
suite report 65/65 job successes, 0 unsafe answers, 0 unauthorized
actions, all gates true; contracts/check.py 20/20 OK. All deterministic,
zero network.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6f1cabbf-33f2-43d9-bb17-d9245969e039

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@SamGu-NRX

Copy link
Copy Markdown
Owner

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review skipped.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@obvious-autobuild

Copy link
Copy Markdown
Author

Independent review (Obvious, Autobuild + Max)

Verdict: changes needed. The tool layer itself held up under everything I threw at it — I mutation-tested the two core safety rules and both are genuinely enforced, and all four verification commands in the body reproduce exactly, with CI green on b8ee5ad. But the tests prove less than the body says they prove: the allergen hard-block is never exercised anywhere, the unauthorized-actions gate cannot count anything by construction, and several body claims don't match the code. Findings are all cheap to fix.

Findings (by severity)

1. [High — the core safety property is untested] The reviewed allergen hard-block has zero coverage in the suite or the unit tests.
allergens.ts has two distinct fail-closed paths: out-of-vocabulary restrictions (confirmation required) and reviewed contains blocks. The suite's eaters are ["wheat-free"] / ["sesame-free"] (src/lechef/suite/scripts.ts:113-114), and neither spelling is in the reviewed vocabulary, so every allergen recheck in all 65 sessions takes the out-of-vocabulary branch. Verified by probe:

  • allergenViolations("olive oil", [{restrictions:["wheat-free"]}]) → "restriction \"wheat-free\" is not in the reviewed restriction vocabulary — confirmation required..."
  • the real compound path works but is never tested: allergenViolations("soy sauce", [{restrictions:["wheat"]}]) → "soy sauce carries soy, wheat" (the exact soy-sauce-contains-wheat case SPEC 3.3 highlights).
  • the new test file has no allergen test at all, and the scorer's allergen-conflict assertion passes via a fallback that is always true — blocked || run.calls.some((c) => c.name === "lookupSubstitution") (src/lechef/suite/scorer.ts:120) — so it would still pass if the recheck were deleted.
    Smallest fix: one unit test with in-vocabulary restrictions ("wheat" vs soy sauce → violation; "sesame" vs tahini → violation; a reviewed free state clears), and drop the || lookupRan fallback so the allergen-conflict session actually requires a block.

2. [Medium] The unauthorizedActionsZero gate is dead by construction.
In scoreRun (src/lechef/suite/scorer.ts:153-158), every possible resolution value — approved-by-user, device-auto, expired, rejected-by-user, rejected-by-phone, still-pending — hits a continue, so unauthorized can never exceed 0. The body's "0 unauthorized actions" is guaranteed, not measured. Smallest fix: count a receipt in state executed whose resolution is neither approved-by-user nor device-auto (drop the blanket continue for the non-executed states), or document the gate as structural.

3. [Medium] Dedupe overwrites the executed receipt — and the test named for it never resubmits.
submitProposal's already-resolved path (src/lechef/fakePhone.ts:196) writes a rejected/"already_resolved" receipt keyed by the same proposalID, replacing the prior receipt. Probe: submit → approve → receipt executed; resubmit the identical wire → receipt is now rejected. Replay is still impossible (the safety property holds), but the audit record of execution is destroyed and the phone does not "return the already-resolved receipt". The test receipts survive a kill-and-relaunch: resubmitting the same proposal resolves at most once (src/__tests__/lechef/lechefTools.test.ts:100) only asserts the receipts map carried over — it never resubmits anything. Smallest fix: on the dedupe path return the existing receipt unchanged; in the test, resubmit the same wire to the relaunched phone and assert the returned receipt is still the executed one.

4. [Low-Medium] Camera-lint refusal is narrower than the absolute claim.
VERDICT_PATTERNS (src/lechef/cameraLint.ts:26) is a closed list of 17 substrings plus a temperature+safety-predicate co-occurrence rule. Verdict-shaped text outside the list relays to the model: "the pork looks safe", "chicken is done", "no longer pink" all pass. The module header says "deliberately narrow", but the body and README state the refusal without the qualifier. Also note the scorer's unsafe-camera check re-uses the same lintCameraObservations, so unsafeAnswersZero cannot independently catch a weak lint — when I disabled the lint entirely in a mutation run, only the cameraRefusals job assertion caught it. Smallest fix: add a verb+verdict pattern class (e.g. \b(is|looks|seems)\s+(done|safe|edible|cooked|spoiled)\b) or qualify the body/README claim to the closed list.

5. [Low] Suite situation labels outrun their behavior.

  • offline (scripts.ts:276) does bumpRevision + approveAll — behaviorally identical to stale-revision. FakePhone.goOffline() is a no-op (src/lechef/fakePhone.ts:130-132, this.cameraMode = this.cameraMode), sessionSnapshot() hardcodes offline: false (fakePhone.ts:160), and the runner never calls goOffline() — so the offline read-path (e.g. cameraLook's framesSent: 0 handling) is never exercised.
  • mic-denied (scripts.ts:280) is behaviorally identical to baseline.
  • duplicate-call (scripts.ts:318) duplicates only a getPantry read in one job; no session ever submits a duplicate proposal, so these sessions do not demonstrate "receipts resolve at most once".
    Smallest fix: wire offline through the phone (the field exists), script one duplicate proposal submission, and give mic-denied a mic-gated path or rename it.

6. [Low] Body inaccuracies.

  • Test counts: the body says "318 pass ... (309 pre-existing + 20 new)". Measured on the base branch (615fc1b): 298 pass across 20 files; head: 318 across 21 files. 298 + 20 = 318, not 309 + 20.
  • The body's 13 situations list "swap-reviewed"; the code's 13th situation is "mic-denied" (scripts.ts:175-189) — "swap-reviewed" does not exist in the matrix.
  • "Proposal ids ... unique across sessions" holds only under a real clock: every suite session's ManualClock starts at the same instant, so ids repeat across sessions (harmless — receipts are per-phone — but the claim as stated is production-only).

What I verified (commands and results)

  • Read the full PR body, all comments (CodeRabbit review was skipped — no seat; no human reviews), and the full diff (24 files, all additions, no workflow/CI changes; single commit on top of base tip 615fc1b with a clean merge-base).
  • CI on the exact head SHA: gh run list --commit b8ee5ad → iOS CI success (build-and-test pass 6m33s, dead-code-advisory pass).
  • Checked out b8ee5ad and ran all four commands from the body, all matching:
    • bun run check → exit 0.
    • bun test src/__tests__ → 318 pass / 0 fail, 21 files.
    • bun run src/run/lechef-suite.ts → 65/65 job successes, unsafeAnswersZero=true unauthorizedActionsZero=true allSessionsRan=true, exit 0.
    • python3 docs/spec/lechef/contracts/check.py → "20 checks over 6 schemas and 8 fixtures — OK".
  • Negative control: worktree at base 615fc1b → bun test src/__tests__ = 298 pass / 20 files; copying the new test file in → 0 pass / 1 fail (module not found): the new tests are bound to the new code.
  • Mutation 1 (fakePhone.ts: isTimer = true, making every kind device-auto-eligible): exactly the two meal-log authorization tests fail — "a meal_log is NEVER device-auto" and "an explicitly approved meal_log executes". Restored; suite green again.
  • Mutation 2 (cameraLint.ts: lint returns []): "committed suite > every session succeeds its job assertion" fails (live-disconnect refusal). Restored; 20/20 green.
  • Read-only probes quoted above for the out-of-vocabulary vs in-vocabulary allergen paths and the dedupe receipt overwrite.

What I couldn't verify

  • Real-model behavior: the fake model stands in by design, so prompt-level honesty is out of scope for this packet (the body says the same).
  • Nothing iOS/native is touched, so there was nothing to build locally without Xcode; the hosted macOS CI ran green on the head SHA.
  • Canon provenance: I read the canon README's provenance claims but did not diff the 16k/18k-line canon JSONs against their overhaul sources; check.py validates structure and temperatures, not verbatim sourcing.
  • Determinism was verified as a same-machine double-run (also asserted in the tests), not across machines.

The layer itself — phone-as-authority, TTL checked at resolution, hash verified at submission, fail-closed reads — is solid. With findings 1–3 fixed and the body corrected, this is in good shape.

🔗 Obvious Project · 🧵 Obvious Thread · 🔀 Obvious PR Artifact

obvious-autobuild Bot added a commit that referenced this pull request Oct 15, 2026
…enforced hash round-trip

Finding 1 (HIGH): proposal.schema.json now carries snapshotRevision
(required, integer >= 1) so §8.1's revision-binding rule has a wire field;
rationale and requiredApproval become optional (absent from #102's wire
and its frozen fixture); schema title/description state the §8.1 rename
map instead of claiming verbatim adoption. SPEC §8 intro/§8.1/§8.3/§14
reworded from 'adopted verbatim' to 'adopted and re-cut', listing #102's
actual camelCase payload kinds (timerStart..note) with the map
(top-level snake_case, note->save_note, id->prop_-prefixed proposalID,
sha256:-prefixed hash). The re-cut is kept rather than reverting to
#102's raw envelope because PR #156 is already cut to these contracts.

Finding 2 (LOW): §8.3's corpus-naming claim replaced — the three
application kinds are named by this spec's registry; #102's corpus and
ProposalTool union contain no plan/swap/adaptation kind strings.

Finding 3 (LOW): check.py now recomputes payloadHash = sha256 over the
compact canonical payload for every proposal fixture (was set-membership
only), matches receipts to proposals by proposalID, fails loudly on an
unknown $defs ref, and fires the shoppingList error only when short
lines exist. New cross-check: #102's frozen action-proposal.json fixture
(added verbatim) is mapped through the §8.1 rename map and must validate
against proposal.schema.json.

Fail-before-fix evidence in PR body; check.py 23/23 green.

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
…thorized counter, receipt dedupe, camera constructions, offline/mic-denied/duplicate-call

Fixes every finding of the independent review of #156:

1. Allergen hard-block (high): allergen-conflict sessions now use
   in-vocabulary eaters ("wheat", "sesame") and substitution-adaptation
   looks up soy sauce, whose reviewed compound fact (carries wheat) must
   hard-block every table row; the scorer's '|| lookupRan' fallback is
   gone, so a deleted or weakened recheck fails the session. New unit
   tests pin the reviewed contains/clears/out-of-vocabulary paths.
2. Unauthorized gate (medium): scoreRun now cross-checks every
   resolution against its receipt — an executed receipt resolving as
   neither approved-by-user nor device-auto (and the mirror case) counts
   as unauthorized. The gate is measured, not structural.
3. Receipt dedupe (medium): resubmitting an already-resolved proposal
   returns the EXISTING receipt unchanged — the executed receipt is the
   audit record and is never overwritten by a rejected/already_resolved
   receipt. Tests actually resubmit the same wire, on the same phone and
   across a relaunch.
4. Camera lint (low-medium): adds a verb+verdict construction class
   (looks/is/seems/smells + done/safe/edible/cooked/spoiled/bad/ready)
   and a 'no longer pink' colour-verdict phrase; module header and docs
   state the closed-list-plus-constructions scope.
5. Situation behavior (low): offline really takes the device offline
   (session snapshot reports it; camera look sends 0 frames and the read
   reports an explicit unknown — scorer asserts it); mic-denied really
   drops the attempted voice device event (counted in RunRecord, timer
   executes by explicit approval); duplicate-call re-submits the same
   proposal wire after approval for every job.
6. Body corrections recorded in the PR description (test counts,
   situation list, id-uniqueness scope).

Verification: tsc clean; bun test src/__tests__ 334 pass / 0 fail;
suite 65/65 job successes, 0 unsafe, 0 unauthorized; contracts/check.py
20/20. Red-first evidence: 9 tests failed before the fixes (dedupe x2,
camera constructions x2, offline x2, unauthorized x2, mic-denied);
mutation recheckSwapAllergens -> [] now fails
substitution-adaptation:allergen-conflict (passed 65/65 before).

Co-authored-by: obvious-autobuild[bot] <262744130+obvious-autobuild[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant