[SamGu-NRX]

What changed

Adds the opt-in live-model path for the LeChef three-design comparison. The offline harness replays scripted turns — the script IS the provider, so runDesignSession has no model seam to inject into. This PR therefore adds a new src/lechef/live/ module and a src/lechef/run/live.ts CLI with their own session loop, reusing the provider-independent seams unchanged: the per-arm design configs, wrapWithPolicy tool gating, buildSessionWorld, and the pinned RunRecord assembly surfaces.

New files (no existing src/lechef/*.ts is edited):

• live/types.ts — the LiveTransport injection contract (request, response, provider usage), LiveCaps, TokenPrices, CostEstimate. No provider SDK, no key handling in types.
• live/pricing.ts — conservative token estimates and per-model prices. Unknown model pricing fails closed: a run cannot silently cost zero.
• live/caps.ts — CapLedger enforcing both LECHEF_LIVE_MAX_SPEND_USD and LECHEF_LIVE_MAX_REQUESTS (whichever binds first) BEFORE each request. A request that would breach a cap is never sent; abort is clean and reports cap state.
• live/config.ts — env-only key resolution (LECHEF_LIVE_API_KEY, provider/model/caps params). Missing key, unsupported provider, unknown pricing, and invalid caps all fail closed.
• live/logs.ts — one append-and-flush JSONL per (arm, session) under the run directory, plus a run summary writer. Entries carry wall-clock timestamps, model, request sizes, estimated cost, and cap state — never request content, never the key.
• live/liveSession.ts — slot-driven session loop: source user turns drive context, generated assistant turns replace scripted ones at source assistant slots, tool calls run through the same wrapWithPolicy gating, approval actions map FIFO to open proposals.
• live/geminiTransport.ts — the Gemini adapter: pure, unit-tested request/response mappers (a structured {text, toolCalls} schema matching the offline scripts' response shape) and an SDK-backed send path constructed only by the explicit live command. LECHEF_LIVE_ENDPOINT overrides the base URL (version suffix stripped) — the documented localhost rehearsal channel.
• run/live.ts — the CLI. Plan mode (default, keyless) prints arms/slots/caps/prices and the exact live command. Live mode (--mode=live) requires the key in the environment, runs the three arms, writes per-session JSONL + RunRecord JSON + summary.json, aborts cleanly at a cap, and exits 1 on a cap breach or when zero sessions complete.
• README-live.md — runbook: env contract (including default caps), outputs, sessions input, rehearsal channel. backend/gemini-agent/.gitignore keeps run outputs out of git; package.json gains lechef:live.

Why

PR #102's review found the three design-arm scorecards not-run because sessions were missing, and asked for the real comparison. Running it needs a real model reachable through an injected seam that does not exist today, with spending bounded and every exchange logged. This PR builds that seam so the comparison becomes one command with an explicit key and hard caps — and is never triggered implicitly. A key appearing in the environment later does nothing; live is an explicit command.

How it works

flowchart LR
    src[source script\nuser turns + slots] --> loop[live session loop\nwrapWithPolicy gating]
    loop -->|request| ledger[CapLedger\nspend + requests]
    ledger -->|authorized| tx[LiveTransport\ninjected — Gemini adapter in live mode]
    tx -->|response + usage| loop
    ledger -->|would breach| abort[clean abort\ncap state]
    loop --> log[per-session JSONL\n+ run summary]

Loading

The caps ledger authorizes every request before the transport fires and charges the pre-request estimate at send time, so a crash between send and response still counts the attempt.

Verification

Checks run in the sandbox (no CI or workflow edits). Branch head ef59034:

• cd backend/gemini-agent && bun run check — clean (tsc, zero errors), before and after the overhaul/lechef-f1 merge (da04d52; the Direction-3 integration gate passed at 528/0 with the merged tool layer).
• bun test src/__tests__ — 543 pass / 0 fail across 45 files: the merged tool-layer suite (503 after the f1 merge) plus 40 live-runner additions:
  • 25 behavior tests on an injected mock transport only — key absence refusal, cap enforcement before the request, conservative pricing fail-closed, JSONL logging, tool dispatch through wrapWithPolicy, FIFO approval mapping, clean aborts, proactive suppression.
  • 14 adapter mapper tests — role/tool-result mapping, args JSON parsing, usage mapping, malformed-response rejection.
  • 1 localhost-only transport test — the real SDK + HTTP path driven against an in-process server via LECHEF_LIVE_ENDPOINT; external traffic is not possible from the test.
• CLI smokes: plan mode exits 0 and prints arms/slots/caps/prices and the exact live command; live mode without LECHEF_LIVE_API_KEY exits 2 before any transport is constructed.
• Incident disclosure: one verification smoke with a placeholder key (default caps) started a live run; the transport attempted three requests to the real endpoint — all failed at auth/connection, zero spend, no successful API call. The run directory was deleted; the CLI now exits 1 when zero sessions complete, and README-live.md documents the localhost rehearsal channel as the only offline-safe way to exercise live mode. Live execution remains unrun.

Risk

• Live execution is UNRUN by design — the operator supplies key + scope + cap authorization explicitly. See Handoff.
• Caps default to $1 / 20 requests when the env vars are absent (documented in README-live.md); explicit caps are still recommended for any real run.
• Costs are conservative pre-request estimates reconciled from usageMetadata when the provider reports usage; models missing from the pricing table fail closed.

Handoff

Done — branch overhaul/lechef-live-runner, head ef59034, every commit authored by Sam Gu:

• c4c6e3d live core modules · 7a3ee89 behavior tests (mock transport only) · a3ec50c merge of overhaul/lechef-f1 (da04d52) + integration fixes · 7f131e8 Gemini adapter + live CLI · ef59034 adapter tests, localhost transport proof, runbook, exit hardening.
• Suite 543 pass / 0 fail; plan and refusal smokes replayable (see Verification).

Not done

• Live execution is UNRUN: no key exists in the environment, and per the operator's standing rule a live run needs Sam's explicit key + scope + cap authorization. The three-design comparison results will exist only after that run.

How to continue

1. When Sam authorizes (key, scope, caps), run from backend/gemini-agent:
   LECHEF_LIVE_API_KEY=... LECHEF_LIVE_MAX_SPEND_USD=<usd> LECHEF_LIVE_MAX_REQUESTS=<n> bun run lechef:live --sessions=<dir-or-file> --mode=live
   Without the key, bun run lechef:live --sessions=<dir-or-file> prints the plan safely (verified).
2. Session scripts come from the harness conversion flow — the same JSON shape the offline CLI consumes (see README-live.md). Outputs land under evaluation/lechef-live-runs/<runId>/ (summary.json, per-session JSONL, RunRecord JSON) and stay uncommitted.
3. A cap breach aborts cleanly with the cap state in summary.json (exit 1); zero completed sessions also exit 1. Nothing sends a request without an explicit keyed --mode=live command.

Checklist

• The scope is focused; related changes belong together
• I explained what changed and why
• N/A — no visible UI change (backend evaluation tooling)
• N/A — no motion or interaction change

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}