feat(agent-cli): add codex, opencode and kilo ori harnesses with in-sandbox generation-id proxy

[abhinav-pola]

TL;DR

Adds codex, opencode and kilo as ori agents. Since none of them emit OpenRouter generation ids in their headless output, each run starts a tiny local HTTP proxy inside the sandbox, points the CLI's OpenRouter base URL at it, and records the gen-* ids from the actual responses the client received. Existing agents (claude, pi, prime-agent, omp) are untouched.

What changed?

• src/benchmarks/agent-cli/generation-proxy.ts: repo-owned Node proxy script (no deps, http + fetch) written to the sandbox via heredoc. Binds 127.0.0.1 on an ephemeral port, forwards method/headers/body to https://openrouter.ai, streams the response back unchanged, scans non-GET response bytes (JSON and SSE) for "id":"gen-<digits>-<suffix>", and emits each new id once as OR_GENERATION_ID <id> on stdout and to a log file. buildGenerationProxyPrelude starts it, waits for the port file, exports OR_GENERATION_PROXY_BASE_URL, and kills it on exit.
• harness.ts: CODEX_HARNESS, OPENCODE_HARNESS, KILO_HARNESS plus JSONL parsers.
  • Codex: ori codex -- exec --json ... with the ori provider overrides repeated at exec level so base_url can point at the proxy. Parses turn.completed.usage, item.completed items (tool calls, errors, final agent_message).
  • OpenCode/Kilo: OPENCODE_CONFIG_CONTENT / KILO_CONFIG_CONTENT built after the prelude so provider.openrouter.options.baseURL is the proxy. Parses step_finish usage/cost, tool_use, text, error.
  • All three: generationIds: parseProxyGenerationIds(stdout) because the proxy shares the client's stdout pipe.
• schema.ts: ORI_AGENTS gains the three ids.
• Tests: generation-proxy.test.ts (real proxy against a local Bun upstream, no network or key), harness.test.ts (parsers and generated scripts), solver-level test in ori-solver.test.ts proving ids reach sample.metadata.generationIds and recordGenerationId. Test helper marker list extended. registry.test.ts swapped its "unwired agent" fixture from codex to grok since codex is now wired.

Why?

Session-id recovery after the fact would also pick up generations from activity retries. Recording ids from the client's own traffic keeps attribution to exactly the trajectory that produced the result.

How to test

bun test src/benchmarks/agent-cli src/benchmarks/terminal-bench/ori-solver.test.ts
bun run lint && bun run build

Manual: run terminal_bench with agent: codex (or opencode, kilo) and confirm generationIds on the sample metadata are non-empty and resolve at /api/v1/generation?id=.

Benchmark impact

None for existing agents. New agents are opt-in via agent.

Reviewer focus

• Proxy strips only host, connection, content-length, transfer-encoding, content-encoding before forwarding. Auth header passes through untouched and is never logged.
• The id regex is intentionally strict (gen-\d+-...) so model catalog ids like gen-eric/model are not captured.
• Codex exec-level -c overrides were verified locally to load (an earlier partial set failed with provider name must not be empty).

Checklist

• Tests cover changed behavior
• Public API or configuration changes are backward compatible, or the break is documented
• Benchmark changes document dataset provenance and licensing
• No credentials, private results, or restricted dataset contents are included
• Documentation is updated where needed

Link to Devin session: https://openrouter.devinenterprise.com/sessions/d7b9666e78ce40b4b64eded0024b7e11
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/d7b9666e78ce40b4b64eded0024b7e11?variant=devin
Requested by: @abhinav-pola

---

[devin-ai-integration]

Original prompt from Abhinav

SYSTEM:
<latest_message>
Abhinav Pola (U090K0G7JF3) [ts=1788471843.371019]: @Devin what other ori harnesses can i introduce to the benchmark harness? need harnesses that expose generation ids
</latest_message>

=== BEGIN THREAD HISTORY (in #brain-abhinav) ===
Abhinav Pola (U090K0G7JF3) [ts=1788471843.371019]: @Devin what other ori harnesses can i introduce to the benchmark harness? need harnesses that expose generation ids
=== END THREAD HISTORY ===
Channel ID: C09JH4HR3DH
Thread URL: https://openrouter.slack.com/archives/C09JH4HR3DH/p1788471843371019?thread_ts=1788471843.371019&amp;cid=C09JH4HR3DH

The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR that start with 'DevinAI' or '@devin'.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring