Skip to content

adversarial-review threads are always ephemeral — no way to verify which model actually ran a review #705

Description

@dswinicki

Summary

We use adversarial-review as an independent code-review gate and need to verify,
after the fact, which model actually executed the review (guarding against silent
local fallback or misconfiguration). Today that is not possible from the job artifacts:

  • adversarial-review starts its thread with ephemeral: true unconditionally
    (scripts/codex-companion.mjs, review path passes no persistThread), so no
    rollout is written under ~/.codex/sessions/.
  • The persisted job record only stores the requested model (request.model),
    not the model that actually served the turn.
  • task threads, by contrast, set persistThread: true and do write rollouts.

We currently work around this by correlating the job's threadId against the
tracing telemetry in ~/.codex/logs_2.sqlite (span attribute model= on
sampling-request spans), which does cover ephemeral threads — but that is an
internal format we would rather not depend on.

Requests

  1. Opt-in persistence for review threads — e.g. adversarial-review --persist-thread (or persist by default), so a rollout exists for audit.
  2. Document the existing --model flag for adversarial-review — the parser
    accepts and forwards it today, but help text does not mention it.
  3. Add --effort to adversarial-review — currently it silently inherits the
    value from ~/.codex/config.toml, which makes review runs non-reproducible
    across machines.
  4. Expose the effective runtime model in the job result — ideally also the
    response.model returned by the Responses API for each sampling request (or an
    execution receipt), with a clear distinction between requested_model,
    effective_runtime_model, and response_model.
  5. Keep the current outputSchema (structured verdict) — it is a strength of the
    review path; none of the above should require dropping it.

Environment

  • plugin openai-codex / codex 1.0.6 (Claude Code marketplace)
  • measured on macOS, Codex CLI with app-server transport

Why it matters

A review gate is only as trustworthy as its provenance. Without any persisted
artifact tying the review thread to the model that actually ran it, "review by
model X" is a request, not a fact. Structured Outputs plus a persisted model
receipt would make adversarial-review fully auditable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions