add sandbox mode option - #8
Conversation
Co-authored-by: OpenAI Codex <noreply@openai.com>
note: this currently fails, apparently due to a bug in codex where the sandbox permissions given to main agent are not passed to sub-agents Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Proof of how this works: sandbox-mode-real-review-proof.md |
andreidavid
left a comment
There was a problem hiding this comment.
This is exactly what the #3 review asked for, and more carefully than asked: the array-based mode args, the fail-safe default, one command with an argument hint, the script in scripts/ with CI and CONTRIBUTING updated to match, the SessionStart warning, and a CHANGELOG entry politely left unversioned. Two things deserve specific credit: the flag-position discovery (--sandbox really is rejected after the review subcommand — I verified against codex 0.144.5) and the state-file mechanism with its install-channel rationale, which is better-reasoned than the pluginConfigs approach it replaces. The tests isolating CLAUDE_CONFIG_DIR from the developer's real state is the kind of detail most contributors miss.
Merging as-is. The INCONCLUSIVE detection has a few integration gaps on our side of the fence (the Stop hook and GC don't know the new verdict, and the prose heuristic can misfire on reviews that merely discuss sandbox failures — this repo's reviews do constantly); rather than bounce it back, I'll fold those fixes into a follow-up PR right behind this one and link it here. Thanks for another thorough round.
|
One favor that would sharpen the follow-up: the fix replaces the prose regex with a structural signal (a review whose JSONL stream contains zero jq 'select(.verdict=="PASS") | {timestamp, short_sha, review_prose}' .git/codex-reviews.jsonl | tail -20— specifically an entry from a run where you know the sandbox failed. And once the follow-up ships in a release, a quick "INCONCLUSIVE now fires correctly here" (or doesn't!) from your setup would close the loop. |
|
Follow-up landed as #9 — the detector ended up structural (no successful execution activity in the JSONL stream, grounded in codex's exec serializer source) rather than prose-based, plus the Stop-hook/GC integration and remote-aware retry guidance. Your redacted phantom-PASS sample would still be valuable to confirm the detector catches your environment's exact failure shape — and once 1.8.0 is released, a quick yes/no on whether INCONCLUSIVE fires correctly on your setup closes the loop. Thanks again! |
#9) Follow-up to #8's phantom-pass detection, fixing the integration gaps found in review: The detector is now structural instead of prose matching, and deliberately minimal: INCONCLUSIVE fires only when the review's event stream shows no successful execution activity at all -- no command_execution item that exited 0 and no code-mode dynamic_tool_call item. That is the one empirically observed failure shape (a nested sandbox that refuses to initialize, so the review runs nothing) and the only fact the stream states reliably. Verifying that activity actually READ the repository is explicitly not attempted: shell output under-determines it (compounds, pipelines, || fallbacks, quoting, error-like content), and a far stricter seven-layer version of this check minted a new counterexample per layer before being retired. Partial sandboxes that permit some execution while denying repo reads are a documented accepted residual. The prose regex the detector replaces matched reviews that merely DISCUSS sandbox failures -- which every review of this repo does. Legacy text-parser mode is exempt (no event stream). The Stop hook clears INCONCLUSIVE state like ERROR/TIMEOUT, and the GC recognizes it as a known verdict and sweeps aged entries -- previously it fell into the unknown-verdict bucket, which is preserved indefinitely as corruption evidence, so abandoned INCONCLUSIVE state was immortal. The guidance message is remote-aware like the fix loop: an unpushed commit pairs /codex-review-sandbox-mode danger-full-access with a repo-anchored git commit --amend --no-edit (new sha, fresh review under the new mode; a same-sha re-run is dedup-skipped); a pushed commit is never amended and gets a one-off elevated review command carrying the configured CODEX_BIN and shell-quoted repo path. Stub modes emit realistic command_execution events; no_inspection, failed_inspection, unrelated_command_pass (documented residual), and dynamic_tool_pass cover the detector from all directions. README documents the verdict. Version 1.8.0 covering #8 plus this hardening. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Happy to. And the follow-up: I updated to 1.8.1 and ran the test — INCONCLUSIVE does not fire on my setup. Details below. Question 1 — phantom-PASS samples. From a work repo, {"timestamp":"2026-07-22T12:37:51Z","short_sha":"<redacted>","review_prose":"The local filesystem sandbox failed, and the commit is not available through the GitHub API. The changes could not be inspected, so this verdict has very low confidence."}
{"timestamp":"2026-07-27T08:59:35Z","short_sha":"<redacted>","review_prose":"The commit could not be inspected: local commands failed because the sandbox could not start, and the commit is not available in the connected GitHub repository. No reliable correctness verdict or findings can be provided."}
{"timestamp":"2026-07-27T10:20:21Z","short_sha":"<redacted>","review_prose":"The commit could not be inspected because local shell access failed and the commit is not available on the connected GitHub repository. No reliable findings could be produced."}5 of 8 PASSes in that history look like this — codex self-declares "sandbox failed" or "could not be inspected" in the prose, then the verdict is recorded PASS. Question 2 — INCONCLUSIVE on my setup. I set {"type":"item.completed","item":{"type":"mcp_tool_call","tool":"list_mcp_resources","status":"completed"}}
{"type":"item.completed","item":{"type":"mcp_tool_call","tool":"list_mcp_resource_templates","status":"completed"}}
{"type":"item.completed","item":{"type":"mcp_tool_call","tool":"github.search_commits","status":"completed"}}
{"type":"item.completed","item":{"type":"agent_message","text":"The review could not be completed because the sandbox rejected all filesystem and Git commands with `sandbox_apply: Operation not permitted`. The commit diff was unavailable, so this verdict is not an assessment of the code."}}Zero Not sure if you'd consider that the "partial sandbox / commands whose output the reviewer ignores" residual the docstring accepts by design, or something worth catching. If it's worth catching, filtering out codex's own introspection tools ( Happy to run any other probe or paste more of the stream if it helps. |
Codex reviews can fail when its sandbox is inside an already sandboxed Claude session.
Add sandbox configuration, with
workspace-writeas the default.Add command to configure and show the setting.