Skip to content

fix(plugin): never start an agent CLI from the plugin bundle - #86

Merged
ursuciprian merged 4 commits into
mainfrom
fix/plugin-no-agent-spawn
Oct 2, 2026
Merged

ursuciprian merged 4 commits into
mainfrom
fix/plugin-no-agent-spawn

Conversation

@ursuciprian

@ursuciprian ursuciprian commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

The plugin directory review rejected the plugin because its System 2 CLI backend launched other agent sessions with their prompts and hooks off (codex exec -c approval_policy="never", claude -p --settings {"disableAllHooks":true}). This removes every way the bundle could start another agent session or skip a prompt. reflex setup and npm installs keep the cli backend unchanged.

What changed in plugin/

  • judge2.mjs: the cli backend (cliArgs, runCli, onPath, CLAUDE_LEAN, CODEX_OFF, --bare), its probe branch, the node:child_process import and the comments about fake CLIs are setup-only. A cli tier that still got this far answers human without making a call (cliJudge). The checkout run with --plugin never assigns runCli either.
  • config.mjs: in plugin mode JUDGE_BACKENDS is anthropic, openai-compatible, none. judgeSettings turns a saved judge.backend of cli (what human-last setup picks when claude is installed) into none and drops cli tiers. BACKEND_DEFAULTS.cli is setup-only.
  • status.mjs: warns when config.json names a cli backend or tier, and the human-last System 2 rung says why it is off. Running each recorded agent's --version and the node:child_process import are setup-only.
  • failsafe.mjs, gate.mjs: the --sh flag, the node:child_process import and the reflex-sh comments are setup-only.
  • report.mjs: a comment that listed Claude Code's permission modes is reworded.

A System 2 approve in the plugin is still a plain pass: APPROVED is pass and claudeOut returns nothing, so Claude Code's own prompt decides.

New forbidden patterns (scripts/build-plugin.mjs)

  • dangerously-skip-permissions, bypassPermissions, approval_policy, ask-for-approval, full-auto, --yolo, disableAllHooks, permission-mode, reflex-sh anywhere in the bundle.
  • A spawn, spawnSync, execFile or execFileSync whose command is not process.execPath, "git" or "sed". The one exception is infra.mjs, which runs the terraform, tofu, kubectl or helm that which() found.
  • Any of those (or exec, execSync, fork) with a literal claude, codex, opencode, pi, omp or hermes.
  • A node:child_process import that is not by name (default, namespace, dynamic, require), uses an alias, or brings in exec, execSync or fork.

Run against the sources at main, the new build fails on all nine places this PR strips.

Tests

  • node scripts/build-plugin.mjs: OK, 62 files, no forbidden pattern.
  • claude plugin validate --strict plugin and .: passed (run with a scratch HOME).
  • CLAUDE_PLUGIN_ROOT=plugin node test.mjs --plugin-only: green. npm test: green.
  • New test in claudePluginChecks, each case next to the setup hook as the control:
    • HTTP System 2 that approves: setup answers allow, the plugin hook writes nothing. The trace shows source judge, decision pass, resolver system2. The stub got exactly one call, with Bearer <judge_api_key option>.
    • cli backend with a fake claude: setup starts it and allows. The plugin asks, the fake never starts, and status shows backend none, the warning and the rung.
    • A cli tier ahead of an HTTP one: the tier is skipped and the HTTP tier approves silently.
    • Against the main bundle this test fails.
  • Local eval (engine local, scratch HOME): golden 199 cases, 1 MISS; MCP 96 cases, 0 MISS. Identical to main, including the known keyless terraform apply -auto-approve row, so no new MISS.
  • Plugin parity: 309 golden and MCP cases through the bundle's own judge in plugin mode. Every outcome and source is the same before and after.

The plugin hook's timeout (follow-up commits)

The plugin's PreToolUse hook has 10 s, and Claude Code runs the command when a hook outlives it. A System 2 call could take up to judge.timeout_ms (20 s by default), so a slow judge made the plugin fail open.

  • plugin.mjs: new PLUGIN_HOOK_MS = 10000. A test checks that it matches hooks/hooks.json.
  • autonomy.mjs ladder: in plugin mode, on the hook path only, judge2 gets a deadline. That is what is left of 10 s since the process started (process.uptime(), which covers precheck and the Jev call), less 2 s.
  • judge2.mjs: the deadline only shortens judge.timeout_ms, and it is shared across tiers. With under 1 s left, no call is made (no time, which the runaway counter and the report do not count as a call). A timeout or a skipped call goes to a human, so the hook asks; with the approval queue on, it parks the command like any other System 2 failure.
  • reflex setup installs are unchanged: their hook timeout is sized from judge.timeout_ms.
  • Tests: the plugin hook against an HTTP judge that sleeps 15 s answers ask in about 8.0 s (asserted under 9 s; the trace has judge.error: timeout). The judge2 selfcheck covers a deadline cutting a call short and the no-call case under 1 s.
  • Docs: docs/SETUP.md drops the fail-open caveat, docs/GUIDE.md "Hook time" gets one line, the CHANGELOG has a Fixed entry, and the reflex status warning no longer says the plugin fails open.

The System 2 cli backend (claude -p, codex exec) and status.mjs's
per-agent --version run are setup-only now, so plugin/ has neither.
In plugin mode a saved judge.backend cli is none and a cli tier is
dropped; status warns. The build fails on prompt-skipping flags and
settings, reflex-sh, and any child process other than node, git, sed
or the infra binaries. A test shows a System 2 approve in the plugin
is a silent pass and a cli judge never starts.
A judge call longer than the plugin's PreToolUse timeout was killed with
the hook, and Claude Code then ran the command. In plugin mode the
ladder gives judge2 a deadline: what is left of 10 s since the process
started, less 2 s, never more than judge.timeout_ms. Under 1 s left the
judge is not called (no time). A timeout goes to a human, so the hook
asks. Setup installs keep their sized hook timeout.
@ursuciprian
ursuciprian merged commit 618c4b9 into main Oct 2, 2026
8 checks passed
@ursuciprian ursuciprian mentioned this pull request Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant