Repository navigation
fix(plugin): never start an agent CLI from the plugin bundle - #86
Merged
Merged
Conversation
The System 2 cli backend (claude -p, codex exec) and status.mjs's per-agent --version run are setup-only now, so plugin/ has neither. In plugin mode a saved judge.backend cli is none and a cli tier is dropped; status warns. The build fails on prompt-skipping flags and settings, reflex-sh, and any child process other than node, git, sed or the infra binaries. A test shows a System 2 approve in the plugin is a silent pass and a cli judge never starts.
A judge call longer than the plugin's PreToolUse timeout was killed with the hook, and Claude Code then ran the command. In plugin mode the ladder gives judge2 a deadline: what is left of 10 s since the process started, less 2 s, never more than judge.timeout_ms. Under 1 s left the judge is not called (no time). A timeout goes to a human, so the hook asks. Setup installs keep their sized hook timeout.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The plugin directory review rejected the plugin because its System 2 CLI backend launched other agent sessions with their prompts and hooks off (
codex exec -c approval_policy="never",claude -p --settings {"disableAllHooks":true}). This removes every way the bundle could start another agent session or skip a prompt.reflex setupand npm installs keep theclibackend unchanged.What changed in
plugin/judge2.mjs: theclibackend (cliArgs,runCli,onPath,CLAUDE_LEAN,CODEX_OFF,--bare), itsprobebranch, thenode:child_processimport and the comments about fake CLIs are setup-only. Aclitier that still got this far answers human without making a call (cliJudge). The checkout run with--pluginnever assignsrunClieither.config.mjs: in plugin modeJUDGE_BACKENDSisanthropic,openai-compatible,none.judgeSettingsturns a savedjudge.backendofcli(what human-last setup picks when claude is installed) intononeand dropsclitiers.BACKEND_DEFAULTS.cliis setup-only.status.mjs: warns when config.json names aclibackend or tier, and the human-last System 2 rung says why it is off. Running each recorded agent's--versionand thenode:child_processimport are setup-only.failsafe.mjs,gate.mjs: the--shflag, thenode:child_processimport and thereflex-shcomments are setup-only.report.mjs: a comment that listed Claude Code's permission modes is reworded.A System 2 approve in the plugin is still a plain pass:
APPROVEDispassandclaudeOutreturns nothing, so Claude Code's own prompt decides.New forbidden patterns (
scripts/build-plugin.mjs)dangerously-skip-permissions,bypassPermissions,approval_policy,ask-for-approval,full-auto,--yolo,disableAllHooks,permission-mode,reflex-shanywhere in the bundle.spawn,spawnSync,execFileorexecFileSyncwhose command is notprocess.execPath,"git"or"sed". The one exception isinfra.mjs, which runs the terraform, tofu, kubectl or helm thatwhich()found.exec,execSync,fork) with a literalclaude,codex,opencode,pi,omporhermes.node:child_processimport that is not by name (default, namespace, dynamic, require), uses an alias, or brings inexec,execSyncorfork.Run against the sources at main, the new build fails on all nine places this PR strips.
Tests
node scripts/build-plugin.mjs: OK, 62 files, no forbidden pattern.claude plugin validate --strict pluginand.: passed (run with a scratch HOME).CLAUDE_PLUGIN_ROOT=plugin node test.mjs --plugin-only: green.npm test: green.claudePluginChecks, each case next to the setup hook as the control:Bearer <judge_api_key option>.clibackend with a fakeclaude: setup starts it and allows. The plugin asks, the fake never starts, and status shows backend none, the warning and the rung.clitier ahead of an HTTP one: the tier is skipped and the HTTP tier approves silently.terraform apply -auto-approverow, so no new MISS.The plugin hook's timeout (follow-up commits)
The plugin's PreToolUse hook has 10 s, and Claude Code runs the command when a hook outlives it. A System 2 call could take up to
judge.timeout_ms(20 s by default), so a slow judge made the plugin fail open.plugin.mjs: newPLUGIN_HOOK_MS = 10000. A test checks that it matcheshooks/hooks.json.autonomy.mjsladder: in plugin mode, on the hook path only,judge2gets a deadline. That is what is left of 10 s since the process started (process.uptime(), which covers precheck and the Jev call), less 2 s.judge2.mjs: the deadline only shortensjudge.timeout_ms, and it is shared across tiers. With under 1 s left, no call is made (no time, which the runaway counter and the report do not count as a call). A timeout or a skipped call goes to a human, so the hook asks; with the approval queue on, it parks the command like any other System 2 failure.reflex setupinstalls are unchanged: their hook timeout is sized fromjudge.timeout_ms.judge.error: timeout). The judge2 selfcheck covers a deadline cutting a call short and the no-call case under 1 s.docs/SETUP.mddrops the fail-open caveat,docs/GUIDE.md"Hook time" gets one line, the CHANGELOG has a Fixed entry, and thereflex statuswarning no longer says the plugin fails open.