Skip to content

fix(fleet): read-only agents run read-only commands; refusals are actionable (#6015) - #6637

Merged
8 commits merged into
mainfrom
fix/6015-readonly-shell
Sep 28, 2026
Merged

8 commits merged into
mainfrom
fix/6015-readonly-shell

Conversation

@Hmbown

@Hmbown Hmbown commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Closes #6015

What was wrong

Read-only children were refused commands that really are read-only. The reported trigger was cd X && git diff. Four defects caused this:

  1. Grammar. The agent read-only classifier refused any ;, &, > or $ anywhere in the string. It also split pipes with a quote-unaware split('|'). So cd sub && git diff, cat a && echo --- && cat b, rg x 2>/dev/null and git log -3; git status were all refused, even though every program in them is an admitted read.
  2. Two authorities. Durable Fleet workers were judged by the parent's parallel grammar plus a special case for bounded sed. In-process children were judged by the agent grammar. So rg x | head, find . -name '*.rs' and git -C . log ran in one path and were refused in the other.
  3. Refusals. The refusal text did not say which rule failed. It said "avoid pipes", and it pointed children at Git and Run tools they do not have. The executor returned Ok(error), so the no-progress guard never counted the refusal. Child tool results were sent with is_error: None.
  4. A separate route to 0 steps. The launch-permit wait shared the child's work deadline. A child that never got a launch slot was reported as child_wall_time_exhausted with 0 steps.

What changed

  • One classifier, the same on every host (command_safety.rs). A new lexer tracks quotes and splits only on unquoted |, &&, || and ;. It admits exactly three redirects: 2>/dev/null, >/dev/null and 2>&1. It refuses every other unquoted metacharacter it does not recognise, so anything unknown fails closed.
    • Each segment must pass the existing per-segment grammar, which is unchanged. The only new segment form is echo/printf with literal text.
    • No new programs are admitted: python, awk, jq and cargo are still refused.
    • agent_readonly_verdict returns the rule that refused the command, so the reason comes from the same code that made the decision.
    • readonly_command_help() now describes the real grammar.
  • A leading cd <dir> && is moved into the cwd field (normalize_readonly_cd) before any gate judges the call. The existing working-directory workspace check then decides whether that directory is allowed.
  • One authority. The registry's durable-worker check now admits exactly agent_readonly_bash_verdict(). This is the same predicate and normalization used by the posture gate and by BashTool::execute. The bounded_sed special case is deleted.
  • gh and npm view reads need the network grant wherever they appear in a command. is_github_readonly_command is deleted. A full-shell parent is unchanged.
  • Workspace operand checks run per segment.
  • Execution rebuilds pipelines and chains from argv that the classifier approved, using only the admitted operators and redirect words.
  • One refusal builder (readonly_refusal) is shared by the posture gate, the durable authority and the executor. The rule text is identical across them, and the next steps name only tools a read-only agent has. The executor now returns a typed PermissionDenied. Child tool results carry is_error: Some(true) when they fail.
  • A queued agent that never starts now fails with "never started: no sub-agent launch slot opened within Ns, so no work ran", with 0 steps. A child that gets a slot after waiting gets its full wall time from launch. Any parent, saved-run or source deadline still caps it.
  • Docs. docs/SUBAGENTS.md gains a "Read-only shell commands" section and an updated launch-clock paragraph. docs/FLEET.md notes that durable workers use the same grammar. CHANGELOG.md [Unreleased] has a new entry.

Deviations from the approved design

  • End-to-end test commands. The e2e test uses git grep -n needle instead of rg, because rg is not guaranteed on CI runners.
  • Durable codewhale exec test. I did not add a new end-to-end test for a durable run where every call is refused. The existing pieces already pin that path: no-progress returns TurnOutcomeStatus::Failed, which exits 1 in exec_agent.rs, and classify_worker_exit(Some(1)) gives Failed, which is already tested in fleet/executor.rs.
  • Where refusal reasons come from. Refusals from the lexer (operator, quote) are produced by the code that decides. For segment-level refusals (program, subcommand, option), the existing boolean grammar decides and a diagnoser names the rule. The diagnoser runs only after a refusal, so it can never widen what is admitted.

Related

#6298 argues for moving away from grammar-defined read-only, towards one grant model and a sandbox. This PR stays inside the current grammar. It fixes what gets refused and how refusals read now; it does not start that rework.

Verification (local)

  • cargo test -p codewhale-execpolicy --lib command_safety: 79 passed, 0 failed.

  • cargo test -p codewhale-tui --lib -- tools::shell tools::registry tools::subagent tools::execution_envelope core::engine::tool_catalog core::engine::dispatch: 1001 passed, 0 failed, 1 ignored.

  • After the launch-clock commit, cargo test -p codewhale-tui --lib -- tools::subagent: 751 passed, 0 failed.

    • late_launch_permit_still_gets_the_full_work_budget fails (BudgetExhausted) with the clock restart disabled, and passes with it.
  • New tests:

    • A Scout runs cd sub && ls, git grep, then git log, and completes.
    • touch evil.txt returns an error result that names program: touch and does not mention Git, Run or /mode. The worker takes another step and completes.
    • When every shell call is refused, the transcript shows (error).
    • For 20 commands, the posture gate, predicate, durable authority and executor give the same answer and byte-identical rule text.
    • gh and npm view reads need the network grant wherever they appear in a command.
    • A late-launched child's saved deadline starts from launch, not spawn.
    • Lexer unit tests cover quoted operators, $ inside double quotes, backslash escapes and redirect edge cases.
  • cargo clippy -p codewhale-execpolicy -p codewhale-tui --all-targets --all-features --locked with CI flags: clean.

  • cargo fmt --all -- --check: clean.

  • check-blocking-calls-budget.py, check-dead-code-budget.py, check-command-crate-boundaries.py and split/module_graph.py --check: pass.

  • Changelog sync, web derive, check-versions.sh --range-audit-advisory, check-contributor-credit.py v0.10.0 and vitest lib/public-copy.test.ts (6 passed): pass.

  • After the review fixes (62739c9): cargo test -p codewhale-execpolicy --lib command_safety 79 passed, 0 failed; cargo test -p codewhale-tui --lib -- tools::shell tools::registry tools::subagent 972 passed, 0 failed, 1 ignored; clippy with CI flags, cargo fmt --check and the lint gates: clean.

Not run here: the full workspace test suite (CI covers it) and a live Fleet run against a real provider. The 0-step report from 9cdfa92 was not reproduced on current main. The evidence for the fix is the scripted-provider end-to-end tests above.

Suggested squash message

fix(fleet): read-only agents run read-only commands; refusals are actionable (#6015) with a body that describes behaviour only: read-only chains and pipelines of admitted reads run; one read-only shell authority for in-session and durable agents; refusals name the rule and are counted; gh and npm view reads need the network grant wherever they appear in a command; a queued agent that never starts says so, and its work clock starts at launch.

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings September 26, 2026 11:18
@Hmbown Hmbown added this to the v0.10.1 milestone Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

CodeWhale Bot and others added 7 commits September 26, 2026 19:46
…he refusal rule (#6015)

The agent read-only classifier refused any ;, & or > anywhere and split
pipes with a quote-unaware split('|'), so 'cd sub && git diff',
'cat a && echo --- && cat b' and 'rg x 2>/dev/null' were refused although
every program in them is an admitted read.

Replace the charset gate with an allow-direction lexer that tracks quotes,
splits only on unquoted |, &&, || and ;, admits exactly 2>/dev/null,
>/dev/null and 2>&1, and refuses every other unquoted metacharacter.
Each segment must pass the unchanged per-segment grammar; literal
echo/printf is the only new segment form. The verdict returns the rule that
refused the command, and readonly_command_help() now describes the real
grammar. readonly_network_reads() replaces is_github_readonly_command and
reports gh and npm reads per segment. split_leading_cd() exposes a leading
'cd <dir> &&' for callers to move into the cwd field.

Tests: cargo test -p codewhale-execpolicy --lib command_safety: 79 passed, 0 failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…fusals (#6015)

Read-only agents were refused commands that are genuinely read-only, and
the refusal could not be acted on:

- Durable Fleet workers were judged by the parent's parallel grammar plus a
  bounded-sed special case, while in-process agents used the agent grammar,
  so 'rg x | head', 'find . -name ...' and 'git -C . log' ran in one path
  and were refused in the other. The registry authority now admits exactly
  agent_readonly_bash_verdict(), the same input normalization the posture
  gate and BashTool::execute use; the bounded_sed carve-out is gone.
- A leading 'cd <dir> &&' is moved into the cwd field
  (normalize_readonly_cd) before any gate judges it, so the existing
  working-directory workspace check decides the directory.
- Network reads are judged per segment (gh and npm view) in the registry,
  the executor's network policy and the no-network child check.
  Workspace operand checks run per segment, so the gh exemption covers only
  the gh segment.
- Execution rebuilds pipelines and chains from classifier-approved argv with
  the admitted operators and redirect words (hardened_readonly_script).
- One refusal builder (readonly_refusal) is used by the posture gate, the
  durable authority and the executor: the rule text is byte-identical and
  the next steps name only tools a read-only agent has. The executor now
  returns a typed PermissionDenied instead of Ok(error), so the Fleet
  no-progress guard counts it.
- Child tool results carry is_error for refused and failed calls, matching
  the parent turn loop.

Tests: cargo test -p codewhale-tui --lib -- tools::shell tools::registry
tools::subagent tools::execution_envelope core::engine::tool_catalog
core::engine::dispatch: 1001 passed, 0 failed, 1 ignored.
cargo test -p codewhale-execpolicy --lib command_safety: 79 passed.
cargo clippy -p codewhale-execpolicy -p codewhale-tui --all-targets
--all-features (CI flags): clean. cargo fmt --check: clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ck starts at launch (#6015)

The launch-permit wait shared the child's work deadline, so a child that
never got a slot ended as child_wall_time_exhausted with zero steps: a
child that never started, reported as one that ran out of time. A child
that got a slot late started its work with part of its budget gone.

The queue wait keeps its bound (the authored wall time, or an earlier
inherited deadline). If it expires the child fails with 'never started: no
sub-agent launch slot opened within Ns, so no work ran' and zero steps.
A child that launches after waiting gets its full wall time from launch,
still capped by any parent, saved-run or source deadline
(SubAgentTask::wall_ceiling_ms). This reverses the #6277 invariant that
queue time counts against the run; docs/SUBAGENTS.md, the queued-row note
and the budget line are updated to match. Kept as its own commit so it can
be dropped while keeping the rest of #6015.

Tests: cargo test -p codewhale-tui --lib -- tools::subagent: 751 passed,
0 failed. late_launch_permit_still_gets_the_full_work_budget fails
(BudgetExhausted) with the clock restart disabled and passes with it.
cargo clippy -p codewhale-tui --all-targets --all-features (CI flags): clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
docs/SUBAGENTS.md gains a 'Read-only shell commands' section: the one
grammar every read-only agent and durable worker uses, the admitted
compositions and redirects, the network grant for gh and npm reads, and
what a refusal looks like. docs/FLEET.md notes that durable workers with a
read-only shell grant use the same grammar. CHANGELOG [Unreleased] gains
the #6015 line and the #6277 queued-row line now says when the agent stops
waiting.

Checks: ./scripts/sync-changelog.sh; web derive-changelog/facts/install;
./scripts/release/check-versions.sh --range-audit-advisory: OK;
python3 scripts/check-contributor-credit.py v0.10.0: OK (5 contributors);
cd web && npx vitest run lib/public-copy.test.ts: 6 passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`gh` and `npm view` reads need the network grant wherever they appear in
a command, including after a leading `cd <dir> &&`. readonly_network_reads
now removes leading `cd` prefixes the same way the shell gates move them
into the working directory, so every caller judges the command as it runs.

Tests: cargo test -p codewhale-execpolicy --lib command_safety: 79 passed,
0 failed. cargo test -p codewhale-tui --lib -- tools::shell tools::registry
tools::subagent: 972 passed, 0 failed, 1 ignored. clippy (CI flags) on
codewhale-execpolicy and codewhale-tui: clean. cargo fmt --check: clean.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#6015)

A child that waited for a launch slot restarts its work clock at launch,
but only the in-memory deadline moved. Continuation reads the worker
record, so continuing a late-launched child was refused as out of budget
while most of its work budget was left. The restarted deadline is now
saved to the record at launch. late_launch_permit_still_gets_the_full_work_budget
also checks that the saved deadline starts from launch, not spawn.
docs/SUBAGENTS.md says so and states the gh/npm network-grant rule as
behaviour only.

Tests: cargo test -p codewhale-tui --lib -- tools::shell tools::registry
tools::subagent: 972 passed, 0 failed, 1 ignored. clippy (CI flags):
clean. cargo fmt --check: clean. Lint gates (blocking-calls, dead-code,
command-crate-boundaries, module_graph --check): pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Windows CI (Test (windows-latest)) failed to compile the tui tests with
`error: function 'git' is never used`: the helper's only caller is the
#[cfg(unix)] read_only_scout_runs_read_only_commands_and_completes test,
so on Windows it was dead code under -D warnings. Gate the helper with the
same cfg as its caller instead of allowing dead code.

Verified locally (macOS): cargo fmt --all -- --check ok; clippy -p
codewhale-tui --all-targets --all-features with CI flags: 0 warnings;
cargo test -p codewhale-tui --lib readonly_shell_6015: 3 passed, 0 failed.
A Windows-target cargo check could not run here (ring needs Windows C
headers); the Windows fix is confirmed by caller analysis only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Hmbown
Hmbown force-pushed the fix/6015-readonly-shell branch from 9af8a23 to 1ef3fdb Compare September 27, 2026 02:55
main (#6641) removed Config's top-level api_key/base_url fields, so the
two new read-only scout fixtures no longer compiled on CI (E0560).

Checks: codewhale-tui lib tests build; read_only_scout* 2 passed,
late_launch_permit 1 passed, launch_gate_wait 1 passed,
read_only_gates_agree 1 passed; 0 failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Hmbown pushed a commit that referenced this pull request Sep 27, 2026
crates/execpolicy/src/command_safety.rs: #6637 (already merged) replaced
the agent read-only classifier with a quote-aware lexer
(lex_readonly_command / agent_readonly_verdict) that admits chains and
three redirects; #6675 hardened the old classifier by rejecting a word
that starts with an unquoted `*` (has_leading_glob), because its matches
can begin with `-` and become options after the option allowlist ran.
Kept #6637's lexer as the single read-only authority and ported #6675's
rule into it: an unquoted `*` whose word has no literal prefix yet (empty
quotes count as no prefix, so `''*` is refused as #6675's test requires)
is an `operator` refusal. has_leading_glob had no remaining caller and is
removed. Doc bullet takes #6675's `*` wording within #6637's list.
Test updates: `echo *` is now refused by the glob rule before the echo
literal rule (row updated, `echo a*` added for the literal rule); a new
unit test covers the glob rule on the lexer incl. chains.

crates/tui/src/tools/approval_cache.rs: #6670 (already merged) replaced
command_prefix with shell_command_grant_scope (family key only for simple,
non-wrapper, inert-option, known families; else shell:cmd:<command>);
#6675 edited the old command_prefix to fall back to an exact key for
dynamic/nested/multi-command expansions or when the canonical prefix is
not leading. Folded #6675's conditions into #6670's family branch, so
either rule failing gives the full-command key; kept #6670's key format.

Verified: cargo test -p codewhale-execpolicy: 221+1+5+7+1 passed, 0 failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Hmbown pushed a commit that referenced this pull request Sep 27, 2026
Each fix below reconciles two PRs that merged textually but not
semantically.

- snapshot/repo.rs, receipts.rs: #6645/#6682 added
  SnapshotRepo::changed_paths_between(from, to) -> Vec<PathBuf> (undo,
  turn artifacts) and #6591 added a different
  changed_paths_between(from, to, limit) -> (Vec<SnapshotPathChange>, bool)
  (receipts). Duplicate definition; #6591's is renamed
  path_changes_between and its one caller (receipts) updated.
- core/engine/turn_loop.rs: #6673's repl-fence approval match did not
  cover #6601's ApprovalResult::TimedOut. A timed-out card now refunds the
  tool-call budget slot and reports a timeout, like direct and code-mode
  calls; the audit line records "timeout" instead of "denied".
- runtime_api/sessions.rs: #6640's session-owner 409 predates #6645's
  ApiError.code field; code: None.
- tools/verifier.rs: #6671's env-scrub test called run_gate(gate) without
  the session_id argument run_gate takes on main (#6508).
- skills/install.rs + integration harness: #6679 made install.rs read
  downloads through crate::utils::read_response_body_capped, but the
  integration harness #[path]-includes install.rs and has no utils
  module, so the integration test target did not compile (also on the
  #6679 branch). The capped reader moves to utils/response_body.rs
  (re-exported from utils, unchanged API) and the harness includes just
  that file as crate::utils.
- Test files where an add-only conflict was auto-resolved by
  concatenating both sides lost the shared closing lines of the first
  test: commands/groups/debug/tests.rs (#6682 + #6591), tui/ui/tests.rs
  (#6635 + main), tools/shell/tests.rs (#6674 + #6679), and
  runtime_api/tests.rs (#6645 merge in round 1). Restored the missing
  `}` / `);` so each test is whole again; no assertions were dropped.
- tools/shell/tests.rs: #6637's executor test expected `sort * | cat`
  to be admitted and then fail at run time; with #6675's rule ported into
  the #6637 lexer (see the #6675 merge), a word-leading unquoted `*` is
  refused before anything runs. The test now asserts that refusal and
  still checks the sentinel and option-named files are untouched.
- core/engine/tests.rs -> tui/history/tests.rs: #6601's engine test
  asserted crate::tui::history on the trust warning, raising the
  runtime->UI test reference ratchet 40 -> 41 (check-command-crate-
  boundaries FAIL). That assertion moved to a tui::history test on
  workspace_trust_runtime_message, so the ratchet is back at 40.
- scripts/check-blocking-calls-budget.json: runtime_api/git.rs 5 -> 6.
  #6648 justified this budget in its PR body (working-tree fingerprint
  reads in sync fns reached only from spawn_blocking); its own branch
  already has six such sites (the File::open used for hashing), so the
  recorded 5 was stale. subagent/worktree.rs tightened 7 -> 6.

Checks: cargo check --workspace --tests clean (no warnings);
cargo test -p codewhale-execpolicy 221+1+5+7+1 passed;
cargo test -p codewhale-tui --test integration 187 passed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Hmbown pushed a commit that referenced this pull request Sep 27, 2026
Codex (read-only) reviewed every hand-resolved merge of this round. Five
findings, all verified in the code and fixed:

- tools/shell.rs readonly_git_dirs (#6637 x #6679, silent merge): the
  git-filter hardening found git segments by splitting on `|` only, while
  #6637's classifier and executor also run `&&`, `||` and `;` chains. In
  `pwd && git diff` no override was installed, so a repository clean
  filter could run. It now splits with agent_readonly_verdict's segments
  (falling back to `|` only if the verdict fails); test added.
- compaction.rs (#6680 merge): starts_with on the bare legacy marker missed
  the real pre-v0.9.6 checkpoint text, which opened with
  "## 📋 Conversation Summary (Auto-Generated)", optionally after the
  "## Pinned Facts (User Anchors)" section. Those openings are recognised
  again (so restore replaces them instead of stacking), while a message
  that only quotes the marker mid-text still is not; test added.
- core/engine/turn_loop.rs (#6673 x #6601): the inline REPL fence refunded
  the tool-call budget only on timeout; Denied, RetryWithPolicy and
  approval errors kept the slot although nothing ran. Every refusal now
  refunds, as direct and code-mode calls do.
- command_safety.rs (#6675 port): the leading-glob rule treated any
  quote-only raw prefix as empty, so `cat '"'*` (literal `"` prefix) was
  refused. It now decodes the prefix with shlex and refuses only when the
  decoded prefix is empty (`''*` still refused); tests added.
- tools/git.rs (#6671 x #6679, silent merge): two hardened git runners
  (run_git_command via read_only_git_command, and run_git_review_command)
  both wrapped Git::review_command with different NotFound handling.
  run_git_review_command is removed and its callers use run_git_command,
  keeping their REVIEW_DIFF_ARGS.

Checks: cargo check --workspace --tests clean; cargo test -p
codewhale-execpolicy 221+1+5+7+1 passed; cargo test -p codewhale-tui
--lib -- compaction tools::shell tools::git turn_loop repl approval
readonly checkpoint: 1133 passed, 1 failed
(approval::tests::required_tool_execution_uses_typed_host_decisions_not_approval_claims,
a 5s event deadline under load; core::engine::approval:: rerun twice: 15
passed, 0 failed). Ratchets: boundaries, blocking budget, dead-code
budget, module graph, cargo fmt --check all pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Hmbown Hmbown closed this pull request by merging all changes into main in 0bfe04e Sep 28, 2026
@Hmbown
Hmbown deleted the fix/6015-readonly-shell branch September 28, 2026 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(fleet): adaptive anti-stall + wider read-only shell grammar (defaults, not per-user config)

2 participants