Skip to content

fix(execpolicy): a redirect is not a command separator - #6929

Merged
Hmbown merged 5 commits into
codewhale-hq:mainfrom
SparkofSpike:fix/safety-gate-ampersand-classification
Oct 13, 2026
Merged

Hmbown merged 5 commits into
codewhale-hq:mainfrom
SparkofSpike:fix/safety-gate-ampersand-classification

Conversation

@SparkofSpike

@SparkofSpike SparkofSpike commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Every shell command containing a lone & was hard-blocked on Windows with:

Built-in safety gate requires approval: Windows command input cannot be
classified safely enough to exclude termination of Codewhale npm launchers;
use a direct PID- or port-specific command.

The reason text names npm launchers, but no launcher check ever ran — the
command was simply unclassifiable, and that string is a canned fail-closed
reason (SessionRuntimeRisk::UnclassifiedWindowsInvocation). Two lines in the
shared classifier made a plain redirect unreadable:

crates/execpolicy/src/command_safety.rs, split_command_segments kept a bare
& inside the word ('&' => current.push(ch)), while command_invocations
re-parsed any word containing &. Because the segmenter never separates a bare
&, that re-parse returned the identical word, recursed again, and hit
MAX_WRAPPER_DEPTH. command_invocations then returned None — "fail closed"
— and windows_session_runtime_risk turned that into
UnclassifiedWindowsInvocation, which the runtime floor converts into a
non-bypassable block (#6928 is the sibling fix; this is the classifier).

So the gate was not too strict about launchers. It could not read the command at
all. Concretely, all of these were refused in a Full Access session with the
sandbox disabled:

  • cargo fmt --all -- --check 2>&1 | Select-Object -First 30
  • git fetch origin 2>&1; git log --oneline -1 origin/main
  • & $GH auth status (PowerShell's call operator — a bare &)
  • git grep -n "2>&1" -- crates/ (the literal text inside quotes)

Removing the & made each one run. That last pair is the tell: the classifier
never gets to decide what the command means, because a bare & makes it stop
reading.

Two changes, one cause:

  • split_command_segments now separates a & that is a command separator
    (POSIX job control, and cmd.exe's own separator) while keeping the & of a
    redirection inside the word: >& copies a file descriptor (2>&1, >&2)
    and &> is bash's combined redirect.
  • command_invocations re-parses only a shell script argument, including
    nested commands with whitespace, ;, &, or |. Ordinary quoted arguments
    and redirects are data; shell payloads still expose glued command separators.
  • A quoted operator stays data. The scanner now tracks quote state, and the
    nested re-parse is limited to a shell's own script argument (sh -c,
    bash -lc). Without that, git commit -m "fix & feature" read as two
    commands and echo 'x & rm -rf /etc' produced an rm invocation the shell
    never runs — both reach the destroyer hold, which can only refuse a command
    that was fine. That is the bug class this change exists to remove, and an
    adversarial review of the first cut caught it.

The second change alone would have been a weakening: a bare & glued to its
neighbours (echo x&rm -rf /etc) names two commands, and left unsplit it
corrupts the command word of the second one. Splitting it is the fix; dropping
the recursion is what makes the redirect readable. echo x&rm -rf /etc now
yields an rm invocation instead of being refused wholesale by the depth cap.

Issue

Refs #6963 (the follow-up report now tracks the same Windows failure modes).

How I tested it

cargo fmt --all -- --check                                    # clean
cargo test -p codewhale-execpolicy command_safety
    -> 88 passed; 0 failed
cargo test -p codewhale-tui auto_review
    -> 72 passed; 0 failed

redirects_stay_classifiable_and_a_lone_ampersand_still_separates (new)
asserts both directions: git log 2>&1, cargo test 2>&1 | tail -5,
make > build.log 2>&1, tool &> out.txt stay classifiable, while
echo x & rm -rf /etc, echo x&rm -rf /etc and
curl https://example.com & rm -rf /etc still yield an rm invocation.
a_quoted_operator_stays_data covers the four quoting shapes.

Run against the previous code, the first of those fails with exactly the
reported symptom:

a redirect must stay readable instead of failing closed: git log 2>&1
test result: FAILED. 0 passed; 1 failed

I also exercised the real binary: git log --oneline -3 2>&1 through the
bash tool runs and returns output on a build with this change, where it was
refused before.

test_cases_09.rs used to assert the Windows floor as the expected outcome
for cargo test & curl https://example.com; that assertion is removed because
the command is classifiable and belongs on the reviewer path like every other
approval-shaped command. The remaining windows_* tests in that module pass
unchanged, and UnclassifiedWindowsInvocation is still covered at
auto_review.rs via the nested sh -c depth case.

Note on tests. These commits add new tests and code comments, which is the
shape AGENTS.md asked for when this work started. Commit 12de671c4 has since
changed that rule to "no new tests and no new code comments unless explicitly
asked". I am leaving them in because the previous-code failure above is the only
proof the defect was real and is fixed. If the current rule is what you want
here, say so and I will drop them.

Checklist

  • One focused change
  • cargo fmt --all passes; tests cover the new behavior
  • No user-facing feature added or changed, so docs/features.toml is unchanged

Maintainer conflict resolution and verification (2026-10-13)

Current head: 454b607bdf. The main conflict is resolved in eb817dbf87, preserving this contributor PR and its tests. Shell payload detection now reuses the existing option parser and recognizes Windows cmd/PowerShell payloads. Follow-up review fixed escaped-backslash parity before a closing double quote in 0b5042a443; its case is added to an existing test.

  • Latest head, macOS: 89 classifier tests passed, 0 failed (147 filtered out); focused all-targets Clippy, formatting and whitespace checks passed.
  • eb817dbf87, macOS: 79 approval tests passed, 0 failed; npm 1,312 passed, 0 failed, 7 skipped and full check:web passed. Web/npm files are unchanged at the new head.
  • Historical hosted Windows on 0b5042a443 passed: 18,299 tests passed, 0 failed, 27 skipped (one test reported leaky), including codewhale-cli::owner_crash_reentry killed_owner_is_replaced_by_the_next_app_server (PASS, 2.386s). Windows job and full log. The leaky test is the existing external-URL nonblocking test, which intentionally starts a delayed child. Ubuntu and Lint also passed on that prior head. The changed head requires fresh hosted checks. These hosted results do not qualify an installed native-app or packaged release.
  • Founder authorized merging this PR as its own contribution after verification. Tagging/publication remain separate. Intended release: 0.10.2, not yet shipped.

Final review follow-up: 454b607bdf keeps literal cmd.exe apostrophes from hiding an ampersand-separated invocation in its explicit /C or /K payload. Existing classifier table extended: 89 passed, 0 failed on macOS; focused all-target Clippy and final formatting pass. A harmless Windows 11 ARM64 echo probe confirmed cmd quote semantics; it did not execute the new Rust classifier. New-head Windows, Ubuntu and macOS CI remain required before merge. No test was disabled.

A lone `&` inside a word was fed back into the classifier's own word walk, and
`split_command_segments` never separates a bare `&`. The word therefore
reproduced itself until `MAX_WRAPPER_DEPTH` blew, `command_invocations` returned
`None`, and the caller treated the whole command as unclassifiable. On Windows
the runtime floor turns that into a non-bypassable block, so any command
carrying `2>&1` — a plain redirect — could not run at all, and the agent
burned turns retrying different spellings of the same refusal.

Two changes, one cause:

- `split_command_segments` now separates a `&` that is a command separator
  (POSIX job control, and cmd.exe's own separator) while keeping the `&` of a
  redirection inside the word: `>&` copies a file descriptor and `&>` is bash's
  combined redirect.
- `command_invocations` re-parses a word only when re-parsing can yield a
  different token stream — whitespace, `;`, or `|`. A `&` that survived
  segmenting is part of a redirect, and recursing on it could only reproduce
  the same word.

Separating the bare `&` also keeps a destructive second stage reachable:
`echo x&rm -rf /etc` now yields an `rm` invocation instead of being refused
wholesale by the depth cap.

Tests: `redirects_stay_classifiable_and_a_lone_ampersand_still_separates` fails
on the previous code (`a redirect must stay readable instead of failing closed:
git log 2>&1`) and passes on this one. `test_cases_09.rs` no longer asserts the
Windows floor as the expected outcome for `cargo test & curl https://example.com`;
that command is classifiable and now reaches the reviewer like every other
approval-shaped command.
The redirect fix made `&` a separator but left the scanner quote-blind, so an
operator inside quotes was split anyway. The adversarial review of this branch
caught the result: `git commit -m "fix & feature"` read as two commands, and
`echo 'x & rm -rf /etc'` produced an `rm` invocation the shell never runs.
Both reach the destroyer hold, which can only refuse a command that was fine —
and a spurious refusal is the bug class this branch exists to remove.

Two parts:

- `split_command_segments` tracks quote state, so an operator inside quotes
  stays in its word. The read-only grammar already promised this
  (`rg 'a && b; c' src`); the shared scanner now agrees.
- `command_invocations` re-parses a word only when it is a shell's script
  argument (`sh -c`, `bash -lc`, ...). An ordinary argument that happens to
  contain whitespace or an operator is not a nested command line; re-parsing it
  invented commands, and once a `&` made a word reproduce itself it also failed
  readable commands closed.

Tests: `a_quoted_operator_stays_data` covers the four shapes from the review
(`git commit -m "fix & feature"`, `echo "a & b"`, `echo 'x & rm -rf /etc'`,
`printf 'a;b | c'`) and asserts one command stays one command. The nested
`sh -c` detection the previous change relies on is still covered by
`command_invocations_reach_past_wrapper_options_and_into_shell_payloads`,
including its depth-cap assertion. `cargo test -p codewhale-execpolicy
command_safety` -> 88 passed; 0 failed.
@SparkofSpike
SparkofSpike requested a review from Hmbown as a code owner October 9, 2026 04:08
… nested-shell detection

Resolve command_safety conflicts against origin/main 008d21d without
rewriting SparkofSpike's commits. Preserve quoted ampersands as data,
recognize redirections and separators, reuse the existing shell payload
parser for option-bearing shell wrappers, and retain cmd /C and PowerShell
-Command payload inspection. Remove the obsolete descriptor-only helper.
Keep the contributor's regression tests and main's redirect coverage;
update the existing expectations for the newly classifiable syntax.

Verification on macOS, native shared build lock and hermetic test home:
- scripts/dev-test.sh execpolicy command_safety:
  89 tests run: 89 passed, 147 skipped (filter exclusions).
- scripts/dev-test.sh tui auto_review:
  79 tests run: 79 passed, 15232 skipped (filter exclusions).
- npm test: 1312 passed; 0 failed; 7 skipped.
- npm run check:web: passed, including latest-release check and build.
- cargo fmt --all -- --check and git diff --check: passed.
- clippy -p codewhale-execpolicy --all-targets --locked -D warnings: passed.
Windows execution and hosted Windows CI on this merge are still pending.
No release, tag, or main push is part of this commit.

Refs codewhale-hq#6929, codewhale-hq#6963
@Hmbown

Hmbown commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

@SparkofSpike, thank you for isolating the ampersand recursion and then checking quoted operators as data—the examples in this PR made both the false refusals and the destructive-command boundary concrete. I have resolved the conflict caused by main’s changes on your branch in eb817dbf87; your commits and PR are preserved. The resolution also reuses the existing shell-payload parser so extra shell options and Windows cmd/PowerShell payloads remain inspected.

Local macOS verification: 89 classifier tests passed, 79 approval tests passed, npm 1,312 passed / 0 failed / 7 skipped; web gate, focused Clippy and formatting passed. Hosted checks are running on the updated head; I do not yet have a Windows execution receipt for it. @jayanthvee, this is the SHA for your offered Windows 11 native-install retest.

I recommend keeping your regression tests and have retained them in this resolution. The current no-new-tests rule constrains agent-written additions; your before/after evidence belongs with your contribution. I have asked the founder to confirm that interpretation. Thank you also for starting the Chinese localization group in #6804. This fix is intended for 0.10.2, which has not shipped yet.

Review of the contributor conflict resolution found that a pair of
backslashes before a closing double quote consumed that quote as escaped.
Consume the escaped backslash as a pair, so the following closing quote
ends its string and a glued command separator remains visible.
Extended the contributor's existing ampersand regression test; no new test
function or code comment was added.

macOS verification: 89 tests run: 89 passed, 147 skipped (filter exclusions).
Focused all-targets execpolicy Clippy, formatting and diff whitespace pass.
Unchanged web/npm source retains the immediately preceding eb817db gate:
npm test 1312 passed; 0 failed; 7 skipped, and npm run check:web passed.
Windows execution remains pending hosted CI; no Windows proof claimed.

Refs codewhale-hq#6929
Refs codewhale-hq#6963
Hmbown pushed a commit that referenced this pull request Oct 13, 2026
Combine SparkofSpike's preserved PR head 0b5042a with the release
candidate so hosted verification covers its classifier and the approved
normal-approval policy together. There are no merge conflicts; the existing
engine assertion changes are identical on both sides. Credit the contributor
and reporter in the release notes.

Landing order: PR #6929 must merge as its own contribution before #6968 is
squash-merged. This merge does not land either PR on main or authorize release.

macOS combined-source verification:
- 81 tests run: 81 passed, 15230 skipped (focused filter exclusions).
- The exact classifier change at 0b5042a: 89 tests run: 89 passed,
  147 skipped (filter exclusions); focused all-target Clippy passed.
- Unchanged web/npm source retains the preceding verified slice's gate:
  npm test 1312 passed; 0 failed; 7 skipped; npm run check:web passed.
- Formatting, whitespace, dated versions and synchronized changelogs pass.
- The notes/credit update followed the Rust run; it changes no Rust source.
Windows execution is pending hosted CI on this combined candidate. The prior
Mac MSVC cross-check could not compile ring without Windows SDK assert.h;
it provides no Windows type-check evidence. Native Windows acceptance is open.

Refs #6929
Refs #6963
cmd.exe treats apostrophes as literal characters, so its /C or /K payload
can execute an ampersand-separated command inside apostrophes. The shared
POSIX-oriented scanner previously grouped that text and missed the later
invocation. Normalize literal apostrophes only in the explicit cmd payload
before the existing bounded nested scan. Keep ordinary quoted arguments and
other shell payloads on their existing path. Extend the existing wrapper case
table; no new test function or code comment.

Refs codewhale-hq#6963. Follow-up review on contributor PR codewhale-hq#6929, retained as its own PR.

macOS: CODEWHALE_DEV_NEXTEST=0 CARGO_BUILD_JOBS=1 scripts/dev-test.sh execpolicy command_safety
  test result: ok. 89 passed; 0 failed; 0 ignored; 0 measured; 147 filtered out; finished in 0.01s
Focused all-target Clippy, fmt and git diff --check passed. The first fmt check
requested only match-arm formatting, applied before commit. Reused unchanged
web/npm gate: 1312 passed; 0 failed; 7 skipped; check:web passed.

Windows 11 ARM64: a harmless cmd /D /S /C echo probe emitted the second-stage
marker between literal apostrophes. This proves shell semantics only; the new
Rust classifier has not yet run on Windows. Hosted Windows CI is required.
The prior PR head's Windows job passed 18299 tests, including owner_crash_reentry;
that receipt does not qualify this changed head. No native-app, package or
Windows 10 1607 acceptance is claimed.
Hmbown pushed a commit that referenced this pull request Oct 13, 2026
Refs #6963. Contributor head passes 89 classifier tests, 0 failed, and focused all-target Clippy on macOS. Unchanged npm/web gate remains 1312 passed, 0 failed, 7 skipped. The merge is conflict-free; hosted Windows verification is pending for this changed head.
@Hmbown
Hmbown merged commit 5ebb2c1 into codewhale-hq:main Oct 13, 2026
28 checks passed
@Hmbown

Hmbown commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

Merged as your own PR at 5ebb2c14f8. Thank you @SparkofSpike: distinguishing quoted ampersands and redirection from command separators removes several ordinary Windows commands from the unclassified safety hold. We resolved the main-branch overlap here and kept your commits and regression coverage.

The final contributor head 454b607bdf passed hosted macOS 19,035 passed, 0 failed, 33 skipped, and Windows 18,299 passed, 0 failed, 27 skipped, including owner-crash recovery. The source branch is preserved. This is now on main for contributors to build on and is planned for 0.10.2; the release has not shipped. Your original contribution is also credited in the candidate changelog.

Hmbown pushed a commit that referenced this pull request Oct 13, 2026
Document the merged contributor sub-agent state fix with its actual
existing-target junction scope. Credit SparkofSpike for #6929 and #6949,
and Lstarsky0 for #6971 in the 0.10.2 contributor list. Regenerate the
embedded changelog from the same source.

Verification on macOS, source410d8310ca plus these two changelog paths:
- npm test: Node540 passed;0 failed;7 skipped; Vitest772 passed.
  Combined1312 passed;0 failed;7 skipped.
- npm run check:web: PASS.
- sync-changelog.sh --check and git diff --check: PASS.

#6949 final PR head hosted CI: Windows18301 passed/27 skipped,
macOS19035 passed/33 skipped, Ubuntu19032 passed/34 skipped. Both
Windows junction cases and owner_crash_reentry passed. Final candidate
CI, dangling-target, native/package and oldest-Windows acceptance are
separate remaining gates. No tests were added in this documentation slice.

Refs #6949
Refs #6971
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants