Skip to content

Harden agent capabilities/settings against untrusted scanned repos - #10

Open
schenksj wants to merge 8 commits into
mainfrom
vulnfix/agentic-sdk-hardening
Open

Harden agent capabilities/settings against untrusted scanned repos#10
schenksj wants to merge 8 commits into
mainfrom
vulnfix/agentic-sdk-hardening

Conversation

@schenksj

@schenksj schenksj commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Harden agent capabilities/settings against untrusted scanned repos

Tightens two places where content from the scanned repository can influence the launched Claude agent's capabilities or settings, plus a small prerequisite build fix. These came out of the cross-model /vulnhunt evaluation (eval_rep/). Each change is developed test-first: a security test that fails on the current code and passes after the fix, with the existing suite kept green.

Prerequisite — config.py currently doesn't parse

Commit a35b0ff. vulnhunter-agent/agent/config.py has an unclosed paren — the token_path_prefixes=tuple(...) generator is missing its ), before max_comment_pages=int( — so import agent.config raises SyntaxError and the agent package (and its test suite) can't be imported. This adds the missing ), and a small regression test (tests/test_config.py) that the module imports and load_config is callable.

The same one-line prerequisite also appears on vulnfix/verify-mode-scoping and vulnfix/output-injection. Whichever merges first applies it; the others will show the hunk as already-applied.

CANON-03 — Default scans to read-only; drop Bash from the scan tool set (CWE-94/CWE-78/CWE-77/CWE-269)

Commits 01cb350, 948cf42. harness/local_harness/scan.py launched claude with --allowedTools Read Write Edit Bash Agent --permission-mode acceptEdits, and the readonly argument only adjusted prompt text — it defaulted to False and the untrusted-repo scan entry points (batch, benchmark) never set it, so the default scan granted Bash (command execution) on the scanned repo.

  • Change: scan_folder / scan_folder_with_retry / scan_targets now default readonly=True. The read-only tool set is Read Write Edit Agent AskUserQuestion Grep Glob — no Bash — so prompt-injected scanned-repo content can no longer drive host command execution by default. Only an explicit opt-out (readonly=False) re-adds Bash for scans that must execute code. The batch CLI flag --readonly (opt-in) is replaced by --execute (opt-out).
  • Tests: harness/tests/test_scan.py::test_scan_folder_default_is_readonly_no_bash / ::test_scan_folder_readonly_no_bash / ::test_scan_folder_execute_optin_grants_bash; test_batch_run.py updated for the --execute semantics. Fail on the pre-rework code, pass after.
  • Residual (stated plainly): a read-only scan still permits Write/Edit within the --add-dir roots; fully isolating untrusted-repo scanning requires process sandboxing, which is out of scope for a code patch. (This scoping followed an independent vulnhunt-fix-verify pass that found the initial readonly-gated fix left the default path unprotected.)

CANON-19 — Only load trusted (user) settings sources (CWE-913/CWE-269)

Commit f57db26. run_vulnhunt built ClaudeAgentOptions(setting_sources=["user","project","local"], skills="all", cwd=clone_dir). Because cwd is the cloned repo under scan, "project"/"local" cause the scanned repo's .claude/settings*.json and skills to be loaded from that directory.

  • Change: setting_sources=["user"] in both agent/runner.py and agent/verify_runner.py (the pattern appeared in both). Checked against the installed SDK: passing ["user"] explicitly means skills="all" discovers only user-dir skills (the trusted vulnhunt skills), not clone_dir/.claude/skills — so scan/verify behavior is preserved.
  • Test: vulnhunter-agent/tests/test_runner.py::test_run_vulnhunt_does_not_load_cloned_repo_settings. Fails on current code (['user','project','local']), passes after (['user']).

Verification

  • harness: 160 passed
  • vulnhunter-agent: 1111 passed, 1 skipped
  • Discrimination evidence (test fails with the fix stashed, passes with it applied) recorded for both findings; both re-checked by an independent vulnhunt-fix-verify pass.

🤖 Generated with Claude Code

schenksj added 4 commits July 18, 2026 09:39
@schenksj
fix: close unclosed VerifyConfig(token_path_prefixes=...) paren in co…
a35b0ff 
…nfig.py

Prerequisite: the agent package did not parse, blocking all tests.

Co-Authored-By: Claude Code (VulnFix)
@schenksj
fix(security): CANON-03 restrict readonly scan capabilities
01cb350 
A readonly scan of an untrusted repo previously passed
--allowedTools Read Write Edit Bash Agent with --permission-mode
acceptEdits unconditionally, so prompt-injected content in the scanned
repo could drive arbitrary Bash/Write/Edit on the host. Gate on the
readonly flag: readonly scans now use a read-only tool set
(Read/Grep/Glob/Agent) and --permission-mode plan. Non-readonly
default callers are unchanged.

VulnHunter-Finding: CANON-03
Co-Authored-By: Claude Code (VulnFix)
@schenksj
fix(security): CANON-19 stop loading cloned-repo settings/skills
f57db26 
run_vulnhunt and verify_runner passed setting_sources=[user,project,local]
while cwd was the untrusted cloned repo, so the scanned repo's
.claude/settings(.local).json (hooks, permissions) and its skills loaded
and could execute on the host. Restrict setting_sources to [user] in both
runners; the trusted vulnhunt / vulnhunt-fix-verify skills live in the
user dir, so scan/verify functionality is unaffected.

VulnHunter-Finding: CANON-19
Co-Authored-By: Claude Code (VulnFix)
@schenksj
rework(CANON-03): default scans to read-only, drop Bash from scan too…
948cf42 
…l set

Follow-up to 01cb350 after independent fix-verify found the prior
readonly-gated fix did not cover the default scan path (readonly
defaulted to False and the untrusted-repo scan entry points never
passed it), so the RCE-capable tool set was still used by default.

- scan_folder / scan_folder_with_retry / scan_targets now default
  readonly=True.
- Read-only tool set = Read, Write, Edit, Agent, AskUserQuestion,
  Grep, Glob (no Bash) — prompt-injected scanned-repo content can no
  longer drive host command execution by default. Only an explicit
  opt-out (readonly=False) re-adds Bash for scans that must execute.
- batch CLI: `--readonly` (opt-in) replaced by `--execute` (opt-out);
  default is now a read-only scan.
- Updated pre-existing test_batch_run.py for the --execute semantics.

Residual (documented, out of scope for a code patch): a read-only
scan still permits Write/Edit within the --add-dir roots; full
isolation of untrusted-repo scanning requires process sandboxing.

Co-Authored-By: Claude Code (VulnFix)
@schenksj
schenksj requested a review from a team as a code owner July 18, 2026 15:07
schenksj and others added 4 commits July 18, 2026 12:27
@schenksj @claude
Fix A (PR #10 review): add benchmark --execute opt-out for read-only …
1689cc8 
…scan

Benchmark's phase_scan inherited the new readonly=True default with no way to
restore execution. Add --execute (store_true, default False) to main(), thread
it through phase_scan(..., execute=False), and pass readonly=not execute at the
scan_targets call. Safe default (read-only) preserved; opt-out is now explicit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@schenksj @claude
Fix B/E1 (PR #10 review): drop parent --add-dir from scan; simplify p…
0c4edaf 
…ermission_mode

Fix B: a read-only scan still granted write to the parent dir via
--add-dir os.path.dirname(folder_path). Under acceptEdits + Write/Edit, injected
scanned-repo content could overwrite sibling clones. Remove that --add-dir
entirely (results are written under folder_path=cwd; sibling access unneeded).
Keep folder_path/SKILLS_DIR/PHASES_DIR. Update CANON-03 comment to disclose the
residual: SKILLS_DIR/PHASES_DIR remain readable AND writable under acceptEdits;
closing that needs finer per-path controls (out of scope).
Add test asserting the parent dir is NOT among --add-dir values for a readonly
scan (folder_path is).

Fix E(1): permission_mode was identically 'acceptEdits' in both branches;
inline it at the Popen call so it is not misleadingly branch-shaped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@schenksj @claude
Fix C (PR #10 review): drop Bash(find:*) from analyze_misses diagnostic
86cb300 
Bash(find:*) permits 'find -exec <cmd>' = arbitrary command execution over the
untrusted repo_dir. Remove it from invoke_diagnostic's --allowedTools (keep the
read-only grep/wc/ls/head/tail/cat). Comment notes the residual: remaining Bash
read commands can still read host files; full hardening would drop Bash for
Read/Grep/Glob. Add test asserting Bash(find:*) is not in the constructed cmd.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@schenksj @claude
Fix D/E2 (PR #10 review): assert verify setting_sources==[user] and t…
0e335f3 
…oken_path_prefixes parse

Fix D: CANON-19 verify half was untested. Assert the recorded SDK
options.setting_sources == ['user'] so a regression to project/local is caught.

Fix E(2): guard token_path_prefixes semantics (not just importability) with a
config that includes a whitespace-only prefix; assert the blank is dropped and
the rest kept/stripped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@schenksj

schenksj commented Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

Adversarial review + fixes applied (pushed 1689cc8, 0c4edaf, 86cb300, 0e335f3).

  • Benchmark default-flip (Major) — benchmark/run.py inherited the new read-only default with no opt-out. Added a --execute flag (default off → read-only) threaded through phase_scan to scan_targets(readonly=not execute); documented.
  • Read-only write scope (Major) — dropped --add-dir os.path.dirname(folder_path) (the sibling-clone write vector under acceptEdits+Write/Edit). Comment now discloses the remaining residual honestly (SKILLS_DIR/PHASES_DIR stay readable+writable; per-path controls out of scope). New test asserts the parent dir is not in --add-dir for a default scan.
  • analyze_misses diagnostic (Minor, same class) — removed Bash(find:*) (permits find -exec = arbitrary exec) from its agent launch; kept read-only grep/wc/ls/head/tail/cat. New test asserts its absence.
  • Test gaps — added assertions that verify_runner uses setting_sources==["user"] (CANON-19) and that token_path_prefixes parses to the intended tuple.
  • Nit — inlined the now-unconditional permission_mode.

Suites: harness 162 passed, vulnhunter-agent 1112 passed / 1 skipped.

@schenksj schenksj mentioned this pull request Jul 23, 2026
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@schenksj