Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: A guard that fails blocks the call instead of counting as no decision
status: implemented
date: 2026-11-09
scope: hookprobe
---

## Decision

The three PreToolUse guards of the Claude engine, the bash guard, the input
guard and the MCP guard, are registered with the SDK's `on_failure="block"`.
A guard callback that raises, or answers in a shape the CLI rejects, now makes
the CLI refuse the call. The telemetry hook on the same event, and the audit,
hygiene and compaction hooks on other events, decide nothing and keep the
default.

## Why

The option arrived in claude-agent-sdk 0.2.177, which came in with this
relock. Before it, the SDK treated a failing hook as "no decision", and the
call ran: a guard that crashed on an input nobody foresaw fell open exactly
when it was broken. The guards are written to survive malformed input, and a
test pins that, but a boundary that depends on its own code never raising is
the shape this repository keeps finding: the selftest exists because a gate
that quietly stopped running is indistinguishable from one with nothing to
catch.

The SDK refuses to connect to a CLI that would not honour the option, so the
boundary cannot revert in silence on an older runtime either. Measured on
this change's image: the engine connects and completes a scripted turn with the
bundled CLI 2.1.319, a protected write is still refused, and prompts stay
verbatim.

## Consequences

- A bug in a guard refuses every call it is asked about, until it is fixed.
That is the right direction for a gate, and it will be visible: the run
fails its tool calls and says why.
- A CLI older than the option cannot run this engine; the SDK says so at
connect. The CLI is the one the SDK bundles, so this does not arise here.
- Not exercised: a guard made to raise on purpose, end to end. The unit test
pins the option on exactly the three guards; the SDK's own tests cover what
the CLI does with it.
- The codex and pi runtimes spawn the gate as a process and are unaffected.
2 changes: 1 addition & 1 deletion docs/containment.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ Every row here is a claim, and a claim is worth what its test is worth. `scripts
| **Chained audit** — every flight-recorder line carries the hash of the one before it; `verify_chain` names the first line that stops adding up, and `/v1/selftest` walks it | A record quietly tidied after the fact. The audit is what an investigation is reconstructed from and what a compliance reader asks about, and an append-only file cannot tell a line rewritten later from one written that way. Edits, deletions and reorderings are all caught, because each line names its predecessor rather than only hashing itself | Somebody who can rewrite the audit here AND reach the pipe's ledger. Rewriting `.chain` alone is not enough since 2026-09-10: every report carries `meta.audit_head` off-box, so a rebuilt chain is locally consistent and still fails `chain_anchored` against a head another service recorded hours earlier. Two disks and two services is the bound, not proof. Lines from before the chain existed are counted, never treated as a break — an alarm that fired on all existing history would be ignored by day two | `hookprobe/gate.py` (`append_audit`, `verify_chain`), `selftest.py` |
| **What an approved procedure may run, and with what** — `HOOKPROBE_REMEDIATION_ALLOWLIST` (full-match regexes, one per line, hot-read; no file means nothing runs), checked over the WHOLE procedure at approval and again before each step; `execution_env()` passes `PATH`/`HOME`/locale, the cloud credentials an operator mounted for the purpose (`AWS_*`, `KUBE*`, `GOOGLE_*`, `AZURE_*`) and the egress proxy variables, and nothing else; a WRITING node names what its credential may do (`HOOKPROBE_BLAST_RADIUS`, one measured line per entry as `/v1/posture` reports it) and under `enforce` a credential that can do more refuses to start, naming the excess | The one path that executes anything running a command nobody wrote down, or a half-executed 1-2-3 when the file is tightened mid-incident; a procedure reading the family's HMAC keys, the Lark app secret or the provider credential out of its own environment — until 2026-09-10 `create_subprocess_exec` passed no `env` at all; a writing node quietly growing, which under `danger-only` had nothing to be compared against for its whole life | A pattern an operator wrote too widely — the file is the decision, this only holds it, and the model proposing anything stays free. A permitted command doing something with the credentials it IS given. Anything inside the declared radius, a grant that was unwise (`iam:*` declared is `iam:*`), and a credential widened while the node runs, which is caught at the next boot and by the hourly selftest, not at the moment it changes | `hookprobe/remediation.py` (`deny_reason`, `approve`, `execute`, `execution_env`), `posture.py` (`declared_radius`, `beyond`, `verdict`) |
| **Approval window, freshness cursor and the version pressed** — a proposal is unrunnable 24h after it was made (the span the pipe gives a card's token), and it records what was true about its condition when the steps were chosen — has it ENDED, has the investigation taken another turn; if either has moved by the time somebody presses, the procedure is retired instead of run, the button is not drawn, and the refusal returns to the chat as a report. And a press names WHICH proposal it approves: the digest of its steps, run, clock and cursor as they were shown — inside the card's signed token, or as the console listed them — and a press naming none or another version is refused with the row untouched; the approved digest goes on the row | Commands chosen from evidence that has since moved being run by a click on a week-old card; approving a fix for a condition that already cleared, or running steps lifted from a report the investigation has since superseded — both reachable at any age under a clock alone, because a clock assumes the world moves at a rate; and commands nobody saw running under an approval of others — a proposal rewritten under the same id between the reading and the press | An approval at 23 hours on evidence that went stale in one is what the cursor is for; what neither sees is anything that happened outside this pipe. Nothing here reaches out to look, and it must not — the production investigator holds no credentials for the systems it writes procedures about — so somebody fixing the target by hand without saying so in chat is invisible | `hookprobe/remediation.py` (`APPROVAL_WINDOW_SECONDS`, `stale`, `cursor`, `moved`, `content_hash`, `Changed`), `actions.py`, `events.py`, `service.py` (`report_superseded`) |
| **Bash guard and input guard** — under `readonly`, mutating verbs of known CLIs with `aws` inverted to deny-unless-read, under `danger-only` only what no credential scope can undo; and writes to the files that steer the next run (`CLAUDE.md` and `AGENTS.md`, `.claude/`, the system prompt, `audit/`, `telemetry/`, `remediation/`, `actions/`, and the `results/` case files the prompt tells every run to read first, and every file configuration points at by path, wherever it lives: the MCP config, roles, allowlists, declared radius, decline patterns, runtime configs) refused in the three shapes a write arrives in — an edit tool's path argument, the paths inside a codex `apply_patch` envelope, and a shell command that redirects, `tee`s, `sed -i`s, `cp`s, `mv`s, removes (`rm`, `find -delete`) or links (`ln`) into one of them; and under every posture, a script piped or handed to a shell (`curl … \| sh`), whose verbs no rule ever reads, and printing a secret's value: a Secret's data from kubectl, a Secrets Manager value, a decrypted SSM parameter, KMS plaintext, AWS credentials; and before any tool runs, the message itself reaches the model as words, not as input typed at the CLI's prompt | An over-eager model running `kubectl delete` or `aws s3 rm`, or an installer's `curl … \| sh`; a run installing its own runbook as a persistence vector, or deleting the case files the next run reads. All three write shapes were measured getting through before they were closed: `apply_patch` because the guard read argument keys and codex puts its paths inside a patch string, `AGENTS.md` because the list only knew Claude's name for the steering file, and the shell redirect on **every** runtime under **both** postures; removing and symlinking from the shell were measured getting through on 2026-10-27; an `@path` in an alert or a chat reply making the CLI read that file into the model's context as a Read result no hook saw, and a leading `/` running a CLI command in place of the investigation, both measured getting through on 2026-11-02; a run planting an MCP config whose command every later run starts, measured getting through the same day; an over-eager model printing a Secret to see what is in it | An adversary: the rules need a binary and a verb in one pipeline segment, and separating them gets through; a redirect assembled at runtime out of two variables, or any of the endless other ways a shell can name a file; a script saved to a file and then run. These catch the obvious forms — the over-eager model, which is the case that happens. Writes anywhere else in the workspace are allowed on purpose. The model can still read a file the text names, on its own initiative; that read is a tool call the audit records. A secret read some other way: a raw API URL built at runtime, a pod that mounts it, another vendor's CLI. For the configured files the read-only mount is what holds, and the startup posture names any this UID could still write. **The read-only credential is what holds** | `hookprobe/guard.py`, `inputs.py`, `gate.py` (`patch_targets`, `shell_write_target`), `engine.py` (`verbatim_prompts`), `posture.py` (`writable_inputs`, `reads_secrets`) |
| **Bash guard and input guard** — under `readonly`, mutating verbs of known CLIs with `aws` inverted to deny-unless-read, under `danger-only` only what no credential scope can undo; and writes to the files that steer the next run (`CLAUDE.md` and `AGENTS.md`, `.claude/`, the system prompt, `audit/`, `telemetry/`, `remediation/`, `actions/`, and the `results/` case files the prompt tells every run to read first, and every file configuration points at by path, wherever it lives: the MCP config, roles, allowlists, declared radius, decline patterns, runtime configs) refused in the three shapes a write arrives in — an edit tool's path argument, the paths inside a codex `apply_patch` envelope, and a shell command that redirects, `tee`s, `sed -i`s, `cp`s, `mv`s, removes (`rm`, `find -delete`) or links (`ln`) into one of them; and under every posture, a script piped or handed to a shell (`curl … \| sh`), whose verbs no rule ever reads, and printing a secret's value: a Secret's data from kubectl, a Secrets Manager value, a decrypted SSM parameter, KMS plaintext, AWS credentials; and before any tool runs, the message itself reaches the model as words, not as input typed at the CLI's prompt; a guard that raises, or answers in a shape the CLI rejects, blocks the call where it used to count as no decision | An over-eager model running `kubectl delete` or `aws s3 rm`, or an installer's `curl … \| sh`; a run installing its own runbook as a persistence vector, or deleting the case files the next run reads. All three write shapes were measured getting through before they were closed: `apply_patch` because the guard read argument keys and codex puts its paths inside a patch string, `AGENTS.md` because the list only knew Claude's name for the steering file, and the shell redirect on **every** runtime under **both** postures; removing and symlinking from the shell were measured getting through on 2026-10-27; an `@path` in an alert or a chat reply making the CLI read that file into the model's context as a Read result no hook saw, and a leading `/` running a CLI command in place of the investigation, both measured getting through on 2026-11-02; a run planting an MCP config whose command every later run starts, measured getting through the same day; an over-eager model printing a Secret to see what is in it | An adversary: the rules need a binary and a verb in one pipeline segment, and separating them gets through; a redirect assembled at runtime out of two variables, or any of the endless other ways a shell can name a file; a script saved to a file and then run. These catch the obvious forms — the over-eager model, which is the case that happens. Writes anywhere else in the workspace are allowed on purpose. The model can still read a file the text names, on its own initiative; that read is a tool call the audit records. A secret read some other way: a raw API URL built at runtime, a pod that mounts it, another vendor's CLI. For the configured files the read-only mount is what holds, and the startup posture names any this UID could still write. **The read-only credential is what holds** | `hookprobe/guard.py`, `inputs.py`, `gate.py` (`patch_targets`, `shell_write_target`), `engine.py` (`verbatim_prompts`, `_GUARD_FAILS_CLOSED`), `posture.py` (`writable_inputs`, `reads_secrets`) |
| **Memory shape check** — a suggested fact whose SHAPE could act waits for a human | An instruction smuggled through alert text into `CLAUDE.md` | A benign-shaped line that is simply wrong | `hookprobe/suggestions.py` |
| **High-risk allowlist and target cooldown** — a step the report marked `risk: high` must also full-match a pattern in `HOOKPROBE_REMEDIATION_HIGH_RISK_ALLOWLIST`, deny-by-default in the same direction, both lists re-read before each step and one uncovered high step refusing the whole procedure at the click; after a procedure acts, a second one sharing its declared `target` **or** its literal command is refused for 15 minutes (`HOOKPROBE_REMEDIATION_COOLDOWN_SECONDS`, 0 disables), and one already `running` holds its target with no window at all | Arming remediation from also arming its worst half by the same gesture — `kubectl rollout restart .*` written to cover restarts has not licensed the steps the model itself calls dangerous; a fix and the rollback of that fix approved a minute apart, and two sequences interleaving their steps against one host. Held, not retired: the row stays `proposed` and the same press works after the window | **Keys the model writes.** A dangerous command the report labelled `low` never meets the strict list — the risk label is the MODEL's, so this can only ADD a requirement, and a mislabelled step is bounded by the ordinary allowlist exactly as before. A step that varies its command by a flag AND labels its target differently is not cooled; two keys because production named one target three ways while running character-identical commands, so the command half is the one that holds. Both count only what THIS node ran — somebody fixing the same box by hand is invisible, the same blind spot the freshness cursor has | `hookprobe/remediation.py` (`step_deny_reason`, `cooldown_keys`, `cooling`), `actions.py`, `service.py` (`report_cooling`) |
| **Refused at config load** — `on_error` per stage (`pass` or `drop`, chosen per stage, never defaulted globally); an inline decider's timeout capped at 10s; no unreachable door, starved exit, or wildcard a return door can reach; a write-back channel may say `POST`, `PUT` or `PATCH` and nothing else | A stage's failure silently changing policy; an inline decider holding the sender's connection until it retries and duplicates the alert; a config change silently feeding a brain its own output; a channel configured with `DELETE`, or a `GET` whose side effect fires when a chat client fetches a link preview — the trap `/card-action` refuses to fall into | Choosing wrong inside what is legal: an explicit fail-open, a node that answers in 9s and is still the wrong place for the work, a graph that is legal and still wrong, a `POST` that does damage at the receiver — the pipe checks the verb it is asked for, never what the far side does with it | `hookrelay/processors.py` (`MAX_INLINE_TIMEOUT_SECONDS`), `config.py` (`_WRITE_METHODS`), `topology.py`, `scripts/assert_topology.py` |
Expand Down
2 changes: 1 addition & 1 deletion hookjudge/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ httpx>=0.28.1

# Test + lint
pytest>=9.2.0
pytest-asyncio>=1.4.1
pytest-asyncio>=1.4.1.post1
pytest-cov>=7.1.0
ruff==0.17.3
bandit>=1.9.4
Expand Down
3 changes: 2 additions & 1 deletion hookprobe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,8 @@ always reach. Two things about the word *default*, before the layers:
parameter, KMS plaintext, or AWS credentials. Whether a secret exists, which
keys it holds and when it changed stay open. The model is offered a closed
list of built-in tools, so a tool a CLI upgrade adds is not offered until it
is added to that list on purpose. It errs toward over-blocking. HTTP verbs are not policed (query APIs POST), and
is added to that list on purpose. A guard that fails blocks the call rather
than counting as no decision. It errs toward over-blocking. HTTP verbs are not policed (query APIs POST), and
cloud CLIs are too many to enumerate — scope their credentials instead.
3. **Container** — non-root, disposable, nothing precious inside, no container
runtime and no socket (a run cannot start a container, and cannot build a
Expand Down
17 changes: 16 additions & 1 deletion hookprobe/hookprobe/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,15 @@ def _context_facts(usage: Any) -> dict[str, Any] | None:
return facts


# A guard that fails is a refusal, not a pass (SDK 0.2.177+). A hook callback
# that raises, or answers in a shape the CLI rejects, used to count as "no
# decision", and the call ran: the gate fell open exactly when it was broken.
# `block` makes the CLI refuse the call instead. The SDK refuses to connect to
# a CLI that would not honour it, so this cannot revert in silence either. The
# telemetry and audit hooks decide nothing and keep the default.
_GUARD_FAILS_CLOSED: Literal["block"] = "block"


def _hook_list(*fns: Callable[..., Any]) -> list[Any]:
"""The SDK types hook inputs as a TypedDict union; ours are plain dicts on
purpose — the tests never import the SDK, so its types cannot appear in
Expand Down Expand Up @@ -1112,19 +1121,25 @@ async def _step_done(input_data: dict[str, Any], tool_use_id: str | None, contex
self._settings.bash_guard, recorder, self._workdir, self._home, self._configured
)
),
on_failure=_GUARD_FAILS_CLOSED,
),
# Matched on every tool, filtered by name inside: the guard
# must not depend on how the SDK interprets a matcher
# pattern for the one thing it exists to stop.
HookMatcher(
matcher=None,
hooks=_hook_list(_input_guard_hook(self._workdir, self._home, recorder, self._configured)),
on_failure=_GUARD_FAILS_CLOSED,
),
# matcher=None for the same reason as the line above: the
# one guard standing between a colleague's chat message and
# a tool that can act as the operator must not depend on
# how the SDK interprets an `mcp__*` matcher pattern.
HookMatcher(matcher=None, hooks=_hook_list(_mcp_guard_hook(self._settings.mcp_tools, recorder))),
HookMatcher(
matcher=None,
hooks=_hook_list(_mcp_guard_hook(self._settings.mcp_tools, recorder)),
on_failure=_GUARD_FAILS_CLOSED,
),
HookMatcher(matcher=None, hooks=_hook_list(_step_begin)),
],
# The flight recorder: every tool call, every run, one JSONL
Expand Down
2 changes: 1 addition & 1 deletion hookprobe/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ requires-python = ">=3.12"
dependencies = [
"fastapi>=0.162.0",
"uvicorn[standard]>=0.54.0",
"claude-agent-sdk>=0.2.175",
"claude-agent-sdk>=0.2.176",
]

[tool.setuptools]
Expand Down
2 changes: 1 addition & 1 deletion hookprobe/requirements.lock
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ CacheControl==0.14.4
certifi==2026.10.19
cffi==2.1.1
charset-normalizer==3.5.1
claude-agent-sdk==0.2.176
claude-agent-sdk==0.2.179
click==8.4.2
coverage==7.16.2
cryptography==50.0.0
Expand Down
Loading
Loading