Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
title: The message reaches the model as words, and which CLI the engine actually runs
status: implemented
date: 2026-11-02
scope: hookprobe
---

## Decision

The investigator hands the SDK its message with `verbatim_prompts=True`. The
message is an alert's text, a reply under a card, or a watch round: written by
whoever can reach those doors, never by an operator at the CLI's prompt. Taken
as typed input, two things in it acted before any tool call:

- an `@path` anywhere in the text made the CLI read that file and give it to
the model as a Read result that no PreToolUse hook saw, so neither the input
guard nor the audit knew a read had happened;
- a message starting with `/` ran a CLI command instead of an investigation:
`/cost show me` came back as the run's report, a cost table.

Measured with the real service in its image, a recording stand-in for the model
API on an internal Docker network, and a marker file mounted into the
container. A request naming the marker with `@` put the marker's content into
the request the stand-in received, on the published 0.9.2 image and on this
change's image before the switch. After the switch the content never arrived,
`/cost show me` reached the model as text, and the scripted turn still ran: an
allowed read executed, and a `touch` into `results/` was refused and left no
file.

The option arrived in claude-agent-sdk 0.2.158 and needs CLI 2.1.248 or newer.
It came with the relock of Dependabot's first proposal for the investigator's
runtime, which moved the SDK from 0.2.140 to 0.2.176.

## Why

The guard and the audit are built on tool calls. A read that happens before any
tool call is outside both, so the only fix that holds is to stop the CLI from
reading on the text's say-so. The SDK ships that switch for exactly this case.
A filter of our own over the text would have to know every way the CLI
interprets input, and that changes with every CLI release.

Nothing relied on the expansion. No message the service builds carries a slash
command or an `@` mention, and skills load through setting sources, not slash
commands.

## Consequences

- A person who writes `@/some/file` in a chat reply no longer gets the file
attached. The model sees the words and may read the file itself, as a tool
call the guard checks and the audit records.
- A slash command in a reply is text for the model, not a CLI command.
- `tests/test_engine_loop.py` pins the option, and
`scripts/assert_guards_are_tested.py` flips it and requires that test to fail.
- Not measured: the codex and pi runtimes, which hand the message to their own
CLIs.
- Found on the way and not decided here: the engine runs the CLI bundled inside
the SDK wheel, not the one the Dockerfile installs from npm with
`CLAUDE_CODE_VERSION`. The SDK looks for its bundled binary first. The 0.9.2
image ran 2.1.235 with 2.1.259 unused on PATH, and this relock moves the
engine to 2.1.316. So the lock, not the build argument, has been pinning the
agent runtime since the SDK began bundling it. Whether the image should drop
the npm CLI or point the engine at it is the operator's call.
2 changes: 1 addition & 1 deletion docs/containment.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ Every row here is a claim, and a claim is worth what its test is worth. `scripts
| **Chained audit** — every flight-recorder line carries the hash of the one before it; `verify_chain` names the first line that stops adding up, and `/v1/selftest` walks it | A record quietly tidied after the fact. The audit is what an investigation is reconstructed from and what a compliance reader asks about, and an append-only file cannot tell a line rewritten later from one written that way. Edits, deletions and reorderings are all caught, because each line names its predecessor rather than only hashing itself | Somebody who can rewrite the audit here AND reach the pipe's ledger. Rewriting `.chain` alone is not enough since 2026-09-10: every report carries `meta.audit_head` off-box, so a rebuilt chain is locally consistent and still fails `chain_anchored` against a head another service recorded hours earlier. Two disks and two services is the bound, not proof. Lines from before the chain existed are counted, never treated as a break — an alarm that fired on all existing history would be ignored by day two | `hookprobe/gate.py` (`append_audit`, `verify_chain`), `selftest.py` |
| **What an approved procedure may run, and with what** — `HOOKPROBE_REMEDIATION_ALLOWLIST` (full-match regexes, one per line, hot-read; no file means nothing runs), checked over the WHOLE procedure at approval and again before each step; `execution_env()` passes `PATH`/`HOME`/locale, the cloud credentials an operator mounted for the purpose (`AWS_*`, `KUBE*`, `GOOGLE_*`, `AZURE_*`) and the egress proxy variables, and nothing else; a WRITING node names what its credential may do (`HOOKPROBE_BLAST_RADIUS`, one measured line per entry as `/v1/posture` reports it) and under `enforce` a credential that can do more refuses to start, naming the excess | The one path that executes anything running a command nobody wrote down, or a half-executed 1-2-3 when the file is tightened mid-incident; a procedure reading the family's HMAC keys, the Lark app secret or the provider credential out of its own environment — until 2026-09-10 `create_subprocess_exec` passed no `env` at all; a writing node quietly growing, which under `danger-only` had nothing to be compared against for its whole life | A pattern an operator wrote too widely — the file is the decision, this only holds it, and the model proposing anything stays free. A permitted command doing something with the credentials it IS given. Anything inside the declared radius, a grant that was unwise (`iam:*` declared is `iam:*`), and a credential widened while the node runs, which is caught at the next boot and by the hourly selftest, not at the moment it changes | `hookprobe/remediation.py` (`deny_reason`, `approve`, `execute`, `execution_env`), `posture.py` (`declared_radius`, `beyond`, `verdict`) |
| **Approval window, freshness cursor and the version pressed** — a proposal is unrunnable 24h after it was made (the span the pipe gives a card's token), and it records what was true about its condition when the steps were chosen — has it ENDED, has the investigation taken another turn; if either has moved by the time somebody presses, the procedure is retired instead of run, the button is not drawn, and the refusal returns to the chat as a report. And a press names WHICH proposal it approves: the digest of its steps, run, clock and cursor as they were shown — inside the card's signed token, or as the console listed them — and a press naming none or another version is refused with the row untouched; the approved digest goes on the row | Commands chosen from evidence that has since moved being run by a click on a week-old card; approving a fix for a condition that already cleared, or running steps lifted from a report the investigation has since superseded — both reachable at any age under a clock alone, because a clock assumes the world moves at a rate; and commands nobody saw running under an approval of others — a proposal rewritten under the same id between the reading and the press | An approval at 23 hours on evidence that went stale in one is what the cursor is for; what neither sees is anything that happened outside this pipe. Nothing here reaches out to look, and it must not — the production investigator holds no credentials for the systems it writes procedures about — so somebody fixing the target by hand without saying so in chat is invisible | `hookprobe/remediation.py` (`APPROVAL_WINDOW_SECONDS`, `stale`, `cursor`, `moved`, `content_hash`, `Changed`), `actions.py`, `events.py`, `service.py` (`report_superseded`) |
| **Bash guard and input guard** — under `readonly`, mutating verbs of known CLIs with `aws` inverted to deny-unless-read, under `danger-only` only what no credential scope can undo; and writes to the files that steer the next run (`CLAUDE.md` and `AGENTS.md`, `.claude/`, the system prompt, `audit/`, `telemetry/`, `remediation/`, `actions/`, and the `results/` case files the prompt tells every run to read first) refused in the three shapes a write arrives in — an edit tool's path argument, the paths inside a codex `apply_patch` envelope, and a shell command that redirects, `tee`s, `sed -i`s, `cp`s, `mv`s, removes (`rm`, `find -delete`) or links (`ln`) into one of them; and under every posture, a script piped or handed to a shell (`curl … \| sh`), whose verbs no rule ever reads | An over-eager model running `kubectl delete` or `aws s3 rm`, or an installer's `curl … \| sh`; a run installing its own runbook as a persistence vector, or deleting the case files the next run reads. All three write shapes were measured getting through before they were closed: `apply_patch` because the guard read argument keys and codex puts its paths inside a patch string, `AGENTS.md` because the list only knew Claude's name for the steering file, and the shell redirect on **every** runtime under **both** postures; removing and symlinking from the shell were measured getting through on 2026-10-27 | An adversary: the rules need a binary and a verb in one pipeline segment, and separating them gets through; a redirect assembled at runtime out of two variables, or any of the endless other ways a shell can name a file; a script saved to a file and then run. These catch the obvious forms — the over-eager model, which is the case that happens. Writes anywhere else in the workspace are allowed on purpose. **The read-only credential is what holds** | `hookprobe/guard.py`, `inputs.py`, `gate.py` (`patch_targets`, `shell_write_target`) |
| **Bash guard and input guard** — under `readonly`, mutating verbs of known CLIs with `aws` inverted to deny-unless-read, under `danger-only` only what no credential scope can undo; and writes to the files that steer the next run (`CLAUDE.md` and `AGENTS.md`, `.claude/`, the system prompt, `audit/`, `telemetry/`, `remediation/`, `actions/`, and the `results/` case files the prompt tells every run to read first) refused in the three shapes a write arrives in — an edit tool's path argument, the paths inside a codex `apply_patch` envelope, and a shell command that redirects, `tee`s, `sed -i`s, `cp`s, `mv`s, removes (`rm`, `find -delete`) or links (`ln`) into one of them; and under every posture, a script piped or handed to a shell (`curl … \| sh`), whose verbs no rule ever reads; and before any tool runs, the message itself reaches the model as words, not as input typed at the CLI's prompt | An over-eager model running `kubectl delete` or `aws s3 rm`, or an installer's `curl … \| sh`; a run installing its own runbook as a persistence vector, or deleting the case files the next run reads. All three write shapes were measured getting through before they were closed: `apply_patch` because the guard read argument keys and codex puts its paths inside a patch string, `AGENTS.md` because the list only knew Claude's name for the steering file, and the shell redirect on **every** runtime under **both** postures; removing and symlinking from the shell were measured getting through on 2026-10-27; an `@path` in an alert or a chat reply making the CLI read that file into the model's context as a Read result no hook saw, and a leading `/` running a CLI command in place of the investigation, both measured getting through on 2026-11-02 | An adversary: the rules need a binary and a verb in one pipeline segment, and separating them gets through; a redirect assembled at runtime out of two variables, or any of the endless other ways a shell can name a file; a script saved to a file and then run. These catch the obvious forms — the over-eager model, which is the case that happens. Writes anywhere else in the workspace are allowed on purpose. The model can still read a file the text names, on its own initiative; that read is a tool call the audit records. **The read-only credential is what holds** | `hookprobe/guard.py`, `inputs.py`, `gate.py` (`patch_targets`, `shell_write_target`), `engine.py` (`verbatim_prompts`) |
| **Memory shape check** — a suggested fact whose SHAPE could act waits for a human | An instruction smuggled through alert text into `CLAUDE.md` | A benign-shaped line that is simply wrong | `hookprobe/suggestions.py` |
| **High-risk allowlist and target cooldown** — a step the report marked `risk: high` must also full-match a pattern in `HOOKPROBE_REMEDIATION_HIGH_RISK_ALLOWLIST`, deny-by-default in the same direction, both lists re-read before each step and one uncovered high step refusing the whole procedure at the click; after a procedure acts, a second one sharing its declared `target` **or** its literal command is refused for 15 minutes (`HOOKPROBE_REMEDIATION_COOLDOWN_SECONDS`, 0 disables), and one already `running` holds its target with no window at all | Arming remediation from also arming its worst half by the same gesture — `kubectl rollout restart .*` written to cover restarts has not licensed the steps the model itself calls dangerous; a fix and the rollback of that fix approved a minute apart, and two sequences interleaving their steps against one host. Held, not retired: the row stays `proposed` and the same press works after the window | **Keys the model writes.** A dangerous command the report labelled `low` never meets the strict list — the risk label is the MODEL's, so this can only ADD a requirement, and a mislabelled step is bounded by the ordinary allowlist exactly as before. A step that varies its command by a flag AND labels its target differently is not cooled; two keys because production named one target three ways while running character-identical commands, so the command half is the one that holds. Both count only what THIS node ran — somebody fixing the same box by hand is invisible, the same blind spot the freshness cursor has | `hookprobe/remediation.py` (`step_deny_reason`, `cooldown_keys`, `cooling`), `actions.py`, `service.py` (`report_cooling`) |
| **Refused at config load** — `on_error` per stage (`pass` or `drop`, chosen per stage, never defaulted globally); an inline decider's timeout capped at 10s; no unreachable door, starved exit, or wildcard a return door can reach; a write-back channel may say `POST`, `PUT` or `PATCH` and nothing else | A stage's failure silently changing policy; an inline decider holding the sender's connection until it retries and duplicates the alert; a config change silently feeding a brain its own output; a channel configured with `DELETE`, or a `GET` whose side effect fires when a chat client fetches a link preview — the trap `/card-action` refuses to fall into | Choosing wrong inside what is legal: an explicit fail-open, a node that answers in 9s and is still the wrong place for the work, a graph that is legal and still wrong, a `POST` that does damage at the receiver — the pipe checks the verb it is asked for, never what the far side does with it | `hookrelay/processors.py` (`MAX_INLINE_TIMEOUT_SECONDS`), `config.py` (`_WRITE_METHODS`), `topology.py`, `scripts/assert_topology.py` |
Expand Down
6 changes: 3 additions & 3 deletions hookjudge/requirements.lock
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ license-expression==30.4.4
markdown-it-py==4.2.0
mdurl==0.1.2
msgpack==1.2.4
mypy==2.4.0
mypy==2.4.1
mypy_extensions==1.1.0
opentelemetry-api==1.45.1
packageurl-python==0.17.6
Expand All @@ -41,14 +41,14 @@ pydantic==2.13.4
pydantic_core==2.46.4
Pygments==2.21.0
pyparsing==3.3.2
pytest==9.1.1
pytest==9.2.0
pytest-asyncio==1.4.1.post1
pytest-cov==7.1.0
python-dotenv==1.2.3
PyYAML==6.0.3
requests==2.34.2
rich==15.0.0
ruff==0.17.2
ruff==0.17.3
sortedcontainers==2.4.0
starlette==1.6.0
stevedore==5.9.0
Expand Down
6 changes: 3 additions & 3 deletions hookjudge/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ aiosqlite>=0.22.1
httpx>=0.28.1

# Test + lint
pytest>=9.1.1
pytest>=9.2.0
pytest-asyncio>=1.4.1
pytest-cov>=7.1.0
ruff==0.17.2
ruff==0.17.3
bandit>=1.9.4
pip-audit>=2.10.1
mypy>=2.4.0
mypy>=2.4.1
12 changes: 12 additions & 0 deletions hookprobe/hookprobe/engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -1047,6 +1047,18 @@ async def _step_done(input_data: dict[str, Any], tool_use_id: str | None, contex
# console is for, and an agent that says nothing for two minutes and
# then everything at once is indistinguishable from a hung one.
include_partial_messages=True,
# The message is not someone typing at the CLI's prompt: it is an
# alert's text, a reply under a card, a watch round, written by
# whoever can reach those doors. Delivered as typed input, an
# `@path` anywhere in it made the CLI read that file and hand it to
# the model as a Read result no PreToolUse hook saw, so neither the
# guard nor the audit knew the read happened; and a message that
# starts with `/` ran a CLI command instead of an investigation
# (`/cost` came back as the report). Both were measured on the
# 0.9.2 image against a recording stand-in model. Verbatim, the
# words arrive as words; the model can still ask to read a file,
# and that read is a tool call the guard and the audit see.
verbatim_prompts=True,
# The declared MCP tools ride along so the CLI exposes them at all.
# This is visibility, not enforcement — with bypassPermissions an
# allowlist is a preference. _mcp_guard_hook is the gate.
Expand Down
8 changes: 4 additions & 4 deletions hookprobe/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[build-system]
requires = ["setuptools>=68"]
requires = ["setuptools>=84.2.1"]
build-backend = "setuptools.build_meta"

[project]
Expand All @@ -9,9 +9,9 @@ description = "Single-purpose deep-analysis agent runner: OpenClaw-compatible co
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
"fastapi>=0.115",
"uvicorn[standard]>=0.34",
"claude-agent-sdk>=0.1.0",
"fastapi>=0.162.0",
"uvicorn[standard]>=0.54.0",
"claude-agent-sdk>=0.2.175",
]

[tool.setuptools]
Expand Down
12 changes: 6 additions & 6 deletions hookprobe/requirements.lock
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ CacheControl==0.14.4
certifi==2026.10.19
cffi==2.1.1
charset-normalizer==3.5.1
claude-agent-sdk==0.2.140
claude-agent-sdk==0.2.176
click==8.4.2
coverage==7.16.2
cryptography==50.0.0
cyclonedx-python-lib==11.12.0
defusedxml==0.7.1
fastapi==0.141.1
fastapi==0.162.0
filelock==4.2.4
h11==0.16.0
httpcore==1.0.9
Expand All @@ -34,7 +34,7 @@ mcp==2.3.0
mcp-types==2.3.0
mdurl==0.1.2
msgpack==1.2.4
mypy==2.4.0
mypy==2.4.1
mypy_extensions==1.1.0
opentelemetry-api==1.45.1
packageurl-python==0.17.6
Expand All @@ -52,7 +52,7 @@ pydantic_core==2.46.4
Pygments==2.21.0
PyJWT==2.15.1
pyparsing==3.3.2
pytest==9.1.1
pytest==9.2.0
pytest-cov==7.1.0
python-dotenv==1.2.3
python-multipart==0.0.32
Expand All @@ -61,7 +61,7 @@ referencing==0.37.0
requests==2.34.2
rich==15.0.0
rpds-py==2026.6.3
ruff==0.17.2
ruff==0.17.3
sniffio==1.3.1
sortedcontainers==2.4.0
sse-starlette==3.4.8
Expand All @@ -73,7 +73,7 @@ truststore==0.10.4
typing-inspection==0.4.4
typing_extensions==4.16.0
urllib3==2.8.0
uvicorn==0.52.4
uvicorn==0.54.0
uvloop==0.22.1
watchfiles==1.2.0
websockets==17.0.1
12 changes: 6 additions & 6 deletions hookprobe/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Three runtime dependencies, chosen to stay three. The agent loop, built-in
# tools, MCP client and SKILL.md loading all come from the Claude Agent SDK —
# hookprobe itself is only a contract shell around one engine call.
fastapi>=0.115
uvicorn[standard]>=0.34
claude-agent-sdk>=0.1.0
fastapi>=0.162.0
uvicorn[standard]>=0.54.0
claude-agent-sdk>=0.2.175

# Test + lint (tests never import the SDK; they inject fake engines)
pytest>=9.1.1
pytest>=9.2.0
httpx>=0.28.1
pytest-cov>=7.1.0
ruff==0.17.2
ruff==0.17.3
bandit>=1.9.4
pip-audit>=2.10.1
mypy>=2.4.0
mypy>=2.4.1
Loading
Loading