diff --git a/.agents/notes/implemented/2026-11-02-the-lock-pins-the-agent-runtime.md b/.agents/notes/implemented/2026-11-02-the-lock-pins-the-agent-runtime.md index 860d9bd..abbb6e8 100644 --- a/.agents/notes/implemented/2026-11-02-the-lock-pins-the-agent-runtime.md +++ b/.agents/notes/implemented/2026-11-02-the-lock-pins-the-agent-runtime.md @@ -34,6 +34,9 @@ have to agree. The operator chose the lock. - A relock of claude-agent-sdk is a CLI bump. The SDK's changelog names the CLI each release bundles, and `claude_agent_sdk/_cli_version.py` names it inside an image. +- A CLI bump can move the tool surface too: 2.1.316 added Monitor, ListAgents + and ToolSearch. The engine's built-in tools are a closed list since + 2026-11-03 (2026-11-03-the-investigators-built-in-tools-are-a-closed-list). - `claude` is no longer on PATH in the image, so `claude --version` there fails; read the version file instead. - The image lost the npm CLI: 1.69 GB before and 1.17 GB after, in a local diff --git a/.agents/notes/implemented/2026-11-03-the-investigators-built-in-tools-are-a-closed-list.md b/.agents/notes/implemented/2026-11-03-the-investigators-built-in-tools-are-a-closed-list.md new file mode 100644 index 0000000..2e3eca4 --- /dev/null +++ b/.agents/notes/implemented/2026-11-03-the-investigators-built-in-tools-are-a-closed-list.md @@ -0,0 +1,53 @@ +--- +title: The investigator's built-in tools are a closed list, because a CLI upgrade moved them unreviewed +status: implemented +date: 2026-11-03 +scope: hookprobe +--- + +## Decision + +The Claude engine passes the SDK a base set of built-in tools (`tools`), and +the list is closed: Bash, Read, Write, Edit, Glob, Grep, WebSearch, WebFetch, +Skill, Agent, and the four task-list tools that replaced TodoWrite. The CLI +offers nothing else, in a run or in its subagents. MCP tools are untouched; +`HOOKPROBE_MCP_TOOLS` and the MCP guard already decide those. ToolSearch is +left out, so nothing is deferred and every listed tool is offered directly. + +## Why + +A second deployment running copies of these files adds each new command its +upstream ships to its deny list as soon as an upgrade brings one. Asked here, +the same question had no answer. Under `bypassPermissions` the engine's +`allowed_tools` restricts nothing, so a run could reach whatever the bundled CLI +shipped, and the CLI moves with every relock of the SDK. Measured with a +recording stand-in model on the published images: CLI 2.1.235 (0.9.2) offered +26 tools, and CLI 2.1.316 (0.10.0) offered 14 directly and 15 more behind +ToolSearch. Monitor, ListAgents and ToolSearch were new, and TaskOutput was +gone. None of it had been reviewed. + +Monitor runs shell commands. Loaded through ToolSearch, its write into +`results/` was still refused, because the CLI runs the Bash hook for it. That +held by the CLI's choice, not by anything in this repository, and the bash +guard's own matcher only names Bash. A closed list turns "the next tool will +probably be covered" into "the next tool is not offered until someone adds it +here". + +Measured on this change's image with a one-tool MCP server: exactly the listed +tools and the MCP tool are offered, nothing is deferred, the MCP tool answers, +and Monitor and ToolSearch are refused as tools that do not exist. + +## Consequences + +- A tool a later CLI adds stays out until it is added to `_TOOLS` and to the + test that pins it, on purpose. +- The CLI ignores a name it does not know. A tool that a later CLI renames + therefore drops out silently, and an SDK bump has to be checked against what + the new CLI offers. It still accepts `Task` as the old name for `Agent`. +- An operator who wants a tool off the list, such as NotebookEdit or a + scheduler, adds it in the engine. There is no setting for it, on purpose. +- Not changed: the console's live plan still reads TodoWrite calls, which no + current CLI offers. The task-list tools that replaced it show as ordinary + steps. +- The codex and pi runtimes have their own tool models and are not covered by + this list. diff --git a/docs/containment.md b/docs/containment.md index d8b75fa..c43bacf 100644 --- a/docs/containment.md +++ b/docs/containment.md @@ -59,7 +59,7 @@ cannot reach. Its limits are in the third column; its key is not model-written. - **Signatures at every door** — HMACs the model has no secret for: the pipe's doors, a card's buttons, the bridge's inbound. - **Per-node credentials, and the agent's bearer reads only** — the real boundary, which lives outside this repository; inside a node the agent's own token can only `GET`. -- **Closed sets at the doors** — a payload, a verdict or a tool call picks from inside a set the config declared; free text cannot widen it. The MCP list binds the agent's TOOL calls and not its sockets, which is why the row below exists. +- **Closed sets at the doors** — a payload, a verdict or a tool call picks from inside a set the config declared, and the model picks its built-in tools from a list the engine declares; free text cannot widen either. The MCP list binds the agent's TOOL calls and not its sockets, which is why the row below exists. - **The chat's tools are on a list in another container** — a network with no route, and a list the agent's process cannot edit. The tools that write are not refused, they are unreachable; a hostile model has nowhere to send the request. - **The one way out** — the same network with no route, and a forward proxy in another container whose list the agent's process cannot edit: an unlisted destination is unreachable, not refused. What it bounds is WHERE, not what — a listed host that accepts bodies carries whatever the model puts in them. - **The watcher's signature is held where no agent runs** — the key is in another container and the check in front of it is against a file the agent does not write. What it bounds is which conversations can carry a signal, not whether one is honest. @@ -139,7 +139,7 @@ Every row here is a claim, and a claim is worth what its test is worth. `scripts |---|---|---|---| | **Signatures at every door** — every pipe door verifies `sha256(secret, "{ts}.{body}")`; a card's button carries a signed single-use token; the bridge accepts a card only with `X-Hook-Timestamp` + `X-Hook-Signature` over the exact bytes, within five minutes | An unsigned or forged event reaching a door; anyone in a group chat pressing on your behalf; a neighbour on the shared container network posting a card into the operator's chat AS the application — the bridge's port has no other guard, and since 2026-10-12 it refuses to start without a secret | A replay inside the skew, which the pipe CLOSES and the bridge only BOUNDS: the pipe claims each signature once (`HOOKRELAY_SIGNATURE_REPLAY_SECONDS`, 300; 0 leaves the window bounded) and answers the second copy as a duplicate, while the bridge's five-minute window is still bounded, not closed, and its legacy in-body sign covers only the timestamp. Nothing when a secret is empty: with `HOOKRELAY_ACTION_SECRET` unset the buttons render and do nothing, and an unset `BRIDGE_INBOUND_SECRET` is now a refusal to start (`BRIDGE_ALLOW_UNSIGNED=1` is the loud opt-out); on the investigator, an empty `HOOKPROBE_ACTION_SECRET`/`HOOKPROBE_EVENT_SECRET` still leaves the door unsigned for followups and rulings, but `approve` — the one kind that executes — answers `503` rather than let a run press its own proposal | `hookrelay/security.py` (`replay_token`), `hookrelay/store.py` (`claim_signature`), `hookprobe/wire.py`, `hookrelay/actions.py`, `app.py`, `deploy/lark-bridge/bridge.py` (`is_authentic_protocol`) | | **Per-node credentials, and the agent's bearer reads only** — each process-external node gets its own secrets, budget and guards; inside a node the console's `HOOKPROBE_TOKEN` is blanked out of the agent's subprocess and replaced with `HOOKPROBE_AGENT_TOKEN`, which the API refuses on every method but `GET` | One node's compromise becoming every node's; an injected instruction reaching a Bash step and using the service's own write surface: `PUT /v1/memory` (no shape check, unlike the suggestion path), `PUT /v1/skills`, `/v1/system-prompt`, approving a remediation, and `POST /hooks/agent`, which spends money. The regret route *claimed* this for its whole life while the token was inherited | Anything inside a node once it is compromised; everything a `GET` exposes, which is most of a node's own state; and anything the agent can do without this API, which is the whole shell. The real boundary is the credential each node is given, and that lives outside this repository | the compose files; `hookprobe/settings.py` (`agent_token`), `app.py` (`require_token`), `gate.py` (`agent_credentials`) | -| **Closed sets at the doors** — config names the channel set a payload may pick from (the bypass lanes); `HOOKPROBE_VERDICTS` and `HOOKPROBE_MCP_TOOLS` are closed sets, and empty denies all | An injected message routing itself around a gatekeeper; an investigator's free text choosing which lane spends money; a chat message asking the agent to use a write tool on a mounted server | A wrong choice among lanes the operator already declared. **A socket.** The MCP allowlist is enforced where the agent asks for a TOOL and says nothing about a request the agent makes itself: measured on 2026-09-30, one plain HTTP POST from inside the watcher reached the chat client's own MCP port, with no credential, and called a tool this list does not name. The row below is the answer; this one alone was advice | `hookrelay/config.py`, `routing.py`; `hookprobe/reports.py` (`verdict`), `engine.py` (`mcp_deny_reason`) | +| **Closed sets at the doors** — config names the channel set a payload may pick from (the bypass lanes); `HOOKPROBE_VERDICTS` and `HOOKPROBE_MCP_TOOLS` are closed sets, and empty denies all; and the built-in tools the model is offered are a closed list in the engine, so a tool a CLI upgrade adds is not offered until it is added on purpose | An injected message routing itself around a gatekeeper; an investigator's free text choosing which lane spends money; a chat message asking the agent to use a write tool on a mounted server; a tool an SDK relock brings in reaching an investigation unreviewed (CLI 2.1.316 added Monitor, ListAgents and ToolSearch, measured 2026-11-03) | A wrong choice among lanes the operator already declared. **A socket.** The MCP allowlist is enforced where the agent asks for a TOOL and says nothing about a request the agent makes itself: measured on 2026-09-30, one plain HTTP POST from inside the watcher reached the chat client's own MCP port, with no credential, and called a tool this list does not name. The row below is the answer; this one alone was advice. A tool on the list that behaves differently in a new CLI, and a listed tool a new CLI renames, which drops out without a word | `hookrelay/config.py`, `routing.py`; `hookprobe/reports.py` (`verdict`), `engine.py` (`mcp_deny_reason`, `_TOOLS`) | | **The chat's tools are on a list in another container** — the probes sit on a docker network with `internal: true`, so they have no route to the host or the internet at all; `mcp-gate` holds the only reach to the chat client and forwards a `tools/call` only when the tool's name is on the calling node's list, recording both halves. Several MCP servers can share its one port as routes: each keeps its own credential, which never reaches another, and a node's list names tools per route | An agent that goes around its own MCP allowlist with an HTTP request, which is what the row above could not stop. The writes are not refused, they are unreachable: the watcher sees 8 read tools where the server offers 34, 20 of them writes | Anything a PERMITTED read does: an argument is not checked, and what a read returns still goes to the model provider. The server-to-client stream is declined rather than relayed (405), so a chat server that pushed notifications would not reach the agent through this. A client token lives in the agent's own MCP config by construction — it admits exactly the tools that agent already had, which is the point of the list being here rather than there | `deploy/sidecars/gate.py`, `deploy/docker-compose.work.yml` (`probe_net`) | | **The one way out** — the probes' only route to anything is the egress proxy in the doors container, on an allowlist of names: a bare name matches exactly, a leading dot admits a tree, and two shapes (`@aws-api`, `@aws-s3`) admit AWS service APIs and buckets without admitting the hostnames anyone can obtain under that suffix. A listed name is resolved and refused if any answer is a loopback, link-local, private, reserved or host-mapped address; compose peers (`EGRESS_INTERNAL`) must resolve inside the compose network; the proxy connects to the address it vetted and never looks the name up twice; only the listed ports open; every refusal is logged and, with `EGRESS_LEDGER` set, written to a ledger that outlives the container | A credential read out of a tool's own environment and posted to an address nobody listed — measured before this existed, a Bash step inside the investigator could read the provider key and reach anywhere; a listed name that resolves to the host or the LAN; a name used as a DNS exfiltration channel, since the list is checked before any lookup; a tunnel to 22 or 5432 | A listed host that accepts attacker-chosen bodies: the model gateway takes POST bodies and anyone can create an S3 bucket, so the shapes narrow the door without closing it; what a permitted destination returns still reaches the model; on a deployment without the internal network a client that ignores the proxy variables goes direct | `deploy/sidecars/proxy.py` (`permitted`, `decide`, `resolve`, `connect`), `deploy/docker-compose.work.yml` (`probe_net`, `EGRESS_ALLOW`, `EGRESS_INTERNAL`) | | **The watcher's signature is held where no agent runs** — `watch-signer` holds `WATCH_INGEST_SECRET`; the watcher posts its signal unsigned to it, and it signs only a signal naming a conversation the scanner offered THIS round, with a level and kind from a closed set, text cut to a length, and a ceiling per round | A round steered by a colleague's message manufacturing a request from a conversation nobody scanned — at `high`, as a `task`, which buys a paid planner run and a card the operator reads as real. Until 2026-09-30 the secret sat in a file the agent could read, so the door's only check answered yes to whatever the round chose to say | A round that read a real conversation and describes it dishonestly: the offer bounds WHICH conversations can carry a signal, not whether the signal is true. Nothing here is a judgement about content, and a deployment with no prescan states no offer, which is passed through rather than read as an offer of nothing. The producer half of an origin is a closed set only where `WATCH_SIGNER_PRODUCERS` names one (the work compose does: the chat tool, Jira, the scanner); the scanner's own `scanner-notes` subject is admitted without an offer at `low`/`note`, so a round can always post one cheap note; and the offer is this round's or the round before it, so a run that outlasts a tick is checked against what it was handed, one round wide. Its ledger is also the record of what was reported (conversation and offered cursor); `scripts/assert_watch_signed.py` reads it against the pipe's ledger once per tick, and a signal in the pipe with no signer row is the one thing it pages about | `deploy/sidecars/signer.py`, `scripts/post_watch_signal.py` | diff --git a/hookprobe/README.md b/hookprobe/README.md index 09c2de9..c498492 100644 --- a/hookprobe/README.md +++ b/hookprobe/README.md @@ -238,7 +238,9 @@ always reach. Two things about the word *default*, before the layers: is refused, because its verbs never reach a rule. So is printing a secret's value: a Secret's data from kubectl, a Secrets Manager value, a decrypted SSM parameter, KMS plaintext, or AWS credentials. Whether a secret exists, which - keys it holds and when it changed stay open. It errs toward over-blocking. HTTP verbs are not policed (query APIs POST), and + keys it holds and when it changed stay open. The model is offered a closed + list of built-in tools, so a tool a CLI upgrade adds is not offered until it + is added to that list on purpose. It errs toward over-blocking. HTTP verbs are not policed (query APIs POST), and cloud CLIs are too many to enumerate — scope their credentials instead. 3. **Container** — non-root, disposable, nothing precious inside, no container runtime and no socket (a run cannot start a container, and cannot build a diff --git a/hookprobe/hookprobe/engine.py b/hookprobe/hookprobe/engine.py index cc5ad99..df3d698 100644 --- a/hookprobe/hookprobe/engine.py +++ b/hookprobe/hookprobe/engine.py @@ -46,7 +46,17 @@ # investigates, enforced by the bash guard plus the credentials mounted into # the container. Task enables parallel sub-investigations for cascading # incidents; hooks (and so both guards) apply inside subagents too. -_ALLOWED_TOOLS = [ +# The built-in tools this engine offers the model: a closed set. Under +# bypassPermissions `allowed_tools` restricts nothing, so until 2026-11-03 a run +# could reach whatever the bundled CLI shipped, and the CLI moves with every +# relock of the SDK. 2.1.316 (v0.10.0) added Monitor, ListAgents and ToolSearch +# and put most of the rest behind ToolSearch, none of it reviewed here. `tools` +# is the base set the CLI offers at all, so a tool a later CLI adds stays out +# until it is added here on purpose. ToolSearch is left out, so nothing is +# deferred and every tool below is offered directly. The CLI ignores a name it +# does not know, so a renamed tool drops out without a word: check this list +# against what a new SDK offers whenever it moves. +_TOOLS = ( "Bash", "Read", "Write", @@ -55,11 +65,15 @@ "Grep", "WebSearch", "WebFetch", - "TodoWrite", "Skill", - "Task", - "Agent", # the CLI's current name for the subagent tool; keep both spellings -] + # Subagents; the CLI still accepts the old name `Task` for it. + "Agent", + # The task list that replaced TodoWrite in the CLI: planning, in-session only. + "TaskCreate", + "TaskGet", + "TaskList", + "TaskUpdate", +) @dataclass(frozen=True, slots=True) @@ -1071,7 +1085,9 @@ async def _step_done(input_data: dict[str, Any], tool_use_id: str | None, contex # The declared MCP tools ride along so the CLI exposes them at all. # This is visibility, not enforcement — with bypassPermissions an # allowlist is a preference. _mcp_guard_hook is the gate. - allowed_tools=[*_ALLOWED_TOOLS, *sorted(self._settings.mcp_tools)], + allowed_tools=[*_TOOLS, *sorted(self._settings.mcp_tools)], + # The base set the CLI offers at all. See _TOOLS for why it is closed. + tools=list(_TOOLS), max_turns=self._settings.max_turns, # Keep the engine's own system prompt; append the operator's # methodology when a system-prompt file is present. diff --git a/hookprobe/hookprobe/gate.py b/hookprobe/hookprobe/gate.py index 303926b..0ecf3b5 100644 --- a/hookprobe/hookprobe/gate.py +++ b/hookprobe/hookprobe/gate.py @@ -93,7 +93,7 @@ def mcp_deny_reason(tool_name: str, allowed: frozenset[str]) -> str | None: second thing to get subtly wrong, and the list is meant to be read by somebody deciding what an agent may do on their behalf. - Non-MCP tools are not this guard's business; `_ALLOWED_TOOLS` and the bash + Non-MCP tools are not this guard's business; `_TOOLS` (engine.py) and the bash and input guards already answer for those. """ if not tool_name.startswith("mcp__"): diff --git a/hookprobe/tests/test_engine_loop.py b/hookprobe/tests/test_engine_loop.py index 1f21fdb..cd73e4c 100644 --- a/hookprobe/tests/test_engine_loop.py +++ b/hookprobe/tests/test_engine_loop.py @@ -128,6 +128,34 @@ def test_the_message_reaches_the_model_as_words_not_as_cli_input(engine) -> None assert _FakeClient.options_seen.verbatim_prompts is True +def test_the_tools_the_model_is_offered_are_a_closed_set(engine) -> None: + """Under bypassPermissions an allowlist restricts nothing; the base set is + what the CLI offers at all. CLI 2.1.316 (v0.10.0) added Monitor, ListAgents + and ToolSearch with nobody choosing them. A tool joins this set on purpose, + in this file and in this test, or not at all.""" + _run(engine, [_result()]) + + offered = _FakeClient.options_seen.tools + assert offered == [ + "Bash", + "Read", + "Write", + "Edit", + "Glob", + "Grep", + "WebSearch", + "WebFetch", + "Skill", + "Agent", + "TaskCreate", + "TaskGet", + "TaskList", + "TaskUpdate", + ] + kept_out = {"Monitor", "Workflow", "SendMessage", "ListAgents", "CronCreate", "ToolSearch", "EnterWorktree"} + assert kept_out.isdisjoint(offered) + + def test_deltas_are_streamed_but_not_counted(engine) -> None: """The 32,294 bug, as an assertion. diff --git a/scripts/assert_guards_are_tested.py b/scripts/assert_guards_are_tested.py index 67dc431..332098f 100755 --- a/scripts/assert_guards_are_tested.py +++ b/scripts/assert_guards_are_tested.py @@ -41,6 +41,13 @@ # (name, file, exact text to replace, replacement, tests that must then fail) MUTATIONS: tuple[tuple[str, str, str, str, tuple[str, ...]], ...] = ( + ( + "the model is offered whatever tools the CLI ships", + "hookprobe/hookprobe/engine.py", + " tools=list(_TOOLS),", + " tools=None,", + ("tests/test_engine_loop.py",), + ), ( "a file the operator configured may be written by a run", "hookprobe/hookprobe/inputs.py",