diff --git a/docs/ACS.md b/docs/ACS.md new file mode 100644 index 00000000..07da6551 --- /dev/null +++ b/docs/ACS.md @@ -0,0 +1,205 @@ +# CTRLRun and the Agent Control Standard + +What was read, what maps, what does not, and how the adapter is built. + +## What was read + +The [Agent Control Standard](https://agentcontrolstandard.org/), a project of the OWASP GenAI +Security Project, in +[`GenAI-Security-Project/agent-control-standard`](https://github.com/GenAI-Security-Project/agent-control-standard) +at commit **`c7ad162f69386daac94b89073e3b751e8cdf28b2`** (2026-08-11, *"Raise the Python floor +to unblock Dependabot security updates"*). The repository was at release **v0.1.1**, with the +specification version decoupled from the release version at commit `1af1f92` (2026-08-11) and +tracked separately at **v0.1.0**. Code is Apache-2.0; documentation is CC BY-SA 4.0. + +Specifically: + +| File | What it gave | +|---|---| +| `specification/ACS/acs_schema.json` | the v0.1.0 aggregator; 22 hooks; `oneOf` a RequestEnvelope or a ResponseEnvelope | +| `specification/v0.1.0/request-envelope.json` | JSON-RPC 2.0; method namespaces; `AcsParams` and its `metadata` identity fields | +| `specification/v0.1.0/response-envelope.json` | `AcsResult`, and the five decisions | +| `specification/v0.1.0/hooks/tool-call-request.json` | `steps/toolCallRequest` — `tool`, `arguments`, `operation`, `capability`, `intent` | +| `specification/v0.1.0/hooks/tool-call-result.json` | `steps/toolCallResult` — `exit_status`, `outputs`, `request_id_ref`, `duration_ms` | +| `specification/v0.1.0/ask-details.json` | what an `ask` decision must carry | +| `CONTRIBUTING.md` | DCO required (`git commit -s`); spec changes open a Discussion first; no CLA | + +Two things the repository does **not** have at that commit, both of which shaped this work: + +- **No reference implementation.** The README's roadmap places a Guardian Agent sample and + FastMCP instrumentation at v1. There is a `pyproject.toml`, but it serves the docs and + version-sync tooling. So there is nothing to conform *to* except the schemas, and this + adapter is written against them directly. +- **No `examples/` directory**, and so no house format for a community example. `examples/acs/` + therefore follows CTRLRun's own convention. + +## What ACS defines + +Three layers: **Instrument** (runtime hooks and the Guardian Agent pattern), **Trace** +(OpenTelemetry and OCSF with agent-specific conventions), and **Inspect** (CycloneDX, SPDX and +SWID for a dynamic Agent BOM). Only Instrument is relevant here. + +The wire format is JSON-RPC 2.0. A host fires a hook as a request whose `method` is +`steps/`; a Guardian answers with an `AcsResult` carrying one of five decisions: + +```text +allow · deny · modify · ask · defer +``` + +`deny` requires `reasoning`. `modify` requires `reasoning` and `modifications`. `ask` requires +`reasoning` and `ask_details`. `defer` requires `reasoning` and `defer_details`. + +Of the 22 hooks, CTRLRun answers **two**, and it is worth being explicit that it declines the +other twenty: `SessionStart`, `SessionEnd`, `AgentTrigger`, `TurnStart`, `TurnEnd`, +`UserMessage`, `AgentResponse`, `KnowledgeRetrieval`, `MemoryContextRetrieval`, `MemoryStore`, +`PreCompact`, `PostCompact`, `SubagentStart`, `SubagentStop`, `SkillRegister`, `SkillLoad`, +`SkillUnload`, `SystemPing`, `AgbomSnapshot`, `AgbomChanged`. Those are checkpoints about what +the model is thinking, remembering or composed of. CTRLRun's product rule is that it decides +actions that can affect the real world, and nothing else — so an unanswered method returns a +JSON-RPC error in ACS's reserved range rather than an opinion. + +## The mapping, hook by hook + +### `steps/toolCallRequest` → build the Action, decide it, take the reservation + +| ACS field | CTRLRun | +|---|---| +| `params.metadata.agent_id` | `Principal.agent` | +| `params.metadata.user_context.user_id` | `Principal.user` | +| `params.metadata.environment` | `Action.environment` (ACS's enum passes through; v0.1 §2.1 takes any non-empty string) | +| `payload.tool.name`, `payload.tool.provider`, `payload.operation` | `Action.name`, as `..[.]` | +| `payload.arguments` | `Action.arguments`, unwrapped from `{name: {value, provenance}}` to `{name: value}` | +| — *(ACS has no resource field)* | `Action.resource`, from the policy's `resource:` template (SPEC-v0.2 §3) | +| `params.request_id` | the continuation that joins this hook to its result | + +`operation` is part of the name because two verbs on one tool are two actions, and a policy has +to be able to say different things about `create` and `void`. + +The decision maps out: + +| CTRLRun | ACS | +|---|---| +| `ALLOW` | `allow` | +| `DENY` | `deny` + `reasoning` + `reason_codes` | +| `APPROVE` (`ApprovalRequired`) | `ask` + `reasoning` + `ask_details` | +| `DuplicateEffect` | `deny`, `reason_codes: ["ctrlrun.duplicate_effect", ]` | +| `AmbiguousEffect` | `deny`, `reason_codes: ["ctrlrun.ambiguous_effect"]` | +| `ApprovalMismatch` | `deny`, `reason_codes: ["ctrlrun.blocked", ]` | + +`ask_details` carries `approver` (`{type: "human", id}`), a `question` naming the request id a +human answers with `ctrlrun approve`, and `timeout_seconds`. All three are required by +`ask-details.json`. + +`ask_details.intent_extension` is **not** used. It grants capabilities for `this_request` or +`session`, which is an authority model — CTRLRun has none until v0.3, and a grant it cannot +represent is one it must not claim to honour. + +### `steps/toolCallResult` → close the reservation + +ACS describes this hook as *"fires after tool execution, before results reach the agent, +serving as an output redaction checkpoint"*. CTRLRun redacts nothing, so it always answers +`allow`; the work is the outcome it records. + +`exit_status` is `success | failure | timeout | blocked`. **ACS does not say what any of them +means for the side effect.** This is the fail-closed reading, and it is the same one +SPEC-v0.2 §6.8 applies to MCP: + +| `exit_status` | Effect state | Why | +|---|---|---| +| `success` | `COMMITTED` | the only status that asserts the effect happened | +| `blocked` | `FAILED` | a control refused it before dispatch — this is what `NotExecuted` means | +| `failure` | `AMBIGUOUS` | a tool that failed *after* acting and one that failed *before* send the same string | +| `timeout` | `AMBIGUOUS` | the same, and the case this library exists for | +| `failure` **with** `mcp.not_executed_on_error: true` | `FAILED` | an operator's per-tool assertion (SPEC-v0.2 §3.1) | + +A result whose `request_id_ref` matches no held reservation — a restarted Guardian, a result +fired twice, one arriving out of order — is logged and nothing is written about an effect. +Guessing which call it meant is how a duplicate gets committed. + +## Where ACS is silent + +These are not criticisms of ACS; they are the seam. ACS governs *whether an action may +proceed*. It has almost nothing to say about *what happened afterwards*, because that is not +what the Instrument layer was built for. + +**1. No effect identity.** `params.request_id` is a per-hook-invocation UUID and +`request_id_ref` links a result to its request. Neither identifies the *effect*: two calls that +would refund the same payment get two unrelated UUIDs. There is no idempotency key, no +`capability`-plus-argument identity, nothing an implementer could use to recognise that a retry +is a repeat. CTRLRun supplies one from the policy's `effect:` template. + +**2. `exit_status` is a status of the call, not an outcome of the effect.** Four values, and +the vocabulary itself carries the confusion: `timeout` sits alongside `failure` as though both +were kinds of not-working. The distinction that matters — *did the side effect land?* — has no +representation. A conformant Guardian reading `timeout` has no way to say "unknown, and a +retry is unsafe until a human resolves it". + +**3. Nothing binds an approval to the exact action.** `ask_details` carries a question and an +approver. What comes back is a decision on *the request*, and `intent_extension` can widen a +capability for the session. Neither pins the approval to a canonical form of the arguments, so +nothing in ACS prevents an agent from getting `refund(2000)` approved and then calling +`refund(5000)`. CTRLRun binds to `action_hash`, which covers the principal, the arguments, the +resource and the environment. + +**4. No terminal unknown state.** ACS's decisions are about the future of a call. There is no +way for a Guardian to record that an effect's outcome is unresolved and that *no* further call +on that effect may proceed until a human says which way it went. `AMBIGUOUS` has no ACS +counterpart, and it is the state most of CTRLRun's design exists to protect. + +**5. The Guardian does not execute.** ACS is advisory by construction — the platform runs the +tool. That is a reasonable separation, but it means a Guardian cannot make reserve-and-execute +atomic. The best available is what this adapter does: reserve at the request hook, close at the +result hook, and accept that a platform which never fires the result hook leaves a reservation +to lease-expire into `AMBIGUOUS` by the ordinary path of v0.1 §5.3 E3. + +## The adapter's design + +`ctrlrun.acs.AcsControlHook`, in `ctrlrun[gateway]` — it needs no new dependency, and it is +kept out of core for the same reason the gateway is: `import ctrlrun` must not grow. + +The shape is forced by the seam above. ACS is advisory and CTRLRun is executing, so one action +is split across two hooks and the reservation is held between them. That is exactly the shape +`Suspended` and `Control.resume` were built for in SPEC-v0.2 §6.9 — a reservation held across a +round trip the kernel does not control — so the adapter reuses them rather than reaching for +the store itself: + +```text +steps/toolCallRequest + → Action ← envelope metadata + payload + policy templates + → Control.execute(action, executor=raise Suspended(request_id), effect_key) + · policy decided, approval consumed, effect reserved, EXECUTION_STARTED + · executor suspends: no outcome, no receipt, lease extended, continuation held + → allow | deny | ask + +steps/toolCallResult + → Control.resume(request_id_ref, executor=report(exit_status)) + · the same outcome mapping execute uses (v0.1 §5.5) + · commit / fail / ambiguous, one receipt, same action_id and attempt + → allow +``` + +`Control` remains the only module that composes the others (ARCHITECTURE §6). The adapter +translates two vocabularies and decides nothing. + +**What it does not do.** It does not use `modify` — CTRLRun refuses or permits an action as +proposed, and rewriting an agent's arguments is a different product. It does not use `defer`. +It does not answer the other twenty hooks. It makes no claim of conformance: the schemas are +`v0.1.0`, the repository is a public preview, there is no reference implementation to test +against, and there is no conformance suite. **The words "ACS-compatible" do not appear in this +repository's README, docstrings or CLI output**, and should not until something exists to be +compatible *with*. + +## What the OTel attributes would have to become + +SPEC-v0.2 §8's sink emits `ctrlrun.*` attributes. ACS's Trace layer extends OpenTelemetry with +its own agent conventions and maps security events to OCSF; `acs_schema.json` carries +`TraceOtelMapping` and `TraceOcsfMapping` definitions for that purpose. + +Aligning would mean renaming `ctrlrun.action.name`, `ctrlrun.principal.agent`, +`ctrlrun.decision` and the rest onto ACS's conventions. The cost is not the rename: it is that +receipts already written carry the old names, and a receipt is evidence that has to outlive the +tool that wrote it. Any alignment is therefore additive — emit both, deprecate neither — or it +is a schema version bump on the receipt, which SPEC-v0.2 §11 explicitly does not do. + +That work is not in this release, and the mapping table is deliberately not written yet: it +would be a compliance claim with nothing behind it. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index a589212b..dddaa461 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -26,7 +26,7 @@ Standards: none. `THREAT_MODEL.md` is the only compliance-adjacent claim. Adoption story: *existing MCP server + one CTRLRun gateway = action safety.* -Standards: OWASP ACS adapter (code), OpenTelemetry export (code), MCP gateway. First use of "ACS-compatible", and only once the adapter exists. +Standards: OpenTelemetry export (code), MCP gateway. The OWASP ACS adapter shipped in v0.2 (see `docs/ACS.md`); "ACS-compatible" is still unearned and waits on an ACS conformance suite to measure against. ## v0.3 — Authority diff --git a/docs/SPEC-v0.2.md b/docs/SPEC-v0.2.md index 2779769a..06892b75 100644 --- a/docs/SPEC-v0.2.md +++ b/docs/SPEC-v0.2.md @@ -1130,9 +1130,23 @@ A retry is a new action and therefore a new span; the two are correlated by --- -## 9. ACS: design note only +## 9. ACS: the design note, and the adapter it earned -Item 7 writes `docs/ACS-NOTE.md`. **No code, no adapter, no claim.** +**Amended.** This section said `docs/ACS-NOTE.md`, no code, no adapter, no claim — on the +reading that ACS had no stable interface to build against. Reading the v0.1.0 schemas at +commit `c7ad162` (2026-08-11) settled that differently: the hook payloads, the envelope and +the five decisions are specified precisely enough to write against, and `steps/toolCallResult` +gives a post-execution checkpoint the earlier draft assumed did not exist. + +So: `docs/ACS.md` (renamed from `ACS-NOTE.md`), **and** an adapter in `ctrlrun[gateway]` — +`ctrlrun.acs.AcsControlHook` — with acceptance tests T51-T55. `ROADMAP.md`'s "OWASP ACS +adapter (code)" line moves from v0.3 into this release. + +**The no-claim rule is untouched and is the reason the adapter is safe to ship.** There is no +ACS reference implementation and no conformance suite at that commit, so there is nothing to +be conformant *with*. The words "ACS-compatible" MUST NOT appear in the README, in docstrings, +or in CLI output. `docs/ACS.md` states what was read, what maps, and — at length — where ACS +is silent. The Agent Control Standard is an open specification for runtime agent governance, published at v0.1.0 on 27 May 2026 as a project of the OWASP GenAI Security Project @@ -1140,7 +1154,7 @@ v0.1.0 on 27 May 2026 as a project of the OWASP GenAI Security Project lifecycle, expresses policy as YAML, and extends OpenTelemetry with agent-specific semantic conventions while mapping security events to OCSF. -The note states, and does no more than state: +`docs/ACS.md` states: - where CTRLRun would sit in that model — the tool-call checkpoint, and only that one; - what an adapter would carry across, and what has no counterpart on the ACS side: effect @@ -1150,8 +1164,8 @@ The note states, and does no more than state: conventions, and the cost of doing that to receipts already written. `ROADMAP.md`'s standards rule governs: integrate first, map second, never claim compliance. -The words "ACS-compatible" MUST NOT appear in the README, in docstrings, or in CLI output in -v0.2. They are earned by an adapter with tests, which is not in this release. +The adapter is the "integrate" step. "ACS-compatible" is the "claim" step, and it is not +earned by an adapter alone — it needs something on the ACS side to be measured against. --- @@ -1304,6 +1318,18 @@ An intercepted `tools/call` answered with an SSE stream carrying `id:` fields re client with those fields stripped, so it cannot request a resume of a stream whose outcome the gateway owns. A non-intercepted stream reaches the client with its `id:` fields intact. +### T51-T55 — The ACS control hook +Against a minimal fake ACS runtime, per §9 and `docs/ACS.md`. **T51**: a hooked +`steps/toolCallRequest` becomes an Action with the name, arguments and principal the envelope +names, arguments unwrapped from their provenance envelopes and the resource from the policy +template. **T52**: `ALLOW`/`DENY`/`APPROVE` map to `allow`/`deny`/`ask`, each carrying what +`response-envelope.json` requires of it. **T53**: an approval is re-presented by re-sending the +identical call, and a mutated call gets a fresh question rather than the granted one; a second +call on a committed or held effect is denied. **T54**: `exit_status` maps by v0.1 §5.5's +asymmetry — `success` committed, `blocked` failed, `failure` and `timeout` ambiguous unless +`not_executed_on_error` says otherwise. **T55**: an unanswered method is a JSON-RPC error in +ACS's reserved range, and `import ctrlrun` does not import the adapter. + ### T31 — Every example runs and every template loads Each script under `examples/` exits 0 with no network, against its own state directory, and prints the refusal it exists to demonstrate. Every `examples/policies/.yaml` loads @@ -1401,6 +1427,6 @@ and OTLP/HTTP exporter). Neither is in `dependencies`. Everything in v0.1 §9 that v0.2 does not deliver, and specifically: the deprecated `2024-11-05` HTTP+SSE transport; relaying SSE resumption for an intercepted call (§6.2); binding an approval across an elicitation round trip (§6.9.5); more than one upstream per -gateway; multi-host reservation; an ACS adapter or any ACS claim; +gateway; multi-host reservation; any ACS compliance claim; authenticating the principal or the approver; signed receipts; MCP `tasks`, `apps` or any other extension; `ctrlrun verify`; framework adapters; Postgres; anything in `VISION.md`. diff --git a/examples/acs/README.md b/examples/acs/README.md new file mode 100644 index 00000000..5de09890 --- /dev/null +++ b/examples/acs/README.md @@ -0,0 +1,24 @@ +# The double-refund scenario, through ACS hooks + +ACS has no `examples/` directory of its own as of commit `c7ad162` (2026-08-11), so this +follows CTRLRun's own convention: one runnable script, its policy beside it, no network, and +its own state directory. + +``` +python examples/acs/main.py +``` + +## What it shows + +A tool call fires two ACS hooks. `steps/toolCallRequest` is where a Guardian decides; +`steps/toolCallResult` is where the platform reports what happened. CTRLRun answers both: +it takes the reservation at the first and closes it at the second, which is what makes the +retry refusable — the first call's effect record is still open when the second arrives. + +The remote commits the refund and the response goes missing. ACS reports `exit_status: +"timeout"`, and **ACS does not say what a timeout means for the side effect**. CTRLRun +records `AMBIGUOUS` and refuses the retry, because a tool that timed out after acting and one +that timed out before acting send the same string. + +See [`docs/ACS.md`](../../docs/ACS.md) for the full mapping and for where the two models +disagree. diff --git a/examples/acs/ctrlrun.yaml b/examples/acs/ctrlrun.yaml new file mode 100644 index 00000000..c63e0900 --- /dev/null +++ b/examples/acs/ctrlrun.yaml @@ -0,0 +1,19 @@ +# The policy this example runs under. Unknown actions are denied; there is no default-allow. +# +# The rule these decisions follow, so you can apply it to actions not listed here: +# cheap to undo → allow (autonomous) +# leaves the building → approve (a human, every time) +# destroys the evidence → deny (not an agent action at any size) +# +# Action names are built by the ACS hook as .., so a policy written +# against an ACS deployment addresses one stable string per tool. +schema: ctrlrun.policy/v2 + +actions: + acs.stripe.create_refund: + effect: "refund:{payment_id}" + resource: "payment:{payment_id}" + rules: + - when: { amount_gte: 0, amount_lte: 50000 } # $0 to $500 — autonomous + decision: allow + - decision: deny diff --git a/examples/acs/main.py b/examples/acs/main.py new file mode 100644 index 00000000..9b1b4a52 --- /dev/null +++ b/examples/acs/main.py @@ -0,0 +1,146 @@ +#!/usr/bin/env python3 +"""The double-refund scenario, through ACS hooks, with CTRLRun behind them. + +ACS is advisory: a Guardian returns a decision and the *platform* runs the tool. So one tool +call fires two hooks — `steps/toolCallRequest` before, `steps/toolCallResult` after — and +CTRLRun holds the reservation between them. That is what makes the second attempt refusable: +the first call's effect record is still open when the second arrives. + +The remote commits the refund and then the response goes missing. ACS reports that as +`exit_status: "timeout"`, and ACS says nothing about what a timeout means for the side +effect. CTRLRun records it as AMBIGUOUS — the fail-closed reading — and refuses the retry. + + python examples/acs/main.py + +No network. Its own state directory, under `.ctrlrun/examples/acs/`. +""" + +from __future__ import annotations + +import json +import uuid +from pathlib import Path +from typing import Any + +from ctrlrun import Control, Policy, SQLiteStateStore +from ctrlrun.acs import ACS_VERSION, TOOL_CALL_REQUEST, TOOL_CALL_RESULT, AcsControlHook + +HERE = Path(__file__).resolve().parent +BLOCKED = " ✗ BLOCKED — " + + +class FakeAcsRuntime: + """The smallest thing that behaves like an ACS host: it fires hooks and obeys verdicts.""" + + def __init__(self, hook: AcsControlHook) -> None: + self._hook = hook + self.session_id = str(uuid.uuid4()) + self.tool_calls = 0 + + def call_tool(self, tool: str, arguments: dict[str, Any], *, lose_response: bool) -> str: + request_id = str(uuid.uuid4()) + verdict = self._hook.handle( + self._envelope( + TOOL_CALL_REQUEST, + request_id, + { + "tool": {"name": tool, "provider": "stripe"}, + "arguments": {name: {"value": value} for name, value in arguments.items()}, + }, + ) + ) + decision = verdict.get("result", {}).get("decision") + if decision != "allow": + return f"{decision}: {verdict['result'].get('reasoning', '')}" + + # The platform runs the tool. CTRLRun did not, and does not know what happened until + # the result hook tells it. + self.tool_calls += 1 + exit_status = "timeout" if lose_response else "success" + + self._hook.handle( + self._envelope( + TOOL_CALL_RESULT, + str(uuid.uuid4()), + { + "tool": {"name": tool, "provider": "stripe"}, + "exit_status": exit_status, + "outputs": [{"value": {"id": f"re_{arguments['payment_id']}"}}], + "request_id_ref": request_id, + }, + ) + ) + return f"allow → the platform ran it → exit_status={exit_status}" + + def _envelope(self, method: str, request_id: str, payload: dict[str, Any]) -> dict[str, Any]: + return { + "jsonrpc": "2.0", + "method": method, + "id": self.tool_calls + 1, + "params": { + "acs_version": ACS_VERSION, + "request_id": request_id, + "timestamp": "2026-09-04T10:00:00Z", + "metadata": { + "agent_id": "refund-agent", + "session_id": self.session_id, + "environment": "production", + "user_context": {"user_id": "alice", "roles": ["support"]}, + }, + "payload": payload, + }, + } + + +def state_dir(root: Path) -> Path: + """A store of this example's own, emptied so the example repeats.""" + evidence = root / ".ctrlrun" / "examples" / HERE.name + evidence.mkdir(parents=True, exist_ok=True) + for name in ("state.db", "state.db-wal", "state.db-shm", "receipts.jsonl", "events.jsonl"): + (evidence / name).unlink(missing_ok=True) + return evidence + + +def main(root: Path) -> None: + store = SQLiteStateStore(state_dir(root) / "state.db") + control = Control(Policy.from_file(HERE / "ctrlrun.yaml"), store) + runtime = FakeAcsRuntime(AcsControlHook(control, prefix="acs")) + + print("Duplicate effect after a lost response — through ACS hooks") + print(" ACS is advisory: the Guardian decides, the platform executes.") + try: + arguments = {"payment_id": "txn_1", "amount": 200} + + first = runtime.call_tool("create_refund", arguments, lose_response=True) + print(f" steps/toolCallRequest → {first}") + print(" ACS says nothing about what a timeout means for the side effect.") + + second = runtime.call_tool("create_refund", arguments, lose_response=False) + if not second.startswith("deny"): + raise SystemExit("the retry was permitted; this example proved nothing") + print(" agent retries the identical call →") + print(f"{BLOCKED}{second}") + + for record in store.list_effects(): + print(f" effect record: {record.effect_key} {record.state}") + print(f" tool calls the platform actually ran: {runtime.tool_calls}") + print(" only a human moves it on: ctrlrun resolve refund:txn_1 --committed|--failed") + print("") + print(" The decision the Guardian returned, in ACS's own shape:") + refused = runtime._hook.handle( + runtime._envelope( + TOOL_CALL_REQUEST, + str(uuid.uuid4()), + { + "tool": {"name": "create_refund", "provider": "stripe"}, + "arguments": {name: {"value": value} for name, value in arguments.items()}, + }, + ) + ) + print(" ", json.dumps(refused["result"], indent=2).replace("\n", "\n ")) + finally: + store.close() + + +if __name__ == "__main__": + main(Path.cwd()) diff --git a/pyproject.toml b/pyproject.toml index 61c84e12..1c3dca81 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -92,6 +92,9 @@ select = ["E", "F", "W", "I", "N", "UP", "B", "ANN", "SIM", "RUF"] # does. N818: the error names follow this codebase's convention (DuplicateEffect, # AmbiguousEffect), not the suffix rule. "src/ctrlrun/gateway/*.py" = ["ANN401", "N818"] +# Same reasoning for the ACS adapter: its subject matter is an untyped JSON-RPC envelope off +# somebody else's wire, and `_Unknown` follows this codebase's error naming. +"src/ctrlrun/acs.py" = ["ANN401", "N818"] # Type hints are a src/ constraint; tests stay cheap to write. N802 lets acceptance # tests carry the SPEC §7 identifier verbatim (test_T7_...). "tests/*" = ["ANN", "N802"] diff --git a/src/ctrlrun/acs.py b/src/ctrlrun/acs.py new file mode 100644 index 00000000..41b4bd27 --- /dev/null +++ b/src/ctrlrun/acs.py @@ -0,0 +1,417 @@ +"""An ACS control hook backed by CTRLRun. Ships in `ctrlrun[gateway]`. + +Read against the Agent Control Standard v0.1.0 schemas in +`GenAI-Security-Project/agent-control-standard` at commit `c7ad162` (2026-08-11): +`specification/v0.1.0/request-envelope.json`, `response-envelope.json`, +`hooks/tool-call-request.json`, `hooks/tool-call-result.json` and `ask-details.json`. +`docs/ACS.md` records what was read and where the two models disagree. + +**ACS is advisory; CTRLRun is executing.** A Guardian returns a decision and the *platform* +runs the tool, which is the opposite way round from `@protect`. So one action is split across +two hooks: + +- `steps/toolCallRequest` builds the Action, decides it, takes the reservation, and suspends + holding it — no outcome, no receipt; +- `steps/toolCallResult` closes that reservation with what actually happened. + +The two are joined by `Suspended` and `Control.resume` (SPEC-v0.2 §6.9), which exist for +exactly this shape: a reservation held across a round trip the kernel does not control. The +adapter adds no second implementation of v0.1 §5.5's asymmetry; it translates ACS's +`exit_status` vocabulary into the kernel's and lets `Control` decide. +""" + +from __future__ import annotations + +import logging +from collections.abc import Mapping +from typing import Any, Final + +from .action import Action, Principal +from .control import Control, with_approval +from .effect import resolve_effect_key, resolve_resource +from .errors import ( + ActionDenied, + AmbiguousEffect, + ApprovalMismatch, + ApprovalRequired, + CTRLRunError, + DuplicateEffect, + EffectKeyError, + InvalidArgument, + NotExecuted, + Suspended, +) + +_LOG = logging.getLogger(__name__) + +#: The ACS revision these mappings were read against. +ACS_VERSION: Final = "0.1.0" + +#: The two `steps/*` hooks CTRLRun answers. Every other method is somebody else's checkpoint. +TOOL_CALL_REQUEST: Final = "steps/toolCallRequest" +TOOL_CALL_RESULT: Final = "steps/toolCallResult" + +#: `response-envelope.json` — the five decisions ACS defines. +ALLOW: Final = "allow" +DENY: Final = "deny" +ASK: Final = "ask" + +#: `tool-call-result.json` — the four statuses ACS defines, and nothing about what any of +#: them means for the side effect. See `docs/ACS.md`. +SUCCESS: Final = "success" +FAILURE: Final = "failure" +TIMEOUT: Final = "timeout" +BLOCKED: Final = "blocked" + +#: ACS reserves -32000 to -32099 for itself (`response-envelope.json`). +METHOD_NOT_ANSWERED: Final = -32001 +MALFORMED_ENVELOPE: Final = -32002 + +#: How long an `ask` says a human has. `ask-details.json` requires a positive integer. +DEFAULT_ASK_TIMEOUT_SECONDS: Final = 900 + + +class AcsControlHook: + """Answer ACS `steps/*` hooks with CTRLRun's decisions and outcomes. + + One `Control`, one prefix. `prefix` names the tool namespace in the action name, the way + the gateway's `--alias` does: `..` — so a policy addresses one + stable string and two providers exposing the same tool name stay distinguishable. + """ + + def __init__( + self, + control: Control, + *, + prefix: str = "acs", + approver_id: str = "cli:local", + ask_timeout_seconds: int = DEFAULT_ASK_TIMEOUT_SECONDS, + ) -> None: + self._control = control + self._prefix = prefix + self._approver_id = approver_id + self._ask_timeout = max(1, ask_timeout_seconds) + + # --- the ACS surface ---------------------------------------------------------------- + + def handle(self, envelope: Mapping[str, Any]) -> dict[str, Any]: + """One ACS request envelope in, one response envelope out.""" + if ( + not isinstance(envelope, Mapping) + or envelope.get("jsonrpc") != "2.0" + or not isinstance(envelope.get("method"), str) + or not isinstance(envelope.get("params"), Mapping) + ): + return _error(None, MALFORMED_ENVELOPE, "not an ACS request envelope") + + method = str(envelope["method"]) + params = envelope["params"] + rpc_id = envelope.get("id") + request_id = params.get("request_id") + if not isinstance(request_id, str) or not request_id: + return _error(rpc_id, MALFORMED_ENVELOPE, "params.request_id is required") + + try: + if method == TOOL_CALL_REQUEST: + return self._on_request(rpc_id, request_id, params) + if method == TOOL_CALL_RESULT: + return self._on_result(rpc_id, request_id, params) + except InvalidArgument as refused: + # A malformed payload is not a decision about an action: there is no action. + return _error(rpc_id, MALFORMED_ENVELOPE, str(refused)) + return _error( + rpc_id, + METHOD_NOT_ANSWERED, + f"{method} is not a checkpoint CTRLRun answers; it decides tool calls only", + ) + + # --- steps/toolCallRequest ---------------------------------------------------------- + + def _on_request( + self, rpc_id: Any, request_id: str, params: Mapping[str, Any] + ) -> dict[str, Any]: + """Decide the call, take the reservation, and hold it for the result hook.""" + payload = _mapping(params.get("payload"), "params.payload") + action = self._action(params, payload) + effect_key = self._effect_key(action) + + # SPEC-v0.2 §6.10's rule, in ACS's shape: a client comes back by re-sending the + # identical call, and the newest granted, unexpired approval for this action's hash + # is what authorizes it. Matching on the hash is safe for the reason v0.1 §4.2 A1 + # exists — the hash covers the principal, the arguments, the resource and the + # environment — and it is still single-use, consumed atomically with the reservation. + granted = ( + self._control.store.find_granted_approval(action.action_hash) + if self._control.policy.evaluate(action).decision.value == "approve" + else None + ) + + def suspend_holding_the_reservation() -> Any: + # The platform executes, not CTRLRun. `Suspended` is how an executor says the + # outcome is not known yet *and no outcome should be recorded* (§6.9): the + # record stays EXECUTING, the lease is extended, and this request_id is what the + # result hook presents to close it. + raise Suspended(request_id) + + try: + if granted is not None: + with with_approval(granted.approval_id): + self._control.execute(action, suspend_holding_the_reservation, effect_key) + else: + self._control.execute(action, suspend_holding_the_reservation, effect_key) + except Suspended: + return _final(rpc_id, request_id, ALLOW) + except ActionDenied as refused: + return _final(rpc_id, request_id, DENY, reasoning=str(refused), codes=[refused.reason]) + except ApprovalRequired as pending: + return self._ask(rpc_id, request_id, action, pending) + except DuplicateEffect as refused: + return _final( + rpc_id, + request_id, + DENY, + reasoning=str(refused), + codes=["ctrlrun.duplicate_effect", refused.state], + ) + except AmbiguousEffect as refused: + return _final( + rpc_id, + request_id, + DENY, + reasoning=str(refused), + codes=["ctrlrun.ambiguous_effect"], + ) + except ApprovalMismatch as refused: + return _final( + rpc_id, + request_id, + DENY, + reasoning=str(refused), + codes=["ctrlrun.blocked", refused.reason], + ) + # An action with no effect key has nothing to hold: `execute` ran the executor, + # which suspended, but with no reservation there is nothing for a result hook to + # close. It is allowed, and it is recorded as unheld (§6.9). + return _final(rpc_id, request_id, ALLOW) + + def _ask( + self, rpc_id: Any, request_id: str, action: Action, pending: ApprovalRequired + ) -> dict[str, Any]: + """CTRLRun's APPROVE in ACS's shape (`ask-details.json`). + + `approver`, `question` and `timeout_seconds` are all required by the schema, so all + three are present or the response is not conformant. The question carries the + request id, because that is what `ctrlrun approve` answers. + """ + return _final( + rpc_id, + request_id, + ASK, + reasoning=f"{action.name} requires a human decision under this policy", + codes=["ctrlrun.approval_required"], + ask_details={ + "approver": {"type": "human", "id": self._approver_id}, + "question": ( + f"Approve {action.name} for {action.principal.agent}? " + f"Answer with 'ctrlrun approve {pending.request_id}' " + f"or 'ctrlrun deny {pending.request_id}'." + ), + "timeout_seconds": self._ask_timeout, + "context": _describe(action), + }, + ) + + # --- steps/toolCallResult ----------------------------------------------------------- + + def _on_result(self, rpc_id: Any, request_id: str, params: Mapping[str, Any]) -> dict[str, Any]: + """Close the reservation the request hook took, with what actually happened. + + ACS describes this hook as an output redaction checkpoint. CTRLRun redacts nothing — + it records — so the decision is always `allow`; the work is the outcome it writes. + """ + payload = _mapping(params.get("payload"), "params.payload") + exit_status = payload.get("exit_status") + if not isinstance(exit_status, str): + raise InvalidArgument("payload.exit_status is required") + held = payload.get("request_id_ref") + if not isinstance(held, str) or not held: + # Nothing links this result to a call, so there is no reservation to close and + # nothing may be written about an effect. Guessing which one it meant is how a + # duplicate gets committed. + _LOG.warning("an ACS toolCallResult arrived with no request_id_ref; ignoring") + return _final(rpc_id, request_id, ALLOW) + + tool = _mapping(payload.get("tool"), "payload.tool") + name = self._action_name(tool, payload.get("operation")) + not_executed_on_error = self._control.policy.mcp_options(name).not_executed_on_error + + def report_what_happened() -> Any: + outcome = _outcome(exit_status, not_executed_on_error) + if outcome is True: + return payload.get("outputs") + if outcome is False: + raise NotExecuted(f"ACS exit_status={exit_status!r} before dispatch") + raise _Unknown(f"ACS exit_status={exit_status!r} says nothing about the effect") + + try: + self._control.resume(held, report_what_happened) + except (NotExecuted, _Unknown): + pass + except InvalidArgument: + # No held suspension matches. A restarted Guardian, a result fired twice, or one + # arriving out of order. There is nothing to close, and inventing an outcome for + # an effect nobody reserved would be worse than recording none. + _LOG.warning("no held CTRLRun reservation matches ACS request_id_ref %s", held) + except CTRLRunError as refused: + _LOG.warning("could not close the reservation for %s: %s", held, refused) + return _final(rpc_id, request_id, ALLOW) + + # --- building the Action (tool-call-request.json) ----------------------------------- + + def _action(self, params: Mapping[str, Any], payload: Mapping[str, Any]) -> Action: + tool = _mapping(payload.get("tool"), "payload.tool") + metadata = _mapping(params.get("metadata"), "params.metadata") + agent = metadata.get("agent_id") + if not isinstance(agent, str) or not agent: + raise InvalidArgument("params.metadata.agent_id is required") + user_context = metadata.get("user_context") + user = user_context.get("user_id") if isinstance(user_context, Mapping) else None + + name = self._action_name(tool, payload.get("operation")) + arguments = _arguments(payload.get("arguments")) + resource_template = self._control.policy.resource_template(name) + return Action( + name=name, + arguments=arguments, + principal=Principal(agent=agent, user=user if isinstance(user, str) else None), + resource=( + None + if resource_template is None + else resolve_resource(resource_template, arguments) + ), + # `environment` is an ACS enum (development|staging|production); v0.1 §2.1 takes + # any non-empty string, so it passes through rather than being mapped. + environment=str(metadata.get("environment") or "production"), + ) + + def _action_name(self, tool: Mapping[str, Any], operation: object) -> str: + name = tool.get("name") + if not isinstance(name, str) or not name: + raise InvalidArgument("payload.tool.name is required") + provider = tool.get("provider") + parts = [self._prefix] + if isinstance(provider, str) and provider: + parts.append(provider) + parts.append(name) + if isinstance(operation, str) and operation: + # Two verbs on one tool are two actions: a policy must be able to say different + # things about `create` and `void`. + parts.append(operation) + return ".".join(parts) + + def _effect_key(self, action: Action) -> str | None: + template = self._control.policy.effect_template(action.name) + if template is None: + return None + try: + return resolve_effect_key(template, action) + except EffectKeyError as refused: + raise InvalidArgument(str(refused)) from refused + + +class _Unknown(CTRLRunError): + """Anything that is not `NotExecuted` is an AMBIGUOUS outcome to `Control` (v0.1 §5.5).""" + + +def _outcome(exit_status: str, not_executed_on_error: bool) -> bool | None: + """ACS's `exit_status` under v0.1 §5.5's asymmetry. `True` committed, `False` provably + not executed, `None` unknown. + + ACS names four statuses and says nothing about what any of them means for a side effect. + This is the fail-closed reading, and it is the same one SPEC-v0.2 §6.8 applies to MCP: + + - `success` is the only status that asserts the effect happened; + - `blocked` is the only one that asserts it did not — a control refused it before + dispatch, which is what `NotExecuted` means; + - `failure` and `timeout` are both unknown, because a tool that failed *after* acting and + one that failed *before* acting report the same string. `not_executed_on_error` (§3.1) + is where an operator asserts otherwise for a tool they know. + """ + if exit_status == SUCCESS: + return True + if exit_status == BLOCKED: + return False + if exit_status == FAILURE and not_executed_on_error: + return False + return None + + +def _arguments(raw: object) -> dict[str, Any]: + """`{name: {value, provenance}}` → `{name: value}` (`tool-call-request.json`). + + An Action built from the envelopes rather than the values would hash something no policy + can address and no human can read. + """ + if raw is None: + return {} + if not isinstance(raw, Mapping): + raise InvalidArgument("payload.arguments must be an object") + arguments: dict[str, Any] = {} + for name, envelope in raw.items(): + if not isinstance(name, str): + raise InvalidArgument("argument names must be strings") + if isinstance(envelope, Mapping) and "value" in envelope: + arguments[name] = envelope["value"] + else: + raise InvalidArgument(f"argument {name!r} must be an object with a 'value'") + return arguments + + +def _describe(action: Action) -> str: + parts = ", ".join(f"{key}={value!r}" for key, value in action.canonical_arguments.items()) + return f"{action.name}({parts}) in {action.environment}" + + +def _mapping(value: object, where: str) -> Mapping[str, Any]: + if not isinstance(value, Mapping): + raise InvalidArgument(f"{where} must be an object") + return value + + +def _final( + rpc_id: Any, + request_id: str, + decision: str, + *, + reasoning: str | None = None, + codes: list[str] | None = None, + ask_details: Mapping[str, Any] | None = None, +) -> dict[str, Any]: + """One `AcsResult` in `response-envelope.json`'s shape.""" + result: dict[str, Any] = { + "type": "final", + "acs_version": ACS_VERSION, + "request_id": request_id, + "decision": decision, + } + if reasoning is not None: + result["reasoning"] = reasoning + if codes: + result["reason_codes"] = [code for code in codes if code] + if ask_details is not None: + result["ask_details"] = dict(ask_details) + result["metadata"] = {"evaluator": "ctrlrun", "version": _version()} + return {"jsonrpc": "2.0", "id": rpc_id, "result": result} + + +def _error(rpc_id: Any, code: int, message: str) -> dict[str, Any]: + return {"jsonrpc": "2.0", "id": rpc_id, "error": {"code": code, "message": message}} + + +def _version() -> str: + from importlib.metadata import PackageNotFoundError, version + + try: + return version("ctrlrun") + except PackageNotFoundError: # pragma: no cover - source checkout without an install + return "unknown" diff --git a/tests/test_acs.py b/tests/test_acs.py new file mode 100644 index 00000000..55d0f0d9 --- /dev/null +++ b/tests/test_acs.py @@ -0,0 +1,491 @@ +"""The ACS control hook. SPEC-v0.2 §9 (amended), acceptance tests T51-T55. + +ACS is an *advisory* interface: a Guardian returns a decision and the platform executes. That +is the opposite way round from `@protect`, where CTRLRun runs the executor — so the adapter +splits one action across two hooks. `steps/toolCallRequest` decides and takes the +reservation; `steps/toolCallResult` closes it with what actually happened. + +The two are joined by `Suspended`/`Control.resume` (§6.9), which already exists for exactly +this shape: a reservation held across a round trip the kernel does not control. + +Read against the ACS v0.1.0 schemas, commit c7ad162 (2026-08-11): + specification/v0.1.0/request-envelope.json + specification/v0.1.0/response-envelope.json + specification/v0.1.0/hooks/tool-call-request.json + specification/v0.1.0/hooks/tool-call-result.json + specification/v0.1.0/ask-details.json +""" + +from __future__ import annotations + +import json +import uuid + +import pytest + +from ctrlrun import Control, EffectState, Policy, SQLiteStateStore +from ctrlrun.acs import ACS_VERSION, TOOL_CALL_REQUEST, TOOL_CALL_RESULT, AcsControlHook +from ctrlrun.receipt import ReceiptResult + +POLICY = """ +schema: ctrlrun.policy/v2 +actions: + acs.stripe.create_refund: + effect: "refund:{payment_id}" + resource: "payment:{payment_id}" + rules: + - when: { amount_gte: 0, amount_lte: 50000 } + decision: allow + - when: { amount_gte: 0, amount_lte: 1000000 } + decision: approve + - decision: deny + acs.stripe.read_balance: + decision: allow + acs.stripe.reports_before_acting: + effect: "report:{payment_id}" + mcp: + not_executed_on_error: true + decision: allow +""" + + +@pytest.fixture +def store(tmp_path): + store = SQLiteStateStore(tmp_path / "state.db") + yield store + store.close() + + +@pytest.fixture +def control(store): + return Control(Policy.from_yaml(POLICY), store) + + +@pytest.fixture +def hook(control): + return AcsControlHook(control, prefix="acs") + + +def _envelope(method, payload, *, request_id=None, agent="refund-agent", user="alice"): + """One ACS request envelope, in the shape request-envelope.json defines.""" + return { + "jsonrpc": "2.0", + "method": method, + "id": 1, + "params": { + "acs_version": ACS_VERSION, + "request_id": request_id or str(uuid.uuid4()), + "timestamp": "2026-09-04T10:00:00Z", + "metadata": { + "agent_id": agent, + "session_id": str(uuid.uuid4()), + "environment": "production", + "user_context": {"user_id": user, "roles": ["support"]}, + }, + "payload": payload, + }, + } + + +def _call(tool="create_refund", *, payment_id="txn_1", amount=200, request_id=None): + return _envelope( + TOOL_CALL_REQUEST, + { + "tool": {"name": tool, "provider": "stripe"}, + "arguments": { + "payment_id": {"value": payment_id}, + "amount": {"value": amount}, + }, + }, + request_id=request_id, + ) + + +def _result(request_id, exit_status="success", tool="create_refund"): + return _envelope( + TOOL_CALL_RESULT, + { + "tool": {"name": tool, "provider": "stripe"}, + "exit_status": exit_status, + "outputs": [{"value": {"id": "re_1"}}], + "request_id_ref": request_id, + }, + ) + + +CALL_ID = "11111111-1111-4111-8111-111111111111" + + +def _both(hook, exit_status="success", **call): + """The two hooks one tool call fires, joined by request_id_ref as ACS joins them.""" + request = hook.handle(_call(request_id=CALL_ID, **call)) + result = hook.handle(_result(CALL_ID, exit_status, tool=call.get("tool", "create_refund"))) + return request, result + + +def _result_of(hook, response): + assert "error" not in response, response + return response["result"] + + +# --- T51: a hooked call becomes an Action ---------------------------------------------- + + +def test_T51_the_hooked_call_becomes_an_action_with_the_right_name(hook, store): + hook.handle(_call()) + + receipt_or_events = store.events() + assert receipt_or_events[0].action_id + assert any(e.data.get("action_hash") for e in receipt_or_events) + + +def test_T51_the_tool_name_is_prefixed_and_the_provider_is_not_in_it(hook, store, control): + """ACS carries `tool.provider` separately from `tool.name`; the action name is built from + the configured prefix and the tool name, so a policy addresses one stable string.""" + hook.handle(_call(request_id=CALL_ID)) + + proposed = store.events()[0] + assert proposed.action_id + hook.handle(_result(CALL_ID)) + assert store.receipts()[-1].action == "acs.stripe.create_refund" + + +def test_T51_the_arguments_are_unwrapped_from_their_provenance_envelopes(hook, store): + """ACS's `arguments` is `{name: {value, provenance}}`, not raw values. An Action built + from the envelopes rather than the values would hash something no policy can address.""" + hook.handle(_call(request_id=CALL_ID, payment_id="txn_9", amount=300)) + hook.handle(_result(CALL_ID)) + + receipt = store.receipts()[-1] + assert receipt.arguments == {"amount": 300, "payment_id": "txn_9"} + + +def test_T51_the_principal_comes_from_the_envelope_metadata(hook, store): + hook.handle(_call(request_id=CALL_ID)) + hook.handle(_result(CALL_ID)) + + receipt = store.receipts()[-1] + assert receipt.principal.agent == "refund-agent" + assert receipt.principal.user == "alice" + assert receipt.environment == "production" + + +def test_T51_the_resource_comes_from_the_policy_template(hook, store): + """ACS has no resource field; §3's template is where one comes from.""" + hook.handle(_call(request_id=CALL_ID)) + hook.handle(_result(CALL_ID)) + + assert store.receipts()[-1].resource == "payment:txn_1" + + +def test_T51_an_operation_is_part_of_the_action_name(hook, store): + """`operation` is ACS's sub-verb for a tool that exposes several; two verbs on one tool + are two actions, and a policy must be able to say different things about them.""" + envelope = _call() + envelope["params"]["payload"]["operation"] = "void" + hook.handle(envelope) + + assert store.events()[0].action_id + + +# --- T52: deny and approve take ACS's decision shape ----------------------------------- + + +def test_T52_an_allowed_call_returns_decision_allow(hook): + response = hook.handle(_call()) + + result = _result_of(hook, response) + assert result["decision"] == "allow" + assert result["type"] == "final" + assert result["acs_version"] == ACS_VERSION + + +def test_T52_the_response_echoes_the_request_id_and_the_jsonrpc_id(hook): + envelope = _call() + response = hook.handle(envelope) + + assert response["jsonrpc"] == "2.0" + assert response["id"] == envelope["id"] + assert response["result"]["request_id"] == envelope["params"]["request_id"] + + +def test_T52_a_denied_call_returns_deny_with_reasoning(hook, store): + """response-envelope.json makes `reasoning` required when the decision is `deny`.""" + response = hook.handle(_call(amount=99999999)) + + result = _result_of(hook, response) + assert result["decision"] == "deny" + assert result["reasoning"] + assert store.list_effects() == () + + +def test_T52_an_action_needing_a_human_returns_ask_with_ask_details(hook, store): + """ACS's `ask` is CTRLRun's APPROVE. ask-details.json requires approver, question and + timeout_seconds, so all three are present or the response is not conformant.""" + response = hook.handle(_call(amount=200000)) + + result = _result_of(hook, response) + assert result["decision"] == "ask" + assert result["reasoning"] + details = result["ask_details"] + assert details["approver"]["type"] == "human" + assert details["approver"]["id"] + assert details["question"] + assert isinstance(details["timeout_seconds"], int) + assert details["timeout_seconds"] >= 1 + + +def test_T52_the_ask_carries_the_request_id_a_human_answers(hook, store): + response = hook.handle(_call(amount=200000)) + + action_hash = store.events()[0].data["action_hash"] + approvals = store.approvals_for(action_hash) + assert approvals + assert approvals[0].approval_id in response["result"]["ask_details"]["question"] + + +def test_T52_an_unknown_tool_is_denied_because_policy_denies_what_it_does_not_list(hook): + response = hook.handle(_call(tool="delete_everything")) + + assert _result_of(hook, response)["decision"] == "deny" + + +def test_T52_a_deny_carries_reason_codes(hook): + result = _result_of(hook, hook.handle(_call(amount=99999999))) + + assert result["reason_codes"] + + +# --- T53: a mutated call after an approval is refused ---------------------------------- + + +def test_T53_the_identical_call_executes_once_the_approval_is_granted(hook, store): + first = _result_of(hook, hook.handle(_call(amount=200000))) + request_id = _pending_approval(store) + store.grant_approval(request_id, "cli:local") + + second = _result_of(hook, hook.handle(_call(amount=200000))) + + assert first["decision"] == "ask" + assert second["decision"] == "allow" + + +def _pending_approval(store): + for record in store.events(): + if record.approval_id: + return record.approval_id + raise AssertionError("no approval requested") + + +def test_T53_a_mutated_call_presenting_the_same_approval_is_refused(hook, store): + """v0.1 §4.2 A1 over ACS: the approval binds to the action's hash, which covers the + principal, the arguments, the resource and the environment. A different amount is a + different action, and the human never saw it.""" + hook.handle(_call(amount=200000)) + store.grant_approval(_pending_approval(store), "cli:local") + + mutated = _result_of(hook, hook.handle(_call(amount=500000))) + + assert mutated["decision"] == "ask" # a fresh question, not the granted one + assert store.list_effects() == () + + +def test_T53_a_second_call_on_a_committed_effect_is_denied(hook, store): + _both(hook) + + again = _result_of(hook, hook.handle(_call())) + + assert again["decision"] == "deny" + assert "duplicate" in " ".join(again["reason_codes"]).lower() + + +def test_T53_a_concurrent_call_while_one_is_outstanding_is_denied(hook, store): + """The reservation is held between the two hooks, which is the whole point of splitting + them: an ACS platform that fires two toolCallRequests before either result must not get + two allows.""" + hook.handle(_call()) + + second = _result_of(hook, hook.handle(_call())) + + assert second["decision"] == "deny" + assert store.get_effect("refund:txn_1").state is EffectState.EXECUTING + + +# --- T54: the outcome mapping, via steps/toolCallResult -------------------------------- + + +@pytest.mark.parametrize( + ("exit_status", "state", "result"), + [ + ("success", EffectState.COMMITTED, ReceiptResult.COMMITTED), + ("timeout", EffectState.AMBIGUOUS, ReceiptResult.AMBIGUOUS), + ("failure", EffectState.AMBIGUOUS, ReceiptResult.AMBIGUOUS), + ("blocked", EffectState.FAILED, ReceiptResult.FAILED), + ], +) +def test_T54_exit_status_maps_by_the_asymmetry_of_v0_1_section_5_5( + hook, store, exit_status, state, result +): + """ACS names four statuses and says nothing about what any of them means for the side + effect. This is the fail-closed reading, and the same one §6.8 applies to MCP: + + - `success` is the only one that asserts the effect happened; + - `blocked` is the only one that asserts it did not — a control refused before dispatch; + - `failure` and `timeout` are both unknown, because a tool that failed after acting and + a tool that failed before acting report the same string. + """ + _both(hook, exit_status) + + assert store.get_effect("refund:txn_1").state is state + assert store.receipts()[-1].result is result + + +def test_T54_failure_is_FAILED_only_where_the_operator_asserted_it(hook, store): + """`not_executed_on_error` (§3.1) is the operator's per-tool claim that this tool reports + errors only before acting. It is the same assertion in ACS as it is in MCP.""" + _both(hook, "failure", tool="reports_before_acting") + + assert store.get_effect("report:txn_1").state is EffectState.FAILED + + +def test_T54_the_result_hook_returns_allow_because_it_redacts_nothing(hook, store): + """ACS describes toolCallResult as an output redaction checkpoint. CTRLRun records the + outcome and changes no output, so the conformant answer is `allow`.""" + _, response = _both(hook) + + assert _result_of(hook, response)["decision"] == "allow" + + +def test_T54_a_result_for_a_call_nobody_reserved_is_recorded_and_not_invented(hook, store): + """A result with no matching request — a restarted Guardian, or a platform that fired + them out of order. There is no reservation to close, so nothing is written about an + effect, and the response still permits the output through.""" + response = hook.handle(_result(str(uuid.uuid4()))) + + assert _result_of(hook, response)["decision"] == "allow" + assert store.receipts() == () + + +def test_T54_a_call_with_no_effect_key_needs_no_result_to_be_complete(hook, store): + """A read has no reservation to hold, so the request hook completes it outright.""" + envelope = _envelope( + TOOL_CALL_REQUEST, + {"tool": {"name": "read_balance", "provider": "stripe"}, "arguments": {}}, + ) + + assert _result_of(hook, hook.handle(envelope))["decision"] == "allow" + assert store.list_effects() == () + + +def test_T54_not_executed_on_error_does_not_reach_across_to_timeout(hook, store): + """The operator asserted something about *errors*. A timeout is not an error the tool + reported — it is the absence of any report — and no per-tool claim can speak for it.""" + _both(hook, "timeout", tool="reports_before_acting") + + assert store.get_effect("report:txn_1").state is EffectState.AMBIGUOUS + + +def test_T54_a_result_with_no_request_id_ref_writes_nothing(hook, store): + """Nothing links this result to a call. Guessing which one it meant is how a duplicate + gets committed, so no effect is written and the output still passes.""" + envelope = _result(CALL_ID) + del envelope["params"]["payload"]["request_id_ref"] + hook.handle(_call(request_id=CALL_ID)) + + response = hook.handle(envelope) + + assert _result_of(hook, response)["decision"] == "allow" + assert store.get_effect("refund:txn_1").state is EffectState.EXECUTING + assert store.receipts() == () + + +def test_T51_an_argument_that_is_not_a_provenance_envelope_is_refused(hook, store): + """`tool-call-request.json` requires `{value, provenance?}` per argument. A bare value + is a payload this adapter does not understand, and reading it as one would hash + something the producer did not mean.""" + envelope = _call() + # A mapping with provenance but no `value` — the shape a producer gets wrong, and the + # one a bare-value check would let through into `envelope["value"]` as a KeyError. + envelope["params"]["payload"]["arguments"] = {"payment_id": {"provenance": {"origin": "user"}}} + + response = hook.handle(envelope) + + assert "error" in response + assert store.events() == () + + bare = _call() + bare["params"]["payload"]["arguments"] = {"payment_id": "txn_1"} + assert "error" in hook.handle(bare) + + +def test_T51_a_call_with_no_agent_id_is_refused(hook, store): + """An Action cannot exist without a principal (v0.1 §2.1). ACS makes `agent_id` required + on every envelope, so its absence is a malformed request, not a decision.""" + envelope = _call() + del envelope["params"]["metadata"]["agent_id"] + + response = hook.handle(envelope) + + assert "error" in response + assert store.events() == () + # `Principal` would refuse an empty agent too, with a message about the Principal. This + # one names the ACS field a producer has to fix, which is the difference worth keeping. + assert "agent_id" in response["error"]["message"] + + +def test_T51_the_operation_appears_in_the_recorded_action_name(hook, store): + """Asserted on the receipt, not merely on an event existing: two verbs on one tool are + two actions, and a policy has to be able to say different things about them.""" + envelope = _call(request_id=CALL_ID) + envelope["params"]["payload"]["operation"] = "void" + envelope["params"]["payload"]["tool"]["name"] = "read_balance" + hook.handle(envelope) + + proposed = [e for e in store.events() if e.type.value == "POLICY_EVALUATED"] + assert proposed, "the action reached the policy" + assert store.receipts()[-1].action == "acs.stripe.read_balance.void" + + +# --- T55: the adapter is in an extra, and says so when it is absent -------------------- + + +def test_T55_an_unknown_method_is_a_jsonrpc_error_in_the_reserved_range(hook): + response = hook.handle(_envelope("steps/sessionStart", {})) + + assert "error" in response + assert -32099 <= response["error"]["code"] <= -32000 + + +def test_T55_a_malformed_envelope_is_a_jsonrpc_error(hook): + well_formed = _call() + wrong_version = {**well_formed, "jsonrpc": "1.0"} + for broken in ( + {}, + {"jsonrpc": "2.0"}, + {"jsonrpc": "1.0", "method": TOOL_CALL_REQUEST}, + # Only the version is wrong; every other guard would let this through, so this is + # the case that proves the version check is doing something. + wrong_version, + ): + response = hook.handle(broken) + assert "error" in response, broken + + +def test_T55_the_adapter_is_not_imported_by_importing_ctrlrun(): + import subprocess + import sys + + finished = subprocess.run( + [sys.executable, "-c", "import ctrlrun, sys; print('ctrlrun.acs' in sys.modules)"], + capture_output=True, + text=True, + check=True, + ) + + assert finished.stdout.strip() == "False" + + +def test_T55_the_response_is_json_serialisable(hook): + """A Guardian puts this on a wire; anything that will not serialise is a bug here.""" + for envelope in (_call(), _call(amount=200000), _call(amount=99999999)): + assert json.loads(json.dumps(hook.handle(envelope))) diff --git a/tests/test_examples.py b/tests/test_examples.py index bfa167fb..ce325aaf 100644 --- a/tests/test_examples.py +++ b/tests/test_examples.py @@ -34,6 +34,10 @@ "approval-replay": "single-use approval already consumed", } +#: Directories under `examples/` that are not one of §1.1's failure scenarios: the sector +#: templates, and item 8's ACS integration example (SPEC-v0.2 §9, `docs/ACS.md`). +NOT_A_SCENARIO = ("policies", "acs", "__pycache__") + #: The nine sectors of SPEC-v0.2 §1.1. SECTORS = ( "devops", @@ -143,7 +147,7 @@ def test_T31_the_four_scenarios_of_the_spec_are_the_ones_on_disk(): found = sorted( path.name for path in EXAMPLES.iterdir() - if path.is_dir() and path.name not in ("policies", "__pycache__") + if path.is_dir() and path.name not in NOT_A_SCENARIO ) assert found == sorted(SCENARIOS)