Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,12 +139,18 @@ Pragmas: `journal_mode=WAL`, `busy_timeout=5000`, `synchronous=NORMAL`.
| `approval.py` | request/grant/consume, providers | executors |
| `effect.py` | key templating, state enum, transition rules | SQLite |
| `state.py` | `StateStore` protocol + SQLite/in-memory impls | policy, decorator, sinks |
| `control.py` | `Control` orchestration, decorator, context | CLI |
| `control.py` | `Control` orchestration, decorator, context, suspend/resume | CLI |
| `receipt.py` | Receipt/Event models, `EventSink`, JSONL sink | everything else |
| `cli/` | click commands, demo | internals beyond `Control` |

Dependencies point downward only. `Control` is the only module that composes the others.

The gateway (v0.2) does not change this. It builds an Action and calls `Control` — including
`Control.resume` for an elicitation's second leg — rather than reserving and committing for
itself. A gateway that owned the reservation would be a second module composing the others,
and a second implementation of SPEC-v0.1 §5.5's asymmetry, which is the one rule in this
codebase that must not drift.

One exception, added in v0.2 and worth stating rather than discovering: `policy.py` imports the
template grammar (`template_placeholders`) from `effect.py`, because SPEC-v0.2 §3.1 requires an
`effect:` / `resource:` template to be validated when the policy loads and the grammar is
Expand Down
41 changes: 38 additions & 3 deletions docs/SPEC-v0.2.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ anything it cannot parse, and it never lets a transport error look like a defini
### 2.1 The hook

```python
Suspended(continuation: bytes | str) # raised by an executor; §6.9
Control.resume(continuation: str, executor) -> Receipt

ReconcileOutcome = Literal["committed", "not_executed", "unknown"]

protect(..., reconcile: Callable[[str], ReconcileOutcome] | None = None,
Expand Down Expand Up @@ -785,6 +788,37 @@ that exchange.

#### 6.9.2 Held reservation, where the upstream made it possible

**How this reaches the kernel.** A held reservation spans two HTTP requests and therefore two
`Control` calls, and `Control.execute` writes a terminal outcome for any executor exception
(v0.1 §5.5) — so the gateway cannot express "suspend, record nothing" through it. The kernel
grows one signal and one method, and suspension is modelled the way v0.1 models *nothing
happened*: an explicit opt-in an executor raises, never a default and never inferred.

- `Suspended(continuation)`, raised by an executor, is handled in `Control.execute` **before**
the generic exception branch, alongside `NotExecuted`. The record stays `EXECUTING`, the
lease is extended by `Control(suspend_timeout=...)`, the continuation is held via
`hold_continuation` **in the same transaction**, `EXECUTION_SUSPENDED` is appended, **no
receipt is written** — the same rule as an action awaiting a human (v0.1 §6.1) — and the
signal propagates so the caller can relay the elicitation.
- `Suspended` on an action with **no effect key** holds nothing: there is no reservation to
hold. `EXECUTION_SUSPENDED` records `held: false` and the signal still propagates.
Suspension without an effect key gives no protection at all — a concurrent caller can start
the same work — exactly as retries without one give none (v0.1 §5.1). It is logged, not
silently pretended.
- `Control.resume(continuation, executor)` consumes the continuation atomically, rehydrates
the action and effect key from the store, checks the record is still `EXECUTING` under a
live lease, appends `EXECUTION_RESUMED`, and runs the executor through **the same** outcome
mapping `execute` uses. That path is one private function called by both: a resumption that
decided outcomes differently would be a second implementation of the only question this
library exists to answer. A resumed executor may raise `Suspended` again; each round
re-holds.
- `resume` re-evaluates the policy for the **receipt**, not to re-decide. The action was
decided and reserved on the first leg, and refusing at resume would strand a reservation
the remote may already be acting on.

`Control` remains the only module that composes the others (ARCHITECTURE §6); the gateway
calls it.

When an intercepted `tools/call` with an effect key returns `input_required` **with a
`requestState`**:

Expand Down Expand Up @@ -1286,7 +1320,7 @@ claim (asserted against a word list including "compliant", "compliance", "certif
from .effect import ReconcileOutcome
from .receipt import EventSink, JSONLEventSink
from .approval import WebhookApprovalProvider
from .errors import MissingDependency
from .errors import MissingDependency, Suspended
# lazily importable, not re-exported at package import:
# ctrlrun.otel.OTelEventSink (ctrlrun[otel])
# ctrlrun.gateway.serve (ctrlrun[gateway])
Expand Down Expand Up @@ -1314,8 +1348,9 @@ StateStore.find_granted_approval(action_hash: str) -> Approval | None
StateStore.find_denied_request(action_hash: str) -> ApprovalRequest | None
StateStore.resolve_effect(effect_key, state, *, resolved_by: str) -> None
StateStore.extend_lease(effect_key: str, action_id: str, until: datetime) -> None
StateStore.hold_continuation(effect_key, action_id, request_state: str) -> None
StateStore.take_continuation(effect_key, request_state: str) -> EffectRecord
StateStore.hold_continuation(action, effect_key, continuation: str, until: datetime) -> int
StateStore.take_continuation(continuation: str) -> HeldContinuation
StateStore.continuation_rounds(effect_key: str) -> int
```

**Removed:** `SQLiteStateStore.journal`, and the `EventLog` behind it — `JSONLEventSink` is
Expand Down
2 changes: 2 additions & 0 deletions src/ctrlrun/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
MissingDependency,
NotExecuted,
PolicyError,
Suspended,
)
from .policy import Decision, Policy
from .receipt import Event, EventSink, JSONLEventSink, Receipt
Expand Down Expand Up @@ -64,6 +65,7 @@
"SQLiteStateStore",
"ScriptedApprovalProvider",
"StateStore",
"Suspended",
"action_hash",
"canonicalize",
"context",
Expand Down
112 changes: 112 additions & 0 deletions src/ctrlrun/control.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
EffectKeyError,
InvalidArgument,
NotExecuted,
Suspended,
)
from .policy import Decision, Evaluation, Policy, discover_policy_path
from .receipt import (
Expand All @@ -70,6 +71,10 @@

DEFAULT_ENVIRONMENT: Final = "production"

#: SPEC-v0.2 §6.9.2 — how long a reservation is held open across one round trip. A client
#: that never returns lets this lapse, and the record becomes AMBIGUOUS by v0.1 §5.3 E3.
DEFAULT_SUSPEND_TIMEOUT: Final = timedelta(minutes=5)

#: Denial reason when a protected function is called outside `ctrlrun.context()`.
NO_PRINCIPAL: Final = "no_principal"

Expand Down Expand Up @@ -209,6 +214,7 @@ def __init__(
approval_ttl: timedelta = DEFAULT_APPROVAL_TTL,
lease: timedelta = DEFAULT_LEASE,
sinks: Sequence[EventSink] = (),
suspend_timeout: timedelta = DEFAULT_SUSPEND_TIMEOUT,
) -> None:
self._policy = policy
self._store = store
Expand All @@ -219,6 +225,7 @@ def __init__(
self._approval_ttl = approval_ttl
self._lease = _checked_lease(lease, "Control(lease=...)")
self._sinks = tuple(sinks)
self._suspend_timeout = _checked_lease(suspend_timeout, "Control(suspend_timeout=...)")

@classmethod
def from_file(cls, path: str | os.PathLike[str] | None = None) -> Control:
Expand Down Expand Up @@ -331,8 +338,74 @@ def execute(
)
raise
self._append(EventType.EXECUTION_STARTED, action, {}, effect_key, approval=approval)
return self._outcome(
action, evaluation, executor, effect_key, attempt, approval, started_at, reconciler
)

def resume(self, continuation: str, executor: Callable[[], Any]) -> Receipt:
"""Finish an action a `Suspended` executor left open (SPEC-v0.2 §6.9).

The continuation is taken and consumed atomically, so one suspension admits exactly
one resumption; the action and its effect key are rehydrated from the store, which is
what lets a process that restarted mid-round still finish one. The record must still
be `EXECUTING` under a live lease — a lease that lapsed while the client was
answering is `AMBIGUOUS` by the ordinary path of v0.1 §5.3 E3.

The executor then runs through the **same** outcome mapping `execute` uses. There is
one implementation of v0.1 §5.5 in this codebase, and a resumed call gets that one:
a resumption that decided outcomes differently would be a second answer to the only
question this library exists to answer.
"""
held = self._store.take_continuation(continuation)
started_at = self._clock()
action = held.action
self._append(
EventType.EXECUTION_RESUMED,
action,
{"round": held.rounds},
held.effect_key,
)
# SPEC: §6.9.2 — the policy is evaluated again for the *receipt*, not to re-decide.
# The action was decided and reserved on the first leg; refusing here would strand a
# reservation the remote may already be acting on, which is the one thing a held
# reservation exists to avoid.
evaluation = self._policy.evaluate(action)
return self._outcome(
action,
evaluation,
executor,
held.effect_key,
held.record.attempt,
None,
started_at,
_Reconciler(None, False),
)

def _outcome(
self,
action: Action,
evaluation: Evaluation,
executor: Callable[[], Any],
effect_key: str | None,
attempt: int,
approval: Approval | None,
started_at: datetime,
reconciler: _Reconciler,
) -> Receipt:
"""Run the executor and record what happened (SPEC-v0.1 §5.5).

The single implementation of the asymmetry: `NotExecuted` is the only outcome that
maps to `FAILED`, `Suspended` records no outcome at all, and everything else is
`AMBIGUOUS`. `execute` and `resume` both come through here, so the two cannot drift.
"""
try:
result = executor()
except Suspended as suspension:
# SPEC-v0.2 §6.9 — no outcome, no receipt: the remote has not said what happened
# and this attempt is not finished. Handled above the generic branch precisely so
# it cannot be mistaken for one.
self._suspend(action, effect_key, suspension, approval)
raise
except NotExecuted as exc:
if effect_key is not None:
# SPEC §5.5 — the executor asserted the remote side did nothing, so this is
Expand Down Expand Up @@ -431,6 +504,45 @@ def execute(
attempt=attempt,
)

def _suspend(
self,
action: Action,
effect_key: str | None,
suspension: Suspended,
approval: Approval | None,
) -> None:
"""Hold the reservation open and record that it is held (SPEC-v0.2 §6.9.2).

With no effect key there is nothing to hold, and the event says so. Suspension
without an effect key gives no protection at all — a second caller can start the same
work while the first is waiting — exactly as retries without one give none (v0.1
§5.1). It is permitted, logged, and recorded as unheld rather than silently pretended.
"""
if effect_key is None:
_LOG.warning(
"%s: suspended with no effect key, so nothing is held; a concurrent caller "
"can start the same work. Declare effect= to make a suspension mean something",
action.name,
)
self._append(
EventType.EXECUTION_SUSPENDED,
action,
{"held": False, "round": 1},
None,
approval=approval,
)
return
rounds = self._store.hold_continuation(
action, effect_key, suspension.continuation, self._clock() + self._suspend_timeout
)
self._append(
EventType.EXECUTION_SUSPENDED,
action,
{"held": True, "round": rounds},
effect_key,
approval=approval,
)

# --- the effect key (SPEC-v0.1 §5.1) ------------------------------------------------

def _resolve_effect(self, action: Action, template: str | None) -> str | None:
Expand Down
29 changes: 29 additions & 0 deletions src/ctrlrun/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,35 @@ class NotExecuted(CTRLRunError):
"""


class Suspended(CTRLRunError):
"""Raised by an executor: the remote asked for something before it will finish.

SPEC-v0.2 §6.9 in the kernel's own terms, and modelled the way v0.1 §5.5 models "nothing
happened" — an explicit opt-in signal, never a default and never inferred. There is no
outcome to record: the effect record stays `EXECUTING`, its lease is extended, the
continuation is held, and the caller gets this back to relay.

`continuation` is whatever the remote said to present again. It is opaque here — CTRLRun
never parses it, and only ever compares it with `hmac.compare_digest`.
"""

def __init__(self, continuation: bytes | str) -> None:
if isinstance(continuation, bytes):
try:
continuation = continuation.decode("utf-8")
except UnicodeDecodeError as exc:
raise InvalidArgument(
"Suspended(continuation=...) must be UTF-8 decodable"
) from exc
if not continuation:
raise InvalidArgument(
"Suspended(continuation=...) must be non-empty; a continuation nobody can "
"present again is a suspension nobody can end"
)
super().__init__(f"suspended awaiting a continuation ({len(continuation)} bytes)")
self.continuation = continuation


class MissingDependency(CTRLRunError):
"""An optional extra is not installed (SPEC-v0.2 §1.1, §11).

Expand Down
Loading
Loading