feat: core API — registry, capabilities, plans, Apply and retries

.github/workflows/ci.yml

@@ -3,6 +3,10 @@ name: CI
 on:
   pull_request:
     branches: [main]
+    # Drafts skip every job, so marking a PR ready must start a run; without
+    # ready_for_review the required checks stay "skipped", which a ruleset
+    # counts as passing.
+    types: [opened, synchronize, reopened, ready_for_review]
   # Re-run by hand, from the Actions tab or `gh workflow run ci.yml --repo simtabi/cloudkit`.
   workflow_dispatch:
   # Nothing re-runs after a merge, so this weekly run on main is what catches

CHANGELOG.md

@@ -20,9 +20,29 @@ All notable changes to this project are documented here. The format follows
 - CI on pull requests, by dispatch and weekly; a release workflow that runs only on a confirmed
   dispatch against a `v` tag.
 - A weekly and on-demand CI leg on macOS and Windows; pull requests still test on Linux only.
+- `Options`, `CredentialSource` and `Registry` (`NewRegistry`, `Open`, `Descriptors`): the caller
+  builds the registry from one `Factory` per provider; it refuses invalid options, incomplete
+  descriptors, empty or non-slug provider IDs and duplicate IDs, and keeps no package-level state.
+- `Provider`, `Descriptor`, `TokenModel`, `Module`, `Resource`, `Filter` and `Account`: each `Open`
+  gives an adapter a fresh client that is limited to its declared hosts, fetches the account's token
+  per request, sends it only to API hosts and never on a redirect, and retries.
+- `Capability`, `Access` (`Allowed`, `Denied`, `Unknown`) and `CapSet`, with the capability catalogue;
+  an unknown capability counts as a write.
+- `Plan`, `Change`, `Money`, `NewPlan` and a SHA-256 digest over the provider, account, changes, what a
+  destroy loses and the live state; plans expire after `PlanTTL`.
+- `Registry.Apply` with the `Applier` interface: refuses an altered or expired plan and a plan whose
+  live state moved (`ErrConflict`, `PlanError`), and a destroy whose resource names were not typed back
+  (`ErrConfirmationRequired`, `Confirmation`), before anything is sent.
+- `RetryPolicy` and `RateLimitError`: jittered backoff, `Retry-After` honoured up to `MaxDelay`, and no
+  retry of a non-idempotent request after an ambiguous failure.
+- `UnsupportedError`, matching `ErrUnsupported`.
+- Policy tests: context first on every interface and registry method (C10), no package-level client
+  or registry (C9), and no shipped exported name removed.
 
 ### Changed
 
 - The architecture guide states the scope split: cloudkit finds and connects to managed databases
   but never administers them, and REST descriptors (planned) support writes behind signing or
   explicit enablement, plans and the contract suite.
+- The architecture guide explains the plan digest and the client's layering, and the getting-started
+  guide covers the registry, plans, `Apply` and retries.

README.md

@@ -62,7 +62,7 @@ Hosted at <https://opensource.simtabi.com/documentation/simtabi/cloudkit/>.
 ### Guides
 
 - [Installation](docs/installation.md) — the module path, Go version and pinning before `v0.1.0`
-- [Getting started](docs/getting-started.md) — transports, egress policies, secrets and errors
+- [Getting started](docs/getting-started.md) — transports, the registry, plans and applying them, retries, secrets and errors
 - [Architecture](docs/architecture.md) — the rules every adapter follows, and why
 - [Release](docs/release.md) — how a version is tagged and released
 

docs/architecture.md

@@ -35,9 +35,12 @@ its own engine adapters.
 | C9 | No global state: no `init()` registration, no package-level clients |
 | C10 | `context.Context` first on every call; the logger is injected |
 
-C1, C3, C4 and C5 are implemented in this release. A policy test in `src/internal/policy` scans
-every Go file for the symbols that would break C1, C4, C8, C9 and C10, with an exemption list that
-records the exact count it allows.
+C1 to C7, C9 and C10 are implemented in this release; C8 arrives with the SSH path. A policy test
+in `src/internal/policy` scans every Go file for the symbols that would break C1, C4, C8, C9 and
+C10, with an exemption list that records the exact count it allows. Three more policy tests read
+the package's syntax: every interface method, `*Registry` method and exported function field takes
+a `context.Context` first (C10); no package-level client or registry exists (C9); and no exported
+name the package has shipped disappears.
 
 ## Why the caller owns the transport
 
@@ -47,6 +50,31 @@ a library that accepted nil, or built its own client, would route around that gu
 noticing. Refusing nil and refusing the default transport makes the bypass impossible to write by
 accident.
 
+## Why plans carry a digest
+
+Both consumers show a plan to a person, collect an approval, and apply it later, possibly minutes
+later and from another process. The digest is SHA-256 over the provider, the account, the changes,
+what a destroy loses, and the live state the plan was built against. `Apply` reads the live state
+again through the adapter and recomputes it, so one comparison refuses a plan whose infrastructure
+moved, a plan edited after approval (a trimmed `Loses` list, another target), and a plan replayed
+against another account. The plan ID is derived from the digest, so an approval recorded against an
+ID cannot be reused for different content. `Cost` and `Expires` are outside the digest: the first is
+informational, and the second is checked on its own.
+
+A destroy is applied only when the consumer passes back the deleted resources' names exactly as
+`Plan.ConfirmText` gives them. Approval levels and step-up remain the consumer's; cloudkit checks
+only the typed name, and does so before it opens the provider, so a refusal makes no request.
+
+## Why the client is layered as it is
+
+The client an adapter receives puts the egress guard outermost, then retries, then the token, then
+the caller's round tripper. A refused host therefore never reaches the credential source or the
+retry loop. The token is added only to requests for the descriptor's API hosts, and never to a
+request that follows a redirect, even to an allowed redirect host: a token for the API is not a
+token for the provider's object store. Retries never resend a non-idempotent request after an
+ambiguous failure, so a create is not sent twice; a 429 is retried because the provider did not act
+on it.
+
 ## Why it is a library without a CLI framework
 
 Each consumer has its own command tree and configuration loader. A library that brought cobra or a
@@ -68,7 +96,7 @@ before it loads.
 
 | Path | Holds |
 |---|---|
-| `src/` | package `cloudkit`: transport, egress policy, secrets, errors, redaction |
+| `src/` | package `cloudkit`: transport, egress policy, secrets, errors, redaction, the registry, capabilities, plans and `Apply`, retries |
 | `src/internal/policy/` | repository-wide tests: forbidden symbols, workflow guards |
 
 Planned packages are added when they have code: `providers/` (wave 1: Hetzner, DigitalOcean,

docs/getting-started.md

@@ -1,7 +1,7 @@
 # Getting started
 
-A walkthrough of the four things this release provides: the transport, the egress policy, secrets
-and the error set.
+A walkthrough of what this release provides: the transport, the egress policy, the registry, plans
+and applying them, retries, secrets and the error set.
 
 ## The transport
 
@@ -46,6 +46,156 @@ client, err := cloudkit.NewHTTPClient(transport, cloudkit.EgressPolicy{
 | Redirect-only hosts | a host listed only in `RedirectHosts` is reachable by redirect, never directly |
 | Refusal | before your round tripper sees the request, as a `*cloudkit.EgressError` matching `cloudkit.ErrEgressDenied` |
 
+## The registry
+
+`cloudkit.Registry` is built by you, from `cloudkit.Options` and one `cloudkit.Factory` per provider.
+Nothing registers itself, and there is no package-level registry or client.
+
+```go
+reg, err := cloudkit.NewRegistry(cloudkit.Options{
+	Transport:   transport,                     // required; nil or the default transport is refused
+	Credentials: vaultTokens,                   // required; a cloudkit.CredentialSource
+	Logger:      logger,                        // required; slog.New(slog.DiscardHandler) to silence
+	Retry:       cloudkit.RetryPolicy{},        // zero value: 3 tries, jittered backoff
+	UserAgent:   "dbpantry/1.4.0",
+}, hetznerFactory, digitaloceanFactory)
+```
+
+`NewRegistry` refuses invalid options, a factory whose descriptor is incomplete, a factory with no
+`New` function, and two factories with the same ID. A descriptor ID is lower-case words joined by
+hyphens (`hetzner`, `digitalocean`), never empty.
+
+| `Options` field | Required | Notes |
+|---|---|---|
+| `Transport` | yes | both `HTTP` and `Dial`; see above |
+| `Credentials` | yes | `Token(ctx, account)` is called for every request, so a rotated token is used on the next one |
+| `Logger` | yes | cloudkit never uses the default logger |
+| `Retry` | no | see [Retries](#retries) |
+| `UserAgent` | no | sent as `User-Agent` when set |
+
+### Credentials
+
+```go
+type vault struct{ store *Vault }
+
+func (v vault) Token(ctx context.Context, account string) (cloudkit.Secret, error) {
+	b, err := v.store.Read(ctx, "providers/"+account)
+	if err != nil {
+		return cloudkit.Secret{}, err
+	}
+	return cloudkit.NewSecret(b), nil
+}
+```
+
+An empty token is refused rather than sent, and a failing source is reported without repeating its
+message, so nothing the vault says about a token reaches an error string.
+
+### Opening a provider
+
+```go
+p, err := reg.Open(ctx, "hetzner", "production")
+if err != nil {
+	return err // cloudkit.ErrNotFound for an unknown provider
+}
+caps, err := p.Probe(ctx)
+if caps.Allowed(cloudkit.CapServerCreate) {
+	// offer "create server"
+}
+servers, err := p.Inventory(ctx, cloudkit.Filter{Kinds: []string{"server"}})
+```
+
+Each `Open` builds a fresh client, so two accounts never share one. The adapter receives a
+`cloudkit.Account` whose `HTTP` client:
+
+| Layer, outermost first | Does |
+|---|---|
+| Egress guard | refuses any host outside the descriptor's `APIHosts`, and redirects outside `RedirectHosts` |
+| Retry | the registry's `RetryPolicy` |
+| Authenticator | adds `Authorization: Bearer <token>` from the credential source, only for API hosts and never on a redirect |
+| Your round tripper | everything else: proxy, TLS, timeouts, your own egress controls |
+
+### Capabilities
+
+`Probe` returns a `cloudkit.CapSet`: each capability it tested and whether the token is `Allowed`
+or `Denied`. A capability it did not test reads as `Unknown`, which is not allowed. A read-only
+token shows every write as `Denied`. `Capability.IsWrite` says which capabilities change anything;
+a name cloudkit does not know counts as a write.
+
+## Plans and applying them
+
+Every write is planned first. An adapter builds a plan with `cloudkit.NewPlan` from the changes,
+what a destroy loses, and the live state it read:
+
+```go
+plan, err := lifecycle.PlanDestroy(ctx, "srv-4821")
+// plan.Changes: delete server web-1
+// plan.Loses:   server web-1, volume data-1, ipv4 203.0.113.7
+// plan.ConfirmText(): "web-1"
+```
+
+| `Plan` field | Holds |
+|---|---|
+| `ID` | stable, derived from the digest; approvals refer to it |
+| `Provider`, `Account` | where it applies; `Apply` opens the same account |
+| `Changes` | each create, update or delete, with the resource before and after |
+| `Loses` | for a destroy, every resource deleted and everything that goes with it |
+| `Cost` | monthly delta, when the provider reports one; not covered by the digest |
+| `Digest` | SHA-256 over the provider, account, changes, loses and the live state |
+| `Expires` | 15 minutes after the plan was built (`cloudkit.PlanTTL`) |
+
+Show the plan, collect the approval your product requires, then apply it:
+
+```go
+result, err := reg.Apply(ctx, plan, cloudkit.Confirmation{Typed: typedByThePerson})
+switch {
+case errors.Is(err, cloudkit.ErrConfirmationRequired):
+	// a destroy, and the name was not typed back exactly
+case errors.Is(err, cloudkit.ErrConflict):
+	// the live state moved, the plan expired, or it was altered: plan again
+}
+```
+
+`Apply` refuses before anything changes, in this order:
+
+| Refusal | Error |
+|---|---|
+| the plan is malformed, was altered after `NewPlan`, or has expired | `ErrConflict` (`*PlanError`) |
+| a destroy whose `ConfirmText` was not typed back exactly | `ErrConfirmationRequired` |
+| the provider cannot apply plans | `ErrUnsupported` (`*UnsupportedError`) |
+| the live state the adapter reads again no longer matches the digest | `ErrConflict` (`*PlanError`) |
+
+`Result` comes back only when the provider reports the work done, not merely accepted.
+
+### Writing the apply side of an adapter
+
+A provider that applies plans implements `cloudkit.Applier`:
+
+```go
+type Applier interface {
+	LiveState(ctx context.Context, p Plan) ([]Resource, error) // same shape as passed to NewPlan
+	Execute(ctx context.Context, p Plan) (Result, error)       // waits for the provider's actions
+}
+```
+
+`NewPlan` refuses a delete that is not listed in `loses`, or whose resource has no name to type
+back.
+
+## Retries
+
+The registry's clients retry by `cloudkit.RetryPolicy`. The zero value is 3 tries, jittered
+exponential backoff from 500 ms, and `Retry-After` honoured up to 30 s; `MaxAttempts: 1` turns
+retries off.
+
+| Response | Retried for |
+|---|---|
+| 429 | any request whose body can be replayed; the provider did not act on it |
+| 502, 503, 504, or a transport error | idempotent methods only (GET, HEAD, OPTIONS, PUT, DELETE), so a create is never sent twice |
+| anything else | not retried |
+
+A 429 that outlasts the budget, or asks to wait longer than `MaxDelay`, ends in a
+`*cloudkit.RateLimitError` matching `cloudkit.ErrRateLimited`, with the `RetryAfter` the provider
+asked for.
+
 ## Secrets
 
 `cloudkit.Secret` holds a credential and prints as `[REDACTED]` in every fmt verb, in `log/slog`,
@@ -69,6 +219,7 @@ req.Header.Set("Authorization", "Bearer "+string(token.Reveal()))
 | `ErrRateLimited` | throttled after the retry budget |
 | `ErrEgressDenied` | the egress policy refused a request or redirect |
 | `ErrUnsupported` | the adapter does not implement the capability |
+| `ErrConfirmationRequired` | a destroy plan was applied without its names typed back |
 
 Match them with `errors.Is`. `cloudkit.Redact` strips Authorization-style headers, bearer
 credentials, credential query parameters and URL userinfo from a string, and `cloudkit.RedactBody`
@@ -77,9 +228,9 @@ URL with them before logging it.
 
 ## What comes next
 
-The registry, capability probes, plans with drift refusal, the contract test kit, the SSH path, the
-cloud-init renderer, the first provider adapters (Hetzner, DigitalOcean, Linode, Vultr), managed
-database discovery and REST descriptors are planned; see [Architecture](architecture.md).
+The contract test kit, the hardened transport, the SSH path, the cloud-init renderer, the first
+provider adapters (Hetzner, DigitalOcean, Linode, Vultr), managed database discovery and REST
+descriptors are planned; see [Architecture](architecture.md).
 
 ---
 

src/capability.go

@@ -0,0 +1,100 @@
+package cloudkit
+
+import (
+	"fmt"
+	"maps"
+	"slices"
+)
+
+// Capability names one thing a token may be allowed to do, as "<area>.<verb>".
+type Capability string
+
+// The capabilities a probe reports on. Reads need no plan; every other one is
+// a write and goes through [Plan] and [Registry.Apply].
+const (
+	CapInventory Capability = "inventory.read"
+
+	CapServerCreate  Capability = "server.create"
+	CapServerResize  Capability = "server.resize"
+	CapServerDestroy Capability = "server.destroy"
+	CapServerPower   Capability = "server.power"  // power and reboot
+	CapServerUpdate  Capability = "server.update" // rename, labels, protection
+
+	CapFirewallRead  Capability = "firewall.read"
+	CapFirewallWrite Capability = "firewall.write"
+
+	CapNetworkRead  Capability = "network.read"
+	CapNetworkWrite Capability = "network.write"
+
+	CapVolumeRead  Capability = "volume.read"
+	CapVolumeWrite Capability = "volume.write"
+
+	CapSnapshotRead  Capability = "snapshot.read"
+	CapSnapshotWrite Capability = "snapshot.write"
+
+	CapObjectStorageRead  Capability = "object-storage.read"
+	CapObjectStorageWrite Capability = "object-storage.write"
+
+	CapDNSRead  Capability = "dns.read"
+	CapDNSWrite Capability = "dns.write"
+
+	CapHostKeysRead Capability = "host-keys.read"
+	CapCostRead     Capability = "cost.read"
+
+	CapManagedDatabasesRead    Capability = "managed-databases.read"
+	CapManagedDatabasesConnect Capability = "managed-databases.connect" // mints a short-lived connect credential; changes nothing
+)
+
+// reads lists the capabilities that change nothing. Anything else, including
+// a capability this release does not know, counts as a write, so an unknown
+// name fails safe.
+var reads = []Capability{
+	CapInventory, CapFirewallRead, CapNetworkRead, CapVolumeRead, CapSnapshotRead,
+	CapObjectStorageRead, CapDNSRead, CapHostKeysRead, CapCostRead,
+	CapManagedDatabasesRead, CapManagedDatabasesConnect,
+}
+
+// IsWrite reports whether c changes anything. An unknown capability is a write.
+func (c Capability) IsWrite() bool { return !slices.Contains(reads, c) }
+
+// Access is what a probe found for one capability.
+type Access int
+
+// The three answers a probe gives. The zero value is Unknown, so a capability
+// a probe did not test is never mistaken for an allowed one.
+const (
+	Unknown Access = iota // not tested, or the provider would not say
+	Allowed               // the token may do it
+	Denied                // the token may not; neither product offers it
+)
+
+func (a Access) String() string {
+	switch a {
+	case Unknown:
+		return "unknown"
+	case Allowed:
+		return "allowed"
+	case Denied:
+		return "denied"
+	}
+	return fmt.Sprintf("Access(%d)", int(a))
+}
+
+// CapSet is a probe's answer: each capability it tested and what it found.
+// A capability missing from the set is Unknown. A nil CapSet reads as all
+// Unknown; Set needs one made with make or a literal.
+type CapSet map[Capability]Access
+
+// Get returns what the probe found for c, Unknown when it did not say.
+func (s CapSet) Get(c Capability) Access { return s[c] }
+
+// Allowed reports whether the probe found c allowed. Unknown is not allowed.
+func (s CapSet) Allowed(c Capability) bool { return s[c] == Allowed }
+
+// Set records a for c.
+func (s CapSet) Set(c Capability, a Access) { s[c] = a }
+
+// Capabilities returns the capabilities in the set, sorted.
+func (s CapSet) Capabilities() []Capability {
+	return slices.Sorted(maps.Keys(s))
+}