Skip to content

feat: add cloudkittest, the contract kit - #3

Merged
imanimanyara merged 4 commits into
mainfrom
feat/contract-kit
Nov 5, 2026
Merged

imanimanyara merged 4 commits into
mainfrom
feat/contract-kit

Conversation

@imanimanyara

Copy link
Copy Markdown
Member

Summary

Phase 2 of cloudkit: the contract kit every adapter passes.

  • cloudkittest.Run / Check run nine contract cases against any Provider: transport only, nil transport, egress, read-only token, plan drift, destroy, redaction, cloud-init and SSH. Each case uses its own registry, a token generated for the run, no retries, refused dialing, and either a replaying Recorder or a fault-injecting transport. Nothing reaches the network. Check returns a Report without a *testing.T, so the descriptor loader can use it later.
  • Recorder replays one cassette and answers each interaction once, in order. It refuses any request the cassette does not hold (ErrUnexpectedRequest) and reports interactions nothing used, and the suite fails on either.
  • ParseRecordings checks every recorded and synthetic response against the provider's OpenAPI document (ErrNotInSpec, ErrSchemaViolation). It refuses any recording that carries an Authorization, cookie or API-key header, or a value cloudkit.Redact would change (ErrCredentialInRecording).
  • The OpenAPI validator (ParseSpec, Spec.ValidateResponse) supports a stated JSON Schema subset: type, nullable, required, properties, additionalProperties, items, enum and local $ref, plus annotations it does not check. Any other keyword fails the load with ErrUnsupportedKeyword. Spec.Synthesize builds a response from the document's examples and marks it Synthetic.
  • cloudkit.RedactError (new export) redacts an error's message and keeps its chain, so an adapter outside the root package can redact and its callers still match ErrEgressDenied.
  • The policy tests now read cloudkittest too: no default client or transport, context first on its hook signatures, and its shipped names preserved.

Stdlib only; no go.sum. Every existing exported symbol is unchanged.

Proof

  • The core tests' fake provider is written again against a hand-written spec in src/cloudkittest/testdata/. It passes all nine cases through Run.
  • One deliberately broken variant per case (TestEachCaseCatchesItsDefect), and each fails its own case with the expected message:
    • bypassing the transport
    • accepting a nil client
    • flattening the egress error
    • a probe that ignores the token
    • stale live state
    • dropped attachments
    • unredacted errors
    • a renderer that ignores the size limit
    • offering password authentication
  • testdata/broken-schema.recordings.json fails to load with ErrSchemaViolation at $.write.
  • I planted a defect for each guard, watched its test fail, then restored it: unexpected request, Authorization and token in a recording, schema violation, unknown keyword, default transport or client in cloudkittest, file floor, context-first, shipped surface, the synthetic marker, and RedactError's chain.

Checks

make ci and make lint (0 issues) pass locally. Coverage is 95.9% for cloudkit and 84.1% for cloudkittest. gitleaks detect over the full history found no leaks.

The cloud-init and SSH cases run here against stand-ins in the test file. The real renderer and SSH client arrive with the generic path and plug into Suite.CloudInit and Suite.SSH. Until then, a suite without SSH reports that case as skipped, with its reason.

An adapter in its own package has to return transport and provider
errors through the one redaction rule, and its callers still match
ErrEgressDenied and ErrDenied with errors.Is. The unexported redactErr
already did both; RedactError exposes it, and redactErr now delegates.
Run and Check run nine contract cases against any Provider: transport
only, nil transport, egress, read-only token, plan drift, destroy,
redaction, cloud-init and SSH. Each case builds its own registry over a
replaying Recorder or a fault-injecting transport, with a token
generated for the run, so nothing reaches the network.

Recorder replays one cassette, answers each interaction once and in
order, and refuses any request it does not hold. Recordings load only
through ParseRecordings, which checks every recorded and synthetic
response against the provider's OpenAPI document and refuses any
recording carrying an Authorization header or a credential value.

The validator implements a stated JSON Schema subset and refuses every
other keyword when the document loads. Synthetic responses come from
the document's examples and are marked as such.

The core tests' fake provider is written again against a hand-written
spec under testdata; it passes the suite, and one planted defect per
case fails that case.
The file floor rises to the exact 28 the scan reads. Context-first now
also checks exported func types and reads cloudkittest's hooks, with
the floor at the measured 14 signatures. The shipped-surface guard
covers cloudkittest's exported names and RedactError.
A reference page for cloudkittest (the nine cases, cassettes,
recordings, Recorder, the validator's subset, errors), the reasons for
spec-checked replay in the architecture guide, and the changelog.
@imanimanyara
imanimanyara marked this pull request as ready for review November 4, 2026 20:49
Copilot AI balanced review requested due to automatic review settings November 4, 2026 20:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@imanimanyara
imanimanyara merged commit c375333 into main Nov 5, 2026
12 checks passed
@imanimanyara
imanimanyara deleted the feat/contract-kit branch November 7, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants