feat(#1277): the CloudFront origin access control family, and a tagged create

CHANGELOG.md

@@ -64,6 +64,41 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   behaves exactly as it did. The cost, stated in `docs/services.md`: an identifier is guessable from
   a request ID, which is already true of the request ID and is acceptable for a test emulator whose
   identifiers name nothing outside it.
+- **CloudFront's origin access control family, and `CreateDistributionWithTags`** (#1277). An origin
+  access control is what lets a distribution serve a **private** S3 bucket — CloudFront signs the
+  origin request with SigV4 and the bucket policy trusts the distribution rather than the world — and
+  substrate routed none of it: `/2020-05-31/origin-access-control` was not a path the plugin knew, so
+  the create reached the unknown-route refusal and a consumer keeping its bucket private could not run
+  against the emulator at all. That was the first of ten missing operations an adopter found by
+  replacing ~1,100 lines of hand-written AWS fakes with substrate (#1274), and the one it named as
+  blocking. `CreateOriginAccessControl` answers 201 with the `ETag` and `Location` headers — neither is
+  in the API Reference's Response Syntax block, both are output members in the CLI and the SDKs, and
+  the ETag is what a later delete has to echo. The four `Required: Yes` config members are validated,
+  three of them against their published enums (`s3|mediastore|mediapackagev2|lambda`,
+  `never|always|no-override`, `sigv4`), case-sensitively: accepting `S3` would pass a request through
+  substrate that AWS refuses, which is the direction a consumer pays for with a failed live deploy.
+  `GetOriginAccessControl` answers the same document and the same version; `ListOriginAccessControls`
+  answers the account's controls whole, and an account using none answers **no `Items` element at
+  all**, which is what the page states and what a decoder cannot distinguish from an empty one — so
+  the test asserts on the raw XML. `DeleteOriginAccessControl` requires `If-Match` and separates the
+  ways it can fail: an absent control is `NoSuchOriginAccessControl`/404, a version that is *missing
+  or malformed* is `InvalidIfMatchVersion`/400 — the code's own published description is "missing or
+  not valid", which is two cases in one sentence — and a well-formed but stale one is
+  `PreconditionFailed`/412, because telling a caller that sent no version, or a typo, that its version
+  was stale is a wrong answer. Malformed is decidable only because the ETag rendering is substrate's
+  own: a value outside the shape substrate mints was never handed out here, so it cannot be a stale
+  one, and that is a statement about substrate's minting rather than about what CloudFront accepts.
+  `CreateDistributionWithTags` is
+  the same path and verb as `CreateDistribution` with `?WithTags` — a bare query key, so the routing
+  tests for the key's *presence*; testing for a value would have sent a tagged create to the untagged
+  handler and dropped the tags while answering 201. Its body is decoded strictly, unlike
+  `CreateDistribution`'s: a caller that asked for tags must not be handed an untagged distribution and
+  a success, which is #883's argument applied to the create. Two published behaviors are deliberately
+  absent and documented as such in `docs/services.md`: `OriginAccessControlInUse`/409 needs to know a
+  distribution's origins and substrate records none (#1271 is where that changes), and
+  `OriginAccessControlAlreadyExists`/409 is published for a control "with the specified parameters"
+  without publishing which parameters, and a control carries no `CallerReference` to key a duplicate
+  on. `UpdateOriginAccessControl` is not implemented.
 
 ### Changed
 
@@ -198,6 +233,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   receives an SQS message, subscribes to an SNS topic and creates an EFS file system with an access
   point — each later request naming what an earlier one minted — now replays with **zero**
   differences and `StateValid` true.
+- **A replayed CloudFront create mints the identifiers its recording minted** (#856). One generator
+  produces every identifier this service publishes — a distribution ID, an invalidation ID, and now an
+  origin access control's ID and its ETag — so the whole service moved with #1277 rather than waiting
+  for its turn in the per-family tiering: adding a fourth `crypto/rand` caller to a list #856 is
+  actively shortening was the wrong direction. `generateCloudFrontID` now takes the request's `IDMint`
+  and has no error to return, the alphabet it maps onto is a named constant the three callers share,
+  and the mapping is byte-for-byte the one it performed before, so an ID recorded by an earlier
+  substrate is still the shape this one mints. A recorded stream that creates a control, reads it back,
+  creates a distribution, invalidates inside it and deletes the control with the ETag the create handed
+  out now replays with **zero** differences and `StateValid` true. That last step is what makes the
+  claim load-bearing rather than cosmetic: a re-minted ETag turns the recorded delete into a
+  `PreconditionFailed` against a control nothing had changed. 28 draw sites remain on `crypto/rand`.
 - **A stream recorded under a seed replays under the same seed** (#1140). Every seedable outcome in
   substrate is written through a control-plane endpoint, and only the AWS path recorded anything — so
   a seed never entered the event stream. A replay opens by resetting the whole `StateManager`, and a

docs/services.md

@@ -2207,7 +2207,9 @@ Three kinds of value stay random, and one more is still migrating:
   CloudFormation's [stack and change-set ARNs](#stack-and-change-set-arns-are-deterministic),
   which predate this rule and are what generalising it was modelled on.
 - EC2, IAM, STS, SQS, SNS, Lambda, EFS, FSx, Transfer, ECS, Step Functions, EventBridge,
-  CloudWatch Logs and Service Quotas identifiers are derived today. The remaining services are
+  CloudWatch Logs, CloudFront and Service Quotas identifiers are derived today. A CloudFront
+  distribution, invalidation and origin access control all draw from one generator, so the three
+  moved together with the origin access control family (#1277). The remaining services are
   migrating one family at a time, tracked on #856; until a service moves, its identifiers are
   still drawn from `crypto/rand` and a replay of a stream creating one of its resources still
   diverges.
@@ -16650,7 +16652,8 @@ Kinesis shard: $0.015 per shard-hour. PUT payload: $0.014 per million 25KB units
 
 | Operation | Notes |
 |-----------|-------|
-| CreateDistribution | Distribution IDs: `E{13-char upper alphanum}` |
+| CreateDistribution | Distribution IDs: `E{13-char upper alphanum}`, derived from the request ID (#1277) |
+| CreateDistributionWithTags | Same path as `CreateDistribution` with `?WithTags`; body is a `<DistributionConfigWithTags>`. A body substrate cannot decode is refused rather than creating an untagged distribution |
 | GetDistribution | |
 | GetDistributionConfig | Answers `DistributionConfig` members only, and two of its five required ones — see [A configuration is not a distribution](#a-configuration-is-not-a-distribution) |
 | UpdateDistribution | Shares the `/config` path with `GetDistributionConfig`, told apart by the verb |
@@ -16662,6 +16665,10 @@ Kinesis shard: $0.015 per shard-hour. PUT payload: $0.014 per million 25KB units
 | TagResource | Body is a `<Tags>` document; a body of another shape is refused rather than read as an empty tag set (#883) |
 | UntagResource | Body is a `<TagKeys><Items><Key>` document. Removing a key the distribution does not carry succeeds — AWS documents no error for it, so that reading is substrate's (#883) |
 | ListTagsForResource | Reports the `<Tags><Items>` members sorted by key — see [A tag set read back out of a map](#a-tag-set-read-back-out-of-a-map) |
+| CreateOriginAccessControl | 201 with the `ETag` and `Location` headers; the four required config members are validated against their published enums — see [The origin access control family](#the-origin-access-control-family) |
+| GetOriginAccessControl | 200 with the `ETag` header; absent → `NoSuchOriginAccessControl` |
+| ListOriginAccessControls | An account using no origin access controls answers **no `Items` element** |
+| DeleteOriginAccessControl | 204. `If-Match` required: missing or malformed → `InvalidIfMatchVersion`, well-formed but stale → `PreconditionFailed`. `OriginAccessControlInUse` is not answered — see below |
 
 All three tagging operations share the `POST`/`GET /2020-05-31/tagging` path and are told apart
 by the query string: `Operation=Tag`, `Operation=Untag`, and a `GET` carrying only `Resource`. A
@@ -16678,6 +16685,55 @@ for why `GetResources` reports a distribution in `us-east-1` alone.
 
 All CloudFront resources are stored under `us-east-1` (global service).
 
+### The origin access control family
+
+An origin access control is what lets a distribution read a private S3 bucket: CloudFront signs the
+origin request with SigV4 and the bucket policy trusts the distribution rather than the world. The
+four operations live on their own path — `POST`/`GET /2020-05-31/origin-access-control` for the
+collection, `GET`/`DELETE …/origin-access-control/{Id}` for one member — and substrate routed none
+of it before #1277, so a consumer keeping its bucket private could not run against the emulator at
+all: the create reached the unknown-route refusal.
+
+`OriginAccessControlConfig` marks four members `Required: Yes`, and three of the four publish a
+closed set of values: `OriginAccessControlOriginType` is `s3|mediastore|mediapackagev2|lambda`,
+`SigningBehavior` is `never|always|no-override`, and `SigningProtocol` is `sigv4`. A missing member
+or a value outside its enum is `InvalidArgument`/400, which is the code the operation publishes for
+both. The comparison is case-sensitive: accepting `S3` would let a request through substrate that
+AWS refuses, which is the direction a consumer pays for with a failed live deploy.
+
+The control carries an **ETag**, and it is the one version substrate models on this service. The
+create answers it as a header alongside `Location`, the get answers it as a header, and
+`DeleteOriginAccessControl` requires it in `If-Match`, and the published description of
+`InvalidIfMatchVersion` — *"The If-Match version is missing or not valid"* — is two cases in one
+sentence: a **missing** header and a value that is **not a version substrate could have issued** are
+both `InvalidIfMatchVersion`/400, while a well-formed version that is not the current one is
+`PreconditionFailed`/412. Three published codes for three different mistakes, so a caller that sent
+no version at all is not told its version was stale.
+
+The `ETag` *shape* is substrate's: AWS publishes only that the value identifies the current version,
+so substrate mints the same `E`-prefixed form it mints IDs in, from the same per-request mint, which
+is what makes a replayed create hand out the version its recording did. That is also what makes
+"malformed" decidable at all — a value outside that shape was never handed out here, so it cannot be
+a stale one — and it is a statement about substrate's own minting rather than a claim about what
+CloudFront accepts. A quoted `If-Match` is accepted as well as a bare one, since HTTP ETags are
+conventionally quoted and CloudFront's are not.
+
+`ListOriginAccessControls` answers the whole list, and an account using none answers **no `Items`
+element at all** rather than an empty one — the page states exactly that, and it is the difference
+between a caller's `len(Items) == 0` and a decode that has nothing to decode. `Marker`, `MaxItems`
+and `NextMarker` are published members and are not rendered, for the reason `ListDistributions`
+omits them: substrate holds no value for a page boundary it never draws.
+
+Two published behaviours are deliberately absent. **`OriginAccessControlInUse`/409** on delete needs
+to know that some distribution names the control, and substrate records no origins — that is the gap
+[A configuration is not a distribution](#a-configuration-is-not-a-distribution) describes, and
+[#1271](https://github.com/scttfrdmn/substrate/issues/1271) is where a distribution starts recording
+its configuration and the check becomes answerable. **`OriginAccessControlAlreadyExists`/409** on
+create is published for a control "with the specified parameters", which parameters is not published,
+and a control carries no `CallerReference` to key a duplicate on the way a distribution does; a
+repeated create mints a second control, so a consumer converging by name should list first.
+`UpdateOriginAccessControl` is not implemented.
+
 ### A configuration is not a distribution
 
 `GetDistribution` returns a `Distribution` and `GetDistributionConfig` returns a
@@ -16701,6 +16757,11 @@ shape from. An `Origins` needs `Items` and a `Quantity`; a `DefaultCacheBehavior
 subtree. Omitting a member substrate holds no value for is the honest answer; inventing one would
 assert a shape AWS has not published.
 
+The unrecorded `Origins` is also what defers `OriginAccessControlInUse`/409: an origin access
+control is referenced *from* an origin, so until a distribution records the origins it was created
+with, a delete cannot tell an unused control from one a live distribution depends on. Both halves
+land together in [#1271](https://github.com/scttfrdmn/substrate/issues/1271).
+
 `API_GetDistributionConfig` publishes, on its `Id` parameter: *"The distribution's ID. If the ID is
 empty, an empty distribution configuration is returned."* An empty ID is reachable — the path
 `/2020-05-31/distribution//config` routes to the operation with an empty ID — and substrate answers

docs/testing-guide.md

@@ -336,8 +336,8 @@ all; before #856 every such stream diverged on its first create, which is why th
 tests above are built on caller-chosen bucket and key names instead.
 
 Two caveats. **Not every service's identifiers are derived yet.** EC2, IAM, STS, SQS, SNS,
-Lambda, EFS, FSx, Transfer, ECS, Step Functions, EventBridge, CloudWatch Logs and Service
-Quotas are; the rest are migrating one family at a time, and until a service moves, a
+Lambda, EFS, FSx, Transfer, ECS, Step Functions, EventBridge, CloudWatch Logs, CloudFront and
+Service Quotas are; the rest are migrating one family at a time, and until a service moves, a
 replay of a stream creating one of its resources still diverges. And a recording made against an
 **unfrozen** clock can still diverge on a `state_hash_after` even when every identifier
 matches, because a handler reading the live clock stamps its record a few hundred

emulator/cfn_resources_v23.go

@@ -123,7 +123,10 @@ const cfnOAIIDLen = 13
 // cfnGeneratedName records for its own suffix: UpdateStack in substrate re-deploys the whole
 // template, so an ID minted from crypto/rand would change on every update and leak the identity
 // it replaced. It is derived rather than reused from generateCloudFrontID
-// (cloudfront_plugin.go), which produces exactly this shape but reads crypto/rand.
+// (cloudfront_plugin.go), which produces exactly this shape but draws from the *request's* mint:
+// that makes a replayed CreateDistribution reproduce its ID (#856), and says nothing about an
+// UpdateStack in the same run, which is a second request with a second request id and so a second
+// mint. What this ID has to be stable across is redeployment, not replay.
 //
 // The two obvious deterministic helpers do not fit. cfnGeneratedName returns a hyphenated
 // {stack}-{logical}-{suffix}, which is not this shape. cfnNameSuffix is pinned to twelve base-36