diff --git a/CHANGELOG.md b/CHANGELOG.md index 17a65b7e..3e59639c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -294,6 +294,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 route, integration or mapping ID — now looks like one AWS would issue. This is the one identifier whose *alphabet* changed when it was derived, and it widened rather than narrowed. 23 draw sites remain on `crypto/rand`. +- **The identity, key and certificate family of draw sites is derived** (#856). Eight generators + across seven services move onto `IDMint`: a Cognito user-pool ID, user-pool client ID and client + secret, a Cognito identity-pool ID and identity ID, an IAM Identity Center permission-set ID, a KMS + key ID, an ACM certificate ID, a Secrets Manager version ID, a WAFv2 web ACL and IP set ID, and an + API Gateway API key's ID and value. Every rendering is byte-for-byte the shape the `crypto/rand` + version produced — the width, the case and the alphabet — so an identifier a previous substrate + recorded is still the shape this one mints; nothing in any of the seven API models distinguishes the + two renderings of a UUID-shaped value, which is why #671's "only what the API model states" leaves + them alone. 15 draw sites remain on `crypto/rand`. +- **A replayed secret version ID answers the read it recorded** (#856). This is the quietest way an + identifier can break a replay, and the reason the family's replay test records the pair: substrate + reports an unknown `VersionId` to `GetSecretValue` by *omitting* `SecretString` rather than by + refusing, so before this a replay answered a recorded read with a **200 that had silently lost a + member**. A version ID is a key a caller hands back, not a bare identifier, which is what puts it + above the rest of the family. Reverting the minter to confirm the assertion is not vacuous produces + 21 differences, of which that lost member is one. +- **A replayed WAFv2 `LockToken` still unlocks the update it recorded** (#856). The optimistic-locking + contract requires a caller to hand the token back to `UpdateWebACL`, so a replay that re-minted it + answered the recorded update with `WAFOptimisticLockException` instead of the recorded success. One + minter serves the web ACL ID, the IP set ID and both lock tokens because API_WebACLSummary publishes + the identical 1–36 character `^[0-9a-f]{8}-(?:[0-9a-f]{4}-){3}[0-9a-f]{12}$` constraint on each. +- **An API Gateway API key is no longer minted by ACM's certificate generator** (#856). `CreateApiKey` + reached into `generateACMCertID` for both UUID-shaped strings it returns, so a change to ACM's + rendering would have silently moved API Gateway's. The two now have their own minters and the `id` + and `value` draw in turn, so they differ. `generateACMCertID` no longer returns an error either — + the mint cannot fail where `rand.Read` could, so the caller's dead error branch went with it. +- **A KMS `GenerateDataKey` response is reproducible in both of its members** (#856). The stub data key + is not an identifier, and it is in scope because a caller observes it twice over in one response: the + bytes are the `Plaintext`, and the same bytes are wrapped into the `CiphertextBlob`. A recorded + `Decrypt` was never affected — substrate's ciphertext carries its own plaintext, so a replayed + decrypt answers from the recorded blob regardless — so the break was in the create, not the read. - **A stream recorded under a seed replays under the same seed** (#1140). Every seedable outcome in substrate is written through a control-plane endpoint, and only the AWS path recorded anything — so a seed never entered the event stream. A replay opens by resetting the whole `StateManager`, and a diff --git a/docs/services.md b/docs/services.md index 256eb3bd..1828d257 100644 --- a/docs/services.md +++ b/docs/services.md @@ -2208,11 +2208,12 @@ Three kinds of value stay random, and one more is still migrating: which predate this rule and are what generalising it was modelled on. - EC2, IAM, STS, SQS, SNS, Lambda, EFS, FSx, Transfer, ECS, Step Functions, EventBridge, CloudWatch Logs, CloudFront, Service Quotas, API Gateway (v1 and v2), AppSync, Batch, EMR - Serverless, ECR, ELB and Route 53 identifiers are derived today. A CloudFront distribution, - invalidation and origin access control all draw from one generator, so the three moved together - with the origin access control family (#1277). The remaining services are migrating one family - at a time, tracked on #856; until a service moves, its identifiers are still drawn from - `crypto/rand` and a replay of a stream creating one of its resources still diverges. + Serverless, ECR, ELB, Route 53, Cognito (both the user-pool and the identity-pool API), IAM + Identity Center, KMS, ACM, Secrets Manager and WAFv2 identifiers are derived today. A CloudFront + distribution, invalidation and origin access control all draw from one generator, so the three + moved together with the origin access control family (#1277). The remaining services are + migrating one family at a time, tracked on #856; until a service moves, its identifiers are still + drawn from `crypto/rand` and a replay of a stream creating one of its resources still diverges. Nine of those services publish an identifier from one shared generator rather than declaring their own, so they moved together: an ECS task ID, a Step Functions execution name, an SQS message ID, @@ -2229,6 +2230,23 @@ digit never did. Drawing a byte per character reaches the whole published set, s resource ID, deployment ID, authorizer ID, usage-plan ID or API Gateway v2 route, integration and mapping ID now looks like one AWS would issue. +**Not every derived value is an identifier.** Three kinds of minted string are in scope because a +caller hands them *back*, so a re-minted one turns a recorded success into a refusal or a silently +incomplete response. A WAFv2 `LockToken` is the clearest: the optimistic-locking contract requires a +caller to return the token to `UpdateWebACL`, so a replay that re-minted it would answer the recorded +update with `WAFOptimisticLockException` instead of the recorded success. A Secrets Manager version +ID is the quietest: `GetSecretValue` reports an unknown `VersionId` by *omitting* `SecretString` +rather than refusing, so a re-minted version ID produces a 200 that has lost a member. And a KMS +`GenerateDataKey` stub data key is observed twice in one response — as `Plaintext` and wrapped inside +`CiphertextBlob` — so both members of a recorded response depend on it. (A recorded `Decrypt` does +not: substrate's ciphertext carries its plaintext, so a replayed decrypt answers from the recorded +blob either way.) + +**One service used to mint another's identifiers.** An API Gateway API key's `id` and `value` were +drawn from ACM's certificate-ID generator, which meant a change to ACM's rendering would silently +move API Gateway's. #856 split them into separate minters; both still render the UUID shape they +always did, because neither API publishes a pattern that would decide the question (#671). + An ECR image digest is minted rather than computed from the manifest, so it is reproducible across a replay but is not the SHA-256 of the image it names, and two pushes of identical manifest bytes store two images where AWS stores one. [#1283](https://github.com/scttfrdmn/substrate/issues/1283) @@ -12532,6 +12550,14 @@ SNS publish: $0.0000005 per message. | UntagResource | Idempotent — an absent key is not an error — but a secret scheduled for deletion is refused even then, see [A deleted secret is scheduled, not removed](#a-deleted-secret-is-scheduled-not-removed) | | RotateSecret | Records the rotation function and schedule and echoes `ClientRequestToken` as `VersionId`; no rotation function is executed, and a secret scheduled for deletion is refused — see [A rotation is configured, not run](#a-rotation-is-configured-not-run) | +A version ID is minted from the request ID (#856), so a replayed `CreateSecret` or `PutSecretValue` +reports the version the recording reported and the recorded `GetSecretValue` naming it still answers +its value. Two things about the value are known-unfaithful and tracked on +[#1285](https://github.com/scttfrdmn/substrate/issues/1285): it is sixteen characters where +`VersionId` publishes a minimum of 32, and `CreateSecret` and `PutSecretValue` mint their own rather +than using the caller's `ClientRequestToken`, which AWS documents as *becoming* the version ID and +builds an idempotency contract on. `RotateSecret` already echoes the token it was sent. + ### A `SecretId` addresses the secret its own ARN names `SecretId` is documented as "the ARN or name of the secret", and the two halves have @@ -16251,7 +16277,7 @@ ECS Fargate vCPU: $0.04048 per vCPU-hour. Memory: $0.004445 per GB-hour. | Operation | Notes | |-----------|-------| -| CreateUserPool | Pool ID format: `{region}_{12-char alphanum}` | +| CreateUserPool | Pool ID format: `{region}_{12-char alphanum}`, derived from the request ID (#856) | | DescribeUserPool | | | UpdateUserPool | Replaces the published configuration and answers an empty body — see below | | DeleteUserPool | | @@ -16324,6 +16350,15 @@ the tag set outright like every other published member. Note that `DescribeUserP set as `Tags` rather than the published `UserPoolTags` ([#1136](https://github.com/scttfrdmn/substrate/issues/1136)). +`CreateUserPool`, `DescribeUserPool` and `UpdateUserPool` report the pool's identifier as +`UserPool.UserPoolId`, where `UserPoolType` publishes it as `Id` and carries no `UserPoolId` member at +all ([#1286](https://github.com/scttfrdmn/substrate/issues/1286)). The cause is that the state struct +is also the wire struct, so its storage-side tag reaches the response; `ListUserPools` escapes it only +because it builds a separate summary type, and already renders `Id`. So the two disagree inside one +service, which is what makes it a shape bug rather than a naming preference — an SDK caller reads +`CreateUserPoolOutput.UserPool.Id` as nil and must fall back to `ListUserPools` to learn the ID of the +pool it just created. + ### CloudFormation resource types | Type | Ref | Notes | diff --git a/docs/testing-guide.md b/docs/testing-guide.md index 4c01db6a..5badd347 100644 --- a/docs/testing-guide.md +++ b/docs/testing-guide.md @@ -337,7 +337,8 @@ tests above are built on caller-chosen bucket and key names instead. Two caveats. **Not every service's identifiers are derived yet.** EC2, IAM, STS, SQS, SNS, Lambda, EFS, FSx, Transfer, ECS, Step Functions, EventBridge, CloudWatch Logs, CloudFront, -Service Quotas, API Gateway (v1 and v2), AppSync, Batch, EMR Serverless, ECR, ELB and Route 53 +Service Quotas, API Gateway (v1 and v2), AppSync, Batch, EMR Serverless, ECR, ELB, Route 53, +Cognito (both APIs), IAM Identity Center, KMS, ACM, Secrets Manager and WAFv2 are; the rest are migrating one family at a time, and until a service moves, a replay of a stream creating one of its resources still diverges. And a recording made against an **unfrozen** clock can still diverge on a `state_hash_after` even when every identifier diff --git a/emulator/acm_plugin.go b/emulator/acm_plugin.go index 9ec9fd42..de078523 100644 --- a/emulator/acm_plugin.go +++ b/emulator/acm_plugin.go @@ -2,7 +2,6 @@ package emulator import ( "context" - "crypto/rand" "encoding/json" "fmt" "net/http" @@ -91,10 +90,7 @@ func (p *ACMPlugin) requestCertificate(ctx *RequestContext, req *AWSRequest) (*A keyAlgo = "RSA_2048" } - certID, err := generateACMCertID() - if err != nil { - return nil, fmt.Errorf("acm requestCertificate generateACMCertID: %w", err) - } + certID := generateACMCertID(ctx.IDs) certArn := fmt.Sprintf("arn:aws:acm:%s:%s:certificate/%s", ctx.Region, ctx.AccountID, certID) tags := make(map[string]string, len(body.Tags)) @@ -405,14 +401,20 @@ func (p *ACMPlugin) listTagsForCertificate(ctx *RequestContext, req *AWSRequest) return acmJSONResponse(http.StatusOK, response{Tags: tags}) } -// generateACMCertID generates a UUID-like string for an ACM certificate ID -// using cryptographically random bytes. -func generateACMCertID() (string, error) { - b := make([]byte, 16) - if _, err := rand.Read(b); err != nil { - return "", fmt.Errorf("generateACMCertID rand.Read: %w", err) - } - return fmt.Sprintf("%x-%x-%x-%x-%x", b[0:4], b[4:6], b[6:8], b[8:10], b[10:16]), nil +// generateACMCertID mints an ACM certificate ID from m, in the UUID shape API_RequestCertificate +// writes into the ARN form it documents: +// `arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012`. +// +// [IDMint.HexUUID] rather than [IDMint.UUID]: the crypto/rand form reshaped sixteen raw bytes +// without setting the RFC 4122 version and variant nibbles, and #856 does not change which bytes a +// caller sees. `CertificateArn`'s published pattern ends `[\w+=,.@-]+(/[\w+=,.@-]+)*`, which admits +// either rendering, so nothing in the API model distinguishes them (#671). +// +// It no longer returns an error. The mint cannot fail where rand.Read could, so the caller's error +// branch went with it — and it no longer serves API Gateway's API keys either, which drew from it +// before [generateAPIGatewayAPIKey] existed. +func generateACMCertID(m *IDMint) string { + return m.HexUUID() } // acmJSONResponse marshals v as JSON and returns an AWSResponse with diff --git a/emulator/apigateway_plugin.go b/emulator/apigateway_plugin.go index 1ba8c3dd..fccff38f 100644 --- a/emulator/apigateway_plugin.go +++ b/emulator/apigateway_plugin.go @@ -1162,14 +1162,8 @@ func (p *APIGatewayPlugin) createAPIKey(ctx *RequestContext, req *AWSRequest) (* } } - keyID, err := generateACMCertID() - if err != nil { - return nil, fmt.Errorf("apigateway createApiKey generateID: %w", err) - } - keyValue, err := generateACMCertID() - if err != nil { - return nil, fmt.Errorf("apigateway createApiKey generateValue: %w", err) - } + keyID := generateAPIGatewayAPIKey(ctx.IDs) + keyValue := generateAPIGatewayAPIKey(ctx.IDs) key := APIKeyState{ ID: keyID, @@ -1514,6 +1508,20 @@ func generateAPIGatewayID(m *IDMint) string { return m.Chars(10, apigwIDAlphabet) } +// generateAPIGatewayAPIKey mints one of the two UUID-shaped strings a `CreateApiKey` response +// carries — the key's `id` and its `value` — each drawn from m in turn, so the two differ. +// +// It replaces a call into ACM's certificate-ID generator, which is where `createAPIKey` used to +// reach for a UUID; one service minting another's identifiers meant a change to ACM's rendering +// silently moved API Gateway's, and #856 splits them. +// +// The rendering is [IDMint.HexUUID]'s, unchanged from the crypto/rand form, and nothing in the API +// model asks otherwise: API_ApiKey publishes **no** length constraint and **no** pattern on either +// `id` or `value`, so the shape is substrate's to keep rather than #856's to revisit (#671). +func generateAPIGatewayAPIKey(m *IDMint) string { + return m.HexUUID() +} + // --- Response helper --------------------------------------------------------- // apigwJSONResponse marshals v as JSON and returns an AWSResponse with diff --git a/emulator/cognito_identity_plugin.go b/emulator/cognito_identity_plugin.go index 98a39065..8ad58ca1 100644 --- a/emulator/cognito_identity_plugin.go +++ b/emulator/cognito_identity_plugin.go @@ -2,8 +2,6 @@ package emulator import ( "context" - "crypto/rand" - "encoding/hex" "encoding/json" "fmt" "net/http" @@ -84,7 +82,7 @@ func (p *CognitoIdentityPlugin) createIdentityPool(ctx *RequestContext, req *AWS return nil, &AWSError{Code: "InvalidParameterException", Message: "IdentityPoolName is required", HTTPStatus: http.StatusBadRequest} } - poolID := generateIdentityPoolID(ctx.Region) + poolID := generateIdentityPoolID(ctx.IDs, ctx.Region) now := p.tc.Now() pool := CognitoIdentityPool{ IdentityPoolID: poolID, @@ -224,7 +222,7 @@ func (p *CognitoIdentityPlugin) listIdentityPools(ctx *RequestContext, req *AWSR // --- Identity operations ----------------------------------------------------- func (p *CognitoIdentityPlugin) getID(ctx *RequestContext, _ *AWSRequest) (*AWSResponse, error) { - identityID := generateIdentityPoolID(ctx.Region) + identityID := generateIdentityPoolID(ctx.IDs, ctx.Region) type response struct { IdentityID string `json:"IdentityId"` } @@ -242,7 +240,7 @@ func (p *CognitoIdentityPlugin) getCredentialsForIdentity(ctx *RequestContext, r } identityID := body.IdentityID if identityID == "" { - identityID = generateIdentityPoolID(ctx.Region) + identityID = generateIdentityPoolID(ctx.IDs, ctx.Region) } expiration := p.tc.Now().Add(time.Hour) @@ -345,25 +343,17 @@ func (p *CognitoIdentityPlugin) loadIdentityPool(ctx *RequestContext, poolID str return &pool, nil } -// generateIdentityPoolID returns a Cognito Identity Pool ID of the form -// {region}:{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx} using crypto/rand. -func generateIdentityPoolID(region string) string { - return region + ":" + generateIdentityUUID() -} - -// generateIdentityUUID generates a lowercase hex UUID in the standard -// xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx format using crypto/rand. -func generateIdentityUUID() string { - b := make([]byte, 16) - _, _ = rand.Read(b) - // Set version 4 and variant bits. - b[6] = (b[6] & 0x0f) | 0x40 - b[8] = (b[8] & 0x3f) | 0x80 - return hex.EncodeToString(b[0:4]) + "-" + - hex.EncodeToString(b[4:6]) + "-" + - hex.EncodeToString(b[6:8]) + "-" + - hex.EncodeToString(b[8:10]) + "-" + - hex.EncodeToString(b[10:16]) +// generateIdentityPoolID mints a Cognito identity pool ID from m, in the form +// API_CreateIdentityPool publishes for its `IdentityPoolId` response element: "an identity pool ID +// in the format REGION:GUID", 1–55 characters matching `[\w-]+:[0-9a-f-]+`. The pattern's `[0-9a-f]` +// is why the GUID half is lowercase hex and not the uppercase the IDP plugin's IDs use. +// +// [IDMint.UUID] rather than [IDMint.HexUUID], because the generator this replaces set the RFC 4122 +// version and variant bits itself, and UUID is the method that sets them; the rendering is +// unchanged. The same minter serves `GetId`'s identity IDs, which the same page publishes in the +// same form. +func generateIdentityPoolID(m *IDMint, region string) string { + return region + ":" + m.UUID() } // cognitoIdentityJSONResponse serializes v as JSON and returns an AWSResponse with diff --git a/emulator/cognito_idp_plugin.go b/emulator/cognito_idp_plugin.go index 87dc895d..538dc666 100644 --- a/emulator/cognito_idp_plugin.go +++ b/emulator/cognito_idp_plugin.go @@ -2,7 +2,6 @@ package emulator import ( "context" - "crypto/rand" "encoding/json" "fmt" "net/http" @@ -132,7 +131,7 @@ func (p *CognitoIDPPlugin) createUserPool(ctx *RequestContext, req *AWSRequest) return nil, &AWSError{Code: "InvalidParameterException", Message: "PoolName is required", HTTPStatus: http.StatusBadRequest} } - poolID := ctx.Region + "_" + generateCognitoID() + poolID := ctx.Region + "_" + generateCognitoID(ctx.IDs) arn := fmt.Sprintf("arn:aws:cognito-idp:%s:%s:userpool/%s", ctx.Region, ctx.AccountID, poolID) providerName := fmt.Sprintf("cognito-idp.%s.amazonaws.com/%s", ctx.Region, poolID) @@ -362,10 +361,10 @@ func (p *CognitoIDPPlugin) createUserPoolClient(ctx *RequestContext, req *AWSReq return nil, err } - clientID := generateCognitoID() + clientID := generateCognitoID(ctx.IDs) var secret string if body.GenerateSecret { - secret = generateCognitoID() + generateCognitoID() + secret = generateCognitoID(ctx.IDs) + generateCognitoID(ctx.IDs) } now := p.tc.Now() @@ -1023,7 +1022,7 @@ func (p *CognitoIDPPlugin) signUp(ctx *RequestContext, req *AWSRequest) (*AWSRes return nil, &AWSError{Code: "ResourceNotFoundException", Message: "Client not found: " + body.ClientID, HTTPStatus: http.StatusNotFound} } - userSub := generateCognitoID() + userSub := generateCognitoID(ctx.IDs) now := p.tc.Now() attrs := body.UserAttributes attrs = append(attrs, CognitoAttribute{Name: "sub", Value: userSub}) @@ -1223,17 +1222,24 @@ func (p *CognitoIDPPlugin) loadUser(ctx *RequestContext, poolID, username string return &user, nil } -// generateCognitoID generates a 12-character uppercase alphanumeric ID using -// crypto/rand for use as Cognito pool IDs, client IDs, and user sub values. -func generateCognitoID() string { - const chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" - b := make([]byte, 12) - _, _ = rand.Read(b) - out := make([]byte, 12) - for i, ch := range b { - out[i] = chars[int(ch)%len(chars)] - } - return string(out) +// cognitoIDAlphabet is the alphabet substrate draws its 12-character Cognito identifiers from. +// +// The uppercase-alphanumeric choice is substrate's, and it satisfies every constraint AWS +// publishes for the three members it renders: API_UserPoolType's `Id` is 1–55 characters matching +// `[\w-]+_[0-9a-zA-Z]+`, which `{region}_{12 chars}` meets; API_UserPoolClientType's `ClientId` is +// 1–128 matching `[\w+]+`; and its `ClientSecret` is **24–64** matching the same pattern, which is +// why a secret is two of these concatenated and exactly reaches the published minimum. Nothing here +// is a shape #856 needs to revisit, so it does not. +const cognitoIDAlphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" + +// generateCognitoID mints a 12-character uppercase alphanumeric ID from m, for use as the suffix +// of a user pool ID, as a client ID, or as a user's `sub`. +// +// The mapping is one byte per character through [cognitoIDAlphabet], which is byte-for-byte what +// the crypto/rand form did, so an ID a previous substrate recorded is still the shape this one +// mints. A client *secret* is two of these concatenated, which is the one caller that draws twice. +func generateCognitoID(m *IDMint) string { + return m.Chars(12, cognitoIDAlphabet) } // cognitoIDPJSONResponse serializes v as JSON and returns an AWSResponse with the diff --git a/emulator/ec2_types.go b/emulator/ec2_types.go index d912b9cf..d1032595 100644 --- a/emulator/ec2_types.go +++ b/emulator/ec2_types.go @@ -495,7 +495,7 @@ func generateAssociationID(m *IDMint) string { // flag day: a caller moves by taking a mint and calling [IDMint.Hex] with the same width. // EC2's own ids no longer come through here. // -// TODO(#856): 23 draw sites remain on crypto/rand, tiered by service family on the issue; +// TODO(#856): 15 draw sites remain on crypto/rand, tiered by service family on the issue; // delete this function when the last caller moves. func randomHex(n int) string { b := make([]byte, n) diff --git a/emulator/ids.go b/emulator/ids.go index 894b4b7f..83fe0500 100644 --- a/emulator/ids.go +++ b/emulator/ids.go @@ -61,7 +61,7 @@ import ( // already derived from its inputs: a public IP from its instance id, a secret's ARN from its // name, CloudFormation's stack UUIDs from account and region. // -// TODO(#856): 23 draw sites remain on crypto/rand, tiered by service family on the issue. +// TODO(#856): 15 draw sites remain on crypto/rand, tiered by service family on the issue. // IDMint mints the identifiers one request publishes, derived from that request's own id so // that replaying the request mints the same ones. diff --git a/emulator/ids_test.go b/emulator/ids_test.go index 8abfadc4..990689b8 100644 --- a/emulator/ids_test.go +++ b/emulator/ids_test.go @@ -1069,3 +1069,416 @@ func idsECRCall(t *testing.T, ts *emulator.TestServer, op string, body any) []by require.Equal(t, http.StatusOK, resp.StatusCode, "%s: %s", op, out) return out } + +// Tier 4 of #856: the identity, keys and certificates family — Cognito (both APIs), IAM Identity +// Center, KMS, ACM, Secrets Manager, WAFv2 and API Gateway's API keys. +// +// What this tier adds to the property the tiers above assert is the kind of value being derived. +// These services mint things a caller *authenticates or decrypts or locks with* and not only things +// it addresses a resource by: a Cognito client secret, a KMS data key, a WAFv2 lock token, a secret +// version id. ids.go's file comment argues why those belong in the mint, and the stream below is the +// assertion that a recorded one replays as itself. + +// TestIDs_OneCreateUserPoolClientMintsAClientIDAndASecret is the ordinal's test for this tier, on +// the one request in the tree that draws three times: the client id, and the two halves a client +// secret is concatenated from. +// +// A mint that ignored its ordinal would answer one value three times over, so a client id would be +// the first half of its own secret — and the secret would be that half twice. +func TestIDs_OneCreateUserPoolClientMintsAClientIDAndASecret(t *testing.T) { + t.Parallel() + ts := emulator.StartTestServer(t) + ts.FreezeTime() + + poolID := idsCognitoUserPool(t, ts, "ids-tier4") + + var created struct { + Client struct { + ClientID string `json:"ClientId"` + ClientSecret string `json:"ClientSecret"` + } `json:"UserPoolClient"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsCognitoIDPHost, + "AWSCognitoIdentityProviderService.CreateUserPoolClient", map[string]any{ + "UserPoolId": poolID, "ClientName": "ids-tier4", "GenerateSecret": true, + }), &created)) + + id, secret := created.Client.ClientID, created.Client.ClientSecret + require.Len(t, id, 12, "a Cognito client id is twelve characters") + require.Len(t, secret, 24, + "and a client secret is two of them, which is exactly the published 24-character minimum") + + assert.NotEqual(t, id, secret[:12], "a client id is not the first half of its own secret") + assert.NotEqual(t, secret[:12], secret[12:], "and a secret's two halves are two draws") +} + +// TestReplay_TheIdentityAndKeyFamiliesReplayWithTheIdentifiersTheyMinted is the wire-level +// assertion for tier 4, and the same claim the three tiers above make for their families: a stream +// whose later requests *name* what its earlier ones minted replays with no differences and reaches +// the recorded state. +// +// Every service contributes a create followed by at least one request that can only succeed against +// what that create minted — a user pool client on a pool id, a sign-up on a client id, credentials +// on an identity id, a permission set on a lazily-minted instance ARN, a Decrypt of a data key's +// ciphertext, tags on a certificate ARN, a GetSecretValue naming a version id, an UpdateWebACL +// holding a lock token, and a GetApiKey naming a key id. A re-minted value breaks one of those +// rather than merely rewording it: a refusal (WAFOptimisticLockException for the ACL, a not-found +// for most), or — see [idsRecordSecretsManager] — a 200 that has quietly lost a member. +func TestReplay_TheIdentityAndKeyFamiliesReplayWithTheIdentifiersTheyMinted(t *testing.T) { + t.Parallel() + ts := emulator.StartTestServer(t, + emulator.WithRecordedBodies(), emulator.WithRecordedStateHashes()) + require.True(t, ts.Store().RecordsStateHashes(), "precondition: state_hash_after is compared") + + idsRecordIdentityAndKeyCreates(t, ts) + + results, err := replayEngineFor(ts, emulator.ReplayConfig{ValidateState: true}). + Replay(t.Context(), replayStreamID) + require.NoError(t, err) + + assert.Positive(t, results.TotalEvents, "the stream has to contain the creates") + assert.Equal(t, results.TotalEvents, results.SuccessEvents, + "every recorded request is re-executed and answers") + assert.Empty(t, results.Differences, + "an identity, key or certificate identifier replays as the one recorded: %s", + replayDifferenceSummary(results)) + assert.True(t, results.StateValid, + "and the state it reaches is the recorded state: %v", results.StateErrors) +} + +// idsRecordIdentityAndKeyCreates records the tier-4 stream, one interlocked group per service. +func idsRecordIdentityAndKeyCreates(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + // Frozen for the reason [idsRecordInterlockedCreates] gives: a replay pins the clock to the + // recorded event's timestamp, so a handler stamping a record off a live clock diverges in the + // state hash for a reason that has nothing to do with an identifier. + ts.FreezeTime() + + idsRecordCognitoIDP(t, ts) + idsRecordCognitoIdentity(t, ts) + idsRecordSSO(t, ts) + idsRecordKMS(t, ts) + idsRecordACM(t, ts) + idsRecordSecretsManager(t, ts) + idsRecordWAFv2(t, ts) + idsRecordAPIGatewayAPIKey(t, ts) +} + +// idsRecordCognitoIDP records a user pool, a client with a secret on it, a sign-up against that +// client id, and two reads naming what was minted. +func idsRecordCognitoIDP(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + poolID := idsCognitoUserPool(t, ts, "ids-tier4-pool") + + var created struct { + Client struct { + ClientID string `json:"ClientId"` + ClientSecret string `json:"ClientSecret"` + } `json:"UserPoolClient"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsCognitoIDPHost, + "AWSCognitoIdentityProviderService.CreateUserPoolClient", map[string]any{ + "UserPoolId": poolID, "ClientName": "ids-tier4", "GenerateSecret": true, + }), &created)) + clientID := created.Client.ClientID + require.NotEmpty(t, clientID) + require.NotEmpty(t, created.Client.ClientSecret) + + // SignUp is addressed by the client id alone — the handler finds the pool through it — and + // mints the user's `sub`, so it is both a read of one minted value and a draw of another. + idsJSONTargetCall(t, ts, idsCognitoIDPHost, "AWSCognitoIdentityProviderService.SignUp", + map[string]any{"ClientId": clientID, "Username": "ids-user", "Password": "Passw0rd!"}) + + idsJSONTargetCall(t, ts, idsCognitoIDPHost, + "AWSCognitoIdentityProviderService.DescribeUserPoolClient", + map[string]any{"UserPoolId": poolID, "ClientId": clientID}) + idsJSONTargetCall(t, ts, idsCognitoIDPHost, + "AWSCognitoIdentityProviderService.AdminGetUser", + map[string]any{"UserPoolId": poolID, "Username": "ids-user"}) +} + +// idsCognitoUserPool creates one user pool and returns the `{region}_{12 chars}` id it minted. +// +// The member is read as `UserPoolId` and not as the `Id` that API_UserPoolType publishes, which is +// substrate's own wire shape rather than AWS's — a fidelity gap this tier found and #1286 tracks. +// Reading the member substrate actually sends is what keeps this a test about the mint. +func idsCognitoUserPool(t *testing.T, ts *emulator.TestServer, name string) string { + t.Helper() + + var created struct { + Pool struct { + ID string `json:"UserPoolId"` + } `json:"UserPool"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsCognitoIDPHost, + "AWSCognitoIdentityProviderService.CreateUserPool", + map[string]any{"PoolName": name}), &created)) + + region, suffix, ok := strings.Cut(created.Pool.ID, "_") + require.True(t, ok, "a user pool id is {region}_{suffix}, got %q", created.Pool.ID) + require.NotEmpty(t, region) + require.Len(t, suffix, 12, "the minted half of a user pool id is twelve characters") + return created.Pool.ID +} + +// idsRecordCognitoIdentity records an identity pool, a read of it, and a GetId followed by +// credentials for the identity id that GetId minted. +func idsRecordCognitoIdentity(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + var pool struct { + IdentityPoolID string `json:"IdentityPoolId"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsCognitoIdentityHost, + "AWSCognitoIdentityService.CreateIdentityPool", map[string]any{ + "IdentityPoolName": "ids-tier4", "AllowUnauthenticatedIdentities": true, + }), &pool)) + require.NotEmpty(t, pool.IdentityPoolID) + + idsJSONTargetCall(t, ts, idsCognitoIdentityHost, + "AWSCognitoIdentityService.DescribeIdentityPool", + map[string]any{"IdentityPoolId": pool.IdentityPoolID}) + + var got struct { + IdentityID string `json:"IdentityId"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsCognitoIdentityHost, + "AWSCognitoIdentityService.GetId", + map[string]any{"IdentityPoolId": pool.IdentityPoolID}), &got)) + require.NotEmpty(t, got.IdentityID) + + idsJSONTargetCall(t, ts, idsCognitoIdentityHost, + "AWSCognitoIdentityService.GetCredentialsForIdentity", + map[string]any{"IdentityId": got.IdentityID}) +} + +// idsRecordSSO records the lazily-minted instance, a permission set under it, an account assignment +// naming that permission set, and a listing naming both. +// +// ListInstances is the request that *creates* the instance, which is the laziness sso_types.go's +// preamble records: the instance ARN and its identity store id belong to the ordinal stream of a +// read. +func idsRecordSSO(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + var instances struct { + Instances []struct { + InstanceArn string `json:"InstanceArn"` + IdentityStoreID string `json:"IdentityStoreId"` + } `json:"Instances"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsSSOHost, + "SWBExternalService.ListInstances", map[string]any{}), &instances)) + require.Len(t, instances.Instances, 1) + instanceArn := instances.Instances[0].InstanceArn + require.NotEmpty(t, instanceArn) + require.NotEmpty(t, instances.Instances[0].IdentityStoreID) + + var permSet struct { + PermissionSet struct { + PermissionSetArn string `json:"PermissionSetArn"` + } `json:"PermissionSet"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsSSOHost, + "SWBExternalService.CreatePermissionSet", map[string]any{ + "InstanceArn": instanceArn, "Name": "ids-tier4", + }), &permSet)) + permSetArn := permSet.PermissionSet.PermissionSetArn + require.NotEmpty(t, permSetArn) + require.Contains(t, permSetArn, instanceArn, + "a permission set ARN is a child of the instance ARN, so one mint carries the other") + + idsJSONTargetCall(t, ts, idsSSOHost, "SWBExternalService.DescribePermissionSet", + map[string]any{"InstanceArn": instanceArn, "PermissionSetArn": permSetArn}) + idsJSONTargetCall(t, ts, idsSSOHost, "SWBExternalService.CreateAccountAssignment", + map[string]any{ + "InstanceArn": instanceArn, "PermissionSetArn": permSetArn, + "TargetId": "123456789012", "TargetType": "AWS_ACCOUNT", + "PrincipalType": "USER", "PrincipalId": "ids-principal", + }) + idsJSONTargetCall(t, ts, idsSSOHost, "SWBExternalService.ListAccountAssignments", + map[string]any{ + "InstanceArn": instanceArn, "PermissionSetArn": permSetArn, + "AccountId": "123456789012", + }) +} + +// idsRecordKMS records a key, a data key wrapped by it, and a Decrypt of the ciphertext that +// GenerateDataKey answered with. +// +// The data key is the one value in this tier that is not an identifier: it is reported as the +// response's Plaintext and wrapped into its CiphertextBlob, so a re-minted key changes both members +// of the recorded response. [kmsStubDataKey]'s comment records why that puts it in #856's scope. +func idsRecordKMS(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + var key struct { + Metadata struct { + KeyID string `json:"KeyId"` + } `json:"KeyMetadata"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsKMSHost, + "TrentService.CreateKey", map[string]any{"Description": "ids-tier4"}), &key)) + keyID := key.Metadata.KeyID + require.NotEmpty(t, keyID) + + var dataKey struct { + CiphertextBlob string `json:"CiphertextBlob"` + Plaintext string `json:"Plaintext"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsKMSHost, + "TrentService.GenerateDataKey", + map[string]any{"KeyId": keyID, "KeySpec": "AES_256"}), &dataKey)) + require.NotEmpty(t, dataKey.CiphertextBlob) + require.NotEmpty(t, dataKey.Plaintext) + + idsJSONTargetCall(t, ts, idsKMSHost, "TrentService.Decrypt", + map[string]any{"CiphertextBlob": dataKey.CiphertextBlob}) + idsJSONTargetCall(t, ts, idsKMSHost, "TrentService.DescribeKey", + map[string]any{"KeyId": keyID}) +} + +// idsRecordACM records a certificate and two requests naming the ARN its id was minted into. +func idsRecordACM(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + var cert struct { + CertificateArn string `json:"CertificateArn"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsACMHost, + "CertificateManager.RequestCertificate", + map[string]any{"DomainName": "ids-tier4.example.com"}), &cert)) + require.NotEmpty(t, cert.CertificateArn) + + idsJSONTargetCall(t, ts, idsACMHost, "CertificateManager.AddTagsToCertificate", + map[string]any{ + "CertificateArn": cert.CertificateArn, + "Tags": []map[string]string{{"Key": "tier", "Value": "4"}}, + }) + idsJSONTargetCall(t, ts, idsACMHost, "CertificateManager.DescribeCertificate", + map[string]any{"CertificateArn": cert.CertificateArn}) +} + +// idsRecordSecretsManager records a secret, a second version of it, and a GetSecretValue naming the +// version id the create minted. +// +// That last request is the quietest interlock in the tier: a re-minted version id does not make it +// fail, it makes it answer 200 with no `SecretString` at all, because substrate reports an unknown +// version by omitting the member. Reverting the minter turns the recorded `"first"` into `""`. +func idsRecordSecretsManager(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + var created struct { + ARN string `json:"ARN"` + VersionID string `json:"VersionId"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsSecretsManagerHost, + "secretsmanager.CreateSecret", + map[string]any{"Name": "ids-tier4", "SecretString": "first"}), &created)) + require.NotEmpty(t, created.ARN) + require.Len(t, created.VersionID, 16, + "a version id is sixteen uppercase hex characters until #1285 widens it") + + var put struct { + VersionID string `json:"VersionId"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsSecretsManagerHost, + "secretsmanager.PutSecretValue", + map[string]any{"SecretId": "ids-tier4", "SecretString": "second"}), &put)) + require.NotEqual(t, created.VersionID, put.VersionID, + "two versions of one secret are two draws") + + idsJSONTargetCall(t, ts, idsSecretsManagerHost, "secretsmanager.GetSecretValue", + map[string]any{"SecretId": "ids-tier4", "VersionId": created.VersionID}) + idsJSONTargetCall(t, ts, idsSecretsManagerHost, "secretsmanager.ListSecretVersionIds", + map[string]any{"SecretId": "ids-tier4"}) +} + +// idsRecordWAFv2 records a web ACL and an update holding the lock token the create answered with, +// which is the tier's one interlock that fails as WAFOptimisticLockException rather than as a +// not-found. +func idsRecordWAFv2(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + var created struct { + Summary struct { + ID string `json:"Id"` + LockToken string `json:"LockToken"` + } `json:"Summary"` + } + require.NoError(t, json.Unmarshal(idsJSONTargetCall(t, ts, idsWAFv2Host, + "AWSWAF_20190729.CreateWebACL", map[string]any{ + "Name": "ids-tier4", "Scope": "REGIONAL", + "DefaultAction": map[string]any{"Allow": map[string]any{}}, + }), &created)) + id, lockToken := created.Summary.ID, created.Summary.LockToken + require.NotEmpty(t, id) + require.NotEmpty(t, lockToken) + require.NotEqual(t, id, lockToken, "an ACL's id and its first lock token are two draws") + + idsJSONTargetCall(t, ts, idsWAFv2Host, "AWSWAF_20190729.UpdateWebACL", map[string]any{ + "Name": "ids-tier4", "Scope": "REGIONAL", "Id": id, "LockToken": lockToken, + "Description": "updated under the recorded lock token", + }) + idsJSONTargetCall(t, ts, idsWAFv2Host, "AWSWAF_20190729.GetWebACL", + map[string]any{"Name": "ids-tier4", "Scope": "REGIONAL", "Id": id}) +} + +// idsRecordAPIGatewayAPIKey records an API key and a read naming its id. +// +// The key is path-routed rather than target-routed, and it is in this tier rather than tier 3 +// because its two values came from ACM's certificate-ID generator until [generateAPIGatewayAPIKey] +// existed — one service minting another's identifiers. +func idsRecordAPIGatewayAPIKey(t *testing.T, ts *emulator.TestServer) { + t.Helper() + + var key struct { + ID string `json:"id"` + Value string `json:"value"` + } + require.NoError(t, json.Unmarshal(idsRESTCall(t, ts, idsAPIGatewayHost, http.MethodPost, + "/apikeys", map[string]any{"name": "ids-tier4", "enabled": true}), &key)) + require.NotEmpty(t, key.ID) + require.NotEmpty(t, key.Value) + require.NotEqual(t, key.ID, key.Value, + "one CreateApiKey draws the key's id and its value separately") + + idsRESTCall(t, ts, idsAPIGatewayHost, http.MethodGet, "/apikeys/"+key.ID, nil) +} + +// The hosts the tier-4 services are addressed at. +const ( + idsCognitoIDPHost = "cognito-idp.us-east-1.amazonaws.com" + idsCognitoIdentityHost = "cognito-identity.us-east-1.amazonaws.com" + idsSSOHost = "sso.us-east-1.amazonaws.com" + idsKMSHost = "kms.us-east-1.amazonaws.com" + idsACMHost = "acm.us-east-1.amazonaws.com" + idsSecretsManagerHost = "secretsmanager.us-east-1.amazonaws.com" + idsWAFv2Host = "wafv2.us-east-1.amazonaws.com" +) + +// idsJSONTargetCall issues one JSON-1.1 request routed by X-Amz-Target and requires a 2xx. target +// is the whole header value, prefix included, because the seven services in this tier publish seven +// different prefixes. +func idsJSONTargetCall(t *testing.T, ts *emulator.TestServer, host, target string, body any) []byte { + t.Helper() + + raw, err := json.Marshal(body) + require.NoError(t, err) + req, err := http.NewRequestWithContext(t.Context(), http.MethodPost, ts.URL+"/", + strings.NewReader(string(raw))) + require.NoError(t, err) + req.Host = host + req.Header.Set("Content-Type", "application/x-amz-json-1.1") + req.Header.Set("X-Amz-Target", target) + + resp, err := http.DefaultClient.Do(req) + require.NoError(t, err) + out, err := io.ReadAll(resp.Body) + require.NoError(t, err) + require.NoError(t, resp.Body.Close()) + require.Less(t, resp.StatusCode, 300, "%s %s: %s", host, target, out) + return out +} diff --git a/emulator/kms_plugin.go b/emulator/kms_plugin.go index 37ed0445..6369456a 100644 --- a/emulator/kms_plugin.go +++ b/emulator/kms_plugin.go @@ -364,7 +364,7 @@ func (p *KMSPlugin) createKey(ctx *RequestContext, req *AWSRequest) (*AWSRespons } } - keyID := generateKMSKeyID() + keyID := generateKMSKeyID(ctx.IDs) arn := kmsKeyARN(ctx.Region, ctx.AccountID, keyID) key := &KMSKey{ KeyID: keyID, @@ -1791,7 +1791,7 @@ func (p *KMSPlugin) generateDataKey(ctx *RequestContext, req *AWSRequest) (*AWSR } // Generate a stub 32-byte data key. - dataKeyHex := randomHex(16) + dataKeyHex := kmsStubDataKey(ctx.IDs) plaintextB64 := base64.StdEncoding.EncodeToString([]byte(dataKeyHex)) ciphertext := kmsEncryptStub(key, kmsSymmetricDefaultAlgorithm, input.EncryptionContext, []byte(dataKeyHex)) @@ -1860,7 +1860,7 @@ func (p *KMSPlugin) generateDataKeyWithoutPlaintext(ctx *RequestContext, req *AW return nil, stateErr } - dataKeyHex := randomHex(16) + dataKeyHex := kmsStubDataKey(ctx.IDs) ciphertext := kmsEncryptStub(key, kmsSymmetricDefaultAlgorithm, input.EncryptionContext, []byte(dataKeyHex)) out := map[string]interface{}{ diff --git a/emulator/kms_types.go b/emulator/kms_types.go index 9b0a49f0..2c576dc6 100644 --- a/emulator/kms_types.go +++ b/emulator/kms_types.go @@ -225,10 +225,39 @@ type KMSTag struct { TagValue string `json:"TagValue"` } -// generateKMSKeyID generates a UUID-like KMS key ID. -func generateKMSKeyID() string { - h := randomHex(32) - return fmt.Sprintf("%s-%s-%s-%s-%s", h[0:8], h[8:12], h[12:16], h[16:20], h[20:32]) +// generateKMSKeyID mints a KMS key ID from m, in the UUID shape AWS's own examples write +// (`1234abcd-12ab-34cd-56ef-1234567890ab`). +// +// [IDMint.HexUUID] rather than [IDMint.UUID]: the crypto/rand form reshaped 32 hex characters +// without setting the RFC 4122 version and variant nibbles, and #856 does not change which bytes a +// caller sees. API_KeyMetadata's `KeyId` is "the globally unique identifier for the KMS key" at a +// length of 1–2048 with **no pattern**, so nothing in the API model distinguishes the two +// renderings — which is also why the shape is not something to revisit here (#671). +func generateKMSKeyID(m *IDMint) string { + return m.HexUUID() +} + +// kmsStubDataKey mints the stub a `GenerateDataKey` response's plaintext and ciphertext are both +// built from — 32 lowercase hex characters, which is what substrate calls the data key's bytes. +// +// Shared by GenerateDataKey and GenerateDataKeyWithoutPlaintext so the two cannot diverge, which is +// the same reason [kmsDataKeyKeySpecError] is shared. +// +// Deriving it is #856's rule applied to a value that is *not* an identifier, and the reason it is in +// scope is that the caller observes it twice over: the bytes are the response's `Plaintext`, and the +// same bytes are wrapped into its `CiphertextBlob`, so a re-minted key makes both members of a +// recorded response unreproducible. (A recorded `Decrypt` still answers the recorded plaintext +// either way — [kmsEncryptStub] carries the plaintext inside the blob, so what a replayed Decrypt +// reports comes from the recorded ciphertext rather than from a fresh draw. The break is in the +// GenerateDataKey response itself.) +// +// STS secret access keys and session tokens are already derived for the same reason, and the +// argument is the one ids.go's file comment makes — a test emulator's stub data key protects +// nothing, and a caller who needs an unguessable key from a test double is asking the wrong tool. +// EC2 key-pair material stays random because it needs a deterministic reader into a key generator +// rather than a string; this needs only a string. +func kmsStubDataKey(m *IDMint) string { + return m.Hex(16) } // kmsKeyMaterialID mints the identifier of a key's key material from the key's own ARN. diff --git a/emulator/secretsmanager_plugin.go b/emulator/secretsmanager_plugin.go index ac9975ed..2bdcab72 100644 --- a/emulator/secretsmanager_plugin.go +++ b/emulator/secretsmanager_plugin.go @@ -169,7 +169,7 @@ func (p *SecretsManagerPlugin) createSecret(ctx *RequestContext, req *AWSRequest now := p.tc.Now() arn := generateSecretARN(ctx.Region, ctx.AccountID, input.Name) - versionID := generateVersionID() + versionID := generateVersionID(ctx.IDs) secret := &SecretState{ ARN: arn, @@ -300,7 +300,7 @@ func (p *SecretsManagerPlugin) putSecretValue(ctx *RequestContext, req *AWSReque return nil, smSecretScheduledForDeletion(input.SecretID, secret.DeletionDate) } - versionID := generateVersionID() + versionID := generateVersionID(ctx.IDs) value := input.SecretString if value == "" { value = input.SecretBinary @@ -476,7 +476,7 @@ func (p *SecretsManagerPlugin) updateSecret(ctx *RequestContext, req *AWSRequest value = input.SecretBinary } if value != "" { - versionID = generateVersionID() + versionID = generateVersionID(ctx.IDs) if err := p.state.Put(goCtx, secretsManagerNamespace, smSecretVersionStateKey(target.AccountID, target.Region, target.Name, versionID), []byte(value)); err != nil { return nil, fmt.Errorf("sm updateSecret store value: %w", err) } diff --git a/emulator/secretsmanager_types.go b/emulator/secretsmanager_types.go index 17acf603..5f023837 100644 --- a/emulator/secretsmanager_types.go +++ b/emulator/secretsmanager_types.go @@ -122,7 +122,21 @@ func generateSecretARN(region, accountID, name string) string { return fmt.Sprintf("arn:aws:secretsmanager:%s:%s:secret:%s", region, accountID, name) } -// generateVersionID returns a new uppercase hex version ID. -func generateVersionID() string { - return strings.ToUpper(randomHex(8)) +// generateVersionID mints a secret version ID from m — sixteen uppercase hex characters, which is +// byte-for-byte the width and case the crypto/rand form produced, so a version ID a previous +// substrate recorded is still the shape this one mints. +// +// Deriving it matters more than a bare identifier would, because a version ID is a *key*: a caller +// reads a specific version back through `GetSecretValue`'s `VersionId`. A replay that re-minted one +// answers that recorded read with no `SecretString` at all — substrate reports an unknown version by +// omitting the member rather than by refusing — which is the quietest way an identifier can break a +// replay and the reason the tier-4 stream in ids_test.go records exactly that pair. +// +// TODO(#1285): the width is short of what AWS publishes, and the caller's own `ClientRequestToken` +// is ignored. API_PutSecretValue gives `VersionId` a length of 32–64 and states that the request's +// `ClientRequestToken` — itself 32–64 — "becomes the VersionId of the new version". Substrate emits +// sixteen characters and mints its own regardless. Both are fidelity gaps rather than derivation +// ones, so #856 preserves the shape and #1285 changes it. +func generateVersionID(m *IDMint) string { + return strings.ToUpper(m.Hex(8)) } diff --git a/emulator/sso_plugin.go b/emulator/sso_plugin.go index 0e7997ab..a8aeeecb 100644 --- a/emulator/sso_plugin.go +++ b/emulator/sso_plugin.go @@ -72,7 +72,7 @@ func (p *SSOPlugin) HandleRequest(reqCtx *RequestContext, req *AWSRequest) (*AWS // ensureInstance auto-creates the singleton SSO instance for an account on first access, // following the same pattern as ensureOrganization in organizations_plugin.go. -func (p *SSOPlugin) ensureInstance(goCtx context.Context, acct string) (*SSOInstance, error) { +func (p *SSOPlugin) ensureInstance(goCtx context.Context, m *IDMint, acct string) (*SSOInstance, error) { key := ssoInstanceKey(acct) data, err := p.state.Get(goCtx, ssoNamespace, key) if err != nil { @@ -87,8 +87,8 @@ func (p *SSOPlugin) ensureInstance(goCtx context.Context, acct string) (*SSOInst } inst := SSOInstance{ - InstanceArn: generateSSOInstanceArn(), - IdentityStoreID: generateSSOIdentityStoreID(), + InstanceArn: generateSSOInstanceArn(m), + IdentityStoreID: generateSSOIdentityStoreID(m), Status: "ACTIVE", CreatedDate: p.tc.Now(), AccountID: acct, @@ -105,7 +105,7 @@ func (p *SSOPlugin) ensureInstance(goCtx context.Context, acct string) (*SSOInst func (p *SSOPlugin) listInstances(reqCtx *RequestContext, _ *AWSRequest) (*AWSResponse, error) { goCtx := context.Background() - inst, err := p.ensureInstance(goCtx, reqCtx.AccountID) + inst, err := p.ensureInstance(goCtx, reqCtx.IDs, reqCtx.AccountID) if err != nil { return nil, err } @@ -140,7 +140,7 @@ func (p *SSOPlugin) createPermissionSet(reqCtx *RequestContext, req *AWSRequest) } goCtx := context.Background() - inst, err := p.ensureInstance(goCtx, reqCtx.AccountID) + inst, err := p.ensureInstance(goCtx, reqCtx.IDs, reqCtx.AccountID) if err != nil { return nil, err } @@ -149,7 +149,7 @@ func (p *SSOPlugin) createPermissionSet(reqCtx *RequestContext, req *AWSRequest) instanceArn = inst.InstanceArn } - permSetArn := generateSSOPermissionSetArn(instanceArn) + permSetArn := generateSSOPermissionSetArn(reqCtx.IDs, instanceArn) ps := SSOPermissionSet{ PermissionSetArn: permSetArn, Name: input.Name, @@ -385,7 +385,7 @@ func (p *SSOPlugin) createAccountAssignment(reqCtx *RequestContext, req *AWSRequ compositeKey := input.TargetID + "/" + input.PrincipalType + "/" + input.PrincipalID updateStringIndex(goCtx, p.state, ssoNamespace, ssoAssignmentKeysKey(reqCtx.AccountID, input.PermissionSetArn), compositeKey) - requestID := generateSSORequestID() + requestID := generateSSORequestID(reqCtx.IDs) return ssoJSONResponse(http.StatusOK, map[string]interface{}{ "AccountAssignmentCreationStatus": map[string]interface{}{ "Status": "SUCCEEDED", @@ -435,7 +435,7 @@ func (p *SSOPlugin) deleteAccountAssignment(reqCtx *RequestContext, req *AWSRequ compositeKey := input.TargetID + "/" + input.PrincipalType + "/" + input.PrincipalID removeFromStringIndex(goCtx, p.state, ssoNamespace, ssoAssignmentKeysKey(reqCtx.AccountID, input.PermissionSetArn), compositeKey) - requestID := generateSSORequestID() + requestID := generateSSORequestID(reqCtx.IDs) return ssoJSONResponse(http.StatusOK, map[string]interface{}{ "AccountAssignmentDeletionStatus": map[string]interface{}{ "Status": "SUCCEEDED", diff --git a/emulator/sso_types.go b/emulator/sso_types.go index 603840b9..09bd109b 100644 --- a/emulator/sso_types.go +++ b/emulator/sso_types.go @@ -1,8 +1,6 @@ package emulator import ( - "crypto/rand" - "encoding/hex" "time" ) @@ -61,36 +59,39 @@ type SSOAccountAssignment struct { InstanceArn string `json:"InstanceArn"` } -// generateSSOInstanceArn generates a random IAM Identity Center instance ARN. -func generateSSOInstanceArn() string { - b := make([]byte, 13) - _, _ = rand.Read(b) - return "arn:aws:sso:::instance/" + hex.EncodeToString(b)[:26] +// The four SSO minters take the request's [IDMint] as a parameter rather than reading it off a +// [RequestContext], because two of them are reached from [SSOPlugin.ensureInstance], which takes an +// account ID and a Go context and not a request context (#856's established shape for a helper that +// mints without a request in scope — see buildSNSEnvelope). +// +// The instance is minted **lazily, by whichever request touches SSO first**, including a read: +// `ListInstances` creates it as readily as `CreatePermissionSet` does. That is replay-stable rather +// than in spite of the laziness — a replay re-issues the recorded requests in their recorded order, +// so the same request creates the instance and mints the same ARN — but it does mean the instance +// ARN belongs to the ordinal stream of a request that did not ask for one. + +// generateSSOInstanceArn mints an IAM Identity Center instance ARN from m. +// +// Thirteen bytes is 26 hex characters, so the `[:26]` the crypto/rand form applied truncated +// nothing; the width is unchanged. +func generateSSOInstanceArn(m *IDMint) string { + return "arn:aws:sso:::instance/" + m.Hex(13) } -// generateSSOIdentityStoreID generates a random identity store ID. -func generateSSOIdentityStoreID() string { - b := make([]byte, 5) - _, _ = rand.Read(b) - return "d-" + hex.EncodeToString(b)[:10] +// generateSSOIdentityStoreID mints an identity store ID from m, in the `d-`-prefixed form. +func generateSSOIdentityStoreID(m *IDMint) string { + return "d-" + m.Hex(5) } -// generateSSOPermissionSetArn generates an ARN for a permission set. -func generateSSOPermissionSetArn(instanceArn string) string { - b := make([]byte, 8) - _, _ = rand.Read(b) - return instanceArn + "/ps-" + hex.EncodeToString(b) +// generateSSOPermissionSetArn mints a permission set ARN from m, as a child of instanceArn. +func generateSSOPermissionSetArn(m *IDMint, instanceArn string) string { + return instanceArn + "/ps-" + m.Hex(8) } -// generateSSORequestID generates a UUID-style request ID for async SSO operations. -func generateSSORequestID() string { - b := make([]byte, 16) - _, _ = rand.Read(b) - return hex.EncodeToString(b[0:4]) + "-" + - hex.EncodeToString(b[4:6]) + "-" + - hex.EncodeToString(b[6:8]) + "-" + - hex.EncodeToString(b[8:10]) + "-" + - hex.EncodeToString(b[10:16]) +// generateSSORequestID mints a UUID-shaped request ID from m, for the async SSO operations that +// report one. +func generateSSORequestID(m *IDMint) string { + return m.HexUUID() } // State key helpers. diff --git a/emulator/wafv2_plugin.go b/emulator/wafv2_plugin.go index dee8167d..20eba734 100644 --- a/emulator/wafv2_plugin.go +++ b/emulator/wafv2_plugin.go @@ -2,8 +2,6 @@ package emulator import ( "context" - "crypto/rand" - "encoding/hex" "encoding/json" "fmt" "net/http" @@ -119,17 +117,20 @@ func wafv2ARN(region, accountID, scope, resourceType, name, id string) string { region, accountID, strings.ToLower(scope), resourceType, name, id) } -// generateWAFv2Token returns a new random UUID string for use as a LockToken or ID. -func generateWAFv2Token() string { - b := make([]byte, 16) - _, _ = rand.Read(b) - b[6] = (b[6] & 0x0f) | 0x40 - b[8] = (b[8] & 0x3f) | 0x80 - return hex.EncodeToString(b[0:4]) + "-" + - hex.EncodeToString(b[4:6]) + "-" + - hex.EncodeToString(b[6:8]) + "-" + - hex.EncodeToString(b[8:10]) + "-" + - hex.EncodeToString(b[10:16]) +// generateWAFv2Token mints a UUID-shaped string from m, for use as a web ACL or IP set `Id` or as a +// `LockToken`. +// +// API_WebACLSummary publishes the identical constraint on both members — 1–36 characters matching +// `^[0-9a-f]{8}-(?:[0-9a-f]{4}-){3}[0-9a-f]{12}$` — which is why one minter serves both and why the +// rendering is lowercase hex in the 8-4-4-4-12 grouping. The pattern is indifferent to the RFC 4122 +// version and variant nibbles, and [IDMint.UUID] sets them exactly as the crypto/rand form did, so +// a token this mints is the shape a previous substrate recorded. +// +// The optimistic-locking contract is what makes deriving this worth more here than elsewhere: a +// caller must hand a `LockToken` back to `UpdateWebACL`, so a replay that re-minted the token would +// answer the recorded update with WAFOptimisticLockException rather than the recorded success. +func generateWAFv2Token(m *IDMint) string { + return m.UUID() } func (p *WAFv2Plugin) createWebACL(reqCtx *RequestContext, req *AWSRequest) (*AWSResponse, error) { @@ -153,8 +154,8 @@ func (p *WAFv2Plugin) createWebACL(reqCtx *RequestContext, req *AWSRequest) (*AW return nil, err } - id := generateWAFv2Token() - lockToken := generateWAFv2Token() + id := generateWAFv2Token(reqCtx.IDs) + lockToken := generateWAFv2Token(reqCtx.IDs) arn := wafv2ARN(reqCtx.Region, reqCtx.AccountID, input.Scope, "webacl", input.Name, id) acl := WAFv2WebACL{ @@ -284,7 +285,7 @@ func (p *WAFv2Plugin) updateWebACL(reqCtx *RequestContext, req *AWSRequest) (*AW acl.VisibilityConfig = input.VisibilityConfig } // Regenerate lock token. - newToken := generateWAFv2Token() + newToken := generateWAFv2Token(reqCtx.IDs) acl.LockToken = newToken data, err := json.Marshal(acl) @@ -483,8 +484,8 @@ func (p *WAFv2Plugin) createIPSet(reqCtx *RequestContext, req *AWSRequest) (*AWS return nil, err } - id := generateWAFv2Token() - lockToken := generateWAFv2Token() + id := generateWAFv2Token(reqCtx.IDs) + lockToken := generateWAFv2Token(reqCtx.IDs) arn := wafv2ARN(reqCtx.Region, reqCtx.AccountID, input.Scope, "ipset", input.Name, id) ipset := WAFv2IPSet{ @@ -578,7 +579,7 @@ func (p *WAFv2Plugin) updateIPSet(reqCtx *RequestContext, req *AWSRequest) (*AWS if input.Addresses != nil { ipset.Addresses = input.Addresses } - newToken := generateWAFv2Token() + newToken := generateWAFv2Token(reqCtx.IDs) ipset.LockToken = newToken data, err := json.Marshal(ipset)