fix(#856): the shared UUID-shaped identifier, and the messaging/storage family

CHANGELOG.md

@@ -169,9 +169,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   `StateValid` true, twice over and in a second process. `generateRequestID`, which is the seed,
   gained a random suffix: it was the wall clock alone, and every other identifier now derives from
   it, so two requests served within one tick of a coarse clock would have minted the same volume ID.
-  Remaining on `crypto/rand`: 32 draw sites in the other services, migrating one family at a time on
+  Remaining on `crypto/rand`: 29 draw sites in the other services, migrating one family at a time on
   #856; EC2 key-pair material, which needs a deterministic reader into the key generator rather than
   a string; and substrate's own event, snapshot and replay IDs, which no AWS call observes.
+- **The shared UUID-shaped identifier, and the messaging/storage family, are derived too** (#856).
+  Substrate had two shared draw sites, not one. `randomHex` was the known one; the other was a
+  UUID-shaped generator declared in the Lambda plugin that **six other services** publish an
+  identifier from — an ECS task ID, a Step Functions execution name, an SQS message ID, an
+  EventBridge event ID, a CloudWatch Logs upload sequence token and a Service Quotas request ID — so
+  it moved to `IDMint` together with Lambda's own revision IDs rather than waiting for each of those
+  services' turn in the per-family tiering. Its rendering is unchanged: sixteen derived bytes in
+  UUID *shape*, `8-4-4-4-12` lowercase hex without the RFC 4122 version and variant bits, because
+  #856 is about reproducing an identifier across a replay and not about changing which bytes a
+  caller sees. Moving with it: SQS receipt handles, SNS subscription IDs and notification-envelope
+  message IDs, EFS file-system/access-point/mount-target IDs, FSx file-system IDs and Lustre mount
+  names, and Transfer Family server IDs. A send mints a message's initial receipt handle and each
+  receive replaces it, which matches real SQS — two `ReceiveMessage` calls returning one message
+  hand back two handles, and only the most recent deletes — so the handle derives from the mint's
+  ordinal rather than from the message, and each call's handle replays as the one that call
+  recorded. Three helpers that minted
+  without a request context in scope now take the mint as a parameter: `buildSNSEnvelope`,
+  `requestServiceQuotaIncrease` and FSx's Lustre mount-name branch. A recorded stream that sends and
+  receives an SQS message, subscribes to an SNS topic and creates an EFS file system with an access
+  point — each later request naming what an earlier one minted — now replays with **zero**
+  differences and `StateValid` true.
 - **A stream recorded under a seed replays under the same seed** (#1140). Every seedable outcome in
   substrate is written through a control-plane endpoint, and only the AWS path recorded anything — so
   a seed never entered the event stream. A replay opens by resetting the whole `StateManager`, and a

docs/services.md

@@ -2206,9 +2206,22 @@ Three kinds of value stay random, and one more is still migrating:
   ID, a NAT gateway's private IP from its gateway ID, a secret's ARN from its name, and
   CloudFormation's [stack and change-set ARNs](#stack-and-change-set-arns-are-deterministic),
   which predate this rule and are what generalising it was modelled on.
-- EC2, IAM and STS identifiers are derived today. The remaining services are migrating one family
-  at a time, tracked on #856; until a service moves, its identifiers are still drawn from
-  `crypto/rand` and a replay of a stream creating one of its resources still diverges.
+- EC2, IAM, STS, SQS, SNS, Lambda, EFS, FSx, Transfer, ECS, Step Functions, EventBridge,
+  CloudWatch Logs and Service Quotas identifiers are derived today. The remaining services are
+  migrating one family at a time, tracked on #856; until a service moves, its identifiers are
+  still drawn from `crypto/rand` and a replay of a stream creating one of its resources still
+  diverges.
+
+Six of those services publish an identifier from one shared generator rather than declaring their
+own, so they moved together: an ECS task ID, a Step Functions execution name, an SQS message ID,
+an EventBridge event ID, a CloudWatch Logs upload sequence token and a Service Quotas request ID
+are all the same sixteen derived bytes rendered in UUID *shape* — `8-4-4-4-12` lowercase hex
+without the RFC 4122 version and variant bits, which is the form substrate published before it
+derived them and is unchanged by deriving them.
+
+An SQS send mints a message's initial receipt handle and each receive replaces it, matching real
+SQS: two `ReceiveMessage` calls that return the same message hand back different handles and only
+the most recent one deletes. A replay of either call reproduces the handle that call recorded.
 
 ---
 

docs/testing-guide.md

@@ -335,9 +335,10 @@ makes `Differences` empty — and `StateValid` true, with `WithRecordedStateHash
 all; before #856 every such stream diverged on its first create, which is why the #1140
 tests above are built on caller-chosen bucket and key names instead.
 
-Two caveats. **Only EC2, IAM and STS identifiers are derived so far**; the remaining
-services are migrating one family at a time, and until a service moves, a replay of a
-stream creating one of its resources still diverges. And a recording made against an
+Two caveats. **Not every service's identifiers are derived yet.** EC2, IAM, STS, SQS, SNS,
+Lambda, EFS, FSx, Transfer, ECS, Step Functions, EventBridge, CloudWatch Logs and Service
+Quotas are; the rest are migrating one family at a time, and until a service moves, a
+replay of a stream creating one of its resources still diverges. And a recording made against an
 **unfrozen** clock can still diverge on a `state_hash_after` even when every identifier
 matches, because a handler reading the live clock stamps its record a few hundred
 nanoseconds after the event's own timestamp — invisible in a response rendering seconds,

emulator/cloudwatchlogs_plugin.go

@@ -408,7 +408,7 @@ func (p *CloudWatchLogsPlugin) createLogStream(ctx *RequestContext, req *AWSRequ
 		LogStreamName:       body.LogStreamName,
 		ARN:                 cwLogStreamARN(ctx.Region, ctx.AccountID, body.LogGroupName, body.LogStreamName),
 		CreationTime:        now,
-		UploadSequenceToken: generateLambdaRevisionID(),
+		UploadSequenceToken: generateLambdaRevisionID(ctx.IDs),
 	}
 	data, err := json.Marshal(ls)
 	if err != nil {
@@ -596,7 +596,7 @@ func (p *CloudWatchLogsPlugin) putLogEvents(ctx *RequestContext, req *AWSRequest
 	var ls CWLogStream
 	if json.Unmarshal(streamData, &ls) == nil {
 		ls.LastIngestionTime = now
-		ls.UploadSequenceToken = generateLambdaRevisionID()
+		ls.UploadSequenceToken = generateLambdaRevisionID(ctx.IDs)
 		if updated, marshalErr := json.Marshal(ls); marshalErr == nil {
 			_ = p.state.Put(goCtx, cloudwatchLogsNamespace, streamKey, updated)
 		}

emulator/ec2_types.go

@@ -495,7 +495,7 @@ func generateAssociationID(m *IDMint) string {
 // flag day: a caller moves by taking a mint and calling [IDMint.Hex] with the same width.
 // EC2's own ids no longer come through here.
 //
-// TODO(#856): 32 draw sites remain on crypto/rand, tiered by service family on the issue;
+// TODO(#856): 29 draw sites remain on crypto/rand, tiered by service family on the issue;
 // delete this function when the last caller moves.
 func randomHex(n int) string {
 	b := make([]byte, n)

emulator/ecs_plugin.go

@@ -830,7 +830,7 @@ func (p *ECSPlugin) runTask(ctx *RequestContext, req *AWSRequest) (*AWSResponse,
 
 	var tasks []ECSTask
 	for i := 0; i < body.Count; i++ {
-		taskID := generateLambdaRevisionID()[:16]
+		taskID := generateLambdaRevisionID(ctx.IDs)[:16]
 		task := ECSTask{
 			TaskArn:           fmt.Sprintf("arn:aws:ecs:%s:%s:task/%s/%s", ctx.Region, ctx.AccountID, clusterName, taskID),
 			TaskDefinitionArn: td.TaskDefinitionArn,

emulator/efs_plugin.go

@@ -94,7 +94,7 @@ func (p *EFSPlugin) createFileSystem(reqCtx *RequestContext, req *AWSRequest) (*
 		input.ThroughputMode = "bursting"
 	}
 
-	fsID := generateEFSFileSystemID()
+	fsID := generateEFSFileSystemID(reqCtx.IDs)
 	arn := fmt.Sprintf("arn:aws:elasticfilesystem:%s:%s:file-system/%s",
 		reqCtx.Region, reqCtx.AccountID, fsID)
 
@@ -256,7 +256,7 @@ func (p *EFSPlugin) createAccessPoint(reqCtx *RequestContext, req *AWSRequest) (
 		return nil, efsBadRequest("FileSystemId is required")
 	}
 
-	apID := generateEFSAccessPointID()
+	apID := generateEFSAccessPointID(reqCtx.IDs)
 	arn := fmt.Sprintf("arn:aws:elasticfilesystem:%s:%s:access-point/%s",
 		reqCtx.Region, reqCtx.AccountID, apID)
 
@@ -380,7 +380,7 @@ func (p *EFSPlugin) createMountTarget(reqCtx *RequestContext, req *AWSRequest) (
 		return nil, efsBadRequest("FileSystemId is required")
 	}
 
-	mtID := generateEFSMountTargetID()
+	mtID := generateEFSMountTargetID(reqCtx.IDs)
 	mt := EFSMountTarget{
 		MountTargetID:  mtID,
 		FileSystemID:   input.FileSystemID,
@@ -695,17 +695,17 @@ func efsJSONResponse(status int, v interface{}) (*AWSResponse, error) {
 	}, nil
 }
 
-// generateEFSFileSystemID generates an EFS file system ID (fs- + 8 hex chars).
-func generateEFSFileSystemID() string {
-	return "fs-" + randomHex(8)
+// generateEFSFileSystemID mints an EFS file system ID (fs- + 16 hex chars) from m.
+func generateEFSFileSystemID(m *IDMint) string {
+	return "fs-" + m.Hex(8)
 }
 
-// generateEFSAccessPointID generates an EFS access point ID (fsap- + 8 hex chars).
-func generateEFSAccessPointID() string {
-	return "fsap-" + randomHex(8)
+// generateEFSAccessPointID mints an EFS access point ID (fsap- + 16 hex chars) from m.
+func generateEFSAccessPointID(m *IDMint) string {
+	return "fsap-" + m.Hex(8)
 }
 
-// generateEFSMountTargetID generates an EFS mount target ID (fsmt- + 8 hex chars).
-func generateEFSMountTargetID() string {
-	return "fsmt-" + randomHex(8)
+// generateEFSMountTargetID mints an EFS mount target ID (fsmt- + 16 hex chars) from m.
+func generateEFSMountTargetID(m *IDMint) string {
+	return "fsmt-" + m.Hex(8)
 }

emulator/eventbridge_plugin.go

@@ -465,7 +465,7 @@ func (p *EventBridgePlugin) putEvents(ctx *RequestContext, req *AWSRequest) (*AW
 			Detail:       entry.Detail,
 			EventBusName: busName,
 			Time:         now,
-			EventID:      generateLambdaRevisionID(),
+			EventID:      generateLambdaRevisionID(ctx.IDs),
 		}
 		existing = append(existing, ev)
 		results = append(results, resultEntry{EventID: ev.EventID})

emulator/fsx_plugin.go

@@ -56,10 +56,10 @@ type FSxTag struct {
 	Value string `json:"Value"`
 }
 
-// generateFSxFileSystemID returns a unique FSx file system ID of the form
-// "fs-" followed by 8 lowercase hex digits.
-func generateFSxFileSystemID() string {
-	return "fs-" + randomHex(8)
+// generateFSxFileSystemID mints a unique FSx file system ID from m, of the form
+// "fs-" followed by 16 lowercase hex digits.
+func generateFSxFileSystemID(m *IDMint) string {
+	return "fs-" + m.Hex(8)
 }
 
 // fsxDNSName derives a DNS name for a file system based on its ID and region.
@@ -141,7 +141,7 @@ func (p *FSxPlugin) createFileSystem(ctx *RequestContext, req *AWSRequest) (*AWS
 		input.StorageType = "SSD"
 	}
 
-	fsID := generateFSxFileSystemID()
+	fsID := generateFSxFileSystemID(ctx.IDs)
 	arn := fmt.Sprintf("arn:aws:fsx:%s:%s:file-system/%s", ctx.Region, ctx.AccountID, fsID)
 
 	// Derive VPC ID from the first subnet when available (simplified).
@@ -171,7 +171,7 @@ func (p *FSxPlugin) createFileSystem(ctx *RequestContext, req *AWSRequest) (*AWS
 		if lustreDeploymentType == "SCRATCH_2" || lustreDeploymentType == "" {
 			lustreMountName = "fsx"
 		} else {
-			lustreMountName = randomHex(8)
+			lustreMountName = ctx.IDs.Hex(8)
 		}
 	}
 

emulator/fsx_plugin_test.go

@@ -339,3 +339,41 @@ func TestFSx_SDKTargetRouting(t *testing.T) {
 	require.NoError(t, json.Unmarshal(body, &out))
 	assert.True(t, strings.HasPrefix(out.FileSystem.FileSystemId, "fs-"))
 }
+
+// TestFSx_LustreMountNameIsMintedForANonScratchDeployment covers the other half of the
+// MountName branch: SCRATCH_2 always reports "fsx", and every other Lustre deployment type
+// reports a minted value instead.
+//
+// Since #856 that value derives from the request's own ID rather than from crypto/rand, so a
+// replayed CreateFileSystem reports the mount name its recording reported. The assertion is on
+// the shape and on its difference from the SCRATCH_2 constant; the derivation itself is
+// asserted end-to-end in ids_test.go.
+func TestFSx_LustreMountNameIsMintedForANonScratchDeployment(t *testing.T) {
+	ts := httptest.NewServer(newFSxTestServer(t))
+	t.Cleanup(ts.Close)
+
+	mountNameFor := func(deploymentType string) string {
+		resp := fsxRequest(t, ts, "CreateFileSystem", `{
+			"FileSystemType": "LUSTRE",
+			"StorageCapacity": 1200,
+			"SubnetIds": ["subnet-12345678"],
+			"LustreConfiguration": {"DeploymentType": "`+deploymentType+`"}
+		}`)
+		body := readFSxBody(t, resp)
+		require.Equal(t, http.StatusOK, resp.StatusCode, "%s", body)
+		var created struct {
+			FileSystem struct {
+				LustreConfiguration struct {
+					MountName string `json:"MountName"`
+				} `json:"LustreConfiguration"`
+			} `json:"FileSystem"`
+		}
+		require.NoError(t, json.Unmarshal(body, &created))
+		return created.FileSystem.LustreConfiguration.MountName
+	}
+
+	assert.Equal(t, "fsx", mountNameFor("SCRATCH_2"),
+		"SCRATCH_2's mount name is the documented constant, not a minted value")
+	assert.Regexp(t, `^[0-9a-f]{16}$`, mountNameFor("PERSISTENT_1"),
+		"a persistent deployment reports a minted mount name")
+}

emulator/ids.go

@@ -61,7 +61,7 @@ import (
 // already derived from its inputs: a public IP from its instance id, a secret's ARN from its
 // name, CloudFormation's stack UUIDs from account and region.
 //
-// TODO(#856): 32 draw sites remain on crypto/rand, tiered by service family on the issue.
+// TODO(#856): 29 draw sites remain on crypto/rand, tiered by service family on the issue.
 
 // IDMint mints the identifiers one request publishes, derived from that request's own id so
 // that replaying the request mints the same ones.