fix(#856): the shared UUID-shaped identifier, and the messaging/storage family - #1275
Merged
Merged
Conversation
…ge family Substrate had two shared draw sites, not one. randomHex was the known one; the other was a UUID-shaped generator declared in the Lambda plugin that six other services publish an identifier from — an ECS task id, a Step Functions execution name, an SQS message id, an EventBridge event id, a CloudWatch Logs upload sequence token and a Service Quotas request id. Tiering per service family would have left it on crypto/rand until the last of the six moved, so it moves here with Lambda's own revision ids. Its rendering is unchanged: sixteen derived bytes in UUID *shape*, 8-4-4-4-12 lowercase hex without the RFC 4122 version and variant bits. IDMint.UUID sets those bits, and using it here would have changed which bytes a caller sees, which is not what #856 is about. Moving with it: SQS receipt handles, SNS subscription ids and notification-envelope message ids, EFS file-system/access-point/mount-target ids, FSx file-system ids and Lustre mount names, and Transfer Family server ids. A send mints a message's initial receipt handle and each receive replaces it, matching real SQS, so the handle derives from the mint's ordinal rather than from the message — deriving from the message would make every receive of one message agree. Three helpers minted without a request context in scope and now take the mint as a parameter: buildSNSEnvelope, requestServiceQuotaIncrease and FSx's Lustre mount-name branch. 29 draw sites remain on crypto/rand, down from 32; the TODO on randomHex and on IDMint carries the new count. Tests: a SendMessageBatch whose three entries carry identical bodies mints three distinct message ids, which is the ordinal's test through the shared generator; and a recorded stream that sends and receives an SQS message, subscribes to an SNS topic and creates an EFS file system with an access point — each later request naming what an earlier one minted — replays with zero differences and StateValid true. Reverting generateSQSReceiptHandle alone to crypto/rand fails that test on the ReceiptHandle difference and on every state hash after it, so it is not vacuous. Two branches the change touched and nothing covered gained tests of their own: an unnamed StartExecution, and a Lustre deployment type other than SCRATCH_2. Refs #856.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tier 2 of #856. Tier 1 (#1265) built
IDMintand moved EC2, IAM and STS onto it.Substrate had two shared draw sites, not one
randomHexwas the known one, documented as the chokepoint that makes the per-familytiering possible. The other was a UUID-shaped generator declared in the Lambda plugin that
six other services publish an identifier from:
runTask)SendMessage,SendMessageBatch)PutEvents)Tiering strictly per service family would have left this generator on
crypto/randuntilthe last of the six moved, and every one of those identifiers diverging on replay in the
meantime. So it moves here, with Lambda's own revision IDs.
Its rendering is unchanged: sixteen derived bytes in UUID shape —
8-4-4-4-12lowercase hex, without the RFC 4122 version and variant bits.
IDMint.UUIDsets those bits,and using it here would have changed which bytes a caller sees. #856 is about reproducing an
identifier across a replay, not about changing it, so the doc comment says why
UUIDisdeliberately not called.
The messaging/storage family, moving with it
SQS receipt handles; SNS subscription IDs and notification-envelope message IDs; EFS
file-system, access-point and mount-target IDs; FSx file-system IDs and Lustre mount names;
Transfer Family server IDs.
One behavioural note worth recording: a send mints a message's initial receipt handle and
each receive replaces it. That matches real SQS, where a handle belongs to a receive and
only the most recent one deletes — so the handle derives from the mint's ordinal rather than
from the message. Deriving from the message would make every receive of one message agree,
which would be a different bug wearing determinism's clothes.
Three helpers minted without a request context in scope and now take the mint as a
parameter:
buildSNSEnvelope,requestServiceQuotaIncrease(whose ownidsname wasalready taken by a local
[]string, so the parameter ismint) and FSx's Lustremount-name branch.
29 draw sites remain on
crypto/rand, down from 32. TheTODO(#856)on bothrandomHexand
IDMintcarries the new count.Tests
SendMessageBatchwhose three entriescarry identical bodies mints three distinct message IDs.
deletes it by the handle the receive returned, creates an SNS topic and subscribes to it,
then creates an EFS file system with an access point naming it, replays with zero
differences and
StateValidtrue, withWithRecordedBodies(),WithRecordedStateHashes()andReplayConfig{ValidateState: true}.generateSQSReceiptHandlealone tocrypto/randfails that test atseq=2 op=ReceiveMessage response_body/…/ReceiptHandle: 74d7… -> 3dbf… (major)plus astate_hash_aftermismatch on every event fromseq=1onward. Restoring it returns thetest to green.
unnamed
StartExecution(asserting both that two starts under one mint differ and that asecond mint over the same seed reproduces the first name), and a Lustre deployment type
other than
SCRATCH_2. Patch coverage of the changed non-test lines is 87/87.Verification
make lint(0 issues),make test(race,-count=1),make docs-reference-check docs-versions version-check discarded-unmarshal-check wire-bookkeeping-check.Refs #856 — the issue stays open until its checklist empties; this PR ships two of the six
remaining family groups plus the shared generator.