Repository navigation
fix(s3): honor Tags in CreateBucketConfiguration - #2557
Merged
Merged
Conversation
CreateBucket accepted the `Tags` tag set that was added to
CreateBucketConfiguration in the 2025 S3 API, but dropped it: only
LocationConstraint was parsed out of the body, so the bucket was created
untagged and GetBucketTagging answered NoSuchTagSet.
That path matters for Terraform. The AWS provider (6.x) tags a bucket
this way on create and skips PutBucketTagging when the create succeeds,
so an `aws_s3_bucket` with `tags` applied cleanly and then read back
`tags = {}`.
- parse the `<Tags>` tag set out of CreateBucketConfiguration with the
same parse_tagging_xml the PutBucketTagging path uses, and store it on
the new bucket
- run validate_tags first, so duplicate keys and `aws:`-prefixed keys
fail with InvalidTag before any bucket is created
- write the Tags subresource alongside the bucket meta, since bucket tags
are not part of BucketMeta and would otherwise be lost on restart
- unit tests for the stored tag set, the no-tags case, InvalidTag
rejection leaving no bucket behind, and XML-entity round-tripping
- e2e coverage for create-with-tags (plus PutBucketTagging still
replacing the create-time set), the duplicate-key rejection, and tags
surviving a restart in persistent mode
- document the behavior on the S3 service page
Fixes #2553
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015paJopoDarw6WeFFoZMDMf
Tagging a bucket at create time was authorized as nothing but
s3:CreateBucket. The S3 model is explicit on CreateBucketConfiguration$Tags:
"You must have the s3:TagResource permission to create a general purpose
bucket with tags", so a principal granted s3:CreateBucket and denied
s3:TagResource could tag a bucket that real S3 would refuse to create.
Fixing that surfaced a second bug one layer down. The AWS SDKs send
CreateBucket as `PUT /<bucket>/`, and dispatch's streaming heuristic read
the trailing slash as an object upload, so the body was handed to the
handler unbuffered. Everything that runs before the handler therefore saw
an empty body: IAM enforcement could not read CreateBucketConfiguration at
all, which is why a policy conditioned on the create-time tag set never
matched. The handler itself was unaffected -- it drains the stream into
`req.body` for non-object PUTs, which is why the tags applied even though
the authorization layer could not see them.
- streaming_route now requires a non-empty key after the bucket on the
path-style route, so `PUT /<bucket>/` is buffered like `PUT /<bucket>`
- S3 overrides iam_actions_for to append s3:TagResource when the create
body carries a tag set; dispatch already requires every returned action
to be allowed, so an untagged create still needs only s3:CreateBucket
- s3_request_tags covers CreateBucket and TagResource, so the create-time
tag set populates aws:RequestTag/<key> and aws:TagKeys
- create_bucket_configuration_tags replaces the handler's ad-hoc
`contains("<Tags>")` gate, which would have silently dropped the tag set
from a client that wrote `<Tags xmlns=...>` or any attribute on the
element
- unit tests for the action pair, the condition-key extraction, and the
helper's gating; e2e tests for the TagResource requirement and for an
aws:RequestTag-conditioned create; a dispatch regression test for the
trailing-slash route
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015paJopoDarw6WeFFoZMDMf
Write the Tags subresource ahead of meta.toml and the in-memory insert, so a store error during CreateBucket leaves nothing usable behind: the loader skips a bucket directory that has no meta.toml, so the failure mode is now "no bucket" rather than "bucket exists, permanently untagged" -- the same guarantee the InvalidTag path already gives. Also document why the us-east-1 idempotent re-create does not re-apply the create-time tag set (it re-applies no create-time setting -- not the canned ACL, object lock, or object ownership either), with a test locking that in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015paJopoDarw6WeFFoZMDMf
…te with routing Two holes the previous commits left: - An untagged CreateBucket now deletes the Tags subresource instead of leaving it alone. Writing tags.toml before meta.toml means a create that fails partway leaves an orphan tag file in a directory the loader skips (no meta.toml) -- but a later untagged create of that name would adopt the never-committed tag set on the next restart. Deleting is a no-op in the normal case, where no such file exists. - streaming_route derived the path-style key with split_once, which calls the key of `PUT /<bucket>//` "/" and streams it, while routing filters empty segments and reads the same path as a bucket-level operation -- the exact empty-body mismatch the trailing-slash fix was for. The key is now derived with the same split-and-drop-empty rule the request builder uses, so the two cannot disagree. Adds a persistence e2e for the orphan tag file (verified failing without the delete), plus dispatch tests for the doubled-slash path and for a real object key that ends in '/', which must still stream. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015paJopoDarw6WeFFoZMDMf
The gate was tightened for path-style addressing only. A virtual-hosted request that also carries the bucket in the path (`Host: b.s3...` with `PUT /b` or `PUT /b/`) still streamed, even though routing has an explicit bucket-root branch that collapses exactly that shape to CreateBucket -- so the handler and IAM saw an empty body, the CreateBucketConfiguration tag set was dropped, and iam_actions_for omitted the s3:TagResource requirement. Same failure mode as the path-style one, left open for the one shape the router already special-cases. Rather than tighten a second predicate, the router's path derivation moves into s3_routing_path and the gate calls it: streaming now requires that the routed path split into a bucket AND a non-empty key, by the same split-and-drop-empty rule the request builder uses for path_segments. The two cannot drift again -- they are the same code -- which is what produced both of these bugs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015paJopoDarw6WeFFoZMDMf
Under virtual-hosted addressing real S3 treats the whole path as the key, so a key beginning with `<bucket>/` (or a key that is the bucket name) is not representable here: the passthrough reads it as a client that put the bucket in the path and drops the prefix. Nothing in a single request tells the two apart, so the behavior resolves the ambiguity in favor of the mixed-addressing client -- pre-existing, moved verbatim into the helper, now stated rather than implied, including in the test that exercises the shape. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015paJopoDarw6WeFFoZMDMf
vieiralucas
added a commit
to Sorttech/fakecloud
that referenced
this pull request
Sep 29, 2026
…ed with CreateBucket set acl_grants, object_lock_config and ownership_controls in memory and wrote none of them. None live in BucketMeta, and the loader restores the latter two as None and falls back to the default owner grant when acl.toml is absent, so after a restart a bucket created with `x-amz-acl: public-read` came back owner-only and one created with `x-amz-bucket-object-lock-enabled: true` came back with no lock configuration at all -- which silently stops retention being enforced. Versioning survived, since that one IS in meta, which is what made the gap easy to miss. Each of the three is now written when the create sets it, and the bucket's stored subresources are cleared first so a create cannot inherit a sidecar left by a create or delete that stopped partway, or by a `/_fakecloud/reset` (which clears memory and deliberately leaves the store alone). The meta is written before that clear, so a create that fails has destroyed nothing; `objects/` is never touched, because the loader skips a bucket whose objects it cannot read and re-creating the name must not be what destroys data that is still recoverable. Persisting the ACL made a set of latent ACL bugs durable, so they are fixed here too: - A canned `x-amz-acl`, the `x-amz-grant-*` headers and an AccessControlPolicy body are mutually exclusive, as on S3. Letting one win silently discarded what the others asked for. - Canned values are validated against the set the target accepts. An unrecognized one fell through to owner-only, so a typo wiped every public grant. The object-scoped `bucket-owner-*` values are accepted on a bucket and resolve to owner-only, which is what S3 ignoring them produces. - A grant naming no resolvable grantee is refused rather than stored as an entry that can never match anyone, and grantee values are trimmed so a pretty-printed <ID> does not bake whitespace into a canonical id. - `emailAddress=` grantees are kept as AmazonCustomerByEmail instead of being dropped, and logged as unenforced: nothing resolves them to a canonical user, so the grant conveys no access. - PutBucketAcl honors the grant headers; without that arm a `put-bucket-acl --grant-read` fell through to an empty body and wiped the ACL. - PutObjectAcl updates the versioned copy of the object too, so a versionId read no longer answers a different ACL than the unversioned one. - An ACL on a bucket whose ownership disables them is refused by every path that can set one, CreateMultipartUpload included. - PutBucketOwnershipControls validates every ObjectOwnership the document carries -- `bucket_owner_enforced` matches the stored text, so a second rule could turn ACLs off while the API reported otherwise -- and persists before mutating memory. And three storage-integrity fixes in the same paths: - Every writer of a persisted snapshot propagates a serialization failure rather than writing an empty document. An empty acl.toml is worse than none: the loader takes its presence to mean an explicit ACL and drops the owner grant. A sidecar whose grants are all unusable now falls back to that owner grant too. - delete_object removes the sidecar before the body. `load` iterates `*.toml` and treats a sidecar with no body as fatal for the bucket, while an orphan body is never read, so the body is the residue an interrupted delete should leave. A body that cannot be unlinked warns rather than failing a delete that did happen. - The analytics, intelligent-tiering and metrics sidecars neither default silently (losing the configurations, which the next Put then makes permanent) nor fail the bucket over a reporting config: they are skipped with a warning. Fixes the create-time half of the gap found while reviewing faiscadev#2557.
fas89
added a commit
to Agenticstiger/forge-cli
that referenced
this pull request
Oct 5, 2026
…does terraform-provider-aws 6.23 and later tag a bucket in its CreateBucket request (CreateBucketConfiguration Tags, added to S3 in 2025) and skip PutBucketTagging when the create succeeds. moto 5.2.3 reads only LocationConstraint from that body and drops the tags, so every bucket the moto lanes applied was left untagged, and the state-drift lane read a clean apply as tag drift (tags.fluid_contract, tags.managed_by). On real S3 the tags are kept: eleven applied AWS products planned no change on provider 6.67. tests/iac/_moto_s3.py wraps moto's bucket PUT so a create also stores the body's tags, the fix fakecloud made in its own S3 (faiscadev/fakecloud#2557), and tests/iac/conftest.py applies it once per process. The drift lane passes locally with it (5 of 5), and failed without it (2 of 5).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CreateBucketaccepted theTagstag set that was added toCreateBucketConfigurationin the 2025 S3 API, but dropped it:create_bucketonly pulledLocationConstraintout of the body, so the bucket came out untagged andGetBucketTaggingansweredNoSuchTagSet.That path matters for Terraform. The AWS provider (6.x) tags a bucket this way on create and skips
PutBucketTaggingwhen the create succeeds, so anaws_s3_bucketwithtagsapplied cleanly and then read backtags = {}.Verified against the vendored Smithy model before implementing:
com.amazonaws.s3#CreateBucketConfigurationdoes carry aTagsmember targetingTagSet(non-flattened,xmlName: "Tag"), and the Rust SDK serializes it as<Tags><Tag><Key/><Value/></Tag></Tags>— the same shapeparse_tagging_xmlalready handles forPutBucketTagging.The bug underneath it
Requiring
s3:TagResource(below) did not work at first, and the reason turned out to be a separate defect one layer down. The AWS SDKs send CreateBucket asPUT /<bucket>/, andstreaming_routeincrates/fakecloud-core/src/dispatch.rsread the trailing slash as an object upload, so the body went to the handler unbuffered. Everything that runs before the handler therefore saw an empty body — IAM enforcement could not readCreateBucketConfigurationat all. The handler itself was unaffected, because it drains the stream intoreq.bodyfor non-object PUTs, which is why the tags applied even though the authorization layer could not see them.This was not limited to tags: with the old predicate, a trailing-slash
CreateBucketalso droppedLocationConstraint(so any non-us-east-1 create sent that way would have failedIllegalLocationConstraintException), andPUT /bucket/?taggingsaw an empty body too.The gate and the router had each derived "where does the bucket end and the key begin" separately, and drifted — twice (the trailing slash, then a doubled slash, then the virtual-hosted bucket-in-path shape). So the router's derivation is now extracted into
s3_routing_pathand the gate calls it: streaming requires the routed path to split into a bucket and a non-empty key, by the same split-and-drop-empty rulepath_segmentsuses. They are the same code now and cannot drift again.Changes
<Tags>tag set out ofCreateBucketConfigurationand store it on the new bucketvalidate_tagsruns before any state mutation, so duplicate keys andaws:-prefixed keys fail withInvalidTagand no bucket is createdTagssubresource (bucket tags are not part ofBucketMeta), written beforemeta.tomland before the in-memory insert, so a store failure leaves no half-created bucket; an untagged create deletes the subresource, so a previously half-writtentags.tomlcan't be adopted by a later untagged create of the same names3:CreateBucketands3:TagResource, per the model text onCreateBucketConfiguration$Tags; an untagged create still needs onlys3:CreateBucketaws:RequestTag/<key>andaws:TagKeysstreaming_routeas described aboveTest plan
fakecloud-s3): tag set stored; no-tags case;InvalidTagon duplicate keys leaves no bucket; XML-entity round-trip; theCreateBucket+TagResourceaction pair (and that an untagged create yields onlyCreateBucket); request-tag extraction for both actions; helper gating on theCreateBucketConfigurationroot; idempotent us-east-1 re-create does not re-apply the tag set.fakecloud-core):streaming_routefor/b/,/b//, a real key ending in/, and the virtual-hosted bucket-in-path shape.PutBucketTaggingstill replacing the set wholesale); duplicate-key rejection; tags surviving a restart; an untagged create clearing an orphantags.toml; thes3:TagResourcerequirement; anaws:RequestTag-conditioned create.NoSuchTagSetafter restart; orphan clear → inherited ghost tag; virtual-hosted gate → still streams).fakecloud-core331 passed,fakecloud-s3448+ passed, e2es3+s3_persistence+iam_enforcement+iam_enforcement_abac+s3_streaming_body+s3_aws_chunked+s3_access_points+s3_anonymous_access+s3_post_object+ecrall green,fakecloud-conformance --test s351 passed.cargo clippy --all-targets -- -D warningsandcargo fmt --checkclean.Surface sync
website/content/docs/services/s3.mdgains a "Bucket tags at create time" bullet;website/content/docs/reference/security.mdupdates theaws:RequestTag/<key>row for theCreateBuckettag set and itss3:TagResourcerequirement./_fakecloud/*surface, so no first-party SDK touches it.conformance-baseline.json, the ops index, the README headline, and the GitHub repo description all stay as they are.CreateBuckettag behavior.Known limitation (pre-existing, now documented)
Under virtual-hosted addressing real S3 treats the whole path as the key, so a key beginning with
<bucket>/is not representable — the router reads it as a client that put the bucket in the path. Nothing in a single request tells the two apart. Moved verbatim intos3_routing_pathand now stated in its doc comment rather than implied.Fixes #2553
🤖 Generated with Claude Code
https://claude.ai/code/session_015paJopoDarw6WeFFoZMDMf
Summary by cubic
CreateBucket now honors the
Tagstag set inCreateBucketConfiguration, storing it on the new bucket soGetBucketTaggingreturns the tags immediately instead ofNoSuchTagSet. This fixes the AWS Terraform provider (6.x) reading backtags = {}after a clean create.A tagged create now also requires
s3:TagResourcealongsides3:CreateBucketand populatesaws:RequestTag/*andaws:TagKeys, matching real S3. Fixes a dispatch bug where bucket-level PUTs likePUT /<bucket>/were streamed unbuffered, so IAM enforcement saw an empty body; they now buffer through the same routing helper as object PUTs.aws:-prefixed keys fail withInvalidTagand no bucket is created.Written for commit c9c5308. Summary will update on new commits.