fix(cloudformation): auto-deploy stack sets when org membership changes - #2540
Open
vieiralucas wants to merge 1 commit into
Open
vieiralucas wants to merge 1 commit into
vieiralucas wants to merge 1 commit into
Conversation
A service-managed stack set with AutoDeployment.Enabled only covered the accounts that were in the target OU at CreateStackInstances time: the flag was stored and reported but nothing ever acted on it, so an account added to the OU afterwards never got the stack set's stacks (#2539). Organizations now carries a change-hook registry, fired after a mutation that actually moves membership (accounts, their parents and status, the OU tree), after a CreateAccount completes, and by the create-admin endpoint that enrolls an account directly. CloudFormation registers an observer that reconciles every service-managed stack set with auto-deployment against the organization as it stands: - an account under a targeted OU with no instance gains one, in that OU's regions, recorded as an ordinary CREATE operation against the OU - an instance whose account has left the OU (moved, removed, closed) is deleted with its stack, or, with RetainStacksOnAccountRemoval=true, the instance is dropped and its stack stays behind in the account - instances the operator deliberately left out - an AccountFilterType that excluded them, or a DeleteStackInstances that removed them - stay out while the account stays in the OU that decision was made under - stack sets without auto-deployment, self-managed ones, instances added by account name, and stacks a refused import only recorded are left alone The OUs and regions a stack set follows are kept on the stack set itself, so an OU that momentarily holds no accounts stays a target. Reconciling against current membership rather than a diff keeps it idempotent whatever triggered it, including a stack that provisions organization resources of its own. Deployment runs inline, so the account is provisioned by the time the Organizations call returns, bounded so a slow template cannot hold that call past a client timeout: past the budget the deployment carries on in the background, as does one blocked behind another operation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2539. A service-managed stack set with
AutoDeployment.Enabledonly ever covered the accounts that were in the target OU atCreateStackInstancestime — the flag was parsed, stored, validated and reported byListStackSetAutoDeploymentTargets, but nothing acted on it. An account added to the OU afterwards never got the stack set's stacks, exactly as the reporter's repro shows.Organizations now carries a change-hook registry, fired after a mutation that actually moves membership (accounts, their parents and status, the OU tree — a tag or a policy change notifies nobody), after a
CreateAccountcompletes, and by the/_fakecloud/iam/create-adminendpoint that enrolls an account directly. CloudFormation registers an observer that reconciles every service-managed auto-deploying stack set against the organization as it stands:CREATEoperation against the OURetainStacksOnAccountRemoval=true, the instance is dropped and its stack stays behind in the accountAccountFilterTypethat excluded them, or aDeleteStackInstancesthat removed them — stay out while the account stays in the OU that decision was made under; leaving, or moving to another target OU, deploys againFAILED_IMPORTrecord are left aloneThe OUs and regions a stack set follows are now kept on the stack set itself (
auto_deployment_targets, backfilled from instances for stack sets persisted by an older build), so an OU that momentarily holds no accounts stays a target. Reconciling against current membership rather than a diff keeps it idempotent whatever triggered it — including a stack that provisionsAWS::Organizations::*resources of its own, which reaches the same observer through the CFN snapshot hook.Deployment runs inline, so the account is provisioned by the time
MoveAccountanswers, but bounded: past a 20s budget for the whole pass the deployment carries on in the background, as does one blocked behind another operation on the same stack set. One reconciliation runs at a time; a trigger that arrives during one is served by a further lap, and a cancelled request (a client that hung up) hands its pass to a task of its own rather than dropping it.Test plan
cargo test -p fakecloud-cloudformation --lib— 461 pass, including 24 new auto-deployment tests: an account joining a target OU is deployed to, one leaving loses its stacks (and the retain variant keeps them), a deleted instance is not resurrected but re-deploys after moving OU,AccountFilterTypeis honoured, per-region OU attribution across nested targets, an OU that ran empty still follows, imports are not torn down and do not deploy the rest of their OU, a refused import's stack is never deleted, a failed target is not retried forever, turning auto-deployment off stops a pass, bookkeeping-only reconciles are persisted, and cancelled passes are carried oncargo test -p fakecloud-organizations— 145 pass, including the membership-fingerprint gating and its reversal racecargo test -p fakecloud-e2e --test cloudformation_stack_sets_auto_deployment— new file replaying the reported repro end to end with the real SDK: an IAM-role stack set on the root OU, a second account created afterwards gets the role, removal deletes it, the retain variant keeps it and the emptied OU still follows, and an account created by a CloudFormation stack is covered toocloudformation_stack_sets,cloudformation_organizations,cloudformation_persistence,organizations{,_lifecycle,_persistence,_handshake},multi_accountall greencargo clippy --workspace --all-targets -- -D warnings,cargo fmtSurface
Docs updated for both services (CloudFormation stack sets, Organizations). No new API surface, endpoint or field is exposed over the wire, so no SDK changes; no service/operation/variant counts move, so no doc-count or repo-description update.
Summary by cubic
Fixes #2539: a service-managed stack set with
AutoDeployment.Enabled=trueis now reconciled against the organization when membership changes. Previously the flag was stored and reported but never acted on, so an account that joined a target OU afterCreateStackInstancesnever got the stack set's stacks; now it gains them, and an account that leaves the OU loses them (or keeps them, perRetainStacksOnAccountRemoval).Bug Fixes
CreateAccountcompletion and direct account enrollment.CREATEoperation; one leaving the OU loses its instances and stacks.AccountFilterTypeexclusion or aDeleteStackInstances) stay out while the account stays in the OU the decision was made under.FAILED_IMPORTrecords are left alone.Written for commit f64e6d3. Summary will update on new commits.