Skip to content

fix(cloudformation): auto-deploy stack sets when org membership changes - #2540

Open
vieiralucas wants to merge 1 commit into
mainfrom
fix/stackset-auto-deployment
Open

vieiralucas wants to merge 1 commit into
mainfrom
fix/stackset-auto-deployment

Conversation

@vieiralucas

@vieiralucas vieiralucas commented Sep 20, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #2539. A service-managed stack set with AutoDeployment.Enabled only ever covered the accounts that were in the target OU at CreateStackInstances time — the flag was parsed, stored, validated and reported by ListStackSetAutoDeploymentTargets, but nothing acted on it. An account added to the OU afterwards never got the stack set's stacks, exactly as the reporter's repro shows.

Organizations now carries a change-hook registry, fired after a mutation that actually moves membership (accounts, their parents and status, the OU tree — a tag or a policy change notifies nobody), after a CreateAccount completes, and by the /_fakecloud/iam/create-admin endpoint that enrolls an account directly. CloudFormation registers an observer that reconciles every service-managed auto-deploying stack set against the organization as it stands:

  • an account under a targeted OU with no instance gains one, in that OU's regions, recorded as an ordinary CREATE operation against the OU
  • an instance whose account has left the OU (moved, removed, closed) is deleted with its stack, or, with RetainStacksOnAccountRemoval=true, the instance is dropped and its stack stays behind in the account
  • instances the operator deliberately left out — an AccountFilterType that excluded them, or a DeleteStackInstances that removed them — stay out while the account stays in the OU that decision was made under; leaving, or moving to another target OU, deploys again
  • stack sets without auto-deployment, self-managed ones, instances added by account name, and the operator's own stack behind a FAILED_IMPORT record are left alone

The OUs and regions a stack set follows are now kept on the stack set itself (auto_deployment_targets, backfilled from instances for stack sets persisted by an older build), so an OU that momentarily holds no accounts stays a target. Reconciling against current membership rather than a diff keeps it idempotent whatever triggered it — including a stack that provisions AWS::Organizations::* resources of its own, which reaches the same observer through the CFN snapshot hook.

Deployment runs inline, so the account is provisioned by the time MoveAccount answers, but bounded: past a 20s budget for the whole pass the deployment carries on in the background, as does one blocked behind another operation on the same stack set. One reconciliation runs at a time; a trigger that arrives during one is served by a further lap, and a cancelled request (a client that hung up) hands its pass to a task of its own rather than dropping it.

Test plan

  • cargo test -p fakecloud-cloudformation --lib — 461 pass, including 24 new auto-deployment tests: an account joining a target OU is deployed to, one leaving loses its stacks (and the retain variant keeps them), a deleted instance is not resurrected but re-deploys after moving OU, AccountFilterType is honoured, per-region OU attribution across nested targets, an OU that ran empty still follows, imports are not torn down and do not deploy the rest of their OU, a refused import's stack is never deleted, a failed target is not retried forever, turning auto-deployment off stops a pass, bookkeeping-only reconciles are persisted, and cancelled passes are carried on
  • cargo test -p fakecloud-organizations — 145 pass, including the membership-fingerprint gating and its reversal race
  • cargo test -p fakecloud-e2e --test cloudformation_stack_sets_auto_deployment — new file replaying the reported repro end to end with the real SDK: an IAM-role stack set on the root OU, a second account created afterwards gets the role, removal deletes it, the retain variant keeps it and the emptied OU still follows, and an account created by a CloudFormation stack is covered too
  • Regression: cloudformation_stack_sets, cloudformation_organizations, cloudformation_persistence, organizations{,_lifecycle,_persistence,_handshake}, multi_account all green
  • cargo clippy --workspace --all-targets -- -D warnings, cargo fmt

Surface

Docs updated for both services (CloudFormation stack sets, Organizations). No new API surface, endpoint or field is exposed over the wire, so no SDK changes; no service/operation/variant counts move, so no doc-count or repo-description update.


Summary by cubic

Fixes #2539: a service-managed stack set with AutoDeployment.Enabled=true is now reconciled against the organization when membership changes. Previously the flag was stored and reported but never acted on, so an account that joined a target OU after CreateStackInstances never got the stack set's stacks; now it gains them, and an account that leaves the OU loses them (or keeps them, per RetainStacksOnAccountRemoval).

Bug Fixes

  • Organizations fires an observer hook only when membership actually moved (accounts, parents, status, OU tree), including CreateAccount completion and direct account enrollment.
  • An account joining a target OU gains the stack set's instances in that OU's regions, recorded as a CREATE operation; one leaving the OU loses its instances and stacks.
  • Instances deliberately left out (an AccountFilterType exclusion or a DeleteStackInstances) stay out while the account stays in the OU the decision was made under.
  • Target OUs and regions are stored on the stack set (backfilled from instances for older stack sets), so an OU that momentarily holds no accounts stays a target.
  • Runs inline so the account is provisioned before the triggering Organizations call returns, capped at 20s then continuing in the background; a pass blocked behind another stack set operation is re-planned, and a cancelled request hands its pass to its own task.
  • Self-managed stack sets, non-auto-deploying ones, instances added by account name, and stacks behind FAILED_IMPORT records are left alone.

Written for commit f64e6d3. Summary will update on new commits.

Review in cubic

A service-managed stack set with AutoDeployment.Enabled only covered the
accounts that were in the target OU at CreateStackInstances time: the flag
was stored and reported but nothing ever acted on it, so an account added
to the OU afterwards never got the stack set's stacks (#2539).

Organizations now carries a change-hook registry, fired after a mutation
that actually moves membership (accounts, their parents and status, the OU
tree), after a CreateAccount completes, and by the create-admin endpoint
that enrolls an account directly. CloudFormation registers an observer that
reconciles every service-managed stack set with auto-deployment against the
organization as it stands:

- an account under a targeted OU with no instance gains one, in that OU's
  regions, recorded as an ordinary CREATE operation against the OU
- an instance whose account has left the OU (moved, removed, closed) is
  deleted with its stack, or, with RetainStacksOnAccountRemoval=true, the
  instance is dropped and its stack stays behind in the account
- instances the operator deliberately left out - an AccountFilterType that
  excluded them, or a DeleteStackInstances that removed them - stay out
  while the account stays in the OU that decision was made under
- stack sets without auto-deployment, self-managed ones, instances added by
  account name, and stacks a refused import only recorded are left alone

The OUs and regions a stack set follows are kept on the stack set itself,
so an OU that momentarily holds no accounts stays a target. Reconciling
against current membership rather than a diff keeps it idempotent whatever
triggered it, including a stack that provisions organization resources of
its own. Deployment runs inline, so the account is provisioned by the time
the Organizations call returns, bounded so a slow template cannot hold that
call past a client timeout: past the budget the deployment carries on in
the background, as does one blocked behind another operation.
@vieiralucas
vieiralucas requested a lite review from Copilot September 20, 2026 22:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adding account to an Organizational Unit does not trigger CloudFormation StackSet deployment

2 participants