Skip to content

fix(cloudformation): report ChangeSetNotFound for an unknown change set - #2563

Merged
vieiralucas merged 3 commits into
faiscadev:mainfrom
Sorttech:fix/cfn-describe-change-set-not-found
Sep 30, 2026
Merged

vieiralucas merged 3 commits into
faiscadev:mainfrom
Sorttech:fix/cfn-describe-change-set-not-found

Conversation

@Sorttech

@Sorttech Sorttech commented Sep 25, 2026 •

Copy link
Copy Markdown

cdk deploy and cdk bootstrap hung forever against a stack that
already exists.

Before creating its change set the CDK CLI deletes any leftover one with
the same name, then polls DescribeChangeSet until the call reports
DELETE_COMPLETE/DELETE_FAILED or raises ChangeSetNotFoundException
(waitForGone). On a lookup miss DescribeChangeSet fabricated
{"Status": "CREATE_COMPLETE", "ExecutionStatus": "AVAILABLE"}, so the
change set was never gone and the poll never terminated: one
DeleteChangeSet followed by DescribeChangeSet every five seconds,
with CreateChangeSet never reached.

ChangeSetNotFoundException is declared on DescribeChangeSet,
DescribeChangeSetHooks and ExecuteChangeSet, with awsQueryError code
ChangeSetNotFound and HTTP 404 (aws-models/cloudformation.json), so
returning it costs no conformance. DeleteChangeSet does not declare it
and still succeeds for a change set that was never created, matching AWS.

All three reads had the same fabricated success with the lookup
copy-pasted, ExecuteChangeSet explicitly so ("pass-through success
rather than hard-fail"). They now share one find_change_set and map a
miss to the error, which is also what CONTRIBUTING's no-stub-responses
rule asks for.

The unit coverage was asserting the stub: change_sets built a fresh
service per call, so DescribeChangeSet("cs") only passed because the
miss was faked. It now runs the lifecycle against one service.

Test plan

Regression test: describe_change_set_reports_unknown_change_set_as_not_found in crates/fakecloud-e2e/tests/cloudformation_execute_change_set.rs, plus the change_sets unit lifecycle in extras.rs which previously asserted the stub

Verification

On this exact head, rebased onto current main:

  • cargo fmt --all --check - clean.
  • cargo clippy --workspace --all-targets -- -D warnings - clean.
  • cargo test --workspace excluding fakecloud-e2e, fakecloud-conformance,
    fakecloud-tfacc and fakecloud-parity (the same set CI's test job runs),
    plus cargo test -p fakecloud-conformance --lib - 9929 passed, 0 failed.
  • cargo test -p fakecloud-e2e --test cloudformation_execute_change_set - the
    regression test above passes against a freshly built target/debug/fakecloud.

Found by deploying a CDK CloudFront + S3 SPA against fakecloud.


Summary by cubic

Fixes cdk deploy and cdk bootstrap hanging forever when the target stack already exists. The CDK CLI deletes its leftover change set, then polls DescribeChangeSet until it 404s, but an unknown change set previously got a fabricated CREATE_COMPLETE response, so the poll never terminated. DescribeChangeSet, DescribeChangeSetHooks, and ExecuteChangeSet now return ChangeSetNotFound (HTTP 404); DeleteChangeSet still succeeds for unknown change sets, matching AWS.

Change sets scoped to their stack

  • DeleteChangeSet and ListChangeSets now honor StackName; one stack's cleanup no longer deletes another stack's same-named change set (the CDK CLI names every stack's cdk-deploy-change-set).
  • CreateChangeSet rejects a duplicate name on the same stack, and deleting a stack removes its change sets.
  • Executing a change set retires the stack's other change sets, and change sets carry their stack id and name, matched by name or ARN and resolved under the write lock so concurrent CREATE change sets share the live stack's id.

Tests

  • The three reads share one find_change_set helper; the unit lifecycle test now runs against a single service.
  • E2e covers the not-found path (never-created and deleted) and the shared-name CDK flow.

Written for commit 519a350. Summary will update on new commits.

Review in cubic

`cdk deploy` and `cdk bootstrap` hung forever against a stack that
already exists.

Before creating its change set the CDK CLI deletes any leftover one with
the same name, then polls `DescribeChangeSet` until the call reports
`DELETE_COMPLETE`/`DELETE_FAILED` or raises `ChangeSetNotFoundException`
(`waitForGone`). On a lookup miss `DescribeChangeSet` fabricated
`{"Status": "CREATE_COMPLETE", "ExecutionStatus": "AVAILABLE"}`, so the
change set was never gone and the poll never terminated: one
`DeleteChangeSet` followed by `DescribeChangeSet` every five seconds,
with `CreateChangeSet` never reached.

`ChangeSetNotFoundException` is declared on `DescribeChangeSet`,
`DescribeChangeSetHooks` and `ExecuteChangeSet`, with awsQueryError code
`ChangeSetNotFound` and HTTP 404 (aws-models/cloudformation.json), so
returning it costs no conformance. `DeleteChangeSet` does not declare it
and still succeeds for a change set that was never created, matching AWS.

All three reads had the same fabricated success with the lookup
copy-pasted, `ExecuteChangeSet` explicitly so ("pass-through success
rather than hard-fail"). They now share one `find_change_set` and map a
miss to the error, which is also what CONTRIBUTING's no-stub-responses
rule asks for.

The unit coverage was asserting the stub: `change_sets` built a fresh
service per call, so `DescribeChangeSet("cs")` only passed because the
miss was faked. It now runs the lifecycle against one service.
- DeleteChangeSet and ListChangeSets honor StackName instead of acting on
  every stack's change set of that name (the CDK CLI names every stack's
  change set cdk-deploy-change-set, so one stack's cleanup deleted another's).
- Store the resolved stack name and id on each change set so a StackName
  filter matches by name or ARN; resolve them under the write lock so
  concurrent CREATE change sets share the live stack's id.
- CreateChangeSet rejects a duplicate name on the same stack with
  AlreadyExistsException.
- DeleteStack removes the stack's change sets; a successful ExecuteChangeSet
  retires the stack's other change sets, as AWS documents.
- Unit tests for each; e2e for the shared-name CDK flow.
@vieiralucas
vieiralucas merged commit 5948fda into faiscadev:main Sep 30, 2026
157 checks passed
@vieiralucas

Copy link
Copy Markdown
Member

Thanks a lot for this, @Sorttech. Tracing the CDK CLI's waitForGone loop back to the fabricated CREATE_COMPLETE was a great catch, and returning the declared ChangeSetNotFound is exactly right. Making lookups honest exposed a bug that the fake success had been hiding, so I pushed a few follow-ups before merging: DeleteChangeSet and ListChangeSets now honor StackName (the CDK names every stack's change set cdk-deploy-change-set, so one stack's cleanup used to delete another's), change sets store the resolved stack name and id so a filter matches by name or ARN (including under concurrent CREATE change sets and for a stack that does not exist yet), CreateChangeSet rejects a duplicate name on the same stack with AlreadyExistsException, DeleteStack removes the stack's change sets, and a successful ExecuteChangeSet retires the stack's other change sets as AWS documents. Unit tests for each plus an e2e for the shared-name CDK flow; the CloudFormation conformance probe stays at 3434/3434. Merged. Really appreciate the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants