fix(cloudformation): report ChangeSetNotFound for an unknown change set - #2563
Conversation
`cdk deploy` and `cdk bootstrap` hung forever against a stack that
already exists.
Before creating its change set the CDK CLI deletes any leftover one with
the same name, then polls `DescribeChangeSet` until the call reports
`DELETE_COMPLETE`/`DELETE_FAILED` or raises `ChangeSetNotFoundException`
(`waitForGone`). On a lookup miss `DescribeChangeSet` fabricated
`{"Status": "CREATE_COMPLETE", "ExecutionStatus": "AVAILABLE"}`, so the
change set was never gone and the poll never terminated: one
`DeleteChangeSet` followed by `DescribeChangeSet` every five seconds,
with `CreateChangeSet` never reached.
`ChangeSetNotFoundException` is declared on `DescribeChangeSet`,
`DescribeChangeSetHooks` and `ExecuteChangeSet`, with awsQueryError code
`ChangeSetNotFound` and HTTP 404 (aws-models/cloudformation.json), so
returning it costs no conformance. `DeleteChangeSet` does not declare it
and still succeeds for a change set that was never created, matching AWS.
All three reads had the same fabricated success with the lookup
copy-pasted, `ExecuteChangeSet` explicitly so ("pass-through success
rather than hard-fail"). They now share one `find_change_set` and map a
miss to the error, which is also what CONTRIBUTING's no-stub-responses
rule asks for.
The unit coverage was asserting the stub: `change_sets` built a fresh
service per call, so `DescribeChangeSet("cs")` only passed because the
miss was faked. It now runs the lifecycle against one service.
- DeleteChangeSet and ListChangeSets honor StackName instead of acting on every stack's change set of that name (the CDK CLI names every stack's change set cdk-deploy-change-set, so one stack's cleanup deleted another's). - Store the resolved stack name and id on each change set so a StackName filter matches by name or ARN; resolve them under the write lock so concurrent CREATE change sets share the live stack's id. - CreateChangeSet rejects a duplicate name on the same stack with AlreadyExistsException. - DeleteStack removes the stack's change sets; a successful ExecuteChangeSet retires the stack's other change sets, as AWS documents. - Unit tests for each; e2e for the shared-name CDK flow.
|
Thanks a lot for this, @Sorttech. Tracing the CDK CLI's waitForGone loop back to the fabricated CREATE_COMPLETE was a great catch, and returning the declared ChangeSetNotFound is exactly right. Making lookups honest exposed a bug that the fake success had been hiding, so I pushed a few follow-ups before merging: DeleteChangeSet and ListChangeSets now honor StackName (the CDK names every stack's change set cdk-deploy-change-set, so one stack's cleanup used to delete another's), change sets store the resolved stack name and id so a filter matches by name or ARN (including under concurrent CREATE change sets and for a stack that does not exist yet), CreateChangeSet rejects a duplicate name on the same stack with AlreadyExistsException, DeleteStack removes the stack's change sets, and a successful ExecuteChangeSet retires the stack's other change sets as AWS documents. Unit tests for each plus an e2e for the shared-name CDK flow; the CloudFormation conformance probe stays at 3434/3434. Merged. Really appreciate the contribution. |
cdk deployandcdk bootstraphung forever against a stack thatalready exists.
Before creating its change set the CDK CLI deletes any leftover one with
the same name, then polls
DescribeChangeSetuntil the call reportsDELETE_COMPLETE/DELETE_FAILEDor raisesChangeSetNotFoundException(
waitForGone). On a lookup missDescribeChangeSetfabricated{"Status": "CREATE_COMPLETE", "ExecutionStatus": "AVAILABLE"}, so thechange set was never gone and the poll never terminated: one
DeleteChangeSetfollowed byDescribeChangeSetevery five seconds,with
CreateChangeSetnever reached.ChangeSetNotFoundExceptionis declared onDescribeChangeSet,DescribeChangeSetHooksandExecuteChangeSet, with awsQueryError codeChangeSetNotFoundand HTTP 404 (aws-models/cloudformation.json), soreturning it costs no conformance.
DeleteChangeSetdoes not declare itand still succeeds for a change set that was never created, matching AWS.
All three reads had the same fabricated success with the lookup
copy-pasted,
ExecuteChangeSetexplicitly so ("pass-through successrather than hard-fail"). They now share one
find_change_setand map amiss to the error, which is also what CONTRIBUTING's no-stub-responses
rule asks for.
The unit coverage was asserting the stub:
change_setsbuilt a freshservice per call, so
DescribeChangeSet("cs")only passed because themiss was faked. It now runs the lifecycle against one service.
Test plan
Regression test:
describe_change_set_reports_unknown_change_set_as_not_foundincrates/fakecloud-e2e/tests/cloudformation_execute_change_set.rs, plus thechange_setsunit lifecycle inextras.rswhich previously asserted the stubVerification
On this exact head, rebased onto current
main:cargo fmt --all --check- clean.cargo clippy --workspace --all-targets -- -D warnings- clean.cargo test --workspaceexcludingfakecloud-e2e,fakecloud-conformance,fakecloud-tfaccandfakecloud-parity(the same set CI'stestjob runs),plus
cargo test -p fakecloud-conformance --lib- 9929 passed, 0 failed.cargo test -p fakecloud-e2e --test cloudformation_execute_change_set- theregression test above passes against a freshly built
target/debug/fakecloud.Found by deploying a CDK CloudFront + S3 SPA against fakecloud.
Summary by cubic
Fixes
cdk deployandcdk bootstraphanging forever when the target stack already exists. The CDK CLI deletes its leftover change set, then pollsDescribeChangeSetuntil it 404s, but an unknown change set previously got a fabricatedCREATE_COMPLETEresponse, so the poll never terminated.DescribeChangeSet,DescribeChangeSetHooks, andExecuteChangeSetnow returnChangeSetNotFound(HTTP 404);DeleteChangeSetstill succeeds for unknown change sets, matching AWS.Change sets scoped to their stack
DeleteChangeSetandListChangeSetsnow honorStackName; one stack's cleanup no longer deletes another stack's same-named change set (the CDK CLI names every stack'scdk-deploy-change-set).CreateChangeSetrejects a duplicate name on the same stack, and deleting a stack removes its change sets.Tests
find_change_sethelper; the unit lifecycle test now runs against a single service.Written for commit 519a350. Summary will update on new commits.