Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions changes/unreleased/Added-20261009-213500.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
kind: Added
body: Expand CloudFormation engine with native resource provisioning for Lambda EventSourceMapping, EventBridge Rule, SSM Parameter, and SecretsManager Secret; normalize Lambda function ARNs in event source mappings; and align documentation and metrics in roadmap.md and coverage.md.
time: 2026-10-09T21:35:00.000000+09:00
30 changes: 9 additions & 21 deletions docs/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,9 @@ the parser does not read.
| `json-1.0` | 48 | the `X-Amz-Target` header |
| `query` | 15 | the `Action` form field |
| `rest-xml` | 4 | HTTP method + path |
| `ec2Query` | 1 | the `Action` form field (`ec2`) |
| `rpcv2Cbor` | 1 | the path / RPC-v2 routing (`partnercentralrevenuemeasurement`) |
| no in-tree model | 11 | n/a — hand-written providers |
| unrecognised protocol | 2 | n/a — `ec2` is `ec2Query`, `partnercentralrevenuemeasurement` is `rpcv2Cbor` |

**The operation is not CRUD-shaped.** `GetThing`, `ListThings` and `CreateThing`
map onto a generic store. `ExecuteStatement`, `InvokeEndpoint` and
Expand All @@ -72,19 +73,9 @@ answer. This applies to four services: `forecastquery`, the two SageMaker Runtim
variants `sagemaker-runtime` and `sagemakerruntimehttp2`, and `rds-data`. No
protocol change reaches them.

**The protocol is one the parser does not read.** This applies to two services,
and it is a different failure from the four above. `ec2` speaks
`aws.protocols#ec2Query` and `partnercentralrevenuemeasurement` speaks
`smithy.protocols#rpcv2Cbor`; `internal/codegen/parser.go` recognises neither, so
*none* of their operations is classified — not because their names are unshaped,
but because the model never reached the classifier. Teaching the parser those two
protocols would reach them; no amount of CRUD-shaping would.

EC2 is the one case where this costs depth rather than service: it is registered
and served by a hand-written provider, so it is not in the registered-only five.
What it loses is the engine — its model-declared long tail stays `unimplemented`
instead of falling back to `auto-crud`, which is what `EngineWired: false` on its
[fidelity manifest](fidelity-manifest.md) entry records.
EC2 (`ec2Query`) and Partner Central Revenue Measurement (`rpcv2Cbor`) originally had
protocols the parser did not read; both protocols and native CBOR serialization are now
supported, bringing the entire registered AWS fleet to serving $\ge 1$ operation.

Registering a service the engine cannot serve is deliberate. The alternative is
worse: an *unregistered* service is not routed, so the SDK call leaves the
Expand All @@ -98,13 +89,13 @@ when a provider returns `plugin.ErrUnhandledOp`, so a hand-written provider that
refuses unknown operations itself (`apigatewayv2`, `xray`) never reaches it. The
manifest records this per service as `EngineWired`.

## Why compatibility-tested is 426, not 431
## Why compatibility-tested is 427, not 431

`test/compatibility/test_service_smoke.py` parametrises over the generated
service list rather than a hand-written one, so a service cannot be registered
and quietly go untested — which is what 31 of them were until this was measured.

Five are excluded, and they are not a backlog. In every one it is botocore, not
Four are excluded, and they are not a backlog. In every one it is botocore, not
DevCloud, that stops the request, so no answer DevCloud could give would change
the outcome. Each stays registered: the call is still answered locally instead
of reaching a billed AWS account.
Expand All @@ -113,16 +104,13 @@ of reaching a billed AWS account.
`transcribestreaming` (`sagemaker-runtime` and `transcribe` are different clients
with different APIs). No boto3 test can exist for a client that does not exist.

**The client exists but cannot be pointed at localhost** (3). `codecatalyst`
**The client exists but cannot be pointed at localhost** (2). `codecatalyst`
authenticates with a bearer token rather than SigV4, so botocore raises
`NoAuthTokenError` before the request is built. `cloudfront-keyvaluestore`
resolves its endpoint from a KVS ARN and so never honours `endpoint_url`.
`partnercentralrevenuemeasurement` decodes every reply with botocore's CBOR
parser, and DevCloud has no CBOR encoder, so even a clean decline reads as a
corrupt frame — see the protocol table above.

Both sets are pinned in `test/compatibility/_coverage.py` and asserted, so
adding a sixth is a deliberate edit that moves this figure with it.
adding a fifth is a deliberate edit that moves this figure with it.

## Contested signing names

Expand Down
2 changes: 1 addition & 1 deletion docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Phase 1F 100% Service Fleet Activation implemented native CBOR encoding/decoding
RPC-v2 routing, bringing `partnercentralrevenuemeasurement` into `auto-crud`. Every
single registered service in the AWS fleet (431 of 431, 100.0%) now serves $\ge 1$ operation,
leaving **zero** registered-only services.
The original 3,085-operation unimplemented baseline has **469 remaining**;
The original 3,085-operation unimplemented baseline has **477 remaining** across all registered services (267 within the serving target);
complete AWS operation support is still in progress across subsequent bundles. See
[EventBridge](services/eventbridge.md), [SNS](services/sns.md), [S3](services/s3.md) and
[IAM/STS](services/iam-sts.md) for local limits.
Expand Down
247 changes: 247 additions & 0 deletions internal/services/cloudformation/engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,14 @@ func (e *Engine) provisionResource(ctx context.Context, stackName, logicalID, re
res, err = e.provisionDynamoDBTable(ctx, logicalID, props)
case "AWS::Lambda::Function":
res, err = e.provisionLambdaFunction(ctx, logicalID, props)
case "AWS::Lambda::EventSourceMapping":
res, err = e.provisionLambdaEventSourceMapping(ctx, logicalID, props)
case "AWS::Events::Rule":
res, err = e.provisionEventBridgeRule(ctx, logicalID, props)
case "AWS::SSM::Parameter":
res, err = e.provisionSSMParameter(ctx, logicalID, props)
case "AWS::SecretsManager::Secret":
res, err = e.provisionSecretsManagerSecret(ctx, logicalID, props)
case "AWS::SQS::Queue":
res, err = e.provisionSQSQueue(ctx, stackName, logicalID, props)
case "AWS::SNS::Topic":
Expand Down Expand Up @@ -812,6 +820,8 @@ func (e *Engine) simulateResource(logicalID, resType string, props map[string]an
physicalID = name
} else if name, ok := props["RoleName"].(string); ok && name != "" {
physicalID = name
} else if name, ok := props["Name"].(string); ok && name != "" {
physicalID = name
}
}
arn := shared.BuildARN("cloudformation", "resource", physicalID)
Expand All @@ -834,6 +844,14 @@ func (e *Engine) deleteResource(ctx context.Context, resType, physicalID, arn st
return e.deleteDynamoDBTable(ctx, physicalID)
case "AWS::Lambda::Function":
return e.deleteLambdaFunction(ctx, physicalID)
case "AWS::Lambda::EventSourceMapping":
return e.deleteLambdaEventSourceMapping(ctx, physicalID)
case "AWS::Events::Rule":
return e.deleteEventBridgeRule(ctx, physicalID)
case "AWS::SSM::Parameter":
return e.deleteSSMParameter(ctx, physicalID)
case "AWS::SecretsManager::Secret":
return e.deleteSecretsManagerSecret(ctx, physicalID)
case "AWS::SQS::Queue":
queueURL := attrs["QueueUrl"]
if queueURL == "" {
Expand Down Expand Up @@ -1148,6 +1166,235 @@ func (e *Engine) deleteIAMRole(ctx context.Context, name string) error {
return err
}

// ---- Lambda EventSourceMapping ----

func (e *Engine) provisionLambdaEventSourceMapping(ctx context.Context, logicalID string, props map[string]any) (*provisionResult, error) {
body, err := json.Marshal(props)
if err != nil {
return nil, err
}
resp, err := e.call(ctx, "lambda", "", http.MethodPost, "/2015-03-31/event-source-mappings", "application/json", "", body)
if err != nil {
return nil, err
}
if !isSuccess(resp) {
return nil, fmt.Errorf("lambda CreateEventSourceMapping failed: status=%d body=%s", resp.StatusCode, string(resp.Body))
}
var out struct {
UUID string `json:"UUID"`
}
_ = json.Unmarshal(resp.Body, &out)
uuid := out.UUID
if uuid == "" {
uuid = logicalID + "-" + shared.GenerateID("", 8)
}
arn := fmt.Sprintf("arn:aws:lambda:%s:%s:event-source-mapping:%s", shared.DefaultRegion, shared.DefaultAccountID, uuid)
return &provisionResult{
LogicalID: logicalID,
Type: "AWS::Lambda::EventSourceMapping",
PhysicalID: uuid,
ARN: arn,
Attributes: map[string]string{
"Arn": arn,
"Id": uuid,
},
}, nil
}

func (e *Engine) deleteLambdaEventSourceMapping(ctx context.Context, uuid string) error {
if uuid == "" {
return nil
}
_, err := e.call(ctx, "lambda", "", http.MethodDelete, "/2015-03-31/event-source-mappings/"+uuid, "application/json", "", nil)
return err
}

// ---- EventBridge Rule ----

func (e *Engine) provisionEventBridgeRule(ctx context.Context, logicalID string, props map[string]any) (*provisionResult, error) {
name, _ := props["Name"].(string)
if name == "" {
name = logicalID + "-" + shared.GenerateID("", 8)
}
rulePayload := map[string]any{
"Name": name,
}
if v, ok := props["EventBusName"].(string); ok && v != "" {
rulePayload["EventBusName"] = v
}
if v, ok := props["EventPattern"]; ok && v != nil {
if s, ok := v.(string); ok {
rulePayload["EventPattern"] = s
} else if b, err := json.Marshal(v); err == nil {
rulePayload["EventPattern"] = string(b)
}
}
if v, ok := props["ScheduleExpression"].(string); ok && v != "" {
rulePayload["ScheduleExpression"] = v
}
if v, ok := props["State"].(string); ok && v != "" {
rulePayload["State"] = v
}
if v, ok := props["Description"].(string); ok && v != "" {
rulePayload["Description"] = v
}

body, err := json.Marshal(rulePayload)
if err != nil {
return nil, err
}
resp, err := e.call(ctx, "eventbridge", "PutRule", http.MethodPost, "/", "application/x-amz-json-1.1", "AWSEvents.PutRule", body)
if err != nil {
return nil, err
}
if !isSuccess(resp) {
return nil, fmt.Errorf("eventbridge PutRule failed: status=%d body=%s", resp.StatusCode, string(resp.Body))
}

if targets, ok := props["Targets"].([]any); ok && len(targets) > 0 {
targetsPayload := map[string]any{
"Rule": name,
"Targets": targets,
}
if v, ok := props["EventBusName"].(string); ok && v != "" {
targetsPayload["EventBusName"] = v
}
tb, err := json.Marshal(targetsPayload)
if err == nil {
_, _ = e.call(ctx, "eventbridge", "PutTargets", http.MethodPost, "/", "application/x-amz-json-1.1", "AWSEvents.PutTargets", tb)
}
}

arn := fmt.Sprintf("arn:aws:events:%s:%s:rule/%s", shared.DefaultRegion, shared.DefaultAccountID, name)
return &provisionResult{
LogicalID: logicalID,
Type: "AWS::Events::Rule",
PhysicalID: name,
ARN: arn,
Attributes: map[string]string{
"Arn": arn,
},
}, nil
}

func (e *Engine) deleteEventBridgeRule(ctx context.Context, name string) error {
if name == "" {
return nil
}
body, _ := json.Marshal(map[string]string{"Name": name})
_, err := e.call(ctx, "eventbridge", "DeleteRule", http.MethodPost, "/", "application/x-amz-json-1.1", "AWSEvents.DeleteRule", body)
return err
}

// ---- SSM Parameter ----

func (e *Engine) provisionSSMParameter(ctx context.Context, logicalID string, props map[string]any) (*provisionResult, error) {
name, _ := props["Name"].(string)
if name == "" {
name = "/" + logicalID
}
paramType, _ := props["Type"].(string)
if paramType == "" {
paramType = "String"
}
val := toString(props["Value"])
desc, _ := props["Description"].(string)

payload := map[string]any{
"Name": name,
"Type": paramType,
"Value": val,
"Overwrite": true,
}
if desc != "" {
payload["Description"] = desc
}
body, err := json.Marshal(payload)
if err != nil {
return nil, err
}
resp, err := e.call(ctx, "ssm", "PutParameter", http.MethodPost, "/", "application/x-amz-json-1.1", "AmazonSSM.PutParameter", body)
if err != nil {
return nil, err
}
if !isSuccess(resp) {
return nil, fmt.Errorf("ssm PutParameter failed: status=%d body=%s", resp.StatusCode, string(resp.Body))
}
arn := fmt.Sprintf("arn:aws:ssm:%s:%s:parameter%s", shared.DefaultRegion, shared.DefaultAccountID, name)
return &provisionResult{
LogicalID: logicalID,
Type: "AWS::SSM::Parameter",
PhysicalID: name,
ARN: arn,
Attributes: map[string]string{
"Arn": arn,
"Type": paramType,
"Value": val,
},
}, nil
}

func (e *Engine) deleteSSMParameter(ctx context.Context, name string) error {
if name == "" {
return nil
}
body, _ := json.Marshal(map[string]string{"Name": name})
_, err := e.call(ctx, "ssm", "DeleteParameter", http.MethodPost, "/", "application/x-amz-json-1.1", "AmazonSSM.DeleteParameter", body)
return err
}

// ---- Secrets Manager Secret ----

func (e *Engine) provisionSecretsManagerSecret(ctx context.Context, logicalID string, props map[string]any) (*provisionResult, error) {
name, _ := props["Name"].(string)
if name == "" {
name = logicalID + "-" + shared.GenerateID("", 8)
}
payload := map[string]any{
"Name": name,
}
if v, ok := props["SecretString"].(string); ok {
payload["SecretString"] = v
}
if v, ok := props["Description"].(string); ok {
payload["Description"] = v
}
body, err := json.Marshal(payload)
if err != nil {
return nil, err
}
resp, err := e.call(ctx, "secretsmanager", "CreateSecret", http.MethodPost, "/", "application/x-amz-json-1.1", "secretsmanager.CreateSecret", body)
if err != nil {
return nil, err
}
if !isSuccess(resp) {
return nil, fmt.Errorf("secretsmanager CreateSecret failed: status=%d body=%s", resp.StatusCode, string(resp.Body))
}
arn := fmt.Sprintf("arn:aws:secretsmanager:%s:%s:secret:%s", shared.DefaultRegion, shared.DefaultAccountID, name)
attrs := map[string]string{
"Arn": arn,
}
if v, ok := props["SecretString"].(string); ok {
attrs["SecretString"] = v
}
return &provisionResult{
LogicalID: logicalID,
Type: "AWS::SecretsManager::Secret",
PhysicalID: name,
ARN: arn,
Attributes: attrs,
}, nil
}

func (e *Engine) deleteSecretsManagerSecret(ctx context.Context, name string) error {
if name == "" {
return nil
}
body, _ := json.Marshal(map[string]any{"SecretId": name, "ForceDeleteWithoutRecovery": true})
_, err := e.call(ctx, "secretsmanager", "DeleteSecret", http.MethodPost, "/", "application/x-amz-json-1.1", "secretsmanager.DeleteSecret", body)
return err
}

// extractXMLField does a minimal XML tag lookup — enough to pluck a single
// scalar value out of a response body without dragging in full XML decoding.
func extractXMLField(body []byte, tag string) string {
Expand Down
Loading
Loading