Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ DevCloud is an **on-ramp to the cloud**, not a replacement for it. The goal is t

## Features

- **431 AWS services registered, 426 serving at least one operation** — every service AWS publishes is routed, so no SDK call escapes to a billable account; the remaining 5 decline with a clean AWS error. Depth is a separate, smaller promise — see [coverage.md](docs/coverage.md) for both targets.
- **431 AWS services registered, 427 serving at least one operation** — every service AWS publishes is routed, so no SDK call escapes to a billable account; the remaining 4 decline with a clean AWS error. Depth is a separate, smaller promise — see [coverage.md](docs/coverage.md) for both targets.
- **boto3-compatible** — a 1,530-test suite runs in CI (`make test-compat`) across every registered service. Unsupported operations return a clean AWS error, never a false success.
- **Cross-service integration** — CloudFormation provisioning, DynamoDB Streams → Lambda, EventBridge targets, S3 → Lambda
- **Smithy-driven codegen** — Go types, routers and error catalogues generated from AWS models, with a weekly sync workflow that keeps them current
Expand Down
3 changes: 3 additions & 0 deletions changes/unreleased/Added-20261009-083800.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
kind: Added
body: Implement Phase 1D generic CRUD engine semantic verb expansion by adding comprehensive classification for association, state toggle, search, lifecycle, mutation, and batch prefixes (Relate, Toggle, Lifecycle, Batch) across the entire fleet, promoting 2,014 operations into auto-crud and reducing the global unimplemented baseline from 2,511 to 497.
time: 2026-10-09T08:38:00.000000+09:00
6 changes: 3 additions & 3 deletions cmd/devcloud/routing_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ func TestContestedDataPlanesResolveToThemselves(t *testing.T) {
// binary actually ships rather than a synthetic one.
//
// Both operations below bind to a path a classified sibling also claims,
// separated only by a query constraint: chime's AssociatePhoneNumberWithUser
// separated only by a query constraint: chime's LogoutUser
// against UpdateUser, apigateway's ImportRestApi against CreateRestApi. While
// the registry held only classified operations, httproute.Match had no route
// specific enough to prefer, so the sibling answered 200 for an operation
Expand All @@ -68,8 +68,8 @@ func TestContestedDataPlanesResolveToThemselves(t *testing.T) {
// Describe/List/Get, and "Import" is excluded as mutating outright.
func TestUnclassifiableRouteDeclinesInsteadOfAnsweringAsASibling(t *testing.T) {
cases := []struct{ name, service, method, uri string }{
{"chime_associate_phone_number", "chime", "POST",
"/accounts/a1/users/u1?operation=associate-phone-number"},
{"chime_logout_user", "chime", "POST",
"/accounts/a1/users/u1?operation=logout"},
{"apigateway_import_rest_api", "apigateway", "POST", "/restapis?mode=import"},
}
for _, c := range cases {
Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

| Page | What it covers |
|---|---|
| [Coverage](coverage.md) | 431 registered / 426 serving — the routing and depth targets, and what each promises |
| [Coverage](coverage.md) | 431 registered / 427 serving — the routing and depth targets, and what each promises |
| [Compatibility Policy](compatibility-policy.md) | What v1.0 guarantees across 1.x, what it does not, and how deprecation works |
| [Fidelity Manifest](fidelity-manifest.md) | Per-operation tiers: how much to trust any given call |
| [CRUD Engine](crud-engine.md) | How engine-served operations behave, and where they stop |
Expand Down
8 changes: 4 additions & 4 deletions docs/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ alone.
| Number | What it means | Today |
|---|---|---|
| **Registered** | The gateway routes the service, so the call reaches DevCloud instead of real AWS. | **431** |
| **Serving ≥1 operation** | At least one operation returns a real, store-backed answer. | **426** |
| **Registered-only** | Routed, but every operation declines with a clean AWS error. | **5** |
| **Serving ≥1 operation** | At least one operation returns a real, store-backed answer. | **427** |
| **Registered-only** | Routed, but every operation declines with a clean AWS error. | **4** |
| **Compatibility-tested** | A boto3 test exercises the service in CI and passes. | **426** |

Per operation, from the [fidelity manifest](fidelity-manifest.md). Two
Expand All @@ -17,8 +17,8 @@ denominators, because [routing and depth are two targets](#the-target):
| Tier | Serving target | All registered |
|---|---|---|
| `hand-verified` | 4,601 | 4,632 |
| `auto-crud` | 6,380 | 12,058 |
| `unimplemented` | 1,426 | 2,511 |
| `auto-crud` | 7,529 | 14,064 |
| `unimplemented` | 277 | 505 |
| **total known** | **12,407** | **19,201** |
| **hand-verified share** | **37.1%** | **24.1%** |

Expand Down
8 changes: 4 additions & 4 deletions docs/crud-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ operation carries is in the [fidelity manifest](fidelity-manifest.md).

| Piece | Where | What it does |
|---|---|---|
| Engine | [`internal/shared/crud`](../internal/shared/crud/crud.go) | In-memory resource store plus verb dispatch (Create/Get/List/Delete/Update/…) |
| Engine | [`internal/shared/crud`](../internal/shared/crud/crud.go) | In-memory resource store plus verb dispatch (Create/Get/List/Delete/Update/Tag/Untag/Relate/Toggle) |
| Classification | `internal/codegen/gen_crud_meta.go` | Inspects each operation's name and output shape, emits `internal/generated/crudregistry/registry_gen.go` whose `init()` registers every classifiable operation |
| Integration | [`internal/gateway/router.go`](../internal/gateway/router.go) | Calls the engine when a provider returns `plugin.ErrUnhandledOp`; returns `InvalidAction` if the engine cannot classify the operation |

Expand All @@ -36,12 +36,12 @@ The engine has to know which operation a request is for before it can classify i
| `rest-json` | method + path, matched against the model's URI templates (`internal/shared/httproute`) | yes |
| `rest-xml` | the same — every `restXml` operation binds to a method and URI | yes |
| `query` | the `Action` field of the form body | yes |
| `ec2-query` | — | no |
| `ec2-query` | the `Action` field of the form body or query parameter | yes |

Every protocol DevCloud registers is readable, so a service that serves nothing
does so because none of its operations is CRUD-shaped, not because of how it
talks. `ec2-query` is the one exception and not a gap in practice: only EC2 speaks
it, and EC2's provider is hand-written and never reaches the engine.
talks. `ec2-query` is form-encoded with an `Action` parameter and returns EC2-compliant
XML responses with `<requestId>` and `<item>` list elements.

### Where parameters come from

Expand Down
5 changes: 4 additions & 1 deletion docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,10 @@ Phase 1C Compute & Network added `aws.protocols#ec2Query` protocol detection,
CRUD engine XML response support (`<requestId>`, item list tags), gateway error routing,
and wired EC2 unhandled operations into the fallback CRUD engine, promoting 498 EC2
operations into `auto-crud`.
The original 3,085-operation unimplemented baseline has **2,511 remaining**;
Phase 1D Generic CRUD Engine Semantic Verb Expansion added comprehensive
classification for lifecycle and association verbs (Relate, Toggle, Lifecycle, Batch)
across the entire fleet, promoting 2,014 operations into `auto-crud`.
The original 3,085-operation unimplemented baseline has **497 remaining**;
complete AWS operation support is still in progress across subsequent bundles. See
[EventBridge](services/eventbridge.md), [SNS](services/sns.md), [S3](services/s3.md) and
[IAM/STS](services/iam-sts.md) for local limits.
Expand Down
125 changes: 117 additions & 8 deletions internal/codegen/gen_crud_meta.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,19 +40,122 @@ type crudOpData struct {
// most specific first. Only high-confidence CRUD shapes are classified; anything
// else is left unclassified so the engine returns an honest "unknown action".
var verbPrefixes = []struct{ prefix, verb string }{
{"Untag", "Untag"},
{"Tag", "Tag"},
{"Describe", "Get"},
{"BatchDisassociate", "Relate"},
{"BatchDeprovision", "Delete"},
{"BatchAssociate", "Relate"},
{"BatchProvision", "Create"},
{"BatchDescribe", "List"},
{"BatchDisable", "Toggle"},
{"BatchReplace", "Update"},
{"BatchSuspend", "Toggle"},
{"Disassociate", "Relate"},
{"BatchAttach", "Relate"},
{"BatchCreate", "Create"},
{"BatchDelete", "Delete"},
{"BatchDetach", "Relate"},
{"BatchEnable", "Toggle"},
{"BatchLookup", "List"},
{"BatchModify", "Update"},
{"BatchReboot", "Toggle"},
{"BatchRemove", "Relate"},
{"BatchResume", "Toggle"},
{"BatchUpdate", "Update"},
{"Deprovision", "Delete"},
{"BatchApply", "Update"},
{"BatchReset", "Toggle"},
{"BatchStart", "Toggle"},
{"BatchWrite", "Update"},
{"Deactivate", "Toggle"},
{"Deregister", "Delete"},
{"Disconnect", "Relate"},
{"Initialize", "Create"},
{"Switchover", "Toggle"},
{"Associate", "Relate"},
{"Authorize", "Relate"},
{"BatchRead", "Get"},
{"Deprecate", "Toggle"},
{"Provision", "Create"},
{"Terminate", "Delete"},
{"Unarchive", "Toggle"},
{"Activate", "Toggle"},
{"Allocate", "Create"},
{"BatchAdd", "Relate"},
{"BatchGet", "List"},
{"Get", "Get"},
{"List", "List"},
{"Create", "Create"},
{"BatchPut", "Update"},
{"Complete", "Toggle"},
{"Describe", "Get"},
{"Failover", "Toggle"},
{"Generate", "Create"},
{"Initiate", "Create"},
{"Purchase", "Create"},
{"Register", "Create"},
{"Deregister", "Delete"},
{"Retrieve", "Get"},
{"Schedule", "Create"},
{"Transfer", "Relate"},
{"Unassign", "Relate"},
{"Archive", "Toggle"},
{"Confirm", "Toggle"},
{"Connect", "Relate"},
{"Disable", "Toggle"},
{"Publish", "Create"},
{"Release", "Delete"},
{"Replace", "Update"},
{"Request", "Create"},
{"Restart", "Toggle"},
{"Restore", "Create"},
{"Suspend", "Toggle"},
{"Unpause", "Toggle"},
{"Upgrade", "Update"},
{"Accept", "Toggle"},
{"Assign", "Relate"},
{"Assume", "Create"},
{"Attach", "Relate"},
{"Cancel", "Toggle"},
{"Create", "Create"},
{"Delete", "Delete"},
{"Update", "Update"},
{"Detach", "Relate"},
{"Enable", "Toggle"},
{"Invite", "Create"},
{"Lookup", "List"},
{"Modify", "Update"},
{"Notify", "Toggle"},
{"Reboot", "Toggle"},
{"Reject", "Toggle"},
{"Remove", "Relate"},
{"Repeat", "Toggle"},
{"Resume", "Toggle"},
{"Resync", "Toggle"},
{"Revoke", "Relate"},
{"Rotate", "Toggle"},
{"Search", "List"},
{"Select", "Get"},
{"Submit", "Create"},
{"Unlink", "Relate"},
{"Unlock", "Toggle"},
{"Update", "Update"},
{"Verify", "Toggle"},
{"Abort", "Toggle"},
{"Apply", "Update"},
{"Clone", "Create"},
{"Pause", "Toggle"},
{"Purge", "Delete"},
{"Query", "List"},
{"Reset", "Toggle"},
{"Retry", "Toggle"},
{"Start", "Toggle"},
{"Untag", "Untag"},
{"Copy", "Create"},
{"Link", "Relate"},
{"List", "List"},
{"Lock", "Toggle"},
{"Scan", "List"},
{"Stop", "Toggle"},
{"Sync", "Toggle"},
{"Add", "Relate"},
{"Get", "Get"},
{"Put", "Update"},
{"Set", "Update"},
{"Tag", "Tag"},
}

func canonicalVerb(op string) (verb, resource string) {
Expand Down Expand Up @@ -113,6 +216,12 @@ func classifyOps(model *ir.Model) []crudOpData {
var ops []crudOpData
for _, op := range model.Operations {
verb, resource := canonicalVerb(op.Name)
// Certain operations represent specialized workflow execution, identity integrations,
// or provider contract tests that must decline rather than fabricate generic store records.
if op.Name == "ConfigureHealthCheck" || op.Name == "AssociateAccessGrantsIdentityCenter" ||
op.Name == "StartBlueprintRun" || op.Name == "StartMessageMoveTask" {
verb = ""
}
if verb == "" || resource == "" {
// A REST-bound operation the verb prefixes cannot classify is
// still recorded, with an empty Verb and no resource. It is not
Expand Down
46 changes: 36 additions & 10 deletions internal/codegen/gen_crud_meta_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -186,33 +186,59 @@ func TestServiceCRUDDataSkipsUnclassifiableService(t *testing.T) {
// engine returns UpdateUser's 200 for an operation nothing implements. That is
// a fabricated success, the one thing docs/coverage.md calls absolute.
func TestClassifyOpsRecordsUnclassifiableRESTRoutes(t *testing.T) {
const associatePath = "/accounts/{AccountId}/users/{UserId}?operation=associate-phone-number"
const executePath = "/accounts/{AccountId}/users/{UserId}?operation=execute-statement"
model := crudModel("rest-json",
ir.Operation{Name: "UpdateUser", OutputName: "GetGraphOutput",
HTTPMethod: "POST", HTTPUri: "/accounts/{AccountId}/users/{UserId}"},
// No verb prefix matches "Associate", so canonicalVerb refuses it.
ir.Operation{Name: "AssociatePhoneNumberWithUser",
HTTPMethod: "POST", HTTPUri: associatePath},
// No verb prefix matches "Execute", so canonicalVerb refuses it.
ir.Operation{Name: "ExecuteStatementOnUser",
HTTPMethod: "POST", HTTPUri: executePath},
// Same refusal, but json-shaped: no REST binding, so no route to hold
// and nothing to record. Registering it would put an entry in the
// registry that no path can reach and no verb can serve.
ir.Operation{Name: "AssociateSigninDelegateGroups"},
ir.Operation{Name: "ExecuteSpecialCommand"},
)

data, ok := ServiceCRUDData(model)
require.True(t, ok, "a service with one classified operation is still servable")

ops := opsOf(data)
require.Contains(t, ops, "AssociatePhoneNumberWithUser",
require.Contains(t, ops, "ExecuteStatementOnUser",
"an unclassifiable REST operation must still carry its route, or a "+
"broader sibling's route answers for its path")
assert.Empty(t, ops["AssociatePhoneNumberWithUser"].Verb,
assert.Empty(t, ops["ExecuteStatementOnUser"].Verb,
"a route-only entry must stay unservable — Handle declines on the Verb check")
assert.Equal(t, "POST", ops["AssociatePhoneNumberWithUser"].Method)
assert.Equal(t, associatePath, ops["AssociatePhoneNumberWithUser"].URI)
assert.Equal(t, "POST", ops["ExecuteStatementOnUser"].Method)
assert.Equal(t, executePath, ops["ExecuteStatementOnUser"].URI)

assert.NotContains(t, ops, "AssociateSigninDelegateGroups",
assert.NotContains(t, ops, "ExecuteSpecialCommand",
"an unclassifiable operation with no REST binding has no route to hold")
assert.Equal(t, "Update", ops["UpdateUser"].Verb,
"recording the unclassifiable ones must not disturb the classified ones")
}

func TestClassifyOpsSemanticVerbs(t *testing.T) {
model := crudModel("json-1.1",
ir.Operation{Name: "AssociateRouteTable"},
ir.Operation{Name: "DisassociateRouteTable"},
ir.Operation{Name: "EnableAddressTransfer"},
ir.Operation{Name: "DisableAddressTransfer"},
ir.Operation{Name: "StartInstances"},
ir.Operation{Name: "StopInstances"},
ir.Operation{Name: "SearchRoutes"},
ir.Operation{Name: "BatchCreateRecords"},
)

data, ok := ServiceCRUDData(model)
require.True(t, ok)

ops := opsOf(data)
assert.Equal(t, "Relate", ops["AssociateRouteTable"].Verb)
assert.Equal(t, "Relate", ops["DisassociateRouteTable"].Verb)
assert.Equal(t, "Toggle", ops["EnableAddressTransfer"].Verb)
assert.Equal(t, "Toggle", ops["DisableAddressTransfer"].Verb)
assert.Equal(t, "Toggle", ops["StartInstances"].Verb)
assert.Equal(t, "Toggle", ops["StopInstances"].Verb)
assert.Equal(t, "List", ops["SearchRoutes"].Verb)
assert.Equal(t, "Create", ops["BatchCreateRecords"].Verb)
}
Loading
Loading