feat: Phase 1 AWS depth & stabilization for v1.0

README.md

@@ -23,8 +23,8 @@ Today DevCloud targets **AWS**. Our long-term goal is to support the full range
 
 ## Features
 
-- **101 AWS services scaffolded** — 4,400+ operations across all major categories (run `make stats` for current counts; see [services-matrix.md](docs/services-matrix.md))
-- **~96% boto3 compatibility** — run `make test-compat` for the current pass rate; works with most boto3 apps
+- **104 AWS services** across all major categories (run `make stats` for current counts; see [services-matrix.md](docs/services-matrix.md))
+- **boto3-compatible** — a 729-test compatibility suite runs in CI (`make test-compat`); works with most boto3 apps, and unsupported operations return a clean AWS error, never a false success
 - **Cross-service integration** — CFN provisioning, DDB Streams → Lambda, EventBridge targets, S3 → Lambda
 - **Smithy-driven codegen** — auto-generate Go interfaces, types, and serializers from Smithy models
 - **Weekly auto-sync** — GitHub Actions keeps generated code up to date with upstream AWS API changes
@@ -60,7 +60,8 @@ Start at the [docs index](docs/) for the full map. Quick links:
 - [Getting Started](docs/getting-started.md) — Installation, first use, boto3 / AWS CLI / Terraform examples
 - [Configuration](docs/configuration.md) — Config options, env-var overrides, tier shortcuts
 - [Architecture](docs/architecture.md) — System design, codegen pipeline, plugin model, multi-CSP vision
-- [Services Matrix](docs/services-matrix.md) — 101 services, coverage status, boto3 pass rate
+- [Services Matrix](docs/services-matrix.md) — services, coverage status, boto3 compatibility
+- [Plugin API](docs/plugin-api.md) — the stable `ServicePlugin` contract and v1.x stability policy
 - [Roadmap](docs/roadmap.md) — Phased plan toward multi-CSP support
 - [FAQ](docs/faq.md) / [Troubleshooting](docs/troubleshooting.md) — Common questions and errors
 - [Contributing](docs/contributing.md) — Development setup, adding new services

changes/unreleased/Added-20260718-222651.yaml

@@ -0,0 +1,5 @@
+kind: Added
+body: Document the stable `ServicePlugin` plugin API in docs/plugin-api.md (method contracts, config keys, error convention, and the v1.x stability policy), enforced by a conformance test over every registered service
+time: 2026-07-18T22:26:51.495862+09:00
+custom:
+    Issue: "96"

changes/unreleased/Added-20260718-222652.yaml

@@ -0,0 +1,5 @@
+kind: Added
+body: Enable and add boto3 compatibility tests for CodeConnections, DMS, and Verified Permissions (previously scaffolded but not enabled)
+time: 2026-07-18T22:26:52.000000+09:00
+custom:
+    Issue: "96"

changes/unreleased/Fixed-20260718-222651.yaml

@@ -0,0 +1,5 @@
+kind: Fixed
+body: Correct `X-Amz-Target` service routing for AWS services whose target prefix is dotted (e.g. CodeConnections, DMS); they previously misrouted to CloudTrail via an overbroad `com` prefix match
+time: 2026-07-18T22:26:51.484673+09:00
+custom:
+    Issue: "96"

changes/unreleased/Fixed-20260718-222652.yaml

@@ -0,0 +1,5 @@
+kind: Fixed
+body: Unimplemented operations now return a `NotImplemented` error (HTTP 501) instead of a misleading empty `200` success, so SDKs no longer see a false success and can tell a not-yet-emulated real operation apart from a genuinely invalid action; applied uniformly across all service providers (the ChangeResourceRecordSets record-action validation path keeps `InvalidChangeBatch`)
+time: 2026-07-18T22:26:52.000000+09:00
+custom:
+    Issue: "96"

changes/unreleased/Removed-20260718-222651.yaml

@@ -0,0 +1,5 @@
+kind: Removed
+body: Remove the dead, unregistered STS provider stub (`internal/services/sts`); STS is served by the IAM package
+time: 2026-07-18T22:26:51.501604+09:00
+custom:
+    Issue: "96"

cmd/devcloud/conformance_test.go

@@ -0,0 +1,63 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package main
+
+import (
+	"testing"
+
+	"github.com/skyoo2003/devcloud/internal/plugin"
+)
+
+// validProtocols is the closed set of wire protocols a ServicePlugin may declare.
+var validProtocols = map[plugin.ProtocolType]bool{
+	plugin.ProtocolRESTXML:  true,
+	plugin.ProtocolRESTJSON: true,
+	plugin.ProtocolJSON10:   true,
+	plugin.ProtocolJSON11:   true,
+	plugin.ProtocolQuery:    true,
+}
+
+// TestServicePluginConformance locks the stable ServicePlugin contract against
+// every service registered via the blank imports in imports.go. It runs after
+// package init, so DefaultRegistry holds the full plugin surface. See
+// docs/plugin-api.md for the contract this enforces.
+func TestServicePluginConformance(t *testing.T) {
+	ids := plugin.DefaultRegistry.RegisteredServices()
+
+	// Conservative floor. Broken registration wiring (a mangled imports.go,
+	// init-order regression, a dropped Register) collapses the live surface far
+	// below this, so a constant catches it without the fragility of parsing
+	// imports.go at runtime. Raise it if the real count ever nears it.
+	const minServices = 50
+	if len(ids) < minServices {
+		t.Fatalf("registered %d services, want >= %d", len(ids), minServices)
+	}
+
+	var prev string
+	for i, id := range ids {
+		if id == "" {
+			t.Error("RegisteredServices() contains an empty service ID")
+			continue
+		}
+		// RegisteredServices() is documented to return a sorted slice.
+		if i > 0 && id < prev {
+			t.Errorf("RegisteredServices() not sorted: %q came before %q", prev, id)
+		}
+		prev = id
+
+		p, ok := plugin.DefaultRegistry.Construct(id)
+		if !ok {
+			t.Errorf("%s: registered but Construct returned not-found", id)
+			continue
+		}
+		if got := p.ServiceID(); got != id {
+			t.Errorf("%s: ServiceID() = %q, want the registry key %q", id, got, id)
+		}
+		if p.ServiceName() == "" {
+			t.Errorf("%s: ServiceName() is empty", id)
+		}
+		if proto := p.Protocol(); !validProtocols[proto] {
+			t.Errorf("%s: Protocol() = %q is not a valid ProtocolType", id, proto)
+		}
+	}
+}

docs/architecture.md

@@ -71,6 +71,8 @@ type ServicePlugin interface {
 
 Plugins are registered in the `Registry` at startup. The gateway routes requests to the correct plugin based on protocol detection and service identification.
 
+The full contract for each method, the configuration keys, the error convention, and the v1.x stability guarantee are documented in [plugin-api.md](plugin-api.md).
+
 ### Supported Protocols
 
 | Protocol | Services | Request Format | Response Format |

docs/contributing.md

@@ -91,7 +91,7 @@ make codegen-s3
    - In-memory for ephemeral data
    - Filesystem for binary blobs
 
-6. **Register the plugin** — create `register.go` that registers the service factory in the plugin registry. Follow existing services as examples.
+6. **Register the plugin** — register the service factory in the plugin registry from an `init()` function. Follow existing services as examples. The `ServicePlugin` interface contract, error convention, and config keys are documented in [plugin-api.md](plugin-api.md).
 
 7. **Update main.go** — add the service initialization in `cmd/devcloud/main.go`
 

docs/plugin-api.md

@@ -0,0 +1,142 @@
+# ServicePlugin API
+
+This document specifies the **`ServicePlugin`** interface — the stable contract
+every DevCloud service implements. As of **v1.0** this interface is a public API:
+see [API stability](#api-stability) for the compatibility guarantee.
+
+The interface and registry live in [`internal/plugin/`](../internal/plugin/).
+Plugins are compiled in-tree and register themselves at startup; there is no
+dynamic/external plugin loading. (The package lives under `internal/` for that
+reason — it is imported by DevCloud's own binaries, not by third-party modules.)
+
+## The interface
+
+```go
+type ServicePlugin interface {
+    ServiceID() string
+    ServiceName() string
+    Protocol() ProtocolType
+    Init(config PluginConfig) error
+    Shutdown(ctx context.Context) error
+    HandleRequest(ctx context.Context, op string, req *http.Request) (*Response, error)
+    ListResources(ctx context.Context) ([]Resource, error)
+    GetMetrics(ctx context.Context) (*ServiceMetrics, error)
+}
+```
+
+### Method contracts
+
+| Method | Contract |
+|--------|----------|
+| `ServiceID()` | Stable lowercase identifier, e.g. `"s3"`. **Must equal the key the plugin is registered under** (the gateway routes by this key). Constant; safe to call before `Init`. |
+| `ServiceName()` | Human-readable name for logs/dashboard. Must be non-empty. Constant; safe before `Init`. |
+| `Protocol()` | One of the [`ProtocolType`](#protocols) constants. Constant; safe before `Init`. |
+| `Init(config)` | Called once at startup before any request. Open stores, read `config.Options`. Return a non-nil error to abort startup (services in the fixed init order) or skip the service (others). |
+| `Shutdown(ctx)` | Called once at graceful shutdown (15s budget). Flush/close resources. Idempotent-friendly. |
+| `HandleRequest(ctx, op, req)` | Handle one API call. `op` is the operation name pre-extracted by the gateway (see [operation names](#operation-names)); it may be `""` for REST protocols, in which case derive it from method+path or `X-Amz-Target`. Return a `*Response`; return an `error` only for unexpected internal failures (the gateway wraps those as a `500 InternalError`). **Model AWS errors as a normal `*Response`** with the right status and error body, not as a Go `error`. |
+| `ListResources(ctx)` | Return the resources this service currently holds (dashboard). May be empty. |
+| `GetMetrics(ctx)` | Return request/error/resource counters (dashboard). May be zero-valued. |
+
+These invariants are enforced for every registered service by
+[`TestServicePluginConformance`](../cmd/devcloud/conformance_test.go).
+
+## Protocols
+
+`Protocol()` returns the wire protocol the gateway uses to detect and serialize
+for the service:
+
+| Constant | Value | Example services |
+|----------|-------|------------------|
+| `ProtocolRESTXML` | `rest-xml` | S3, Route53, CloudFront |
+| `ProtocolRESTJSON` | `rest-json` | ACM, API Gateway, Lambda REST |
+| `ProtocolJSON10` | `json-1.0` | DynamoDB, Kinesis, CodeConnections |
+| `ProtocolJSON11` | `json-1.1` | ECS, Lambda, DMS, many others |
+| `ProtocolQuery` | `query` | IAM, STS, SQS, SNS, RDS, EC2 |
+
+## Operation names
+
+The gateway derives `op` in
+[`extractOperationName`](../internal/gateway/router.go):
+
+- **JSON protocols** — the suffix of the `X-Amz-Target` header after the last
+  `.` (e.g. `DynamoDB_20120810.GetItem` → `GetItem`).
+- **Query protocol** — the `Action` parameter.
+- **REST protocols** — `""`; the operation is implicit in the URL + method, so
+  the plugin resolves it itself.
+
+Service routing uses the target prefix / signing name; see
+[`serviceFromTarget` and `normalizeServiceID`](../internal/gateway/protocol.go).
+Target prefixes may be dotted (e.g.
+`com.amazonaws.codeconnections.CodeConnections_20231201`); the last dotted
+segment is treated as the `ServiceName_Date` token.
+
+## Configuration
+
+```go
+type PluginConfig struct {
+    DataDir string
+    Options map[string]any
+}
+```
+
+- `DataDir` — per-service data directory from config (`data_dir:`).
+- `Options` — cross-cutting values injected by
+  [`buildOptions`](../cmd/devcloud/main.go). Keys a plugin may rely on:
+  - `server_port` (`int`) — the HTTP port, for building resource URLs (e.g. SQS
+    queue URLs). Passed to every service.
+  - `iam_store` — set for `sts` so it shares the IAM store; services needing a
+    peer's store receive it here.
+
+## Error convention
+
+Return AWS-shaped errors as a `*Response`, using the shared helpers in
+[`internal/shared/response.go`](../internal/shared/response.go):
+
+- JSON protocols → `shared.JSONError(code, message, status)` (`{"__type", "message"}`).
+- Query/XML protocols → `shared.QueryXMLError(...)` or the service's XML error helper.
+
+**Unknown / unimplemented operations must return an error, never a false
+success.** The convention is `InvalidAction` with HTTP 400:
+
+```go
+default:
+    return shared.JSONError("InvalidAction", "unknown action: "+op, http.StatusBadRequest), nil
+```
+
+A silent `200 OK {}` for an unimplemented op is a bug: it makes an SDK believe
+the call succeeded.
+
+## Registering a service
+
+Each service registers a factory in its `init()`:
+
+```go
+func init() {
+    plugin.DefaultRegistry.Register("myservice", func() plugin.ServicePlugin {
+        return &MyProvider{}
+    })
+}
+```
+
+The service package is blank-imported in
+[`cmd/devcloud/imports.go`](../cmd/devcloud/imports.go) so its `init()` runs, and
+enabled in [`internal/config/default.yaml`](../internal/config/default.yaml) so
+it is initialized at startup. See [contributing.md](contributing.md) for the
+full "add a service" checklist.
+
+The registry also exposes `RegisteredServices()` (all registered IDs) and
+`Construct(id)` (build an instance without `Init`, for introspection/tests).
+
+## API stability
+
+Starting at **v1.0**, `ServicePlugin`, `PluginConfig`, `Response`, `Resource`,
+`ServiceMetrics`, and the `ProtocolType` constants are stable within the `v1.x`
+series:
+
+- No method will be **removed** from `ServicePlugin` and no existing method
+  **signature** will change in a `v1.x` release.
+- Struct fields will only be **added**, never removed or repurposed.
+- Any breaking change to this contract requires a **major** version bump.
+
+New optional capability is introduced additively (new `Options` keys, new
+`ProtocolType` values) so existing plugins keep compiling and behaving.

docs/roadmap.md

@@ -22,11 +22,11 @@ We pursue this vision through a **phased rollout** to manage scope, architectura
 - [x] Deep hand-written coverage on core, integration, and major extended services (see [services-matrix.md](services-matrix.md) for per-tier depth)
 - [x] Cross-service integration (CloudFormation, DynamoDB Streams → Lambda, SQS → Lambda, S3 → Lambda, EventBridge, SNS → SQS)
 - [x] boto3 compatibility suite (`make test-compat`); core services — S3, SQS, DynamoDB, Lambda, IAM, STS, SNS, CloudWatch, KMS, Secrets Manager, EventBridge, CloudFormation — pass 100%
-- [x] boto3 compatibility ≥ 95% overall (run `make test-compat` for the current rate)
-- [ ] Reduce remaining `NotImplementedError` stubs (less-common operations and scaffolded "Category Expansion" services)
-- [ ] Close remaining boto3 compatibility edge cases in newly-added services
-- [ ] Finalize and document a stable `ServicePlugin` API (`internal/plugin/`)
-- [ ] v1.0 release
+- [x] boto3 compatibility suite green in CI (`make test-compat`) — a failing test fails the build
+- [x] Unimplemented operations return a consistent AWS error (`InvalidAction`, HTTP 400) instead of a false `200` success; dead scaffold code removed
+- [x] boto3 compatibility coverage added for previously-untested services (CodeConnections, DMS, Verified Permissions)
+- [x] Stable `ServicePlugin` API finalized and documented ([plugin-api.md](plugin-api.md)), enforced by a conformance test over every registered service
+- [ ] v1.0 release (pending maintainer tag)
 
 ### Phase 2 — Architectural Preparation (v1.x)
 

docs/services-matrix.md

@@ -1,8 +1,8 @@
 # DevCloud Services Matrix
 
-**Total**: 101 services, 4,460 operations, ~96% boto3 compatibility
+**Total**: 104 services (run `make stats`). The boto3 compatibility suite — 729 tests in `tests/compatibility/` — passes in CI.
 
-> To refresh these numbers, run `make stats` (services/operations) and `make test-compat` (compatibility), then update the line above.
+> To refresh these numbers, run `make stats` (service/handler counts) and `make test-compat` (compatibility suite), then update the line above. Note: `make stats`' "Operations" counts dispatch cases (a service carrying both Query and JSON protocols counts each), not distinct AWS operations.
 
 _Last updated with each release. For unreleased changes, see [CHANGELOG.md](../CHANGELOG.md)._
 
@@ -15,7 +15,7 @@ DevCloud is a Go-based local cloud environment with AWS API compatibility. This
 | Tier 1 (Big 6) | S3, SQS, DynamoDB, Lambda, IAM, STS | 128+ | Core services (SQS at full 23/23 op coverage) |
 | Tier 2 (Integration) | EventBridge, SNS, CW Logs, CloudWatch, KMS, Secrets Manager, SSM, ECR | 157+ | Integration services |
 | Tier 3 (Extended) | EFS, EBS, EC2, Route53, ACM, ECS, Bedrock, Account, Pipes, CloudControl, RGTAPI, AppAutoScaling, Firehose, S3Tables, MWAA, Scheduler, Support, IdentityStore, MediaConvert, Textract, ServerlessRepo, DDB Streams, SFN, Kinesis, CloudFormation | 900+ | Extended platform services, networking, and services requiring custom integration logic |
-| Category Expansion | 40+ additional services | 2,000+ | All remaining AWS services scaffolded from Smithy models |
+| Category Expansion | remaining services | varies | Smithy-scaffolded services with working dispatch — common operations implemented; less-common ones return a clean `InvalidAction` error rather than a false success |
 
 ## Top 25 services (by ops count)
 
@@ -61,15 +61,16 @@ DevCloud is a Go-based local cloud environment with AWS API compatibility. This
 
 ## boto3 compatibility
 
-- Tests: `tests/compatibility/`
-- Pass rate: **~96%** (run `make test-compat` for the current count)
+- Tests: `tests/compatibility/` (729 tests)
+- Status: the full suite passes in CI (`.github/workflows/compat.yml`); any failing test fails the build
 - Run: `make test-compat`
 
-Remaining failures are concentrated in edge cases of newly-added services
-(ARN path parsing in S3Tables, restJson1 jsonName in ServerlessRepo, specialized
-Textract/Support endpoints). Core services (S3, SQS, DynamoDB, Lambda, IAM, STS,
-SNS, CloudWatch, KMS, Secrets Manager, EventBridge, CloudFormation) all pass
-100% of their tests.
+Every service the suite covers passes — including S3Tables (ARN path parsing),
+ServerlessRepo (restJson1 `jsonName`), Textract, and Support, which were once
+rough edges. Core services (S3, SQS, DynamoDB, Lambda, IAM, STS, SNS, CloudWatch,
+KMS, Secrets Manager, EventBridge, CloudFormation) have the deepest coverage.
+Services implement their common operations; less-common operations return a clean
+AWS error rather than a false success, so an SDK always gets a truthful response.
 
 ## Supported protocols
 

internal/config/default.yaml

@@ -304,6 +304,15 @@ services:
   serverlessrepo:
     enabled: true
     data_dir: ./data/serverlessrepo
+  codeconnections:
+    enabled: true
+    data_dir: ./data/codeconnections
+  dms:
+    enabled: true
+    data_dir: ./data/dms
+  verifiedpermissions:
+    enabled: true
+    data_dir: ./data/verifiedpermissions
 
 auth:
   enabled: false