feat: multi-CSP groundwork — IR, ModelSource, provider config, auth adapters (Phase 2)

changes/unreleased/Added-20260905-135309.yaml

@@ -0,0 +1,5 @@
+kind: Added
+body: 'Provider-namespaced configuration: service blocks can be written as `providers.aws.services.*`, forward-compatible with `providers.azure.*`. The top-level `services` block is the same block under its historical name and keeps working; a block for an unserved provider loads with a warning instead of failing'
+time: 2026-09-05T13:53:09.000000+09:00
+custom:
+    Issue: "135"

changes/unreleased/Added-20260905-135310.yaml

@@ -0,0 +1,5 @@
+kind: Added
+body: 'Per-provider auth adapters (`internal/auth`): SigV4 credentials are parsed once and the caller''s claimed identity — access key, region, signing name, session token — is available to every plugin via `auth.FromContext`, and reported as a new `region` field on `GET /devcloud/api/logs`. Nothing is verified; DevCloud still accepts any credentials'
+time: 2026-09-05T13:53:10.000000+09:00
+custom:
+    Issue: "135"

changes/unreleased/Changed-20260905-135311.yaml

@@ -0,0 +1,5 @@
+kind: Changed
+body: 'Codegen now runs through a provider-neutral intermediate representation (`internal/codegen/ir`) behind a `ModelSource` interface, so a second API description format is an added file rather than a rewrite. Smithy is the first source; generated output is byte-identical'
+time: 2026-09-05T13:53:11.000000+09:00
+custom:
+    Issue: "135"

cmd/codegen/main.go

@@ -52,8 +52,14 @@ func main() {
 	// called incomplete generation clean.
 	skipped := false
 
+	// The driver knows nothing about model formats: it hands every file to
+	// codegen.SourceFor and generates from whatever IR comes back. Adding
+	// OpenAPI or Protobuf support is a new ModelSource, not a change here.
+	// A file no source claims is an error rather than a quiet skip — the
+	// models directory holds models, and a silently ignored one produces
+	// generated output that is wrong without saying so.
 	for _, entry := range entries {
-		if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".json") {
+		if entry.IsDir() {
 			continue
 		}
 
@@ -64,9 +70,16 @@ func main() {
 			continue
 		}
 
-		model, err := codegen.ParseSmithyJSON(data)
+		source, err := codegen.SourceFor(entry.Name(), data)
 		if err != nil {
-			fmt.Fprintf(os.Stderr, "Error parsing %s: %v\n", entry.Name(), err)
+			fmt.Fprintf(os.Stderr, "Error: %v\n", err)
+			skipped = true
+			continue
+		}
+
+		model, err := source.Parse(data)
+		if err != nil {
+			fmt.Fprintf(os.Stderr, "Error parsing %s as %s: %v\n", entry.Name(), source.Name(), err)
 			skipped = true
 			continue
 		}

cmd/devcloud/main.go

@@ -62,8 +62,20 @@ func main() {
 
 	registry := plugin.DefaultRegistry
 
+	// A service's provider comes from the plugin itself, so config resolution is
+	// provider-aware without the config package needing to know which services
+	// exist. Construct builds an instance without Init, which is all the
+	// metadata methods need.
+	providerOf := func(name string) string {
+		p, ok := registry.Construct(name)
+		if !ok {
+			return plugin.DefaultProvider
+		}
+		return plugin.ProviderOf(p)
+	}
+
 	initService := func(name string, fatal bool) {
-		svcCfg := cfg.Service(name)
+		svcCfg := cfg.ProviderService(providerOf(name), name)
 		if !svcCfg.Enabled {
 			return
 		}

docs/architecture.md

@@ -6,12 +6,24 @@ DevCloud's long-term direction is to support multiple Cloud Service Providers (A
 
 A phased refactor is planned (see [roadmap.md](roadmap.md)):
 
-- **Phase 1 (complete, shipped as v1.0)** — AWS services via Smithy codegen, single-port gateway, AWS SigV4 auth.
-- **Phase 2 (current)** — Introduce an Intermediate Representation (IR) between API models and codegen. Abstract `ModelSource` so OpenAPI (Azure) and Protocol Buffers / Discovery Documents (GCP) can feed the same pipeline. Extract a per-provider auth adapter interface.
+- **Phase 1 (complete, shipped as v1.0)** — AWS services via Smithy codegen, single-port gateway.
+- **Phase 2 (complete)** — Intermediate Representation (IR) between API models and codegen; `ModelSource` so OpenAPI (Azure) and Protocol Buffers / Discovery Documents (GCP) can feed the same pipeline; provider namespacing in config; per-provider auth adapters.
 - **Phase 3 (pilot)** — First non-AWS service (candidate: Azure Blob Storage) validates the multi-CSP architecture.
 - **Phase 4 (breadth)** — Additional services across CSPs; community-owned providers.
 
-The existing plugin system, protocol detector, and storage abstractions are intentionally CSP-agnostic where possible, to minimize rework when providers are added.
+After Phase 2, the four places a second CSP would have touched are each an
+addition rather than an edit:
+
+| Seam | Package | How a provider joins |
+|---|---|---|
+| Model → code | [`internal/codegen/ir`](../internal/codegen/ir/ir.go), [`source.go`](../internal/codegen/source.go) | Implement `ModelSource`, append it to `DefaultSources` |
+| Configuration | [`internal/config`](../internal/config/config.go) | `providers.<name>.services.*`; add the name to `knownProviders` |
+| Service contract | [`internal/plugin`](../internal/plugin/plugin.go) | Implement the optional `ProviderScoped` on the plugin |
+| Credentials | [`internal/auth`](../internal/auth/auth.go) | Implement `Adapter`, append it to `Adapters` |
+
+The plugin system, protocol detector, and storage abstractions stay CSP-agnostic
+by convention rather than by enforcement — `ProtocolType` is an open string type,
+not an enum, for exactly this reason.
 
 ## Request Flow
 
@@ -27,6 +39,7 @@ API Gateway (port 4747)
   │   ├─ CORS (cross-origin handling)
   │   ├─ RequestID (X-Amz-Request-Id)
   │   ├─ RequestLogger (structured logging)
+  │   ├─ Identity (auth.Adapter → claimed identity on the request context)
   │   └─ LogCollector (admin API live logs)
   │
   ├─ Route: /devcloud/api/* → Admin API
@@ -36,13 +49,10 @@ Protocol Detector
   │
   ├─ X-Amz-Target header present       → JSON protocol (DynamoDB, SQS JSON)
   ├─ Content-Type: x-www-form-urlencoded + Action= → Query protocol (IAM, STS, SQS Query)
-  ├─ SigV4 with Lambda path            → REST-JSON (Lambda)
+  ├─ SigV4 credential scope            → REST-JSON (Lambda, and the signing name decides the service)
   └─ Default                           → REST-XML (S3)
   │
   ▼
-Auth Validator (SigV4 format check, account ID extraction)
-  │
-  ▼
 Service Router → Plugin Registry → ServicePlugin.HandleRequest()
   │
   ▼
@@ -83,18 +93,42 @@ The full contract for each method, the configuration keys, the error convention,
 
 SQS supports both Query and JSON protocols. The protocol is auto-detected per request based on Content-Type and headers.
 
-## Smithy Code Generation
+## Authentication
+
+DevCloud reads credentials and never verifies them. `internal/auth` holds one
+`Adapter` per provider — `SigV4` for AWS today — which parses the credential
+scope off the `Authorization` header or a presigned URL's query string and
+reports the access key, region, signing name and session token the caller
+claimed. `IdentityMiddleware` puts the result on the request context, where any
+plugin can read it with `auth.FromContext`, and the protocol detector uses the
+same parse to route by signing name so routing and identity cannot disagree.
 
-DevCloud auto-generates Go code from AWS Smithy JSON models. This enables rapid tracking of AWS API changes with minimal manual work.
+Signature validation is deliberately absent: a local server that rejected calls
+whose signature did not match would break every SDK pointed at it with
+placeholder credentials. See
+[compatibility-policy.md](compatibility-policy.md#not-guaranteed).
+
+## Code Generation
+
+DevCloud auto-generates Go code from API models. This enables rapid tracking of
+API changes with minimal manual work. AWS Smithy is the only format read today;
+the pipeline is built so a second one is an added file rather than a rewrite.
 
 ### Pipeline
 
 ```
 smithy-models/*.json  (AWS Smithy model files)
        │
        ▼
-  Parser (internal/codegen/parser.go)
-  Reads service definitions, operations, input/output shapes
+  ModelSource (internal/codegen/source.go)
+  Each source detects its own format and parses it. SmithySource is
+  the first; OpenAPI and Protobuf join by implementing the interface
+  and appending to DefaultSources. cmd/codegen names no format.
+       │
+       ▼
+  IR — *ir.Model (internal/codegen/ir)
+  Provider-neutral: service id, protocol, operations, shapes.
+  Everything downstream reads only this.
        │
        ▼
   Generator (internal/codegen/generator.go)
@@ -159,9 +193,11 @@ devcloud/
 │   └── codegen/            # Smithy code generator CLI (main.go)
 ├── internal/
 │   ├── gateway/            # HTTP server, middleware, protocol detection, routing
-│   ├── plugin/             # ServicePlugin interface, Registry
-│   ├── codegen/            # Smithy parser, code generators, templates
-│   ├── config/             # YAML config loading, env overrides
+│   ├── plugin/             # ServicePlugin interface, Registry, ProviderScoped
+│   ├── auth/               # Per-provider credential adapters (SigV4 today)
+│   ├── codegen/            # ModelSource implementations, generators, templates
+│   │   └── ir/             # Provider-neutral intermediate representation
+│   ├── config/             # YAML config loading, provider namespacing, env overrides
 │   ├── generated/          # Auto-generated code (DO NOT EDIT; run `make stats` for count)
 │   ├── services/           # Service implementations (run `make stats` for count)
 │   ├── admin/              # Admin REST API

docs/compatibility-policy.md

@@ -24,6 +24,7 @@ removed or repurposed. Defined in [`internal/config/config.go`](../internal/conf
 | `services` | map | Presence of the block is authoritative — only the services it lists run, and an empty block runs nothing. Absent means every registered service runs. |
 | `services.<id>.enabled` | bool | Whether that service starts. |
 | `services.<id>.data_dir` | string | Where that service stores data. |
+| `providers.aws.services` | map | The same block as `services`, namespaced by provider. Identical semantics; wins over `services` when both are written, with a warning. |
 | `admin.enabled` | bool | Whether the admin API is served. Default `false`. |
 | `logging.level` | string | Log level. |
 | `logging.format` | string | Log format. |
@@ -131,8 +132,10 @@ Depending on any of the following will break, and breaking it is **not** a major
   normalizing it is a minor release, not a major one.
 - **Log output.** Format, levels and wording of server logs are operational, not an API.
 - **Everything under `internal/`.** Go forbids importing it from another module, and DevCloud
-  reserves the right to restructure it freely across 1.x — explicitly including the planned
-  intermediate representation and `ModelSource` work on the [roadmap](roadmap.md). Internal
+  reserves the right to restructure it freely across 1.x. The Phase 2 refactor on the
+  [roadmap](roadmap.md) is the precedent: the intermediate representation, `ModelSource`,
+  `ProviderScoped` and the `auth` adapters all landed inside a 1.x minor without a
+  compatibility event, because none of them is reachable from outside this module. Internal
   churn is not a compatibility event. The in-tree `ServicePlugin` contract in
   [plugin-api.md](plugin-api.md#api-stability) is not an exception to this: it is a convention
   that keeps in-tree plugins compiling, and it does not gate release versioning.

docs/configuration.md

@@ -45,6 +45,40 @@ Service-specific options:
 | `services.lambda.warm_containers` | `0` | Number of warm containers to keep |
 | `services.iam.enforce_policies` | `false` | Enforce IAM policies (experimental) |
 
+### Provider namespacing
+
+DevCloud serves AWS today and is being prepared to serve more (see the
+[roadmap](roadmap.md)). Service configuration can therefore be written under an
+explicit provider:
+
+```yaml
+providers:
+  aws:
+    services:
+      s3:
+        enabled: true
+        data_dir: ./data/s3
+```
+
+`providers.aws.services` and the top-level `services` block are **the same
+block** — one under its forward-compatible name, one under its historical one.
+Rules:
+
+- Write either. The top-level `services` block keeps working exactly as before
+  and is not deprecated.
+- If both are present, `providers.aws.services` wins and a warning names the
+  ignored block. Do not write both.
+- A block for a provider this build does not serve (`providers.azure`) parses
+  without error — a config written for a later DevCloud still loads — and logs
+  a warning saying it is ignored. That warning is also your typo check:
+  `providers.awss` produces it.
+- `DEVCLOUD_SERVICES` and `DEVCLOUD_DATA_DIR` are AWS-scoped. There is no
+  syntax for naming another provider's services, so they cannot silently
+  disable one.
+- AWS data directories stay flat (`<base>/<id>`). Any future provider nests
+  under its own name (`<base>/<provider>/<id>`) so two CSPs offering a
+  same-named service cannot collide.
+
 ### Admin API
 
 | Key | Default | Description |

docs/plugin-api.md

@@ -38,6 +38,60 @@ type ServicePlugin interface {
 These invariants are enforced for every registered service by
 [`TestServicePluginConformance`](../cmd/devcloud/conformance_test.go).
 
+## Providers and CSP neutrality
+
+Phase 2 of the [roadmap](roadmap.md) reviewed this interface for CSP neutrality.
+The conclusion was that **`ServicePlugin` needs no change**: every method is
+about serving an HTTP API, and none of them assumes AWS.
+
+What is AWS-specific is the vocabulary around it, and each piece is open rather
+than closed:
+
+| Surface | Why it is already neutral |
+|---|---|
+| `ProtocolType` | An open `string` type, not an enum. The constants are AWS's wire protocols; another CSP declares its own values. |
+| `ServiceID()` | An opaque registry key. A non-AWS service registers under whatever id it wants. |
+| `DefaultAccountID` | An AWS concept used only by AWS services. Nothing in the interface refers to it. |
+
+The one thing missing was a way to ask *which CSP a plugin serves*. That is the
+optional `ProviderScoped` interface:
+
+```go
+type ProviderScoped interface {
+    Provider() string
+}
+
+func ProviderOf(p ServicePlugin) string // defaults to plugin.DefaultProvider ("aws")
+```
+
+It is **not** a `ServicePlugin` method on purpose. Adding one would force an
+edit to every service in the tree so it could state the value it already
+defaults to, and would break the [v1.x contract](#api-stability) below for no
+gain. Always read the provider through `ProviderOf`, never by type-asserting.
+
+Startup uses it to resolve configuration: `cmd/devcloud` looks up each service
+with `cfg.ProviderService(plugin.ProviderOf(p), id)`, so a plugin lands in its
+own provider's config namespace without the config package needing to know which
+services exist. See [configuration.md](configuration.md#provider-namespacing).
+
+## Caller identity
+
+The gateway parses whatever credentials a request carried and attaches the
+result to the request context. Inside `HandleRequest`:
+
+```go
+if id, ok := auth.FromContext(ctx); ok {
+    region := id.Region // what the SDK signed for; "" if unsigned
+}
+```
+
+`auth.Identity` carries `Provider`, `AccessKeyID`, `Region`, `Service` (the
+signing name as presented) and `SessionToken`. **None of it is verified** —
+DevCloud accepts any credentials, so treat these as a claim, never as an
+authorization decision. Adapters live in [`internal/auth`](../internal/auth/);
+`SigV4` is AWS's, and Azure (AAD/SAS) and GCP (OAuth2) join by implementing
+`auth.Adapter`.
+
 ## Protocols
 
 `Protocol()` returns the wire protocol the gateway uses to detect and serialize
@@ -146,4 +200,5 @@ and the `ProtocolType` constants are stable within the `v1.x` series:
   governed by [compatibility-policy.md](compatibility-policy.md).
 
 New optional capability is introduced additively (new `Options` keys, new
-`ProtocolType` values) so existing plugins keep compiling and behaving.
+`ProtocolType` values, optional side interfaces such as `ProviderScoped`) so
+existing plugins keep compiling and behaving.

docs/roadmap.md

@@ -29,17 +29,19 @@ We pursue this vision through a **phased rollout** to manage scope, architectura
 - [x] Generic CRUD fallback engine ([crud-engine.md](crud-engine.md)) auto-serves ~2,200 CRUD-shaped operations across all 46 JSON-protocol services with plausible, store-backed responses; every registered JSON service is wired. **Follow-up**: promote high-value auto-crud ops to hand-verified fidelity.
 - [x] v1.0 release — see [compatibility-policy.md](compatibility-policy.md) for what 1.x guarantees
 
-### Phase 2 — Architectural Preparation (Current, v1.x)
+### Phase 2 — Architectural Preparation (Complete, v1.x)
 
 **Goal**: internal refactor so adding a new CSP doesn't require forking the project.
 
-- [ ] Introduce Intermediate Representation (IR) between API models and codegen
-- [ ] Refactor `internal/codegen/parser.go` behind a `ModelSource` interface (Smithy being the first implementation; OpenAPI/Protobuf to follow)
-- [ ] Provider namespacing in config (`providers.aws.*`, forward-compatible with `providers.azure.*`)
-- [ ] Plugin interface review — ensure `ServicePlugin` is CSP-agnostic
-- [ ] Per-provider auth adapter interface (SigV4, AAD/SAS, OAuth2)
+- [x] Intermediate Representation (IR) between API models and codegen — [`internal/codegen/ir`](../internal/codegen/ir/ir.go). The generators read `*ir.Model` and nothing in the IR names Smithy; `ir.Model.Provider` carries the owning CSP.
+- [x] `internal/codegen/parser.go` refactored behind a [`ModelSource`](../internal/codegen/source.go) interface. `SmithySource` is the first implementation and owns its own format detection, so `cmd/codegen` hands every file to `SourceFor` and never names a format. OpenAPI/Protobuf are an added file.
+- [x] Provider namespacing in config — `providers.aws.services.*`, forward-compatible with `providers.azure.*`. The top-level `services` block is the same AWS block under its historical name and keeps working ([configuration.md](configuration.md#provider-namespacing)).
+- [x] Plugin interface review — `ServicePlugin` needs no change; the CSP is carried by the optional [`ProviderScoped`](plugin-api.md#providers-and-csp-neutrality) interface, read through `plugin.ProviderOf`. Adding a method would have broken every in-tree plugin to make it state a value it already defaults to.
+- [x] Per-provider auth adapter interface — [`internal/auth`](../internal/auth/auth.go). `Adapter` reads one provider's credential form; `SigV4` is the AWS implementation, and AAD/SAS and OAuth2 slot in beside it without touching the gateway. The caller's claimed identity reaches plugins via `auth.FromContext`.
 
-### Phase 3 — First Non-AWS Service (v2.0, exploratory)
+What Phase 2 deliberately did **not** do: no non-AWS service, no second `ModelSource`, and no signature verification. Those are Phase 3 and beyond — Phase 2's job was to make each of them an addition rather than a fork.
+
+### Phase 3 — First Non-AWS Service (Next, v2.0, exploratory)
 
 **Goal**: validate the multi-CSP architecture with a single, well-scoped pilot.
 
@@ -76,6 +78,6 @@ We pursue this vision through a **phased rollout** to manage scope, architectura
 | Version | Focus |
 |---------|-------|
 | 0.x | AWS services, unstable API |
-| 1.x ← current | AWS depth, stable plugin API |
+| 1.x ← current | AWS depth, stable plugin API, multi-CSP groundwork (IR, `ModelSource`, provider namespacing, auth adapters) |
 | 2.x | Multi-CSP architecture, Azure pilot |
 | 3.x+ | Broad CSP coverage, community-owned providers |

internal/admin/logger.go

@@ -15,6 +15,10 @@ type RequestLog struct {
 	Duration  string    `json:"duration"` // e.g. "1.234ms"
 	Timestamp time.Time `json:"timestamp"`
 	Service   string    `json:"service"` // detected service ID
+	// Region is the region the caller signed for, read off the request's
+	// credential scope. Empty when the client did not sign — DevCloud is
+	// regionless, so this reports what the SDK believed, not where data lives.
+	Region string `json:"region"`
 }
 
 // LogCollector is a thread-safe ring buffer for recent request logs.