diff --git a/architecture/control-plane/api-versioning.md b/architecture/control-plane/api-versioning.md new file mode 100644 index 0000000..6f323a3 --- /dev/null +++ b/architecture/control-plane/api-versioning.md @@ -0,0 +1,402 @@ +--- +Document Status: ✅ Complete +Document Type: Architecture Reference — API Versioning and Lifecycle +Maps to: udlm/governance/registry-governance.md +--- + +# DCM Data Model — API Versioning Strategy + +> **Implements contracts defined in UDLM**: +> [udlm/governance/registry-governance.md](https://github.com/croadfeldt/udlm/blob/main/governance/registry-governance.md) +> and [udlm/contracts/event-catalog.md](https://github.com/croadfeldt/udlm/blob/main/contracts/event-catalog.md). +> UDLM defines the registry-governance versioning and deprecation lifecycle and +> the event-catalog event-versioning contract. DCM operationalizes a concrete +> versioning strategy across every public API surface — Consumer, Admin, +> Operator Interface (Provider), and Flow GUI — including URL major versioning, +> deprecation lead time, and event schema versioning. + +**Document Status:** ✅ Complete +**Document Type:** Architecture Reference — API Versioning and Lifecycle +**Related Documents:** [Consumer API Specification](../../docs/specifications/consumer-api-spec.md) | [Admin API Specification](../../docs/specifications/dcm-admin-api-spec.md) | [Operator Interface Specification](../../docs/specifications/dcm-operator-interface-spec.md) | [Event Catalog](https://github.com/croadfeldt/udlm/blob/main/contracts/event-catalog.md) | [Registry Governance](https://github.com/croadfeldt/udlm/blob/main/governance/registry-governance.md) | [Design Priorities](https://github.com/croadfeldt/udlm/blob/main/design-principles/design-priorities.md) + +> **This document governs all DCM API surfaces.** Every public API endpoint — Consumer, Admin, Operator Interface (Provider), Flow GUI — follows this versioning strategy. The strategy is designed to make the secure, compatible path the easy path: clients that do nothing get the version they requested; breaking changes are announced with sufficient lead time; the newest version is always the supported version. + +--- + +## 1. Versioning Model + +### 1.1 URL-Based Major Version + +DCM APIs use **URL path versioning** for major versions. The version is the first path segment after the API surface prefix: + +``` +Consumer API: https://{dcm-instance}/api/v1/ +Admin API: https://{dcm-instance}/api/v1/admin/ +Provider API (OIS):https://{dcm-instance}/provider/api/v1/ +Flow GUI API: https://{dcm-instance}/flow/api/v1/ +``` + +The version segment (`v1`, `v2`, etc.) represents a major version. It increments only on breaking changes. Multiple major versions may coexist during a transition window (see Section 4). + +The four surfaces are separate because they are distinct **trust zones** — Consumer, Admin, Provider (OIS), and Flow each carry their own authentication and RBAC, and each versions independently (the Consumer API can be at `v2` while the Provider/OIS surface is still `v1`). + +### 1.2 Version Granularity — Per-API, Not Per-Endpoint + +Versioning is **per-API surface**, not per-endpoint. When a breaking change occurs to any endpoint within an API surface, the entire surface increments to the next major version. This means: + +- `v2` of the Consumer API is a complete API surface, not a patchwork of versioned endpoints +- All endpoints within a surface version are internally consistent +- Clients target a single version for all their interactions with that surface + +Individual endpoints are not independently versioned. If a single endpoint needs a breaking change, the API surface version increments and all other endpoints continue unchanged under the new version. + +### 1.3 Minor and Revision Changes + +DCM follows [Semantic Versioning](https://semver.org/) — that baseline (major = breaking, minor = additive, patch/revision = fixes) is assumed and not restated here. This section records only the **DCM-specific** application of it. + +Non-breaking changes within a major version are documented in the API changelog but do not change the URL. Clients do not need to take any action for non-breaking changes. + +- **Minor change**: new optional fields, new endpoints, expanded enum values with version-compatible defaults +- **Revision**: documentation corrections, clarifications, non-functional specification updates + +--- + +## 2. Breaking Change Definition + +A change is **breaking** if it requires any existing client to modify its code or configuration to continue working correctly. The general taxonomy below is standard REST practice — it is enumerated here not as novel guidance but as DCM's **explicit, authoritative checklist** so that "is this breaking?" has one answer across every surface. The DCM-specific entries to note are idempotency semantics, the response-envelope structure, authentication-method removal, and enum-value handling (clients MUST tolerate unknown enum values): + +**Request changes:** +- Removing a field that was previously accepted +- Changing a field from optional to required +- Changing a field's type (e.g. string → integer) +- Removing an accepted enum value +- Changing URL path structure (endpoint rename or restructure) +- Changing HTTP method for an existing operation +- Removing an endpoint + +**Response changes:** +- Removing a field from any response +- Changing a field's type in any response +- Changing a field's name in any response +- Removing a previously returned enum value +- Changing HTTP status code semantics (e.g. 200 → 202, or changing when 4xx vs 5xx is returned) +- Changing the response envelope structure + +**Behavior changes:** +- Changing default values in ways that alter existing behavior +- Changing idempotency semantics +- Removing a previously supported authentication method +- Tightening validation (rejecting previously accepted inputs) +- Changing pagination behavior in ways that break existing cursor patterns + +**The following are NOT breaking changes:** +- Adding new optional request fields (with sensible defaults) +- Adding new response fields (existing clients safely ignore unknown fields) +- Adding new endpoints +- Expanding an enum with new values (clients must handle unknown enum values gracefully) +- Relaxing validation (accepting previously rejected inputs) +- Adding new error codes (clients that handle errors generically are unaffected) +- Performance improvements, infrastructure changes, security patches +- Documentation improvements + +--- + +## 3. Version Discovery + +Clients can discover available API versions and their status without prior knowledge: + +### 3.1 Well-Known Discovery Endpoint + +``` +GET https://{dcm-instance}/.well-known/dcm-api-versions + +Response 200: +{ + "dcm_version": "1.2.0", + "api_surfaces": { + "consumer": { + "current": "v2", + "supported": ["v1", "v2"], + "versions": { + "v1": { + "status": "deprecated", + "sunset_date": "2027-06-01", + "deprecation_date": "2026-06-01", + "base_url": "/api/v1/", + "changelog_url": "/api/v1/changelog" + }, + "v2": { + "status": "stable", + "released_date": "2026-06-01", + "base_url": "/api/v2/", + "changelog_url": "/api/v2/changelog" + } + } + }, + "admin": { + "current": "v1", + "supported": ["v1"], + "versions": { + "v1": { "status": "stable", "base_url": "/api/v1/admin/" } + } + }, + "provider": { + "current": "v1", + "supported": ["v1"], + "versions": { + "v1": { "status": "stable", "base_url": "/provider/api/v1/" } + } + } + } +} +``` + +### 3.2 Per-Version Changelog + +``` +GET /api/v1/changelog + +Response 200: +{ + "version": "v1", + "changes": [ + { + "date": "2026-01-15", + "type": "minor", + "description": "Added optional `score_drivers` field to request status response", + "affected_endpoints": ["GET /api/v1/requests/{uuid}/status"] + } + ] +} +``` + +--- + +## 4. Deprecation and Sunset Lifecycle + +### 4.1 Deprecation Timeline + +When a new major version is released, the previous version enters a **deprecation period**. The deprecation timeline is profile-governed — production deployments require longer support windows than development environments: + +```yaml +api_version_support_lifecycle: + minimal: + deprecation_notice_period: P90D # 90 days notice before sunset + deprecated_version_support: P180D # old version supported 180 days after deprecation + + dev: + deprecation_notice_period: P0D # none — dev has no deprecation guarantee + deprecated_version_support: P0D # old versions may be removed immediately + # Dev is for iteration, not stable consumers; versions can break without a + # support window. Use minimal+ if you need any deprecation lead time. + + standard: + deprecation_notice_period: P180D + deprecated_version_support: P365D # 1 year + + prod: + deprecation_notice_period: P365D # 1 year notice + deprecated_version_support: P730D # 2 years support after deprecation + + fsi: + deprecation_notice_period: P548D # 18 months notice + deprecated_version_support: P1095D # 3 years support after deprecation + + sovereign: + deprecation_notice_period: P730D # 2 years notice + deprecated_version_support: P1460D # 4 years support after deprecation +``` + +**Deprecation ≠ Sunset.** A deprecated version continues to function. Sunset is when it stops working. The deprecation period is the window between "we recommend you migrate" and "you must migrate." + +### 4.2 Deprecation Headers + +When a client calls a deprecated API version, the response includes standard deprecation headers (per [RFC 8594](https://datatracker.ietf.org/doc/html/rfc8594) and [RFC 9745](https://datatracker.ietf.org/doc/html/rfc9745)): + +```http +HTTP/1.1 200 OK +Deprecation: @1749340800 # Unix timestamp when this version was deprecated +Sunset: @1781049600 # Unix timestamp when this version will stop working +Link: ; rel="successor-version" +Link: ; rel="deprecation" +``` + +### 4.3 Deprecation Events + +When a version is deprecated or sunsetted, DCM fires notification events: + +- `governance.api_version_deprecated` — version entered deprecation; Sunset header begins appearing +- `governance.api_version_sunset_warning` — 30 days before sunset; high urgency +- `governance.api_version_sunset` — version has reached sunset date; calls now return 410 Gone + +Platform admins should configure notification routing for these events to ensure API consumers receive timely warning. + +### 4.4 Sunset Behavior + +After the sunset date, calls to the deprecated version return: + +```http +HTTP/1.1 410 Gone +Content-Type: application/json + +{ + "error": "api_version_sunset", + "message": "API version v1 reached its sunset date on 2027-06-01. Migrate to v2.", + "successor_version": "v2", + "migration_guide_url": "/api/v2/migration-guide", + "sunset_date": "2027-06-01" +} +``` + +--- + +## 5. Version Negotiation + +### 5.1 How Clients Specify a Version + +The URL path is the primary versioning mechanism. No headers or query parameters are required — the URL is authoritative: + +``` +GET /api/v1/resources → Consumer API v1 +GET /api/v2/resources → Consumer API v2 (when available) +``` + +### 5.2 Version Preference Header (Optional) + +For clients that need to pin to a specific version or test against a new version before migrating, an optional `DCM-API-Version` header is supported: + +```http +GET /api/v1/resources +DCM-API-Version: v1 # explicit pin; returns 406 if v1 is sunsetted +``` + +If the header specifies a sunsetted version, the response is `406 Not Acceptable` with a migration guide reference. + +### 5.3 Latest-Version Alias + +``` +GET /api/latest/resources # always routes to current stable version +``` + +The `latest` alias is provided for development and testing. It is **not recommended for production** — production clients should pin to a specific version to avoid inadvertent breaking changes when a new major version becomes `latest`. + +--- + +## 6. Beta and Preview Endpoints + +New capabilities that are not yet stable may be released as **preview endpoints** within the current major version: + +``` +GET /api/v1/preview/new-feature +``` + +Preview endpoints: +- Are not covered by the stability guarantees of the parent version +- May change or be removed without a major version increment +- Are marked in the API changelog and discovery endpoint as `status: preview` +- Must not be used in production automation without explicit acknowledgment of instability + +```yaml +# Discovery response for a preview endpoint +"new-feature": { + "status": "preview", + "stability_commitment": "none", + "planned_graduation": "v2", + "feedback_url": "https://github.com/dcm-project/discussions" +} +``` + +Preview endpoints graduate to stable when they are included in a new major version release. + +--- + +## 7. Provider API (OIS) Versioning + +The Operator Interface Specification (OIS) governs how DCM calls providers. Provider implementations must support the version of the OIS they declare in their capability registration. + +### 7.1 OIS Version in Capability Registration + +```yaml +provider_registration: + ois_version: "1.0" # which OIS version this provider implements + ois_version_min: "1.0" # minimum OIS version supported + ois_version_max: "1.x" # maximum OIS version supported (x = any minor) +``` + +### 7.2 OIS Compatibility + +DCM maintains version-compatible with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 until the version is sunset. + +When the OIS version is incremented: +1. DCM announces the new OIS version via the event `governance.ois_version_released` +2. Providers have the deprecation notice period to upgrade their implementation +3. DCM dispatches using the appropriate OIS version per the provider's declared capability +4. After sunset, providers still on deprecated OIS versions receive `410 Gone` on dispatch + +**Why an event, not a REST response (step 1).** A version/capability change is a **one-to-many** announcement — every registered provider and every interested subscriber needs to learn about it, and they are not in the middle of a request when it happens. That is a fan-out, so it is published on the event bus (CloudEvents), not returned synchronously. By contrast, an actual **dispatch** (step 3) is **point-to-point** and needs an immediate result, so it stays a synchronous REST call. The rule across DCM: state/capability *changes* broadcast as events; *operations* that need a result are REST. + +### 7.3 Provider-Initiated API Versioning + +Providers that expose their own management APIs (beyond the standard OIS surface) are responsible for their own versioning. DCM does not version-manage provider-internal APIs. Providers should follow the same breaking-change definition (Section 2) and announce breaking changes via `provider_update.submitted` events. + +--- + +## 8. Version Upgrade Path + +When a new major API version is published, a machine-readable change log is available at: + +``` +GET /api/v{N}/migration-guide +``` + +This endpoint returns all breaking changes from the previous major version: + +```json +{ + "from_version": "v1", + "to_version": "v2", + "breaking_changes": [ + { + "change_id": "BC-001", + "type": "field_removed", + "endpoint": "GET /api/v2/resources/{uuid}", + "description": "Field 'legacy_id' removed — use 'entity_uuid' instead" + } + ], + "new_capabilities": [] +} +``` + +Clients declare the API version they target via the `Accept-Version` header or URL prefix. DCM supports all non-sunset major versions simultaneously. When a version reaches sunset, responses include `Deprecation` and `Sunset` headers (RFC 8594) before support is withdrawn. + +--- + + +## 9. Internal API Versioning + +DCM internal component APIs (Control Plane components communicating with each other) follow a simpler model: + +- Internal APIs are not exposed externally and not subject to the external versioning lifecycle +- Internal breaking changes require a coordinated deployment of all affected components +- DCM release versions (e.g. `1.2.0`) cover the complete set of internal APIs for that release +- Operators upgrading DCM must upgrade all components together per the release upgrade guide + +--- + +## 10. System Policies + +| Policy | Rule | +|--------|------| +| `VER-001` | All DCM public API surfaces use URL path versioning. The version path segment is the only authoritative version indicator. | +| `VER-002` | A change is breaking if any existing client must modify code or configuration to continue working. When in doubt, treat a change as breaking. | +| `VER-003` | Deprecated API versions must return `Deprecation`, `Sunset`, and `Link` headers on every response during the deprecation period (per RFC 8594 / RFC 9745). | +| `VER-004` | Deprecated versions must remain fully functional until the sunset date. Bugs in deprecated versions are fixed; new features are not backported. | +| `VER-005` | The deprecation notice period and deprecated version support window are profile-governed. Production deployments require longer windows than development. See Section 4.1. | +| `VER-006` | The `latest` version alias is available but must not be recommended for production use. Production clients must pin to a specific version. | +| `VER-007` | Preview endpoints are not stable. They may change or be removed without a major version increment. They are identified by the `/preview/` path segment. | +| `VER-008` | Every new major version must publish a machine-readable migration guide at `/api/v{N}/migration-guide`. | +| `VER-009` | DCM must maintain dispatch compatibility with providers registered on supported OIS versions until the OIS version is sunset. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/architecture/control-plane/components.md b/architecture/control-plane/components.md new file mode 100644 index 0000000..bdd12c3 --- /dev/null +++ b/architecture/control-plane/components.md @@ -0,0 +1,852 @@ +--- +Document Status: ✅ Complete +Document Type: Architecture Reference +--- + +# DCM Data Model — Control Plane Components + +> **DCM-native control-plane runtime; no single UDLM contract counterpart.** +> The control-plane components are runtime implementations of the three UDLM +> abstractions (Data, Provider, Policy) — not a fourth abstraction and not a +> distinct UDLM contract. A peer DCM realization could decompose its control +> plane differently and still satisfy every UDLM contract. + + +**Document Status:** ✅ Complete +**Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions — Data, Provider, and Policy — are defined in +> [udlm/foundations/foundations.md](https://github.com/croadfeldt/udlm/blob/main/foundations/foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> See also: [Provider Contract](https://github.com/croadfeldt/udlm/blob/main/contracts/provider-contract.md) | [Policy Contract](https://github.com/croadfeldt/udlm/blob/main/contracts/policy-contract.md) +> +> **This document maps to: RUNTIME** +> +> Runtime implementations of the three abstractions — not a fourth abstraction + + +**Related Documents:** [Internal Component Authentication](internal-component-auth.md) | [Context and Purpose](https://github.com/croadfeldt/udlm/blob/main/foundations/context-and-purpose.md) | [Four States](https://github.com/croadfeldt/udlm/blob/main/foundations/four-states.md) | [Resource/Service Entities](https://github.com/croadfeldt/udlm/blob/main/entities/resource-service-entities.md) | [Operational Models](https://github.com/croadfeldt/udlm/blob/main/lifecycle/operational-models.md) | [Policy Profiles](../governance-enforcement/policy-profiles.md) + +--- + + +The DCM Control Plane consists of **nine components** that implement the three foundational abstractions at runtime. + +## 1. Purpose + +> **Internal component authentication:** See [Internal Component Authentication](internal-component-auth.md) for the mTLS and interaction credential model governing all component-to-component calls within the DCM control plane. + + +This document formally defines the DCM control plane components that are referenced throughout the data model documents. Two components are defined here: + +1. **The Request Orchestrator** — the event bus and coordinator of the request lifecycle pipeline +2. **The Cost Analysis Component** — the internal DCM component that provides cost signals for placement, catalog, and attribution + +--- + +## 2. The Request Orchestrator + +### 2.1 Role + +The Request Orchestrator is the **event bus and pipeline coordinator** for all DCM request lifecycle operations. It does not perform any pipeline work itself — it listens for events, evaluates which components need to act on them, and routes work to the appropriate components. + +The Request Orchestrator embodies DCM's **data-driven, policy-triggered orchestration model**: the pipeline is not a fixed procedural sequence. It is a cascade of event-condition-action responses, where policies define what happens when specific payload states are observed. + +### 2.2 Data-Driven Orchestration Principle + +**Policies ARE the orchestration.** The Request Orchestrator does not contain hardcoded pipeline logic. It publishes events to the Policy Engine; policies match on payload type and state; policy actions produce new payload states; those new states trigger further policy evaluations. + +This means: +- Adding a new pipeline step = writing a new policy (no code change) +- Removing a step = deactivating a policy +- Changing when a step fires = changing a policy condition +- A static workflow (e.g., always require human approval for prod VMs) = a policy that always matches for those conditions +- A dynamic workflow (e.g., route to different approval processes based on cost) = a policy with conditional logic + +Static and dynamic flows compose naturally — a static policy defines a guaranteed step; a dynamic policy defines a conditional step. Both are expressed as policies, evaluated by the same engine, producing deterministic outcomes. + +**Determinism guarantee:** Dynamic execution remains deterministic because: +- The payload type vocabulary is a closed set +- Policy evaluation order within a domain level is deterministic (domain precedence) +- The payload mutation model is immutable (each policy produces a new payload version) +- The same input state always produces the same output state + +### 2.3 The Payload Type Vocabulary + +Every event in DCM carries a payload with a declared type. Policies pattern-match on these types. The payload type vocabulary is the foundational contract of the orchestration model. + +```yaml +payload_types: + # Request lifecycle + request.initiated: # consumer submitted a request + request.intent_captured: # Intent State written + request.layers_assembled: # layer assembly complete + request.policies_evaluated: # all active policies evaluated + request.placement_complete: # provider selected + request.dispatched: # sent to provider + request.realized: # provider confirmed realization + request.failed: # terminal failure + request.cancelled: # cancelled + + # Provider update + provider_update.received: # provider submitted update notification + provider_update.evaluated: # policy evaluation complete + provider_update.accepted: # accepted; Realized State updating + provider_update.rejected: # rejected; becomes drift + + # Drift and discovery + discovery.cycle_complete: + drift.detected: + drift.resolved: + + # Recovery + recovery.timeout_fired: + recovery.late_response: + recovery.compensation_triggered: + + # Governance + policy.activated: + layer.updated: + profile.changed: +``` + +### 2.4 Event Routing Model + +``` +Event published: { type: "request.initiated", payload: {...}, entity_uuid: X } + │ + ▼ Request Orchestrator receives event + │ Routes to Policy Engine: "evaluate all policies matching request.initiated" + │ + ▼ Policy Engine evaluates in domain precedence order + │ Matching policies fire; payload mutations accumulated + │ New payload state produced: { type: "request.layers_assembled", ... } + │ + ▼ Request Orchestrator receives new event + │ Routes to Policy Engine for next evaluation cycle + │ (parallel if no data dependencies between active policies) + │ + ▼ Continues until terminal state (request.realized or request.failed) +``` + +**Parallel execution:** Policies that have no data dependencies on each other evaluate concurrently. The Request Orchestrator tracks dependency declarations between policies and executes in parallel where safe. + +**Why policy evaluation is event-driven but provider dispatch is synchronous REST.** Policy evaluation is **one-to-many**: a single lifecycle event (e.g. `request.layers_assembled`) may match any number of policies — gating, transformation, recovery — that the orchestrator does not know in advance and that fire independently. That fan-out is exactly what an event bus is for; modeling it as a chain of REST calls would hard-wire a caller↔callee coupling the policy model is designed to avoid. **Provider dispatch is the opposite shape**: it is **point-to-point** (one entity → one selected provider) and the orchestrator needs the realized result *before* it can advance the request, so it is a synchronous REST call ([Provider Contract](../../contracts/provider-contract.md)). The rule across the control plane: *fan-out / "who cares about this happening?"* → events; *point-to-point / "I need this result to continue"* → REST. + +### 2.5 Static Flow Support + +Organizations that require guaranteed sequential flows express them as ordered policy sets: + +```yaml +static_flow_policy_group: + handle: "org/flows/prod-vm-approval-flow" + concern_type: orchestration_flow + ordered: true # policies execute in declared sequence, not parallel + policies: + - step: 1 + handle: "org/policies/cost-check" + condition: "request.initiated AND resource_type=Compute.VirtualMachine AND tenant.profile=prod" + on_fail: halt + - step: 2 + handle: "org/policies/manager-approval" + condition: "request.cost_estimated > 500" + on_fail: halt + - step: 3 + handle: "org/policies/security-review" + condition: "always" + on_fail: halt +``` + +A static flow is a Policy Group with `concern_type: orchestration_flow` and `ordered: true`. The Request Orchestrator respects the declared order. Static flows integrate with dynamic policies — a dynamic policy can fire alongside the static flow steps. + +#### Match-condition grammar + +The `condition:` field above is a **match-condition expression** — the same grammar every policy uses to decide whether it fires. It is a boolean expression over the current payload and context: + +``` +condition := term (("AND" | "OR") term)* | "always" +term := | +operand := | +comparator:= "=" | "!=" | ">" | ">=" | "<" | "<=" | "IN" +``` + +- **`always`** is a **reserved keyword** meaning the condition is unconditionally true — the policy fires on every evaluation of its event. It is the explicit form of "no guard"; use it instead of a tautology so the intent is visible. (Other reserved tokens: `AND`, `OR`, `IN`, `true`, `false`.) +- **Variable paths** (e.g. `resource_type`, `tenant.profile`, `request.cost_estimated`) resolve **by reference** against the current payload + request context at evaluation time. A bare `request.initiated`-style token is an **event-type match** (true when the triggering event is that type). +- **Literals** are values, not references: an unquoted number (`500`) or boolean is a literal; `resource_type=Compute.VirtualMachine` compares the resolved `resource_type` variable to the literal enum value. To compare two variables, both sides are paths (`a.x = b.y`); to compare a variable to a fixed value, the right side is a literal. Quote a literal that contains spaces. +- Evaluation is **side-effect-free** and short-circuits left to right; an unresolvable variable path evaluates the term to `false` (it never errors the request). + +### 2.5a Named Workflows vs Dynamic Policies — How They Compose + +The Request Orchestrator does not distinguish between named workflows and dynamic policies — both arrive as events and are routed to the Policy Engine. The distinction is in *how they are declared*: + +**Named Workflow Artifacts** (Orchestration Flow Policies with `ordered: true`) declare an explicit step sequence. An operator reading the workflow can see every step in order. Steps reference payload types from the closed vocabulary. Named workflows are the *explicit, visible skeleton* of a process. + +**Dynamic Policies** (Gating Policy, Transformation, Recovery) fire when their match conditions are satisfied, regardless of workflow position. They are not declared in the workflow artifact. They are the *conditional behavior* that fills in the skeleton. + +**Example — request lifecycle:** +``` +Named workflow "system/workflows/request-lifecycle" declares: + Step 1: request.initiated → capture intent + Step 2: request.intent_captured → run layer assembly + Step 3: request.layers_assembled → run placement + Step 4: request.placement_complete → dispatch + +Dynamic policies also fire: + Gating Policy "vm-size-limits" fires on request.layers_assembled + if cpu_count > 32 → deny + Transformation "inject-monitoring" fires on request.layers_assembled + → adds monitoring_endpoint field + Recovery "notify-on-timeout" fires on recovery.timeout_fired + → NOTIFY_AND_WAIT action +``` + +The named workflow and the dynamic policies are independent artifacts. Adding a new Gating Policy does not modify the workflow. Modifying the workflow does not affect dynamic policies. They compose through the same Policy Engine evaluation on the same events. + +### 2.6 Request Orchestrator Responsibilities + +| Responsibility | Description | +|----------------|-------------| +| Event routing | Receive all request lifecycle events; route to appropriate components | +| Pipeline coordination | Sequence component interactions per data dependencies | +| Timeout monitoring | Track dispatch_timeout and assembly_timeout; fire recovery triggers | +| Dependency resolution | For composite services, sequence component provisioning per dependency graph | +| Status tracking | Maintain current status of all in-flight requests; respond to status queries | +| Recovery coordination | On timeout/failure, invoke Recovery Policy evaluation | + +--- + +## 3. The Cost Analysis Component + +### 3.1 Role + +The Cost Analysis Component is an **internal DCM control plane component** that provides cost signals to other components. It is not a billing system and not a provider type. It does not manage financial transactions, produce invoices, or serve as the authoritative financial record. It provides cost *signals* that DCM uses for placement decisions, pre-request estimation, and ongoing attribution. + +The authoritative billing record lives in the organization's financial system. A billing system can register as an Information Provider to push authoritative cost data back into DCM for attribution records. + +### 3.2 Three Cost Functions + +**Function 1 — Pre-request cost estimation:** +Given a catalog item and assembled field values, compute the estimated lifecycle cost. Used by: +- Service Catalog describe endpoint (consumer sees cost before requesting) +- CI pipeline pre-validation (cost estimate in PR comment) +- Placement engine tie-breaker step 4 (cheapest eligible provider) + +**Function 2 — Placement cost input:** +During Step 6 placement, provide current cost data per eligible provider for the requested resource type. If Cost Analysis data is unavailable, the placement engine falls back to static declared costs per REG-011. + +**Function 3 — Ongoing cost attribution:** +For realized entities, track ongoing consumption and attribute costs to the owning Tenant. Consumed by OBS-005 (consumer cost view) and the resource describe endpoint (`estimated_cost_per_hour` field). + +### 3.3 Cost Data Sources + +The Cost Analysis Component ingests cost data from two sources, following the REG-011 hybrid model: + +```yaml +cost_data_sources: + static: + source: provider_registration # declared at provider registration time + update_frequency: manual # updated when rates change + fields: [capex_per_unit, opex_per_unit_per_hour, currency] + + dynamic: + source: external_cost_api # external billing API or cloud pricing API + registered_as: information_provider + query_interval: PT1H + fallback: static # use static if dynamic unavailable + fallback_max_age: PT24H +``` + +### 3.4 Cost Estimation Model + +```yaml +cost_estimation_request: + catalog_item_uuid: + assembled_fields: + cpu_count: 4 + memory_gb: 8 + storage_gb: 100 + tenant_uuid: + requested_duration: P30D # optional; lifecycle estimate + +cost_estimation_response: + estimated_cost: + per_hour: 0.32 + per_month: 230.40 + lifecycle_estimate: 691.20 # if requested_duration provided + currency: USD + confidence: high # high: current Cost Analysis data + # medium: data > PT1H old + # low: static fallback + breakdown: + - component: compute + per_hour: 0.28 + - component: ip_allocation + per_hour: 0.04 + cost_data_timestamp: +``` + +**Field reference.** `assembled_fields` — the post-layer-assembly resource spec the estimate is computed from (the cost-relevant subset; provider-agnostic). `requested_duration` — optional ISO-8601 duration; when present, `lifecycle_estimate` = projected total over that duration. `confidence` — provenance of the figure, not a probability: `high` = live Cost Analysis data, `medium` = data older than PT1H, `low` = static fallback table. `breakdown[]` — per-cost-component contribution (`component` is a free-form cost driver such as `compute`/`ip_allocation`; the sum reconciles to `per_hour`). `cost_data_timestamp` — when the underlying rate data was sourced (drives the `confidence` downgrade). On the attribution side, `billing_state` governs whether accrual is charged (`billable` | `non_billable` | `reduced_rate`) and `cost_data_source` records whether the rate came from live `cost_analysis`, a `static` table, or is `unknown`. + +### 3.5 Cost Attribution for Realized Entities + +```yaml +entity_cost_attribution: + entity_uuid: + tenant_uuid: + billing_state: billable # billable | non_billable | reduced_rate + current_rate: + per_hour: 0.32 + currency: USD + rate_effective_since: + monthly_accrual: 230.40 + cost_data_source: cost_analysis # cost_analysis | static | unknown +``` + +### 3.6 Integration with Placement Engine + +The placement engine queries Cost Analysis at step 4 of the tie-breaking hierarchy: + +``` +Step 4 — Cost Analysis (if available and determinable): + Query Cost Analysis for each eligible provider + Cost Analysis returns: estimated cost per unit per provider + Placement engine prefers lowest cost among equally-ranked candidates + If Cost Analysis unavailable: skip step 4; proceed to step 5 + # Cost Analysis unavailability never blocks placement +``` + +--- + +## 4. Related Policies + +| Policy | Rule | +|--------|------| +| `CTL-001` | The Request Orchestrator is the single event bus for all request lifecycle events. No component communicates directly with another component outside of events published to the Request Orchestrator. | +| `CTL-002` | Policies ARE the orchestration. The Request Orchestrator does not contain hardcoded pipeline logic. Pipeline behavior is modified by adding, removing, or changing policies — not by changing the orchestrator. | +| `CTL-003` | Dynamic and static flows compose naturally. Static flows are Policy Groups with concern_type: orchestration_flow and ordered: true. Both types are evaluated by the same Policy Engine. | +| `CTL-004` | Cost Analysis is not a billing system. It provides cost signals for placement and attribution. The authoritative billing record lives in the organization's financial system, which may register as an Information Provider. | +| `CTL-005` | Cost Analysis unavailability never blocks placement. The placement engine falls back to static declared costs per REG-011 and skips the Cost Analysis tie-breaking step. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + +--- + +## 4. The Placement Engine + +### 4.1 Role + +The Placement Engine selects the specific Service Provider that will fulfill a resource request. It runs as step 6 of the Request Payload Processor assembly pipeline and is invoked by the Request Orchestrator after all policies have been evaluated and the assembled payload is ready for dispatch. + +The Placement Engine does not make business decisions — those are made by policies (which inject constraints and preferences). The Placement Engine applies those constraints to find eligible providers, then deterministically resolves ties using a declared hierarchy. + +### 4.2 Input and Output + +**Input:** +- Assembled payload with full field-level provenance +- Sovereignty constraints (from compliance domain profile and any policy-injected constraints) +- Accreditation requirements (from Data/Capability Authorization Matrix) +- Preference scores or preferred provider UUIDs (if injected by Transformation policy) +- Tenant affinity declarations + +**Output:** +- Selected provider UUID +- Placement reason (why this provider was selected) +- Sovereignty satisfaction record (which constraints were checked and passed) +- Reserve confirmation (provider confirmed it has capacity for this specific request) + +This output is written to `placement.yaml` in the Requested Store directory. + +### 4.3 Placement Algorithm — Six Steps + +``` +Step 1: Sovereignty Pre-Filter + Eliminate any provider whose sovereignty_declaration does not satisfy + the request's sovereignty constraints. + → Providers that fail this step are never contacted. + → If zero providers remain: RESERVE_QUERY_ALL_EXHAUSTED recovery trigger fires. + +Step 2: Accreditation Filter + Eliminate any provider that does not hold the required accreditations + for the data classifications present in the assembled payload. + → Checked against the active Data/Capability Authorization Matrix. + → Providers with accreditation gaps are excluded. + +Step 3: Capability Filter + Eliminate any provider that does not declare support for the + requested resource type and all required capabilities. + → Based on provider registration catalog item declarations. + +Step 4: Reserve Query + Send a reserve query to each remaining candidate provider in parallel. + → Providers have reserve_query_timeout to respond (profile-governed: PT5–30S). + → Providers that do not respond within timeout: excluded from this placement cycle. + → Providers that respond with INSUFFICIENT_CAPACITY: excluded; DCM updates + internal capacity rating for that provider. + → Providers that confirm capacity: advance to tie-breaking. + +Step 5: Tie-Breaking (deterministic hierarchy) + Applied when multiple providers confirmed capacity in Step 4: + + Priority 1: Policy preference + A Transformation policy injected a preference_score or preferred_provider_uuid. + Highest preference_score wins. preferred_provider_uuid is absolute — skips Steps 2-6. + + Priority 2: Provider declared priority + Providers declare a numeric priority at registration (default: 50). + Higher value = preferred when all else equal. + + Priority 3: Tenant affinity + Tenant's Policy Group declares preferred providers for specific resource types. + Affinity preference is a soft preference — does not override accreditation or sovereignty. + + Priority 4: Cost Analysis + Cost Analysis component provides current cost per unit per candidate provider. + Prefer lower total cost (CapEx + OpEx + licensing). + Skip if: Cost Analysis unavailable, data stale > PT1H, or cost difference < 5%. + + Priority 5: Least loaded + Prefer provider with lower current capacity utilization from reserve_query response. + Skip if: utilization difference < 10%, or utilization data not returned. + + Priority 6: Consistent hash (final tiebreaker — always resolves) + SHA-256(request_uuid + resource_type + sorted_candidate_uuids) + Deterministic — same request always resolves to the same provider in a stable cluster. + Never round-robin. + +Step 6: Reserve Confirmation + Notify the selected provider that its reservation is confirmed. + Other providers that responded to the reserve query receive a reservation release. + → Prevents capacity holds from accumulating across providers for the same request. +``` + +### 4.4 Reserve Query Protocol + +```yaml +reserve_query: + query_uuid: # idempotency key + entity_uuid: + resource_type: Compute.VirtualMachine + resource_type_spec_version: "2.1.0" + requested_fields: + cpu_count: 4 + memory_gb: 8 + storage_gb: 100 + sovereignty_requirements: + data_residency: EU + reservation_hold_ttl: PT5M # provider holds capacity for this duration + # released when: confirmed, rejected, or TTL expires +``` + +```yaml +reserve_query_response: + query_uuid: + provider_uuid: + status: confirmed | insufficient_capacity | capability_not_supported + capacity_held_until: # if confirmed + utilization_pct: 42 # current load; used for Step 5 tiebreaking + cost_per_hour: 0.32 # if Cost Analysis integration enabled + currency: USD +``` + +### 4.5 Placement Configuration + +```yaml +placement_engine_config: + reserve_query_timeout: PT10S # profile-governed default + parallel_reserve_queries: true # always true; all candidates queried simultaneously + max_candidates_per_placement: 10 # cap on parallel reserve queries + cost_freshness_max: PT1H + cost_difference_threshold: 0.05 # 5% — skip cost step if within this band + utilization_difference_threshold: 0.10 # 10% — skip utilization step if within this band + reservation_hold_ttl: PT5M +``` + +### 4.6 Placement Failure and Recovery + +When placement cannot find an eligible provider: + +| Failure Reason | Recovery Trigger | +|---------------|-----------------| +| All providers fail sovereignty filter | `RESERVE_QUERY_ALL_EXHAUSTED` | +| All providers fail accreditation filter | `RESERVE_QUERY_ALL_EXHAUSTED` | +| All providers respond INSUFFICIENT_CAPACITY | `RESERVE_QUERY_ALL_EXHAUSTED` | +| All reserve queries time out | `RESERVE_QUERY_ALL_EXHAUSTED` | + +The `RESERVE_QUERY_ALL_EXHAUSTED` trigger fires the active Recovery Policy. Default action per profile: standard/prod → `NOTIFY_AND_WAIT`; dev → `RETRY` with exponential backoff. + +### 4.7 Placement System Policies + +| Policy | Rule | +|--------|------| +| `PLC-001` | Sovereignty pre-filter runs before any provider is contacted. Providers that fail sovereignty constraints never receive reserve queries. | +| `PLC-002` | Accreditation filter runs before reserve queries. Providers without required accreditations for the payload's data classifications are excluded. | +| `PLC-003` | Reserve queries are sent in parallel to all eligible candidates. Sequential querying is not permitted — it introduces latency and prevents fair capacity comparison. | +| `PLC-004` | The consistent hash tiebreaker is always the final tiebreaker. It ensures deterministic provider selection for identical inputs without round-robin non-determinism. | +| `PLC-005` | A confirmed reservation hold must be released when not used — either by confirmation dispatch or by explicit release on timeout. Capacity holds must not accumulate silently. | +| `PLC-006` | The Placement Engine never selects a provider based solely on network position or co-location. Every selection is based on declared constraints, policies, and the tie-breaking hierarchy. | + +--- + +## 5. The Lifecycle Constraint Enforcer + +### 5.1 Role + +The Lifecycle Constraint Enforcer is a DCM control plane component that monitors all realized entities against their declared lifecycle constraints and fires expiry actions when constraints are reached. It is the authoritative enforcer of TTL, expiry date, and maximum execution time declarations. + +Lifecycle constraint enforcement is a DCM concern — not a provider concern. The provider does not need to know about or implement any TTL logic. + +### 5.2 What It Monitors + +The Lifecycle Constraint Enforcer monitors three categories of constraint: + +**Category 1 — Entity TTL:** +Duration-based: entity expires T duration after a reference point (realization, creation, last modification). + +**Category 2 — Entity Expiry Date:** +Calendar-based: entity expires at an absolute timestamp. + +**Category 3 — Process Resource Maximum Execution Time:** +Process Resources must declare `max_execution_time`. The Enforcer monitors all executing Process Resources and fires `on_max_exceeded` when the limit is reached. + +### 5.3 Monitoring Loop + +``` +Lifecycle Constraint Enforcer runs continuously: + + Every cycle (interval: PT1M for standard/prod; PT5M for minimal/dev): + + Query Realized Store for entities with: + lifecycle_state IN [OPERATIONAL, SUSPENDED, EXECUTING] + AND lifecycle_constraints declared + AND NOT already in terminal state + + For each entity: + Compute time_remaining = constraint_expiry - now() + + If time_remaining <= warn_before_expiry: + If warn_not_yet_sent: + Emit: entity.ttl_warning notification + Record: WARNING_EMITTED in entity provenance + + If time_remaining <= 0: + Execute on_expiry action (see Section 5.4) +``` + +### 5.4 Expiry Action Execution + +When a lifecycle constraint fires, the Enforcer executes the declared `on_expiry` action: + +| Action | Behavior | +|--------|---------| +| `decommission` | Submit a decommission request through the standard pipeline — produces Requested State, dispatches to provider, full audit trail | +| `suspend` | Submit a suspend request through the standard pipeline | +| `notify` | Fire `entity.ttl_expired` notification to entity owner; no automated action | +| `review` | Entity enters PENDING_EXPIRY_ACTION state; Platform Admin and owner notified | +| `escalate` | Immediately escalate to Platform Admin; entity enters PENDING_EXPIRY_ACTION state | + +**Grace period:** Expiry actions are not immediate. The Enforcer respects the declared `grace_period` (default PT1H) — the action fires `grace_period` after the constraint expires, giving human operators a window to intervene. + +**Action failure:** If the expiry action fails to execute (provider unreachable, dependency conflict), the entity enters `PENDING_EXPIRY_ACTION` state (LTC-005). The Enforcer retries per the active Recovery Policy. Platform Admin is notified with urgency: high. + +### 5.5 Expiry Audit Records + +Every expiry-related event produces an audit record: + +```yaml +audit_record: + action: EXPIRY_WARNING | EXPIRY_ACTION_FIRED | EXPIRY_ACTION_FAILED | + PENDING_EXPIRY_ACTION_ENTERED + actor: + type: system + system_actor: + component: lifecycle_constraint_enforcer + trigger: ttl_reached | expires_at_reached | max_execution_time_reached + entity_uuid: + details: + constraint_type: ttl | expires_at | max_execution_time + constraint_value: + action_taken: decommission | suspend | notify | review | escalate + grace_period_remaining: +``` + +### 5.6 Process Resource Enforcement + +Process Resources require `max_execution_time` (mandatory). The Enforcer monitors all EXECUTING Process Resources: + +``` +Process Resource enters EXECUTING state + │ + ▼ Enforcer records: execution_started_at; computes execution_timeout_at + │ + ▼ On every monitoring cycle: + │ If now() >= execution_timeout_at: + │ Emit: PROCESS_TIMEOUT event + │ Entity state → FAILED + │ Recovery Policy: COMPENSATION_FAILED trigger if resources were modified + │ Notification: entity owner + Platform Admin (urgency: high) +``` + +### 5.7 Lifecycle Constraint Enforcer Policies + +| Policy | Rule | +|--------|------| +| `LCE-001` | The Lifecycle Constraint Enforcer runs as a continuous monitor. It does not rely on provider callbacks or event triggers for expiry detection — it polls based on declared constraints. | +| `LCE-002` | Expiry actions are submitted through the standard DCM request pipeline. Decommission-on-expiry produces a Requested State record with `actor: system/lifecycle-constraint-enforcer`. | +| `LCE-003` | The Enforcer respects the declared grace_period before firing expiry actions. Grace period gives human operators a window to intervene before automated action. | +| `LCE-004` | Process Resource max_execution_time enforcement fires immediately on breach — no grace period. Hung processes are failed immediately to prevent resource leaks. | +| `LCE-005` | Expiry action failures enter PENDING_EXPIRY_ACTION state. The Enforcer retries per the active Recovery Policy. Indefinite retry without escalation is not permitted. | + +--- + +## 6. The Search Index + +### 6.1 Role + +The Search Index is a **non-authoritative, queryable projection** of the GitOps stores (Intent Store and Requested Store). It enables millisecond-latency queries against stored entities without traversing Git history, while the GitOps stores remain the authoritative source of truth. + +The Search Index is a PostgreSQL store contract. It has its own registration, health check, and sovereignty declaration. It is never the source of truth — if the Search Index and the GitOps store disagree, the GitOps store wins unconditionally. + +### 6.2 What It Indexes + +The Search Index maintains a projection of key fields from Intent State and Requested State records, enabling queries without retrieving full payloads from Git: + +```yaml +search_index_record: + entity_uuid: + entity_handle: + resource_type: Compute.VirtualMachine + resource_type_category: Compute + tenant_uuid: + lifecycle_state: OPERATIONAL + drift_status: clean + provider_uuid: + deployment_posture: prod + compliance_domains: [hipaa] + data_classifications: [restricted] # highest classification in entity + created_at: + updated_at: + cost_per_hour: 0.32 + currency: USD + git_path: intent-store/tenant-uuid/Compute/VirtualMachine/entity-uuid/intent.yaml + # git_path is the pointer back to the authoritative record + tags: { environment: production, team: payments } +``` + +### 6.3 Required Query Operations + +| Operation | Description | +|-----------|-------------| +| `find_by_uuid(entity_uuid)` | Return index record for a single entity | +| `find_by_tenant(tenant_uuid, filters)` | Return all entities for a Tenant with optional field filters | +| `find_by_resource_type(fqn, filters)` | Return all entities of a resource type | +| `find_by_provider(provider_uuid, filters)` | Return all entities hosted at a provider | +| `find_by_lifecycle_state(state, tenant_uuid)` | Return entities in a given lifecycle state | +| `find_by_drift_status(status, tenant_uuid)` | Return drifted or clean entities | +| `find_by_data_classification(classification)` | Return entities containing data of a given classification | +| `full_text_search(query, tenant_uuid)` | Full-text search across handle, display_name, tags | + +All queries return the `git_path` — consumers fetch the full payload from Git if needed. + +### 6.4 Consistency Model + +The Search Index is **eventually consistent** with the GitOps stores. There is a defined maximum staleness: + +```yaml +search_index_consistency: + max_staleness: PT5M # index must be within 5 minutes of GitOps store + profile_overrides: + prod: PT2M + fsi: PT1M + sovereign: PT1M + on_staleness_exceeded: + action: degrade_with_warning # serve results with staleness warning + alert: platform_admin # alert on staleness exceeding 2× max + rebuild_on_recovery: true # full index rebuild from Git history on failure + rebuild_max_duration: PT4H # must complete within 4 hours for standard+ +``` + +### 6.5 Unavailability Behavior + +If the Search Index is unavailable: +- DCM degrades search operations gracefully: returns a `503 Service Degraded` response with a reference to the authoritative Git store +- Writes are not affected — GitOps stores are written directly; the index is updated asynchronously +- On recovery: the Search Index rebuilds from Git history +- No data is lost if the index is lost — it is always reconstructable from Git + +### 6.6 Search Index Policies + +| Policy | Rule | +|--------|------| +| `SIX-001` | The Search Index is non-authoritative. GitOps stores win on any disagreement. Consumers must be prepared to receive a git_path and fetch from the authoritative store. | +| `SIX-002` | The Search Index must be rebuildable from Git history at any time. Implementations that cannot perform a full index rebuild are non-conformant. | +| `SIX-003` | Search Index staleness beyond the profile-governed maximum triggers a platform admin alert. Staleness is surfaced in query responses — consumers are never served stale data silently. | +| `SIX-004` | Search Index unavailability degrades queries without impacting writes. Write operations proceed directly to the authoritative GitOps stores regardless of Search Index availability. | + +--- + + +--- + +## 7. The Drift Reconciliation Component + +### 7.1 Role + +The Drift Reconciliation Component compares the Discovered State of entities against their Realized State to detect, classify, and respond to drift. It is the consumer of Discovered Store data and the producer of drift records that feed into the Policy Engine for response evaluation. + +Drift Reconciliation is purely a read-and-compare component — it never writes to the Realized Store. It reads Discovered State, reads Realized State, computes differences, classifies severity, and fires events into the Request Orchestrator. The Policy Engine and Recovery Policies determine what happens next. + +### 7.2 Inputs and Outputs + +**Inputs:** +- Discovered State snapshots (from Discovered Store, written by Discovery Scheduler) +- Realized State snapshots (from Realized Store) +- Resource Type Specifications (for field criticality declarations used in severity classification) +- Active governance profile (for magnitude thresholds used in severity classification) + +**Outputs:** +- Drift records (written to Drift Record Store — a lightweight operational store) +- Drift events published to the Request Orchestrator: `drift.detected`, `drift.resolved`, `drift.severity_escalated` +- Unsanctioned change events: `unsanctioned_change.detected` + +### 7.3 Comparison Algorithm + +``` +Discovery cycle completes → Discovered State snapshot written + │ + ▼ Drift Reconciliation Component receives discovery.cycle_complete event + │ + ▼ For each entity UUID in the discovery snapshot: + │ + │ Load: latest Realized State snapshot for entity UUID + │ Load: Discovered State snapshot (just written) + │ Load: Resource Type Specification (field criticality per field) + │ + ▼ Field-by-field comparison: + │ For each field in Realized State: + │ Does Discovered State contain this field? + │ If yes: are the values equal? + │ If no: field is absent — severity based on field criticality + │ For each field in Discovered State not in Realized State: + │ New field appeared — severity based on field criticality + │ + ▼ Severity classification (per field): + │ Field criticality (from Resource Type Spec) × Change magnitude (profile-governed) + │ → severity matrix → minor | significant | critical + │ Unsanctioned? → elevate one level + │ Multiple drifted fields? → overall = highest individual severity + │ + ▼ Unsanctioned check: + │ Is there a Requested State record that explains this change? + │ If yes: sanctioned change (may still be drift if realization didn't match) + │ If no: unsanctioned_change.detected event fired (in addition to drift.detected) + │ + ├── No drift detected: + │ Update entity.last_discovered_at + │ Update entity.drift_status = clean + │ No drift record created + │ + └── Drift detected: + Create drift record + Publish drift.detected to Request Orchestrator + Policy Engine evaluates → response action +``` + +### 7.4 Drift Record Structure + +```yaml +drift_record: + uuid: + entity_uuid: + detected_at: + discovery_snapshot_uuid: # the Discovered State snapshot that triggered this + realized_state_uuid: # the Realized State snapshot compared against + + overall_severity: minor | significant | critical + unsanctioned: true | false # true if no corresponding Requested State record + + drifted_fields: + - field_path: "fields.memory_gb" + realized_value: 8 + discovered_value: 16 + field_criticality: medium # from Resource Type Spec + change_magnitude: significant # 100% increase, threshold: standard 10-50% + field_severity: significant + elevated_for_unsanctioned: true # elevated from significant → critical + + status: open | acknowledged | resolved | escalated + resolution: + resolved_at: + resolution_type: reverted | updated_definition | accepted | escalated | null + resolved_by_requested_state_uuid: +``` + +### 7.5 Drift Resolution Tracking + +Drift records are not resolved by the Drift Reconciliation Component — they are resolved by the Policy Engine's response actions. The Drift Reconciliation Component monitors for resolution: + +``` +REVERT action taken: + New Requested State submitted → provider reverts → new Realized State written + Next discovery cycle: Discovered State matches new Realized State + Drift Reconciliation: no drift detected → drift_record.status = resolved + drift.resolved event published + +UPDATE_DEFINITION action taken: + Consumer submits UPDATE_DEFINITION → new Realized State written with discovered values + Next discovery cycle: Discovered State matches new Realized State + Drift record.status = resolved with resolution_type: updated_definition + +Entity decommissioned: + Drift record.status = resolved with resolution_type: decommissioned +``` + +### 7.6 Governance Matrix Integration + +Before classifying a discovered change as drift, the Drift Reconciliation Component evaluates the governance matrix to determine if the change is expected: + +``` +Field value in Discovered State differs from Realized State + │ + ▼ Check: Is there a governance matrix rule that permits this provider + │ to make this type of change to this field? + │ + ├── Yes → This may be a Provider Update Notification that wasn't submitted + │ DCM logs a warning: "Provider changed field without submitting update notification" + │ Still treated as drift — provider should have submitted update notification + │ + └── No → Standard drift detection; severity classification runs +``` + +### 7.7 Drift Reconciliation Policies + +| Policy | Rule | +|--------|------| +| `DRC-001` | The Drift Reconciliation Component never writes to the Realized Store. It produces drift records and events only. | +| `DRC-002` | Drift detection runs after every discovery cycle. An entity with no corresponding Realized State record is an orphan candidate — not a drift event. | +| `DRC-003` | Unsanctioned changes are always elevated one severity level above the matrix classification. An unsanctioned significant drift is reported as critical. | +| `DRC-004` | Drift records are retained until the entity is decommissioned plus the configured audit retention period. They are not deleted on resolution — resolution is recorded within the record. | +| `DRC-005` | Drift detection produces events into the Request Orchestrator. The Policy Engine determines the response action. The Drift Reconciliation Component does not initiate remediation directly. | + + +## 8. Related Policies — Full Component Set + +| Policy | Rule | +|--------|------| +| `CTL-001` | The Request Orchestrator is the single event bus for all request lifecycle events. No component communicates directly with another component outside of events published to the Request Orchestrator. | +| `CTL-002` | Policies ARE the orchestration. The Request Orchestrator does not contain hardcoded pipeline logic. | +| `CTL-003` | Dynamic and static flows compose naturally. Static flows are Policy Groups with concern_type: orchestration_flow and ordered: true. | +| `CTL-004` | Cost Analysis is not a billing system. It provides cost signals for placement and attribution. | +| `CTL-005` | Cost Analysis unavailability never blocks placement. | +| `PLC-001` through `PLC-006` | Placement Engine policies (see Section 4.7) | +| `LCE-001` through `LCE-005` | Lifecycle Constraint Enforcer policies (see Section 5.7) | +| `SIX-001` through `SIX-004` | Search Index policies (see Section 6.6) | +| `DRC-001` through `DRC-005` | Drift Reconciliation policies (see Section 7.7) | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/architecture/control-plane/internal-component-auth.md b/architecture/control-plane/internal-component-auth.md new file mode 100644 index 0000000..8a3e27d --- /dev/null +++ b/architecture/control-plane/internal-component-auth.md @@ -0,0 +1,379 @@ +--- +Maps to: udlm/governance/accreditation-and-authorization-matrix.md +--- + +# DCM Data Model — Internal Component Authentication + +> **Implements contracts defined in UDLM**: +> [udlm/governance/accreditation-and-authorization-matrix.md](https://github.com/croadfeldt/udlm/blob/main/governance/accreditation-and-authorization-matrix.md). +> UDLM defines the five-check boundary model and the "network position grants zero trust" +> principle. This document specifies how DCM applies that boundary model to internal +> control-plane component-to-component authentication. + +**Document Status:** ✅ Complete +**Document Type:** Architecture Reference — Zero Trust Internal Auth +**Related Documents:** [Accreditation and Zero Trust](https://github.com/croadfeldt/udlm/blob/main/governance/accreditation-and-authorization-matrix.md) | [Deployment and Redundancy](../runtime-features/deployment-redundancy.md) | [credential management service Model](https://github.com/croadfeldt/udlm/blob/main/governance/credentials.md) | [Auth Providers](https://github.com/croadfeldt/udlm/blob/main/governance/auth-providers.md) | [Session Revocation](session-revocation.md) | [Design Priorities](https://github.com/croadfeldt/udlm/blob/main/design-principles/design-priorities.md) + +> **This document maps to: DATA + POLICY** +> +> Internal component identities are Data — each component has a UUID, certificate, and service account. Internal auth is Policy — the same five-check boundary model from doc 26 applies at every internal call boundary, with no exceptions for "trusted internal network." This document specifies how DCM's control plane components authenticate to each other in a distributed deployment. + +--- + +## 1. The Core Principle + +**Network position grants zero trust.** This is stated in doc 26 for external interactions. It applies equally to internal component communication. A call from the Policy Engine to the Placement Engine receives the same boundary checks as a call from an external consumer. The service mesh enforces this at the infrastructure level; DCM enforces it at the application level. + +**Two-layer enforcement:** +1. **Mesh layer (infrastructure):** mTLS mutual authentication (RFC 8446 TLS 1.3), certificate validation (RFC 5280), traffic policies — enforced by the service mesh (Istio or equivalent) +2. **Application layer (DCM):** component identity verification, operation authorization, scoped interaction credentials — enforced by DCM's Ingress and Auth subsystems + +Neither layer alone is sufficient. The mesh layer prevents impersonation at the transport level; the application layer enforces what each component is permitted to do. + +--- + +## 2. Component Identity Model + +Every DCM control plane component has a **component identity** — a stable, verifiable identity used for both mTLS and application-layer authorization. + +```yaml +component_identity: + component_uuid: # stable; assigned at deployment time + component_type: api_gateway | policy_engine | placement_engine | request_orchestrator | + scoring_engine | drift_reconciler | lifecycle_enforcer | notification_router | + audit_store | session_store | message_bus | service_provider_proxy + component_name: # human-readable; e.g. "policy-engine-eu-west-1" + deployment_uuid: # identifies the DCM deployment instance + + # Certificate identity + mtls_certificate: + subject: "CN=-,O=dcm-internal" + san: [, , ] + issuer_ca: + issued_at: + expires_at: + + # Service account (application layer) + service_account_uuid: # DCM actor of type "component_service_account" + allowed_operations: [] # what this component may call + allowed_targets: [] # which components it may call +``` + +### 2.1 Component Types and Communication Graph + +Not every component may call every other. The allowed communication graph is declared and enforced: + +``` +Consumer/Admin/Provider → API Gateway +API Gateway → Request Orchestrator +API Gateway → Policy Engine (for direct policy evaluation) +API Gateway → Session Store (token validation) + +Request Orchestrator → Policy Engine +Request Orchestrator → Placement Engine +Request Orchestrator → Scoring Engine +Request Orchestrator → Audit Store +Request Orchestrator → Message Bus + +Policy Engine → Audit Store +Policy Engine → Message Bus (policy evaluation events) + +Placement Engine → Audit Store +Placement Engine → Message Bus + +Scoring Engine → Audit Store + +Drift Reconciler → API Gateway (discovery dispatch) +Drift Reconciler → Audit Store +Drift Reconciler → Message Bus + +Lifecycle Enforcer → API Gateway (decommission dispatch) +Lifecycle Enforcer → Audit Store +Lifecycle Enforcer → Message Bus + +Notification Router → Message Bus (subscribe) +Notification Router → credential management service Proxy (notification channel credentials) + +All components → Session Store (revocation check) +All components → credential management service Proxy (interaction credential requests) +``` + +**ICOM-004:** Components may only call components declared in their `allowed_targets` list. A call from an unexpected source component is rejected with `403 Forbidden` and an audit record. + +--- + +## 3. Certificate Issuance and Internal CA + +### 3.1 Certificate Authority for Internal Components + +Each DCM deployment uses a **registered Certificate Authority (CA)** for issuing component mTLS certificates. This may be: + +**Option A — Built-in Internal CA (default):** DCM operates its own CA per deployment. Simple to configure; no external dependencies; suitable for minimal through standard profiles. + +**Option B — External CA via credential management service:** An enterprise CA registered as a credential management service (HashiCorp Vault PKI, Venafi TLS Protect, EJBCA, AWS ACM Private CA, Azure Key Vault). The external CA issues component certificates using the standard credential management service interface — DCM requests certificates via the provider's API (ACME/EST/SCEP/CMP). See [credential management service Model](https://github.com/croadfeldt/udlm/blob/main/governance/credentials.md) for registration. Recommended for fsi and sovereign profiles where the enterprise PKI chain must be maintained. + +Both options satisfy ICOM-001 (mTLS required). The distinction is who issues the certificates, not whether mTLS is used. + +**The registered CA's root certificate is installed in all component trust stores at deployment time.** For Option B, the credential management service's CA root (which may itself be a subordinate of an enterprise root) is the trust anchor. + +```yaml +internal_ca: + ca_uuid: + deployment_uuid: + ca_type: built_in | external_service_provider + service_provider_uuid: # if ca_type: external + external_ca_protocol: acme | est | scep | cmp | null # if external + root_cert_fingerprint: + certificate_lifetime: P90D # profile-governed — see table below + renewal_trigger: P14D + algorithm: ECDSA-P-384 # FIPS-compliant; all profiles + crl_endpoint: + ocsp_endpoint: +``` + +```yaml +internal_ca: + ca_uuid: + deployment_uuid: + root_cert_fingerprint: + certificate_lifetime: P90D # all component certs valid 90 days + renewal_trigger: P14D # renew 14 days before expiry + algorithm: ECDSA-P-384 # FIPS-compliant for all profiles + crl_endpoint: # revocation list for component certs + ocsp_endpoint: # online status check +``` + +### 3.2 Profile-Governed Certificate Configuration + +| Profile | Cert lifetime | Renewal trigger | Bootstrap token TTL | Min key algorithm | +|---------|--------------|-----------------|--------------------|--------------------| +| `minimal` | P180D | P30D | PT4H | RSA-2048 (min) | +| `dev` | P90D | P14D | PT1H | RSA-2048 (min) | +| `standard` | P90D | P14D | PT1H | ECDSA-P-256 (min) | +| `prod` | P90D | P14D | PT1H | ECDSA-P-384 | +| `fsi` | P30D | P7D | PT30M | ECDSA-P-384 | +| `sovereign` | P14D | P3D | PT15M | ECDSA-P-384 (HSM-backed if hardware_attested) | + +> **sovereign profile:** Certificates must be HSM-backed if the deployment posture is `hardware_attested`. The external CA option (Option B) using an HSM-backed Vault PKI backend satisfies this requirement. + +### 3.3 Certificate Lifecycle + +``` +Component starts + │ + ▼ Does component have a valid certificate? + │ YES → Use existing certificate + │ NO (first start or expired) → Request certificate from Internal CA + │ + ▼ Certificate request to Internal CA: + │ component_uuid, component_type, deployment_uuid + │ CSR signed with bootstrap key (see Section 5) + │ + ▼ Internal CA issues certificate + │ Subject: CN=-,O=dcm-internal + │ SAN: component_uuid, component_name, internal DNS name + │ Valid for: P90D (profile-governed) + │ + ▼ Component stores certificate; begins accepting mTLS connections + │ + ▼ 14 days before expiry: auto-renewal + Background thread requests new certificate + Transition: both old and new cert valid for PT1H + Old cert retired after transition +``` + +--- + +## 4. Application-Layer Authorization + +mTLS verifies **who** is calling. Application-layer authorization verifies **what** the caller is permitted to do. + +### 4.1 Interaction Credential for Internal Calls + +Every internal component call follows the same ZTS-002 scoped interaction credential model used for external provider dispatch: + +``` +Component A prepares to call Component B + │ + ▼ Request interaction credential from credential management service Proxy: + │ credential_type: dcm_interaction + │ issued_to.component_uuid: + │ scope.operations: [] + │ scope.target_component: + │ expires_at: + │ + ▼ Call Component B with: + │ mTLS certificate (transport identity) + │ Interaction credential in Authorization header (operation authorization) + │ Correlation ID (tracing) + │ + ▼ Component B validates: + │ 1. mTLS cert from Component A's known CA ✓ + │ 2. Interaction credential: not revoked, not expired, scoped to this operation ✓ + │ 3. Component A is in allowed_sources for this endpoint ✓ + │ 4. Operation matches declared scope ✓ + │ → All pass: process request + │ → Any fail: 403 + audit record ICOM_AUTH_FAILURE +``` + +### 4.2 Internal Endpoint Authorization + +Each internal component endpoint declares which source components are permitted to call it: + +```yaml +internal_endpoint: + component: policy_engine + endpoint: POST /internal/evaluate + allowed_sources: + - api_gateway + - request_orchestrator + required_scope: policy.evaluate + audit_every_call: true # all internal calls are audited +``` + +**ICOM-003:** Internal endpoints that receive calls from unauthorized source components return 403 and write an `ICOM_UNAUTHORIZED_SOURCE` audit record. This audit record has urgency: high — unexpected internal call patterns are security signals. + +--- + +## 5. Bootstrap — First Certificate + +The bootstrap problem: a new component needs a certificate, but it has no certificate yet to authenticate its request. DCM solves this with a **bootstrap token** mechanism. + +### 5.1 Bootstrap Token + +At deployment time, the platform admin generates a one-time bootstrap token for each component: + +``` +POST /api/v1/admin/components/bootstrap-tokens + +{ + "component_type": "policy_engine", + "component_uuid": "", + "deployment_uuid": "", + "expires_at": "" // short-lived: PT1H maximum +} + +Response 201: +{ + "bootstrap_token": "", // one-time use; stored as env var or secret + "component_uuid": "", + "expires_at": "" +} +``` + +### 5.2 First Certificate Acquisition + +``` +New component starts with bootstrap_token in environment + │ + ▼ POST /internal/ca/issue-certificate + │ Authorization: Bootstrap + │ Body: { component_uuid, component_type, deployment_uuid, csr_pem } + │ + ▼ Internal CA validates: + │ Bootstrap token not expired + │ Bootstrap token not previously used (one-time) + │ component_uuid matches token's declared component_uuid + │ + ▼ Certificate issued + │ Bootstrap token invalidated immediately after use + │ + ▼ Component uses certificate for all subsequent communication + No further need for bootstrap token +``` + +**ICOM-007:** Bootstrap tokens are one-time-use and must expire within PT1H of creation. A bootstrap token that is not used within PT1H is automatically invalidated. Platform admins must generate new tokens if a component fails to start within the window. + +### 5.3 Kubernetes Deployment Integration + +In Kubernetes deployments, bootstrap tokens are injected as Kubernetes Secrets and mounted as environment variables. The component reads the bootstrap token on startup, acquires its certificate, then deletes the Kubernetes Secret. This ensures the bootstrap credential is not persisted beyond initial use. + +```yaml +# Kubernetes Secret (deleted by component after first cert acquisition) +apiVersion: v1 +kind: Secret +metadata: + name: dcm-policy-engine-bootstrap +type: Opaque +stringData: + DCM_BOOTSTRAP_TOKEN: "" + DCM_COMPONENT_UUID: "" + DCM_INTERNAL_CA_ENDPOINT: "https://dcm-internal-ca.dcm-system.svc.cluster.local" +``` + +--- + +## 6. Certificate Compromise Response + +If a component certificate is compromised, the response follows the same emergency pattern as credential compromise: + +``` +Certificate compromise detected + │ + ▼ Compromised cert added to Internal CA CRL + │ CRL update propagated to all components within SLA: + │ standard/prod: PT1M + │ fsi/sovereign: PT15S + │ + ▼ Component identity suspended in DCM + │ All active interaction credentials for this component → revoked + │ ICOM_CERT_COMPROMISED audit record written + │ + ▼ Platform admin notified (urgency: critical) + │ + ▼ New certificate issued for legitimate component instance + │ Previous certificate remains in CRL permanently + │ + ▼ Component resumes with new certificate +``` + +**ICOM-008:** Compromised internal component certificates are added to the Internal CA CRL immediately. All other components refresh their CRL cache within the profile-governed SLA and reject connections presenting the revoked certificate. + +--- + +## 7. Deployment Architecture Summary + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ DCM Control Plane │ +│ │ +│ ┌──────────┐ mTLS+cred ┌──────────────────┐ │ +│ │API Gateway│───────────→│Request Orchestrator│ │ +│ └──────────┘ └────────┬─────────┘ │ +│ │ mTLS+cred (each call) │ +│ ┌───────────────┼───────────────┐ │ +│ ↓ ↓ ↓ │ +│ ┌─────────────┐ ┌──────────┐ ┌──────────────┐ │ +│ │Policy Engine│ │Placement │ │Scoring Engine│ │ +│ └─────────────┘ │Engine │ └──────────────┘ │ +│ └──────────┘ │ +│ │ +│ ┌──────────────────┐ ┌─────────────┐ ┌──────────────────┐ │ +│ │credential management service│ │Session Store│ │Internal CA │ │ +│ │Proxy │ │ │ │(cert authority) │ │ +│ └──────────────────┘ └─────────────┘ └──────────────────┘ │ +│ │ +│ Service Mesh (Istio): mTLS enforcement at transport layer │ +│ All calls: authenticated + authorized + audited │ +└─────────────────────────────────────────────────────────────────┘ +``` + +--- + +## 8. System Policies + +| Policy | Rule | +|--------|------| +| `ICOM-001` | All internal component-to-component communication must use mTLS with certificates issued by the deployment's Internal CA. Plaintext internal communication is prohibited in all profiles. | +| `ICOM-002` | Every internal call must present a scoped interaction credential (ZTS-002) in addition to the mTLS certificate. The mTLS certificate proves identity; the interaction credential proves authorization for the specific operation. | +| `ICOM-003` | Internal endpoints reject calls from components not in their `allowed_sources` list with 403 and an `ICOM_UNAUTHORIZED_SOURCE` audit record (urgency: high). | +| `ICOM-004` | Components may only call components declared in their `allowed_targets` list. Attempts to call unauthorized components are rejected at the mesh layer (traffic policy) and, if they reach the application layer, at the application layer. | +| `ICOM-005` | All internal component calls are audited: source component, target component, operation, interaction credential UUID, outcome. Internal audit records are written to the same Audit Store as external interactions. | +| `ICOM-006` | Component certificates are issued by the Internal CA with a maximum validity of P90D and renewed automatically P14D before expiry. Component certificates may not be issued by external CAs. | +| `ICOM-007` | Bootstrap tokens are one-time-use and expire within PT1H. A bootstrap token that has been used is immediately invalidated. Unused tokens are invalidated at expiry. | +| `ICOM-008` | Compromised internal component certificates are added to the Internal CA CRL immediately. All components refresh their CRL cache within the profile-governed SLA. | +| `ICOM-009` | The trust anchor for internal component mTLS is a registered root or intermediate CA whose certificate is installed in all component trust stores at deployment time. The trust anchor may be the built-in Internal CA or an external CA registered as a Certificate Provider (e.g. HashiCorp Vault PKI, Venafi, EJBCA) — see [credential management service Model](https://github.com/croadfeldt/udlm/blob/main/governance/credentials.md) Section on External CAs. Components do not accept certificates from unregistered trust anchors. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/architecture/control-plane/self-health.md b/architecture/control-plane/self-health.md new file mode 100644 index 0000000..f45fa5f --- /dev/null +++ b/architecture/control-plane/self-health.md @@ -0,0 +1,463 @@ +--- +Document Status: ✅ Complete +Document Type: Architecture Reference — Operational Health +Maps to: udlm/contracts/provider-contract.md +--- + +# DCM Data Model — DCM Self-Health Endpoints + +> **Implements contracts defined in UDLM**: +> [udlm/contracts/provider-contract.md](https://github.com/croadfeldt/udlm/blob/main/contracts/provider-contract.md). +> UDLM defines the provider health contract that Service Providers must expose. +> DCM operationalizes its own liveness, readiness, and component-health +> endpoints — DCM exposes the same health contract it requires of the providers +> it integrates, so it is itself a well-behaved provider to its operator and +> load-balancer. + +**Document Status:** ✅ Complete +**Document Type:** Architecture Reference — Operational Health +**Related Documents:** [Deployment and Redundancy](../runtime-features/deployment-redundancy.md) | [Internal Component Authentication](internal-component-auth.md) | [Operator Interface Specification](../../docs/specifications/dcm-operator-interface-spec.md) | [Admin API Specification](../../docs/specifications/dcm-admin-api-spec.md) + +> **Events:** Health state change events fire as `provider.healthy` / `provider.unhealthy` for external systems, and `governance.profile_changed` when health thresholds are adjusted. See [Event Catalog](https://github.com/croadfeldt/udlm/blob/main/contracts/event-catalog.md). + +> **This document maps to: PROVIDER** +> +> DCM itself must expose the same health contract it requires of Service Providers (doc OIS §4). This document specifies DCM's own liveness, readiness, and component health endpoints — required for Kubernetes operator deployment, load balancer health checking, and operational monitoring. + +--- + +## 1. Three Health Endpoints + +DCM exposes three distinct health endpoints, following Kubernetes conventions: + +| Endpoint | Purpose | Failure action | Authentication | +|----------|---------|---------------|---------------| +| `GET /livez` | Is DCM alive? | Kubernetes restarts the pod | None | +| `GET /readyz` | Is DCM ready to serve traffic? | Kubernetes removes from load balancer | None | +| `GET /api/v1/admin/health` | Detailed component status | Informational — no automatic action | Admin auth required | + +Liveness and readiness are unauthenticated because they must work before authentication infrastructure is operational (e.g. during startup). + +--- + +## 2. Liveness — `/livez` + +Liveness answers: **is this DCM process alive?** + +A liveness failure means the process is deadlocked, in an unrecoverable state, or otherwise unable to continue. Kubernetes responds by restarting the pod. + +```http +GET /livez HTTP/1.1 + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass" +} +``` + +**Liveness checks (minimal — fast):** +- Process is responding +- No deadlock detected in core event loop +- Internal CA is reachable (for deployments with component auth) + +**Liveness failure response:** +```http +HTTP/1.1 503 Service Unavailable +Content-Type: application/health+json + +{ + "status": "fail", + "failure_reason": "event_loop_deadlock | internal_ca_unreachable | oom_imminent" +} +``` + +**Liveness SLA:** Must respond within PT5S. No external calls. No database reads. + +--- + +## 3. Readiness — `/readyz` + +Readiness answers: **is this DCM instance ready to serve requests?** + +A readiness failure removes the instance from the load balancer rotation without restarting it. This handles startup, migration, and graceful drain scenarios. + +```http +GET /readyz HTTP/1.1 + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass", + "checks": { + "session_store": "pass", + "audit_store": "pass", + "policy_engine": "pass", + "message_bus": "pass", + "auth_provider": "pass" + } +} +``` + +**Readiness checks:** +- Session Store: can write and read a test record +- Audit Store: reachable and writable +- Policy Engine: responding to internal health ping +- Message Bus: connected and subscribed +- Auth Provider: at least one Auth Provider is responding +- Schema version: database schema matches running code version + +**Readiness failure** (any check fails): +```http +HTTP/1.1 503 Service Unavailable +Content-Type: application/health+json + +{ + "status": "fail", + "checks": { + "session_store": "pass", + "audit_store": "fail", + "policy_engine": "pass", + "message_bus": "pass", + "auth_provider": "pass" + }, + "failing_checks": ["audit_store"] +} +``` + +**Readiness SLA:** Must respond within PT10S. Performs lightweight connectivity checks — no heavy queries. + +### 3.1 Startup vs Operational Readiness + +During startup, DCM goes through a startup sequence before becoming ready: + +``` +Process starts + │ + ▼ /livez → pass (process alive) + │ /readyz → fail (not ready yet) + │ + ▼ Internal CA connects → component certs verified + ▼ Session Store connected → revocation registry loaded + ▼ Audit Store connected → schema version validated + ▼ Policy Engine ready → policies loaded and shadow mode initialized + ▼ Auth Providers connected → at least one responding + ▼ Message Bus connected → subscriptions established + │ + ▼ /readyz → pass (ready to serve traffic) +``` + +`startupProbe` in Kubernetes uses `/readyz` with a longer `failureThreshold` to allow startup time before the liveness probe takes over. + +--- + +## 4. Detailed Health — `/api/v1/admin/health` + +The detailed health endpoint provides per-component status for operational monitoring. Requires admin authentication. + +```http +GET /api/v1/admin/health HTTP/1.1 +Authorization: Bearer + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass | warn | fail", + "dcm_version": "1.2.0", + "dcm_instance_uuid": "", + "deployment_profile": "prod", + "uptime_seconds": 864023, + "checked_at": "", + + "components": { + "api_gateway": { + "status": "pass", + "latency_p99_ms": 12, + "requests_per_minute": 340 + }, + "request_orchestrator": { + "status": "pass", + "queue_depth": 3, + "in_flight": 7 + }, + "policy_engine": { + "status": "pass", + "active_policies": 42, + "shadow_policies": 3, + "evaluations_per_minute": 280 + }, + "placement_engine": { + "status": "pass" + }, + "scoring_engine": { + "status": "pass", + "evaluations_per_minute": 280 + }, + "request_scheduler": { + "status": "pass", + "scheduled_requests_queued": 5, + "next_dispatch_at": "" + }, + "drift_reconciler": { + "status": "pass", + "last_cycle_completed_at": "", + "open_drift_records": 2 + }, + "lifecycle_enforcer": { + "status": "pass", + "entities_monitored": 1240, + "ttl_warnings_pending": 3 + }, + "discovery_scheduler": { + "status": "pass", + "pending_jobs": 1, + "last_completed_at": "" + }, + "notification_router": { + "status": "pass", + "providers_active": 2, + "delivery_backlog": 0 + }, + "session_store": { + "status": "pass", + "active_sessions": 47, + "revocation_registry_size": 3 + }, + "audit_store": { + "status": "pass", + "records_last_hour": 1840, + "chain_integrity": "verified" + }, + "message_bus": { + "status": "pass", + "lag_consumer_group_ms": 12 + }, + "internal_ca": { + "status": "pass", + "certificates_active": 12, + "next_expiry_at": "" + } + }, + + "providers": { + "registered": 4, + "healthy": 4, + "degraded": 0, + "unhealthy": 0 + }, + + "auth_providers": { + "registered": 2, + "healthy": 2, + "unhealthy": 0 + } +} +``` + +### 4.1 Status Semantics + +| Status | Meaning | +|--------|---------| +| `pass` | Component fully operational | +| `warn` | Operational but degraded (high latency, reduced capacity, elevated error rate) | +| `fail` | Component not operational; DCM degraded | + +The top-level `status` is the worst status across all components: +- Any `fail` → top-level `fail` +- Any `warn`, no `fail` → top-level `warn` +- All `pass` → top-level `pass` + +--- + +## 5. Kubernetes Manifest + +```yaml +# Standard Kubernetes probe configuration for DCM +livenessProbe: + httpGet: + path: /livez + port: 8443 + scheme: HTTPS + initialDelaySeconds: 10 + periodSeconds: 10 + failureThreshold: 3 + timeoutSeconds: 5 + +readinessProbe: + httpGet: + path: /readyz + port: 8443 + scheme: HTTPS + initialDelaySeconds: 30 + periodSeconds: 5 + failureThreshold: 6 + timeoutSeconds: 10 + +startupProbe: + httpGet: + path: /readyz + port: 8443 + scheme: HTTPS + initialDelaySeconds: 10 + periodSeconds: 10 + failureThreshold: 30 # allow up to 300s for startup + timeoutSeconds: 10 +``` + +--- + +## 6. Prometheus Metrics + +DCM exposes Prometheus-compatible metrics alongside health endpoints: + +``` +GET /metrics # Prometheus scrape endpoint (unauthenticated in cluster; + # configurable for external exposure) +``` + +Key metric families: + +``` +# Request pipeline +dcm_requests_total{status, resource_type, profile} +dcm_request_duration_seconds{quantile, resource_type} +dcm_requests_pending_dependency_total +dcm_requests_scheduled_total + +# Policy engine +dcm_policy_evaluations_total{outcome, enforcement_class} +dcm_policy_shadow_divergences_total + +# Sessions +dcm_sessions_active_total +dcm_session_revocations_total{trigger} + +# Drift +dcm_drift_open_records_total{severity} +dcm_drift_detected_total + +# Providers +dcm_providers_registered_total +dcm_providers_healthy_total +dcm_provider_dispatch_duration_seconds{provider_type, quantile} + +# Internal +dcm_internal_ca_certificates_active +dcm_internal_ca_days_until_next_expiry +``` + +--- + +## 7. Per-Provider Metrics Contract + +In addition to DCM control plane metrics, each registered Service Provider must +expose a Prometheus-compatible `/metrics` endpoint meeting the contract defined in +the Registration Specification (GATE-SP-05). + +### 7.1 Required Provider Metric Families + +``` +# Dispatch metrics — how many requests DCM sent to this provider +dcm_provider_dispatches_total{resource_type="Compute.VirtualMachine", outcome="success|failed|timeout"} +dcm_provider_dispatch_duration_seconds{resource_type="Compute.VirtualMachine", quantile="0.5|0.95|0.99"} + +# Realization metrics — outcomes of provisioning +dcm_provider_realizations_total{resource_type="Compute.VirtualMachine", status="OPERATIONAL|FAILED"} + +# Health signal +dcm_provider_health_status # gauge: 1=healthy, 0.5=degraded, 0=unhealthy +``` + +### 7.2 Recommended Provider Metric Families + +``` +# Capacity +dcm_provider_capacity_remaining{resource_type="Compute.VirtualMachine"} # gauge +dcm_provider_capacity_total{resource_type="Compute.VirtualMachine"} # gauge + +# Queue depth (for async-only providers) +dcm_provider_queue_depth{resource_type="Compute.VirtualMachine"} # gauge + +# Tenant usage +dcm_provider_active_resources_total{tenant_uuid="...", resource_type="..."} +``` + +### 7.3 DCM Control Plane Aggregated Provider Metrics + +The DCM control plane exposes aggregated provider metrics at its own `/metrics` +endpoint alongside the control plane metrics from Section 6: + +``` +# Already in Section 6 — shown here for cross-reference +dcm_providers_registered_total # count of registered providers +dcm_providers_healthy_total # count currently healthy +dcm_provider_dispatch_duration_seconds{...} # aggregated across all providers +``` + +### 7.4 Tenant Metadata Endpoint + +Service Providers in `standard` and above profiles (GATE-SP-04) must implement +a tenant metadata endpoint that DCM calls to retrieve per-tenant usage summaries: + +``` +GET /api/v1/tenants/{tenant_uuid}/metadata +Authorization: Bearer + +Response 200: +{ + "tenant_uuid": "", + "active_resources": { + "Compute.VirtualMachine": 12, + "Storage.Block": 8 + }, + "capacity_consumed": { + "Compute.VirtualMachine": { + "cpu_cores": 96, + "ram_gb": 384 + } + }, + "quota_consumed_pct": { + "Compute.VirtualMachine": 48.0 + } +} +``` + +This data is used by DCM's Cost Analysis component and multi-tenant quota +enforcement. It is also exposed to tenant administrators via the consumer API. + +--- + +## 8. Profile-Governed Health Exposure + +| Profile | /livez | /readyz | /api/v1/admin/health | /metrics scraping | +|---------|--------|---------|----------------------|-------------------| +| `minimal` | Unauthenticated | Unauthenticated | Admin auth | Internal network only | +| `dev` | Unauthenticated | Unauthenticated | Admin auth | Internal network only | +| `standard` | Unauthenticated | Unauthenticated | Admin auth | mTLS client cert or auth token | +| `prod` | Unauthenticated | Unauthenticated | Admin auth | mTLS client cert or auth token | +| `fsi` | Unauthenticated | Unauthenticated | Admin auth (MFA required) | mTLS + authorized scraper registration | +| `sovereign` | Unauthenticated within cluster | Unauthenticated within cluster | Admin auth (MFA + step-up) | Disabled externally; internal only | + +**Notes:** +- `/livez` and `/readyz` are always unauthenticated *within the cluster* — Kubernetes probes cannot present auth credentials. However, at the ingress boundary (external load balancer), these paths may be network-restricted. +- For `fsi` and `sovereign` profiles, `/api/v1/admin/health` requires MFA-verified sessions (mfa_verified: true). Step-up MFA is required for sovereign. +- The `sovereign` profile does not expose `/metrics` externally. Prometheus must scrape from within the cluster network only. +- Component-level detail in `/api/v1/admin/health` may be redacted in fsi/sovereign profiles based on the requesting actor's role — SRE sees full detail; read-only admin sees summary only. + +## 7. System Policies + +| Policy | Rule | +|--------|------| +| `HLT-001` | DCM must expose `/livez` and `/readyz` endpoints on the same port as the API, unauthenticated, following RFC 8615 / IANA health+json format. | +| `HLT-002` | `/livez` must respond within PT5S with no external calls or database reads. A non-response within PT5S is treated as liveness failure. | +| `HLT-003` | `/readyz` returns `fail` if the Session Store, Audit Store, Policy Engine, Message Bus, or any Auth Provider is unreachable. It returns `warn` if any optional component is degraded. | +| `HLT-004` | `GET /api/v1/admin/health` requires admin authentication and provides per-component status. It must include the DCM version, instance UUID, and deployment profile. | +| `HLT-005` | DCM must expose Prometheus-compatible metrics at `GET /metrics`. Metrics must include request pipeline, policy engine, session, drift, and provider metrics at minimum. | +| `HLT-006` | The startup sequence must be observable via `/readyz`. DCM must not report `pass` on `/readyz` until the Session Store, Audit Store, Policy Engine, Auth Provider, and Message Bus are all reachable. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/architecture/control-plane/session-revocation.md b/architecture/control-plane/session-revocation.md new file mode 100644 index 0000000..7293c7f --- /dev/null +++ b/architecture/control-plane/session-revocation.md @@ -0,0 +1,361 @@ +--- +Maps to: udlm/governance/auth-providers.md +--- + +# DCM Data Model — Session Token Revocation + +> **Implements contracts defined in UDLM**: +> [udlm/governance/auth-providers.md](https://github.com/croadfeldt/udlm/blob/main/governance/auth-providers.md). +> UDLM defines the Auth Provider model and authenticated-session contract. This document +> extends that model with the explicit session-token revocation lifecycle, triggers, and +> the Session Revocation Registry that UDLM leaves unspecified. + +**Document Status:** ✅ Complete +**Document Type:** Architecture Reference — Session Lifecycle and Revocation +**Related Documents:** [Auth Providers](https://github.com/croadfeldt/udlm/blob/main/governance/auth-providers.md) | [credential management service Model](https://github.com/croadfeldt/udlm/blob/main/governance/credentials.md) | [Accreditation and Zero Trust](https://github.com/croadfeldt/udlm/blob/main/governance/accreditation-and-authorization-matrix.md) | [Event Catalog](https://github.com/croadfeldt/udlm/blob/main/contracts/event-catalog.md) | [Design Priorities](https://github.com/croadfeldt/udlm/blob/main/design-principles/design-priorities.md) + +> **This document maps to: DATA + POLICY** +> +> A session is a Data artifact with a UUID, lifecycle state, and audit trail. Session revocation is a Policy concern — it fires on triggers defined here and enforced by the Auth Provider and Ingress layer. This document extends the Auth Provider model (doc 19) with the explicit revocation lifecycle that was previously unspecified. +> +> **Relationship to credential revocation:** CPX-006 (doc 31) governs credential revocation — when an actor is deprovisioned, all credentials issued to that actor are revoked. This document governs the complementary concern: active *session tokens* must also be invalidated on the same trigger. Credential revocation and session revocation are parallel processes that both fire on actor deprovisioning. + +--- + +## 1. What a Session Is + +A DCM session represents an authenticated actor's active interaction context. It is created when an actor successfully authenticates through an Auth Provider and is destroyed (or expires) when the session ends. + +```yaml +session_record: + session_uuid: + actor_uuid: + auth_provider_uuid: # which provider issued the session + auth_method: oidc | ldap | api_key | mtls | built_in + mfa_verified: # whether per-session MFA was completed + step_up_verified_at: # last step-up MFA completion + + created_at: + last_active_at: + expires_at: # absolute expiry (from token_ttl) + + refresh_token_uuid: # if refresh_enabled: true + refresh_expires_at: + + status: active | refreshing | revoked | expired + revocation_reason: + revoked_at: + revoked_by: + + # Provenance + client_ip: + user_agent: + tenant_uuid: + + # Concurrent session position + session_sequence: # 1 = oldest active session for this actor +``` + +### 1.1 Session Store + +Active sessions are maintained in a **Session Store** — a fast-queryable, low-latency store separate from the Realized State Store. The Session Store is not GitOps-backed; it is operational state that does not need version history. + +```yaml +session_store: + implementation: redis | postgres | in_memory # profile-governed + ttl_enforcement: hard # sessions expire at expires_at regardless + revocation_index: true # fast lookup by session_uuid for revocation + actor_index: true # fast lookup by actor_uuid for bulk revocation +``` + +**Profile-governed defaults:** + +| Profile | Store | Session TTL | Refresh TTL | Max concurrent | +|---------|-------|-------------|-------------|---------------| +| `minimal` | in_memory or sqlite | PT8H | P7D | unlimited | +| `dev` | redis or postgres | PT4H | P3D | 10 | +| `standard` | redis or postgres | PT1H | P1D | 5 | +| `prod` | redis or postgres | PT30M | PT8H | 3 | +| `fsi` | redis or postgres | PT15M | PT1H | 2 | +| `sovereign` | redis or postgres (HSM-backed) | PT15M | PT30M | 1 | + +--- + +## 2. Revocation Triggers + +Session revocation invalidates a session immediately — regardless of its remaining TTL. The following triggers cause revocation: + +| Trigger | Scope | Who initiates | Behavior | +|---------|-------|--------------|---------| +| `actor_logout` | Single session | Actor (self) | Immediate; that session only | +| `actor_logout_all` | All sessions for actor | Actor (self) | Immediate; all active sessions for this actor | +| `actor_deprovisioned` | All sessions for actor | SCIM / Platform admin | Immediate; fires before deprovisioning acknowledged | +| `actor_suspended` | All sessions for actor | Platform admin | Immediate | +| `security_event` | Specified sessions or all | Platform admin / security automation | Immediate; emergency channel notification | +| `concurrent_limit_exceeded` | Oldest session(s) | System | Oldest session revoked when new session created beyond limit | +| `auth_provider_deregistered` | All sessions from that provider | Platform admin | Immediate; actors must re-authenticate via another provider | +| `credential_compromised` | All sessions for actor | Security automation | Immediate; correlates with CPX emergency rotation | +| `admin_forced_logout` | Specified session(s) | Platform admin | Immediate | + +--- + +## 3. Revocation Lifecycle + +### 3.1 Standard Revocation + +``` +Revocation trigger fires + │ + ▼ Session record status → revoked + │ revoked_at, revocation_reason, revoked_by written + │ + ▼ Refresh token invalidated (if exists) + │ Cannot be exchanged; refresh endpoint returns 401 + │ + ▼ Session UUID added to Session Revocation Registry + │ (fast-queryable; all DCM components check this on every request) + │ + ▼ Revocation event published to Message Bus + │ event_type: auth.session_revoked + │ session_uuid, actor_uuid, revocation_trigger, revoked_at + │ + ▼ Audit record written + session_uuid, actor_uuid, revocation_trigger, revoked_by, revoked_at +``` + +### 3.2 Actor Deprovisioning Revocation (parallel with CPX-006) + +Actor deprovisioning fires both credential revocation (CPX-006) and session revocation simultaneously. Neither blocks the other; both must complete before the deprovisioning is acknowledged. + +``` +Actor deprovisioning initiated + │ + ├──→ Credential revocation (CPX-006) + │ All credentials issued to actor_uuid → revoked + │ Credential Revocation Registry updated + │ + └──→ Session revocation (this document) + All active sessions for actor_uuid → revoked + Session Revocation Registry updated + auth.session_revoked events published per session + │ + ▼ Both complete → deprovisioning acknowledged + actor_deprovisioned event published + Audit record for deprovisioning written +``` + +### 3.3 Emergency Revocation (Security Event) + +Security events bypass the standard pipeline. Revocation is immediate with no grace period. + +``` +Security event detected + │ + ▼ Target sessions determined + │ (single session, all sessions for actor, or all sessions from a provider) + │ + ▼ Sessions → revoked immediately + │ Session Revocation Registry updated within SLA: + │ standard/prod: PT30S + │ fsi: PT10S + │ sovereign: PT5S + │ + ▼ auth.security_session_revoked event published (critical urgency) + │ Routed to security team via configured notification service + │ + ▼ Platform admin notified regardless of profile + │ + ▼ All in-flight requests from these sessions → 401 Unauthorized +``` + +--- + +## 4. Session Revocation Registry + +The Session Revocation Registry is the authoritative list of revoked-but-not-yet-expired session UUIDs. Every DCM component that accepts bearer tokens must check this registry on each request. + +```yaml +session_revocation_registry: + # Session UUID → revocation record + # Fast in-memory cache with TTL equal to original session TTL + # After the original session TTL would have expired, the entry is + # removed (the session would have been invalid anyway) + + entry: + session_uuid: + revoked_at: + original_expires_at: # entry removed after this time + revocation_trigger: +``` + +**Cache refresh behavior by profile:** + +| Profile | Max cache age | Behavior on cache miss | +|---------|--------------|----------------------| +| `minimal` | PT5M | Check authoritative store; cache result | +| `standard` | PT1M | Check authoritative store; cache result | +| `prod` | PT30S | Check authoritative store; cache result | +| `fsi` | PT10S | Check authoritative store; cache result | +| `sovereign` | PT5S | No cache — always check authoritative store | + +--- + +## 5. Token Introspection + +DCM's Ingress layer exposes a token introspection endpoint for internal components and external systems that need to validate a token without maintaining their own cache: + +``` +POST /api/v1/auth:introspect + +Authorization: Bearer +Content-Type: application/json + +{ + "token": "" +} + +Response 200 (active session): +{ + "active": true, + "session_uuid": "", + "actor_uuid": "", + "expires_at": "", + "mfa_verified": true, + "tenant_uuid": "", + "roles": ["consumer"], + "scopes": ["read", "write"] +} + +Response 200 (revoked or expired): +{ + "active": false, + "reason": "revoked | expired | not_found" +} +``` + +Session tokens use JWT format (RFC 7519). This introspection endpoint follows [RFC 7662 (OAuth 2.0 Token Introspection)](https://datatracker.ietf.org/doc/html/rfc7662). + +--- + +## 6. Consumer API — Session Management Endpoints + +### 6.1 Logout (Single Session) + +``` +DELETE /api/v1/auth/session + +Response 204 No Content +``` + +Revokes the session corresponding to the bearer token in the `Authorization` header. No body required. + +### 6.2 Logout All Sessions + +``` +DELETE /api/v1/auth/sessions + +Response 204 No Content +``` + +Revokes all active sessions for the authenticated actor. + +### 6.3 List Active Sessions + +``` +GET /api/v1/auth/sessions + +Response 200: +{ + "items": [ + { + "session_uuid": "", + "created_at": "", + "last_active_at": "", + "expires_at": "", + "auth_method": "oidc", + "client_ip": "", + "current": true // true for the session making this request + } + ], + "total": 2 +} +``` + +### 6.4 Revoke Specific Session + +``` +DELETE /api/v1/auth/sessions/{session_uuid} + +Response 204 No Content +Response 404: session not found or does not belong to this actor +``` + +### 6.5 Admin: Force Revoke Session(s) + +``` +POST /api/v1/admin/actors/{actor_uuid}:revoke-sessions + +{ + "scope": "all | session", + "session_uuid": "", // required if scope: session + "reason": "" // required for audit trail +} + +Response 204 No Content +Response 404: actor not found +``` + +--- + +## 7. Concurrent Session Enforcement + +When `concurrent_sessions: N` is declared and a new session would exceed the limit, the oldest active session is revoked automatically: + +``` +New authentication succeeds + │ + ▼ Count active sessions for actor_uuid + │ If count >= concurrent_sessions limit: + │ Revoke oldest session (by created_at) + │ Trigger: concurrent_limit_exceeded + │ + ▼ New session created +``` + +The evicted actor receives an `auth.session_revoked` notification if a notification service is configured with the actor's notification preferences. The event does not block the new session creation. + +--- + +## 8. Relationship to the Credential Revocation Model + +Session revocation (this document) and credential revocation (doc 31, CPX-001–CPX-012) are parallel but distinct: + +| | Session Revocation | Credential Revocation | +|--|---|---| +| **What** | Bearer token / session cookie validity | API key, x509, SSH key, service account token | +| **Store** | Session Revocation Registry | Credential Revocation Registry | +| **Propagation** | Auth layer cache refresh | Message Bus → all components | +| **Actor deprovision** | All sessions revoked | All credentials revoked | +| **TTL** | Session TTL (minutes to hours) | Credential TTL (hours to years) | +| **Emergency SLA** | PT5S–PT30S | PT30S–PT5M | +| **Event** | `auth.session_revoked` | `credential.revoked` | + +**AUTH-016:** On actor deprovisioning, session revocation and credential revocation are parallel operations. The deprovisioning is not acknowledged until both are confirmed complete. + +--- + +## 9. System Policies + +| Policy | Rule | +|--------|------| +| `AUTH-016` | On actor deprovisioning, session revocation and credential revocation (CPX-006) are parallel operations. Deprovisioning is not acknowledged until both complete. | +| `AUTH-017` | Session revocation must propagate to the Session Revocation Registry within the profile-governed SLA: minimal PT5M, standard PT1M, prod PT30S, fsi PT10S, sovereign PT5S. | +| `AUTH-018` | All DCM components that accept bearer tokens must check the Session Revocation Registry on each request. Cache age must not exceed the profile-governed maximum (sovereign: no cache). | +| `AUTH-019` | Emergency session revocation (security_event trigger) fires immediately with no grace period. The `auth.security_session_revoked` event has `urgency: critical` and is non-suppressable. | +| `AUTH-020` | The token introspection endpoint (`POST /api/v1/auth:introspect`) must be authenticated. Access requires an actor or service account with the `introspection` scope. | +| `AUTH-021` | When concurrent session limits are enforced, the oldest session is revoked before the new session is created. The evicted actor is notified via notification service if configured. | +| `AUTH-022` | Refresh tokens are invalidated when their parent session is revoked. A revoked refresh token returns 401 on exchange; it cannot be used to create a new session. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*