From 76781d4347b10d711e51654fb4a934d7d33cea75 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Wed, 25 Mar 2026 10:29:15 -0500 Subject: [PATCH 01/49] Added Architecture & Data model. Signed-off-by: Chris Roadfeldt --- content/_index.md | 70 +- content/docs/_index.md | 14 - content/docs/architecture/_index.md | 20 + content/docs/architecture/ai-prompt.md | 1439 +++++++++++++++++ .../docs/architecture/data-model/_index.md | 28 + .../audit-provenance-observability.md | 368 +++++ .../data-model/context-and-purpose.md | 374 +++++ .../data-model/entity-relationships.md | 472 ++++++ .../architecture/data-model/four-states.md | 449 +++++ .../data-model/information-providers.md | 441 +++++ .../data-model/layering-and-versioning.md | 1265 +++++++++++++++ .../data-model/resource-grouping.md | 299 ++++ .../data-model/resource-service-entities.md | 479 ++++++ .../data-model/resource-type-hierarchy.md | 450 ++++++ .../data-model/service-dependencies.md | 378 +++++ .../data-model/storage-providers.md | 306 ++++ content/docs/architecture/overview.md | 261 +++ .../architecture/specifications/_index.md | 19 + .../specifications/cncf-strategy.md | 234 +++ .../kubernetes-compatibility.md | 354 ++++ .../specifications/operator-interface-spec.md | 837 ++++++++++ .../specifications/operator-sdk-api.md | 645 ++++++++ 22 files changed, 9126 insertions(+), 76 deletions(-) delete mode 100644 content/docs/_index.md create mode 100644 content/docs/architecture/_index.md create mode 100644 content/docs/architecture/ai-prompt.md create mode 100644 content/docs/architecture/data-model/_index.md create mode 100644 content/docs/architecture/data-model/audit-provenance-observability.md create mode 100644 content/docs/architecture/data-model/context-and-purpose.md create mode 100644 content/docs/architecture/data-model/entity-relationships.md create mode 100644 content/docs/architecture/data-model/four-states.md create mode 100644 content/docs/architecture/data-model/information-providers.md create mode 100644 content/docs/architecture/data-model/layering-and-versioning.md create mode 100644 content/docs/architecture/data-model/resource-grouping.md create mode 100644 content/docs/architecture/data-model/resource-service-entities.md create mode 100644 content/docs/architecture/data-model/resource-type-hierarchy.md create mode 100644 content/docs/architecture/data-model/service-dependencies.md create mode 100644 content/docs/architecture/data-model/storage-providers.md create mode 100644 content/docs/architecture/overview.md create mode 100644 content/docs/architecture/specifications/_index.md create mode 100644 content/docs/architecture/specifications/cncf-strategy.md create mode 100644 content/docs/architecture/specifications/kubernetes-compatibility.md create mode 100644 content/docs/architecture/specifications/operator-interface-spec.md create mode 100644 content/docs/architecture/specifications/operator-sdk-api.md diff --git a/content/_index.md b/content/_index.md index 4115fd6..4585ceb 100644 --- a/content/_index.md +++ b/content/_index.md @@ -1,68 +1,14 @@ --- -title: DCM Project -toc: false +title: Documentation +type: docs +sidebar: + open: true --- -
+# DCM Documentation -{{< hextra/hero-headline >}} - Data Center Management -{{< /hextra/hero-headline >}} +Documentation for the DCM project. -
+## Documentation Sections -{{< hextra/hero-subtitle >}} - Hyperscaler-like cloud experience for 
enterprise on-premises and sovereign cloud infrastructure -{{< /hextra/hero-subtitle >}} - -
- -{{< hextra/hero-button text="Get Started" link="docs" >}} -{{< hextra/hero-button text="View on GitHub" link="https://github.com/dcm-project" >}} - -
- -## About DCM - -The Data Center Management (DCM) project provides a hyperscaler-like cloud experience for enterprise on-premises and sovereign cloud infrastructure. DCM is an **API-first, technology-agnostic framework** that relies on a declarative data model where the desired state of infrastructure is defined as code. - -The overarching goal is to define the configuration of an entire data center in version-controlled code, enabling rapid deployment and full environment recovery. - -
- -## Core Principles - -{{< hextra/feature-grid >}} - {{< hextra/feature-card - title="๐Ÿ”Œ API-First Design" - subtitle="Technology-agnostic framework built on modern API principles for maximum flexibility." - style="background: radial-gradient(ellipse at 50% 80%,rgba(194,97,254,0.15),hsla(0,0%,100%,0));" - >}} - {{< hextra/feature-card - title="โ˜๏ธ Cloud-Native Approach" - subtitle="Moving away from monolithic management toward distributed, cloud-native principles." - style="background: radial-gradient(ellipse at 50% 80%,rgba(62,151,255,0.15),hsla(0,0%,100%,0));" - >}} - {{< hextra/feature-card - title="๐Ÿ“š Service Catalog" - subtitle="Comprehensive catalog system for managing services and infrastructure offerings." - >}} - {{< hextra/feature-card - title="โš–๏ธ Rules Engine" - subtitle="Built-in governance and policy enforcement through a flexible rules engine." - >}} -{{< /hextra/feature-grid >}} - -
- ---- - -
- -## Explore More - -{{< cards >}} - {{< card link="docs" title="Documentation" icon="book-open" subtitle="Guides, tutorials, and technical documentation" >}} - {{< card link="docs/enhancements" title="Enhancements" icon="light-bulb" subtitle="Enhancement proposals documenting new features and changes" >}} - {{< card link="blog" title="Blog & Updates" icon="newspaper" subtitle="Latest news, releases, and articles" >}} -{{< /cards >}} +- **[Enhancements](enhancements/)** - Enhancement proposals documenting new features, architectural decisions, and significant changes to the DCM project. diff --git a/content/docs/_index.md b/content/docs/_index.md deleted file mode 100644 index 4585ceb..0000000 --- a/content/docs/_index.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -title: Documentation -type: docs -sidebar: - open: true ---- - -# DCM Documentation - -Documentation for the DCM project. - -## Documentation Sections - -- **[Enhancements](enhancements/)** - Enhancement proposals documenting new features, architectural decisions, and significant changes to the DCM project. diff --git a/content/docs/architecture/_index.md b/content/docs/architecture/_index.md new file mode 100644 index 0000000..53e7e20 --- /dev/null +++ b/content/docs/architecture/_index.md @@ -0,0 +1,20 @@ +--- +title: Architecture +type: docs +weight: 1 +sidebar: + open: true +--- + +> **โš ๏ธ Active Development** โ€” Architecture documentation is actively being developed. Concepts and specifications are work in progress and subject to change. Open questions are explicitly tracked. [Contribute on GitHub](https://github.com/dcm-project). + +# DCM Architecture + +Architectural documentation for the DCM project โ€” covering the high-level design, +data model, and technical specifications. + +{{< cards >}} + {{< card link="overview" title="High Level Design" icon="template" subtitle="Overview of DCM architecture, core principles, the four states, provider model, and request lifecycle." >}} + {{< card link="data-model" title="Data Model" icon="database" subtitle="The unified data model: layers, entities, states, relationships, providers, and governance." >}} + {{< card link="specifications" title="Specifications" icon="document-text" subtitle="Operator Interface Specification, Kubernetes compatibility, SDK API, and CNCF strategy." >}} +{{< /cards >}} diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md new file mode 100644 index 0000000..707a8b1 --- /dev/null +++ b/content/docs/architecture/ai-prompt.md @@ -0,0 +1,1439 @@ +# DCM Project โ€” AI Model Prompt Script + +**Purpose:** This script provides an AI model with the full context needed to participate effectively in DCM project work. It should be provided at the start of any AI-assisted session involving DCM architecture, documentation, code, or design work. + +**Usage:** Paste this document into the AI model's context at the start of a session. Follow with the specific task or question. + +**Maintainers:** Update this document whenever significant architectural decisions are made, new concepts are established, or open questions are resolved. + +**Last Updated:** 2026-03 + +> **โš ๏ธ Active Development Notice** +> The DCM data model, architecture, and specifications are under active development. Decisions documented here represent current best understanding โ€” open questions are explicitly tracked and will be resolved over time. When working on DCM, always check the DISCUSSION-TOPICS.md for unresolved questions before making assumptions. + +--- + +## SECTION 1 โ€” PROJECT IDENTITY + +You are assisting with the **DCM (Data Center Management)** project, an open-source strategic framework developed by the Red Hat FlightPath team developed by the Red Hat FlightPath Team. + +**Key facts:** +- DCM is NOT a provisioning tool โ€” it is a **governing framework** +- DCM is a **top-down orchestration and policy enforcement layer** +- DCM enables enterprises to achieve a "Private Cloud" / "Sovereign Cloud" experience with public cloud efficiencies on-premises +- DCM is **API-first, data-driven, and policy-governed** +- DCM is inspired by Kubernetes' declarative control plane model +- GitHub: https://github.com/dcm-project +- Website: https://dcm-project.github.io + +**Authors:** Chris Roadfeldt (Principal Architect), Ryan Goodson (Senior Principal Architect), Adam Seeley (Global Director) โ€” Red Hat FlightPath. + +**Vision:** To make life better for all customers, internal teams, and future-looking entities. + +**Mission:** Seamlessly manage the complete lifecycle of all data center infrastructure by providing a policy-governed, data-driven, and unified platform to enable and ensure sovereignty. + +--- + +## SECTION 2 โ€” THE PROBLEM DCM SOLVES + +Organizations face these core challenges that DCM addresses: + +1. **Fragmented Operations** โ€” disparate tools, no unified management, functionality trapped in monoliths +2. **No Single Source of Truth** โ€” multiple CMDBs diverge, no trustworthy representation of infrastructure state +3. **High Time-to-Market** โ€” a single VM lifecycle may be managed by dozens of teams; firewall changes can take weeks +4. **The Private Cloud Gap** โ€” "Private Cloud" โ‰  "On-Premises Compute"; a true private cloud requires networking, storage, identity, catalog, FinOps, observability, auditing, and risk management +5. **Drift and State Discrepancy** โ€” no reconciliation between discovered inventory and intended inventory +6. **Sovereignty Requirements** โ€” organizations must enforce data residency, compliance, and operational control across their infrastructure + +**The definitional clarification DCM makes:** +- Private Cloud โ‰  On-Premises Compute +- Private Cloud > On-Premises IaaS +- Private Cloud = Hyperscale experience for on-premises infrastructure + +--- + +## SECTION 3 โ€” FOUNDATIONAL PRINCIPLES + +These three constraints apply to ALL data, entities, and operations in DCM **universally and without exception:** + +### 3.1 Declarative +Data describes **what something is or should be**, not how to achieve it. Every entity is a complete, self-describing statement of state. The procedures required to achieve that state are the concern of the Service Provider, not the data model. + +### 3.2 Idempotent in Operation +Applying the same data to the same system multiple times must always produce the same result. No operation on DCM data should have different outcomes based on how many times it has been applied. + +### 3.3 Immutable if Versioned +Once a version of any entity is published, it cannot be modified. Changes produce a new version. Previous versions remain intact and accessible forever. + +--- + +## SECTION 4 โ€” THE DCM DATA MODEL + +**The data model is the most foundational element of DCM.** Everything in DCM acts on data in some way โ€” reading, validating, triggering, enriching, gatekeeping, parsing, transforming, or comparing. The data model is the lingua franca โ€” the API between all components. + +### 4.1 Universal Identity Requirement +Every data object in DCM **must have a UUID**. This applies without exception to: resource definitions, catalog items, data layers, policies, policy sets, components, service providers, consumers, requests, and all other entities. UUIDs are used for provenance anchoring, dependency mapping, audit fidelity, and cross-state correlation. + +### 4.2 The Four States +DCM tracks every resource through four distinct states. Together they provide complete visibility into what was wanted, what was asked for, what was built, and what actually exists. + +| State | Description | Store | Created By | +|-------|-------------|-------|------------| +| **Intent** | What the consumer wants โ€” raw declared desire before any processing | Intent Store | Consumer (Web UI / API) | +| **Requested** | Fully processed, policy-validated, enriched payload submitted to a provider | Request Store | Request Payload Processor | +| **Realized** | What was actually provisioned, returned by the provider in DCM unified format | Realized Store | Service Provider (via Denaturalization) | +| **Discovered** | What actually exists, as independently interrogated by a provider during discovery | Discovered Store | Service Provider (via Discovery) | + +**Key operations across states:** +- **Drift Detection:** Discovered State vs. Realized State +- **Request Validation:** Requested State vs. Policy definitions +- **Intent Portability:** Intent State โ†’ re-process through current policies โ†’ new Requested State +- **Brownfield Ingestion:** Discovered State โ†’ enrichment โ†’ Realized State (lifecycle ownership) + +**State lifecycle flow:** +``` +Consumer Request โ†’ INTENT โ†’ (Policy Engine) โ†’ REQUESTED โ†’ (Service Provider) โ†’ REALIZED + โ†• compare + DISCOVERED โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บ Drift Detection +``` + +### 4.3 Field-Level Provenance and Data Lineage +This is a **structural requirement** of the data model โ€” not a logging concern. + +For any field, at any point in the pipeline, it must be possible to determine: +- What is the current value? +- Where did this value originate? (catalog item, layer, policy, consumer input, discovery) +- Has it been modified? If so โ€” what changed it, which entity (by UUID), when, what was the previous value, and why? +- What is the complete chain of custody from origin to current value? + +**Provenance is carried within the data object itself** โ€” co-located with the data, not in an external log. + +Every component that modifies data carries a **provenance obligation** โ€” it must record its UUID, operation type, timestamp, and reason for every field it modifies. This is non-optional. + +Conceptual provenance structure per field: +```yaml +field_name: + value: + metadata: + override: + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + constraint_schema: + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_type: + source_uuid: + operation_type: + actor: + timestamp: + reason: +``` +The `metadata` block is set exclusively by the Policy Engine. `operation_type: lock` is used when a GateKeeper sets `override: immutable`. `operation_type: grant` is used when a trusted_grant is issued. + +### 4.4 Universal Versioning +All entities, definitions, and data objects in DCM follow one versioning scheme: **Major.Minor.Revision** + +| Component | Trigger | +|-----------|---------| +| **Major** | Breaking changes to the contract | +| **Minor** | Additive changes, backward compatible | +| **Revision** | Data/configuration changes, no contract impact | + +This applies universally to: resource types, layers, policies, catalog items, provider registrations, registry entries, and all other definitions. + +### 4.5 Universal Artifact Status Lifecycle +Every DCM artifact โ€” layers, policies, resource types, catalog items, provider registrations โ€” follows a five-status lifecycle: + +| Status | Meaning | Applied? | Shadow? | +|--------|---------|---------|---------| +| `developing` | In development | No โ€” dev mode only | No | +| `proposed` | Submitted for review/validation | No | Yes โ€” policies shadow-execute against real traffic, output captured not applied | +| `active` | Live and governing | Yes | Yes โ€” audit records | +| `deprecated` | Being phased out, replacement available | Yes โ€” with warning | Yes | +| `retired` | End of life | No | No โ€” terminal | + +**Status transitions:** `developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired` +Deprecated artifacts must include: replacement UUID, deprecation reason, migration guidance, sunset date. + +**Previously:** The model had only `active โ†’ deprecated โ†’ retired`. The `developing` and `proposed` statuses were added to support the full artifact development workflow โ€” especially the shadow execution validation model for policies. + +### 4.6 Artifact Metadata Standard +Every DCM artifact carries a universal metadata block โ€” applies to all artifacts without exception: +- `uuid` โ€” immutable, DCM-assigned at creation +- `handle` โ€” human-readable stable ID: `{domain}/{layer_type}/{name}` e.g. `platform/core/security-cpu-limits` +- `version` โ€” Major.Minor.Revision +- `status` โ€” five-status lifecycle (Section 4.5) +- `created_by` โ€” audit record (who submitted): UUID optional + display_name required +- `owned_by` โ€” accountability record (who is responsible, receives notifications): UUID optional + display_name required +- `created_via` โ€” ingestion path: `pr | api | migration | system` +- `modifications` โ€” append-only history of all changes with who/when/why + +**Contact modes:** UUID+display_name when Identity Provider registered; display_name+email standalone/air-gapped. Both supported. + +### 4.7 The Four Provider Types +All four provider types follow the same base contract (registration, health check, trust, provenance emission): + +| Type | Purpose | DCM Owns Result? | +|------|---------|-----------------| +| **Service Provider** | Realizes resources โ€” KubeVirt, VMware, AAP, Terraform | Yes | +| **Information Provider** | Serves authoritative external data DCM references but does not own | No โ€” external system is authoritative | +| **Meta Provider** | Composes multiple providers into higher-order services | Yes | +| **Storage Provider** | Persists DCM state โ€” GitOps stores, event streams, audit store, observability | Yes โ€” DCM is authoritative | + +--- + +## SECTION 5 โ€” THE FOUR STATES AND STORAGE MODEL + +### 5.1 The Four States + +Every DCM entity exists across four independently maintained state records: + +| State | Question | Store Type | Characteristics | +|-------|----------|------------|----------------| +| **Intent State** | What did the consumer ask for? | GitOps | Immutable, branched, PR reviewed, CI/CD triggered | +| **Requested State** | What was approved and dispatched? | GitOps | Immutable, committed, CD triggered, full provenance | +| **Realized State** | What did the provider actually build? | Event Stream | Append-only, entity-keyed stream, high-frequency | +| **Discovered State** | What actually exists right now? | Event Stream (ephemeral) | Machine-generated, drift detection source | + +### 5.2 Storage Architecture โ€” Contract, Not Implementation + +DCM defines store **contracts** โ€” what capabilities, guarantees, and obligations each store must satisfy. Implementation technology is a deployment choice. Storage Provider is the fourth formal DCM provider type. + +``` +GitOps Stores: Intent, Requested, Layers, Policies +Event Stream Stores: Realized, Discovered, Audit events +Search Index: Queryable projection of GitOps stores (non-authoritative) +Audit Store: Compliance-grade, immutable, long-retention +Observability Store: Time-series metrics, traces, logs +``` + +### 5.3 The Entity UUID โ€” Universal Key +The entity UUID is assigned at Intent State creation and links the entity across all four states and all stores. Given an entity UUID, DCM can reconstruct the complete lifecycle history from any store. + +### 5.4 CI/CD Integration +GitOps stores are the natural CI/CD integration point: +- **CI pipeline** fires on branch/update: policy pre-validation (dry run), cost estimation, sovereignty check, auto-approve evaluation โ€” results posted as PR comments +- **CD pipeline** fires on merge: layer assembly, full policy evaluation (binding), provider dispatch +- **Third rail โ€” direct API ingress**: bypasses PR workflow, not governance. Same pipeline, no human review unless required by policy. + +### 5.5 Rehydration โ€” Three Sources, Six Modes + +Rehydration uses a prior state record as the starting point for a new request. Governance always applies โ€” rehydration is a new request, not a shortcut. + +**Three sources:** Intent Store (full assembly runs), Requested Store (assembly skipped, governance runs), Realized Store (provider fields stripped, assembly skipped, governance runs) + +**Two axes โ€” six modes:** + +| Mode | re_evaluate | policy_version | Use Case | +|------|-------------|----------------|----------| +| Faithful | false | current | Same provider, current governance | +| Provider-Portable | true | current | New provider, current governance | +| Historical Exact | false | pinned | Same provider, historical governance (audit evidence) | +| Historical Portable | true | pinned | New provider, historical governance | + +**Placement flag on Requested/Realized rehydration:** +```yaml +placement: + re_evaluate: false # Honor original provider selection (default) + # re_evaluate: true # Strip provider, run placement policies fresh +governance: + apply_all_policies: true # Always true โ€” never skippable + policy_version: current # current | pinned (pinned requires elevated auth) +``` + +**Named concept:** Provider-Portable Rehydration โ€” rehydration with provider selection re-evaluated through current placement policies and current provider landscape. + +**Partial Q54 resolution:** Policies set placement constraints. The placement component selects the specific provider within those constraints. Policies never name a specific provider โ€” that would be portability-breaking. + +### 5.6 Audit, Provenance, and Observability โ€” Three Distinct Concerns + +| Concern | What It Is | Audience | Retention | +|---------|-----------|----------|-----------| +| **Provenance** | Field-level data lineage embedded in every payload | System | Permanent โ€” part of the data | +| **Audit** | Compliance-grade queryable record of all actions | Auditors, Compliance | Regulatory period (7+ years FSI) | +| **Observability** | Real-time metrics, traces, logs | SRE, Platform Engineers | Operational window (90 days) | + +Audit is a **separate component** fed by provenance events emitted by all Storage Providers (contractual obligation). Surfaced through the DCM API Gateway โ€” not a separate endpoint. + +All DCM capabilities โ€” catalog, requests, entities, policies, audit, observability โ€” are surfaced through a **unified API Gateway hierarchy**. + +--- + +## SECTION 6 โ€” DATA LAYERS AND THE ASSEMBLY PROCESS + +### 6.1 What is a Layer? +A Layer is a **declarative, immutable, versioned unit of data** that contributes fields to a merged payload. Layers do not execute โ€” they declare. Every layer has a UUID, follows universal versioning, is immutable once published, carries a parent entity reference, and contributes field-level provenance metadata for every field it sets. + +### 6.2 Layer Types + +| Layer Type | Scope | Ownership | Purpose | +|------------|-------|-----------|---------| +| **Base Layer** | Type-agnostic or type-scoped | DCM platform / implementor | Foundation entity โ€” minimum required fields and defaults. Every chain starts here. | +| **Core Layers** | Type-agnostic by default | Infrastructure teams / implementors | Organizational, infrastructure, and contextual data applicable across all resource types (DC, Zone, Rack, Region, Environment) | +| **Intermediate / Customization Layers** | Type-agnostic or type-scoped | Organizational teams / domain owners | Organizational hierarchy and deployment context overrides stacked between Core and Service Layers | +| **Service Layers** | **Must be type-scoped** | Service Providers / service domain teams | Service-specific configuration for a specific Resource Type. Invalid without declared type scope. | +| **Request Layer** | Scoped to requested Resource Type | Consumer | Consumer's declared intent. Becomes Intent State on submission. Has higher precedence than all data layers. | +| **Policy Layers** | Scoped by policy domain | Policy creators / security / compliance | Governance layer โ€” operates on the assembled payload after data layers are merged | + +### 6.3 Service Layer Type Scope +Service Layers must declare their Resource Type scope and scope inheritance behavior: +```yaml +type_scope: + resource_type_uuid: + resource_type_fully_qualified_name: + scope_inheritance: + # exact: applies only to the declared Resource Type + # descendants: applies to the declared type and all child types via inheritance +``` + +### 6.4 Precedence Order +From lowest to highest precedence: +``` +1. Base Layer (foundation defaults) +2. Core Layers (organizational and infrastructure context) +3. Intermediate/Customization (organizational hierarchy overrides) +4. Service Layers (service-specific configuration) +5. Request Layer (consumer intent โ€” overrides all data layers) +6. Transformation Policies (enrich / modify โ€” additive) +7. Validation Policies (pass/fail โ€” no field modification) +8. GateKeeper Policies (highest authority โ€” overrides everything including consumer input) +``` + +### 6.5 Policy Layer Behavior +- **Validation** โ€” checks data against rules, does not modify. Pass/fail. Failure rejects the request. +- **Transformation** โ€” enriches or modifies the payload. Adds missing fields, applies standards. Recorded in provenance. +- **GateKeeper** โ€” highest authority. Can override any field including consumer-declared values. Used for sovereignty constraints, security mandates, and hard compliance rules. All overrides recorded in provenance. + +### 6.6 Assembly Process (Seven Steps) +1. **Intent Capture** โ€” Request Layer stored as Intent State in Intent Store. No modification. +2. **Layer Resolution** โ€” Processor identifies applicable layers by Resource Type and organizational context. +3. **Layer Merge** โ€” Layers merged in precedence order. Each field records source layer UUID in provenance. +4. **Request Layer Application** โ€” Consumer values applied last in data layer merge. Overrides recorded in provenance. +5. **Transformation Policies** โ€” Enrich and modify payload. Provenance recorded per field. +6. **Validation Policies** โ€” Check payload. Failures reject request with reason. +7. **GateKeeper Policies** โ€” Hard overrides applied. Provenance recorded. Payload stored as Requested State. + +### 6.7 Key Rules +- Core Layers are type-agnostic โ€” applied to every request regardless of Resource Type +- Service Layers must be type-scoped โ€” only applied when request Resource Type matches declared scope +- A Service Layer without a declared type scope is invalid and must be rejected +- Conflicting fields at same precedence: resolved by priority if declared; CONFLICT ERROR if not declared or equal +- Conflicts detected at **ingestion time** โ€” not assembly time โ€” all active layers are pre-validated conflict-free +- All layer modifications are recorded in field-level provenance + +### 6.8 Layer Identity โ€” Domain, Handle, Priority + +**Layer Domain** โ€” organizational home and authorization: + +| Domain | Scope | Can Override | +|--------|-------|-------------| +| `system` | DCM built-in | Nothing above | +| `platform` | All tenants | tenant, service, provider | +| `tenant` | Single tenant | service, provider within tenant | +| `service` | Service Provider | provider | +| `provider` | Catalog Item | Nothing above | + +**Layer Handle** โ€” `{domain}/{layer_type}/{name}` โ€” human-readable stable ID, unique in DCM, mirrors Git path. +Example: `platform/core/security-cpu-limits` โ†’ `dcm-layers/platform/core/security-cpu-limits/v1.2.0.yaml` + +**Priority Schema** โ€” deterministic conflict resolution: +- Format: `{int}.{int}.{int}...` โ€” unlimited depth, higher value = higher priority +- `900.10` beats `800.10`; `900.20` beats `900.10`; no ceiling โ€” infinite upward insertability +- Priority label: semantic context only, does not affect ordering +- Reference taxonomy (advisory, not enforced): 900=Compliance, 800=Security, 700=Sovereignty, 600=Operations, 500=Platform, 400=Service, 300=Organization, 200=Site, 100=Custom +- Organizations needing authority above all standard categories use `1000.*` or higher โ€” no renumbering required + +**Immutable ceiling model (Q51 resolved):** +- `override: immutable` (default) โ€” protected by execution order; first policy to lock wins during this execution +- `override: immutable` + `immutable_ceiling: absolute` โ€” protected against all future policies; attempted overrides are rejected and logged in audit + +### 6.9 Artifact Metadata Standard + +**Every DCM artifact** carries a universal artifact metadata block. Universal โ€” not optional, not per-artifact. + +**Five artifact statuses:** + +| Status | Applied? | Shadow? | Key Behavior | +|--------|---------|---------|-------------| +| `developing` | No | No | Dev mode only | +| `proposed` | No | Yes (policies) | Shadow output captured for validation | +| `active` | Yes | Yes (audit) | Live and governing | +| `deprecated` | Yes (warning) | Yes | Replacement available | +| `retired` | No | No | Terminal โ€” cannot be used | + +**Key fields:** `created_by` (audit โ€” who submitted), `owned_by` (accountability โ€” who gets notified), `created_via` (pr/api/migration/system โ€” audit quality), `modifications` (append-only history) + +**Contact โ€” two modes:** UUID+display_name when IdP registered; display_name+email in standalone/air-gapped mode. Both fully supported. + +**Proposed shadow (policies):** Shadow output captured per request as `proposed_evaluation_record` โ€” what the policy would have done, never applied. Feeds Validation Dashboard. + +**Notifications to `owned_by.notification_endpoint`:** layer conflict, deprecation, provider deregistered, policy violation, drift, high-impact shadow, approaching sunset. + +### 6.10 Field Override Control โ€” Two Categories + +**Structural Rules (Request Payload Processor โ€” non-overridable DCM System behavior):** +- Layer immutability โ€” a published version cannot be modified +- A child layer cannot remove a parent field โ€” only override its value +- Layer precedence order is fixed โ€” Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request โ†’ Policy +- Circular references and typeless Service Layers are always rejected + +**Business Rules (Policy Engine โ€” configurable):** +The Policy Engine is the **sole authority** for field override control. It sets override control metadata on fields using the standard policy mechanism. Data layers and the Request Payload Processor never set override control. + +### 6.11 Field Override Control โ€” Three Levels + +**Design Principle: Simple by default, powerful when needed. Use only the level you need.** + +**Level 1 โ€” No declaration (default)** +Field is fully overridable by any actor. Zero configuration. Covers the majority of fields. + +**Level 2 โ€” Simple declaration** +Single `override` property โ€” sufficient for most governed fields: +- `override: allow` โ€” explicit allow (same as default, self-documenting) +- `override: constrained` โ€” any actor can override within `constraint_schema` +- `override: immutable` โ€” no actor can override at any level + +**Level 3 โ€” Matrix declaration** +Full per-actor permission matrix for fields requiring nuanced governance: + +```yaml +override_matrix: + default: allow + inheritance: restrict_only + actors: + - actor: policy.global # can_expand: true + - actor: policy.tenant # can_expand: true (within global ceiling) + - actor: policy.user # can_expand: false + - actor: consumer_request # can_expand: false + - actor: process_resource # permission: deny by default + - actor: provider # can_expand: false + - actor: sre_override # can_expand: false + - actor: admin_override # can_expand: true (within scope) + trusted_grants: + - granted_to_uuid: + actor_type: process_resource + permission: allow + granted_by_policy_uuid: + expires: +``` + +**Where declared:** Resource Type Specification sets the ceiling. Catalog Item can only restrict further. Policy Engine applies at runtime within those bounds. + +**Expansion rules:** `policy.global`, `policy.tenant`, `admin_override` can grant expansion. `policy.user`, `consumer_request`, `provider` can never expand. `process_resource` and `sre_override` denied by default โ€” require trusted grant. + +**Actor extensibility:** Custom actors default to `deny`, require explicit grants, follow universal versioning and deprecation model. + +--- + +## SECTION 7 โ€” RESOURCE TYPE HIERARCHY AND SERVICE CATALOG + +The Resource Type Hierarchy is how DCM achieves **resource portability** โ€” expressing what a consumer needs independently of which specific provider delivers it. + +### 7.1 The DCM Resource Type Registry +- DCM maintains an official registry of standard Resource Types +- Registry is **open** โ€” community and implementors can propose new types +- All registry entries are **vendor-neutral by hard requirement** +- Exception: `exclusive` classification where one provider is the sole implementor +- Registry entries are versioned, immutable once published, and can be deprecated + +### 7.2 Four Hierarchy Levels + +**Level 1 โ€” Resource Type Category** (broadest) +Organizational container. Examples: `Compute`, `Network`, `Storage`, `Platform`, `Security`, `Observability`, `Data` + +**Level 2 โ€” Resource Type** (abstract) +Defines a class of resource. Must be vendor-neutral. Examples: `Compute.VirtualMachine`, `Network.FirewallRule` + +**Level 3 โ€” Resource Type Specification** (standard contract) +The data contract for a Resource Type โ€” all fields with types, constraints, and portability classifications. + +**Level 4 โ€” Provider Catalog Item** (concrete) +A specific provider's implementation of a Resource Type Specification. + +### 7.3 Portability Classification +Every field in every Resource Type Specification carries a portability classification: + +| Classification | Meaning | Portability | +|---|---|---| +| `universal` | All providers must support it | Fully portable | +| `conditional` | Some providers support it | Portable to supporting providers | +| `provider-specific` | One provider only | Portability-breaking โ€” must be marked | +| `exclusive` | One provider for entire tech stack | Not applicable โ€” declared | + +**Hard requirements:** +- All `universal` fields MUST be supported by ALL implementing providers +- `provider-specific` fields MUST be marked portability-breaking +- Consumers MUST be warned when their request contains portability-breaking fields +- Portability warning enforcement is **organizational policy**: `block` | `warn` | `allow` + +### 7.4 Inheritance +Resource Types support inheritance. Rules: +- Child inherits ALL parent fields โ€” none can be removed or redefined +- Child may add new fields +- Child portability can only be equal to or more restrictive than parent +- Each level is independently versioned with parent UUID reference + +### 7.5 Request Resolution โ€” Specificity Narrowing +Provider selection is **never explicit**. It emerges from progressive specificity: +``` +Resource Type declared โ†’ matches all providers for that type +Universal fields specified โ†’ still matches all providers +Conditional fields specified โ†’ narrows to supporting providers +Provider-specific fields used โ†’ narrows to single provider (portability warning issued) +Placement/sovereignty applied โ†’ final provider selected by Policy Engine +``` + +--- + +## SECTION 8 โ€” RESOURCE/SERVICE ENTITIES + +### 8.1 Core Terminology + +| Term | Definition | +|------|-----------| +| **Resource/Service Request** | What a consumer submits to DCM โ€” the declared intent to consume a resource or service. The consumer side of the transaction. | +| **Resource/Service Entity** | The "thing" produced by a provider as a result of fulfilling a request โ€” the allocation made real. The provider side of the transaction. | + +### 8.2 DCM as Authoritative Owner โ€” Always +DCM is ALWAYS the system of record for Resource/Service Entity data. DCM is ALWAYS authoritative for the resource definition. DCM ALWAYS owns the lifecycle. This applies regardless of the operational ownership model. + +Providers are **custodians** of the underlying infrastructure โ€” they are not the system of record. + +### 8.3 Four Ownership Models + +| Model | Description | Example | +|-------|-------------|---------| +| **Allocation** | Provider retains internal ownership. Consumer owns the Entity (the allocation). Provider has reclaim rights on decommission. | VM, Container, IP Address | +| **Whole Allocation** | Entire resource allocated as indivisible unit. Provider retains ownership. Consumer has exclusive use. Not subdivided or shared. | Dedicated Bare Metal (provider-owned) | +| **Full Transfer** | Provider transfers complete ownership to consumer's DCM Tenant. Consumer controls full lifecycle including decommission. | Transferred Bare Metal, Licensed asset | +| **Hybrid Transfer** | Ownership can transfer multiple times. Current owner is always exactly one DCM Tenant. Every transfer is tracked and auditable. | Bare Metal reallocated between tenants | + +Every Provider Catalog Item must declare which ownership model(s) it supports. + +### 8.4 Resource/Service Entity Lifecycle +``` +REQUESTED โ†’ PENDING โ†’ PROVISIONING โ†’ REALIZED โ†’ OPERATIONAL + โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ โ–ผ + DEGRADED MAINTENANCE SUSPENDED + โ”‚ + DECOMMISSIONING โ†’ DECOMMISSIONED +``` +`DECOMMISSIONED` is the only terminal state. Records are retained permanently and are immutable. + +### 8.5 Process Resource Entities +A distinct entity class for ephemeral execution resources โ€” automation jobs, playbooks, pipelines, workflows. +- Ephemeral lifecycle โ€” exists for duration of execution +- Execution record retained permanently after terminal state +- Must belong to a DCM Tenant +- Must be in provenance chain of any Resource/Service Entity they affect +- Lifecycle: `REQUESTED โ†’ INITIATED โ†’ EXECUTING โ†’ COMPLETED | FAILED | CANCELLED` + +### 8.6 Provider Capacity Model โ€” Three Modes +- **Mode 1 โ€” Dynamic Query**: DCM queries provider on-demand during request processing +- **Mode 2 โ€” Provider Registration (preferred)**: Provider registers capacity on configurable schedule โ€” default minimum twice daily +- **Mode 3 โ€” Provider Denial (mandatory)**: Provider validates fulfillment capability before executing. Denies with `INSUFFICIENT_RESOURCES`. Triggers immediate DCM capacity rating update. + +All three modes are always available. Mode 3 is mandatory for all providers. + +### 8.7 Provider Lifecycle Events +Any provider event affecting Resource/Service Entity availability or operational characteristics MUST be reported to DCM immediately. This is a non-negotiable contractual obligation. + +When DCM receives a provider event, the Policy Engine evaluates and determines response: +`ALERT | REVERT | UPDATE_DEFINITION | INVESTIGATE | DECOMMISSION | ESCALATE` + +DCM acts as the **Tenant advocate** โ€” protecting Tenant interests in all provider interactions. + +--- + +## SECTION 9 โ€” SERVICE DEPENDENCIES + +### 9.1 Why Dependencies Must Be Declared in Advance +Dependencies must be declared in the data model โ€” not discovered at runtime. This is required for: +- **Auditability** โ€” complete dependency graph known before execution +- **Cost Analysis** โ€” full resource footprint known before provisioning +- **Placement** โ€” Policy Engine needs complete resource footprint for optimal placement +- **Idempotency** โ€” same request always produces same dependency graph + +### 9.2 Hybrid Dependency Declaration Model + +**Type-Level Dependencies (Resource Type Specification)** +- Portable and provider-agnostic +- Apply to all Provider Catalog Items implementing that Resource Type +- Use Resource Type UUIDs โ€” not provider-specific references +- Required for all implementations + +**Provider-Specific Dependencies (Provider Catalog Item)** +- Provider-specific additions beyond type-level dependencies +- Must be marked `portability_breaking: true` +- Visible to Policy Engine and consumers as portability warnings + +### 9.3 Dependency Types and Cardinality + +| Type | Behavior | +|------|---------| +| `hard` | Must be realized before/alongside dependent. Failure fails dependent. | +| `soft` | Preferred but not blocking. Failure recorded but does not block. | +| `conditional` | Required only if specific request payload conditions are met. | + +| Cardinality | Description | +|-------------|-------------| +| `one_to_one` | Exactly one dependency resource required | +| `one_to_many` | One or more dependency resources required | +| `one_to_optional` | Zero or one dependency resource | +| `one_to_range` | Specific numeric range required | + +### 9.4 Dependency Graph +When a request is processed, a complete **Dependency Graph** is constructed โ€” all resources that must be created including all transitive dependencies. Each resource appears exactly once. Circular references are invalid. + +The graph is attached to the Requested State. Nodes updated to REALIZED as providers fulfill each dependency. + +### 9.5 Dependency Payload Passing +When a dependency is realized, its Realized State payload is passed to the dependent provider. The dependent resource's Requested State payload is enriched with the dependency entity's UUID and relevant realized data. Recorded in field-level provenance with `source_type: dependency_payload`. + +### 9.6 Failure Handling โ€” Configurable per Request or Policy + +| Mode | Behavior | +|------|---------| +| `fail_all` | Any hard dependency failure fails entire request. Partially realized nodes decommissioned. | +| `fail_dependent` | Failure fails only the dependent and its dependents. Independent branches continue. | +| `retry` | Failed dependencies retried with same or alternative provider. | +| `partial_complete` | Request marked partially complete. Failed nodes flagged for retry or manual intervention. | + +### 9.7 Rehydration and Dependencies +Rehydration uses **Intent State** โ€” not Realized State โ€” to reconstruct the dependency graph. This ensures current policies and standards are applied. Resources can be rehydrated to a different provider as long as type-level dependencies are satisfiable. + +--- + +## SECTION 10 โ€” RESOURCE GROUPING AND TENANCY + +### 10.1 DCM Tenant โ€” Mandatory First-Class Ownership + +Every Resource/Service Entity must belong to exactly one DCM Tenant. This is a **non-overridable DCM System Policy**. + +Tenant provides: ownership, isolation, cost attribution, policy scope, drift detection scope, rehydration scope, audit scope, sovereignty boundary. + +**Ownership vs Consumption:** A resource belongs to one Tenant (owner) but can be consumed by multiple Tenants via the Service Catalog. Consumption is tracked through service requests โ€” not Tenant membership. + +### 10.2 Resource Groups โ€” Flexible Composable Grouping + +Resource Groups function like **structured tags** โ€” a resource accumulates group memberships describing its context from multiple dimensions simultaneously. + +**Two classes โ€” equal capability:** +- **DCM Default Resource Group** โ€” built-in, standard grouping mechanism +- **Custom Resource Group** โ€” implementor-defined, tied to business structures (CostCenter, BusinessUnit, RegulatoryScope, etc.) + +Both implement the same **Resource Group Interface**. + +### 10.3 Multi-Group Membership +A resource can belong to multiple groups across all classes. Membership constraints are configurable per group definition: +- `exclusive: true` โ€” resource can only belong to one group of this type at a time +- `exclusive: false` โ€” resource can belong to multiple groups of this type +- Organizational policies can further restrict multi-group membership + +### 10.4 Nesting +Groups that declare `nesting: true` can contain other groups as members. Nesting is configurable per group definition. Circular nesting is invalid. Child groups inherit policy scope from parent groups. + +### 10.5 DCM System Policies for Grouping + +| Policy | Rule | +|--------|------| +| `TEN-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | +| `TEN-002` | A Tenant must exist before resources can be created in it | +| `GRP-003` | Circular nesting in Resource Groups is invalid | +| `GRP-004` | Custom Resource Groups must implement the full Resource Group Interface | +| `GRP-005` | Exclusive membership groups must reject violating membership requests | + +--- + +## SECTION 11 โ€” ENTITY RELATIONSHIPS + +### 11.1 Design Principle +**Single model. Minimum variance. Simple by default.** Every relationship in DCM โ€” VM requires storage, application contains web server, resource references Business Unit โ€” uses the same structure. No separate binding mechanism, no separate dependency graph, no separate business data association. One universal model. + +### 11.2 Universal Relationship Structure +Every relationship is a first-class data object with its own UUID. Recorded **bidirectionally** โ€” on both participating entities. The same `relationship_uuid` appears on both sides. + +```yaml +relationship: + relationship_uuid: + this_entity_uuid: + this_role: + related_entity_uuid: + related_entity_type: + related_entity_role: + information_provider_uuid: + information_type: + relationship_type: + nature: + lifecycle_policy: + on_related_destroy: + on_related_suspend: + on_related_modify: + status: + provenance: +``` + +### 11.3 Relationship Types (fixed vocabulary) + +| Type | Inverse | Meaning | +|------|---------|---------| +| `requires` | `required_by` | Cannot function without the related entity | +| `depends_on` | `dependency_of` | Uses the related entity but can degrade without it | +| `contains` | `contained_by` | Logical container for the related entity | +| `references` | `referenced_by` | References without owning or requiring | +| `peer` | `peer` | Equal relationship | +| `manages` | `managed_by` | Has lifecycle management authority | + +### 11.4 Relationship Nature + +| Nature | Lifecycle Policy | Example | +|--------|-----------------|---------| +| `constituent` | Required | VM requires its boot disk | +| `operational` | Required | Web server depends on load balancer | +| `informational` | Not applicable | Resource references its Business Unit | + +### 11.5 Lifecycle Policy Authority +``` +Resource Type Specification default (lowest) + โ†’ Provider Catalog Item default + โ†’ Consumer declaration + โ†’ DCM System Policy (non-overridable) +``` + +### 11.6 Relationship Roles +Standard roles: `compute`, `storage`, `networking`, `security`, `database`, `web`, `app`, `cache`, `queue`, `pipeline`, `identity`, `monitoring`, `business_unit`, `cost_center`, `product_owner`, `regulatory_scope` + +Custom roles: extensible โ€” organizations register domain-specific roles (e.g., `trading_engine`, `risk_calculator`). Semantic labels only โ€” do not affect system behavior. + +### 11.7 Where Relationships Are Declared +- **Resource Type Specification** โ€” declares possible relationships (ceiling) +- **Catalog Item** โ€” declares actual relationships for an offering (can only restrict) +- **Request time** โ€” consumer declares instance relationships +- **External data** โ€” same model for Business Unit, Cost Center, Person, etc. + +### 11.8 Bundled Declaration Expansion +When consumer bundles storage/networking in a compute request: +1. **Processor** creates Resource/Service Entity stubs (PENDING) with UUIDs +2. **Processor** creates bidirectional Relationship records +3. **Processor** applies lifecycle policy hierarchy +4. **Policy Engine** validates binding type and lifecycle policy +5. **Service Provider** provisions natively, returns realized payloads in DCM format +6. **DCM** updates entities PENDING โ†’ REALIZED, activates relationships + +### 11.9 The Entity Relationship Graph +All relationships form a traversable graph. Used for: rehydration (full graph traversal), cost rollup (accumulates across constituent relationships), drift detection (discovered vs realized graph comparison), decommission ordering (lifecycle policies at each edge), placement (pre-realization footprint), impact analysis (change propagation). + +### 11.10 Supersedes Dependency Graph +The Entity Relationship model unifies the previously separate dependency graph concept. The dependency graph IS the relationship graph at pre-realization time โ€” same structure, different lifecycle state. + +--- + +## SECTION 12 โ€” INFORMATION PROVIDERS + +### 12.1 Purpose +Information Providers are a first-class DCM provider type that serves authoritative external data DCM needs to reference but does not own (HR systems, finance systems, CMDBs). DCM references but never caches or owns external data. + +### 12.2 Four Provider Types + +| Type | Purpose | DCM Owns Result? | +|------|---------|-----------------| +| **Service Provider** | Realizes resources โ€” KubeVirt, VMware, AAP, etc. | Yes | +| **Information Provider** | Serves authoritative external data DCM references but does not own | No โ€” external system is authoritative | +| **Meta Provider** | Composes multiple providers into higher-order services | Yes | +| **Storage Provider** | Persists DCM state โ€” GitOps stores, event streams, audit, observability | Yes โ€” DCM is authoritative | + +All four provider types follow the same base contract: registration, health check, trust, and provenance emission obligation. + +### 12.3 Same Contract as Service Providers +Information Providers follow the same registration, health check, trust, and capacity model as Service Providers where applicable. Capacity = query capacity (requests/sec). Naturalization/Denaturalization = translating native format to DCM unified format. + +### 12.4 Standard vs Extended Data +- **Standard data** โ€” DCM-defined fields. Used for lookups, policy evaluation, operational decisions. Portable across all implementations. +- **Extended data** โ€” organization-defined fields. Carried in payload for downstream consumers. DCM core does not rely on extended data for operational decisions. + +### 12.5 Stable External Key Model +```yaml +external_entity_reference: + uuid: + external_uuid: + information_provider_uuid: + information_type_name: Business.BusinessUnit + lookup_method: + primary_key: external_uuid + fallback_keys: + - field: code + value: "BU-PAY" + display_name: + verification: + last_verified: + verification_status: +``` +DCM UUID wraps external UUID โ€” if external system changes its UUID, only this record changes. All relationships remain valid. + +### 12.6 Three-Mode Verification +- **Mode 1** โ€” DCM-initiated scheduled verification (configurable frequency) +- **Mode 2** โ€” Information Provider push (contractual obligation โ€” same as SP lifecycle events) +- **Mode 3** โ€” On-demand verification fallback when reference is stale + +### 12.7 Information Types in the Registry +Same DCM registry as Resource Types โ€” distinguished by category prefix: +- `Business.*` โ€” BusinessUnit, CostCenter, ProductOwner +- `Identity.*` โ€” Person, ServiceAccount, Group +- `Compliance.*` โ€” RegulatoryScope, AuditFramework +- `Operations.*` โ€” Runbook, SLA, SupportContract + +--- + +## SECTION 13 โ€” KUBERNETES SUPERSET STRATEGY + +### 13.1 Position +DCM is a **superset of Kubernetes** โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries. Kubernetes manages the execution plane. DCM manages the management plane. + +### 13.2 What DCM Adds Beyond Kubernetes + +| Capability | Kubernetes | DCM | +|------------|-----------|-----| +| Scope | Single cluster | Multi-cluster, multi-infrastructure | +| Tenancy | Namespace isolation | First-class Tenant ownership model | +| Policy | RBAC + admission webhooks | Full Policy Engine โ€” Validation/Transformation/GateKeeper | +| Data lineage | Not provided | Field-level provenance on all data | +| Cost attribution | Not provided | Full lifecycle cost analysis | +| Drift detection | Controller reconciles | Four-state model โ€” Intent/Requested/Realized/Discovered | +| Service catalog | Not provided | Self-service catalog with RBAC-governed presentation | +| Sovereignty | Not provided | Placement constraints, compliance evidence | +| Non-Kubernetes resources | Not provided | VMware, bare metal, OpenStack managed through same model | + +### 13.3 Operator Integration โ€” The Adapter Pattern +Kubernetes operators become DCM Service Providers through the DCM Operator Interface Specification. The pattern: +``` +DCM Control Plane โ†’ Operator Adapter (Service Provider) โ†’ Kubernetes Operator โ†’ Cluster +``` +The adapter handles Naturalization (DCM Requested State โ†’ Kubernetes CR) and Denaturalization (CR status โ†’ DCM Realized State). + +### 13.4 Native Support Strategy +Goal: influence the Kubernetes operator ecosystem to adopt DCM as a superset. Three-phase approach: +1. **Generic Operator Adapter** โ€” declarative field mappings, no operator changes needed +2. **DCM Operator SDK** โ€” Go library, adds DCM support with minimal code changes, one day to Level 1 +3. **Upstream contributions** โ€” contribute DCM support directly to priority operators (KubeVirt, CloudNativePG, Strimzi, Cert-Manager) + +### 13.5 Conformance Levels +- **Level 1** โ€” Registration + health + basic status reporting. One day with SDK. Unlocks: catalog, health monitoring, basic cost tracking. +- **Level 2** โ€” + Capacity + lifecycle events + full realized payloads + field mappings. 2-3 days. Unlocks: placement, drift detection, cross-cluster management. +- **Level 3** โ€” + Sovereignty + provenance + discovery + decommission confirmation. 3-5 days. Unlocks: full audit chain, brownfield ingestion, sovereignty enforcement. + +### 13.6 CNCF Strategy +Target CNCF Sandbox submission. FSI consortium (leading FSI consortium members) provides multi-organization adopter evidence. Key artifacts needed before submission: DCM Operator Interface Spec v1.0, SDK v0.1.0, KubeVirt Level 2 reference implementation, conformance test suite, governance model. + +### 13.7 Key Kubernetes-to-DCM Concept Mappings +| Kubernetes | DCM | +|-----------|-----| +| CRD | Resource Type Specification | +| Custom Resource | Requested State โ†’ Realized State | +| Reconciliation loop | Realization + Drift Detection | +| Namespace | DCM Tenant boundary | +| ownerReference | Entity Relationship (contains/contained_by) | +| Finalizers | Lifecycle policy enforcement | +| Labels/Annotations | DCM entity metadata | +| Kubernetes conditions | DCM lifecycle states | + +--- + +## SECTION 14 โ€” WEBHOOK INTEGRATION + +### 14.1 Purpose +Webhooks provide a **push-based notification model** for consumers, providers, and external systems that cannot or do not poll DCM. They complement the API-first model and Message Bus by enabling real-time outbound event notifications. + +### 14.2 Architectural Position +Webhooks are an **Egress capability** โ€” outbound notifications from DCM to external systems. They fit within the existing Egress zone alongside the Messaging Protocol and Interoperability API. + +### 14.3 Core Use Cases + +| Audience | Example Events | +|----------|---------------| +| **Consumer/CI-CD** | Resource request transitions to REALIZED; Entity enters DEGRADED state; dependency graph node fails | +| **Provider** | New request payload dispatched; discovery request initiated; decommission requested | +| **External Systems** | ITSM notification on request create/update/complete; FinOps platform on Entity realization/decommission | +| **Operational** | Provider capacity below threshold; unsanctioned change detected; GateKeeper policy fired | +| **Compliance** | Sovereignty constraint applied; ownership transfer initiated/completed; audit-relevant policy triggered | + +### 14.4 Key Design Principles (Established) +- Webhook events are **typed and versioned** โ€” consistent with DCM universal versioning model +- Webhook payloads carry **provenance information** โ€” sufficient context to trace back to the originating request, entity, and policy +- **Policy Engine integration** โ€” the Policy Engine can fire webhooks as a response action (alongside ALERT, REVERT, UPDATE_DEFINITION, etc.) +- Webhook registrations are **scoped** โ€” consumer-facing webhooks registered via Consumer API; provider-facing webhooks registered via Provider Registration +- Events should align with a **DCM Event Type Registry** โ€” extensible, versioned, following the same registry model as Resource Types + +### 14.5 Open Design Questions +See [DISCUSSION-TOPICS.md โ€” TOPIC-001](DISCUSSION-TOPICS.md) for the full list of design questions. Key unresolved items: +- Webhook registration model and API +- Full event taxonomy and registry structure +- Payload format โ€” full state vs reference + event type +- Authentication model for outbound webhook calls +- Retry and reliability obligations +- Ordering guarantees +- Relationship to the Message Bus +- Whether provider webhook support is mandatory in the Provider Contract +- Tenant vs platform-level scoping + +### 14.6 Status +**Under active discussion** โ€” see DISCUSSION-TOPICS.md TOPIC-001. Do not make implementation assumptions until design questions are resolved. + +--- + +## SECTION 15 โ€” DCM ARCHITECTURE COMPONENTS + +### 15.1 The Five Domains +DCM is organized into five horizontal domains from bottom to top: + +| Domain | Persona | Content | +|--------|---------|---------| +| **Data Center Domain** | CIO | Physical infrastructure โ€” Compute, GPU, RAM, Storage, Networking, HSM | +| **Resource Domain** | SRE | Declarative resources โ€” VMs, containers, pods, clusters, external IPs | +| **Control Plane Domain** | CTO | Policy, Validate & Placement, Audit, Orchestration | +| **Application Domain** | Engineering | Data Center Pipeline, CI/CD | +| **Value Domain** | Line of Business | Software Build & Deploy, business outcomes | + +### 15.2 Control Plane Components + +| Component | Purpose | +|-----------|---------| +| **API Gateway** | Central clearing house โ€” ingress for consumers, egress to providers | +| **Job Queue** | Manages asynchronous task execution | +| **Request Payload Processor** | Assembles, enriches, and merges data layers into complete request payload | +| **Policy Engine** | Validates, transforms, gates, and enriches data based on policy definitions | +| **IDM / IAM** | Authentication and identity โ€” source of truth for personas and RBAC | +| **Service Catalog** | Presents available services/resources per RBAC policy | +| **Orchestration** | Coordinates multi-step workflows and manages request lifecycle | +| **Cost Analysis** | Tracks service costs throughout the full resource lifecycle | +| **Audit** | Records all operations โ€” reads provenance intrinsic to data objects | +| **Observability** | Monitoring, logging, and metrics | +| **Resource Discovery** | Interrogates providers to discover existing resource state | +| **Message Bus** | Async communication between control plane and external systems | + +### 15.3 Data Stores โ€” Storage Provider Model + +All DCM stores are **Storage Providers** โ€” DCM defines the contract, implementors choose the technology. Four store contract types: + +| Contract Type | Stores | Key Characteristics | +|--------------|--------|-------------------| +| **GitOps Store** | Intent, Requested, Layer, Policy | Branch/PR/merge semantics, immutable history, CI/CD hooks, Search Index companion | +| **Event Stream Store** | Realized, Discovered, Audit events | Append-only, entity-keyed streams, replayable, high-throughput | +| **Audit Store** | Compliance audit records | Compliance-grade, immutable, long-retention (7+ years FSI), hash-verified | +| **Observability Store** | Metrics, traces, logs | Time-series, short-to-medium retention, OpenTelemetry format | + +**Search Index** โ€” non-authoritative queryable projection of GitOps stores. Rebuilt from Git on demand. Git always wins on disagreement. + +**DCM-internal caches** (Layer Cache, Policy Cache, Catalog Cache) โ€” not Storage Providers. Non-authoritative, cache-aside pattern, invalidated on writes. + +### 15.4 Consumer Ingress +- **Web UI** โ€” web interface for human consumers +- **Consumer API** โ€” API interface for programmatic consumers and external systems + +### 15.5 Egress +- **Messaging Protocol** โ€” protocol translation to external systems +- **Interoperability API** โ€” common API spec and data model for Service Provider communication + +--- + +## SECTION 16 โ€” SERVICE PROVIDERS + +### 16.1 Core Principle +DCM is **not concerned with how a provider accomplishes its work** โ€” only with: +- The data that crosses the boundary (conformant data in, conformant data out) +- The trust and contractual obligations the provider has declared and honored + +### 16.2 Naturalization and Denaturalization +- **Naturalization** โ€” provider transforms DCM unified data into its own tool-specific format for execution +- **Denaturalization** โ€” provider transforms tool-specific results back into DCM unified format for return to the control plane + +### 16.3 Provider Contract Dimensions +Providers must honor a multi-dimensional contract: + +**Data Contract** +- Accept DCM unified data format +- Return complete DCM unified data format (not just status codes) +- Implement Naturalization and Denaturalization + +**Sovereignty Contract** +- Explicitly declare which sovereignty dimensions they can satisfy +- Declaration is binding โ€” if declared, it must be delivered +- Sovereignty dimensions: Data/Content, Operational, Security/Compliance, Placement/Mobility + +**Capability Contract** +- Declare which Resource Types they implement +- Declare which lifecycle operations they support (CRUD + Discovery) +- Declare what they do NOT support + +**Lifecycle Contract** +- Support all declared lifecycle states +- Participate in Discovery when requested +- Report Realized State completely and accurately +- Handle drift detection requests + +**Trust Contract** *(validation mechanism โ€” to be detailed)* +- Providers must be validated and certified to participate +- Trust is established at onboarding +- Chain of trust must be maintained + +**Compliance/Audit Contract** +- Maintain audit trail of actions taken +- Make audit data available to DCM in DCM unified format +- Must not take actions outside of DCM-initiated requests (for managed resources) + +**SLA/Operational Contract** +- Response time expectations +- Availability requirements +- Error handling and reporting standards +- Retry and idempotency guarantees + +### 16.4 Provider Types +- **Atomic Providers** โ€” manage a single fundamental resource type (VM, IP, VLAN, container) +- **Meta Providers** โ€” compose multiple providers as components of their own service +- **Process Providers** โ€” purely process-based (no infrastructure resource, but a workflow or automation) +- **Real-world providers** are typically combinations of all three + +--- + +## SECTION 17 โ€” THE POLICY ENGINE + +### 17.1 Purpose +The Policy Engine is the **single authoritative logic gate for all business rules** in DCM. It enforces governance without embedding business logic into the control plane. It is the **sole authority for field override control** โ€” no other component sets override metadata on fields. + +### 17.2 Policy Categories + +| Category | Description | Modifies Data? | Example | +|----------|-------------|----------------|---------| +| **Transformation** | Enriches or modifies the payload. Adds missing fields, applies standards. All changes recorded in provenance. May set `override: constrained`. | Yes | Inject PCI-compliant cryptography standard | +| **Validation** | Checks payload against rules. Pass/fail โ€” no field modification. Failure rejects request. | No | VM class allowed in DMZ Zone A | +| **GateKeeper** | Highest authority. Overrides any field including consumer input. Sets `override: immutable`. Halts execution. | Yes โ€” overrides everything | Block request violating sovereignty | + +### 17.3 Policy Hierarchy +Three-tier execution โ€” Global first, User last. Within each tier sorted by priority (higher value = higher authority): + +``` +Global (Super Admin) โ€” runs first, highest authority +Tenant (Tenant Admin) โ€” runs second +User (End User) โ€” runs last, lowest authority +``` + +A Global policy cannot be overridden by Tenant or User. Field locks set by higher-authority policies cannot be unlocked by lower-authority policies. + +### 17.4 Policy Implementation +- **Engine:** OPA (Open Policy Agent) with Rego policy language +- **Storage:** GitOps โ€” all policies in Git, versioned, immutable once published +- **Execution:** Stored-policy model โ€” OPA pre-loads policies; evaluation calls pre-loaded modules +- **Outputs:** `rejected` (bool), `patch` (field mutations), `constraints` (JSON Schema locks), provider placement constraints (NOT direct provider naming) + +### 17.5 Policy Rules +- Policies operate only on policy definition, core data, and request payload data +- Outcomes must be **deterministic** โ€” same input always produces same output for a given version +- All modifications recorded in field-level provenance with policy UUID, tier level, and reason +- Policies follow the five-status lifecycle: `developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired` + +### 17.6 Policy Scope +- **Core Policies** โ€” all DCM actions regardless of provider +- **Service Policies** โ€” specific to a Service Provider's services +- **Organizational Policies** โ€” defined by the implementing organization +- **Domain Policies** โ€” scoped to a specific domain or business unit + +### 17.7 Proposed Policy โ€” Shadow Execution +When a policy is in `proposed` status it runs in **shadow mode** against real request traffic: +- Executes alongside active policies on every relevant request +- Output captured in `proposed_evaluation_record` โ€” what it would have done +- Output is **never applied** to the actual request +- Feeds the Validation Dashboard for reviewer analysis before activation +- Impact categories: `none | low | medium | high | critical` +- Activation requires approval after review period + +### 17.8 Policy and Override Control +The Policy Engine exclusively sets `override_control` metadata. Three levels (see Section 6.10โ€“6.11): +- Level 1 โ€” no declaration โ†’ fully overridable (default) +- Level 2 โ€” simple `override: allow|constrained|immutable` +- Level 3 โ€” full `override_matrix` with per-actor permissions and trusted grants + +--- + +## SECTION 18 โ€” KEY USE CASES + +### 18.1 Datacenter Rehydration (Repave) +Reconstruct ALL required components and configurations from code after catastrophic failure (ransomware, DR event, mandatory 90-day repave). Industry benchmark: leading FSI organization 60-day repave. DCM must beat this. Key metric: TTR (Time to Recovery). + +### 18.2 Intelligent Placement +Automated placement of resources and workloads based on metadata, policies from CMDB/CISO/platform teams. Consumer declares criteria (location, SLA, security zones) โ€” DCM determines optimal placement automatically. + +### 18.3 Application as a Service +Meta Service Provider that consumes application code and provides its full execution lifecycle. Consumer defines metadata for required technologies or SLAs โ€” provider handles the rest. + +### 18.4 Regional Sovereignty +Enforce workload placement within specific regions or sovereignty constraints. CISO/CCO-driven policy ensuring data residency and jurisdictional compliance. + +### 18.5 Data Enrichment +System enriches consumer requests with ancillary implementation details the consumer should not need to know. Consumer declares intent โ€” DCM fills in the details. + +### 18.6 Greening the Brownfield +Bring existing unmanaged resources under DCM lifecycle management: +1. **Discovery** โ€” provider interrogates existing resources, creates Discovered State payload +2. **Enrichment** โ€” business data associated with discovered resources (owner, cost center, purpose) +3. **Lifecycle Ownership** โ€” Discovered State promoted to Realized State; DCM assumes lifecycle management + +--- + +## SECTION 19 โ€” DIGITAL SOVEREIGNTY + +DCM addresses four dimensions of digital sovereignty: + +| Dimension | DCM Enabler | Impact | +|-----------|-------------|--------| +| **Data and Content Sovereignty** | Data Model, Policy Engine, Validated Providers | Data residency and jurisdictional compliance | +| **Operational Sovereignty** | Policy Engine | Sovereign Execution Posture, Hard Tenancy enforcement | +| **Security and Compliance Sovereignty** | Audit, GRC | Evidence for strict regional mandates | +| **Mobility, Placement, Modernization** | Policy Engine, Providers, Data Model | Automated placement, provider mobility, brownfield ingestion | + +**Sovereign Execution Posture** โ€” the target end state where all operations are governed, auditable, and compliant with sovereignty requirements. This is the north star concept of DCM. + +--- + +## SECTION 20 โ€” PERSONAS + +| Persona | Primary Concern | +|---------|----------------| +| **Consumer** | Self-service access to resources and services | +| **Service Provider** | Exposing services through DCM catalog | +| **Auditor** | End-to-end transaction review and validation | +| **Policy Creator** | Defining and maintaining governance policies | +| **SRE** | Stability, uptime, drift reconciliation, brownfield management | +| **CTO** | Accelerate innovation, reduce risk, enable sovereignty | +| **CIO/MD** | Lifecycle management, agility, IT investment maximization | +| **CISO/CCO** | Sovereignty enforcement, compliance, risk reduction | +| **Application Owner** | Focus on application code, not infrastructure specifics | +| **Line of Business** | Business outcomes โ€” new products, revenue, compliance | + +--- + +## SECTION 21 โ€” TERMINOLOGY GLOSSARY + +| Term | Definition | +|------|-----------| +| **DCM** | Data Center Management โ€” the framework itself | +| **Sovereign Execution Posture** | Target state where all operations are governed, auditable, and sovereignty-compliant | +| **Hard Tenancy** | Strict isolation between tenants at the infrastructure level | +| **UDM** | Unified Data Model โ€” the centralized data schema and single source of truth | +| **Naturalization** | Converting UDM format to provider-specific format for execution | +| **Denaturalization** | Converting provider-specific results back to UDM format | +| **Greening the Brownfield** | Bringing existing unmanaged resources under DCM lifecycle management | +| **Intent Portability** | Replaying an Intent State through current policies to produce a new Requested State | +| **Resource Type** | Abstract, vendor-neutral definition of a class of resource | +| **Provider Catalog Item** | Concrete provider implementation of a Resource Type Specification | +| **Portability-Breaking** | A field or operation that ties a request to a specific provider | +| **CMDB** | Configuration Management Database โ€” DCM aims to replace the fragmented multi-CMDB problem | +| **GRC** | Governance, Risk, and Compliance | +| **TTR** | Time to Recovery โ€” key metric for rehydration use case | +| **MTTD** | Mean Time to Deploy or Detect | +| **MTTR** | Mean Time to Recovery/Repair | +| **FSI** | Financial Services Institution | +| **IaC** | Infrastructure as Code | +| **GitOps** | Managing infrastructure definitions through Git workflows | +| **CRUD** | Create, Read, Update, Delete โ€” full lifecycle operations | +| **IPU** | In-Place Upgrade | +| **EOL** | End of Life | +| **Field-Level Provenance** | Structural mechanism carrying data lineage within each data object | +| **Data Lineage** | Complete chain of custody of any field value from origin through all modifications | +| **Base Layer** | Foundation entity for a resource โ€” every layer chain starts here | +| **Core Layer** | Type-agnostic data layer carrying organizational and infrastructure context | +| **Service Layer** | Type-scoped data layer carrying service-specific configuration โ€” must declare Resource Type scope | +| **Request Layer** | Consumer's declared intent โ€” becomes Intent State on submission | +| **Layer Chain** | Ordered sequence of layers merged to produce an assembled payload | +| **Assembly Process** | Seven-step process by which the Request Payload Processor builds a Requested State payload | +| **GateKeeper Policy** | Highest-authority policy that can override any field including consumer input | +| **Transformation Policy** | Policy that enriches or modifies payload fields โ€” all changes recorded in provenance | +| **Validation Policy** | Policy that checks payload against rules โ€” pass/fail, no field modification | +| **Resource/Service Request** | What a consumer submits to DCM โ€” declared intent to consume a resource or service | +| **Resource/Service Entity** | The "thing" produced by a provider fulfilling a request โ€” the allocation made real | +| **DCM Tenant** | Mandatory first-class ownership boundary for all Resource/Service Entities | +| **Allocation Model** | Provider retains infrastructure ownership; consumer owns the Entity allocation | +| **Whole Allocation Model** | Entire resource allocated as indivisible unit; provider retains ownership | +| **Full Transfer Model** | Provider transfers complete ownership of underlying resource to consumer Tenant | +| **Hybrid Transfer Model** | Ownership can transfer multiple times; always exactly one owning Tenant | +| **Process Resource Entity** | Ephemeral execution resource โ€” playbook, pipeline, workflow. Permanent execution record. | +| **Dependency Graph** | Complete map of all resources required to fulfill a request including transitive dependencies | +| **Type-Level Dependency** | Portable, provider-agnostic dependency declared at Resource Type Specification level | +| **Provider-Specific Dependency** | Additional dependency declared at Provider Catalog Item level โ€” must be marked portability-breaking | +| **Resource Group** | Flexible composable grouping entity โ€” functions like a structured tag | +| **Custom Resource Group** | Implementor-defined grouping entity with full parity to DCM Default Resource Group | +| **Tenant Advocate** | DCM's role in protecting Tenant interests in all provider interactions | +| **DCM System Policy** | Non-overridable policy built into DCM โ€” cannot be disabled or overridden by organizational policy | +| **Webhook** | Push-based outbound notification from DCM to an external system triggered by a DCM event | +| **DCM Event Type** | A versioned, typed event that DCM can emit โ€” follows universal versioning model | +| **Event Type Registry** | DCM-maintained registry of standard event types โ€” extensible like the Resource Type Registry | +| **Webhook Registration** | Declaration by a consumer, provider, or external system of which DCM events they want to receive and where | +| **Discussion Topics** | Living document (DISCUSSION-TOPICS.md) capturing unresolved design decisions and topics requiring further discussion | +| **Override Preference** | Level 2 simple override declaration โ€” single `override: allow|constrained|immutable` on a field | +| **Override Matrix** | Level 3 per-actor permission matrix for fields requiring nuanced governance | +| **Field Override Control** | Graduated mechanism (Levels 1-3) governing who can change what field, under what conditions | +| **Structural Layer Rules** | Non-configurable rules enforced by the Request Payload Processor โ€” layer immutability, precedence order, chain integrity | +| **Business Override Rules** | Configurable override control rules enforced by the Policy Engine via override metadata | +| **Trusted Grant** | Explicit expansion of override permissions issued by a higher-authority actor to a specific entity UUID | +| **Actor Registry** | Extensible registry of override actors โ€” built-in (policy.global, consumer_request, sre_override, etc.) plus custom actors | +| **Basis for Value** | Field metadata documenting why a particular value was set | +| **Baseline Value** | Field metadata recording the original default value before any override was applied | +| **Entity Relationship** | Universal bidirectional relationship between any two entities โ€” internal or external | +| **Entity Relationship Graph** | Complete traversable graph of all entity relationships in DCM | +| **Relationship UUID** | UUID identifying a specific relationship โ€” same on both sides of the bidirectional record | +| **Relationship Type** | Fixed vocabulary describing the nature of a relationship (requires, depends_on, contains, references, peer, manages) | +| **Relationship Role** | Semantic label describing the function a related entity serves (compute, storage, networking, business_unit, etc.) | +| **Relationship Nature** | Structural character of a relationship โ€” constituent, operational, or informational | +| **Lifecycle Policy** | Declares what happens to an entity when its related entity changes state | +| **Bundled Declaration Expansion** | Processor mechanism expanding bundled fields (e.g., storage in VM request) into first-class entities and relationships | +| **Information Provider** | DCM provider type serving authoritative external data DCM references but does not own | +| **External Entity Reference** | Stable pointer record DCM uses to reference data in an external system | +| **Standard Data** | DCM-defined fields on an information type โ€” used for lookups and operational decisions | +| **Extended Data** | Organization-defined fields added to an information type โ€” carried in payload but not used for DCM core operations | +| **Information Type** | Registry entry for a category of external data (Business.BusinessUnit, Identity.Person, etc.) | +| **Stable External Key** | The external system's UUID used as the primary lookup anchor for an external entity reference | +| **Trust But Verify** | DCM's approach to external references โ€” trusts external data is correct, verifies references remain valid | +| **DCM Operator Interface Specification** | The formal technical contract defining how Kubernetes operators integrate with DCM as Service Providers | +| **DCM Operator SDK** | Go library implementing the Operator Interface Specification โ€” enables Level 1 conformance in one day | +| **Conformance Level** | The level of DCM integration an operator implements โ€” Level 1 (basic), Level 2 (standard), Level 3 (full) | +| **Naturalization (Kubernetes)** | Translating DCM Requested State into a Kubernetes CR | +| **Denaturalization (Kubernetes)** | Translating Kubernetes CR status back into DCM Realized State format | +| **Unsanctioned Change** | A change to a DCM-managed CR that did not originate from a DCM request โ€” detected via missing DCM request annotation | +| **Operator Adapter** | A component implementing the DCM Service Provider API on behalf of an operator that cannot be modified directly | +| **CNCF Sandbox** | The initial CNCF maturity level โ€” target for initial DCM project submission | +| **Conformance Test Suite** | The test suite that validates an operator's implementation against the DCM Operator Interface Specification | +| **Intent State** | The immutable record of a consumer's original declaration โ€” captured before any assembly or policy evaluation | +| **Requested State** | The fully assembled, policy-processed, provider-ready payload โ€” the authoritative record of what DCM instructed a provider to build | +| **Realized State** | The provider-confirmed record of what was actually built โ€” append-only event stream keyed by entity UUID | +| **Discovered State** | What DCM observes actually existing through active discovery โ€” ground truth for drift detection | +| **Storage Provider** | The fourth formal DCM provider type โ€” the interface through which DCM persists and streams all state data | +| **GitOps Store** | Storage Provider type for Intent and Requested State โ€” branch, PR, merge, CI/CD hook semantics | +| **Event Stream Store** | Storage Provider type for Realized and Discovered State โ€” append-only, entity-keyed, replayable | +| **Search Index** | Queryable projection of GitOps stores โ€” explicitly non-authoritative, rebuilt from Git on demand | +| **Provider-Portable Rehydration** | Rehydration with provider selection re-evaluated through current placement policies | +| **Faithful Rehydration** | Rehydration honoring the original provider selection from the source record | +| **Pinned Policy Version** | Rehydration using policies as of a specific historical timestamp โ€” requires elevated authorization | +| **Audit Component** | Separate DCM component aggregating provenance events from all stores โ€” compliance-grade, long-retention | +| **Observability Store** | Time-series metrics, traces, and logs โ€” operational, not compliance-grade | +| **Third Rail** | Direct API ingress path โ€” bypasses PR workflow, never bypasses governance | +| **Unsanctioned Change** | A resource modification not traceable to a DCM request โ€” triggers UNSANCTIONED_CHANGE event | +| **Layer Domain** | Organizational and architectural home of a layer โ€” system, platform, tenant, service, provider | +| **Layer Handle** | Human-readable stable identifier for a layer โ€” format: domain/layer_type/name | +| **Priority Schema** | Hierarchical dotted-notation priority system for deterministic layer conflict resolution | +| **Priority Value** | Numeric dotted-notation priority โ€” higher value wins; no ceiling, infinitely insertable in both directions | +| **Immutable Ceiling** | `immutable_ceiling: absolute` โ€” explicit declaration that a field lock cannot be overridden by any future higher-priority policy; the nuclear option for true non-negotiables | +| **Priority Label** | Semantic context for a priority value โ€” human-readable, does not affect ordering | +| **Reference Priority Taxonomy** | DCM's advisory priority category ranges โ€” not enforced, organizations adopt/adapt/ignore | +| **Artifact Metadata** | Universal metadata block on every DCM artifact โ€” identity, ownership, creation, modification history, contact | +| **created_by** | Artifact metadata field โ€” the audit record of who physically submitted the artifact | +| **owned_by** | Artifact metadata field โ€” the accountability record of who is responsible and receives notifications | +| **created_via** | Artifact metadata field โ€” ingestion path: pr, api, migration, system | +| **Proposed Shadow Execution** | Policy artifact in proposed status executing against real traffic โ€” output captured, never applied | +| **Proposed Evaluation Record** | Shadow output record for a proposed policy โ€” what it would have done on a real request | +| **Validation Dashboard** | Review interface showing aggregate shadow output for proposed policies before activation | + +--- + +## SECTION 22 โ€” OPEN QUESTIONS + +These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. + +| # | Question | Area | +|---|----------|------| +| 1 | Where should data caches live? (Shore, Ship, Enclave, all?) | Data Model | +| 2 | Should cache synchronization be push, pull, or both? | Data Model | +| 3 | Which cache is authoritative when caches diverge? | Data Model | +| 4 | What mechanism maintains consistency across distributed caches? | Data Model | +| 5 | Should the data model allow embedded target-technology-specific data bundles? | Data Model | +| 6 | How are the four states represented physically? | Data Model | +| 7 | Performance impact of field-level provenance at scale โ€” optimization strategies? | Data Model | +| 8 | Should provenance metadata be inline or in a linked provenance document? | Data Model | +| 9 | What is the governance model for proposing new Resource Types to the registry? | Catalog | +| 10 | Should the registry support a formal review/approval workflow? | Catalog | +| 11 | What is the minimum sunset period for deprecated definitions? | Catalog | +| 12 | Should version constraints in requests be strictly enforced or advisory? | Catalog | +| 13 | How are conflicts resolved when multiple providers satisfy all narrowing criteria equally? | Catalog | +| 14 | Should the registry be distributed or centralized? Sovereignty implications? | Catalog | +| 15 | Trust validation mechanism for provider certification | Providers | +| 16 | Audit vs. Observability โ€” are these truly separate components? | Control Plane | +| 17 | Message Bus โ€” should it be exposed as consumer ingress or egress only? | Control Plane | +| 18 | GateKeeper vs. Validation policy distinction โ€” needs better examples | Policy Engine | +| 19 | How are conflicting Service Layers at the same precedence level resolved? | Data Layers | โœ… Resolved โ€” priority schema + ingestion conflict detection | +| 20 | Should Core Layers be ordered within their precedence level? | Data Layers | โœ… Resolved โ€” priority schema provides deterministic ordering | +| 21 | Can a consumer explicitly exclude a layer from their request? | Data Layers | +| 22 | How are Service Layers registered and versioned relative to their Service Provider registration version? | Data Layers | +| 23 | Should assembly support conditional layer inclusion โ€” a layer only applied if a specific field value is present? | Data Layers | +| 24 | How does the layer chain interact with service dependencies โ€” does each dependent service get its own chain? | Data Layers | +| 25 | For Hybrid Transfer โ€” what is the maximum number of ownership transfers allowed? | Entities | +| 26 | For Whole Allocation of bare metal โ€” how is indivisibility enforced at the provider level? | Entities | +| 27 | Should capacity confidence ratings trigger automatic actions (e.g., LOW triggers Mode 1 query)? | Entities | +| 28 | For Process Resources โ€” should there be a maximum execution time before DCM escalates? | Entities | +| 29 | How does SUSPENDED state interact with cost analysis โ€” is a suspended Entity still billable? | Entities | +| 30 | How are dependency graphs versioned relative to catalog item versions? | Dependencies | +| 31 | Should the dependency graph be stored as a separate entity or embedded in the request payload? | Dependencies | +| 32 | How are cross-tenant dependencies handled? | Dependencies | +| 33 | Should there be a maximum dependency graph depth? | Dependencies | +| 34 | How does the dependency graph interact with the Meta Provider model? | Dependencies | +| 35 | Should DCM maintain a registry of well-known custom group types? | Grouping | +| 36 | How does group membership interact with sovereignty โ€” can a group span sovereignty boundaries? | Grouping | +| 37 | When a Tenant is decommissioned, what happens to its resources and group memberships? | Grouping | +| 38 | Should Resource Groups support time-bounded membership? | Grouping | +| 39 | How are group-level policies inherited by nested child groups โ€” opt-in or opt-out? | Grouping | +| 40 | Webhook registration model โ€” Consumer API, Provider Registration, or dedicated Webhook API? | Webhooks | +| 41 | Full DCM event taxonomy and whether it should be a versioned registry | Webhooks | +| 42 | Webhook payload format โ€” full state payload vs reference + event type | Webhooks | +| 43 | Webhook authentication model for outbound calls | Webhooks | +| 44 | Webhook retry and reliability obligations | Webhooks | +| 45 | Webhook ordering guarantees | Webhooks | +| 46 | Relationship between webhooks and the Message Bus | Webhooks | +| 47 | Should provider webhook support be mandatory in the Provider Contract? | Webhooks | +| 48 | Tenant vs platform-level webhook scoping | Webhooks | +| 49 | Should webhook registrations declare which payload schema version they expect? | Webhooks | +| 50 | Should override_preference be declarable in layer definitions as a hint to the Policy Engine? | Override Control | +| 51 | When immutable is set by a Global policy, can a higher-priority Global policy still override it? | Override Control | โœ… Resolved โ€” execution order makes default immutable effectively absolute; immutable_ceiling: absolute provides explicit forward-looking protection | +| 52 | Should constraint_schema on a constrained field be visible to consumers in the Service Catalog UI? | Override Control | +| 53 | Enhancement gaps: storage/networking bundling vs. dependency model โ€” V1 simplification or new concept needed? | Enhancements | +| 54 | Enhancement gaps: selected_provider as policy output vs. placement component concern | Enhancements | +| 55 | Enhancement gaps: migration path from V1 (no Tenant) to Tenant-mandatory | Enhancements | +| 56 | Enhancement gaps: should editable field concept from Catalog Item Schema be incorporated into Resource Type Spec? | Enhancements | +| 57 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Entity Relationships | +| 58 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Entity Relationships | +| 59 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Entity Relationships | +| 60 | Should there be a maximum relationship graph depth? | Entity Relationships | +| 61 | How are shared entities represented โ€” an entity required by multiple parents? | Entity Relationships | +| 62 | How are conflicting Information Provider push events handled โ€” two providers claim authority for the same record? | Information Providers | +| 63 | Should Information Providers support write-back โ€” DCM updating external records via the provider? | Information Providers | +| 64 | How is the extended schema versioned when a provider adds or removes extended fields? | Information Providers | +| 65 | Should DCM maintain a registry of well-known Information Providers to simplify onboarding? | Information Providers | +| 66 | How does Information Provider verification interact with air-gapped environments? | Information Providers | +| 67 | Should CNCF submission be for DCM as a whole or for the Operator Interface Specification as a standalone standard? | CNCF Strategy | +| 68 | Which FSI consortium members will be named as public adopters in the CNCF submission? | CNCF Strategy | +| 69 | How does the Namespace-to-Tenant mapping work for clusters with pre-existing namespaces? | Kubernetes Compatibility | +| 70 | How does DCM interact with Kubernetes admission webhooks โ€” duplicate or complement Policy Engine? | Kubernetes Compatibility | +| 71 | Should the Kubernetes Information Provider be a built-in DCM component or separately deployed? | Kubernetes Compatibility | +| 72 | How does DCM interact with managed Kubernetes services (EKS, GKE, AKS) where cluster management is outside user control? | Kubernetes Compatibility | +| 73 | Should the SDK support non-Go operator frameworks via a language-agnostic REST adapter? | SDK Design | +| 74 | How should the SDK handle DCM endpoint unavailability โ€” queue events locally or drop? | SDK Reliability | +| 75 | Should the entity UUID be preserved or regenerated on rehydration? | Four States | +| 76 | For pinned policy version rehydration โ€” what is the minimum authorization level? | Four States | +| 77 | How are concurrent rehydration requests for the same entity handled? | Four States | +| 78 | Should the Discovered Store retain full history or only a configurable window? | Four States | +| 79 | Git repository structure for Intent and Requested stores โ€” deferred pending Q54 | Storage | +| 80 | Should Storage Providers support multi-region replication as a declared capability? | Storage | +| 81 | How are Storage Provider failures handled โ€” failover, queuing, or rejection? | Storage | +| 82 | Should the Search Index be a separate Storage Provider or bundled with GitOps store? | Storage | +| 83 | Should the Audit Store be a specialized Storage Provider or a general Event Stream Store? | Audit | +| 84 | Should DCM provide a default observability dashboard or only the telemetry? | Observability | +| 85 | Should the background conflict validation job run on schedule or be event-triggered? | Data Layers | +| 86 | What is the minimum validation review period for a proposed policy before activation? | Policy Engine | +| 87 | Should the proposed shadow evaluation record be stored in the Audit Store or a separate validation store? | Storage | +| 88 | Should organizations be able to define their own artifact status extensions beyond the five standard statuses? | Artifact Metadata | + +--- + +## SECTION 23 โ€” DOCUMENTATION STRUCTURE + +DCM documentation follows a hierarchical structure: + +``` +dcm-docs/ (internal working docs) +โ”œโ”€โ”€ README.md +โ”œโ”€โ”€ DCM-AI-PROMPT.md # This file +โ”œโ”€โ”€ DISCUSSION-TOPICS.md +โ”œโ”€โ”€ data-model/ # โ†’ website: /docs/architecture/data-model/ +โ”‚ โ”œโ”€โ”€ 00-context-and-purpose.md โœ… +โ”‚ โ”œโ”€โ”€ 02-four-states.md โœ… +โ”‚ โ”œโ”€โ”€ 03-layering-and-versioning.md โœ… +โ”‚ โ”œโ”€โ”€ 05-resource-type-hierarchy.md โœ… +โ”‚ โ”œโ”€โ”€ 06-resource-service-entities.md โœ… +โ”‚ โ”œโ”€โ”€ 07-service-dependencies.md โœ… +โ”‚ โ”œโ”€โ”€ 08-resource-grouping.md โœ… +โ”‚ โ”œโ”€โ”€ 09-entity-relationships.md โœ… +โ”‚ โ”œโ”€โ”€ 10-information-providers.md โœ… +โ”‚ โ”œโ”€โ”€ 11-storage-providers.md โœ… +โ”‚ โ””โ”€โ”€ 12-audit-provenance-observability.md โœ… +โ””โ”€โ”€ specifications/ # โ†’ website: /docs/architecture/specifications/ + โ”œโ”€โ”€ dcm-operator-interface-spec.md โœ… + โ”œโ”€โ”€ 11-kubernetes-compatibility.md โœ… + โ”œโ”€โ”€ dcm-operator-sdk-api.md โœ… + โ””โ”€โ”€ cncf-strategy.md โœ… + +Website structure (Hugo / Hextra): +content/ +โ”œโ”€โ”€ _index.md # Homepage โ€” 4 bottom cards +โ””โ”€โ”€ docs/ + โ”œโ”€โ”€ _index.md # Docs index + โ”œโ”€โ”€ architecture/ + โ”‚ โ”œโ”€โ”€ _index.md # Architecture section โ€” 3 cards + โ”‚ โ”œโ”€โ”€ overview.md # High Level Design โœ… + โ”‚ โ”œโ”€โ”€ data-model/ + โ”‚ โ”‚ โ”œโ”€โ”€ _index.md # Data Model section โ€” 11 cards + โ”‚ โ”‚ โ””โ”€โ”€ (11 data model docs) โœ… + โ”‚ โ””โ”€โ”€ specifications/ + โ”‚ โ”œโ”€โ”€ _index.md # Specifications section โ€” 4 cards + โ”‚ โ””โ”€โ”€ (4 specification docs) โœ… + โ””โ”€โ”€ enhancements/ + โ”œโ”€โ”€ _index.md + โ””โ”€โ”€ (existing enhancement stubs โ€” unchanged) +``` + +--- + +## SECTION 24 โ€” WORKING INSTRUCTIONS FOR AI MODELS + +When working on this project, follow these instructions: + +1. **Data model is foundational** โ€” all design decisions must be evaluated against their impact on the data model first +2. **Four states are always relevant** โ€” when designing any component or flow, identify which states it reads from and writes to +3. **Provenance is non-negotiable** โ€” any component that modifies data must record provenance; never design around this requirement +4. **UUIDs everywhere** โ€” every entity, definition, and data object must have a UUID; never reference by name alone +5. **Portability first** โ€” when designing resource types or catalog items, start with universal fields; justify any deviation toward conditional or provider-specific +6. **Policy over hardcoding** โ€” business logic belongs in the Policy Engine, not in component code +7. **Declarative over procedural** โ€” data describes state, not steps; procedures belong in providers +8. **Universal versioning** โ€” every definition is versioned using Major.Minor.Revision; never create an unversioned definition +9. **Universal artifact lifecycle** โ€” every definition must support the five-status lifecycle: `developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired`; never design an artifact with only the old three-status model +10. **Flag open questions** โ€” do not make assumptions about unresolved items; surface them and ask +11. **Documentation format** โ€” Markdown, hierarchical structure, following the established document style +12. **Provider agnosticism** โ€” DCM does not care how providers accomplish their work; only the data contract matters +13. **Layer type scoping** โ€” Core Layers are type-agnostic; Service Layers must always be type-scoped; a Service Layer without a declared type scope is always invalid +14. **GateKeeper is supreme** โ€” GateKeeper policies can override anything including consumer input; this is by design for sovereignty and security enforcement; never design around it +15. **DCM always owns the data** โ€” regardless of operational ownership model, DCM is always the authoritative system of record for all Resource/Service Entity data and lifecycle +16. **Tenant is mandatory** โ€” every Resource/Service Entity must belong to exactly one Tenant; this is a non-overridable DCM System Policy; no exceptions +17. **Ownership vs consumption** โ€” a resource belongs to one Tenant (owner) but can be consumed by multiple Tenants via the Service Catalog; never conflate ownership and consumption +18. **Dependencies declared in advance** โ€” all dependencies must be declared in the data model before execution; provider-discovered runtime dependencies are not acceptable +19. **Process Resources need provenance** โ€” if a Process Resource modifies an Infrastructure Entity, that Entity's provenance must reference the Process Resource UUID +20. **Two-tier policies** โ€” DCM System Policies are non-overridable; Organizational Policies are configurable; never design a System Policy as organizational or vice versa +21. **Check DISCUSSION-TOPICS.md first** โ€” before designing any component or capability, check the discussion topics document for active or parked topics that may affect the design; never proceed on a topic marked ๐Ÿ”ด Blocking without resolution +22. **Webhooks are unresolved** โ€” webhook integration is under active design (DISCUSSION-TOPICS.md TOPIC-001); do not make implementation assumptions about webhook mechanics, payload format, authentication, or retry behavior until design questions are resolved; webhooks ARE confirmed as an Egress capability with Policy Engine integration +23. **Override control belongs to Policy Engine** โ€” the Request Payload Processor enforces structural layer rules only; field-level override control (allow/constrained/immutable) is set exclusively by the Policy Engine; never design around this boundary +24. **Enhancement gaps are tracked** โ€” DISCUSSION-TOPICS.md TOPIC-011 documents compatibility gaps between existing enhancement documents and the data model; when working on components covered by those enhancements, check TOPIC-011 for known gaps that need resolution +25. **Single relationship model** โ€” all entity relationships use the universal bidirectional model in doc 09; never create a separate binding or dependency mechanism; the entity relationship graph supersedes the dependency graph concept +26. **Information Providers are not Service Providers** โ€” Information Providers serve data DCM references but does not own; DCM never caches external data authoritatively; only `display_name` is cached non-authoritatively for UI convenience +27. **Standard data only for operational decisions** โ€” DCM core only relies on standard information type fields for lookups, policy evaluation, and operational decisions; extended fields are carried in payloads but never used for DCM core operations +28. **DCM is a Kubernetes superset, not a replacement** โ€” Kubernetes manages the execution plane; DCM manages the management plane; operators become DCM Service Providers through the Operator Interface Specification; never design DCM as competing with Kubernetes +29. **Operator adapter pattern** โ€” when an operator cannot be modified directly, an adapter implements the DCM Service Provider API on its behalf; the adapter handles Naturalization (DCM โ†’ CR) and Denaturalization (CR status โ†’ DCM); this is the standard pattern for existing operators +30. **Conformance levels gate capabilities** โ€” Level 1 operators get catalog and basic monitoring; Level 2 adds placement and drift detection; Level 3 adds sovereignty and brownfield ingestion; always check what level an operator has declared before assuming capabilities are available +31. **Storage Providers define contracts, not implementations** โ€” DCM specifies what a store must do; implementors choose the technology; never reference a specific technology (Kafka, Git, Elasticsearch) as a DCM requirement โ€” reference the store type and contract instead +32. **Governance is never skippable in rehydration** โ€” all relevant policies always apply regardless of rehydration source, mode, or urgency; the only variable is current vs pinned policy version, and pinned requires elevated authorization +33. **Audit and Observability are separate concerns** โ€” Audit is compliance-grade, long-retention, persona-restricted; Observability is operational, short-retention, SRE-accessible; never conflate them or design them as the same component +34. **All DCM capabilities surface through the API Gateway** โ€” Audit, Observability, Catalog, Requests, Entities, Policies all live in a unified API hierarchy; there are no separate endpoints outside the Gateway +35. **The Search Index is non-authoritative** โ€” if Search Index and GitOps store disagree, Git always wins; the Search Index is a performance layer only; it can be cleared and rebuilt from Git at any time +36. **All artifacts carry artifact metadata** โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions โ€” everything. No artifact is exempt from the universal metadata block +37. **created_by โ‰  owned_by** โ€” created_by is the audit record of who submitted the artifact; owned_by is the accountability record of who is responsible and receives notifications; these may be different people/teams +38. **Conflicts are resolved at ingestion, not assembly** โ€” all active layers in DCM are pre-validated conflict-free; the assembly process never encounters an ambiguous merge; if a conflict is found at ingestion, the PR is blocked until resolved +39. **Priority schema is advisory for categories, mandatory for ordering** โ€” the reference taxonomy (900=Compliance, 800=Security, etc.) is advisory and organizations may adapt it; however, the numeric comparison rule is always enforced and always deterministic +40. **Proposed status enables shadow validation** โ€” policy artifacts in proposed status execute in shadow mode against real traffic; output is captured in proposed_evaluation_record but never applied; this is the required validation step before activation + +--- + +*This prompt script is a living document. Update it whenever architectural decisions are made or open questions are resolved.* diff --git a/content/docs/architecture/data-model/_index.md b/content/docs/architecture/data-model/_index.md new file mode 100644 index 0000000..739ff75 --- /dev/null +++ b/content/docs/architecture/data-model/_index.md @@ -0,0 +1,28 @@ +--- +title: Data Model +type: docs +weight: 2 +sidebar: + open: true +--- + +# DCM Data Model + +The DCM Data Model is the foundational layer that governs how all data in DCM is +represented, versioned, assembled, and governed. It is the single source of truth +for all resources across the full lifecycle โ€” from consumer intent through +realization, operation, and decommission. + +{{< cards >}} + {{< card link="context-and-purpose" title="Context and Purpose" icon="information-circle" subtitle="Why the data model exists, core principles, field-level provenance, and the artifact metadata standard." >}} + {{< card link="four-states" title="Four States" icon="arrows-expand" subtitle="Intent, Requested, Realized, and Discovered states. Rehydration, drift detection, and CI/CD integration." >}} + {{< card link="layering-and-versioning" title="Data Layers and Assembly" icon="collection" subtitle="Layer types, assembly process, override control, Layer Domain/Handle/Priority, and conflict detection." >}} + {{< card link="resource-type-hierarchy" title="Resource Type Hierarchy" icon="cube" subtitle="The Resource Type Registry, catalog model, portability classification, and provider hierarchy." >}} + {{< card link="resource-service-entities" title="Resource and Service Entities" icon="server" subtitle="Entity definitions, ownership models, lifecycle states, and process resources." >}} + {{< card link="service-dependencies" title="Service Dependencies" icon="share" subtitle="Dependency rehydration ordering and failure handling on the entity relationship graph." >}} + {{< card link="resource-grouping" title="Resource Grouping" icon="user-group" subtitle="Tenant model, Resource Groups, grouping model, and multi-tenancy." >}} + {{< card link="entity-relationships" title="Entity Relationships" icon="link" subtitle="Universal bidirectional relationship model for all entity connections โ€” internal and external." >}} + {{< card link="information-providers" title="Information Providers" icon="globe" subtitle="External data providers, information types, lookup model, and verification." >}} + {{< card link="storage-providers" title="Storage Providers" icon="archive" subtitle="Storage provider contracts for GitOps stores, event streams, audit, and observability." >}} + {{< card link="audit-provenance-observability" title="Audit, Provenance, and Observability" icon="eye" subtitle="The three distinct concerns of audit, data lineage, and operational observability." >}} +{{< /cards >}} diff --git a/content/docs/architecture/data-model/audit-provenance-observability.md b/content/docs/architecture/data-model/audit-provenance-observability.md new file mode 100644 index 0000000..9bbec19 --- /dev/null +++ b/content/docs/architecture/data-model/audit-provenance-observability.md @@ -0,0 +1,368 @@ +--- +title: "Audit, Provenance, and Observability" +type: docs +weight: 11 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Four States](../four-states/) | [Storage Providers](../storage-providers/) | [Context and Purpose](../context-and-purpose/) + +--- + +## 1. Purpose + +Audit, Provenance, and Observability are three distinct but related capabilities in DCM. They are often conflated โ€” this document separates them precisely, defines their relationship, and establishes the architectural model for each. + +| Capability | Question Answered | Audience | Time Orientation | +|------------|------------------|----------|-----------------| +| **Provenance** | Where did this data come from and how did it change? | System โ€” embedded in data | Embedded in every payload | +| **Audit** | What happened, who authorized it, can you prove it? | Auditors, Compliance, Security | Backward-looking | +| **Observability** | Is the system healthy and performing within expectations? | SRE, Platform Engineers | Forward-looking, real-time | + +--- + +## 2. Provenance + +### 2.1 Definition + +Provenance is the structural data lineage mechanism embedded in every field of every DCM payload. It is not a separate system โ€” it is part of the data itself. Every field that can be created or modified by any DCM process carries provenance metadata alongside its value. + +Provenance answers: "where did this value come from, what modified it, and why?" + +Audit queries provenance to answer its questions. Observability does not use provenance directly โ€” it operates on event streams and metrics. + +### 2.2 Provenance Structure + +See [Context and Purpose โ€” Section 4.4](../context-and-purpose/) for the complete field-level provenance structure. The key elements: + +```yaml +field_name: + value: + metadata: + override: + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_uuid: + operation_type: + actor_uuid: + timestamp: + reason: +``` + +### 2.3 Provenance Obligations + +Every DCM component that modifies data carries a provenance obligation โ€” it must record its UUID, operation type, actor, timestamp, and reason for every field it touches. A component that modifies data without recording provenance violates the data model contract. + +| Component | Provenance Obligation | +|-----------|----------------------| +| Request Payload Processor | Record source UUID and type for every field assembled from layers | +| Policy Engine | Record policy UUID, level, operation type, and reason for every field modified or locked | +| Service Provider (Denaturalization) | Record provider UUID and timestamp for every field in the realized payload | +| Storage Provider | Emit provenance event to Audit component on every write | +| Resource Discovery | Record provider UUID, timestamp, and method for every discovered field | +| Rehydration Pipeline | Record source store, source record UUID, rehydration reason, and actor UUID | + +### 2.4 Provenance Across the Full Lifecycle + +The provenance chain for a single field may span multiple lifecycle stages: + +``` +Base Layer sets encryption_standard: AES-128 + origin: {source_type: base_layer, source_uuid: layer-uuid-001} + +Transformation Policy enriches to AES-256 + modification: {source_uuid: policy-uuid-001, operation: transformation, + reason: "Security standard requires AES-256 minimum"} + +GateKeeper Policy locks as immutable + modification: {source_uuid: policy-uuid-002, operation: lock, + reason: "CISO mandate โ€” encryption standard non-negotiable"} + +Provider reports realized value: AES-256 + modification: {source_uuid: provider-uuid-001, operation: denaturalization, + reason: "Provider confirmed encryption standard applied"} + +Drift detected: discovered value AES-128 + modification: {source_uuid: discovery-uuid-001, operation: discovery, + reason: "Direct modification detected outside DCM lifecycle"} +``` + +The complete chain tells the full story of that field across its entire existence. + +--- + +## 3. Audit + +### 3.1 Definition + +Audit is the compliance-grade, queryable record of all significant actions across the DCM lifecycle. It is backward-looking, human-readable, and access-controlled by persona. It answers: "what happened, who authorized it, can you prove it?" + +Audit is a **separate component** โ€” not a query against the GitOps stores, not a view into provenance directly. It aggregates and indexes provenance events from all stores and presents them through a structured query API surfaced by the DCM API Gateway. + +### 3.2 Architecture + +``` +All Storage Providers emit provenance events (contractual obligation) + โ”‚ + โ”‚ Events include: entity_uuid, operation, actor_uuid, + โ”‚ timestamp, payload_hash, store_reference + โ–ผ +Audit Component + โ”‚ Receives provenance events from all stores + โ”‚ Correlates events by entity_uuid across all stores + โ”‚ Indexes for structured query: by entity, tenant, actor, + โ”‚ time range, operation type, policy UUID + โ”‚ Maintains immutable records โ€” audit records are never modified + โ”‚ Enforces long retention (regulatory periods โ€” configurable, + โ”‚ minimum 7 years for FSI deployments) + โ”‚ Verifies payload hashes โ€” detects store tampering + โ–ผ +DCM API Gateway + โ”‚ Surfaces Audit query API with persona-based access control + โ”‚ Auditor: full access โ€” all entities, all tenants, all time + โ”‚ SRE: full access within operational scope + โ”‚ Admin: full access within administrative scope + โ”‚ Consumer: own entities and requests only + โ”‚ Provider: own provider's operations only +``` + +### 3.3 Audit API (via DCM API Gateway) + +``` +GET /api/v1/audit/entities/{uuid}/history + Returns: complete lifecycle history for an entity + Fields: all state transitions, all provenance events, all actor actions + +GET /api/v1/audit/requests/{uuid}/provenance + Returns: complete provenance chain for a specific request + Fields: intent, assembly, policy evaluation, provider dispatch, realization + +GET /api/v1/audit/policies/{uuid}/evaluations + Returns: all evaluations of a specific policy across all requests + Fields: when it ran, what it did, which entities it affected + +GET /api/v1/audit/actors/{uuid}/activity + Returns: all actions taken by a specific actor + Fields: requests submitted, approvals given, policy evaluations triggered + +GET /api/v1/audit/tenants/{uuid}/activity + Returns: all activity within a specific tenant + Fields: requests, realizations, drift events, policy violations + +POST /api/v1/audit/query + Body: structured audit query with field filters, time range, pagination + Returns: matching audit records +``` + +### 3.4 Audit Record Structure + +```yaml +audit_record: + audit_uuid: + entity_uuid: + tenant_uuid: + event_type: + timestamp: + actor_uuid: + actor_type: + + source_store: + store_type: + store_uuid: + store_reference: + payload_hash: + + provenance_summary: + + + policy_context: + policies_evaluated: [] + policies_applied: [] + policies_rejected: [] + override_control_changes: [] + + related_records: + intent_record_uuid: + requested_record_uuid: + realized_event_uuid: + rehydration_source_uuid: +``` + +### 3.5 Audit Integrity + +Audit records are immutable. The Audit component verifies payload hashes against the Storage Provider's stored values on every read โ€” if a hash mismatch is detected, the Audit component flags the record as potentially tampered and escalates to the Policy Engine. + +The Audit Store itself is a Storage Provider with the highest consistency and durability requirements โ€” linearizable consistency, synchronous replication, cryptographic payload hashing, and compliance-grade retention. + +--- + +## 4. Observability + +### 4.1 Definition + +Observability is real-time insight into the health, performance, and behavior of the DCM system. It is forward-looking, machine-readable, and aggregated. It answers: "is the system healthy, where are the bottlenecks, what is the error rate?" + +Observability is operationally oriented โ€” SREs and platform engineers use it to understand system behavior and respond to incidents. It does not carry the compliance obligations of Audit. + +### 4.2 The Three Pillars + +**Metrics** โ€” quantitative measurements of system state over time +- Request throughput: requests/second by resource type, tenant, provider +- Latency: assembly time, policy evaluation time, provider dispatch time, end-to-end time +- Error rates: policy rejection rate, provider failure rate, drift detection rate +- Capacity: provider utilization, store capacity, queue depth +- Cost: accumulated cost by tenant, resource type, provider + +**Traces** โ€” distributed traces of request execution across components +- Full request trace from Intent State creation through provider dispatch +- Policy evaluation trace โ€” which policies ran, in what order, how long each took +- Assembly trace โ€” which layers were applied, in what order, what each contributed + +**Logs** โ€” structured event logs from all DCM components +- Component startup and shutdown +- Registration events (provider registration, deregistration) +- Error conditions +- Drift detection events +- Unsanctioned change events + +### 4.3 Architecture + +``` +DCM components emit metrics, traces, and logs + โ”‚ + โ”‚ All telemetry in OpenTelemetry format + โ”‚ Standardized metric names, trace context propagation, + โ”‚ structured log format + โ–ผ +Observability Store + โ”‚ Time-series metrics store (Prometheus-compatible) + โ”‚ Distributed trace store (Jaeger/Zipkin compatible) + โ”‚ Log aggregation (structured, indexed) + โ”‚ Short-to-medium retention (configurable โ€” typically 90 days) + โ–ผ +DCM API Gateway + โ”‚ GET /api/v1/observability/metrics + โ”‚ GET /api/v1/observability/traces/{request_uuid} + โ”‚ GET /api/v1/observability/health + โ”‚ GET /api/v1/observability/providers/{uuid}/performance + โ–ผ +Dashboards and alerting (external tooling) + โ”‚ Grafana, DataDog, Splunk โ€” implementor choice + โ”‚ DCM provides OpenTelemetry-compatible telemetry + โ”‚ Dashboards are deployment artifacts, not DCM artifacts +``` + +### 4.4 Standard DCM Metrics + +``` +# Request lifecycle +dcm_requests_total{resource_type, tenant, status} +dcm_request_duration_seconds{resource_type, stage} +dcm_requests_in_flight{resource_type, tenant} + +# Policy Engine +dcm_policy_evaluations_total{policy_type, result} +dcm_policy_evaluation_duration_seconds{policy_type} +dcm_policy_rejections_total{policy_uuid, resource_type} + +# Provider +dcm_provider_requests_total{provider_uuid, resource_type, status} +dcm_provider_response_duration_seconds{provider_uuid} +dcm_provider_capacity_available{provider_uuid, resource_type} +dcm_provider_health_status{provider_uuid} + +# Drift +dcm_drift_detections_total{resource_type, severity} +dcm_unsanctioned_changes_total{resource_type, provider_uuid} +dcm_drift_resolution_duration_seconds{resolution_type} + +# Storage +dcm_store_write_duration_seconds{store_type, store_uuid} +dcm_store_read_duration_seconds{store_type, store_uuid} +dcm_store_health_status{store_type, store_uuid} + +# Rehydration +dcm_rehydrations_total{source_store, placement_mode, policy_version} +dcm_rehydration_duration_seconds{source_store} +``` + +### 4.5 Observability vs Audit โ€” The Key Distinctions + +| Dimension | Audit | Observability | +|-----------|-------|---------------| +| **Retention** | Regulatory period (years) | Operational window (days-months) | +| **Access control** | Strict persona-based | Operational teams | +| **Data volume** | Moderate โ€” per-entity events | High โ€” continuous time series | +| **Query model** | Structured, entity-centric | Aggregated, time-series | +| **Immutability** | Absolute โ€” records never modified | Aggregated data may be downsampled | +| **Compliance** | Compliance-grade โ€” hash-verified | Operational โ€” best effort | +| **Use case** | Prove what happened | Understand what is happening | + +--- + +## 5. The API Gateway โ€” Unified Access + +All three capabilities โ€” Provenance (embedded in data), Audit (structured history), and Observability (operational telemetry) โ€” are surfaced through the DCM API Gateway. There is no separate endpoint for audit or observability. All DCM capabilities live in a unified API hierarchy. + +``` +DCM API Gateway + โ”‚ + โ”œโ”€โ”€ /api/v1/catalog/ # Service Catalog + โ”œโ”€โ”€ /api/v1/requests/ # Request submission and management + โ”œโ”€โ”€ /api/v1/entities/ # Entity lifecycle management + โ”œโ”€โ”€ /api/v1/providers/ # Provider registration and management + โ”œโ”€โ”€ /api/v1/policies/ # Policy management + โ”œโ”€โ”€ /api/v1/audit/ # Audit queries + โ”œโ”€โ”€ /api/v1/observability/ # Operational metrics and traces + โ””โ”€โ”€ /api/v1/admin/ # Administrative functions +``` + +Persona-based access control is enforced at the API Gateway level for all endpoints. The same authentication and authorization model applies across the entire API surface. + +--- + +## 6. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the Audit Store be a specialized Storage Provider or can a general Event Stream Store satisfy the audit contract? | Architecture | โ“ Unresolved | +| 2 | How are audit records replicated across sites in air-gapped or geographically distributed deployments? | Sovereignty | โ“ Unresolved | +| 3 | Should DCM provide a default observability dashboard or only the telemetry? | Deployment | โ“ Unresolved | +| 4 | How does the Audit component handle provenance events from a Storage Provider that has been deregistered? | Operational | โ“ Unresolved | + +--- + +## 7. Related Concepts + +- **Provenance** โ€” field-level data lineage embedded in every DCM payload +- **Audit Store** โ€” compliance-grade, immutable store of all audit records +- **Observability Store** โ€” time-series metrics, traces, and logs +- **Storage Provider** โ€” formal provider type for all DCM stores +- **API Gateway** โ€” unified access point for all DCM capabilities including audit and observability +- **Drift Detection** โ€” uses discovered vs realized state comparison; drift events feed the Audit component +- **Unsanctioned Change** โ€” a specific audit event type triggered by unauthorized resource modification + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md new file mode 100644 index 0000000..8b10ffe --- /dev/null +++ b/content/docs/architecture/data-model/context-and-purpose.md @@ -0,0 +1,374 @@ +--- +title: "Context and Purpose" +type: docs +weight: 1 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Entity Types](01-entity-types.md) | [Four States](../four-states/) | [Layering and Versioning](../layering-and-versioning/) | [Examples](04-examples.md) + +--- + +## 1. Purpose + +The DCM Data Model is the foundational layer upon which the entire DCM framework operates. It is not a storage mechanism or a database schema โ€” it is the **lingua franca of DCM**. Every component in the DCM architecture communicates through the data model in some form, whether reading, writing, validating, enriching, transforming, or comparing data. + +The data model exists to solve a problem that is endemic to enterprise IT: **there is no single, trustworthy, consistent representation of infrastructure state**. Tools proliferate, CMDBs diverge, and the result is that no one knows with confidence what exists, what was requested, what was provisioned, or whether the current state matches the intended state. + +The DCM Data Model establishes a **unified, versioned, declarative single source of truth** for all infrastructure state across the full lifecycle of every resource DCM manages. + +--- + +## 2. Role in the DCM Architecture + +The data model is not owned by any single component โ€” it is the contract between all components. Every major DCM capability acts on data in a specific and well-defined way: + +| Component | Relationship to Data | +|-----------|---------------------| +| Request Payload Processor | Assembles and enriches data into a complete request payload | +| Policy Engine | Reads, validates, transforms, and gates data based on policy definitions | +| Service Provider | Consumes data (via Naturalization) and returns data (via Denaturalization) | +| Orchestration | Coordinates component interactions based on data state and dependencies | +| Audit | Records data at every state transition for compliance evidence | +| Drift Reconciliation | Compares versions of data across states to detect and remediate drift | +| Cost Analysis | Derives cost information from data throughout the resource lifecycle | +| Resource Discovery | Produces data representing the current discovered state of resources | +| IDM / IAM | Gates access to data and operations based on identity and role | +| Service Catalog | Exposes available services based on data definitions and RBAC policy | + +This means the data model is effectively the **API between all DCM components** โ€” even components that do not communicate directly are coupled through the data model. A well-designed data model makes every component easier to build, test, and evolve independently. + +--- + +## 3. Universal Identity Requirement + +Every data object in DCM must have a **UUID (Universally Unique Identifier)**. This is not optional โ€” it is a foundational requirement that applies to every entity in the data model without exception. + +UUIDs serve several critical functions: + +- **Unambiguous reference** โ€” any component, policy, layer, catalog item, or process that touches a data object can reference it precisely and without ambiguity +- **Provenance anchoring** โ€” every change recorded in a data object's lineage references the UUID of the entity that caused the change +- **Dependency mapping** โ€” relationships between resources, services, and components are expressed as UUID references, never by name alone +- **Audit fidelity** โ€” audit records reference UUIDs, ensuring that even if names or labels change, the audit trail remains accurate and traceable +- **Cross-state correlation** โ€” the same resource across Intent, Requested, Realized, and Discovered states can be correlated via UUID chains + +This applies to all entities including but not limited to: resource definitions, catalog items, data layers, policies, policy sets, components, service providers, consumers, and requests. + +--- + +## 4. Field-Level Provenance and Data Lineage + +One of the most critical requirements of the DCM Data Model is the ability to trace the complete lineage of any piece of data at any stage of the pipeline. This is not a logging concern โ€” it is a **structural requirement of the data model itself**. + +### 4.1 The Requirement + +At any point in the DCM pipeline, for any field in any data object, it must be possible to answer: + +- What is the current value of this field? +- Where did this value originate? (catalog item, base layer, intermediate layer, policy, consumer input, discovery) +- Has this value been modified since origination? +- If modified: + - What is the complete history of modifications? + - Which entity caused each modification? (identified by UUID) + - What type of entity caused it? (policy, layer, component, provider) + - When did each modification occur? + - What was the value before each modification? + - Why was the modification made? (enrichment, validation, transformation, gatekeeping) +- What is the complete chain of custody of this field from origin to current value? + +### 4.2 Why Field-Level Lineage Matters + +Document-level versioning alone is insufficient for DCM's requirements. Consider a resource request flowing through the pipeline: + +1. Consumer selects a catalog item โ€” catalog item UUID recorded +2. Base resource definition layer applied โ€” base layer UUID recorded, fields established +3. Intermediate layers applied โ€” each layer UUID recorded, field overrides recorded +4. Policy Engine validates โ€” policy UUID recorded, validation outcome recorded +5. Policy Engine enriches โ€” policy UUID recorded, enriched field values and their source recorded +6. Policy Engine transforms โ€” policy UUID recorded, transformation recorded with before/after values +7. Request payload submitted โ€” complete provenance chain intact across all fields + +Without field-level provenance, it is impossible to determine after the fact whether a specific field value came from a consumer request, a data layer, a security policy, or a business rule. This ambiguity is unacceptable in a governed, auditable system. + +### 4.3 Provenance as a Structural Element + +Field-level provenance must be carried within the data object itself โ€” not in an external log. This ensures that: + +- The data and its lineage are always co-located and cannot be separated +- Any consumer of the data can inspect its lineage without querying an external system +- Provenance survives data export, migration, and portability scenarios +- The audit capability reads provenance that is intrinsic to the data, not reconstructed from logs + +### 4.4 Provenance Metadata Structure + +Every field that can be created or modified by any DCM process carries provenance metadata alongside its value. The conceptual structure is: + +```yaml +field_name: + value: + metadata: + # Simple override declaration (Level 2) โ€” most fields only need this + override: + # OR matrix declaration (Level 3) โ€” for actor-specific governance + override_matrix: + default: + actors: + trusted_grants: + + # Always present regardless of level + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + constraint_schema: + + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_type: + source_uuid: + operation_type: + actor: + timestamp: + reason: +``` + +**Note:** The `metadata` block is set exclusively by the Policy Engine. Data layers and the Request Payload Processor never set override control. `operation_type: lock` is used when a GateKeeper policy sets `override: immutable`. `operation_type: grant` is used when a trusted_grant is issued. The three levels of override control are: Level 1 (no declaration โ€” fully overridable), Level 2 (simple `override:` property), Level 3 (full `override_matrix:` with per-actor permissions). See the Layering and Assembly document Section 5a for the complete model. + +### 4.5 Provenance Obligations + +Every DCM component that reads and modifies data carries a provenance obligation: + +| Component | Provenance Obligation | +|-----------|----------------------| +| Request Payload Processor | Record source UUID and type for every field assembled from layers and catalog items | +| Policy Engine | Record policy UUID, operation type, and reason for every field it enriches, transforms, or validates | +| Service Provider (Denaturalization) | Record provider UUID and timestamp for every field returned in the realized payload | +| Resource Discovery | Record provider UUID, discovery timestamp, and interrogation method for every field in the discovered payload | +| Data Layers | Each layer must declare its UUID so downstream provenance records can reference it | +| Catalog Items | Each catalog item must declare its UUID so downstream provenance records can reference it | + +Provenance recording is **not optional** for any component that modifies data. A component that modifies data without recording provenance violates the data model contract. + +### 4.6 Relationship to Audit + +The Audit capability in DCM reads provenance data that is intrinsic to every data object. This means: + +- Audit does not reconstruct history from logs โ€” it reads lineage that was recorded at the point of change +- Any data object can be audited at any time, in any state, by any authorized persona +- The audit trail is as durable and immutable as the data itself +- Compliance evidence is produced from the data, not from a separate audit system that could diverge from the data + +--- + +## 5. Foundational Constraints + +The DCM Data Model is governed by three foundational constraints that apply universally and without exception: + +### 3.1 Declarative + +Data in DCM describes **what something is or should be**, not how to achieve it. Every entity in the data model is a complete, self-describing statement of state. The procedures required to achieve that state are the concern of the Service Provider, not the data model. + +This means: +- A resource definition declares its desired configuration, not the steps to configure it +- A policy declares its conditions and outcomes, not its execution logic +- A layer declares its overrides, not the merge algorithm used to apply them + +### 3.2 Idempotent in Operation + +Applying the same data to the same system multiple times must always produce the same result. No operation on DCM data should have different outcomes based on how many times it has been applied. + +This is critical for: +- **Drift reconciliation** โ€” reapplying desired state to a drifted resource must produce correct results +- **DC rehydration** โ€” replaying the full set of declared states must reconstruct the environment correctly +- **Retry scenarios** โ€” failed operations can be safely retried without risk of inconsistent state +- **Audit and compliance** โ€” the same data, applied by anyone at any time, produces the same verifiable outcome + +### 3.3 Immutable if Versioned + +Once a version of any entity is published, it cannot be modified. If a change is required, a new version must be created. The previous version remains intact and accessible. + +This constraint is what makes the following capabilities trustworthy: +- **Audit trails** โ€” every state at every point in time is preserved and verifiable +- **Drift detection** โ€” comparison between states is meaningful because neither state can change retroactively +- **Intent portability** โ€” a previously declared intent can be replayed against current policies with confidence that the original intent is unchanged +- **Rollback** โ€” reverting to a previous version is always possible because previous versions are never destroyed +- **Chain of trust** โ€” the provenance of any configuration can be traced through an unbroken chain of immutable versions + +--- + +## 5a. Artifact Metadata Standard + +Every DCM artifact โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions, and all other defined or stored objects โ€” carries a universal **Artifact Metadata** block. This is a structural requirement that applies to all artifacts without exception. + +### The Purpose + +Artifact metadata answers: **who created this, when, who owns it, what changed, and how do we contact them?** It is the identity and accountability record for the artifact itself โ€” distinct from field-level provenance which tracks data value lineage. + +### The Five Artifact Statuses + +All DCM artifacts follow a five-status lifecycle: + +| Status | Meaning | Key Behavior | +|--------|---------|-------------| +| `developing` | In active development | Dev mode / dev pipeline only. Not applied in production. | +| `proposed` | Submitted for validation | Shadow execution for policies โ€” output captured, not applied. In PR review for data artifacts. | +| `active` | Live and governing | Applied to all relevant requests. | +| `deprecated` | Being phased out | Still works, replacement available, warning on use. | +| `retired` | End of life | Cannot be used. Terminal status. | + +### Key Design Decisions + +**created_by vs owned_by:** Deliberately separate. The creator is the audit record โ€” who physically submitted the artifact. The owner is the accountability record โ€” who is responsible and receives notifications for conflicts, deprecation warnings, and policy violations. + +**Contact info โ€” two modes:** When an Identity Provider is registered, the `uuid` field links to the IdP record and `display_name` is a non-authoritative display cache. In standalone/air-gapped mode, `uuid` is absent and `display_name` + `email` are the primary identity fields. Both modes are fully supported. + +**created_via:** Declares the ingestion path โ€” `pr` (full GitOps review history), `api` (direct submission), `migration` (imported, limited provenance), `system` (DCM-created). Makes audit quality transparent. + +**Proposed shadow execution:** Policy artifacts in `proposed` status execute in shadow mode against real traffic โ€” output is captured and reported but never applied. Enables safe validation before activation. + +See [Data Layers and Assembly โ€” Section 4b](../layering-and-versioning/) for the complete artifact metadata structure and all field definitions. + +--- + +## 6. The Four States + +DCM tracks the lifecycle of every resource through four distinct states. Together, these four states provide complete visibility into the gap between what was wanted, what was asked for, what was built, and what actually exists. + +### 6.1 Intent State + +The **Intent State** represents what a consumer wants to happen. It is the declared desire, captured at the moment a consumer initiates a request, before any processing, validation, or enrichment has occurred. + +- **When it is created:** When a consumer submits a request via the Web UI or Consumer API +- **Where it is stored:** Intent Store +- **Key characteristic:** Captures the consumer's raw intent โ€” what they asked for in their own terms +- **Primary use:** Source for Intent Portability โ€” replaying an intent through current policies to produce a new request for a different environment or provider + +### 6.2 Requested State + +The **Requested State** represents the fully processed, policy-validated, and enriched payload that has been submitted to a Service Provider for execution. It is the output of the Request Payload Processor after all policies have been applied and all data layers have been merged. + +- **When it is created:** When the Request Payload Processor completes processing and submits to the API Gateway +- **Where it is stored:** Request Store +- **Key characteristic:** Represents a complete, validated, provider-ready declaration of desired state +- **Primary use:** Record of what was formally requested; input to audit and drift processes + +### 6.3 Realized State + +The **Realized State** represents what was actually provisioned or executed by a Service Provider, returned to DCM in unified data model format via Denaturalization. It is the ground truth of what was built. + +- **When it is created:** When a Service Provider completes execution and returns the realized payload to the API Gateway +- **Where it is stored:** Realized Store +- **Key characteristic:** Must be a complete representation of the provisioned resource in DCM unified format โ€” not a status code, but a full state description +- **Primary use:** Baseline for drift detection; source of truth for audit and reporting; input to cost analysis + +### 6.4 Discovered State + +The **Discovered State** represents what actually exists in the environment as interrogated by a Service Provider during a discovery operation. It is an independent observation of reality, not derived from any previous DCM state. + +- **When it is created:** When a Service Provider completes a discovery cycle and returns the discovered payload to DCM +- **Where it is stored:** Discovered Store +- **Key characteristic:** Produced independently of the Realized State โ€” it is what is actually there, regardless of what was supposed to be there +- **Primary use:** Drift detection (compared against Realized State); brownfield ingestion (pathway to lifecycle ownership of unmanaged resources) + +### 6.5 State Relationships and Lifecycle Flow + +The four states relate to each other as follows: + +``` +Consumer Request + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ INTENT โ”‚ โ—„โ”€โ”€ What the consumer wants +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Policy Engine processes, enriches, validates + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REQUESTED โ”‚ โ—„โ”€โ”€ What was formally submitted to the provider +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Service Provider executes + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REALIZED โ”‚ โ—„โ”€โ”€ What was actually built +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ Compare โ—„โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”‚ DISCOVERED โ”‚ โ—„โ”€โ”€ What actually exists now + โ”‚ โ”‚ STATE โ”‚ + โ–ผ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + Drift Detection +``` + +**Key operations across states:** +- **Drift Detection:** Discovered State vs. Realized State +- **Request Validation:** Requested State vs. Policy definitions +- **Intent Portability:** Intent State โ†’ re-process through current policies โ†’ new Requested State +- **Brownfield Ingestion:** Discovered State โ†’ enrichment โ†’ Realized State (lifecycle ownership) + +--- + +## 7. Data as the Provider Contract Boundary + +The data model defines the boundary between DCM and its Service Providers. DCM is explicitly **not concerned with how a provider accomplishes its work** โ€” only with the data that crosses the boundary in both directions. + +This means: +- Providers are interchangeable as long as they honor the data contract +- New providers can be added without changing DCM's core data model +- Provider implementation can evolve independently of DCM +- The contract is enforced at the data level โ€” conformant data in, conformant data out + +The two mechanisms that enforce this boundary are: +- **Naturalization** โ€” the provider's responsibility to transform DCM unified data into its own tool-specific format for execution +- **Denaturalization** โ€” the provider's responsibility to transform its tool-specific result data back into DCM unified format for return to the control plane + +This separation of concerns is what makes DCM technology-agnostic while maintaining a consistent and trustworthy data model across all providers. + +--- + +## 8. Open Questions + +The following questions remain unresolved and require decisions before the data model specification can be considered complete: + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Where should data caches live? (Shore, Ship, Enclave, all?) | Cache architecture, latency, sovereignty | โ“ Unresolved | +| 2 | Should cache synchronization be push, pull, or both? | Consistency model, network requirements | โ“ Unresolved | +| 3 | Which cache is authoritative when caches diverge? | Conflict resolution, drift detection | โ“ Unresolved | +| 4 | What mechanism maintains consistency across distributed caches? | Data integrity, operational complexity | โ“ Unresolved | +| 5 | Should the data model allow embedded target-technology-specific data bundles? | Portability vs. practicality | โ“ Unresolved | +| 6 | How are the four states represented physically? (fields, documents, separate stores?) | Implementation architecture | โ“ Unresolved | +| 7 | What is the performance impact of field-level provenance at scale? What optimization strategies are acceptable? | Scalability, storage cost | โ“ Unresolved | +| 8 | Should provenance metadata be stored inline with field data or in a linked provenance document? | Data model structure, query performance | โ“ Unresolved | + +--- + +## 9. Related Concepts + +- **Sovereign Execution Posture** โ€” the target end state the data model enables by providing a verified, auditable chain of custody through the full resource lifecycle +- **CMDB Replacement** โ€” DCM's four-state model is intended to replace the fragmented multi-CMDB problem by becoming the singular resource domain +- **GitOps** โ€” all entities in the data model are stored in Git, enabling version control, change tracking, and standard software lifecycle practices +- **Data Lineage** โ€” the complete chain of custody of any field value from its origin through every modification, recorded within the data object itself +- **Field-Level Provenance** โ€” the structural mechanism by which data lineage is captured, carried, and made available for audit and compliance purposes +- **UUID** โ€” the universal identity mechanism that makes provenance references, dependency mapping, and cross-state correlation unambiguous and durable + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md new file mode 100644 index 0000000..db25dd1 --- /dev/null +++ b/content/docs/architecture/data-model/entity-relationships.md @@ -0,0 +1,472 @@ +--- +title: "Entity Relationships" +type: docs +weight: 8 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Resource Type Hierarchy](../resource-type-hierarchy/) | [Resource/Service Entities](../resource-service-entities/) | [Service Dependencies](../service-dependencies/) | [Resource Grouping](../resource-grouping/) | [Information Providers](../information-providers/) + +--- + +## 1. Purpose + +The DCM Entity Relationship model is the **universal mechanism for expressing relationships between any two entities in DCM** โ€” whether between two Resource/Service Entities, between an entity and external business data, or between entities at the service definition level. + +A single relationship model is used everywhere. There is no separate binding mechanism for storage, no separate dependency graph structure, no separate business data association mechanism. One model serves all relationship types across the full lifecycle โ€” from pre-realization planning through to post-realization management, drift detection, cost rollup, and rehydration. + +This document supersedes the dependency graph concept from the Service Dependencies document for data structure purposes. The Service Dependencies document retains content on rehydration ordering and failure handling, which operate on the relationship graph defined here. + +--- + +## 2. Design Principle + +**Single model. Minimum variance. Simple by default.** + +The worst outcome is a data model with different mechanisms for expressing similar concepts. Every relationship in DCM โ€” whether a VM requires storage, an application contains a web server, or a resource references a Business Unit โ€” is expressed using the same structure. The only things that vary are the relationship type, role, and nature โ€” all of which are declared fields, not structural differences. + +--- + +## 3. The Universal Relationship Structure + +Every relationship is a first-class data object with its own UUID. It is recorded **bidirectionally** โ€” on both participating entities. The same `relationship_uuid` appears on both sides, identifying the relationship itself. + +### 3.1 Relationship Record Structure + +```yaml +relationship: + relationship_uuid: + + # This entity's perspective + this_entity_uuid: + this_role: + + # The related entity + related_entity_uuid: + related_entity_type: + related_entity_role: + + # For external entities only + information_provider_uuid: + information_type: + lookup_method: + + # Relationship semantics + relationship_type: + nature: + + # Lifecycle policy โ€” for constituent and operational relationships only + lifecycle_policy: + on_related_destroy: + on_related_suspend: + on_related_modify: + + # Metadata + version: + status: + created_timestamp: + created_by_uuid: + + provenance: + +``` + +### 3.2 Bidirectional Recording + +Every relationship is recorded on both participating entities. The `relationship_uuid` is identical on both sides โ€” it identifies the relationship itself, not one side of it. + +**Example โ€” VM requires Storage:** + +```yaml +# On the VM Entity +relationships: + - relationship_uuid: "rel-uuid-001" + this_entity_uuid: "vm-uuid-001" + this_role: compute + related_entity_uuid: "storage-uuid-001" + related_entity_type: internal + related_entity_role: storage + relationship_type: requires + nature: constituent + lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + on_related_modify: notify + +# On the Storage Entity +relationships: + - relationship_uuid: "rel-uuid-001" + this_entity_uuid: "storage-uuid-001" + this_role: storage + related_entity_uuid: "vm-uuid-001" + related_entity_type: internal + related_entity_role: compute + relationship_type: required_by + nature: constituent + lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + on_related_modify: notify +``` + +--- + +## 4. Relationship Types + +Relationship types form a fixed standard vocabulary. Every type has an inverse โ€” when you record the relationship on both entities, the type is expressed from each entity's perspective. + +| Type | Inverse | Meaning | +|------|---------|---------| +| `requires` | `required_by` | This entity cannot function without the related entity | +| `depends_on` | `dependency_of` | This entity uses the related entity but can degrade without it | +| `contains` | `contained_by` | This entity is a logical container for the related entity | +| `references` | `referenced_by` | This entity references the related entity without owning or requiring it | +| `peer` | `peer` | Equal relationship โ€” neither owns, requires, or contains the other | +| `manages` | `managed_by` | This entity has lifecycle management authority over the related entity | + +--- + +## 5. Relationship Roles + +Roles describe the **function** a related entity serves in a relationship. They are semantic labels that carry meaning for humans and for policy evaluation โ€” they do not affect system behavior directly. + +### 5.1 Standard Roles (DCM-defined) + +| Role | Description | +|------|-------------| +| `compute` | Processing resource โ€” VM, container, bare metal | +| `storage` | Storage resource โ€” block, object, file | +| `networking` | Network resource โ€” IP, VLAN, subnet, port | +| `security` | Security resource โ€” firewall rule, certificate, HSM | +| `database` | Database resource โ€” relational, NoSQL, time-series | +| `web` | Web tier resource โ€” web server, reverse proxy, CDN | +| `app` | Application tier resource โ€” app server, runtime | +| `cache` | Caching resource โ€” in-memory cache, CDN layer | +| `queue` | Messaging resource โ€” message queue, event stream | +| `pipeline` | Pipeline resource โ€” CI/CD, data pipeline | +| `identity` | Identity resource โ€” service account, credential | +| `monitoring` | Monitoring resource โ€” metrics, logging, alerting | +| `business_unit` | Business Unit association | +| `cost_center` | Cost Center association | +| `product_owner` | Product Owner association | +| `regulatory_scope` | Regulatory or compliance scope association | + +### 5.2 Custom Roles (extensible) + +Organizations register custom roles for domain-specific relationship semantics. Custom roles are semantic labels only โ€” they do not change system behavior. DCM core ignores unknown custom roles in operational decisions but carries them in payloads for downstream consumers. + +```yaml +custom_role_registration: + uuid: + name: + description: + registered_by_tenant_uuid: + category: + version: + status: +``` + +--- + +## 6. Relationship Nature + +Nature describes the **structural character** of a relationship โ€” what it means for the entities involved. + +| Nature | Meaning | Lifecycle Policy | Example | +|--------|---------|-----------------|---------| +| `constituent` | The related entity is a required component of this entity's definition | Required โ€” declared on relationship | VM requires its boot disk | +| `operational` | The related entity is needed for operation but is not part of the definition | Required โ€” declared on relationship | Web server depends on load balancer | +| `informational` | The related entity provides context or reference only โ€” no operational dependency | Not applicable | Resource references its Business Unit | + +--- + +## 7. Lifecycle Policies + +Lifecycle policies declare what happens to an entity when its related entity changes state. They apply to `constituent` and `operational` relationships only โ€” `informational` relationships have no lifecycle implications. + +### 7.1 Policy Actions + +| Action | Meaning | +|--------|---------| +| `destroy` | Destroy this entity when the related entity is destroyed | +| `retain` | Keep this entity when the related entity is destroyed โ€” it becomes independent | +| `detach` | Detach this entity from the relationship โ€” relationship terminated, entity retained | +| `notify` | Notify appropriate personas and trigger Policy Engine evaluation โ€” no automatic action | +| `suspend` | Suspend this entity when the related entity is suspended | +| `cascade` | Cascade the change from the related entity to this entity | +| `ignore` | Take no action โ€” the change to the related entity does not affect this entity | + +### 7.2 Lifecycle Policy Authority Hierarchy + +Lifecycle policies follow the same three-tier authority model as override control: + +``` +Resource Type Specification default (lowest โ€” portable default) + โ”‚ + โ–ผ +Provider Catalog Item default (provider preference) + โ”‚ + โ–ผ +Consumer declaration (at request time โ€” within Resource Type bounds) + โ”‚ + โ–ผ +DCM System Policy (non-overridable โ€” sovereignty and compliance mandates) +``` + +**Example:** A DCM System Policy might declare that all storage entities in a PCI-DSS scope must `retain` when their parent VM is destroyed โ€” regardless of what the provider default or consumer declared. + +--- + +## 8. Relationship Declarations โ€” Where They Live + +Relationship declarations exist at multiple levels, each building on the previous: + +### 8.1 Resource Type Specification (structural ceiling) + +Declares what relationships are **possible** for a resource type. Sets the ceiling โ€” lower levels can only declare relationships within these bounds. + +```yaml +resource_type: Compute.VirtualMachine +possible_relationships: + - role: storage + relationship_type: requires + nature: constituent + permitted_related_types: + - Storage.Block + - Storage.File + default_lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + binding_types_permitted: [owned, referenced] + consumer_declarable: true + # Consumer can declare binding_type and lifecycle_policy override + + - role: networking + relationship_type: requires + nature: constituent + permitted_related_types: + - Network.IPAddress + default_lifecycle_policy: + on_related_destroy: destroy + consumer_declarable: false + # DCM manages this automatically โ€” consumer cannot override +``` + +### 8.2 Catalog Item (offering-specific) + +Declares the **actual relationships** for a specific curated offering. Can only be more restrictive than the Resource Type Specification. + +```yaml +catalog_item: Production VM +relationships: + - role: storage + relationship_type: requires + nature: constituent + related_catalog_item_uuid: + lifecycle_policy: + on_related_destroy: retain + # Overrides Resource Type default of destroy + # Storage persists even if VM is destroyed โ€” production data protection + binding_type: owned +``` + +### 8.3 Request Time (consumer-declared) + +The consumer declares relationships in their request. Bundled declarations (storage fields within a VM request) are automatically expanded into relationship records by the Request Payload Processor. + +```yaml +# Explicit relationship declaration in a request +request: + resource_type: Compute.VirtualMachine + # ... other fields ... + relationships: + - role: storage + relationship_type: requires + binding_type: referenced + related_entity_uuid: + # Consumer referencing existing storage โ€” not creating new + +# Bundled declaration โ€” expanded automatically +request: + resource_type: Compute.VirtualMachine + storage: + disks: + - name: boot + capacity: 100GB + # Processor expands this into a Storage Entity stub + # and a relationship record with binding_type: owned +``` + +### 8.4 External Data Relationships + +Relationships to external data entities follow the same structure with `related_entity_type: external`: + +```yaml +# On a VM Entity โ€” relationship to external Business Unit +relationships: + - relationship_uuid: + this_entity_uuid: + this_role: + related_entity_uuid: + related_entity_type: external + information_provider_uuid: + information_type: Business.BusinessUnit + relationship_type: references + role: business_unit + nature: informational + lookup_method: primary_key +``` + +--- + +## 9. Bundled Declaration Expansion + +When a consumer includes resource configuration as bundled fields (e.g., storage within a VM request), the Request Payload Processor expands these into first-class entities and relationship records. + +### 9.1 Expansion Process + +``` +Consumer submits bundled VM request with storage fields + โ”‚ + โ–ผ +Request Payload Processor + โ”‚ Reads expansion rules from Resource Type Specification + โ”‚ For each expandable field: + โ”‚ 1. Creates a Resource/Service Entity stub (PENDING state) + โ”‚ with its own UUID, Tenant membership, Resource Type + โ”‚ 2. Creates a Relationship record on both the parent stub + โ”‚ and the child stub + โ”‚ 3. Applies lifecycle policy from: + โ”‚ consumer declaration โ†’ provider default โ†’ Resource Type default + โ”‚ โ†’ DCM System Policy override + โ”‚ 4. Adds the child entity stub to the relationship graph + โ–ผ +Policy Engine validates: + โ”‚ Binding type is permitted by Resource Type Specification + โ”‚ Consumer has override_matrix permission to declare binding type + โ”‚ Lifecycle policy is not overridden by a DCM System Policy + โ–ผ +Service Provider receives: + โ”‚ Parent entity request payload + โ”‚ Child entity stub UUIDs embedded in parent payload + โ”‚ Provisions resources natively + โ”‚ Returns realized payloads for all entities in DCM unified format + โ–ผ +DCM updates: + โ”‚ Parent entity: PENDING โ†’ REALIZED + โ”‚ Child entities: PENDING โ†’ REALIZED + โ”‚ All relationship records: status โ†’ active + โ”‚ Full provenance recorded on all entities and relationships +``` + +### 9.2 Expansion Rules in Resource Type Specification + +The expansion rule declares which fields expand into entities and how: + +```yaml +field_definition: + field_name: storage + type: object + expansion: + expand_to_entity: true + entity_resource_type_uuid: + entity_resource_type_name: Storage.Block + default_binding_type: owned + binding_types_permitted: [owned, referenced] + default_lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + consumer_can_override_lifecycle: true + consumer_can_override_binding_type: true +``` + +--- + +## 10. The Entity Relationship Graph + +All relationships across all entities form a traversable **Entity Relationship Graph** โ€” the complete map of how all entities in DCM relate to each other. + +### 10.1 Graph Properties + +- Every node is a Resource/Service Entity (internal or external reference) +- Every edge is a Relationship with a UUID +- The graph is bidirectional โ€” traversable from any node in any direction +- Every node exists exactly once โ€” shared entities appear once with multiple relationship edges +- Circular relationships are invalid and must be rejected + +### 10.2 Graph and the Four States + +The relationship graph exists across all four states: + +| State | Graph Role | +|-------|-----------| +| Intent State | Graph declared at request time โ€” nodes are intent stubs | +| Requested State | Graph fully assembled โ€” nodes are PENDING entity stubs with UUIDs | +| Realized State | Graph populated โ€” nodes are REALIZED entities with full provenance | +| Discovered State | Graph used for comparison โ€” discovered entities matched against realized graph | + +### 10.3 Graph Applications + +| Application | How the Graph is Used | +|-------------|----------------------| +| **Rehydration** | Full graph traversal from a root entity โ€” all related entities identified and realized in dependency order | +| **Cost Rollup** | Graph traversal accumulates costs across all related constituent entities | +| **Drift Detection** | Discovered State graph compared against Realized State graph โ€” structural and data differences identified | +| **Decommission** | Graph traversal determines decommission order โ€” lifecycle policies applied at each edge | +| **Placement** | Pre-realization graph used to understand full resource footprint for placement decisions | +| **Impact Analysis** | Graph traversal from any node identifies all entities affected by a change | + +--- + +## 11. Relationship Integrity + +### 11.1 DCM System Policies for Relationships + +| Policy | Rule | +|--------|------| +| `REL-001` | Every relationship must have a UUID | +| `REL-002` | Every relationship must be recorded on both participating entities | +| `REL-003` | Circular relationships are invalid and must be rejected | +| `REL-004` | A constituent relationship must declare a lifecycle policy | +| `REL-005` | External relationships must reference a registered Information Provider | +| `REL-006` | Relationship types must be from the standard vocabulary | +| `REL-007` | Consumer-declared binding types must be permitted by the Resource Type Specification | + +### 11.2 Relationship Versioning and Deprecation + +Relationships follow the universal versioning and deprecation model. A relationship version changes when its lifecycle policy, nature, or role changes. Terminated relationships are retained in provenance permanently. + +--- + +## 12. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Policy model | โ“ Unresolved | +| 2 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Operational complexity | โ“ Unresolved | +| 3 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Multi-tenancy | โ“ Unresolved | +| 4 | Should there be a maximum relationship graph depth to prevent runaway complexity? | Operational governance | โ“ Unresolved | +| 5 | How are shared entities represented in the relationship graph โ€” an entity required by multiple parents? | Graph model | โ“ Unresolved | + +--- + +## 13. Related Concepts + +- **Entity Relationship Graph** โ€” the complete traversable graph of all entity relationships in DCM +- **Information Provider** โ€” provider type for external data entities referenced in relationships +- **Bundled Declaration Expansion** โ€” processor mechanism for expanding bundled fields into entities and relationships +- **Lifecycle Policy** โ€” declares what happens to an entity when its related entity changes state +- **Service Dependencies** โ€” document covering rehydration ordering and failure handling on the relationship graph +- **Resource Type Specification** โ€” declares possible relationships for a resource type +- **External Entity Reference** โ€” stable pointer to data owned by an external system + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md new file mode 100644 index 0000000..c993e3f --- /dev/null +++ b/content/docs/architecture/data-model/four-states.md @@ -0,0 +1,449 @@ +--- +title: "Four States" +type: docs +weight: 2 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Entity Relationships](../entity-relationships/) | [Storage Providers](../storage-providers/) | [Audit, Provenance, and Observability](../audit-provenance-observability/) + +--- + +## 1. Purpose + +The four states are the foundational model for how DCM tracks the complete lifecycle of any resource or service. Every entity in DCM exists in one or more of these states simultaneously. The states are not sequential stages โ€” they are parallel, independently maintained records that together provide a complete, auditable picture of what was requested, what was approved, what was built, and what actually exists. + +The four states answer four distinct questions: + +| State | Question Answered | Store Type | +|-------|------------------|------------| +| **Intent State** | What did the consumer ask for? | GitOps Store | +| **Requested State** | What was approved and dispatched to the provider? | GitOps Store | +| **Realized State** | What did the provider actually build? | Event Stream Store | +| **Discovered State** | What does DCM observe actually existing right now? | Event Stream Store (ephemeral) | + +--- + +## 2. State Definitions + +### 2.1 Intent State + +The **Intent State** is the immutable record of a consumer's original declaration. It is captured at the moment a request is submitted โ€” before any layer assembly, before any policy evaluation, before any provider selection. + +**Characteristics:** +- Immutable once created โ€” the consumer's original intent is never modified +- Stored in a GitOps store โ€” branched, reviewed, merged +- The CI/CD pipeline operates on the Intent State โ€” policy pre-validation, cost estimation, sovereignty check, approval workflow +- Versioned via Git history โ€” every revision of an intent is traceable +- Supports human review and debate via the PR mechanism +- The entity UUID is assigned at Intent State creation โ€” it follows the entity through all subsequent states + +**When created:** Every request submission, every rehydration operation, every drift remediation authorization + +**Content:** The consumer's raw declaration in DCM Unified Data Model format โ€” what they want, not what will be built + +### 2.2 Requested State + +The **Requested State** is the fully assembled, policy-processed, provider-ready payload. It is produced by the Request Payload Processor from the Intent State โ€” after layer assembly, after all policy evaluation, after provider selection. + +**Characteristics:** +- Immutable once created โ€” a new Requested State is created for each request cycle +- Stored in a GitOps store โ€” committed, versioned, triggering CD pipeline +- The CD pipeline dispatches from the Requested State to the provider +- Contains the complete assembled payload with full field-level provenance +- Contains the results of all policy evaluations โ€” which policies ran, what they did, what they locked +- Contains provider selection โ€” which provider will realize this request +- Is the authoritative record of what DCM instructed a provider to build + +**When created:** After Intent State approval (merge), after successful policy processing + +**Content:** The complete assembled payload in DCM Unified Data Model format, with full provenance chain, policy evaluation results, provider selection, and override control metadata + +### 2.3 Realized State + +The **Realized State** is the provider-confirmed record of what was actually built. It is produced by the provider after successful realization โ€” the denaturalized result of the provider's execution, translated back to DCM Unified Data Model format. + +**Characteristics:** +- Append-only event stream โ€” each state change is a new event, never an overwrite +- Stored in an Event Stream Store โ€” high-frequency writes, entity-keyed streams +- The entity UUID is the stream key โ€” all realized state events for an entity share the same stream +- Contains provider-specific details not in the Requested State โ€” assigned IPs, generated passwords, actual storage sizes, provider-internal IDs +- Is the authoritative record of what actually exists from DCM's perspective +- Drift is detected by comparing Realized State against Discovered State + +**When created:** After provider confirms realization, updated on every provider lifecycle event + +**Content:** The realized entity in DCM Unified Data Model format, with provider-added fields, full field-level provenance including provider attribution + +### 2.4 Discovered State + +The **Discovered State** is what DCM observes actually existing through active discovery โ€” polling providers, querying Kubernetes APIs, interrogating infrastructure. It is the ground truth of what physically exists, independent of what DCM thinks exists. + +**Characteristics:** +- Append-only snapshot stream โ€” each discovery cycle produces a new snapshot +- Stored in an Event Stream Store (ephemeral) โ€” recent history retained, older snapshots archived or discarded +- High-frequency and machine-generated โ€” not appropriate for human review +- Used exclusively for drift detection โ€” comparing against Realized State +- May contain resources DCM did not provision โ€” brownfield resources discovered for ingestion + +**When created:** On every discovery cycle, on demand for specific entities + +**Content:** Raw discovered resource state in DCM Unified Data Model format, with discovery metadata (timestamp, discovery method, provider interrogated) + +--- + +## 3. The Entity UUID โ€” Universal Linking Key + +Every entity has a single UUID assigned at Intent State creation. This UUID is the universal key linking the entity across all four states and all stores: + +``` +Intent Store: file path includes entity_uuid, content declares entity_uuid +Requested Store: file path includes entity_uuid, content declares entity_uuid +Realized Store: event stream keyed by entity_uuid +Discovered Store: snapshot stream keyed by entity_uuid (matched via provider labels) +Audit Store: all provenance events indexed by entity_uuid +Search Index: entity_uuid โ†’ git_path mapping for Git stores +``` + +Given an entity UUID, DCM can reconstruct the complete history of that entity across its entire lifecycle โ€” from the consumer's original intent through every state transition to the current discovered state. + +--- + +## 4. Physical Representation โ€” Storage Provider Model + +DCM describes store **contracts**, not implementations. Each store is a Storage Provider โ€” a formal DCM provider type with registration, health check, and trust obligations. Implementors choose the technology that satisfies the contract. + +See [Storage Providers](../storage-providers/) for the complete contract specifications. + +### 4.1 GitOps Stores (Intent and Requested) + +**Contract characteristics:** +- Branch-per-request โ€” each request is a branch in the store +- Pull Request semantics โ€” review, comment, approve, merge +- Immutable history โ€” commits are permanent records +- CI/CD hook support โ€” commits trigger pipeline execution +- Indexed for query โ€” a Search Index projection enables field-based queries at scale +- Entity UUID โ†’ file path mapping maintained in the Search Index + +**Typical implementations:** GitHub, GitLab, Gitea, Forgejo (with Elasticsearch/OpenSearch as the Search Index) + +**Repository structure:** Deferred pending Q54 resolution (provider selection in Requested State affects directory structure). Will be documented in `04-examples.md`. + +### 4.2 Event Stream Stores (Realized and Discovered) + +**Contract characteristics:** +- Append-only โ€” events are never overwritten or deleted +- Entity-keyed streams โ€” each entity has its own event stream identified by entity UUID +- Queryable by entity UUID โ€” O(1) lookup of an entity's event stream +- Replayable โ€” the stream can be replayed from any point to reconstruct state at any timestamp +- Distributed and redundant โ€” data is replicated across nodes with configurable consistency guarantees +- High throughput โ€” designed for machine-generated, high-frequency writes + +**Typical implementations:** Kafka with log compaction, EventStoreDB, Apache Pulsar + +### 4.3 Search Index (Git Store Projection) + +**Contract characteristics:** +- Derived from Git stores โ€” rebuilt from Git history on demand +- Explicitly non-authoritative โ€” Git always wins if index and Git disagree +- Queryable by indexed fields: entity_uuid, tenant_uuid, resource_type, lifecycle_state, timestamp, cost_center, business_unit, provider_uuid +- Lightweight โ€” stores indexed fields only, not full payloads +- Fast โ€” designed for millisecond query response at millions of records + +**Typical implementations:** Elasticsearch, OpenSearch, Meilisearch + +--- + +## 5. Rehydration + +Rehydration is the process of using a previously stored state record as the starting point for a new request. It is not a shortcut around governance โ€” **all relevant governance policies always apply regardless of rehydration source.** Rehydration is a new request that happens to start from a known prior state. + +### 5.1 Three Rehydration Sources + +**From Intent State:** +- The consumer's original declaration is replayed +- Full layer assembly runs โ€” current layers applied +- All governance policies run โ€” current policies applied +- Provider selection runs fresh +- Most likely to produce a different result than the original โ€” policies and layers may have changed +- Use cases: upgrade resource to current standards, apply new sovereignty constraints, environment refresh + +**From Requested State:** +- The previously assembled, policy-processed payload is loaded +- Layer assembly is skipped โ€” layers were already applied +- All governance policies run โ€” current policies applied +- Provider selection: configurable via flag (see Section 5.3) +- Use cases: reproduce a resource as closely as possible to the approved specification + +**From Realized State:** +- The provider-confirmed realized payload is loaded +- Provider-specific fields are stripped โ€” DCM unified format only +- Layer assembly is skipped +- All governance policies run โ€” current policies applied +- Provider selection: configurable via flag +- Use cases: exact reproduction for disaster recovery, environment cloning, replacing a failed resource + +### 5.2 The Common Governance Pipeline + +Regardless of rehydration source, all requests flow through the same governance pipeline: + +``` +Rehydration source selected and loaded + โ”‚ + โ”‚ Source payload becomes the basis for a new Intent State record + โ”‚ New entity UUID assigned (or existing UUID preserved โ€” policy decision) + โ”‚ Rehydration provenance recorded: source_store, source_record_uuid, + โ”‚ rehydration_reason, requested_by_uuid, rehydration_timestamp + โ–ผ +If source = Intent: + โ”‚ Full layer assembly runs (Steps 1-7) + โ”‚ Current layers applied + โ–ผ +If source = Requested or Realized: + โ”‚ Layer assembly skipped + โ”‚ Payload loaded as pre-assembled + โ”‚ If source = Realized: provider-specific fields stripped + โ–ผ +Placement evaluation + โ”‚ See Section 5.3 โ€” configurable + โ–ผ +Policy Engine โ€” ALL governance policies applied + โ”‚ Authorization policies: does this actor have permission to rehydrate? + โ”‚ Transformation policies: current enrichment applied + โ”‚ Validation policies: current constraints checked + โ”‚ GateKeeper policies: current field locks applied + โ”‚ Gatekeeping policies: is this resource type still permitted? + โ”‚ + โ”‚ Governance is NEVER skippable โ€” not for any rehydration source, + โ”‚ not for any actor, not for any urgency claim + โ–ผ +New Requested State produced and stored + โ”‚ New record โ€” never overwrites the source record + โ”‚ Source record remains immutable + โ”‚ Provenance chain links to source record + โ–ผ +Provider dispatch + โ”‚ Dispatched to selected provider + โ–ผ +New Realized State events produced + โ”‚ New event stream or continuation of existing stream + โ”‚ Provenance links to rehydration Requested State +``` + +### 5.3 Placement Flag โ€” Provider-Portable Rehydration + +When rehydrating from Requested State or Realized State, provider selection is configurable via an explicit flag in the rehydration request: + +```yaml +rehydration_request: + uuid: + source_store: + source_record_uuid: + + placement: + re_evaluate: false + # false (default): honor provider selection from source record + # Use when: original provider is available and appropriate + # Result: resource reproduced on same provider + # + # true: strip provider selection, run placement policies fresh + # Use when: original provider unavailable, decommissioned, + # at capacity, or no longer sovereign-compliant + # Result: placement policies select provider from current landscape + # Named concept: Provider-Portable Rehydration + + placement_constraints: + # Optional โ€” additional constraints for re-evaluation + # Only applicable when re_evaluate: true + exclude_provider_uuids: [, ...] + require_region: + require_sovereignty_capability: + + governance: + apply_all_policies: true + # Always true โ€” governance is never skippable + # Included explicitly for auditability โ€” the rehydration record + # must declare that governance was applied + + policy_version: current + # current (default): apply today's policies + # pinned: apply policies as of a specific timestamp + # Use when: exact historical reproduction required + # (audit evidence, regulatory examination, environment reconstruction) + # Requires elevated authorization โ€” bypasses current GateKeeper policies + # Only SRE and Admin actors may use pinned policy version + + pinned_timestamp: + # Required when policy_version: pinned + + rehydration_reason: + requested_by_uuid: +``` + +### 5.4 The Four Rehydration Modes + +Two independent axes โ€” placement and policy version โ€” produce four distinct rehydration configurations: + +| Mode | re_evaluate | policy_version | Use Case | +|------|-------------|----------------|----------| +| **Faithful** | false | current | Same provider, current governance | +| **Provider-Portable** | true | current | New provider, current governance | +| **Historical Exact** | false | pinned | Same provider, historical governance (audit evidence) | +| **Historical Portable** | true | pinned | New provider, historical governance | + +Historical modes require elevated authorization. All modes run governance โ€” the difference is whether governance uses current or pinned policies. + +### 5.5 Partial Resolution of Q54 โ€” Provider Selection + +The placement flag model clarifies the Q54 question (selected_provider as policy output vs placement component). The emerging answer: + +**Policies set placement constraints โ€” the placement component selects the provider.** + +A GateKeeper policy may output: "must be in region EU-WEST, must support sovereignty capability PCI-DSS." The placement component reads these constraints and selects the specific provider within those constraints. The policy does not name the provider. The placement component names the provider. + +This is consistent with the portability model โ€” a policy that names a specific provider would be portability-breaking. Policies set constraints. Placement honors constraints and selects. + +--- + +## 6. Drift Detection + +Drift is the difference between what DCM believes exists (Realized State) and what actually exists (Discovered State). + +### 6.1 Drift Detection Flow + +``` +Discovery cycle completes + โ”‚ Provider interrogated โ†’ Discovered State snapshot written + โ–ผ +Drift Detection component + โ”‚ Loads latest Discovered State for entity UUID + โ”‚ Loads latest Realized State events for entity UUID + โ”‚ Field-by-field comparison + โ–ผ +No drift detected + โ”‚ Discovery timestamp updated + โ”‚ No action + โ–ผ +Drift detected + โ”‚ Drift record created with: + โ”‚ - entity_uuid + โ”‚ - drifted_fields: [{field_path, realized_value, discovered_value}] + โ”‚ - discovery_timestamp + โ”‚ - drift_severity: + โ–ผ +Policy Engine evaluates drift + โ”‚ Drift response policy determines action: + โ”‚ REVERT: submit a rehydration request from Realized State to restore + โ”‚ UPDATE_DEFINITION: promote discovered state to new Realized State + โ”‚ ALERT: notify personas, no automatic action + โ”‚ ESCALATE: trigger human review workflow + โ”‚ + โ”‚ Response determined by drift severity, resource type, + โ”‚ resource ownership, and organizational policy + โ–ผ +Audit Store records drift event with full provenance +``` + +### 6.2 Unsanctioned Changes + +A specific category of drift โ€” a change made directly to a resource without a corresponding DCM request. Detected by: +- Kubernetes: CR spec change without DCM request annotation +- VMware/OpenStack: resource modification not traceable to a DCM Requested State record +- General: any Discovered State field value that differs from Realized State without a Requested State record explaining the change + +Unsanctioned changes are always reported to the Policy Engine as `UNSANCTIONED_CHANGE` events. Policy determines the response. + +--- + +## 7. CI/CD Integration + +The GitOps stores are the natural integration point for CI/CD pipelines. DCM does not prescribe a specific CI/CD tool โ€” the GitOps store contract requires hook support, and the CI/CD tool is a deployment choice. + +### 7.1 CI Pipeline (Intent State) + +Triggered on: branch creation or update (new or revised intent) + +``` +CI pipeline executes: + 1. Policy pre-validation (dry run โ€” no state changes) + โ†’ Reports: which policies would apply, what they would do + 2. Cost estimation + โ†’ Reports: estimated cost for lifecycle of this resource + 3. Dependency graph validation + โ†’ Reports: all required dependent resources, any conflicts + 4. Sovereignty constraint check + โ†’ Reports: which sovereignty constraints apply, any violations + 5. Authorization check + โ†’ Reports: does this actor have permission to request this resource type? + 6. Auto-approve evaluation + โ†’ Reports: can this be merged automatically, or does it require human review? + +All results posted as PR comments on the Intent State branch +Consumer and approvers can review and debate before merge +``` + +### 7.2 CD Pipeline (Requested State) + +Triggered on: Intent State merge (PR merged to main) + +``` +CD pipeline executes: + 1. Request Payload Processor assembles full payload + 2. Full policy evaluation (binding โ€” not dry run) + 3. Provider selection (or re-evaluation if placement flag set) + 4. Requested State committed to Git store + 5. Provider dispatch via API Gateway + 6. Status monitoring โ€” poll or receive callbacks until terminal state + 7. Status written back to PR or status file + 8. Consumer notification +``` + +### 7.3 The Third Rail โ€” Direct API Ingress + +Not all requests come through the GitOps PR workflow. Some requests come through direct API submission โ€” automated systems, CI/CD pipelines, Terraform providers, programmatic consumers. These bypass the human review workflow but not governance. + +Direct API ingress: +- Creates an Intent State record (the submitted payload becomes the intent) +- Runs the same CI validation pipeline but non-interactively +- If auto-approve policy permits: proceeds directly to assembly and dispatch +- If human review required: creates a PR for review before proceeding +- Same governance pipeline regardless of ingress path + +The three ingress paths โ€” PR workflow, direct API, and programmatic (Terraform/Ansible) โ€” all converge on the same governance pipeline. The ingress path affects the review workflow; it never affects governance. + +--- + +## 8. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Git repository structure for Intent and Requested stores โ€” deferred pending Q54 resolution | Store design | โ“ Unresolved | +| 2 | Should the entity UUID be preserved or regenerated on rehydration? | Entity identity | โ“ Unresolved | +| 3 | For pinned policy version rehydration โ€” what is the minimum authorization level required? | Security | โ“ Unresolved | +| 4 | How are concurrent rehydration requests for the same entity handled โ€” serialized or rejected? | Concurrency | โ“ Unresolved | +| 5 | Should the Discovered Store retain full history or only a configurable window? | Retention | โ“ Unresolved | +| 6 | How does the Search Index handle Git store unavailability โ€” serve stale results or fail? | Reliability | โ“ Unresolved | + +--- + +## 9. Related Concepts + +- **Storage Provider** โ€” the formal provider type for all DCM stores +- **Entity UUID** โ€” the universal linking key across all four states +- **Rehydration** โ€” using a prior state record as the starting point for a new request +- **Provider-Portable Rehydration** โ€” rehydration with provider selection re-evaluated +- **Drift Detection** โ€” comparing Realized State against Discovered State +- **Unsanctioned Change** โ€” a resource modification not traceable to a DCM request +- **CI/CD Integration** โ€” GitOps stores as the natural CI/CD integration point +- **Search Index** โ€” queryable projection of GitOps stores, explicitly non-authoritative + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/information-providers.md b/content/docs/architecture/data-model/information-providers.md new file mode 100644 index 0000000..0bcf996 --- /dev/null +++ b/content/docs/architecture/data-model/information-providers.md @@ -0,0 +1,441 @@ +--- +title: "Information Providers" +type: docs +weight: 9 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Resource Type Hierarchy](../resource-type-hierarchy/) | [Resource/Service Entities](../resource-service-entities/) | [Entity Relationships](../entity-relationships/) + +--- + +## 1. Purpose + +An **Information Provider** is a registered DCM provider that serves as the authoritative source for a specific category of data that DCM needs to reference but does not own. It exposes external data to DCM through a standard interface, enabling DCM to look up, verify, and relate external records without caching or owning them. + +Information Providers are a first-class provider type in DCM alongside Service Providers and Meta Providers. They follow the same registration, health check, trust, and contract model as Service Providers โ€” adapted where applicable to the lookup-only nature of information retrieval. + +--- + +## 2. Why Information Providers Exist + +DCM manages the lifecycle of resources it provisions. But resources exist in a broader organizational context โ€” they are owned by business units, attributed to cost centers, associated with product owners, governed by regulatory scopes. This contextual data lives in authoritative external systems (HR systems, finance systems, CMDBs, ITSM tools) that DCM does not and should not own. + +Without a formal model for referencing external data, organizations face two bad choices: +- **Copy the data into DCM** โ€” creating duplication, staleness, and an ownership conflict with the authoritative system +- **Ignore the data** โ€” losing business context, cost attribution, and compliance traceability + +Information Providers solve this by giving DCM a standard, stable, governed interface to external data without requiring ownership transfer. + +--- + +## 3. The Three Provider Types + +DCM recognizes three provider types. All follow the same base contract model: + +| Provider Type | Purpose | Data Direction | DCM Owns Result? | +|--------------|---------|---------------|-----------------| +| **Service Provider** | Executes work, realizes resources | DCM โ†’ Provider โ†’ DCM | Yes โ€” DCM owns the realized entity | +| **Information Provider** | Serves authoritative external data | DCM โ†’ Provider (lookup only) | No โ€” external system is authoritative | +| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Child Providers โ†’ DCM | Yes โ€” DCM owns the composite result | + +--- + +## 4. Information Provider Contract + +Information Providers follow the same provider contract model as Service Providers where applicable. The contract dimensions are: + +### 4.1 Registration Contract +Same model as Service Providers. Information Providers register with DCM declaring their endpoint, the information types they implement, their lookup capabilities, and their extended schema. + +### 4.2 Health Check Contract +Same model as Service Providers. Information Providers expose a `/health` endpoint. DCM polls it on the same configurable interval. `Ready`/`NotReady` state machine applies. An `NotReady` Information Provider is excluded from lookups โ€” relationships referencing it are flagged for on-demand verification fallback. + +### 4.3 Trust Contract +Same model as Service Providers. Information Providers must be registered, validated, and certified before DCM will accept their data. The chain of trust applies to data returned by Information Providers โ€” provenance records the provider UUID for every field sourced from an Information Provider. + +### 4.4 Capacity Contract +Adapted for lookup capacity rather than resource provisioning capacity. Information Providers declare and report their query capacity โ€” requests per second, rate limits, availability windows. + +```yaml +capacity_registration: + provider_uuid: + registration_timestamp: + capacity_by_information_type: + - information_type_uuid: + queries_per_second: 1000 + rate_limit_window: 60s + availability: 99.9% +``` + +### 4.5 Lifecycle Event Contract +Same model as Service Providers. Information Providers have a contractual obligation to notify DCM when records they have provided references for change status. DCM receives the notification and updates the external entity reference record accordingly. + +**Reportable event types for Information Providers:** + +| Event Type | Description | DCM Response | +|------------|-------------|--------------| +| `RECORD_DEACTIVATED` | A referenced record has been deactivated | Update reference status, Policy Engine evaluation | +| `RECORD_MERGED` | Two records merged โ€” UUID may change | Update external_uuid in reference record | +| `RECORD_SPLIT` | One record split into multiple | Policy Engine evaluation โ€” which new record applies? | +| `UUID_CHANGED` | Record UUID changed in external system | Update external_uuid, re-verify all references | +| `DATA_UPDATED` | Standard field values changed | Update last_verified, notify relationships | +| `PROVIDER_DEGRADED` | Provider is degraded but operational | DCM flags affected references for on-demand verification | + +### 4.6 Naturalization/Denaturalization Contract +Information Providers translate their native data format (HR system JSON, finance system XML, LDAP records, REST APIs) into the DCM unified data model format. The translation is the provider's responsibility โ€” DCM always receives data in DCM format. + +--- + +## 5. Standard vs Extended Data + +### 5.1 Standard Data (DCM-defined) + +Fields that are part of the DCM-specified schema for an information type. DCM core uses these fields for lookups, relationship matching, policy evaluation, and display. They are portable across all implementations of that information type. + +DCM only relies on standard data for operational decisions. Extended data is carried in the payload but is not used for DCM core operations. + +### 5.2 Extended Data (organization-defined) + +Additional fields organizations add to enrich the standard schema for their specific needs. Declared in the provider's extended schema registration. DCM carries extended data in the payload for downstream consumers โ€” policy engines, cost analysis tools, reporting โ€” that know how to use them. + +```yaml +# Standard + Extended data example โ€” Business.BusinessUnit +business_unit_record: + # Standard fields โ€” DCM defined, used for lookups + uuid: "bu-uuid-001" + name: "Payments Platform" + code: "BU-PAY" + parent_uuid: "bu-uuid-root" + organization_uuid: "org-uuid-001" + status: active + + # Extended fields โ€” organization defined + extensions: + profit_center_code: "PC-4421" + regulatory_jurisdiction: "EU" + trading_desk_id: "TD-007" + risk_tier: 1 + internal_charge_code: "IC-PAY-001" +``` + +--- + +## 6. Lookup Key Model + +DCM looks up external records using a stable primary key โ€” always the external UUID where available โ€” with a fallback chain for systems that don't support UUID-based lookup. + +### 6.1 External Entity Reference Structure + +```yaml +external_entity_reference: + uuid: + # DCM UUID is what gets stored in relationship declarations + # If the external system changes its UUID, only this record changes + # All relationships pointing to dcm-uuid remain valid + + external_uuid: + information_provider_uuid: + information_type_uuid: + information_type_name: Business.BusinessUnit + + lookup_method: + primary_key: external_uuid + # Always attempted first + fallback_keys: + - field: code + value: "BU-PAY" + - field: name + value: "Payments Platform" + # Fallback keys tried in order if primary_key lookup fails + + # Non-authoritative display cache โ€” for UI convenience only + display_name: "Payments Platform" + display_name_authoritative: false + + verification: + last_verified: + last_verified_method: + verification_status: + next_scheduled_verification: + + status: + state: + + provenance: + +``` + +### 6.2 Why DCM UUID Wraps External UUID + +The DCM-generated UUID is the stable internal anchor. This means: +- All relationship declarations inside DCM reference the DCM UUID +- If the external system changes its UUID (migration, system upgrade), only the `external_entity_reference` record needs updating +- All relationships pointing to the DCM UUID remain valid without modification +- The provenance chain tracks the change via the `UUID_CHANGED` lifecycle event + +--- + +## 7. Three-Mode Verification Model + +DCM uses a trust-but-verify approach to external entity references. The external system is trusted as authoritative for the data โ€” DCM does not validate content. But DCM verifies that references remain valid โ€” the UUID still exists and the record is still active. + +### 7.1 Mode 1 โ€” Scheduled Verification (DCM-initiated) + +DCM calls the Information Provider's `/verify/{uuid}` endpoint on a configurable schedule for all registered external entity references. Default frequency: configurable โ€” suggested minimum twice daily. Updates `last_verified` and `verification_status`. + +### 7.2 Mode 2 โ€” Provider Push (Information Provider obligation) + +The Information Provider notifies DCM when a referenced record changes status. This is a contractual obligation โ€” same model as Service Provider lifecycle events. DCM receives the notification, updates the external entity reference, and the Policy Engine evaluates the appropriate response. + +### 7.3 Mode 3 โ€” On-Demand Verification (fallback) + +When a relationship involving an external entity reference is accessed during request processing, policy evaluation, or drift detection, DCM can verify the reference in real time before relying on it. Used when: +- `last_verified` is beyond the acceptable staleness window +- The operation requires high confidence +- Scheduled verification returned `stale` or `unverifiable` + +### 7.4 Verification Fallback Chain + +``` +External entity reference accessed + โ”‚ + โ–ผ +Is verification_status: verified AND last_verified within window? + โ”‚ Yes โ†’ proceed with reference + โ”‚ No โ†“ + โ–ผ +Mode 3 โ€” on-demand verify via Information Provider /verify/{uuid} + โ”‚ Success โ†’ update last_verified, verification_status: verified, proceed + โ”‚ Failure โ†“ + โ–ผ +Policy Engine evaluates: + Options (configurable per information type and organizational policy): + block_request โ€” reject request until reference is verified + warn_and_proceed โ€” proceed with warning recorded in provenance + use_display_only โ€” use display_name only, no operational reliance + escalate โ€” notify appropriate personas for human resolution +``` + +--- + +## 8. Information Type Registry + +Information types live in the same DCM Resource Type Registry as Resource Types, distinguished by category prefix. Same versioning, same deprecation model, same governance. + +### 8.1 Standard Information Type Categories + +| Category | Description | Examples | +|----------|-------------|---------| +| `Business.*` | Business organizational data | BusinessUnit, CostCenter, ProductOwner | +| `Identity.*` | Identity and access data | Person, ServiceAccount, Group | +| `Compliance.*` | Regulatory and compliance data | RegulatoryScope, AuditFramework | +| `Operations.*` | Operational reference data | Runbook, SLA, SupportContract | + +### 8.2 DCM Default Information Types + +```yaml +# Business.BusinessUnit +information_type: + uuid: + name: Business.BusinessUnit + category: Business + version: 1.0.0 + standard_fields: + - name: uuid + type: string + required: true + lookup_supported: true + - name: name + type: string + required: true + lookup_supported: true + - name: code + type: string + required: false + lookup_supported: true + - name: parent_uuid + type: string + required: false + lookup_supported: false + - name: organization_uuid + type: string + required: true + lookup_supported: false + - name: status + type: enum + values: [active, inactive] + required: true + lookup_supported: false + extended_fields_permitted: true + status: active + +# Business.CostCenter +information_type: + uuid: + name: Business.CostCenter + standard_fields: + - name: uuid + lookup_supported: true + - name: name + lookup_supported: true + - name: code + lookup_supported: true + - name: owner_uuid + lookup_supported: false + - name: budget_period + lookup_supported: false + - name: status + lookup_supported: false + +# Identity.Person +information_type: + uuid: + name: Identity.Person + standard_fields: + - name: uuid + lookup_supported: true + - name: name + lookup_supported: true + - name: email + lookup_supported: true + - name: employee_id + lookup_supported: true + - name: department_uuid + lookup_supported: false + - name: status + lookup_supported: false +``` + +### 8.3 Custom Information Types + +Organizations register custom information types following the same model: + +```yaml +custom_information_type: + uuid: + name: + # Must use a non-reserved category prefix or register a new one + category: + version: + registered_by_tenant_uuid: + standard_fields: + + extended_fields_permitted: + status: +``` + +--- + +## 9. Information Provider Registration + +```yaml +information_provider_registration: + uuid: + name: + display_name: + version: + + implements: + - information_type_uuid: + information_type_name: Business.BusinessUnit + information_type_version: + lookup_methods_supported: [primary_key, code] + extended_fields_supported: true + extended_schema: + + + endpoint: + + capacity: + queries_per_second: + rate_limit_window: + update_frequency: + + sovereignty_constraints: + + + trust_declaration: + + + health_check: + endpoint: /health + poll_interval_seconds: + + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + + provenance: + +``` + +--- + +## 10. Mandatory Information Provider API Endpoints + +All Information Providers must implement these endpoints as part of their provider contract: + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/health` | Provider health check โ€” same as Service Provider | +| `GET` | `/lookup/{uuid}` | Returns standard + extended data for a record by external UUID | +| `GET` | `/verify/{uuid}` | Lightweight โ€” confirms UUID exists and is active | +| `POST` | `/search` | Finds records matching standard field criteria (fallback lookup) | +| `POST` | `/notify` | DCM calls this to acknowledge receipt of provider push events | + +--- + +## 11. Internally Owned Business Data + +When an organization decides to manage business context data in DCM rather than reference an external system, they define it as a DCM Resource Type in the `Business.*` or custom category. Internally owned business data follows the **standard resource entity model** exactly: + +- Has a UUID +- Has a Resource Type (`Business.BusinessUnit`, `Business.CostCenter`, etc.) +- Has provenance +- Has versioning +- Has relationships to other entities +- Follows the universal lifecycle (active โ†’ deprecated โ†’ retired) +- Can be grouped under Tenants and Resource Groups + +The relationship model is identical whether the related entity is internal or external โ€” the `related_entity_type` field (`internal` vs `external`) is the only difference from the consuming entity's perspective. + +This means an organization can start with an external Information Provider reference and migrate to internally owned business data later โ€” relationships remain structurally the same, only the `related_entity_type` changes. + +--- + +## 12. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are conflicting provider push events handled โ€” two Information Providers claim authority for the same record? | Data integrity | โ“ Unresolved | +| 2 | Should Information Providers support write-back โ€” DCM updating external records via the provider? | Scope expansion | โ“ Unresolved | +| 3 | How is the extended schema versioned โ€” if a provider adds or removes extended fields, how are existing references affected? | Versioning | โ“ Unresolved | +| 4 | Should DCM maintain a registry of well-known Information Providers (HR systems, finance systems) to simplify onboarding? | Adoption | โ“ Unresolved | +| 5 | How does the verification model interact with air-gapped environments where Information Providers may be unreachable? | Sovereignty | โ“ Unresolved | + +--- + +## 13. Related Concepts + +- **External Entity Reference** โ€” the stable pointer record DCM uses to reference external data +- **Entity Relationships** โ€” the universal relationship model that uses Information Provider references +- **Service Provider** โ€” counterpart provider type for resource provisioning +- **Resource Type Registry** โ€” the unified registry containing both Resource Types and Information Types +- **Trust Contract** โ€” the provider trust model shared across all provider types +- **Naturalization/Denaturalization** โ€” translation between external native format and DCM unified format + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md new file mode 100644 index 0000000..cbd4eeb --- /dev/null +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -0,0 +1,1265 @@ +--- +title: "Data Layers and Assembly" +type: docs +weight: 3 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) + +--- + +## 1. Purpose + +Data Layers are the mechanism by which DCM assembles a complete, contextually correct request payload from a set of composable, reusable data definitions. Rather than requiring consumers to specify every field of every resource they request, layers allow standards, organizational context, service-specific configuration, and consumer intent to be declared independently and merged into a unified payload at request time. + +Layers are the answer to the question: **how does a single consumer request become a complete, policy-validated, provider-ready payload?** + +The layering model enables: +- **Reuse** โ€” a base configuration defined once is inherited by thousands of resources +- **Standardization** โ€” organizational standards are encoded in layers, not in every individual request +- **Separation of concerns** โ€” infrastructure teams own core and service layers; consumers own request layers; policy owners own policy layers +- **Scale** โ€” 36 layer definitions can govern 40,000 VMs without duplication +- **Auditability** โ€” every field in the merged payload knows which layer set it and why + +--- + +## 2. What is a Layer? + +A Layer is a **declarative, immutable, versioned unit of data** that contributes some or all of its fields to a merged payload. Layers do not execute โ€” they declare. The assembly process is what merges them. + +Every layer: +- Has a **UUID** that uniquely identifies it +- Has a **version** following the universal Major.Minor.Revision scheme +- Is **immutable once published** โ€” changes produce a new version +- Carries a **reference to its parent entity** (UUID and version) +- Has an **origination timestamp** +- Can be **deprecated** following the universal deprecation model +- Contributes **provenance metadata** for every field it sets โ€” any field set by a layer records that layer's UUID as its source + +Layers are stored in Git following GitOps practices. They are the configuration source of truth โ€” not the assembled payload. + +--- + +## 3. Layer Types + +DCM defines six layer types. Each has a distinct purpose, scope, ownership model, and position in the assembly precedence chain. + +### 3.1 Base Layer + +**Purpose:** The foundation entity for a resource. Defines the minimum required fields and their default values for a given resource context. Everything starts with a Base Layer. + +**Scope:** Can be type-agnostic (a universal base) or type-scoped (a base specific to a Resource Type). A Base Layer that is type-scoped must declare its Resource Type. + +**Ownership:** DCM platform or platform implementor. + +**Characteristics:** +- Every layer chain must begin with a Base Layer +- Base Layers contain only universal fields โ€” no provider-specific data +- A Base Layer for a typed resource must conform to the Resource Type Specification's universal field requirements +- Multiple Base Layers can exist for the same context โ€” the applicable one is selected based on the request context + +**Examples:** +- CIS Benchmark base configuration +- Baseline OS configuration +- DMZ network base configuration + +--- + +### 3.2 Core Layers + +**Purpose:** Provide data that is applicable across any resource type. Core Layers carry organizational, infrastructure, and contextual data that is not specific to any one service. + +**Scope:** Type-agnostic by default. Core Layers apply to all resource types unless explicitly scoped. This is the primary distinction from Service Layers. + +**Ownership:** DCM platform, infrastructure teams, or platform implementors. + +**Characteristics:** +- Applied to every request regardless of resource type +- Cannot contain service-specific or provider-specific data +- Carry location, organizational, and infrastructure context +- Stored in the Core Layer Store +- Cached in the Service Layer Cache at deployment time + +**Examples:** +- Data Center layer (DC1, DC2) +- Zone layer (Zone 1, Zone 2) +- Rack layer +- Geographic region layer +- Environment layer (production, staging, development) + +--- + +### 3.3 Intermediate / Customization Layers + +**Purpose:** Provide organizational or contextual overrides and customizations that sit between the base standards and the service-specific configuration. These layers encode the organizational hierarchy and deployment context. + +**Scope:** Can be type-agnostic or type-scoped. Scope is declared per layer. + +**Ownership:** Organizational teams, domain owners, platform implementors. + +**Characteristics:** +- Stack between Core Layers and Service Layers in the precedence chain +- Encode organizational structure (business unit, enclave, logical unit) +- Allow organizational customization without modifying base standards +- The Git repo hierarchy typically mirrors the intermediate layer hierarchy + +**Examples:** +- Ship layer (in Navy context: specific vessel configuration) +- Enclave layer (isolated network segment configuration) +- Business unit layer +- DMZ customization layer +- Production web tier layer + +--- + +### 3.4 Service Layers + +**Purpose:** Provide service-specific data required to build a complete request payload for a specific Resource Type. Service Layers are the bridge between general organizational context and provider-ready configuration. + +**Scope:** **Must be type-scoped.** A Service Layer without a declared Resource Type scope is invalid. The scope inheritance behavior is configurable per Service Layer declaration. + +**Type Scope Declaration:** +```yaml +type_scope: + resource_type_uuid: + resource_type_fully_qualified_name: + scope_inheritance: + # exact: applies only to the declared Resource Type + # descendants: applies to the declared Resource Type and all child types via inheritance +``` + +**Ownership:** Service Providers or service domain teams. Stored in Service Layer SCM (source control management). Registered with DCM as part of Service Provider registration. + +**Characteristics:** +- Only applied when the request resource type matches the layer's declared type scope +- Carry service-specific configuration, defaults, and constraints +- Must not contain provider-specific data unless marked as portability-breaking +- Cached in the Service Layer Cache at Service Provider registration time + +**Examples:** +- VM sizing layer (small, medium, large configurations for `Compute.VirtualMachine`) +- Web server configuration layer for `Compute.VirtualMachine` +- Network port configuration layer for `Network.Port` +- CL Web Service Data Layer for `Compute.VirtualMachine` (exact scope) +- General compute placement layer for `Compute.VirtualMachine` and descendants + +--- + +### 3.5 Request Layer + +**Purpose:** Carries the consumer's declared intent. The Request Layer is what the consumer provides โ€” the fields they explicitly specify for their resource request. + +**Scope:** Scoped to the Resource Type the consumer is requesting. + +**Ownership:** Consumer (via Web UI or Consumer API). + +**Characteristics:** +- Created at the time the consumer submits a request +- Contains only what the consumer explicitly declares โ€” it does not need to be complete +- The gap between what the consumer declares and what the provider needs is filled by the lower layers in the chain +- Has higher precedence than all data layers below it โ€” consumer-declared values override layer defaults +- Is the direct source of the **Intent State** โ€” the Request Layer as submitted by the consumer is stored in the Intent Store before any processing occurs +- After assembly and policy processing, the enriched payload becomes the **Requested State** + +**Examples:** +- Consumer requests a VM with `cpu_count: 8`, `ram_gb: 32`, `os: RHEL9`, `environment: production` +- Consumer requests a firewall rule with source/target network and port + +--- + +### 3.6 Policy Layers + +**Purpose:** Policy Layers are not data layers in the traditional sense โ€” they do not add fields to the merge chain. Instead, they operate on the assembled payload after the data layers have been merged. They are the governance layer of the assembly process. + +**Scope:** Scoped by policy type and domain. Core Policies apply to all requests. Service Policies apply to specific Resource Types. Organizational and domain policies apply to specific organizational scopes. + +**Ownership:** Policy creators, security teams, compliance teams, organizational domain owners. + +**Policy Layer Types and Their Behavior:** + +| Policy Type | Behavior | Precedence Effect | +|-------------|----------|-------------------| +| **Validation** | Checks data against rules. Does not modify data. Returns pass/fail. If fail, request is rejected. | No precedence โ€” pass/fail only | +| **Transformation** | Enriches or modifies data in the payload. Adds missing fields, applies standards, fills gaps. | Adds to or modifies the assembled payload โ€” recorded in provenance | +| **GateKeeper** | Highest authority. Can override any field regardless of what was declared in lower layers or the Request Layer. Can halt execution entirely. Used for sovereignty constraints, security mandates, and hard compliance rules. | Overrides everything โ€” including consumer input | + +**Characteristics:** +- Policies operate only on the policy definition, core data, and the data in the request payload +- Policy outcomes are deterministic โ€” same input always produces same output for a given policy version +- All policy modifications are recorded in field-level provenance with policy UUID, operation type, and reason +- Policies are versioned using the universal versioning scheme +- Policies are maintained via GitOps practices + +--- + +## 4. Layer Identity โ€” Domain, Handle, and Priority + +Every layer has a formal identity model with three components that together make it uniquely identifiable, locatable, and orderable within DCM. + +### 4.1 Layer Domain + +The **Layer Domain** is the organizational and architectural home of a layer. It declares ownership, storage location, and authorization scope โ€” who can create and modify this layer, and which other layers it can override. + +| Domain | Meaning | Authorization | Can Override | +|--------|---------|--------------|-------------| +| `system` | DCM built-in layers, shipped with DCM | DCM maintainers only | Nothing above system | +| `platform` | Platform team layers, apply across all tenants | Platform team | tenant, service, provider | +| `tenant` | Tenant-specific layers | Tenant Admin | service, provider within tenant | +| `service` | Service Provider contributed layers | Service Provider owner | provider | +| `provider` | Provider Catalog Item layers | Provider owner | Nothing above provider | + +A lower-domain layer cannot override a higher-domain layer. A `tenant` layer cannot override a `platform` layer. This is enforced at ingestion โ€” the conflict detection pipeline checks domain authority before allowing a merge. + +### 4.2 Layer Handle + +The **Layer Handle** is the human-readable, stable identifier for a layer within DCM. It complements the UUID (machine-meaningful) with a reference that humans can use in conversation, documentation, policy declarations, and audit records. + +**Format:** `{domain}/{layer_type}/{name}` + +**Examples:** +``` +platform/core/cis-benchmark-linux +platform/core/security-cpu-limits +tenant/service/payments-vm-standards +system/base/universal-defaults +service/service/kubevirt-vm-defaults +provider/service/cloudnativepg-database-config +``` + +**Rules:** +- Unique within DCM โ€” enforced at ingestion +- Stable across versions โ€” the handle does not change when a new version is published +- URL-safe characters only โ€” lowercase, hyphens, forward slashes +- The Git file path mirrors the handle structure exactly + +**Git path from handle:** +``` +{layer_store_root}/{domain}/{layer_type}/{name}/v{Major}.{Minor}.{Revision}.yaml + +# Example: +dcm-layers/platform/core/security-cpu-limits/v1.2.0.yaml +dcm-layers/tenant/{tenant-uuid}/service/payments-vm-standards/v1.0.0.yaml +``` + +### 4.3 Priority Schema + +The **Priority Schema** is the deterministic ordering mechanism for resolving conflicts between layers of the same type and scope. It uses a hierarchical dotted-notation system that supports infinite differentiation โ€” new priority values can always be inserted between any two existing values. + +**Format:** `{integer}.{integer}.{integer}...` โ€” unlimited depth + +**Comparison:** Left-to-right, segment by segment. **Higher numeric value = higher priority.** No ceiling โ€” you can always go higher. + +``` +900.10 beats 800.10 (900 > 800 at segment 1) +900.20 beats 900.10 (20 > 10 at segment 2) +900.10.5 beats 900.10 (longer path with matching prefix โ€” 5 at segment 3 > nothing) +900.10.10 beats 900.10.5 (10 > 5 at segment 3) +``` + +**Infinite insertability โ€” symmetric in both directions:** +Between `900.10` and `900.20` insert `900.15`. Between existing values or above the current maximum โ€” there is no ceiling. You can always go higher. This is the key advantage over a lower-wins model which would have a hard floor at 1. + +**Priority Label:** Semantic context for the numeric value โ€” human-readable, does not affect ordering. + +**Reference Priority Taxonomy (advisory โ€” not enforced by DCM):** + +DCM ships a reference taxonomy as documentation and starter templates. Organizations adopt, adapt, or ignore it โ€” DCM resolves conflicts purely by numeric comparison. The taxonomy is a suggested convention, not a system constraint. + +| Suggested Range | Category | Rationale | +|-----------------|----------|-----------| +| `900.*` | Compliance | Regulatory mandates โ€” highest authority | +| `800.*` | Security | Security standards | +| `700.*` | Sovereignty | Data residency constraints | +| `600.*` | Operations | SRE and operational standards | +| `500.*` | Platform | Platform-level defaults | +| `400.*` | Service | Service-specific configuration | +| `300.*` | Organization | Organizational defaults | +| `200.*` | Site | Location-specific overrides | +| `100.*` | Custom | Implementor-defined โ€” lowest standard category | + +Higher number = higher priority. An organization that needs a mandate above all standard categories simply uses `1000.*` or above โ€” no renumbering required. An organization that prioritizes sovereignty above compliance would simply swap their `700.*` and `900.*` ranges. + +**Priority in a layer definition:** + +```yaml +priority: + value: "800.30.10" + label: "security.container.cpu_limits" + category: security + rationale: > + CPU limit enforcement for container workloads per + CISO mandate SEC-2024-047. Overrides platform defaults + to ensure no container can exceed approved limits. +``` + +--- + +## 4b. Artifact Metadata Standard + +Every DCM artifact โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions, and all other defined or stored objects โ€” carries a standard **Artifact Metadata** block. This is a structural requirement, not optional. + +The artifact metadata block answers: **who created this, when, who owns it, what changed, and how do we contact them?** + +### 4b.1 Universal Artifact Metadata Structure + +```yaml +artifact_metadata: + + # Identity + uuid: + handle: + + # Versioning + version: + status: + + # Status detail โ€” populated per status + status_detail: + # When status: proposed + proposed_at: + proposed_by: + uuid: + display_name: + email: + shadow_execution: + enabled: + started_at: + evaluation_count: + validation_dashboard_url: + + # When status: deprecated + deprecated_at: + deprecated_by: + uuid: + display_name: + replacement_uuid: + replacement_handle: + deprecation_reason: + migration_guidance: + sunset_date: + + # When status: retired + retired_at: + retired_by: + uuid: + display_name: + + # Origination + created_by: + uuid: + display_name: + email: + notification_endpoint: + created_at: + created_via: + # pr: submitted via GitOps PR workflow โ€” full review history available + # api: submitted via direct API + # migration: imported from external system โ€” provenance depth may be limited + # system: created by DCM itself (entity stubs, system artifacts) + + # Ownership โ€” may differ from creator + owned_by: + uuid: + display_name: + email: + notification_endpoint: + # Note: created_by is the audit record (who physically submitted it) + # owned_by is the accountability record (who is responsible and gets notified) + + # Modification history โ€” append-only + modifications: + - sequence: 1 + modified_by: + uuid: + display_name: + email: + modified_at: + modification_type: + version_before: + version_after: + change_summary: + pr_reference: + reason: +``` + +### 4b.2 The Five Artifact Statuses + +| Status | Meaning | Executes? | Output Applied? | Output Captured? | Merges to Active? | +|--------|---------|-----------|----------------|-----------------|------------------| +| `developing` | In active development. Development mode / dev pipeline only. | Dev mode only | No | Dev logs only | No โ€” must transition to proposed first | +| `proposed` | Development complete. Submitted for validation. Shadow mode for policies. | Yes (shadow) | No | Yes โ€” validation report | Yes โ€” after review approval | +| `active` | Live and governing. Applied to all relevant requests. | Yes | Yes | Yes โ€” audit/provenance | N/A | +| `deprecated` | Being phased out. Replacement available. Works but warns. | Yes | Yes | Yes โ€” with deprecation warning | N/A | +| `retired` | End of life. Cannot be used. | No | No | No | No | + +**Status transition rules:** +``` +developing โ†’ proposed (author submits for review) +developing โ†’ retired (author abandons without proposing) +proposed โ†’ active (reviewers approve โ€” via PR merge or API approval) +proposed โ†’ developing (returned for rework) +active โ†’ deprecated (replacement available โ€” sunset date declared) +deprecated โ†’ retired (sunset date reached or manual retirement) +retired โ†’ (terminal โ€” no transitions out) +``` + +### 4b.3 Proposed Status โ€” Shadow Execution for Policies + +When a policy artifact is in `proposed` status, it runs in **shadow mode** against real request traffic: + +- Executes alongside active policies on every relevant request +- Output is captured in a `proposed_evaluation_record` โ€” what it would have done +- Output is **never applied** to the actual request +- Shadow output feeds the Validation Dashboard for reviewer analysis +- Policy authors can see aggregate impact before activation + +```yaml +# Shadow output record โ€” captured per real request evaluated +proposed_evaluation_record: + policy_uuid: + policy_version: + request_uuid: + tenant_uuid: + evaluated_at: + would_have_applied: + shadow_output: + would_have_rejected: + rejection_reason: + would_have_patched: + - field: + current_value: + would_have_set: + reason: + would_have_locked: + - field: + lock_type: + reason: + would_have_selected_provider: + impact_assessment: + category: + # none: policy would not have applied to this request + # low: minor enrichment only + # medium: significant field modifications + # high: would have rejected or locked critical fields + # critical: would have rejected or overridden consumer intent +``` + +### 4b.4 Contact Info โ€” Two Modes + +Contact information supports both IdP-backed and standalone deployments: + +**Mode 1 โ€” Identity Provider backed:** +The `uuid` field contains the DCM external entity reference UUID linking to an Identity.Person or Identity.Team in a registered Information Provider. The `display_name` is cached non-authoritatively for UI display. DCM can resolve the full identity record via the Information Provider on demand. + +**Mode 2 โ€” Standalone (no Identity Provider):** +The `uuid` field is absent. `display_name`, `email`, and `notification_endpoint` are the primary identity fields. DCM accepts and records these directly without external verification. This mode supports bootstrapping, air-gapped deployments, and organizations that have not yet registered an Identity Information Provider. + +Both modes are fully supported. An organization can start in standalone mode and migrate to IdP-backed mode by adding `uuid` fields to existing artifact metadata โ€” no other changes required. + +### 4b.5 Notifications from Artifact Metadata + +The `owned_by.notification_endpoint` is the target for all proactive DCM notifications about an artifact: + +| Event | Who Is Notified | +|-------|----------------| +| Layer conflict detected at ingestion | Owner of new layer AND owner of conflicting existing layer | +| Layer deprecated | Owners of all artifacts that reference the deprecated layer | +| Provider deregistered | Owners of all catalog items backed by that provider | +| Policy violation | Owner of the entity that violated the policy | +| Drift detected | Owner of the entity that drifted | +| Proposed policy shadow shows high/critical impact | Policy owner and designated reviewers | +| Artifact approaching sunset date | Artifact owner | + +--- + +## 4c. Conflict Detection at Ingestion + +Conflict detection runs at layer ingestion time โ€” not at request assembly time. This ensures all layers in DCM are conflict-free before they are ever used. + +### 4c.1 Ingestion CI Pipeline + +When a layer is committed to the Layer Store (Git branch created or updated): + +``` +Layer committed to Git branch + โ”‚ + โ–ผ +CI Pipeline fires automatically + โ”‚ + โ”œโ”€โ”€ 1. Schema validation + โ”‚ Is the layer well-formed per the layer schema? + โ”‚ Does it carry required artifact metadata? + โ”‚ Is the version correctly incremented? + โ”‚ + โ”œโ”€โ”€ 2. Handle validation + โ”‚ Is the handle unique in DCM? + โ”‚ Does the handle match the Git path? + โ”‚ Does the domain match the submitting actor's authorization? + โ”‚ + โ”œโ”€โ”€ 3. Scope validation + โ”‚ If type-scoped: do declared resource types exist in the registry? + โ”‚ Is the layer type consistent with the domain? + โ”‚ + โ”œโ”€โ”€ 4. Priority validation + โ”‚ Is the priority value in valid dotted-notation format? + โ”‚ Does the priority category match the domain advisory range? + โ”‚ (Warning only if category/domain mismatch โ€” not a block) + โ”‚ + โ”œโ”€โ”€ 5. Conflict detection + โ”‚ For each field in this layer: + โ”‚ Find all active layers of the same type and overlapping scope + โ”‚ Check if any declare the same field + โ”‚ If conflict found: + โ”‚ โ†’ Does the new layer declare a higher priority? โ†’ Allowed, documented + โ”‚ โ†’ Does the existing layer declare a higher priority? โ†’ Allowed, documented + โ”‚ โ†’ Neither declares priority? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ โ†’ Both declare equal priority? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ โ†’ Domain authority violation? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ + โ”‚ Conflict notification: + โ”‚ Posted as PR comment with: conflicting layer UUID, handle, owner + โ”‚ Both layer owners notified via notification_endpoint + โ”‚ + โ”œโ”€โ”€ 6. Deprecation reference validation + โ”‚ If status: deprecated โ€” does replacement UUID exist? + โ”‚ + โ””โ”€โ”€ 7. Result + All checks pass โ†’ PR approved for merge + Any check fails โ†’ PR blocked, detailed error comment posted +``` + +### 4c.2 Conflict Resolution Rules + +| Situation | Resolution | Action | +|-----------|-----------|--------| +| New layer and existing layer conflict, no priority on either | CONFLICT ERROR | PR blocked. Both owners notified. One must declare priority or remove the conflicting field. | +| New layer has higher priority (higher value) than existing | Allowed โ€” new layer wins | Documented in provenance. Warning posted if domain authority is unusual. | +| Existing layer has higher priority | Allowed โ€” existing layer wins | New layer is a lower-priority alternative. Documented. | +| Both layers have equal priority | CONFLICT ERROR | PR blocked. Priority must be differentiated. | +| New layer from lower domain overrides higher domain | CONFLICT ERROR | Domain authority violation. Platform cannot be overridden by service layer. | +| Priority category suggests domain mismatch | WARNING | PR comment posted, not blocked. Merge allowed but reviewers are notified. | + +### 4c.3 Pre-Validation of All Layers + +Because conflict detection runs at ingestion, all layers resident in DCM are pre-validated: + +- No two active layers of the same type and scope conflict without explicit priority resolution +- The assembly process never encounters an ambiguous merge โ€” all conflicts are resolved at definition time +- If a conflict is discovered after the fact (e.g., a new layer is activated that conflicts with an existing one that was already active when the new layer was ingested), the newer layer's ingestion pipeline should have caught this. A background validation job runs periodically to detect any edge cases. + +--- + +## 4d. Complete Layer Definition Structure + +Combining all elements โ€” identity, artifact metadata, scope, priority, and fields: + +```yaml +# Complete layer definition +layer: + # === ARTIFACT METADATA (universal โ€” required on all artifacts) === + artifact_metadata: + uuid: "layer-uuid-001" + handle: "platform/core/security-cpu-limits" + version: "1.2.0" + status: active + created_by: + uuid: "actor-uuid-001" # Optional โ€” present if IdP registered + display_name: "Jane Smith" + email: "jane.smith@example.com" + notification_endpoint: "https://notify.example.com/webhooks/jane" + created_at: "2026-01-15T10:30:00Z" + created_via: pr + owned_by: + uuid: "team-uuid-security" # Optional โ€” present if IdP registered + display_name: "Platform Security Team" + email: "platform-security@example.com" + notification_endpoint: "https://notify.example.com/webhooks/platform-security" + modifications: + - sequence: 1 + modified_by: + display_name: "Jane Smith" + email: "jane.smith@example.com" + modified_at: "2026-01-15T10:30:00Z" + modification_type: create + version_before: null + version_after: "1.0.0" + change_summary: "Initial creation โ€” CPU limits per CISO mandate SEC-2024-047" + pr_reference: "https://github.com/org/dcm-layers/pull/42" + reason: "CISO mandate SEC-2024-047 requires CPU limits on all containers" + - sequence: 2 + modified_by: + display_name: "Bob Jones" + email: "bob.jones@example.com" + modified_at: "2026-02-20T14:00:00Z" + modification_type: update + version_before: "1.0.0" + version_after: "1.2.0" + change_summary: "Increased CPU limit from 4 to 8 per updated mandate" + pr_reference: "https://github.com/org/dcm-layers/pull/67" + reason: "Updated CISO mandate SEC-2024-047-rev2 allows 8 CPU" + + # === LAYER IDENTITY === + domain: platform + layer_type: core + + scope: + resource_types: + - Compute.Container + - Compute.Pod + # Empty list = type-agnostic (applies to all resource types) + + priority: + value: "200.30.10" + label: "security.container.cpu_limits" + category: security + rationale: > + CPU limit enforcement for container workloads per + CISO mandate SEC-2024-047. Overrides platform defaults. + + # === LAYER CHAIN === + parent_chain: + - uuid: "base-layer-uuid-001" + handle: "system/base/universal-defaults" + version: "1.0.0" + layer_type: base + + # === FIELDS === + fields: + cpu_limit: + value: 8 + metadata: + basis_for_value: "CISO mandate SEC-2024-047-rev2" + baseline_value: 4 + override: constrained + constraint_schema: + minimum: 1 + maximum: 8 +``` + +--- + +--- + +## 5. Precedence and Merge Rules + +When layers are merged to produce the assembled payload, fields from higher-precedence layers override fields from lower-precedence layers. The precedence order from lowest to highest is: + +``` +1. Base Layer (lowest precedence โ€” foundation defaults) +2. Core Layers (organizational and infrastructure context) +3. Intermediate/Customization (organizational hierarchy overrides) +4. Service Layers (service-specific configuration) +5. Request Layer (consumer intent โ€” overrides all data layers) +6. Transformation Policies (enrichment โ€” adds or modifies fields) +7. Validation Policies (pass/fail โ€” no field modification) +8. GateKeeper Policies (highest authority โ€” overrides everything) +``` + +### 5.1 Override Behavior + +- A higher-precedence layer that declares a field **overrides** the value from all lower-precedence layers +- A higher-precedence layer that does **not** declare a field leaves the lower-precedence value intact +- Fields not declared at any layer level are absent from the payload โ€” providers must declare all required fields as being covered by at least one layer in the chain +- GateKeeper policies can override **any** field including consumer-declared Request Layer values โ€” this is the mechanism for enforcing sovereignty constraints, security mandates, and hard compliance rules + +### 5.2 Additive vs. Override Fields + +Some fields are **scalar** (a single value โ€” one layer wins) and some are **additive** (a list or set โ€” layers contribute to a collection). The field type in the Resource Type Specification declares which behavior applies: + +```yaml +field_name: + type: + merge_behavior: + # override: higher precedence layer's value replaces lower precedence value + # additive: all layers contribute their values to a merged collection +``` + +### 5.3 Conflict Resolution + +When two layers at the same precedence level declare conflicting values for the same field: +- The conflict is recorded and surfaced as a validation error +- The request is not processed until the conflict is resolved +- Conflict resolution is never silent โ€” it is always recorded in provenance + +--- + +## 5a. Field Override Control + +Field override control is the mechanism by which DCM governs **who can change what, under what conditions**, across the layer precedence chain. It was present in the original data model rules as "override preference" metadata on fields โ€” this section formalizes that concept as a graduated model that is **simple by default and powerful when needed**. + +**Design Principle:** A field with no override declaration is fully overridable by anyone. Restrictions are always opt-in. The model has three levels โ€” you use only the level you need. Levels 1 and 2 cover the vast majority of real-world cases. Level 3 exists for fields that genuinely require nuanced, actor-specific governance. + +--- + +### 5a.1 Two Categories of Override Rule + +**Category 1 โ€” Structural Rules (Request Payload Processor โ€” non-overridable)** + +Enforced by the Request Payload Processor as DCM System behavior. Not configurable. Always applied: + +- A layer entity is immutable once versioned โ€” no override can modify a published version +- A child layer cannot remove a field declared in a parent layer โ€” it can only override the value +- The layer precedence order is fixed โ€” Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request โ†’ Policy +- Circular layer references are rejected unconditionally +- A Service Layer without a declared type scope is rejected unconditionally + +**Category 2 โ€” Business Rules (Policy Engine โ€” configurable)** + +Enforced by the Policy Engine using the Validation/Transformation/GateKeeper mechanism. Override control metadata is set exclusively by the Policy Engine and carried in the payload as part of field-level provenance. Data layers and the Request Payload Processor never set override control. + +--- + +### 5a.2 Where Override Control is Declared + +Override control can be declared at two static levels and applied dynamically at runtime: + +**Level A โ€” Resource Type Specification (portable, sets the ceiling)** +Declares the default override behavior for a field across all implementations of that Resource Type. These defaults travel with the type definition and apply to all providers and catalog items that implement the type. This sets the maximum permissiveness ceiling โ€” lower levels can only restrict further. + +**Level B โ€” Catalog Item (offering-specific, can only restrict)** +Declares additional restrictions for a specific curated offering beyond the Resource Type defaults. A "PCI Production VM" catalog item can lock `encryption_standard` to a single value even if the VM Resource Type allows a broader enum. Cannot expand beyond what the Resource Type permits. + +**Level C โ€” Policy Engine (runtime, within static bounds)** +Applies override control at request processing time based on current organizational policies. Can only restrict within the bounds established by the Catalog Item (or Resource Type if no Catalog Item restriction exists). Higher-authority policy levels (Global) can grant expansion to trusted actors within their authority scope. + +**Inheritance Rule:** Override control can only be made more restrictive as it flows down the declaration hierarchy โ€” Resource Type โ†’ Catalog Item โ†’ Runtime Policy. The sole exception is explicit trusted grants made by higher-authority actors (see Section 5a.6). + +--- + +### 5a.3 Level 1 โ€” No Declaration (Default) + +No override control declaration on a field means it is fully overridable by any actor. This is the default for all fields. Zero configuration required. + +```yaml +# Level 1 โ€” fully overridable, no declaration needed +cpu_count: + value: 4 +``` + +This covers the majority of fields in most implementations. + +--- + +### 5a.4 Level 2 โ€” Simple Declaration + +A single `override` property covers the most common governance needs without requiring a full matrix. Sufficient for most governed fields. + +```yaml +# Level 2a โ€” nobody can change this +sovereignty_zone: + value: us-east + override: immutable + +# Level 2b โ€” anyone can change but only within these values +encryption_standard: + value: AES-256 + override: constrained + constraint_schema: + enum: [AES-256, AES-128] + +# Level 2c โ€” explicit allow (same as default, but self-documenting) +display_name: + value: my-vm + override: allow +``` + +| Value | Meaning | Enforcement | +|-------|---------|-------------| +| `allow` | Default. Any actor may override. | Structural rules | +| `constrained` | Any actor may override within `constraint_schema` | Policy Engine โ€” Validation | +| `immutable` | No actor may override at any level | Policy Engine โ€” GateKeeper | + +--- + +### 5a.5 Level 3 โ€” Matrix Declaration + +Full actor-level control for fields that require nuanced governance. Used only when Level 2 is insufficient. + +```yaml +billing_tag: + value: engineering + override_matrix: + default: allow + # Default permission for any actor not explicitly listed + # Options: allow | constrained | deny + + inheritance: restrict_only + # Catalog Items and lower-level declarations can only restrict + # Higher-authority actors can grant expansion via trusted_grants + + actors: + - actor: policy.global + permission: allow + can_expand: true + # Global policies can always override and can grant expansion + # to lower actors via trusted_grants + + - actor: policy.tenant + permission: allow + can_expand: true + # Tenant policies can override and grant within global ceiling + + - actor: policy.user + permission: deny + can_expand: false + # User policies cannot override and cannot grant to others + + - actor: consumer_request + permission: constrained + constraint_schema: + pattern: "^[a-z0-9-]+$" + can_expand: false + # Consumers can override within pattern, cannot grant expansion + + - actor: process_resource + permission: deny + can_expand: false + # Automation denied by default โ€” grant via trusted_grants + + - actor: provider + permission: deny + can_expand: false + # Providers cannot modify this field + + - actor: sre_override + permission: allow + can_expand: false + # SREs have operational authority but cannot grant to others + + - actor: admin_override + permission: allow + can_expand: true + # Admins can override and grant within their scope level + + trusted_grants: + # Explicit expansion grants from higher-authority actors + # Used when an actor needs more permission than their default + - granted_to_uuid: + actor_type: process_resource + permission: allow + granted_by_policy_uuid: + reason: Patching automation trusted to update billing_tag + expires: + + constraint_schema: + pattern: "^[a-z0-9-]+$" + # Applied to all actors with permission: constrained +``` + +--- + +### 5a.6 Actor Registry + +The actor list is extensible. DCM ships with built-in actors. Organizations register custom actors following the same model. Custom actors default to `deny` until explicitly granted permissions. + +**Built-in actors:** + +| Actor | Default Scope | Can Expand | Notes | +|-------|--------------|------------|-------| +| `policy.global` | All tenants | โœ… | Highest authority โ€” can grant to any actor | +| `policy.tenant` | Single tenant | โœ… | Within global ceiling | +| `policy.user` | Single user | โŒ | Can only restrict | +| `consumer_request` | Request submitter | โŒ | Can only restrict | +| `process_resource` | Automation execution | โŒ by default | Requires trusted grant | +| `provider` | Service Provider | โŒ | Can only restrict | +| `sre_override` | SRE team | โŒ | Operational authority, cannot grant | +| `admin_override` | DCM Admin | โœ… | Within their scope level | + +**Custom actor registration:** + +```yaml +custom_actor: + uuid: + name: + description: + registered_by_tenant_uuid: + default_permission: deny + # Custom actors always default to deny until explicitly granted + can_expand: false + # Custom actors cannot expand by default โ€” requires explicit grant + version: + status: + provenance: + +``` + +Custom actors follow the universal versioning and deprecation model. A custom actor registered at Tenant scope cannot be granted Global-level authority. + +--- + +### 5a.7 Expansion Rules + +Actor expansion follows a strict hierarchy: + +- **`policy.global`** and **`admin_override`** at global scope โ€” can grant expansion to any actor for any field, including fields declared `immutable` at lower levels +- **`policy.tenant`** and **`admin_override`** at tenant scope โ€” can grant expansion within their tenant, cannot expand beyond what Global permits +- **`policy.user`**, **`consumer_request`**, **`provider`** โ€” can never grant expansion regardless of what they receive +- **`sre_override`** โ€” can never grant expansion but can be granted expansion by Tenant or Global +- **`process_resource`** โ€” denied by default, can be granted expansion by Tenant or Global via `trusted_grants` +- **Custom actors** โ€” denied by default, can be granted expansion by the level that registered them or higher + +**Trusted grants expire** โ€” if an `expires` timestamp is set, the grant is automatically revoked at that time. Expired grants are retained in provenance for audit purposes but are no longer applied. + +--- + +### 5a.8 Override Control in the Assembly Process + +Override control is applied during Step 5 (Policy Processing) of the assembly process: + +``` +Layer Merge complete (Steps 1-4) + โ”‚ Fields have values โ€” all fields default to Level 1 (allow) + โ”‚ Static override declarations from Resource Type and Catalog Item are loaded + โ–ผ +Transformation Policies + โ”‚ May set override: constrained or override_matrix on fields + โ”‚ May set baseline_value and basis_for_value metadata + โ”‚ Records policy UUID, level, and reason in field provenance + โ–ผ +Validation Policies + โ”‚ Verify existing override declarations are not violated + โ”‚ Verify actor permissions against current override_matrix + โ”‚ Pass/fail โ€” no modification to override control + โ–ผ +GateKeeper Policies + โ”‚ May set override: immutable on fields + โ”‚ May override field values before locking + โ”‚ May issue trusted_grants to specific actors + โ”‚ Records policy UUID, level, lock type, and reason in provenance + โ–ผ +Requested State + โ”‚ All governed fields carry full override control metadata + โ”‚ Provenance chain complete โ€” every lock and grant is traceable + โ–ผ +``` + +--- + +### 5a.9 Override Control and Rehydration + +During rehydration, the Intent State is replayed through the **current** Policy Engine. Override control declared in current policies is applied fresh. A field that was `allow` in the original request may be `immutable` if a new GateKeeper policy was added since. This is by design โ€” rehydration applies current governance standards, not historical ones. + +The original consumer intent is preserved unchanged in the Intent Store. The new realized state reflects current governance. Both are auditable and traceable. + +The one exception is `pinned` policy version rehydration (Historical Exact or Historical Portable modes) โ€” this deliberately replays historical policies and may bypass current immutable locks. Pinned rehydration requires elevated authorization precisely for this reason. + +--- + +### 5a.11 Global Policy Self-Override โ€” The Immutable Ceiling Model + +**Q51 resolved:** When a Global GateKeeper policy sets `override: immutable` on a field, can a higher-priority Global policy still override it? + +**The answer emerges from execution order.** Policies execute highest-priority-first (highest numeric value first within a tier). The first policy to set `override: immutable` on a field locks it. All subsequent policies โ€” including other Global policies with lower priority values โ€” find the field locked and cannot modify it. In normal request processing, **default `immutable` is effectively absolute** โ€” not through a special rule, but through execution order. + +**The `immutable_ceiling` declaration** is a forward-looking protection for fields that must remain locked even if a higher-priority policy is **added to the system later**: + +```yaml +# Default immutable โ€” protected by execution order during this request +# The highest-priority Global GateKeeper to run first locks it +# No subsequent policy in this execution can change it +sovereignty_zone: + value: eu-west + override: immutable + # Safe in practice โ€” execution order guarantees the first-runner wins + # Does NOT protect against a new higher-priority policy being added tomorrow + +# Absolute immutable โ€” explicit forward-looking protection +# Protected even if a new higher-priority policy is added to the system +classification_level: + value: RESTRICTED + override: immutable + immutable_ceiling: absolute + # Cannot be overridden by ANY policy, ever + # If a policy attempts to override this, it receives a hard rejection + # The attempted override is logged in audit with full provenance + # Use for: sovereignty zones, data classification, hard compliance mandates +``` + +**The formal rule:** + +| Declaration | Protected During Execution? | Protected Against Future Policies? | Use Case | +|-------------|----------------------------|-------------------------------------|---------| +| `override: immutable` (default) | โœ… Yes โ€” execution order | โŒ No | Most governed fields | +| `override: immutable` + `immutable_ceiling: absolute` | โœ… Yes | โœ… Yes โ€” hard rejection | True non-negotiables | + +**`immutable_ceiling: absolute` is the nuclear option.** Use it sparingly โ€” only for fields where the governance requirement is genuinely non-negotiable regardless of any future organizational policy change. Sovereignty zone on a sovereign deployment. Data classification on a restricted system. Encryption standard under a regulatory mandate with no variance permitted. + +**Audit behavior:** When a policy attempts to override a field with `immutable_ceiling: absolute`, the attempt is rejected silently from the requesting policy's perspective (the field simply doesn't change) but is fully logged in the Audit Store with the policy UUID, the attempted value, the rejection reason, and the UUID of the policy that set the ceiling. + +--- + +### 5a.10 Override Control Metadata โ€” Full Structure + +The complete field metadata structure carrying override control in the payload: + +```yaml +field_name: + value: + metadata: + # Simple declaration (Level 2) โ€” set by Policy Engine at runtime + override: + # OR matrix declaration (Level 3) โ€” set by Policy Engine at runtime + override_matrix: + + + # Always present regardless of level + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + constraint_schema: + + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_uuid: + operation_type: + actor: + timestamp: + reason: +``` + +--- + +The Request Payload Processor assembles the final payload by executing the following steps in order. Each step is recorded in the payload's provenance chain. + +### Step 1 โ€” Intent Capture +The consumer's Request Layer is received and stored as the **Intent State** in the Intent Store. No modification occurs at this step. The Intent State is the immutable record of what the consumer asked for. + +### Step 2 โ€” Layer Resolution +The Request Payload Processor determines which layers apply to this request: +- Identifies the Resource Type from the Request Layer +- Retrieves the applicable Base Layer for the request context +- Retrieves all applicable Core Layers (type-agnostic โ€” all apply) +- Retrieves applicable Intermediate/Customization Layers based on organizational context +- Retrieves applicable Service Layers whose declared type scope matches the request Resource Type +- Orders all retrieved layers according to the precedence chain + +### Step 3 โ€” Layer Merge +Layers are merged in precedence order (lowest to highest). For each field: +- The value from the highest-precedence layer that declares it is used +- The source layer UUID and layer type are recorded in the field's provenance metadata +- Additive fields accumulate values from all layers that declare them + +### Step 4 โ€” Request Layer Application +The consumer's Request Layer is applied last in the data layer merge. Consumer-declared values override all data layer values. Each override is recorded in provenance. + +### Step 5 โ€” Policy Processing +Policies are applied to the merged payload in order. Each policy step may read and set `override_control` metadata on fields โ€” see Section 5a for full detail. +1. **Transformation Policies** โ€” enrich and modify the payload. May set `override_preference: constrained` on fields, declaring that future overrides must satisfy a constraint schema. Each transformation records the policy UUID, operation type, reason, and any override control declarations in the affected fields' provenance. +2. **Validation Policies** โ€” check the payload against rules and verify that existing `override_preference` declarations have not been violated. Failures reject the request with a detailed reason. No field modification occurs. +3. **GateKeeper Policies** โ€” apply hard overrides and blocks. May set `override_preference: immutable` on fields, permanently locking them against further modification. GateKeeper overrides record the policy UUID, the overridden value, the new value, the lock type, and the reason in provenance. + +### Step 6 โ€” Requested State Storage +The fully assembled, policy-processed payload is stored as the **Requested State** in the Request Store. This is the complete, provider-ready payload with full provenance chain intact. + +### Step 7 โ€” Provider Dispatch +The Requested State payload is dispatched to the appropriate Service Provider via the API Gateway. + +--- + +## 7. Layer Assembly Diagram + +``` +Consumer Request + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REQUEST LAYER โ”‚ โ† Consumer declared intent โ†’ stored as INTENT STATE +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ LAYER RESOLUTION โ”‚ +โ”‚ โ”‚ +โ”‚ Base Layer (lowest precedence) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Core Layers (type-agnostic context) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Intermediate Layers (organizational context) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Service Layers (type-scoped service config) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Request Layer (consumer intent โ€” highest โ”‚ +โ”‚ data layer precedence) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Merged payload with full provenance + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ POLICY PROCESSING โ”‚ +โ”‚ โ”‚ +โ”‚ Transformation Policies (enrich / modify) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Validation Policies (pass / fail check) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ GateKeeper Policies (override / block) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Complete, validated, policy-processed payload + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REQUESTED STATE โ”‚ โ† Stored in Request Store +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ–ผ + Service Provider +``` + +--- + +## 8. Layer Scope and Type Enforcement + +### 8.1 Core Layer Scope Enforcement +Core Layers are type-agnostic by default. They are applied to every request regardless of Resource Type. A Core Layer that contains service-specific or provider-specific data is invalid and must be rejected. + +### 8.2 Service Layer Scope Enforcement +Service Layers must declare a Resource Type scope. The Request Payload Processor enforces this during Layer Resolution: +- A Service Layer whose declared Resource Type does not match the request Resource Type is excluded from the merge +- A Service Layer with `scope_inheritance: exact` is only included if the request Resource Type exactly matches the declared type +- A Service Layer with `scope_inheritance: descendants` is included if the request Resource Type is the declared type or any descendant type in the inheritance hierarchy +- A Service Layer with no declared type scope is invalid and must be rejected + +### 8.3 Unanticipated Data Interaction Prevention +The type scoping rules for Service Layers are the primary mechanism for preventing unanticipated data interactions โ€” one of the core data model objectives. Because Service Layers can only contribute to requests of their declared type, data from one service domain cannot inadvertently affect requests in another service domain. + +--- + +## 9. Layer Versioning + +All layers follow the universal DCM versioning scheme: **Major.Minor.Revision** + +| Component | Trigger | +|-----------|---------| +| **Major** | Breaking changes โ€” removing fields, changing field types, changing a field from optional to required | +| **Minor** | Additive changes โ€” adding new optional fields, adding new contextual data | +| **Revision** | Data/configuration changes โ€” updating field values, updating descriptions, updating metadata | + +**Immutability:** Once a layer version is published it cannot be modified. Any change produces a new version. Previous versions remain accessible and can be referenced by existing realized entities. + +**Parent Chain Versioning:** A layer's parent chain references specific versions of parent layers. Updating a parent layer does not automatically update child layers โ€” child layers must be explicitly updated to reference the new parent version, producing a new version of the child layer. + +--- + +## 10. Artifact Lifecycle โ€” The Five Statuses + +All DCM artifacts โ€” layers, policies, resource types, catalog items, and all other defined objects โ€” follow a five-status lifecycle. The statuses are defined in Section 4b.2. For layers specifically: + +| Status | Layer Behavior | +|--------|---------------| +| `developing` | Layer is in active development. Only usable in development mode pipelines. Not loaded by the assembly process in production. | +| `proposed` | Layer has been submitted for review (PR open). Not yet active. For policy layers: shadow execution runs. For data layers: layer is visible in the registry but not applied. Cannot merge to active until PR is approved. | +| `active` | Layer is current and applied in assembly. Can be included in new layer chains. | +| `deprecated` | Layer is being phased out. Existing chains using it continue to function. New chains should use the replacement. Deprecation warning recorded in assembly provenance. Must include replacement UUID, reason, migration guidance, and sunset date. | +| `retired` | Layer cannot be included in new layer chains. Existing realized entities that reference it retain the reference for audit purposes but cannot be used for new requests. | + +**Status transition rules for layers:** +``` +developing โ†’ proposed (author submits PR) +developing โ†’ retired (author abandons) +proposed โ†’ active (PR merged โ€” approval complete) +proposed โ†’ developing (PR returned for rework) +active โ†’ deprecated (replacement available โ€” sunset declared) +deprecated โ†’ retired (sunset date reached or manual retirement) +``` + +--- + +## 11. Scale Example โ€” 40,000 Linux VMs + +This example illustrates the power of the layering model at scale. 40,000 distinct VM configurations are governed by 36 layer definitions: + +``` +Base Entity (3 variants) +โ”œโ”€โ”€ CIS Benchmark +โ”œโ”€โ”€ Baseline +โ””โ”€โ”€ DMZ / Payments + + โ””โ”€โ”€ Layer Entity โ€” OS Family (3 variants per base = 9 total) + โ”œโ”€โ”€ Common Linux Config / RHEL + โ”œโ”€โ”€ Common Linux Config / CoreOS + โ””โ”€โ”€ Common Linux Config / OEL + + โ””โ”€โ”€ Layer Entity โ€” OS Version (4 variants per OS layer = 36 total) + โ”œโ”€โ”€ RHEL 6 + โ”œโ”€โ”€ RHEL 7 + โ”œโ”€โ”€ RHEL 8 + โ””โ”€โ”€ RHEL 9 + + โ””โ”€โ”€ Realized Entity โ€” one per VM (40,000 total) + Each realized entity carries FK references to its + full layer chain (Base UUID + Layer UUIDs) + and is stored in the CMDB +``` + +**Result:** 3 ร— 3 ร— 4 = **36 layer definitions** govern **40,000 VM configurations**. Each VM's realized entity is a lightweight reference to its layer chain โ€” not a copy of all the configuration data. + +This also means: +- Updating the CIS Benchmark base layer creates one new layer version that cascades to all 40,000 VMs at their next realization +- Drift detection compares each VM's discovered state against its realized entity's layer chain +- Any VM can be reproduced exactly by replaying its layer chain through the assembly process + +--- + +## 12. Relationship to the Four States + +| Layer | State Relationship | +|-------|-------------------| +| Request Layer (as submitted) | Directly captured as **Intent State** โ€” stored in Intent Store before any processing | +| Assembled payload (post-merge, pre-policy) | Intermediate โ€” not a named state, internal to assembly process | +| Assembled payload (post-policy) | Becomes **Requested State** โ€” stored in Request Store | +| Provider execution result | Becomes **Realized State** โ€” stored in Realized Store | +| Discovery interrogation result | Becomes **Discovered State** โ€” stored in Discovered Store | + +The layer chain of a Realized Entity is always traceable โ€” given a Realized State record, the complete layer chain that produced it can be reconstructed, providing full audit capability back to the original Base Layer. + +--- + +## 13. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are conflicting Service Layers at the same precedence level resolved? | Assembly determinism | โœ… Resolved โ€” priority schema + conflict detection at ingestion | +| 2 | Should Core Layers be ordered within their precedence level? | Merge determinism | โœ… Resolved โ€” priority schema provides deterministic ordering | +| 3 | Can a consumer explicitly exclude a layer from their request? | Consumer control vs. standardization | โ“ Unresolved | +| 4 | How are Service Layers registered and versioned relative to Service Provider registration? | Provider contract | โ“ Unresolved | +| 5 | Should assembly support conditional layer inclusion? | Assembly flexibility | โ“ Unresolved | +| 6 | How does the layer chain interact with service dependencies? | Dependency model | โ“ Unresolved | +| 7 | Should `override_preference` be declarable in layer definitions as a hint to the Policy Engine? | Override control | โ“ Unresolved | +| 8 | When `override_preference: immutable` is set by a Global policy, can a higher-priority Global policy still override it? | Override control precedence | โ“ Unresolved | +| 9 | Should the `constraint_schema` on a constrained field be visible to consumers in the Service Catalog UI? | Consumer experience | โ“ Unresolved | +| 10 | Should the background validation job for detecting post-ingestion conflicts run on a schedule or be event-triggered? | Operational | โ“ Unresolved | +| 11 | What is the minimum validation review period for a proposed policy before it can be activated? | Policy governance | โ“ Unresolved | + +--- + +## 14. Related Concepts + +- **Request Payload Processor** โ€” the control plane component that executes the assembly process; enforces structural layer rules +- **Policy Engine** โ€” executes Policy Layers (Validation, Transformation, GateKeeper) during the assembly process; the sole authority for setting field override control +- **Field Override Control** โ€” the mechanism governing who can change what field, under what conditions, at what policy level +- **Override Preference** โ€” per-field metadata declaring `allow`, `constrained`, or `immutable` โ€” the formalization of the original data model "override preference" subtag +- **Service Layer Cache** โ€” caches Service Layer data at Service Provider registration time for efficient retrieval during assembly +- **Core Layer Store** โ€” stores all Core Layer definitions +- **Intent State** โ€” the Request Layer as submitted, before assembly +- **Requested State** โ€” the fully assembled, policy-processed payload +- **Field-Level Provenance** โ€” every field in the assembled payload records which layer set it and which policy modified it +- **Resource Type Hierarchy** โ€” defines the type scope that Service Layers must declare and that the assembly process enforces +- **GitOps** โ€” all layers are stored in Git, versioned and immutable + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md new file mode 100644 index 0000000..d90db24 --- /dev/null +++ b/content/docs/architecture/data-model/resource-grouping.md @@ -0,0 +1,299 @@ +--- +title: "Resource Grouping" +type: docs +weight: 7 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Resource/Service Entities](../resource-service-entities/) | [Service Dependencies](../service-dependencies/) + +--- + +## 1. Purpose + +This document defines how Resource/Service Entities are organized into groups within DCM. Grouping provides the ownership, organizational context, cost attribution, policy scope, and rehydration targeting that makes DCM operationally meaningful at scale. + +Two concepts are defined here: +1. **DCM Tenant** โ€” the mandatory, first-class ownership boundary for all Resource/Service Entities +2. **Resource Groups** โ€” flexible, composable grouping entities that provide additional organizational context + +--- + +## 2. DCM Tenant + +### 2.1 Definition + +A **DCM Tenant** is the primary ownership and isolation boundary for Resource/Service Entities in DCM. Every Resource/Service Entity โ€” including Process Resources โ€” must belong to exactly one DCM Tenant at any point in time. + +Tenant membership is the answer to the question: **who owns this resource?** + +### 2.2 Tenant as a DCM System Policy + +Mandatory Tenant membership is a **non-overridable DCM System Policy**: + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `TEN-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation โ€” no Tenant = request rejected | +| `TEN-002` | Tenant membership cannot be empty โ€” a Tenant must exist before resources can be created in it | Enforced at request processing | +| `TEN-003` | A Resource/Service Entity cannot exist without a Tenant | Enforced at all lifecycle states | + +### 2.3 What Tenant Provides + +The Tenant boundary enables the following DCM capabilities for all resources it owns: + +| Capability | Description | +|------------|-------------| +| **Ownership** | Unambiguous answer to "who owns this resource" โ€” always answerable, always auditable | +| **Isolation** | Resources in one Tenant are isolated from resources in another โ€” hard tenancy enforcement | +| **Cost Attribution** | All resource costs roll up to the owning Tenant | +| **Policy Scope** | Tenant-level policies apply to all resources in the Tenant | +| **Drift Detection Scope** | Drift detection can be scoped to a Tenant | +| **Rehydration Scope** | A full Tenant can be targeted for rehydration | +| **Audit Scope** | All activity within a Tenant is auditable as a unit | +| **Sovereignty Boundary** | Sovereignty constraints can be applied at the Tenant level | + +### 2.4 Tenant Entity Definition + +```yaml +dcm_tenant: + uuid: + name: + description: + version: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + ownership: + owner_uuid: + owner_type: + created_timestamp: + membership_policy: + exclusive: true + # A resource belongs to exactly one Tenant + # This is non-overridable + sovereignty_constraints: + + policies: + + provenance: + +``` + +### 2.5 Tenant and Resource Consumption + +A resource belongs to exactly one Tenant โ€” its **owner**. However, a resource can be **consumed** by multiple Tenants via the DCM Service Catalog. Ownership and consumption are distinct: + +- **Ownership** (Tenant membership) โ€” who is responsible for the lifecycle, cost, and compliance of this resource +- **Consumption** โ€” who uses or depends on this resource as a service + +Cross-tenant consumption is tracked through service requests and cost attribution โ€” not through Tenant membership. A shared DNS service owned by a Platform Tenant can be consumed by any number of application Tenants. The DNS Entity belongs to the Platform Tenant. Consumption is tracked via service requests from each consuming Tenant. + +--- + +## 3. Resource Groups + +### 3.1 Definition + +A **Resource Group** is a flexible, composable grouping entity that provides organizational context, operational scope, and policy targeting beyond what Tenant membership provides. + +Resource Groups function like **structured tags** โ€” a resource accumulates group memberships that describe its context from multiple dimensions simultaneously. A VM could simultaneously belong to: +- `Deployment: WebApp-v2` (what deployment it is part of) +- `BusinessUnit: Payments` (which business unit owns the workload) +- `RegulatoryScope: PCI-DSS` (which compliance regime applies) +- `CostCenter: CC-4421` (where costs are attributed) + +Each group membership is a different dimension of context โ€” not a hierarchy within a single dimension. + +### 3.2 Resource Group Classes + +DCM defines two classes of Resource Group, both implementing the same **Resource Group Interface**: + +**Class 1 โ€” DCM Default Resource Group** +Built into DCM. The standard mechanism for grouping resources. No implementor customization required to use it. + +**Class 2 โ€” Custom Resource Group** +Implementor-defined grouping entities. Tied to internal business structures โ€” business units, product lines, regulatory scopes, cost centers, etc. Full parity with DCM Default Resource Groups in terms of DCM capabilities. + +Both classes implement the same interface. The DCM Default Resource Group is simply DCM's own implementation of the Resource Group Interface. Custom groups are implementor-defined implementations of the same interface. + +### 3.3 The Resource Group Interface + +Every Resource Group โ€” both DCM default and custom โ€” must implement this interface: + +```yaml +resource_group: + uuid: + name: + description: + group_class: + group_type: + version: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + nesting: + supported: + # If true, this group can contain other groups as members + max_depth: + membership: + members: + - member_uuid: + member_type: + joined_timestamp: + joined_by_uuid: + membership_policy: + exclusive: + # If true, a resource can only belong to one group of this type at a time + # If false, a resource can belong to multiple groups of this type + max_memberships: + # Maximum number of groups of this type a resource can belong to + allowed_entity_types: + + policies: + + provenance: + +``` + +### 3.4 Multi-Group Membership + +A Resource/Service Entity can belong to multiple Resource Groups across all classes. This multi-dimensional membership is what gives groups their tag-like flexibility. + +**Membership constraints are configurable per group definition:** +- A group can declare `exclusive: true` โ€” meaning a resource can only belong to one group of that type at a time +- Example: A `RegulatoryScope` group might declare `exclusive: true` โ€” a resource cannot be in both EU-GDPR and US-FISMA regulatory scopes simultaneously +- Example: A `Deployment` group might declare `exclusive: false` โ€” a resource could participate in multiple deployments + +**Policy-governed membership:** +Organizational policies can further restrict multi-group membership. For example, a sovereignty policy could declare that resources in a PCI-DSS scope cannot be in the same group as resources in a non-PCI scope. + +### 3.5 Nesting + +Resource Groups that declare `nesting: true` can contain other Resource Groups as members in addition to individual Resource/Service Entities. + +**Example nesting structure:** +``` +Tenant: Payments Platform + โ”‚ + โ””โ”€โ”€ Resource Group: Deployment โ€” WebApp-v2 (nesting: true) + โ”œโ”€โ”€ Resource Group: Service โ€” Frontend (nesting: true) + โ”‚ โ”œโ”€โ”€ Entity: Web Server VM 1 + โ”‚ โ”œโ”€โ”€ Entity: Web Server VM 2 + โ”‚ โ””โ”€โ”€ Entity: Load Balancer + โ””โ”€โ”€ Resource Group: Service โ€” Backend (nesting: true) + โ”œโ”€โ”€ Entity: App Server VM 1 + โ”œโ”€โ”€ Entity: App Server VM 2 + โ””โ”€โ”€ Entity: Database +``` + +**Nesting rules:** +- Circular nesting is invalid โ€” a group cannot contain itself directly or transitively +- Nesting depth is declared per group โ€” `max_depth: unlimited` allows arbitrary depth +- A child group inherits policy scope from parent groups โ€” policies applied to a parent group propagate to all child groups and their members +- Cost rollup propagates up the nesting hierarchy + +--- + +## 4. DCM System Policies for Resource Grouping + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `GRP-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation | +| `GRP-002` | A Resource/Service Entity cannot be removed from its Tenant without being transferred to another Tenant | Enforced at all lifecycle states | +| `GRP-003` | Circular nesting in Resource Groups is invalid | Enforced at group membership modification | +| `GRP-004` | Custom Resource Groups must implement the full Resource Group Interface | Enforced at group registration | +| `GRP-005` | Exclusive membership groups must reject membership requests that violate exclusivity | Enforced at group membership addition | + +--- + +## 5. Grouping and DCM Capabilities + +Resource Groups enable the following DCM capabilities at the group scope: + +| Capability | Tenant | Resource Group | +|------------|--------|---------------| +| Cost Attribution | โœ… Primary | โœ… Rollup within group | +| Policy Scope | โœ… | โœ… | +| Drift Detection Scope | โœ… | โœ… | +| Rehydration Scope | โœ… Full Tenant | โœ… Group and dependencies | +| Audit Scope | โœ… | โœ… | +| Placement Constraints | โœ… | โœ… | +| Sovereignty Boundary | โœ… | โœ… | + +--- + +## 6. Process Resources and Grouping + +Process Resource Entities follow the same grouping rules as Infrastructure Resource Entities: + +- Must belong to exactly one DCM Tenant โ€” non-overridable +- Can optionally belong to Resource Groups +- Typically grouped under the same Deployment or Service group as the resources they operate on +- Tenant membership ensures cost attribution for execution resources +- Group membership enables operational scoping โ€” "show me all automation jobs that ran against this Deployment" + +--- + +## 7. Custom Resource Group Registration + +Implementors register custom Resource Group types as part of their DCM implementation. Custom group types must declare their full interface implementation: + +```yaml +custom_group_type_registration: + uuid: + type_name: + version: + description: + implementing_organization_uuid: + interface_version: + default_membership_policy: + exclusive: + max_memberships: + nesting_supported: + allowed_entity_types: + status: + state: +``` + +--- + +## 8. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should there be a DCM-maintained registry of well-known custom group types to encourage standardization? | Interoperability | โ“ Unresolved | +| 2 | How does group membership interact with sovereignty โ€” can a group span sovereignty boundaries? | Sovereignty model | โ“ Unresolved | +| 3 | When a Tenant is decommissioned, what happens to its resources and group memberships? | Lifecycle management | โ“ Unresolved | +| 4 | Should Resource Groups support time-bounded membership โ€” a resource belongs to a group for a defined period? | Operational flexibility | โ“ Unresolved | +| 5 | How are group-level policies inherited by nested child groups โ€” is inheritance opt-in or opt-out? | Policy model | โ“ Unresolved | + +--- + +## 9. Related Concepts + +- **DCM Tenant** โ€” primary ownership boundary, mandatory for all entities +- **Resource/Service Entity** โ€” the thing being grouped +- **Policy Engine** โ€” enforces grouping system policies and evaluates group-level organizational policies +- **Cost Analysis** โ€” rolls up costs through group hierarchies +- **Drift Detection** โ€” can be scoped to a group +- **Rehydration** โ€” can target a group as the unit of reconstruction +- **Field-Level Provenance** โ€” group membership changes are recorded in entity provenance + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md new file mode 100644 index 0000000..81c1707 --- /dev/null +++ b/content/docs/architecture/data-model/resource-service-entities.md @@ -0,0 +1,479 @@ +--- +title: "Resource and Service Entities" +type: docs +weight: 5 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Layering and Versioning](../layering-and-versioning/) | [Resource Type Hierarchy](../resource-type-hierarchy/) | [Service Dependencies](../service-dependencies/) | [Resource Grouping](../resource-grouping/) + +--- + +## 1. Purpose + +This document defines the two fundamental transactional concepts in DCM โ€” the **Resource/Service Request** and the **Resource/Service Entity** โ€” and establishes the ownership models, lifecycle principles, and provider relationship rules that govern them. + +Understanding the distinction between a Request and an Entity, and understanding DCM's role as the authoritative owner of all resource data regardless of operational ownership, is essential to understanding how DCM achieves its core goals of auditability, lifecycle management, and sovereignty. + +--- + +## 2. Core Terminology + +### 2.1 Resource/Service Request + +A **Resource/Service Request** is what a consumer submits to DCM โ€” the declared intent to consume a resource or service. It is the consumer side of the transaction. + +- Created when a consumer submits a request via the Web UI or Consumer API +- Captured as the **Intent State** before any processing +- Processed into the **Requested State** after assembly and policy validation +- Is the initiating event that causes a Resource/Service Entity to be created + +A Request is not a thing โ€” it is an **instruction**. It describes what the consumer wants. The provider acts on the Requested State to produce an Entity. + +### 2.2 Resource/Service Entity + +A **Resource/Service Entity** is the "thing" produced by a provider as a result of fulfilling a Resource/Service Request. It is the provider side of the transaction โ€” the allocation made real. + +- Created when a provider fulfills a Requested State payload +- Returned to DCM in unified data model format via Denaturalization +- Captured as the **Realized State** in the Realized Store +- Assigned to a **DCM Tenant** โ€” the ownership boundary +- Has a UUID, full provenance chain, and complete lifecycle from creation to decommission +- Is the unit of consumption, cost attribution, drift detection, and audit in DCM + +A Resource/Service Entity IS a thing โ€” it exists, it has state, it has an owner, and DCM manages its lifecycle. + +### 2.3 The Critical Distinction + +``` +Consumer submits โ†’ Resource/Service REQUEST โ†’ Intent/Requested State +Provider fulfills โ†’ Resource/Service ENTITY โ†’ Realized State +DCM manages lifecycle โ†’ ENTITY persists โ†’ Drift/Audit/Cost/Rehydration +``` + +--- + +## 3. DCM as Authoritative Owner of All Resource Data + +This is the most fundamental principle governing Resource/Service Entities: + +**DCM is ALWAYS the system of record for Resource/Service Entity data. DCM is ALWAYS authoritative for the resource definition. DCM ALWAYS owns the lifecycle. This applies regardless of the operational ownership model.** + +The operational ownership model (described in Section 4) determines who has authority to operate on a Resource/Service Entity. It does not affect DCM's data ownership. Specifically: + +- DCM owns the **data definition** of every Resource/Service Entity โ€” what it is, what it should be, what it was +- DCM owns the **lifecycle** โ€” from Requested through Realized to Decommissioned +- DCM is **authoritative** โ€” if a provider reports a change DCM was not aware of, DCM acts on it according to policy +- DCM acts as the **Tenant advocate** โ€” it protects the Tenant's interests in all provider interactions +- Providers are **custodians** of the underlying infrastructure โ€” they are not the system of record + +**When a provider reports an unsanctioned change:** + +If a provider reports a state change that was not initiated by a DCM request, the Policy Engine evaluates the change and determines the appropriate response: + +| Response | Description | +|----------|-------------| +| `ALERT` | Notify appropriate personas โ€” Tenant owner, SRE, Auditor | +| `REVERT` | Instruct provider to revert to DCM-declared realized state | +| `UPDATE_DEFINITION` | Accept the change and update the realized state definition | +| `INVESTIGATE` | Flag for human review before action | +| `DECOMMISSION` | Initiate decommission if the change represents unrecoverable deviation | +| `ESCALATE` | Escalate to higher policy tier for decision | + +The response is determined by Policy Engine evaluation against: +- The Resource/Service definition +- Service/Resource dependencies +- Consumer preferences +- Organizational and Tenant policies +- Sovereignty requirements + +--- + +## 4. Ownership Models + +DCM supports four ownership models for Resource/Service Entities. Every Provider Catalog Item must declare which ownership model(s) it supports. The ownership model is recorded in the Resource/Service Entity's provenance at creation time. + +### 4.1 Allocation Model + +The provider retains internal ownership of the underlying infrastructure. The consumer owns the Resource/Service Entity (the allocation) in their DCM Tenant. The provider can reclaim the underlying resource when the entity is decommissioned. + +**Characteristics:** +- Provider retains asset ownership +- Consumer owns the allocation โ€” the Entity in their DCM Tenant +- Provider has reclaim rights on decommission +- Underlying infrastructure may be shared or subdivided +- DCM manages the Entity lifecycle; provider manages the underlying resource + +**Examples:** Virtual Machine, Container, Network Port, IP Address, Firewall Rule, Database Instance + +--- + +### 4.2 Whole Allocation Model + +The entire physical or logical resource is allocated as a single indivisible unit to one consumer's DCM Tenant. The provider retains internal ownership but the consumer has exclusive use of the whole resource. The resource cannot be subdivided or shared during the allocation period. + +**Characteristics:** +- Provider retains asset ownership +- Consumer has exclusive, indivisible use +- The resource is not shared or subdivided +- Provider has reclaim rights on decommission +- DCM manages the Entity lifecycle + +**Examples:** Dedicated Bare Metal server (provider-owned), Dedicated Network appliance, Whole storage array allocation + +--- + +### 4.3 Full Transfer Model + +The provider transfers complete ownership of the underlying resource to the consumer's DCM Tenant. The Resource/Service Entity IS the resource โ€” there is no separation between the allocation and the underlying infrastructure from DCM's perspective. The consumer controls the full lifecycle including decommissioning. The provider has no reclaim rights after transfer. + +**Characteristics:** +- Ownership of the underlying resource transfers to consumer's DCM Tenant +- The Entity IS the resource โ€” no allocation/infrastructure separation +- Consumer controls full lifecycle including decommission +- Provider has no reclaim rights post-transfer +- Transfer is recorded in provenance โ€” permanent audit record +- DCM remains authoritative for data and lifecycle regardless of transfer + +**Examples:** Transferred Bare Metal server, Licensed software asset, Dedicated hardware appliance transferred to consumer + +--- + +### 4.4 Hybrid Transfer Model + +Ownership can transfer multiple times across the lifecycle of the Resource/Service Entity. The current owner is always exactly one DCM Tenant, but ownership can be formally reassigned through a DCM-governed ownership transfer process. Every transfer is tracked, auditable, and policy-governed. + +**Characteristics:** +- Ownership is held by exactly one DCM Tenant at any point in time +- Ownership can be transferred to another DCM Tenant through a formal DCM process +- Every transfer is recorded in the Entity's provenance chain โ€” complete ownership history +- Transfer requires Policy Engine validation and authorization +- The receiving Tenant must accept the transfer โ€” it cannot be forced +- DCM remains authoritative for data and lifecycle through all transfers + +**Transfer Provenance Record:** +```yaml +ownership_transfer: + sequence: + from_tenant_uuid: + to_tenant_uuid: + transfer_timestamp: + authorized_by: + transfer_reason: + policy_uuid: +``` + +**Examples:** Bare Metal server reallocated between tenants, Hardware asset transferred between business units, Licensed resource reassigned + +--- + +### 4.5 Ownership Model Declaration + +Every Provider Catalog Item must declare the ownership model(s) it supports: + +```yaml +catalog_item: + uuid: + ownership_models_supported: + - allocation + - whole_allocation + - full_transfer + - hybrid_transfer + default_ownership_model: + transfer_policy_required: + # If true, a policy must be referenced in any transfer request +``` + +--- + +## 5. Resource/Service Entity Lifecycle + +Every Resource/Service Entity progresses through a defined lifecycle. The lifecycle states are: + +``` +REQUESTED โ†’ PENDING โ†’ PROVISIONING โ†’ REALIZED โ†’ OPERATIONAL + โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ โ–ผ + DEGRADED MAINTENANCE SUSPENDED + โ”‚ โ”‚ โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ–ผ + DECOMMISSIONING + โ”‚ + โ–ผ + DECOMMISSIONED +``` + +| State | Description | +|-------|-------------| +| `REQUESTED` | Request submitted, Intent State captured | +| `PENDING` | Requested State assembled, awaiting provider dispatch | +| `PROVISIONING` | Provider is fulfilling the request | +| `REALIZED` | Provider has fulfilled the request, Entity exists, Realized State captured | +| `OPERATIONAL` | Entity is in active use | +| `DEGRADED` | Entity is functioning but below expected operational characteristics | +| `MAINTENANCE` | Entity is undergoing planned maintenance | +| `SUSPENDED` | Entity is temporarily suspended โ€” not operational but not decommissioned | +| `DECOMMISSIONING` | Decommission process initiated | +| `DECOMMISSIONED` | Entity no longer exists. Record retained permanently for audit. | + +**Terminal states:** `DECOMMISSIONED` is the only terminal state. Once decommissioned, the Entity record is immutable and retained permanently. + +--- + +## 6. Process Resource Entities + +A **Process Resource Entity** is a distinct class of Resource/Service Entity representing ephemeral execution resources โ€” automation jobs, playbooks, pipelines, workflows, and similar process-oriented resources. + +### 6.1 Characteristics + +- **Ephemeral lifecycle** โ€” exists for the duration of execution, then terminates +- **No ongoing realized state to manage** โ€” lifecycle ends at COMPLETED or FAILED +- **Execution record retained permanently** โ€” the record of what the process did is immutable and permanent +- **Must belong to a DCM Tenant** โ€” even ephemeral resources must be owned +- **Must be in the provenance chain** of any Resource/Service Entity they affect + +### 6.2 Process Resource Lifecycle + +``` +REQUESTED โ†’ INITIATED โ†’ EXECUTING โ†’ COMPLETED + โ†’ FAILED + โ†’ CANCELLED +``` + +| State | Description | +|-------|-------------| +| `REQUESTED` | Process request submitted | +| `INITIATED` | Provider has begun execution | +| `EXECUTING` | Process is actively running | +| `COMPLETED` | Process completed successfully โ€” terminal | +| `FAILED` | Process failed โ€” terminal | +| `CANCELLED` | Process cancelled before completion โ€” terminal | + +All terminal states are permanent. The execution record is immutable after reaching a terminal state. + +### 6.3 Process Resource Entity Data Model + +```yaml +process_resource_entity: + uuid: + entity_class: process + process_type: + tenant_uuid: + version: + lifecycle_state: + input_payload: + + output_payload: + + affected_entities: + - entity_uuid: + effect_type: + effect_description: + execution_record: + initiated_timestamp: + completed_timestamp: + executing_provider_uuid: + authorized_by_policy_uuid: + provenance: + +``` + +### 6.4 Provenance Obligation for Process Resources + +If a Process Resource modifies the state of a Resource/Service Entity, that Entity's realized state provenance MUST reference the Process Resource Entity UUID as the source of the modification. This ensures that every change to an Infrastructure Entity can be traced back to the Process that caused it. + +--- + +## 7. Provider Internal Lifecycle Model + +Providers have their own internal infrastructure that underpins the Resource/Service Entities they create. While that internal infrastructure is opaque to consumers, DCM needs visibility into it for placement, cost analysis, and operational governance. + +### 7.1 Provider Capacity Model + +DCM supports three capacity information modes. Mode 3 is mandatory for all providers. Modes 1 and 2 are configurable per provider registration. + +**Mode 1 โ€” Dynamic Query (on-demand)** +DCM queries the provider for current capacity as part of request processing. Used when real-time accuracy is critical or when the provider cannot maintain a registration schedule. + +```yaml +capacity_query_response: + provider_uuid: + resource_type_uuid: + location_uuid: + query_timestamp: + available_capacity: + reserved_capacity: + committed_capacity: + sovereignty_capabilities: +``` + +**Mode 2 โ€” Provider Registration (scheduled, preferred)** +Provider registers capacity data with DCM on a configurable schedule. DCM maintains an internal capacity rating per provider, per Resource Type, per location. Default minimum update frequency: twice daily. Update frequency is configurable per provider registration. + +```yaml +capacity_registration: + provider_uuid: + registration_timestamp: + next_scheduled_registration: + capacity_by_resource_type: + - resource_type_uuid: + location_uuid: + available_capacity: + reserved_capacity: + committed_capacity: + sovereignty_capabilities: +``` + +**Mode 3 โ€” Provider Denial (reactive, mandatory)** +The provider validates it can fulfill a request before executing. If it cannot, it denies the request with reason `INSUFFICIENT_RESOURCES`. DCM receives the denial and can retry with an alternative provider. The denial triggers an immediate update to DCM's internal capacity rating for that provider. + +```yaml +provider_denial: + provider_uuid: + request_uuid: + denial_reason: INSUFFICIENT_RESOURCES + denial_timestamp: + resource_type_uuid: + location_uuid: + estimated_available_at: +``` + +### 7.2 Provider Lifecycle Events + +Any provider event that affects Resource/Service Entity availability or operational characteristics MUST be reported to DCM immediately. Providers have a contractual obligation to report these events โ€” this is non-negotiable. + +**Reportable Event Types:** + +| Event Type | Description | DCM Response | +|------------|-------------|--------------| +| `CAPACITY_CHANGE` | Available capacity increased or decreased | Update internal capacity rating | +| `DEGRADATION` | Underlying resource is degraded | Policy Engine evaluation โ†’ ALERT/REVERT/ESCALATE | +| `MAINTENANCE_SCHEDULED` | Planned maintenance window declared | Policy Engine evaluation โ†’ notify, migrate if needed | +| `MAINTENANCE_STARTED` | Maintenance has begun | Update Entity state to MAINTENANCE | +| `MAINTENANCE_COMPLETED` | Maintenance completed | Restore Entity state, trigger drift detection | +| `UNSANCTIONED_CHANGE` | Change occurred that was not initiated by DCM | Policy Engine evaluation โ†’ REVERT/UPDATE/ALERT | +| `ENTITY_HEALTH_CHANGE` | Entity health status changed | Policy Engine evaluation | +| `PROVIDER_DEGRADATION` | Provider itself is degraded | Policy Engine evaluation โ†’ reroute new requests | +| `DECOMMISSION_NOTICE` | Provider is decommissioning underlying resource | Policy Engine evaluation โ†’ migrate or decommission Entity | + +**Event Payload Format:** +All provider lifecycle events must be reported in DCM unified data model format: + +```yaml +provider_lifecycle_event: + event_uuid: + event_type: + provider_uuid: + affected_entity_uuids: + - + event_timestamp: + event_details: + + severity: + requires_immediate_action: +``` + +**Maximum Reporting Latency:** +Providers must report lifecycle events within the timeframe declared in their provider registration. For CRITICAL severity events, immediate reporting is required. The reporting latency SLA is part of the Provider SLA/Operational Contract. + +### 7.3 DCM Capacity Rating + +DCM maintains an internal capacity rating per provider, per Resource Type, per location. This rating is used by the Policy Engine for placement decisions. + +```yaml +dcm_capacity_rating: + provider_uuid: + resource_type_uuid: + location_uuid: + last_updated: + update_source: + available_capacity: + capacity_confidence: + # high: updated within last scheduled window + # medium: updated within 2x scheduled window + # low: stale โ€” beyond 2x scheduled window + next_scheduled_update: +``` + +--- + +## 8. Entity Relationships + +Every Resource/Service Entity carries a `relationships` section declaring its relationships to other entities โ€” internal DCM entities, external data entities, and business context entities. The relationship model is universal โ€” the same structure is used for all relationship types. + +See [Entity Relationships](../entity-relationships/) for the complete relationship model. + +```yaml +resource_service_entity: + uuid: + # ... other entity fields ... + relationships: + - relationship_uuid: + this_entity_uuid: + this_role: + related_entity_uuid: + related_entity_type: + relationship_type: + nature: + lifecycle_policy: + on_related_destroy: + on_related_suspend: + on_related_modify: + status: + provenance: + +``` + +--- + +## 9. DCM System Policies for Resource/Service Entities + +The following are **non-overridable DCM System Policies** that apply to all Resource/Service Entities: + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `RSE-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation โ€” no Tenant = request rejected | +| `RSE-002` | Every Resource/Service Entity must have a UUID | Enforced at Entity creation | +| `RSE-003` | Every Resource/Service Entity must have a complete provenance chain | Enforced at every state transition | +| `RSE-004` | Realized State payloads must be complete โ€” not a status code | Enforced at provider response receipt | +| `RSE-005` | Decommissioned Entity records are immutable and permanent | Enforced at decommission โ€” records cannot be deleted | +| `RSE-006` | Provider lifecycle events must be recorded in Entity provenance | Enforced at event receipt | +| `RSE-007` | Ownership transfers must be authorized by policy | Enforced at transfer initiation | +| `RSE-008` | Process Resource Entities must reference all affected Entity UUIDs | Enforced at process completion | + +--- + +## 10. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | For Hybrid Transfer โ€” what is the maximum number of ownership transfers allowed, or is it unlimited? | Operational complexity | โ“ Unresolved | +| 2 | For Whole Allocation of bare metal โ€” how is the indivisibility enforced at the provider level? | Provider contract | โ“ Unresolved | +| 3 | Should capacity confidence ratings trigger automatic actions (e.g., LOW confidence triggers a Mode 1 query)? | Capacity model | โ“ Unresolved | +| 4 | For Process Resources โ€” should there be a maximum execution time after which DCM escalates? | Operational governance | โ“ Unresolved | +| 5 | How does the SUSPENDED state interact with cost analysis โ€” is a suspended Entity still billable? | Cost model | โ“ Unresolved | + +--- + +## 11. Related Concepts + +- **DCM Tenant** โ€” the mandatory ownership boundary for all Resource/Service Entities +- **Four States** โ€” Intent, Requested, Realized, Discovered โ€” the state lifecycle of a Resource/Service Request and Entity +- **Field-Level Provenance** โ€” every state transition and ownership transfer is recorded in Entity provenance +- **Policy Engine** โ€” evaluates provider events and unsanctioned changes, determines response actions +- **Service Dependencies** โ€” Resource/Service Entities declare dependencies on other Entities +- **Resource Grouping** โ€” Entities belong to a Tenant and optionally to additional Resource Groups +- **Provider Contract** โ€” governs provider obligations including capacity reporting and event notification + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-type-hierarchy.md b/content/docs/architecture/data-model/resource-type-hierarchy.md new file mode 100644 index 0000000..091bfaa --- /dev/null +++ b/content/docs/architecture/data-model/resource-type-hierarchy.md @@ -0,0 +1,450 @@ +--- +title: "Resource Type Hierarchy" +type: docs +weight: 4 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Entity Types](01-entity-types.md) | [Four States](../four-states/) | [Layering and Versioning](../layering-and-versioning/) | [Examples](04-examples.md) + +--- + +## 1. Purpose + +The DCM Resource Type Hierarchy is the structural model that defines how services and resources are categorized, specified, and exposed through the DCM Service Catalog. It is the mechanism by which DCM achieves **resource portability** โ€” the ability to express what a consumer needs independently of which specific provider delivers it. + +The hierarchy serves four goals: + +1. **Portability** โ€” consumer intent can be fulfilled by any provider that satisfies the resource type contract, without the consumer needing to know which provider that is +2. **Standardization** โ€” a common vocabulary and data contract for all resource types encourages interoperability across providers, implementors, and the broader DCM community +3. **Extensibility** โ€” the model can be extended at every level without breaking existing definitions +4. **Transparency** โ€” any deviation from full portability is explicitly declared, versioned, and surfaced to consumers + +--- + +## 2. The DCM Resource Type Registry + +DCM maintains an official **Resource Type Registry** โ€” the authoritative source of standard resource type definitions. The registry is the foundation of portability across the DCM ecosystem. + +### 2.1 Registry Principles + +- The registry is **open** โ€” third parties, implementors, and the community can propose new resource type definitions +- Registry entries are **versioned and immutable** once published โ€” changes produce new versions +- Registry definitions are **vendor-neutral by hard requirement** โ€” no vendor-specific data is permitted in a DCM-specified resource type unless that vendor is the exclusive provider of that technology stack +- The registry itself is subject to the same **deprecation model** as all other DCM definitions +- All registry entries follow the **universal versioning scheme** (Major.Minor.Revision) + +### 2.2 Default Resource Type Categories + +DCM ships with a default set of Resource Type Categories. Implementors may define additional categories following the specification. The registry contains both **Resource Types** (for provisioned resources) and **Information Types** (for external data references) โ€” distinguished by category prefix. + +**Resource Type Categories:** + +| Category | Description | +|----------|-------------| +| `Compute` | Processing resources โ€” virtual machines, containers, bare metal | +| `Network` | Networking resources โ€” IP addresses, VLANs, firewall rules, load balancers | +| `Storage` | Storage resources โ€” block, object, file, databases | +| `Platform` | Platform services โ€” Kubernetes clusters, application platforms | +| `Security` | Security resources โ€” certificates, secrets, HSMs, identity | +| `Observability` | Monitoring and logging resources | +| `Data` | Data services โ€” streams, queues, pipelines | + +**Information Type Categories:** + +| Category | Description | +|----------|-------------| +| `Business` | Business organizational data โ€” BusinessUnit, CostCenter, ProductOwner | +| `Identity` | Identity and access data โ€” Person, ServiceAccount, Group | +| `Compliance` | Regulatory and compliance data โ€” RegulatoryScope, AuditFramework | +| `Operations` | Operational reference data โ€” Runbook, SLA, SupportContract | + +All categories follow the same versioning, deprecation, and registry governance model. The `implements_type` field on provider registrations distinguishes whether a provider is a Service Provider (`service`) or an Information Provider (`information`). + +### 2.3 Registry Entry Structure + +Every entry in the Resource Type Registry carries the following metadata: + +```yaml +registry_entry: + uuid: + name: + fully_qualified_name: + version: + parent_uuid: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + replacement_version: + deprecation_reason: + migration_guidance: + portability: + classification: + portability_breaking: + portability_notes: + ownership: + owner: + owner_uuid: + origination_date: + description: + specification_ref: +``` + +--- + +## 3. Resource Type Hierarchy Levels + +The hierarchy has four levels, from most abstract to most concrete. Each level builds on the one above it. + +### Level 1 โ€” Resource Type Category + +The broadest classification. Defines the domain of a resource without any specificity about what the resource is. + +- DCM ships with default categories (see Section 2.2) +- Implementors may define additional categories +- Categories have no data fields โ€” they are organizational containers +- Categories are versioned and can be deprecated + +**Example:** `Compute`, `Network`, `Storage` + +--- + +### Level 2 โ€” Resource Type + +Defines an abstract resource within a category. A Resource Type represents a class of resource that multiple providers can implement. Resource Types are the primary unit of portability in DCM. + +- DCM maintains default Resource Types in the registry +- Community and implementors can define and register new Resource Types +- Resource Types must be **vendor-neutral** โ€” no provider-specific data +- Resource Types declare their **base field specification** (universal fields only) +- Resource Types are versioned and can be deprecated + +**Example:** `Compute.VirtualMachine`, `Network.IPAddress`, `Network.FirewallRule` + +--- + +### Level 3 โ€” Resource Type Specification + +The data contract for a Resource Type. Defines all fields โ€” universal, conditional, and any declared extension points โ€” along with their types, constraints, and portability classifications. + +- Every field in a specification carries a **portability classification** (see Section 4) +- Specifications define which fields are required vs. optional +- Specifications define validation constraints for each field +- Specifications declare **extension points** where providers may add fields +- Specifications are versioned independently of their Resource Type +- Specifications can be deprecated + +**Example:** `Compute.VirtualMachine` specification defines: `cpu_count` (universal, required), `ram_gb` (universal, required), `storage_gb` (universal, required), `os_image` (universal, required), `high_availability` (conditional, optional) + +--- + +### Level 4 โ€” Provider Catalog Item + +A specific provider's concrete implementation of a Resource Type Specification. This is where provider-specific detail lives and where the abstract becomes actionable. + +- Provider Catalog Items are registered against a specific Resource Type Specification version +- They must implement **all universal fields** of the parent specification +- They may implement **conditional fields** (declared in their registration) +- They may add **provider-specific extension fields** (must be marked portability-breaking) +- They are versioned and can be deprecated +- They declare their **sovereignty capabilities** (see Section 6) +- They declare their **supported lifecycle operations** (see Section 7) + +**Example:** `Nutanix.VM.Small` implements `Compute.VirtualMachine` with `cpu_count: 4`, `ram_gb: 16`, `storage_gb: 60` + +--- + +## 4. Portability Classification + +Every field in every Resource Type Specification carries a portability classification. This classification is part of the field's metadata and is immutable once published for a given version. + +### 4.1 Classification Levels + +| Classification | Description | Portability Impact | +|---|---|---| +| `universal` | Part of the DCM standard spec. All providers implementing this type must support it. | Fully portable across all implementing providers | +| `conditional` | Supported by multiple providers but not all. Providers declare support in their registration. | Portable across providers that declare support | +| `provider-specific` | Specific to one provider or technology stack. Using this field locks the request to that provider. | Portability-breaking โ€” must be explicitly marked | +| `exclusive` | Only one provider supports this technology stack. Portability is not applicable by definition. | Not applicable โ€” acknowledged and declared | + +### 4.2 Hard Portability Requirements + +The following are non-negotiable requirements for any DCM-specified Resource Type: + +1. All **universal** fields MUST be supported by ALL providers implementing that Resource Type +2. **Provider-specific** fields MUST be explicitly marked as portability-breaking in the field metadata +3. Consumers MUST be warned when their request contains portability-breaking fields +4. The only exception to vendor-neutrality is the **exclusive** classification โ€” where one provider is the sole implementor of a technology stack, explicitly acknowledged and declared in the registry +5. Any Resource Type in the DCM registry that contains provider-specific fields as universal fields is invalid and must be rejected + +### 4.3 Portability Field Metadata + +Every field in a Resource Type Specification carries the following portability metadata: + +```yaml +field_name: + type: + required: + description: + portability: + classification: + portability_breaking: + portability_notes: + supported_by: + constraints: + - + default_value: + provenance: + +``` + +--- + +## 5. Inheritance Model + +Resource Types support inheritance, enabling specialization without duplication. A child type inherits all fields from its parent and may add new fields. + +### 5.1 Inheritance Rules + +1. A child type inherits **all fields** from its parent type โ€” no field can be removed or redefined +2. A child type may **add new fields** beyond its parent's specification +3. A child type's portability classification can only be **equal to or more restrictive** than its parent โ€” a child of a `universal` type may be `conditional`, but not vice versa +4. Each level of the hierarchy is **independently versioned** +5. Each level maintains a **reference to its parent UUID and version** +6. Deprecating a parent type **does not automatically deprecate child types** โ€” each must be independently deprecated with appropriate migration guidance + +### 5.2 Inheritance Example + +``` +Compute # Category + โ””โ”€โ”€ VirtualMachine # Base Resource Type + โ”œโ”€โ”€ VirtualMachine.GPU # Inherits VirtualMachine + โ”‚ โ”œโ”€โ”€ gpu_count (conditional) + โ”‚ โ”œโ”€โ”€ gpu_memory_gb (conditional) + โ”‚ โ””โ”€โ”€ VirtualMachine.GPU.HighMemory # Inherits VirtualMachine.GPU + โ”‚ โ””โ”€โ”€ extended_memory_gb (conditional) + โ””โ”€โ”€ VirtualMachine.HighAvailability # Inherits VirtualMachine + โ”œโ”€โ”€ ha_mode (conditional) + โ””โ”€โ”€ failover_policy (conditional) +``` + +### 5.3 Inheritance Metadata + +Every Resource Type that inherits from a parent carries the following inheritance metadata: + +```yaml +inheritance: + parent_uuid: + parent_version: + parent_fully_qualified_name: + inherited_fields: + added_fields: +``` + +--- + +## 6. Provider Registration and Catalog Item Declaration + +For a provider to participate in the DCM ecosystem and have its catalog items available for request resolution, it must register against the Resource Type Hierarchy. + +### 6.1 Provider Registration Declaration + +A provider's registration is a machine-readable declaration that DCM consumes to understand what the provider offers and how to route requests to it: + +```yaml +provider_registration: + uuid: + name: + version: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + catalog_items: + - + sovereignty_capabilities: + + supported_lifecycle_operations: + + trust_declaration: + +``` + +### 6.2 Catalog Item Declaration + +Each catalog item a provider offers is declared against a specific Resource Type Specification version: + +```yaml +catalog_item: + uuid: + name: + version: + implements: + resource_type_uuid: + resource_type_version: + resource_type_fully_qualified_name: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + universal_fields: + + conditional_fields_supported: + + provider_specific_extensions: + + + portability_warning: +``` + +--- + +## 7. Request Resolution โ€” Specificity Narrowing + +Provider selection in DCM is never explicit. The consumer declares intent using Resource Types and field values. The appropriate provider catalog item is selected by the DCM Policy Engine through progressive specificity narrowing. + +### 7.1 Resolution Steps + +``` +Step 1: Resource Type declared + โ†’ matches all providers implementing that Resource Type + +Step 2: Universal fields specified + โ†’ still matches all providers (all must support universal fields) + +Step 3: Conditional fields specified + โ†’ narrows to providers that declare support for those fields + +Step 4: Provider-specific fields used + โ†’ narrows to single provider + โ†’ portability warning issued and recorded in request provenance + โ†’ enforcement mode applied (block|warn|allow) per organizational policy + +Step 5: Placement and sovereignty constraints applied + โ†’ Policy Engine applies placement policies + โ†’ Provider sovereignty capabilities matched against request requirements + โ†’ Final provider catalog item selected + +Step 6: Provider catalog item UUID recorded in request payload provenance +``` + +### 7.2 Portability Warning Enforcement + +When a request contains portability-breaking fields, the Policy Engine applies the configured enforcement mode. This is organizational policy โ€” configurable at the organization, domain, or service level: + +| Enforcement Mode | Behavior | +|---|---| +| `block` | Request is rejected. Consumer must remove portability-breaking fields or explicitly acknowledge the lock-in. | +| `warn` | Request proceeds. Portability warning is recorded in request provenance and surfaced to the consumer. | +| `allow` | Request proceeds silently. Portability-breaking fields are still recorded in provenance but no warning is surfaced. | + +The enforcement mode is itself a versioned, auditable policy โ€” subject to the same provenance tracking as all other data in DCM. + +--- + +## 8. Deprecation Model + +Every definition at every level of the Resource Type Hierarchy can be deprecated. Deprecation is a first-class concept in DCM โ€” not an afterthought. + +### 8.1 Deprecation Lifecycle + +``` +active โ†’ deprecated โ†’ retired +``` + +| State | Meaning | System Behavior | +|---|---|---| +| `active` | Definition is current and fully supported | Normal operation | +| `deprecated` | Definition is being phased out. Replacement is available. | Deprecation warning surfaced to consumers. Requests still processed. Warning recorded in provenance. | +| `retired` | Definition is no longer honored. | Requests using retired definitions are rejected by the Policy Engine. | + +### 8.2 Deprecation Cascade Rules + +- Deprecating a **Resource Type** does not automatically deprecate its child types or provider catalog items โ€” each must be independently deprecated +- Deprecating a **Provider Catalog Item** does not affect other catalog items implementing the same Resource Type +- Retiring a **Resource Type Specification version** causes all catalog items registered against that version to require re-registration against a current version +- **Sunset dates** must provide sufficient migration runway โ€” minimum notice periods may be defined by organizational policy + +### 8.3 Migration Guidance Requirement + +Any definition marked `deprecated` MUST include: +- A reference to the replacement definition (UUID and version) +- A human-readable deprecation reason +- Human-readable migration guidance explaining how to transition +- A sunset date giving consumers time to migrate + +--- + +## 9. Versioning + +All definitions in the Resource Type Hierarchy follow the universal DCM versioning scheme. + +### 9.1 Version Scheme + +`Major.Minor.Revision` + +| Component | Trigger | +|---|---| +| **Major** | Breaking changes to the contract โ€” removing fields, changing field types, changing required/optional status of universal fields | +| **Minor** | Additive changes, backward compatible โ€” adding new optional fields, adding new conditional fields, adding new extension points | +| **Revision** | Data or configuration changes with no contract impact โ€” updating descriptions, updating constraints that don't break existing data, updating metadata | + +### 9.2 Version Constraints in Requests + +Consumers and dependencies may declare version constraints in their requests: + +```yaml +resource_type: + uuid: + version_constraint: + version: +``` + +### 9.3 Version Immutability + +Once a version is published it is immutable. Any change โ€” even a documentation correction โ€” produces a new version. This applies to all definitions at all levels of the hierarchy. + +--- + +## 10. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | What is the governance model for proposing and approving new Resource Types to the DCM registry? | Community adoption, quality control | โ“ Unresolved | +| 2 | Should the registry support a formal review/approval workflow before a Resource Type becomes `active`? | Registry integrity, community trust | โ“ Unresolved | +| 3 | What is the minimum sunset period for deprecated definitions? | Migration planning, operational stability | โ“ Unresolved | +| 4 | Should version constraints in requests be strictly enforced or advisory? | Operational flexibility vs. predictability | โ“ Unresolved | +| 5 | How are conflicts resolved when multiple providers satisfy all narrowing criteria equally? | Request resolution determinism | โ“ Unresolved | +| 6 | Should the registry be distributed or centralized? How does this interact with sovereignty requirements? | Registry availability, sovereignty | โ“ Unresolved | + +--- + +## 11. Related Concepts + +- **Portability** โ€” the ability to fulfill a resource intent using any provider that satisfies the resource type contract +- **Naturalization** โ€” provider's responsibility to transform DCM unified data into provider-specific format +- **Denaturalization** โ€” provider's responsibility to transform provider-specific results back into DCM unified format +- **Sovereign Execution Posture** โ€” sovereignty capabilities declared in provider registration inform placement decisions +- **Policy Engine** โ€” applies portability enforcement, placement policies, and request resolution logic +- **Field-Level Provenance** โ€” every field modification during request resolution is recorded with source UUID and operation type +- **Universal Versioning** โ€” Major.Minor.Revision applies to all definitions at all levels of the hierarchy +- **Deprecation** โ€” universal model for phasing out definitions at any level with migration guidance + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/service-dependencies.md b/content/docs/architecture/data-model/service-dependencies.md new file mode 100644 index 0000000..f989ce7 --- /dev/null +++ b/content/docs/architecture/data-model/service-dependencies.md @@ -0,0 +1,378 @@ +--- +title: "Service Dependencies" +type: docs +weight: 6 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Entity Relationships](../entity-relationships/) | [Resource Type Hierarchy](../resource-type-hierarchy/) | [Resource/Service Entities](../resource-service-entities/) + +> **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](../entity-relationships/). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document. + +--- + +## 1. Purpose + +This document defines how service dependencies are declared, resolved, and managed within DCM. Dependencies are a core data model concern โ€” not an orchestration concern. The data structures defined here enable DCM to know the complete resource footprint of any request before execution begins, which is essential for cost analysis, placement decisions, rehydration, and audit. + +--- + +## 2. Why Dependencies Must Be Declared in Advance + +Dependencies must be declared in the data model โ€” not discovered at runtime by providers. This is a hard requirement driven by four core DCM goals: + +**Auditability** โ€” the complete dependency graph must be known before execution. Every resource that will be created as part of fulfilling a request must be visible in the request's provenance chain from the start. + +**Cost Analysis** โ€” accurate cost estimation and cost-based placement require knowing the full resource footprint before provisioning. Hidden dependencies produce hidden costs that only become visible after the fact. + +**Placement** โ€” the Policy Engine cannot make optimal placement decisions without knowing all resources that will be created. A Web Server request that implicitly spawns a VM, IP address, and firewall rule has placement requirements that span multiple resource types. + +**Idempotency and Consistency** โ€” if dependencies are declared in the service definition, the same request always produces the same dependency graph. Provider-driven dependency discovery at runtime breaks idempotency โ€” different provider implementations could produce different dependency graphs for the same logical request. + +--- + +## 3. Hybrid Dependency Declaration Model + +DCM uses a hybrid model for dependency declaration that operates at two levels: + +### 3.1 Type-Level Dependencies (Resource Type Specification) + +Dependencies declared at the Resource Type Specification level are **portable and provider-agnostic**. They define what kinds of resources are needed โ€” not which specific provider supplies them. + +- Declared in the Resource Type Specification +- Apply to all Provider Catalog Items implementing that Resource Type +- Use Resource Type UUIDs โ€” not provider-specific references +- Required for all implementations of the Resource Type +- Portable โ€” the dependency can be fulfilled by any provider implementing the required Resource Type + +**Example:** +```yaml +resource_type: Compute.VirtualMachine +type_level_dependencies: + - dependency_uuid: + required_resource_type_uuid: + required_resource_type_name: Network.IPAddress + dependency_type: hard + cardinality: one_to_one + description: Every VM requires exactly one IP address + - dependency_uuid: + required_resource_type_uuid: + required_resource_type_name: Network.FirewallRule + dependency_type: hard + cardinality: one_to_many + description: Every VM requires at least one firewall rule +``` + +### 3.2 Provider-Specific Dependencies (Provider Catalog Item) + +Dependencies declared at the Provider Catalog Item level are **provider-specific additions** beyond the type-level dependencies. They must be marked as portability-breaking. + +- Declared in the Provider Catalog Item registration +- Apply only to requests fulfilled by that specific provider +- Must be marked `portability_breaking: true` +- Visible to the Policy Engine for governance decisions +- Surfaced to consumers as portability warnings + +**Example:** +```yaml +catalog_item: Nutanix.VM.Small +provider_specific_dependencies: + - dependency_uuid: + required_resource_type_uuid: + required_resource_type_name: Nutanix.StorageContainer + dependency_type: hard + portability_breaking: true + description: Nutanix VMs require a Nutanix Storage Container + portability_warning: This dependency locks this request to Nutanix providers +``` + +--- + +## 4. Dependency Types + +Every declared dependency must specify its type: + +| Type | Description | Behavior | +|------|-------------|----------| +| `hard` | Must be realized before or alongside the dependent resource | Failure of dependency fails the dependent resource | +| `soft` | Preferred but not blocking | Failure of dependency is recorded but does not block the dependent resource | +| `conditional` | Required only if specific conditions in the request payload are met | Evaluated by Policy Engine against request data | + +--- + +## 5. Dependency Cardinality + +Every declared dependency must specify its cardinality: + +| Cardinality | Description | Example | +|-------------|-------------|---------| +| `one_to_one` | Exactly one dependency resource required | One VM needs exactly one primary IP | +| `one_to_many` | One or more dependency resources required | One VM needs one or more firewall rules | +| `one_to_optional` | Zero or one dependency resource | One VM may optionally have a secondary IP | +| `one_to_range` | A specific numeric range required | One load balancer needs 2-6 backend VMs | + +--- + +## 6. Dependency Graph + +When a request is processed, the Request Payload Processor constructs a **Dependency Graph** โ€” a complete map of all resources that must be created to fulfill the request, including all transitive dependencies. + +### 6.1 Dependency Graph Structure + +```yaml +dependency_graph: + graph_uuid: + root_request_uuid: + tenant_uuid: + created_timestamp: + nodes: + - node_uuid: + resource_type_uuid: + resource_type_name: + request_uuid: + entity_uuid: + lifecycle_state: + dependencies: + - dependency_uuid: + dependent_node_uuid: + dependency_type: + status: + edges: + - from_node_uuid: + to_node_uuid: + dependency_uuid: + dependency_type: +``` + +### 6.2 Transitive Dependencies + +DCM resolves transitive dependencies โ€” the full chain of dependencies, not just direct ones. + +**Example โ€” Web Server request:** +``` +Web Server (requested) + โ”œโ”€โ”€ VM (hard dependency of Web Server) + โ”‚ โ”œโ”€โ”€ IP Address (hard dependency of VM) + โ”‚ โ”‚ โ””โ”€โ”€ Network (hard dependency of IP Address) + โ”‚ โ””โ”€โ”€ Firewall Rule (hard dependency of VM) + โ”‚ โ””โ”€โ”€ IP Address (reference โ€” already in graph) + โ””โ”€โ”€ DNS Record (soft dependency of Web Server) + โ””โ”€โ”€ IP Address (reference โ€” already in graph) +``` + +The dependency graph contains each resource exactly once โ€” circular references and duplicate nodes are detected and resolved. A resource that appears as a dependency of multiple nodes is represented as a single node with multiple incoming edges. + +### 6.3 Dependency Graph and the Four States + +The dependency graph is part of the request's data from the moment it is constructed: + +- **Intent State** โ€” consumer's request, no dependency graph yet +- **Requested State** โ€” dependency graph constructed and attached, all nodes in PENDING state +- **Realized State** โ€” nodes updated to REALIZED as providers fulfill each dependency +- **Discovered State** โ€” dependency graph used to scope discovery โ€” discover all nodes in the graph + +--- + +## 7. Dependency Payload Passing + +When a dependency resource is realized, its realized payload must be passed to the dependent resource's provider. This is how a provider knows the details of the resources it depends on โ€” IP addresses, network configurations, security group IDs, etc. + +### 7.1 The Payload Passing Mechanism + +``` +Dependency Resource realized + โ”‚ + โ–ผ +Realized State payload captured in Realized Store + โ”‚ + โ–ผ +Dependency node in graph updated: entity_uuid recorded, status โ†’ SATISFIED + โ”‚ + โ–ผ +Dependent resource's Requested State payload enriched with dependency data + โ”‚ Recorded in field-level provenance โ€” source_type: dependency_payload + โ”‚ source_uuid: + โ–ผ +Enriched payload dispatched to dependent resource's provider +``` + +### 7.2 Dependency Data in Request Payloads + +When a dependency is satisfied, the dependent resource's Requested State payload is enriched with the dependency entity's UUID and relevant realized data: + +```yaml +# Original request payload for VM +vm_request: + cpu_count: 8 + ram_gb: 32 + os: RHEL9 + +# After IP Address dependency is realized +vm_request: + cpu_count: 8 + ram_gb: 32 + os: RHEL9 + dependencies: + ip_address: + entity_uuid: + ip_address: 192.168.1.45 + network_uuid: + subnet: 192.168.1.0/24 + provenance: + source_type: dependency_payload + source_uuid: + timestamp: +``` + +--- + +## 8. Dependency Resolution Order + +The dependency graph determines resolution order. Resources with no unsatisfied hard dependencies can be dispatched immediately. Resources with unsatisfied hard dependencies wait until their dependencies are satisfied. + +### 8.1 Resolution Rules + +- A resource node can only be dispatched when all its `hard` dependencies are in SATISFIED state +- `soft` dependencies do not block dispatch โ€” they are attempted but failure does not block +- `conditional` dependencies are evaluated by the Policy Engine before the graph is constructed โ€” if conditions are not met, the conditional dependency node is not added to the graph +- Independent branches of the dependency graph can be resolved in parallel โ€” the Orchestration component determines parallelism +- Circular dependencies are invalid โ€” the Policy Engine rejects any dependency graph with circular references + +### 8.2 Failure Handling + +Dependency failure handling is **configurable per request or per policy**: + +| Failure Mode | Behavior | +|-------------|---------| +| `fail_all` | Any hard dependency failure fails the entire request. All partially realized nodes are decommissioned. | +| `fail_dependent` | A hard dependency failure fails only the dependent resource and its dependents. Independent branches continue. | +| `retry` | Failed dependencies are retried with the same or alternative provider before failing. Retry count and provider selection policy are configurable. | +| `partial_complete` | Request is marked partially complete. Failed nodes are flagged for retry or manual intervention. | + +The failure mode is declared in the request payload or in an applicable organizational policy. + +--- + +## 9. Rehydration and the Dependency Graph + +The dependency graph is the primary mechanism enabling **DC Rehydration** โ€” the ability to reconstruct any resource and its dependencies from scratch. + +### 9.1 Rehydration Process + +Rehydration uses the **Intent State** of the original request โ€” not the Realized State โ€” to reconstruct the dependency graph. This ensures that rehydration applies current policies and standards rather than replaying an old realized state. + +``` +Rehydration initiated for a Tenant / Group / Entity + โ”‚ + โ–ผ +Intent State(s) retrieved from Intent Store + โ”‚ + โ–ผ +Dependency graphs reconstructed from Intent States + โ”‚ + โ–ผ +Graphs processed through current Policy Engine + โ”‚ Current policies applied โ€” may differ from original request + โ”‚ Current placement policies applied + โ”‚ Current sovereignty constraints applied + โ–ผ +New Requested State payloads generated + โ”‚ + โ–ผ +Resources realized in dependency order + โ”‚ + โ–ผ +New Realized States recorded +``` + +### 9.2 Intent Portability in Rehydration + +Because rehydration uses Intent State rather than Realized State: +- Resources can be rehydrated to a different provider โ€” as long as the provider supports the required Resource Types +- Current organizational standards and policies are applied โ€” ensuring rehydrated resources meet current compliance requirements +- Provider-specific dependencies (portability-breaking) may prevent rehydration to a different provider โ€” this is surfaced as a portability warning during rehydration planning + +### 9.3 Rehydration Scope + +Rehydration can be scoped to: +- A single Resource/Service Entity and its full dependency graph +- A Resource Group โ€” all entities in the group and their dependency graphs +- A Tenant โ€” all entities owned by the Tenant +- A full Data Center โ€” all entities across all Tenants in a location + +The dependency graph ensures that rehydration is always complete โ€” no orphaned resources, no missing dependencies. + +--- + +## 10. Dependency Declaration in Service Catalog Items + +Service Catalog Items must declare their dependencies as part of their definition. A catalog item with undeclared dependencies is invalid and will be rejected by the Policy Engine. + +```yaml +catalog_item: + uuid: + name: Web Server Service + resource_type_uuid: + type_level_dependencies: + - dependency_uuid: + required_resource_type_uuid: + dependency_type: hard + cardinality: one_to_one + provider_specific_dependencies: [] + conditional_dependencies: + - dependency_uuid: + required_resource_type_uuid: + dependency_type: conditional + condition: + field: high_availability + operator: equals + value: true + description: Load balancer required when high_availability is true +``` + +--- + +## 11. DCM System Policies for Dependencies + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `DEP-001` | All dependencies must be declared before a catalog item is active | Enforced at catalog item registration | +| `DEP-002` | Circular dependencies are invalid | Enforced at dependency graph construction | +| `DEP-003` | Provider-specific dependencies must be marked portability-breaking | Enforced at provider catalog item registration | +| `DEP-004` | Dependency payloads must be passed to dependent providers in DCM unified format | Enforced at dependency satisfaction | +| `DEP-005` | Every node in a dependency graph must have a UUID | Enforced at graph construction | + +--- + +## 12. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are dependency graphs versioned โ€” does a new version of a catalog item invalidate existing dependency graphs? | Versioning model | โ“ Unresolved | +| 2 | Should the dependency graph be stored as a separate entity or embedded in the request payload? | Data model structure | โ“ Unresolved | +| 3 | How are cross-tenant dependencies handled โ€” where one tenant's resource depends on another tenant's resource? | Multi-tenancy | โ“ Unresolved | +| 4 | Should there be a maximum dependency graph depth to prevent runaway transitive dependencies? | Operational complexity | โ“ Unresolved | +| 5 | How does the dependency graph interact with the Meta Provider model โ€” where one provider orchestrates others? | Provider model | โ“ Unresolved | + +--- + +## 13. Related Concepts + +- **Resource Type Specification** โ€” declares type-level dependencies for a Resource Type +- **Provider Catalog Item** โ€” declares provider-specific additional dependencies +- **Request Payload Processor** โ€” constructs the dependency graph during assembly +- **Policy Engine** โ€” evaluates conditional dependencies, enforces dependency policies, governs failure handling +- **Intent Portability** โ€” rehydration uses Intent State to allow replay with different providers +- **Field-Level Provenance** โ€” dependency payload data is recorded with source Entity UUID +- **Resource Grouping** โ€” rehydration can be scoped to groups and tenants + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md new file mode 100644 index 0000000..c22e8b2 --- /dev/null +++ b/content/docs/architecture/data-model/storage-providers.md @@ -0,0 +1,306 @@ +--- +title: "Storage Providers" +type: docs +weight: 10 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Four States](../four-states/) | [Audit, Provenance, and Observability](../audit-provenance-observability/) | [Information Providers](../information-providers/) + +--- + +## 1. Purpose + +A **Storage Provider** is the fourth formal DCM provider type. It is the interface through which DCM persists, retrieves, and streams all state data. DCM defines the contract โ€” the characteristics, capabilities, and obligations each store must satisfy. The implementation technology is a deployment choice made by implementors. + +This is consistent with DCM's governing framework philosophy: DCM does not prescribe technology. It defines what is required and what is guaranteed. An organization using GitHub and Kafka satisfies the same contracts as one using Gitea and EventStoreDB. + +--- + +## 2. The Four Provider Types + +| Provider Type | Purpose | Data Direction | DCM Owns Result? | +|--------------|---------|---------------|-----------------| +| **Service Provider** | Realizes resources | DCM โ†’ Provider โ†’ DCM | Yes | +| **Information Provider** | Serves external authoritative data | DCM โ†’ Provider (lookup) | No | +| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Children โ†’ DCM | Yes | +| **Storage Provider** | Persists and streams DCM state | DCM โ†” Provider | Yes โ€” DCM is authoritative | + +--- + +## 3. Storage Provider Contract โ€” Base Requirements + +All Storage Providers share these base contract requirements regardless of store type: + +### 3.1 Registration +Same model as Service and Information Providers. Storage Providers register with DCM declaring their endpoint, store type, capabilities, and sovereignty characteristics. + +```yaml +storage_provider_registration: + uuid: + name: + display_name: + store_type: + version: + endpoint: + capabilities: + sovereignty_constraints: + trust_declaration: + status: +``` + +### 3.2 Health Check +Same model as all providers. `GET /health` endpoint, DCM polls on configurable interval. + +### 3.3 Trust +Same model as all providers. DCM validates Storage Provider identity before writing or reading state data. A compromised Storage Provider is treated as a sovereignty incident. + +### 3.4 Provenance Emission Obligation +Every Storage Provider that holds state data has a contractual obligation to emit provenance events to the Audit component when state is written or modified. This is not optional โ€” it is part of the Storage Provider contract. + +```yaml +# Provenance emission event โ€” sent to Audit component on every write +provenance_emission: + store_type: + operation: + entity_uuid: + record_uuid: + actor_uuid: + timestamp: + payload_hash: + store_reference: +``` + +### 3.5 Consistency Guarantee Declaration +Each Storage Provider must declare its consistency model in registration. DCM components read this declaration and adapt their behavior accordingly. + +```yaml +consistency_declaration: + consistency_model: + replication_factor: + durability_guarantee: + max_data_loss_window: +``` + +--- + +## 4. GitOps Store Contract + +Used for: Intent State, Requested State, Layer Store, Policy Store + +### 4.1 Required Capabilities + +```yaml +gitops_capabilities: + branching: true # Branch-per-request support + pull_request: true # PR creation, review, merge + immutable_history: true # Commits are permanent + ci_cd_hooks: true # Webhook triggers on push/merge + search_index_integration: true # Search Index companion required + access_control: true # Per-branch, per-path access control + signed_commits: optional # Recommended for audit integrity +``` + +### 4.2 Required API Operations + +| Operation | Description | Used By | +|-----------|-------------|---------| +| `create_branch` | Create a new branch from main | Intent State creation | +| `commit_file` | Commit a file to a branch | Intent and Requested State write | +| `create_pr` | Open a Pull Request for review | Intent State review workflow | +| `merge_pr` | Merge an approved PR to main | Intent State approval | +| `get_file` | Retrieve a file by path or commit | State retrieval | +| `get_history` | Retrieve commit history for a path | Audit and rehydration | +| `trigger_ci` | Trigger CI pipeline on branch | Policy pre-validation | +| `trigger_cd` | Trigger CD pipeline on merge | Requested State assembly and dispatch | +| `post_comment` | Post a comment on a PR | CI pipeline result reporting | + +### 4.3 File Structure Convention + +``` +{store_root}/ + tenants/ + {tenant_uuid}/ + {entity_uuid}/ + intent.yaml # Intent State record + # OR + requested-state.yaml # Requested State record +``` + +### 4.4 Search Index Companion + +Every GitOps store deployment requires a companion Search Index. The Search Index is a separate Storage Provider that maintains a queryable projection of the GitOps store. See Section 6. + +--- + +## 5. Event Stream Store Contract + +Used for: Realized State, Discovered State + +### 5.1 Required Capabilities + +```yaml +event_stream_capabilities: + append_only: true # Events are never modified or deleted + entity_keyed_streams: true # Each entity has its own event stream + stream_replay: true # Streams can be replayed from any offset + entity_uuid_lookup: true # O(1) lookup of stream by entity UUID + at_least_once_delivery: true # Events are never silently lost + configurable_retention: true # Retention period configurable per stream type + distributed_replication: true # Data replicated across nodes + high_throughput_write: true # Optimized for machine-generated writes +``` + +### 5.2 Stream Naming Convention + +``` +dcm.realized.{entity_uuid} # Realized State stream per entity +dcm.discovered.{entity_uuid} # Discovered State stream per entity +dcm.audit.{tenant_uuid} # Audit event stream per tenant +dcm.system # DCM system-level events +``` + +### 5.3 Required API Operations + +| Operation | Description | Used By | +|-----------|-------------|---------| +| `append_event` | Append an event to an entity stream | Provider callbacks, discovery | +| `read_stream` | Read events from an entity stream from offset | State retrieval, drift detection | +| `read_latest` | Read the most recent event in a stream | Current state queries | +| `replay_stream` | Replay all events from beginning | Audit, historical reconstruction | +| `list_streams` | List streams matching a pattern | Tenant-level queries | +| `get_stream_metadata` | Get stream statistics and metadata | Health monitoring | + +### 5.4 Event Envelope + +Every event written to the Event Stream Store uses this envelope: + +```yaml +event_envelope: + event_uuid: + stream_id: + entity_uuid: + tenant_uuid: + event_type: + sequence_number: + timestamp: + schema_version: + payload_hash: + payload: + provenance: + written_by_uuid: + triggered_by_request_uuid: + triggered_by_actor_uuid: +``` + +### 5.5 Retention Model + +| Stream Type | Default Retention | Rationale | +|-------------|------------------|-----------| +| Realized State | Permanent | Complete audit trail required | +| Discovered State | Configurable window | Operational use only โ€” older snapshots archived | +| Audit | Regulatory period (configurable โ€” minimum 7 years for FSI) | Compliance requirement | + +--- + +## 6. Search Index Contract + +Used for: Queryable projection of GitOps stores + +### 6.1 Role and Authority + +The Search Index is explicitly **non-authoritative**. If the Search Index and the GitOps store disagree on any record, the GitOps store wins unconditionally. The Search Index is a performance layer โ€” it is never the source of truth. + +The Search Index can be rebuilt from scratch from Git history at any time. This replaceability is a contract requirement โ€” implementors must support full index rebuild from the GitOps store. + +### 6.2 Required Indexed Fields + +At minimum the Search Index must index these fields from Intent and Requested State records: + +```yaml +indexed_fields: + - entity_uuid # Universal linking key + - tenant_uuid # Tenant ownership + - resource_type_name # e.g., Compute.VirtualMachine + - resource_type_uuid # Registry UUID + - lifecycle_state # Current state + - provider_uuid # Selected provider + - created_timestamp # When the record was created + - updated_timestamp # When the record was last updated + - cost_center # Business context (if declared) + - business_unit_uuid # Business context (if declared) + - git_path # Path in GitOps store โ€” used to retrieve full record + - git_commit_hash # Specific commit โ€” used for point-in-time retrieval +``` + +### 6.3 Required Query Operations + +| Operation | Example | Used By | +|-----------|---------|---------| +| `find_by_entity_uuid` | Find all records for entity xyz | Rehydration, audit | +| `find_by_tenant` | All entities for Tenant A | Tenant management | +| `find_by_resource_type` | All VMs across all tenants | Catalog reporting | +| `find_by_lifecycle_state` | All PENDING entities | Operational monitoring | +| `find_by_field` | All entities with cost_center=BU-PAY | FinOps reporting | +| `full_text_search` | Search across all indexed text fields | Discovery, debugging | +| `count_by_field` | Count entities grouped by resource_type | Analytics | + +--- + +## 7. DCM-Internal Caches + +DCM may maintain internal performance caches between components and stores. These are not Storage Providers โ€” they are internal implementation details that do not require external registration or trust. + +### 7.1 Cache Characteristics + +- **Non-authoritative** โ€” explicitly marked. Cache hits are not treated as ground truth. +- **Cache-aside pattern** โ€” DCM checks cache first; on miss, reads from authoritative store and populates cache +- **Invalidation on write** โ€” any write to an authoritative store invalidates the corresponding cache entry +- **Bounded staleness** โ€” maximum staleness window configured per cache; entries older than the window are treated as misses +- **Rebuildable** โ€” any cache can be cleared and rebuilt from its authoritative store + +### 7.2 Candidate Cache Locations + +| Cache | Authoritative Source | Purpose | +|-------|---------------------|---------| +| Layer Cache | Layer Store (Git) | Avoid repeated Git reads for frequently used layers | +| Policy Cache | Policy Store (Git) | OPA policy bundles cached in Policy Engine memory | +| Catalog Cache | Catalog Store (Git) | Service catalog items cached for presentation | +| Provider Registry Cache | Provider Registry | Registered provider list cached for routing | +| Search Index | GitOps stores | Queryable projection (also functions as a cache) | + +--- + +## 8. Storage Provider vs Service Provider โ€” Key Differences + +| Dimension | Service Provider | Storage Provider | +|-----------|-----------------|-----------------| +| **Purpose** | Realizes resources | Persists DCM state | +| **Data direction** | DCM sends, provider executes | DCM reads and writes | +| **Naturalization** | Required โ€” DCM format โ†’ native | Not required โ€” DCM format throughout | +| **Denaturalization** | Required โ€” native โ†’ DCM format | Not required | +| **Provenance emission** | Required (realized state) | Required (all writes) | +| **Capacity model** | Resource capacity | Storage capacity and throughput | +| **Health model** | Is provider healthy? | Is store reachable and consistent? | + +--- + +## 9. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should Storage Providers support multi-region replication as a declared capability? | Sovereignty | โ“ Unresolved | +| 2 | How are Storage Provider failures handled โ€” failover, queuing, or rejection? | Reliability | โ“ Unresolved | +| 3 | Should the Search Index be a separate registered Storage Provider or bundled with the GitOps store? | Architecture | โ“ Unresolved | +| 4 | How does the Storage Provider model interact with air-gapped environments? | Sovereignty | โ“ Unresolved | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/overview.md b/content/docs/architecture/overview.md new file mode 100644 index 0000000..1b2a136 --- /dev/null +++ b/content/docs/architecture/overview.md @@ -0,0 +1,261 @@ +--- +title: High Level Design +type: docs +weight: 1 +--- + +# DCM High Level Design + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +## What is DCM? + +DCM (Data Center Management) is an open-source **governing framework** for +enterprise on-premises and sovereign cloud infrastructure. It provides a +hyperscaler-like cloud experience โ€” the operational model and self-service +capabilities of a public cloud provider โ€” on infrastructure that organizations +own and control. + +DCM is **not a provisioning tool**. It is the management plane that sits above +provisioning tools, governing what gets requested, approved, built, owned, and +decommissioned. Provisioning tools (Ansible, Terraform, Kubernetes operators) +become Service Providers that DCM orchestrates. + +**Mission:** Seamlessly manage the complete lifecycle of all data center +infrastructure by providing a policy-governed, data-driven, and unified platform +to enable and ensure sovereignty. + +--- + +## The Problem DCM Solves + +Enterprise organizations managing private cloud infrastructure face consistent +challenges that public cloud providers have already solved โ€” and that DCM brings +to on-premises: + +| Challenge | DCM Response | +|-----------|-------------| +| **Fragmented operations** โ€” disparate tools, no unified control | Single control plane โ€” one API, one data model, one policy engine | +| **No source of truth** โ€” multiple CMDBs diverge | Four-state model provides authoritative record of intent, request, realized, and discovered state | +| **High time-to-market** โ€” a VM may require dozens of teams | Self-service catalog with policy-governed automation | +| **Drift and state discrepancy** โ€” no reconciliation between intended and actual | Continuous drift detection comparing realized vs discovered state | +| **Sovereignty requirements** โ€” data residency, compliance, audit evidence | Policy Engine with sovereignty enforcement, complete provenance chain | + +--- + +## Core Principles + +| Principle | Meaning | +|-----------|---------| +| **Declarative** | Data describes what should exist, not how to achieve it | +| **API-First** | Every capability is available via a standard API | +| **Policy-Governed** | All business logic flows through the Policy Engine โ€” not hard-coded | +| **Idempotent** | Applying the same data multiple times always produces the same result | +| **Immutable if Versioned** | Published versions never change โ€” changes produce new versions | +| **Provider-Agnostic** | DCM defines contracts, not implementations | +| **GitOps-Native** | Intent and Requested state are Git-native โ€” branched, reviewed, versioned | +| **Kubernetes Superset** | DCM extends Kubernetes upward โ€” operators become DCM Service Providers | + +--- + +## Architecture Overview + +DCM consists of four major architectural layers: + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ CONSUMER INGRESS โ”‚ +โ”‚ Web UI โ”‚ Consumer API โ”‚ Direct API (3rd Rail) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ CONTROL PLANE โ”‚ +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ Service โ”‚ โ”‚ Request โ”‚ โ”‚ Policy Engine โ”‚ โ”‚ +โ”‚ โ”‚ Catalog โ”‚ โ”‚ Payload โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ Processor โ”‚ โ”‚ Transformation โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ Validation โ”‚ โ”‚ +โ”‚ โ”‚ GateKeeper โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ IDM / IAM โ”‚ โ”‚ Audit & โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ Observ. โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ API Gateway โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ PROVIDERS โ”‚ +โ”‚ โ”‚ +โ”‚ Service Providers โ”‚ Information Providers โ”‚ +โ”‚ Meta Providers โ”‚ Storage Providers โ”‚ +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ KubeVirt โ”‚ โ”‚ VMware โ”‚ โ”‚ OpenStackโ”‚ โ”‚ HR / Finance โ”‚ โ”‚ +โ”‚ โ”‚ AAP โ”‚ โ”‚ Bare โ”‚ โ”‚ CAPI โ”‚ โ”‚ CMDB / ITSM โ”‚ โ”‚ +โ”‚ โ”‚ CloudNPG โ”‚ โ”‚ Metal โ”‚ โ”‚ Storage โ”‚ โ”‚ Custom โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +--- + +## The Four States + +Every resource in DCM exists across four independently maintained state records: + +| State | Question | Store | +|-------|----------|-------| +| **Intent** | What did the consumer ask for? | GitOps โ€” immutable, branched, PR reviewed | +| **Requested** | What was approved and dispatched? | GitOps โ€” assembled, policy-processed, full provenance | +| **Realized** | What did the provider actually build? | Event Stream โ€” append-only, entity-keyed | +| **Discovered** | What actually exists right now? | Event Stream โ€” ground truth for drift detection | + +The **entity UUID** is the universal linking key โ€” assigned at Intent State creation, it links the entity across all four states and all stores throughout its entire lifecycle. + +--- + +## The Data Model + +The DCM Data Model is the foundational layer that governs how all data is +represented, versioned, assembled, and governed. Key concepts: + +**Data Layers** assemble a complete request payload from composable, versioned +units of configuration. 36 layer definitions can govern 40,000 VMs without +duplication. Layers are organized by Domain (system, platform, tenant, service, +provider), identified by a human-readable Handle, and ordered by a hierarchical +Priority Schema for deterministic conflict resolution. + +**Resource Types** are portable, vendor-neutral definitions of resource classes. +They live in the DCM Resource Type Registry alongside Information Types +(Business.*, Identity.*, Compliance.*) โ€” same registry, different category prefix. + +**Entity Relationships** use a universal bidirectional model for all connections +between entities โ€” whether VM-to-Storage, Application-to-WebServer, or +Resource-to-BusinessUnit. One model, all relationships. + +**Field Override Control** uses a graduated three-level model: no declaration +(allow by default), simple `override: immutable`, or a full actor-permission +matrix. The Policy Engine is the sole authority for setting override control. + +**Artifact Metadata** is universal โ€” every layer, policy, resource type, catalog +item, and provider registration carries a standard metadata block with creator, +owner, modification history, and contact information. + +--- + +## Provider Model + +DCM defines **contracts**, not implementations. Four provider types: + +| Type | Purpose | +|------|---------| +| **Service Provider** | Realizes resources โ€” KubeVirt, VMware, Ansible, Terraform | +| **Information Provider** | Serves authoritative external data DCM references but does not own | +| **Meta Provider** | Composes multiple providers into higher-order services | +| **Storage Provider** | Persists DCM state โ€” GitOps stores, event streams, audit store | + +All providers follow the same registration, health check, trust, and contract model. + +--- + +## Policy Engine + +The Policy Engine is the single authoritative logic gate for all business rules. +Three policy types in execution order: + +1. **Transformation** โ€” enriches and modifies the payload; may set field constraints +2. **Validation** โ€” checks payload against rules; pass/fail, no modification +3. **GateKeeper** โ€” highest authority; can override any field; enforces sovereignty + +Policies follow a three-tier hierarchy: Global โ†’ Tenant โ†’ User. A Global policy +cannot be overridden by Tenant or User policies. The Policy Engine uses OPA/Rego +for policy implementation. + +Policies support five statuses: **developing** (dev mode only), **proposed** +(shadow execution โ€” output captured but not applied, for validation), +**active**, **deprecated**, and **retired**. + +--- + +## Kubernetes as a Superset + +DCM is designed as a **superset of Kubernetes** โ€” extending Kubernetes' declarative +model upward to the management plane: + +| Kubernetes | DCM | +|-----------|-----| +| Single cluster | Multi-cluster, multi-infrastructure | +| Namespace isolation | First-class Tenant ownership model | +| RBAC + admission webhooks | Policy Engine with field-level override control | +| No cost attribution | Full lifecycle cost analysis | +| No cross-cluster management | Unified management plane | + +Kubernetes operators become DCM Service Providers through the +[DCM Operator Interface Specification](specifications/operator-interface-spec/). +The [DCM Operator SDK](specifications/operator-sdk-api/) enables Level 1 conformance +in one day. + +--- + +## Digital Sovereignty + +DCM addresses four sovereignty dimensions: + +| Dimension | DCM Enabler | +|-----------|-------------| +| **Data and Content Sovereignty** | Data Model, Policy Engine, Validated Providers | +| **Operational Sovereignty** | Policy Engine โ€” Sovereign Execution Posture | +| **Security and Compliance** | Audit, GRC, complete provenance chain | +| **Mobility and Placement** | Policy Engine placement constraints, provider portability | + +**Sovereign Execution Posture** โ€” the target end state where all operations are +governed, auditable, and compliant with sovereignty requirements. This is the +north star concept of DCM. + +--- + +## Request Lifecycle + +A complete request lifecycle from consumer intent to realized resource: + +``` +Consumer submits request + โ”‚ + โ–ผ [Git branch created โ€” CI pipeline fires] +Intent State captured (immutable consumer declaration) + โ”‚ CI: policy pre-validation, cost estimate, sovereignty check + โ”‚ Human review via PR (if policy requires) + โ–ผ [PR merged โ€” CD pipeline fires] +Request Payload Processor + โ”‚ Layer assembly (Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request) + โ”‚ Policy Engine (Transformation โ†’ Validation โ†’ GateKeeper) + โ–ผ +Requested State committed to Git (full provenance chain) + โ”‚ Provider selected by placement component + โ–ผ +Provider dispatch via API Gateway + โ”‚ Naturalization: DCM format โ†’ provider native format + โ”‚ Provider realizes resource + โ”‚ Denaturalization: provider native โ†’ DCM format + โ–ผ +Realized State (event stream, provider-confirmed) + โ”‚ + โ–ผ [Continuous] +Drift Detection: Discovered State vs Realized State + โ”‚ Unsanctioned changes โ†’ Policy Engine response + โ”‚ Drift โ†’ REVERT | UPDATE | ALERT | ESCALATE +``` + +--- + +## Related Documents + +- [Data Model](data-model/) โ€” Complete data model documentation +- [Specifications](specifications/) โ€” Operator Interface Specification, Kubernetes compatibility, SDK API, CNCF strategy +- [Enhancements](../enhancements/) โ€” Enhancement proposals for the DCM project diff --git a/content/docs/architecture/specifications/_index.md b/content/docs/architecture/specifications/_index.md new file mode 100644 index 0000000..79c3bf7 --- /dev/null +++ b/content/docs/architecture/specifications/_index.md @@ -0,0 +1,19 @@ +--- +title: Specifications +type: docs +weight: 3 +sidebar: + open: true +--- + +# DCM Specifications + +Formal technical specifications for DCM integration interfaces, the Kubernetes +superset strategy, and community engagement. + +{{< cards >}} + {{< card link="operator-interface-spec" title="Operator Interface Specification" icon="puzzle" subtitle="Formal contract for Kubernetes operators integrating with DCM. Conformance levels, registration, field mapping, and SDK." >}} + {{< card link="kubernetes-compatibility" title="Kubernetes Compatibility" icon="chip" subtitle="Formal Kubernetes-to-DCM concept mappings. DCM as a superset of Kubernetes." >}} + {{< card link="operator-sdk-api" title="Operator SDK API" icon="code" subtitle="Go library API design for the DCM Operator SDK โ€” Level 1 conformance in one day." >}} + {{< card link="cncf-strategy" title="CNCF Strategy" icon="globe-alt" subtitle="CNCF submission strategy, community engagement plan, and operator ecosystem approach." >}} +{{< /cards >}} diff --git a/content/docs/architecture/specifications/cncf-strategy.md b/content/docs/architecture/specifications/cncf-strategy.md new file mode 100644 index 0000000..b17c35e --- /dev/null +++ b/content/docs/architecture/specifications/cncf-strategy.md @@ -0,0 +1,234 @@ +--- +title: "CNCF Strategy" +type: docs +weight: 4 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Version:** 0.1.0-draft +**Status:** Draft +**Document Type:** Strategic Planning +**Maintainers:** Red Hat FlightPath Team +**Last Updated:** 2026-03 + +--- + +## 1. Strategic Intent + +DCM's goal is to become the community standard for enterprise data center and private cloud management โ€” a neutral, open standard that the industry adopts the way it adopted CSI, CNI, and CRI. This requires DCM to exist in a community-trusted home, not as a vendor product. + +The CNCF (Cloud Native Computing Foundation) is the appropriate home. It provides the neutral governance model, the community infrastructure, and the ecosystem relationships needed to drive broad adoption. The FSI consortium already engaging with DCM (leading FSI consortium members and others) provides the multi-organization sponsorship and real production use case evidence needed for a credible CNCF proposal. + +--- + +## 2. CNCF Landscape and Positioning + +### 2.1 Where DCM Fits + +The CNCF landscape has strong coverage of Kubernetes runtime concerns โ€” container runtimes, networking, storage, service mesh, observability. It has weaker coverage of the management plane โ€” the layer above Kubernetes that governs what gets provisioned, owned, and decommissioned across multiple clusters and infrastructure types. + +DCM fills this gap. It is not competing with existing CNCF projects โ€” it extends and governs them. + +**Related CNCF projects and how DCM relates:** + +| CNCF Project | Relationship to DCM | +|-------------|---------------------| +| **Kubernetes** | DCM is a superset โ€” extends Kubernetes upward to the management plane | +| **Crossplane** | Complementary โ€” Crossplane provisions cloud resources via Kubernetes CRDs; DCM governs what Crossplane provisions and adds the management plane | +| **Cluster API (CAPI)** | DCM can manage Kubernetes clusters via CAPI as a Service Provider | +| **Argo CD / Flux** | Complementary โ€” DCM governs provisioning requests; GitOps manages deployment | +| **OpenCost** | DCM's cost analysis is a superset โ€” OpenCost data can feed DCM cost attribution | +| **Kessel** | Shares inventory and relationship goals โ€” potential collaboration or alignment | +| **OPA/Gatekeeper** | DCM's Policy Engine uses OPA internally; Gatekeeper is the cluster-level enforcement | + +### 2.2 The Gap DCM Fills + +No current CNCF project addresses all of: +- Multi-cluster, multi-infrastructure lifecycle management from a single control plane +- First-class multi-tenancy with Tenant ownership model +- Policy governance with field-level override control across the full request lifecycle +- Data sovereignty and compliance evidence for regulated industries +- Service catalog with self-service consumer experience +- Cost attribution across heterogeneous infrastructure + +This is the gap DCM fills. The positioning is not "another Kubernetes tool" โ€” it is "the management plane that governs your entire data center, of which Kubernetes is one component." + +--- + +## 3. CNCF Submission Path + +### 3.1 CNCF Maturity Levels + +CNCF accepts projects at three maturity levels: + +| Level | Requirements | DCM Target Timeline | +|-------|-------------|---------------------| +| **Sandbox** | Alignment with CNCF mission, basic governance, active development | Target for initial submission | +| **Incubating** | Production users, healthy contributor base, defined governance, security audit | 12-18 months post-Sandbox | +| **Graduated** | Broad adoption, stable API, long-term maintainer commitment | 24-36 months post-Sandbox | + +### 3.2 Sandbox Submission Requirements + +For CNCF Sandbox acceptance, DCM needs: + +**Technical requirements:** +- Clear alignment with CNCF's cloud native mission +- Open source license (Apache 2.0 โ€” already in place) +- Publicly accessible source code (GitHub โ€” already in place) +- Documented roadmap +- Basic security practices (vulnerability disclosure process, etc.) + +**Governance requirements:** +- Defined governance model (maintainers, decision process) +- Code of conduct +- Multi-organization contributor base (this is the key requirement โ€” Red Hat alone is insufficient) + +**Community requirements:** +- Evidence of community interest beyond the founding organization +- At least one non-founding organization actively contributing + +**DCM's strong position:** +The FSI consortium provides exactly the multi-organization evidence CNCF requires. Having leading FSI consortium members as active contributors or committed users is an unusually strong foundation for a Sandbox proposal. Most projects submit to Sandbox without any production users โ€” DCM can submit with evidence of production interest from systemically important financial institutions. + +### 3.3 Recommended Submission Path + +**Step 1 โ€” CNCF TAG (Technical Advisory Group) engagement** +Before formal submission, engage with CNCF TAG App Delivery and TAG Runtime. These groups review cloud native tooling proposals and can provide informal feedback before the formal Due Diligence process. Presenting DCM at a TAG meeting builds awareness and surfaces concerns early. + +**Step 2 โ€” Prepare the Due Diligence document** +The CNCF Due Diligence document is a detailed technical and governance questionnaire. Key sections: project description, statement on alignment with CNCF mission, comparison to similar projects, security practices, roadmap, adopters. The FSI consortium adopters section will be a significant differentiator. + +**Step 3 โ€” TOC sponsor identification** +CNCF Technical Oversight Committee (TOC) members sponsor project proposals. Red Hat's relationships in the Kubernetes community make identifying a TOC sponsor feasible. Target TOC members with expertise in multi-cluster management or enterprise Kubernetes. + +**Step 4 โ€” Sandbox vote** +TOC votes on Sandbox acceptance. With a strong Due Diligence document, FSI adopter evidence, and a TOC sponsor, acceptance probability is high. + +--- + +## 4. Community Engagement Strategy + +### 4.1 Operator Ecosystem โ€” The Primary Leverage Point + +The DCM Operator Interface Specification is the primary community artifact for driving ecosystem adoption. The strategy is to make conformance attractive enough that operator maintainers want to implement it. + +**Priority operator communities for engagement:** + +| Operator | Community | Why Priority | Engagement Approach | +|----------|-----------|-------------|---------------------| +| **KubeVirt** | Red Hat/Community | Active DCM development already | Direct contribution โ€” DCM team contributes Level 2 support | +| **CloudNativePG** | CNPG Community | High FSI adoption โ€” databases in regulated environments | Present DCM at CNPG community calls, contribute SDK example | +| **Strimzi (Kafka)** | Red Hat/Community | Messaging infrastructure โ€” DCM Message Bus use case | Direct contribution via Red Hat maintainership | +| **Cert-Manager** | Jetstack/Venafi | Security resources โ€” every DCM deployment needs certificates | SDK contribution, present at KubeCon | +| **ACM** | Red Hat | Cluster management โ€” natural DCM complement | Direct โ€” internal Red Hat alignment | +| **Rook (Ceph)** | CNCF | Storage operator โ€” core DCM service provider use case | CNCF relationship โ€” present at SIG Storage | + +### 4.2 KubeCon Strategy + +KubeCon is the primary conference for Kubernetes ecosystem influence. DCM needs a presence at KubeCon North America and Europe: + +**KubeCon NA (target โ€” next edition):** +- Submit a talk: "DCM โ€” Managing the Management Plane: Kubernetes as a Component of Enterprise Infrastructure" +- Submit a contribfest session: hands-on DCM Operator SDK implementation workshop +- Engage Kubernetes SIG Cluster Lifecycle about CAPI integration + +**KubeCon EU (following year):** +- Present CNCF Sandbox submission (if accepted by then) +- Case study talk with FSI consortium member (FSI consortium members presenting their DCM deployment) +- Operator Interface Specification BOF (Birds of a Feather) session + +### 4.3 The Developer Value Proposition โ€” What We Need to Communicate + +The community message must be concrete and compelling, not abstract. Avoid "unified management plane" as the opener โ€” lead with what operators get: + +**For operator developers:** +> "Add DCM support to your operator and your users get self-service catalog, multi-tenancy, cost attribution, and cross-cluster management โ€” for free. It takes one day using our SDK." + +**For platform engineering teams:** +> "Manage your entire data center from one control plane. VMs, databases, Kubernetes clusters, networking โ€” all with the same declarative model, the same policy engine, and the same audit trail." + +**For FSI/regulated industry teams:** +> "Every provisioning request produces a complete audit chain โ€” who asked for what, what policies applied, what was approved, what was built. Sovereignty constraints enforced at the management plane, not bolted on afterward." + +### 4.4 Contributor Onboarding + +A project cannot become a standard without contributors beyond the founding organization. The contributor onboarding strategy: + +**Good first issues:** +Maintain a curated list of well-scoped, well-documented issues labeled `good-first-issue`. These should be achievable in a few hours without deep DCM knowledge โ€” documentation improvements, test coverage, example implementations, SDK feature additions. + +**Operator SDK examples:** +Each operator SDK example is a potential contributor touchpoint. An operator maintainer who wants to add DCM support to their operator is a natural contributor. The example for their specific CRD framework (kubebuilder, operator-sdk, raw controller-runtime) lowers the barrier. + +**RFC process:** +Establish a lightweight RFC (Request for Comments) process for significant changes to the DCM Operator Interface Specification. This gives external contributors a formal path to influence the specification direction โ€” which is essential for community trust. + +**Monthly community calls:** +Regular community calls (video, recorded, published) signal active project health and give contributors a forum to discuss ideas. Target: bi-weekly during active development, monthly once stable. + +--- + +## 5. Standards Positioning + +Beyond CNCF, DCM should engage with relevant standards bodies where appropriate: + +### 5.1 DMTF (Distributed Management Task Force) +DMTF maintains the TOSCA (Topology and Orchestration Specification for Cloud Applications) and other cloud management standards. DCM's data model has some conceptual overlap with TOSCA. Rather than competing, DCM should position as a Kubernetes-native, GitOps-native evolution of the same problem TOSCA addressed โ€” bringing the conversation into the cloud native era. + +### 5.2 FinOS Foundation +FinOS is the open source community for financial services. The FSI consortium involvement makes FinOS a natural secondary community for DCM. Presenting DCM at FinOS events reaches exactly the regulated industry audience that benefits most from DCM's sovereignty and compliance capabilities. + +### 5.3 OpenInfra Foundation +The OpenInfra Foundation hosts OpenStack, Kata Containers, and StarlingX โ€” all relevant to DCM's target environments (private cloud, edge, regulated infrastructure). DCM should present at OpenInfra Summit to the platform engineering teams who manage these environments. + +--- + +## 6. What Needs to Exist Before CNCF Submission + +The following artifacts must be ready before a CNCF Sandbox submission is credible: + +| Artifact | Status | Owner | Target | +|----------|--------|-------|--------| +| DCM Operator Interface Specification v1.0 | ๐Ÿ”„ Draft | DCM Project | Ready | +| DCM Operator SDK v0.1.0 (Level 1 + Level 2) | ๐Ÿ“‹ Not started | DCM Project | 3-6 months | +| KubeVirt reference implementation (Level 2) | ๐Ÿ”„ In progress | DCM/KubeVirt teams | 3-6 months | +| Conformance test suite (Level 1 + Level 2) | ๐Ÿ“‹ Not started | DCM Project | 3-6 months | +| CNCF Due Diligence document | ๐Ÿ“‹ Not started | DCM Project | 6 months | +| Governance model document | ๐Ÿ“‹ Not started | Red Hat/Consortium | 3 months | +| Security vulnerability disclosure process | ๐Ÿ“‹ Not started | Red Hat Security | 1 month | +| FSI consortium adopter statements | ๐Ÿ“‹ Not started | Consortium members | 3 months | +| Second non-Red Hat maintainer | ๐Ÿ“‹ Not started | Community | 6 months | + +--- + +## 7. Risk Considerations + +| Risk | Likelihood | Impact | Mitigation | +|------|-----------|--------|------------| +| CNCF TAG sees overlap with Crossplane | Medium | Medium | Prepare clear differentiation โ€” DCM governs, Crossplane provisions; they are complementary | +| Operator communities resist specification adoption | Medium | High | Lead with SDK ease, reference implementations, concrete value; don't mandate, make it attractive | +| Red Hat perceived as controlling the standard | Medium | High | Establish CNCF governance early, actively recruit non-Red Hat maintainers, FSI consortium co-ownership | +| Specification fragmentation โ€” forks or competing standards | Low | High | CNCF neutral governance prevents this; be the first mover in this space | +| Key contributor departure | Low | Medium | CNCF governance ensures project continuity beyond any single contributor | + +--- + +## 8. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the CNCF submission be for DCM as a whole or for the DCM Operator Interface Specification as a standalone standard? | Scope of submission | โ“ Unresolved | +| 2 | Which FSI consortium members are willing to be named as public adopters in the CNCF submission? | Submission strength | โ“ Unresolved | +| 3 | Is there a TOC member with relevant expertise who could sponsor the DCM proposal? | Submission path | โ“ Unresolved | +| 4 | Should DCM engage with the Kubernetes SIG structure before or after CNCF Sandbox submission? | Community positioning | โ“ Unresolved | +| 5 | What is the timeline for the KubeVirt reference implementation reaching Level 2 conformance? | Readiness milestone | โ“ Unresolved | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/kubernetes-compatibility.md b/content/docs/architecture/specifications/kubernetes-compatibility.md new file mode 100644 index 0000000..d1eb6de --- /dev/null +++ b/content/docs/architecture/specifications/kubernetes-compatibility.md @@ -0,0 +1,354 @@ +--- +title: "Kubernetes Compatibility" +type: docs +weight: 2 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](../../specifications/operator-interface-spec/) + +--- + +## 1. Purpose + +DCM is designed as a **superset of Kubernetes** โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries that Kubernetes alone cannot address. + +This document serves three purposes: + +1. **Defines the formal mapping** between Kubernetes concepts and DCM concepts โ€” enabling implementors to understand how the two models relate and where DCM extends beyond Kubernetes +2. **Establishes DCM Resource Types** for standard Kubernetes resources โ€” so that Kubernetes-managed resources participate in the DCM registry alongside non-Kubernetes resources +3. **Documents the boundary** between what Kubernetes governs and what DCM governs โ€” making clear that DCM extends Kubernetes rather than replacing it + +--- + +## 2. The Superset Relationship + +### 2.1 What Kubernetes Provides + +Kubernetes is a container orchestration platform that provides: +- Declarative desired-state management within a single cluster +- A controller/operator pattern for extending resource management +- Namespace-based isolation within a cluster +- RBAC for access control within a cluster +- A rich ecosystem of operators for managing complex stateful resources + +### 2.2 What DCM Adds + +DCM extends Kubernetes upward by providing: + +| Capability | Kubernetes | DCM | +|------------|-----------|-----| +| Scope | Single cluster | Multi-cluster, multi-infrastructure | +| Tenancy | Namespace isolation | First-class Tenant model with ownership | +| Policy | RBAC + admission webhooks | Full Policy Engine with Validation/Transformation/GateKeeper | +| Data lineage | Not provided | Field-level provenance on all data | +| Cost attribution | Not provided | Full lifecycle cost analysis | +| Drift detection | Basic โ€” controller reconciles | Full four-state model with Intent/Requested/Realized/Discovered | +| Service catalog | Not provided | Full self-service catalog with RBAC-governed presentation | +| Sovereignty | Not provided | Sovereignty declarations, placement constraints, compliance evidence | +| Information context | Labels/annotations | First-class Information Provider relationships | +| Non-Kubernetes resources | Not provided | VMware, bare metal, OpenStack, etc. all managed through same model | + +### 2.3 What DCM Does Not Replace + +DCM does not replace Kubernetes at the runtime level. Kubernetes continues to: +- Schedule and run containers +- Manage Pod lifecycle within a cluster +- Enforce network policies within a cluster +- Provide the Kubernetes API for cluster-native tooling +- Run operators that manage complex stateful resources + +DCM manages the management plane โ€” the lifecycle of what gets requested, provisioned, owned, governed, and decommissioned. Kubernetes manages the execution plane โ€” the runtime behavior of what is running. + +--- + +## 3. Core Concept Mappings + +### 3.1 Resource Model + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Custom Resource Definition (CRD) | Resource Type Specification | CRD schema โ†’ DCM Resource Type fields | DCM Resource Type is the portable, provider-agnostic equivalent. CRD is the Kubernetes-specific implementation schema. | +| Custom Resource (CR) | Requested State payload โ†’ Realized State entity | CR is the naturalized form of the DCM payload | The operator translates DCM Requested State into a CR (Naturalization) and translates CR status back to DCM Realized State (Denaturalization). | +| Built-in resource (Pod, Service, PV) | DCM Resource Type in Compute.*, Network.*, Storage.* | Kubernetes built-ins are valid DCM Resource Types | See Section 5 for standard Kubernetes resource type mappings. | +| Kubernetes object | Resource/Service Entity | Every Kubernetes object managed by DCM has a corresponding DCM entity with UUID and provenance | | + +### 3.2 Control Loop + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Operator reconciliation loop | Realization + Drift Detection combined | Reconciliation IS the realization process โ€” the operator drives actual state toward desired state | DCM's Drift Detection compares Discovered State against Realized State. The operator's reconciliation loop is the mechanism that corrects drift. | +| Desired state (CR spec) | Requested State | CR spec is the naturalized form of the DCM Requested State | DCM stores the Requested State in DCM format. The operator translates it to CR spec format. | +| Actual state (CR status) | Realized State | CR status is the Kubernetes-native form of the DCM Realized State | The operator must denaturalize CR status back to DCM Realized State format and report it to DCM. | +| Watch/Inform pattern | DCM Discovered State polling | Kubernetes watch events are the mechanism for keeping DCM Discovered State current | | + +### 3.3 Isolation and Multi-tenancy + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Namespace | DCM Tenant boundary | One namespace per DCM Tenant (per_tenant strategy) | Kubernetes namespace provides the physical isolation enforcement. DCM Tenant provides the ownership and governance model. A single DCM Tenant maps to exactly one namespace per cluster. | +| Namespace | DCM Resource Group | In shared namespace strategies, Resource Group labels replace namespace isolation | When multiple Tenants share a namespace, DCM Resource Group labels provide logical separation. | +| Kubernetes RBAC | DCM IDM/IAM + Policy Engine | Kubernetes RBAC is the runtime enforcement mechanism. DCM Policy Engine governs who can request what via the service catalog. | DCM policies determine what a user can request. Kubernetes RBAC determines what a running workload can do. These are complementary, not duplicative. | +| ServiceAccount | DCM Identity.ServiceAccount Information Type | Kubernetes ServiceAccounts that DCM provisions or references are modeled as DCM Information Type entities | | + +### 3.4 Relationships and Dependencies + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| ownerReference | Entity Relationship (`contains`/`contained_by`) | Kubernetes ownerReferences are a subset of DCM entity relationships โ€” ownership only | DCM relationships are richer โ€” supporting `requires`, `depends_on`, `references`, `peer`, `manages` in addition to ownership. During Denaturalization, ownerReferences are translated to DCM `contains` relationships. | +| Finalizers | Lifecycle policy (`retain`, `detach`) | Kubernetes finalizers implement DCM lifecycle policies at the Kubernetes level | When DCM declares `on_parent_destroy: retain` for a storage entity, the operator implements this using Kubernetes finalizers to prevent deletion until DCM confirms the lifecycle policy has been applied. | +| Label selectors | Resource Group membership | Kubernetes label selectors used for DCM Resource Group filtering | DCM mandatory labels (`dcm-tenant-id`, `dcm-entity-id`) are used as label selectors for Resource Group queries. | + +### 3.5 Data Model + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Labels | DCM entity metadata + relationships | DCM-mandatory labels (`dcm-managed`, `dcm-tenant-id`, `dcm-entity-id`, etc.) carry core DCM identity data. Custom labels may map to DCM Information Type relationships. | | +| Annotations | DCM field-level provenance + metadata | Annotations used by DCM to carry request correlation data during the request lifecycle | `dcm-request-id` annotation on a CR identifies the DCM request that created or last modified it โ€” enabling unsanctioned change detection. | +| Resource version | Entity version (Revision component) | Kubernetes resource versions map to DCM entity Revision increments | Major and Minor versions are managed by DCM based on breaking/non-breaking changes. Kubernetes resource version increments map to DCM Revision increments. | +| Generation | Requested State version | CR generation increments correspond to new DCM Requested State records | Each new generation of a CR corresponds to a new intent/request cycle in DCM. | + +### 3.6 Lifecycle + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Pod phases (Pending, Running, Succeeded, Failed, Unknown) | DCM lifecycle states | Pod phases map to DCM lifecycle states via condition_mappings declaration | | +| CRD conditions | DCM lifecycle states and events | Standard conditions (Ready, Degraded, Progressing) map to DCM states and events via the field mapping specification | | +| Kubernetes events | DCM lifecycle events | Kubernetes watch events trigger DCM lifecycle event reports | The operator translates Kubernetes events into DCM lifecycle event types (ENTITY_HEALTH_CHANGE, DEGRADATION, UNSANCTIONED_CHANGE, etc.) | +| Cluster deletion | DCM decommission workflow | Cluster deletion triggers DCM's full decommission lifecycle โ€” lifecycle policies applied to all related entities | | + +--- + +## 4. Where DCM Extends Beyond Kubernetes + +These are concepts that exist in DCM but have no Kubernetes equivalent. They are the capabilities DCM adds that justify the superset positioning. + +### 4.1 Intent State + +Kubernetes has no concept of a consumer's original intent separate from the desired state. Once you apply a manifest, Kubernetes only knows the current desired state โ€” not what the consumer originally asked for or why. + +DCM's Intent State is the immutable record of what the consumer asked for, stored before any policy processing or layer enrichment. This enables: +- Rehydration โ€” replaying the original intent through current policies to produce a new request +- Intent portability โ€” the same intent applied to a different provider +- Audit โ€” answering "what did the consumer originally ask for?" independently of what was realized + +### 4.2 Field-Level Provenance + +Kubernetes has no concept of where a field value came from or why it was set. A field in a CR spec is a field โ€” there is no lineage. + +DCM's field-level provenance carries the full lineage of every field value through the entire lifecycle โ€” which layer set it, which policy modified it, which provider realized it, and why each change was made. This enables complete audit trails and sovereignty evidence. + +### 4.3 Data Layers and Assembly + +Kubernetes has no equivalent to DCM's layering model. A Kubernetes manifest is a flat declaration โ€” there is no concept of organizational standards, site-specific configuration, and service-specific configuration being separate layers that compose into a final manifest. + +DCM's layering model enables 36 layer definitions to govern 40,000 VMs without duplication โ€” impossible in the Kubernetes model. + +### 4.4 Policy Engine + +Kubernetes admission webhooks provide some policy capability (validation, mutation) but are cluster-scoped, apply at admission time only, and have no concept of hierarchy (Global โ†’ Tenant โ†’ User policy levels) or field-level override control. + +DCM's Policy Engine operates at the management plane level, applies across all clusters and providers, enforces a three-level hierarchy with field-level override control (allow/constrained/immutable), and carries policy decisions as provenance metadata in the payload. + +### 4.5 Cost Analysis + +Kubernetes has no native cost attribution model. Tools like Kubecost exist but are add-ons with no integration into the request lifecycle. + +DCM's cost analysis is built into the lifecycle model โ€” cost attribution is tracked from request time through realization, operation, and decommission for every entity. + +### 4.6 Information Providers + +Kubernetes has no concept of structured relationships to external organizational data (Business Units, Cost Centers, Product Owners). Labels and annotations are unstructured key-value pairs with no type safety, no external system integration, and no verification model. + +DCM's Information Provider model gives every entity structured, verified, versioned relationships to external organizational data with a stable external key model. + +### 4.7 Cross-Cluster Lifecycle + +Kubernetes manages resources within a single cluster. Multi-cluster management requires additional tools (ACM, Argo CD, Fleet) that are not part of the core Kubernetes model. + +DCM manages the lifecycle of resources across multiple clusters as a first-class capability โ€” the same Resource Type can be instantiated on any cluster that has a conformant Service Provider registered. + +--- + +## 5. Standard Kubernetes Resource Type Mappings + +These are the DCM Resource Type registry entries for standard Kubernetes resource types. Operators implementing these types should use these registry UUIDs and field definitions. + +### 5.1 Compute + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Compute.Pod` | Pod | Lowest-level compute unit | +| `Compute.Container` | Container (within a Pod) | Sub-entity of Pod โ€” expanded via bundled declaration | +| `Compute.Deployment` | Deployment | Managed set of Pods | +| `Compute.StatefulSet` | StatefulSet | Stateful managed set of Pods | +| `Compute.Job` | Job | One-time execution workload | +| `Compute.CronJob` | CronJob | Scheduled execution workload | + +### 5.2 Network + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Network.Service` | Service | In-cluster service discovery and load balancing | +| `Network.Ingress` | Ingress | External HTTP/HTTPS routing | +| `Network.NetworkPolicy` | NetworkPolicy | In-cluster network isolation | + +### 5.3 Storage + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Storage.PersistentVolume` | PersistentVolume | Cluster-level storage resource | +| `Storage.PersistentVolumeClaim` | PersistentVolumeClaim | Consumer's storage declaration โ€” expanded into Storage.PersistentVolume relationship | +| `Storage.StorageClass` | StorageClass | Storage type definition โ€” maps to DCM Provider Catalog Item | +| `Storage.ConfigMap` | ConfigMap | Configuration data storage | +| `Storage.Secret` | Secret | Sensitive data storage | + +### 5.4 Platform + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Platform.KubernetesCluster` | Kubernetes Cluster (via CAPI or managed service) | The cluster itself is a DCM-managed resource | +| `Platform.Namespace` | Namespace | Maps to DCM Tenant boundary in per_tenant strategy | +| `Platform.CustomResourceDefinition` | CRD | CRD registration maps to DCM Resource Type registration | + +### 5.5 Identity + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Security.ServiceAccount` | ServiceAccount | Kubernetes identity for workloads | +| `Security.Role` | Role / ClusterRole | Kubernetes RBAC role | +| `Security.RoleBinding` | RoleBinding / ClusterRoleBinding | Kubernetes RBAC binding | + +--- + +## 6. The Kubernetes Information Provider + +Kubernetes clusters function as both Service Providers (for provisioning resources) and Information Providers (for querying existing state). As an Information Provider, a Kubernetes cluster exposes its current resource state to DCM for: + +- **Brownfield ingestion** โ€” discovering existing resources and bringing them under DCM lifecycle management +- **Discovered State** โ€” DCM's Discovered State for Kubernetes resources comes from querying the Kubernetes API +- **Drift detection** โ€” comparing DCM Realized State against what Kubernetes actually has + +### 6.1 Kubernetes as Information Provider Registration + +```yaml +information_provider_registration: + name: kubernetes-cluster-01 + implements: + - information_type: Platform.KubernetesCluster + - information_type: Compute.Pod + - information_type: Storage.PersistentVolume + # ... all resource types the cluster contains + endpoint: + kubernetes_credentials: + auth_method: + discovery_capabilities: + label_selector: "dcm-managed=true" + # Only returns DCM-managed resources by default + full_discovery: true + # Can also return all resources for brownfield ingestion +``` + +### 6.2 Discovered State from Kubernetes + +DCM queries the Kubernetes API using the Kubernetes Information Provider to populate Discovered State: + +``` +DCM Drift Detection + โ”‚ + โ–ผ +Kubernetes Information Provider + โ”‚ GET /apis/{group}/{version}/namespaces/{ns}/{kind} + โ”‚ Filter: label dcm-entity-id = {entity_uuid} + โ–ผ +Discovered State payload (DCM format) + โ”‚ Kubernetes object denaturalized to DCM format + โ–ผ +Compare against Realized State + โ”‚ Field-by-field comparison + โ–ผ +UNSANCTIONED_CHANGE if differences found + โ”‚ Reported to Policy Engine for response determination +``` + +--- + +## 7. Kubernetes-Native Patterns and DCM Equivalents + +### 7.1 GitOps + +Kubernetes GitOps (Argo CD, Flux) manages Kubernetes manifests in Git and synchronizes them to clusters. DCM's data model is also Git-based โ€” all layers, Resource Type definitions, and policy definitions are stored in Git. + +The relationship: DCM manages the **request lifecycle** (what gets asked for, approved, and provisioned). GitOps manages the **deployment lifecycle** (what gets deployed to a cluster from a Git repository). These are complementary: + +- DCM governs the provisioning request โ€” "is this consumer allowed to provision this resource?" +- GitOps deploys application code to the provisioned resource +- DCM and GitOps together form a complete lifecycle: DCM provisions the cluster, GitOps deploys applications to it + +### 7.2 Helm + +Helm charts are packages of Kubernetes manifests that can be parameterized. In DCM terms, a Helm chart is a form of Catalog Item โ€” a curated, parameterized offering of a set of Kubernetes resources. + +DCM does not replace Helm โ€” it can use Helm as a delivery mechanism inside a Service Provider. The Service Provider receives the DCM Requested State, translates it to Helm values, and uses Helm to deploy the resources. The operator pattern is preferred for Day 2 management (Helm has limited reconciliation), but Helm remains valid for initial provisioning. + +### 7.3 Cluster API (CAPI) + +CAPI is the Kubernetes sub-project for managing Kubernetes clusters themselves using the Kubernetes API and operator pattern. CAPI clusters are a natural fit for DCM's `Platform.KubernetesCluster` Resource Type โ€” a CAPI-based operator would be the Service Provider for provisioning new Kubernetes clusters as DCM-managed resources. + +This is particularly significant: DCM managing the lifecycle of Kubernetes clusters through CAPI means DCM can provision the very infrastructure that operators run on. The superset relationship becomes concrete โ€” DCM provisions the cluster, the cluster runs the operators, the operators provision the resources that DCM manages. + +--- + +## 8. Migration Path โ€” Kubernetes-Native to DCM-Managed + +Organizations running Kubernetes today can adopt DCM incrementally: + +### Phase 1 โ€” Observation (no operator changes) +Deploy DCM with the Kubernetes Information Provider. DCM observes existing resources via the Kubernetes API and builds a Discovered State inventory. No changes to existing operators or workloads. + +### Phase 2 โ€” Brownfield Ingestion (no operator changes) +DCM promotes Discovered State records to Realized State โ€” assuming lifecycle management of existing resources. Resources get DCM UUIDs, Tenant assignments, and provenance records. Existing resources are now DCM-managed without any operator changes. + +### Phase 3 โ€” Level 1 Conformance (minimal operator changes) +Operators implement Level 1 of this specification via the DCM Operator SDK. New resources are provisioned through DCM's service catalog. Existing resources managed via brownfield ingestion continue as-is. + +### Phase 4 โ€” Level 2 Conformance (moderate operator changes) +Operators implement Level 2 โ€” full field mappings, capacity reporting, lifecycle events. DCM gains placement intelligence, drift detection, and cross-cluster management capabilities. + +### Phase 5 โ€” Level 3 Conformance (complete integration) +Operators implement Level 3 โ€” sovereignty declarations, provenance, discovery endpoint. Full DCM capabilities available. + +--- + +## 9. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How does the Namespace-to-Tenant mapping work when a cluster has existing namespaces that predate DCM adoption? | Brownfield migration | โ“ Unresolved | +| 2 | Should `Platform.KubernetesCluster` be the boundary for a DCM deployment, or can DCM manage resources across clusters without treating the cluster as a DCM entity? | Architecture scope | โ“ Unresolved | +| 3 | How does DCM interact with Kubernetes admission webhooks โ€” do they duplicate Policy Engine functions or complement them? | Policy model | โ“ Unresolved | +| 4 | Should the Kubernetes Information Provider be a built-in DCM component or a separately deployed provider? | Deployment architecture | โ“ Unresolved | +| 5 | How does the DCM superset model interact with managed Kubernetes services (EKS, GKE, AKS) where cluster management is outside the user's control? | Cloud provider integration | โ“ Unresolved | + +--- + +## 10. Related Concepts + +- **DCM Operator Interface Specification** โ€” the technical contract for operators integrating with DCM +- **DCM Operator SDK** โ€” Go library implementing this specification for operator developers +- **Entity Relationships** โ€” DCM's universal relationship model, of which Kubernetes ownerReferences are a subset +- **Resource Type Hierarchy** โ€” the DCM registry where Kubernetes Resource Types are registered +- **Information Providers** โ€” the DCM model for the Kubernetes API as a discoverable information source +- **Four States** โ€” DCM's Intent/Requested/Realized/Discovered model, which extends Kubernetes' desired/actual model + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md new file mode 100644 index 0000000..44d7cc9 --- /dev/null +++ b/content/docs/architecture/specifications/operator-interface-spec.md @@ -0,0 +1,837 @@ +--- +title: "DCM Operator Interface Specification" +type: docs +weight: 1 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Not yet ratified +**Document Type:** Technical Specification +**Maintainers:** Red Hat FlightPath Team +**GitHub:** https://github.com/dcm-project +**Last Updated:** 2026-03 + +--- + +## Abstract + +This specification defines the interface by which Kubernetes operators integrate with the DCM (Data Center Management) control plane as first-class Service Providers. An operator that conforms to this specification becomes a DCM Service Provider, enabling its managed resources to participate in DCM's unified lifecycle management, multi-tenancy, policy governance, cost analysis, drift detection, and service catalog. + +DCM is designed as a superset of Kubernetes โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries. This specification is the technical contract that enables that extension without requiring operators to abandon their existing Kubernetes-native design. + +--- + +## 1. Introduction + +### 1.1 Motivation + +Kubernetes operators are the most mature pattern for managing complex, stateful resources declaratively on Kubernetes. However, operators operate within a single cluster and lack the cross-cluster lifecycle management, multi-tenancy, cost attribution, sovereignty governance, and policy enforcement that enterprise organizations require at scale. + +DCM provides these capabilities at the management plane level โ€” above individual clusters. By conforming to this specification, an operator's managed resources become: + +- **Multi-tenant** โ€” DCM Tenant ownership and isolation applied automatically +- **Cost-attributed** โ€” resource costs tracked and attributed across the full lifecycle +- **Policy-governed** โ€” organizational policies applied at request time via DCM's Policy Engine +- **Cross-cluster** โ€” the same resource type managed across multiple clusters through DCM +- **Self-service** โ€” automatically available in the DCM Service Catalog for consumer request +- **Sovereignty-compliant** โ€” placement and operational constraints enforced by DCM's GateKeeper policies +- **Audit-complete** โ€” full provenance chain from intent through realization + +### 1.2 Scope + +This specification defines: +- The HTTP API an operator must expose to participate in DCM +- The data format for all API payloads (DCM Unified Data Model) +- The registration, health, capacity, status, and lifecycle event contracts +- The field mapping specification for translating between DCM format and CRD format +- Conformance levels and what each level unlocks in DCM + +This specification does not define: +- How operators implement their internal reconciliation logic +- Which specific Kubernetes distributions operators must support +- The internal architecture of the DCM control plane +- Provider-specific business logic or domain knowledge + +### 1.3 Relationship to the DCM Service Provider Contract + +This specification is a Kubernetes-specific instantiation of the DCM Service Provider Contract. All general Service Provider Contract requirements apply. This specification adds Kubernetes-specific requirements and guidance. Where this specification and the general Service Provider Contract conflict, this specification takes precedence for Kubernetes operator implementations. + +### 1.4 Terminology + +- **Operator** โ€” a Kubernetes controller that manages custom resources via a Custom Resource Definition (CRD) +- **DCM Control Plane** โ€” the DCM management system that routes requests and manages lifecycle +- **Adapter** โ€” a component that sits between DCM and an operator, implementing this specification on the operator's behalf (used when the operator cannot be modified directly) +- **Native implementation** โ€” an operator that implements this specification directly, without an adapter +- **CR** โ€” Custom Resource โ€” an instance of a CRD managed by the operator +- **CRD** โ€” Custom Resource Definition โ€” the Kubernetes schema definition for a CR +- **Reconciliation loop** โ€” the operator's control loop that drives actual state toward desired state + +--- + +## 2. Conformance Levels + +This specification defines three conformance levels. Higher levels unlock additional DCM capabilities. An operator may implement any level โ€” DCM accepts operators at all levels, with capabilities gated by the declared conformance level. + +**Design principle:** Level 1 must be achievable in a single day of work for an existing operator. Level 3 is the target for operators that want full DCM integration. The SDK (see Section 9) handles all protocol concerns โ€” operator developers only implement business logic. + +### 2.1 Level 1 โ€” Basic + +**What it requires:** +- Operator registration with DCM on startup +- Health check endpoint (`GET /health`) +- Basic status reporting to DCM when resource state changes + +**What it unlocks:** +- Operator resources appear in the DCM Service Catalog +- Basic lifecycle state tracking (PROVISIONING, OPERATIONAL, FAILED, DECOMMISSIONED) +- Health monitoring via DCM Observability +- Basic cost tracking (resource exists/does not exist) + +**Estimated implementation effort:** 1 day using the DCM Operator SDK + +### 2.2 Level 2 โ€” Standard + +**What it requires:** All Level 1 requirements, plus: +- Capacity reporting to DCM (scheduled registration) +- Full lifecycle event reporting (DEGRADED, MAINTENANCE, UNSANCTIONED_CHANGE, etc.) +- Complete realized state payloads in DCM Unified Data Model format +- Field mapping declaration (CRD fields mapped to DCM Resource Type fields) + +**What it unlocks:** All Level 1 capabilities, plus: +- Intelligent placement โ€” DCM can route requests based on real capacity data +- Drift detection โ€” DCM compares discovered state against realized state +- Full cost attribution โ€” granular resource cost tracking throughout lifecycle +- Cross-cluster management โ€” DCM can route the same resource type to multiple clusters +- Dependency graph participation โ€” operator resources participate in DCM entity relationships + +**Estimated implementation effort:** 2-3 days using the DCM Operator SDK + +### 2.3 Level 3 โ€” Full + +**What it requires:** All Level 2 requirements, plus: +- Sovereignty capability declaration +- Field-level provenance in realized state payloads +- Override control metadata support +- Discovery endpoint (`POST /discover`) โ€” operator can discover existing resources for brownfield ingestion +- Decommission confirmation callback + +**What it unlocks:** All Level 2 capabilities, plus: +- Sovereignty enforcement โ€” DCM can enforce placement and operational constraints per regulatory requirements +- Full audit chain โ€” complete provenance from intent through realization +- Brownfield ingestion โ€” existing resources can be imported into DCM lifecycle management +- Override control enforcement โ€” policy-set field locks honored in operator requests + +**Estimated implementation effort:** 3-5 days using the DCM Operator SDK + +--- + +## 3. Registration API + +### 3.1 Overview + +Operators register with DCM on startup. Registration informs DCM of the operator's endpoint, the resource types it manages, its capabilities, and its conformance level. Registration is idempotent โ€” re-registering with the same name updates the existing registration rather than creating a duplicate. + +### 3.2 Registration Endpoint + +**DCM endpoint:** `POST /api/v1/providers` + +**Timing:** Called by the operator (or adapter) during startup, after the HTTP server is ready. Retried with exponential backoff on failure. Registration failure does not block operator startup โ€” the operator functions normally for Kubernetes consumers even if DCM registration fails. + +### 3.3 Registration Payload + +```yaml +# Registration request payload +provider_registration: + name: + display_name: + conformance_level: <1|2|3> + endpoint: + version: + + service_types: + - service_type: + service_type_uuid: + crd_reference: + group: + version: + kind: + operations_supported: [CREATE, READ, UPDATE, DELETE, DISCOVER] + # DISCOVER only required for Level 3 + field_mapping_ref: + + kubernetes: + cluster_id: + cluster_endpoint: + namespace_strategy: + # per_tenant: one namespace per DCM Tenant + # shared: all DCM resources in one namespace, isolated by labels + # per_resource: one namespace per resource instance + + metadata: + region: + zone: + cluster_type: + cluster_version: + + # Level 2+ required + capacity: + update_mode: + update_frequency_seconds: + + # Level 3 required + sovereignty_capabilities: + data_residency_regions: [] + operational_sovereignty: + hard_tenancy_supported: + air_gapped_capable: + compliance_frameworks: [] +``` + +### 3.4 Registration Response + +```yaml +# Success response +provider_registration_response: + provider_id: + name: + status: + conformance_level_accepted: <1|2|3> + capabilities_enabled: + - service_catalog + - health_monitoring + - cost_tracking + # Level 2+ + - placement + - drift_detection + - cross_cluster_management + # Level 3 + - sovereignty_enforcement + - brownfield_ingestion + - full_audit_chain +``` + +--- + +## 4. Health Check API + +### 4.1 Overview + +DCM polls the operator's health endpoint every 10 seconds (configurable). A healthy operator is eligible to receive new resource requests. An unhealthy operator is excluded from placement decisions. + +### 4.2 Health Endpoint + +**Endpoint:** `GET /health` +**Authentication:** Unauthenticated (or internally secured โ€” operator choice) +**Expected response:** HTTP 200 OK for healthy, any non-200 for unhealthy + +```yaml +# Health response body (optional but recommended) +health_response: + status: pass # pass | warn | fail + version: + uptime_seconds: + kubernetes_connectivity: + dcm_registration_status: + details: + +``` + +### 4.3 State Machine + +- **Ready** โ€” HTTP 200 received. Operator eligible for new requests. +- **NotReady** โ€” Non-200 or timeout received 3 consecutive times (configurable threshold). Operator excluded from placement. Existing resources not affected. +- **Recovery** โ€” Single HTTP 200 transitions NotReady back to Ready immediately. + +--- + +## 5. Capacity Reporting API + +*Required for Level 2 conformance.* + +### 5.1 Overview + +DCM maintains an internal capacity rating per operator, per service type, per location. Operators report capacity on a configurable schedule. DCM uses capacity data for intelligent placement decisions. + +### 5.2 Capacity Registration + +**DCM endpoint:** `POST /api/v1/providers/{provider_id}/capacity` + +```yaml +capacity_report: + provider_id: + report_timestamp: + next_report_at: + capacity_by_service_type: + - service_type_uuid: + available_units: + reserved_units: + committed_units: + unit_definition: + kubernetes_resources: + available_cpu: + available_memory: + available_storage: + node_count: +``` + +### 5.3 Capacity Denial + +When DCM dispatches a request the operator cannot fulfill, the operator **must** reject it with `INSUFFICIENT_RESOURCES`. DCM receives the denial and retries with an alternative provider. + +```yaml +# Denial response to a resource creation request +denial_response: + request_id: + denial_reason: INSUFFICIENT_RESOURCES + denial_timestamp: + service_type_uuid: + estimated_available_at: + details: +``` + +DCM updates its internal capacity rating for this operator immediately upon receiving a denial. + +--- + +## 6. Resource Lifecycle API + +### 6.1 Overview + +DCM dispatches resource lifecycle operations to the operator via standard REST endpoints. The operator translates these into Kubernetes CR operations (Naturalization) and reports results back to DCM in DCM Unified Data Model format (Denaturalization). + +### 6.2 Standard Endpoints + +| Method | Endpoint | Description | Required Level | +|--------|----------|-------------|---------------| +| `POST` | `/api/v1/{service_type}` | Create a new resource | Level 1 | +| `GET` | `/api/v1/{service_type}` | List all resources | Level 1 | +| `GET` | `/api/v1/{service_type}/{resource_id}` | Get a specific resource | Level 1 | +| `PUT` | `/api/v1/{service_type}/{resource_id}` | Update a resource | Level 2 | +| `DELETE` | `/api/v1/{service_type}/{resource_id}` | Delete a resource | Level 1 | +| `POST` | `/api/v1/{service_type}/discover` | Discover existing resources | Level 3 | + +### 6.3 Create Request + +DCM sends the Requested State payload to the operator. The operator naturalizes it to a Kubernetes CR and submits it. The operator responds immediately with a PROVISIONING status โ€” not waiting for reconciliation to complete. + +```yaml +# Create request from DCM โ€” Requested State payload in DCM format +create_request: + request_id: + tenant_uuid: + resource_type_uuid: + resource_type_name: Storage.Database + spec: + + relationships: + + metadata: + override_control: + +``` + +```yaml +# Create response โ€” immediate acknowledgment +create_response: + resource_id: + dcm_request_id: + lifecycle_state: PROVISIONING + kubernetes_reference: + namespace: + name: + uid: +``` + +### 6.4 Realized State Payload + +When the operator's reconciliation loop completes provisioning, it pushes the realized state to DCM. This is the critical Denaturalization step โ€” translating Kubernetes-native status into DCM Unified Data Model format. + +**DCM endpoint:** `PUT /api/v1/instances/{resource_id}/status` + +```yaml +# Realized state payload โ€” DCM Unified Data Model format +realized_state: + resource_id: + dcm_entity_uuid: + lifecycle_state: + realized_timestamp: + + spec: + + + + # Level 3 โ€” provenance for each field + field_provenance: + : + source_type: provider + source_uuid: + timestamp: + + kubernetes_reference: + namespace: + name: + uid: + resource_version: + + relationships: + +``` + +### 6.5 Delete and Decommission + +When DCM requests deletion, the operator deletes the CR and confirms decommission via the realized state endpoint with `lifecycle_state: DECOMMISSIONED`. + +For **Level 3**, the operator must wait for DCM confirmation before deleting โ€” this allows DCM to apply lifecycle policies (retain, detach) before the operator acts. + +```yaml +# Decommission confirmation callback (Level 3) +# DCM calls this before the operator deletes +decommission_confirmation: + resource_id: + lifecycle_policies_applied: + - entity_uuid: + policy_applied: retain + # storage was retained, not deleted with the parent + - entity_uuid: + policy_applied: destroy + proceed_with_deletion: +``` + +--- + +## 7. Field Mapping Specification + +*Required for Level 2 conformance.* + +### 7.1 Overview + +The field mapping declaration tells DCM how to translate between DCM Unified Data Model fields and the operator's CRD fields. This mapping enables DCM to: +- Generate CRs from DCM Requested State payloads (Naturalization) +- Extract DCM Realized State from CR status (Denaturalization) +- Understand which DCM fields correspond to which CRD fields for drift detection + +### 7.2 Field Mapping Declaration Format + +```yaml +field_mapping: + service_type: Storage.Database + service_type_uuid: + crd_reference: + group: postgresql.cnpg.io + version: v1 + kind: Cluster + + # DCM Requested State โ†’ Kubernetes CR (Naturalization) + dcm_to_cr: + - dcm_path: resources.cpu + cr_path: spec.instances[0].resources.requests.cpu + transform: + required: true + + - dcm_path: resources.memory + cr_path: spec.instances[0].resources.requests.memory + transform: gigabytes_to_kubernetes_memory + required: true + + - dcm_path: engine + cr_path: spec.imageName + transform: engine_version_to_image + # engine: postgresql, version: 15 โ†’ imageName: ghcr.io/cloudnative-pg/postgresql:15 + required: true + + - dcm_path: metadata.name + cr_path: metadata.name + required: true + + - dcm_path: tenant_uuid + cr_path: metadata.labels.dcm-tenant-id + required: true + + - dcm_path: dcm_entity_uuid + cr_path: metadata.labels.dcm-entity-id + required: true + # All DCM-managed CRs must be labeled with their DCM entity UUID + # This enables discovery and drift detection + + # Kubernetes CR status โ†’ DCM Realized State (Denaturalization) + cr_status_to_dcm: + - cr_path: status.phase + dcm_path: lifecycle_state + transform: cr_phase_to_dcm_state + # Mapping defined in condition_mappings below + + - cr_path: status.readyInstances + dcm_path: realized_data.ready_instances + transform: none + + - cr_path: status.instancesStatus[0].ip + dcm_path: realized_data.connection.host + transform: none + + - cr_path: status.certificates.serverCASecret + dcm_path: realized_data.tls.ca_secret_ref + transform: none + + # Kubernetes conditions โ†’ DCM lifecycle states + condition_mappings: + - kubernetes_condition: "Ready=True" + dcm_lifecycle_state: OPERATIONAL + + - kubernetes_condition: "Ready=False,Progressing=True" + dcm_lifecycle_state: PROVISIONING + + - kubernetes_condition: "Ready=False,Progressing=False" + dcm_lifecycle_state: FAILED + + - kubernetes_condition: "Degraded=True" + dcm_lifecycle_state: DEGRADED + + # Kubernetes events โ†’ DCM lifecycle events + lifecycle_event_mappings: + - kubernetes_event: condition_change + condition: "Ready=False" + dcm_event: ENTITY_HEALTH_CHANGE + severity: WARNING + + - kubernetes_event: condition_change + condition: "Degraded=True" + dcm_event: DEGRADATION + severity: CRITICAL + + - kubernetes_event: spec_change_without_dcm_request + dcm_event: UNSANCTIONED_CHANGE + severity: WARNING + # Detected when CR spec changes without a corresponding DCM request ID + # Indicates drift โ€” someone modified the CR directly in Kubernetes + + # Namespace strategy implementation + namespace_strategy: + type: per_tenant + namespace_name_pattern: "dcm-{tenant_uuid_short}" + # {tenant_uuid_short} = first 8 chars of tenant UUID + labels_required: + dcm-managed: "true" + dcm-tenant-id: "{tenant_uuid}" + dcm-entity-id: "{entity_uuid}" +``` + +### 7.3 Mandatory CR Labels + +All CRs created by a DCM-conformant operator must carry these labels. These labels enable DCM's discovery and drift detection capabilities: + +| Label | Value | Purpose | +|-------|-------|---------| +| `dcm-managed` | `"true"` | Identifies this CR as DCM-managed | +| `dcm-tenant-id` | DCM Tenant UUID | Tenant ownership | +| `dcm-entity-id` | DCM Entity UUID | Links CR to DCM entity record | +| `dcm-provider-id` | DCM Provider UUID | Which provider created this | +| `dcm-request-id` | DCM Request UUID | Which request created this | + +Any CR change that does not have a corresponding DCM request ID in its update metadata is flagged as an UNSANCTIONED_CHANGE and reported to DCM. + +--- + +## 8. Lifecycle Event API + +*Required for Level 2 conformance.* + +### 8.1 Overview + +Operators must notify DCM of any event that affects the operational status of a managed resource. DCM acts as the Tenant advocate โ€” it receives events, evaluates them through the Policy Engine, and determines the appropriate response. + +### 8.2 Event Endpoint + +**DCM endpoint:** `POST /api/v1/instances/{resource_id}/events` + +### 8.3 Standard Event Types + +| Event Type | Trigger | Severity | Required Level | +|------------|---------|----------|---------------| +| `ENTITY_HEALTH_CHANGE` | CR condition changes | INFO/WARNING | Level 2 | +| `DEGRADATION` | Resource is degraded but operational | WARNING | Level 2 | +| `MAINTENANCE_SCHEDULED` | Planned maintenance window | INFO | Level 2 | +| `MAINTENANCE_STARTED` | Maintenance has begun | INFO | Level 2 | +| `MAINTENANCE_COMPLETED` | Maintenance completed | INFO | Level 2 | +| `UNSANCTIONED_CHANGE` | CR modified without DCM request | WARNING | Level 2 | +| `CAPACITY_CHANGE` | Available capacity changed significantly | INFO | Level 2 | +| `DECOMMISSION_NOTICE` | Operator is shutting down | CRITICAL | Level 2 | +| `PROVIDER_DEGRADATION` | Operator itself is degraded | CRITICAL | Level 2 | + +```yaml +# Event payload +lifecycle_event: + event_uuid: + event_type: UNSANCTIONED_CHANGE + provider_id: + resource_id: + dcm_entity_uuid: + event_timestamp: + severity: WARNING + requires_immediate_action: true + + details: + changed_fields: + - field_path: spec.instances[0].resources.requests.cpu + previous_value: "2000m" + current_value: "4000m" + changed_by: + changed_at: + + kubernetes_reference: + namespace: + name: + resource_version: +``` + +--- + +## 9. DCM Operator SDK + +### 9.1 Overview + +The DCM Operator SDK is an open source Go library that handles all DCM protocol concerns for operator developers. Using the SDK, an operator developer only needs to: + +1. Import the SDK +2. Configure field mappings (declarative YAML) +3. Add SDK hooks at key points in the reconciliation loop + +The SDK handles registration, health check endpoint exposure, capacity reporting, status translation, lifecycle event emission, provenance generation, and label management. + +### 9.2 SDK Initialization + +```go +import dcmsdk "github.com/dcm-project/operator-sdk" + +func main() { + // Load field mapping configuration + mappings, err := dcmsdk.LoadFieldMappings("dcm-mappings.yaml") + + // Initialize DCM SDK + dcm, err := dcmsdk.New(dcmsdk.Config{ + ProviderName: "cloudnativepg-provider", + DisplayName: "CloudNativePG Service Provider", + ConformanceLevel: dcmsdk.Level2, + DCMEndpoint: os.Getenv("DCM_ENDPOINT"), + OperatorEndpoint: os.Getenv("OPERATOR_ENDPOINT"), + FieldMappings: mappings, + CapacityReporter: &PostgresCapacityReporter{}, + }) + + // Start HTTP server with DCM endpoints automatically registered + dcm.StartServer(":8080") + + // Register with DCM on startup + dcm.Register(context.Background()) + + // Start operator manager + mgr.Start(ctrl.SetupSignalHandler()) +} +``` + +### 9.3 Reconciliation Loop Integration + +```go +func (r *ClusterReconciler) Reconcile( + ctx context.Context, + req ctrl.Request, +) (ctrl.Result, error) { + + cluster := &cnpgv1.Cluster{} + if err := r.Get(ctx, req.NamespacedName, cluster); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + + // Check if this CR is DCM-managed + if !r.DCM.IsManagedResource(cluster) { + return ctrl.Result{}, nil + // Not a DCM resource โ€” normal operator behavior + } + + // Detect unsanctioned changes + if r.DCM.IsUnsanctionedChange(cluster) { + r.DCM.ReportEvent(ctx, cluster, dcmsdk.UnsanctionedChange{ + ChangedFields: r.DCM.DetectChangedFields(cluster), + }) + } + + // ... existing reconciliation logic ... + + // Report current state to DCM + realizedState, err := r.DCM.TranslateStatus(cluster) + if err != nil { + return ctrl.Result{}, err + } + r.DCM.ReportStatus(ctx, cluster, realizedState) + + return ctrl.Result{}, nil +} +``` + +### 9.4 SDK Responsibilities + +The SDK automatically handles: +- Self-registration on startup with retry and exponential backoff +- Health check HTTP endpoint (`GET /health`) +- Capacity reporting on configurable schedule +- CR label injection on creation (`dcm-managed`, `dcm-tenant-id`, etc.) +- Unsanctioned change detection (spec change without DCM request ID) +- Status translation using field mapping configuration +- Lifecycle event formatting and delivery to DCM +- Provenance metadata generation for realized state payloads (Level 3) + +--- + +## 10. Kubernetes-to-DCM Concept Mappings + +Understanding how Kubernetes concepts map to DCM concepts is essential for implementing this specification correctly. + +| Kubernetes Concept | DCM Concept | Notes | +|-------------------|-------------|-------| +| Custom Resource Definition (CRD) | Resource Type Specification | CRD schema maps to DCM Resource Type fields | +| Custom Resource (CR) | Requested State โ†’ Realized State | CR is the naturalized form of the DCM payload | +| Operator reconciliation loop | Realization + Drift Detection | Reconciliation IS the realization process | +| CR status subresource | Realized State payload | Status must be denaturalized to DCM format | +| Kubernetes Namespace | DCM Tenant boundary | One namespace per Tenant (per_tenant strategy) | +| ownerReference | Entity Relationship | ownerReferences map to `contains`/`contained_by` relationships | +| Labels/Annotations | DCM Entity metadata | DCM-specific labels declared as mandatory | +| Finalizers | Lifecycle policy enforcement | Finalizers implement `retain` lifecycle policies | +| Kubernetes conditions | DCM lifecycle states | Mapped via condition_mappings declaration | +| Watch events | DCM lifecycle events | Kubernetes watch โ†’ DCM event translation | +| Kubernetes RBAC | DCM IDM/IAM + Policy Engine | Kubernetes RBAC is the runtime enforcement; DCM Policy Engine governs the request | +| Kubernetes cluster | DCM Resource Type: Platform.KubernetesCluster | The cluster itself is a DCM-managed resource | + +--- + +## 11. Conformance Testing + +### 11.1 Overview + +The DCM project provides a conformance test suite that validates an operator's implementation against this specification. Operators that pass the conformance test suite at their declared level can claim DCM conformance. + +### 11.2 Test Suite Structure + +``` +dcm-operator-conformance/ +โ”œโ”€โ”€ level1/ +โ”‚ โ”œโ”€โ”€ registration_test.go +โ”‚ โ”œโ”€โ”€ health_check_test.go +โ”‚ โ””โ”€โ”€ basic_status_test.go +โ”œโ”€โ”€ level2/ +โ”‚ โ”œโ”€โ”€ capacity_test.go +โ”‚ โ”œโ”€โ”€ lifecycle_events_test.go +โ”‚ โ”œโ”€โ”€ realized_state_test.go +โ”‚ โ””โ”€โ”€ field_mapping_test.go +โ””โ”€โ”€ level3/ + โ”œโ”€โ”€ sovereignty_test.go + โ”œโ”€โ”€ provenance_test.go + โ”œโ”€โ”€ discovery_test.go + โ””โ”€โ”€ decommission_confirmation_test.go +``` + +### 11.3 Running the Conformance Tests + +```bash +# Run Level 1 conformance tests against a running operator +dcm-conformance test \ + --level 1 \ + --operator-endpoint https://my-operator:8080 \ + --dcm-endpoint https://dcm-control-plane:8080 \ + --service-type Storage.Database + +# Run all levels +dcm-conformance test --level 3 --operator-endpoint ... +``` + +### 11.4 Conformance Certification + +Operators that pass the conformance test suite may: +- Use the "DCM Compatible โ€” Level N" badge in their documentation +- Be listed in the DCM Operator Registry +- Receive inclusion in the DCM default Service Catalog for participating organizations + +--- + +## 12. Security Considerations + +### 12.1 Authentication + +DCM authenticates outbound requests to operators using the trust model established during registration. Operators must validate that incoming requests originate from the DCM control plane. The specific authentication mechanism is declared in the provider registration: + +```yaml +trust_declaration: + auth_method: + auth_config: +``` + +### 12.2 Namespace Isolation + +When using the `per_tenant` namespace strategy, operators must enforce that resources in one namespace cannot access resources in another namespace. This is the physical enforcement of DCM's hard tenancy model at the Kubernetes level. + +### 12.3 Unsanctioned Change Detection + +Operators must monitor for changes to DCM-managed CRs that did not originate from a DCM request. Any such change is an UNSANCTIONED_CHANGE event and must be reported to DCM immediately. DCM's Policy Engine determines the appropriate response (REVERT, UPDATE_DEFINITION, ALERT, etc.). + +--- + +## 13. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the specification be submitted to CNCF as a sandbox project or proposed as a Kubernetes SIG? | Community adoption strategy | โ“ Unresolved | +| 2 | Should conformance certification be self-certified (test suite passes) or require DCM project review? | Community trust | โ“ Unresolved | +| 3 | How should the specification handle operators that manage cluster-scoped (non-namespaced) resources? | Namespace strategy | โ“ Unresolved | +| 4 | Should the SDK support non-Go operator frameworks (Java Operator SDK, Python kopf)? | Ecosystem breadth | โ“ Unresolved | +| 5 | How does the specification interact with Kubernetes Cluster API โ€” can CAPI clusters be DCM-managed resources? | Scope | โ“ Unresolved | +| 6 | Should there be a Level 0 โ€” a pure label-based passive mode requiring no operator changes? | Adoption friction | โ“ Unresolved | + +--- + +## Appendix A โ€” Example Implementation Checklist + +### Level 1 Checklist +- [ ] Operator registers with DCM on startup via `POST /api/v1/providers` +- [ ] Registration retried with exponential backoff on failure +- [ ] `GET /health` endpoint returns HTTP 200 when healthy +- [ ] `GET /health` returns non-200 when operator cannot fulfill requests +- [ ] Status reported to DCM when resource transitions to OPERATIONAL, FAILED, or DECOMMISSIONED +- [ ] All DCM-managed CRs labeled with mandatory DCM labels +- [ ] Create response returns PROVISIONING state immediately + +### Level 2 Checklist +- [ ] All Level 1 items complete +- [ ] Capacity reported to DCM on configurable schedule +- [ ] Capacity denial returns `INSUFFICIENT_RESOURCES` with proper payload +- [ ] Full realized state payload in DCM Unified Data Model format +- [ ] Field mapping declaration complete and validated +- [ ] All standard lifecycle event types implemented +- [ ] Unsanctioned change detection active +- [ ] CR condition changes translated to DCM lifecycle events + +### Level 3 Checklist +- [ ] All Level 2 items complete +- [ ] Sovereignty capabilities declared in registration +- [ ] Field-level provenance included in realized state payloads +- [ ] `POST /discover` endpoint implemented +- [ ] Decommission confirmation callback handled +- [ ] Override control metadata honored in CR creation + +--- + +## Appendix B โ€” Relationship to Other Specifications + +- **DCM Data Model** โ€” defines the Unified Data Model format used in all API payloads +- **DCM Service Provider Contract** โ€” the general provider contract this specification extends +- **DCM Resource Type Registry** โ€” where DCM Resource Types are registered; operators must reference registry UUIDs +- **AEP (API Enhancement Proposals)** โ€” the DCM API follows AEP standards for REST API design +- **OpenAPI 3.1.0** โ€” all API schemas are defined in OpenAPI 3.1.0 + +--- + +*This specification is maintained by the DCM Project. For questions, contributions, or conformance certification see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/operator-sdk-api.md b/content/docs/architecture/specifications/operator-sdk-api.md new file mode 100644 index 0000000..b2ab69b --- /dev/null +++ b/content/docs/architecture/specifications/operator-sdk-api.md @@ -0,0 +1,645 @@ +--- +title: "DCM Operator SDK API" +type: docs +weight: 3 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Version:** 0.1.0-draft +**Status:** Design โ€” Not yet implemented +**Document Type:** Technical Design +**Language:** Go +**Repository:** https://github.com/dcm-project/operator-sdk +**Related Documents:** [DCM Operator Interface Specification](../../specifications/operator-interface-spec/) | [Kubernetes Compatibility](../../specifications/kubernetes-compatibility/) + +--- + +## 1. Purpose + +This document defines the public API of the DCM Operator SDK โ€” the Go library that enables Kubernetes operators to implement the DCM Operator Interface Specification with minimal code changes. The SDK handles all DCM protocol concerns so that operator developers only need to implement business logic โ€” field mappings and reconciliation hooks. + +**Design principle:** The SDK must be adoptable in a single day. If implementing Level 1 takes more than a day, the API is too complex. + +--- + +## 2. Package Structure + +``` +github.com/dcm-project/operator-sdk/ +โ”œโ”€โ”€ pkg/ +โ”‚ โ”œโ”€โ”€ client/ # DCM control plane client +โ”‚ โ”œโ”€โ”€ config/ # SDK configuration +โ”‚ โ”œโ”€โ”€ mapping/ # Field mapping engine +โ”‚ โ”œโ”€โ”€ reconciler/ # Reconciliation loop helpers +โ”‚ โ”œโ”€โ”€ registration/ # Provider registration +โ”‚ โ”œโ”€โ”€ server/ # HTTP server with DCM endpoints +โ”‚ โ”œโ”€โ”€ status/ # Status translation and reporting +โ”‚ โ”œโ”€โ”€ events/ # Lifecycle event types and emission +โ”‚ โ”œโ”€โ”€ discovery/ # Brownfield discovery helpers (Level 3) +โ”‚ โ””โ”€โ”€ provenance/ # Provenance metadata generation (Level 3) +โ”œโ”€โ”€ api/ +โ”‚ โ””โ”€โ”€ v1/ # DCM API type definitions +โ””โ”€โ”€ examples/ + โ”œโ”€โ”€ level1/ # Minimal Level 1 implementation example + โ”œโ”€โ”€ level2/ # Full Level 2 implementation example + โ””โ”€โ”€ level3/ # Complete Level 3 implementation example +``` + +--- + +## 3. Core Types + +### 3.1 Config + +```go +// Config is the primary SDK configuration structure. +// All fields have sensible defaults โ€” only DCMEndpoint, +// OperatorEndpoint, and ProviderName are required. +type Config struct { + // Required + ProviderName string + DCMEndpoint string + OperatorEndpoint string + + // Required โ€” at least one ServiceType must be declared + ServiceTypes []ServiceTypeConfig + + // Optional โ€” defaults to Level1 if not specified + ConformanceLevel ConformanceLevel + + // Optional โ€” defaults to "unknown" if not specified + DisplayName string + Version string + + // Optional โ€” field mappings loaded from file if not inline + FieldMappings []FieldMapping + FieldMappingFiles []string + + // Level 2+ โ€” capacity reporter + // If nil and ConformanceLevel >= Level2, SDK returns error on init + CapacityReporter CapacityReporter + + // Level 3 โ€” sovereignty and provenance + SovereigntyCapabilities *SovereigntyCapabilities + + // Optional โ€” HTTP server configuration + ServerConfig ServerConfig + + // Optional โ€” registration retry configuration + RegistrationConfig RegistrationConfig + + // Optional โ€” health check configuration + HealthConfig HealthConfig + + // Optional โ€” logger (defaults to zap logger) + Logger logr.Logger +} + +// ConformanceLevel declares the operator's DCM conformance level +type ConformanceLevel int + +const ( + Level1 ConformanceLevel = 1 + Level2 ConformanceLevel = 2 + Level3 ConformanceLevel = 3 +) + +// ServiceTypeConfig declares a DCM Resource Type this operator implements +type ServiceTypeConfig struct { + // DCM Resource Type name โ€” e.g., "Storage.Database" + ServiceTypeName string + // DCM Resource Type UUID from the registry + ServiceTypeUUID string + // Kubernetes CRD this service type maps to + CRDReference CRDReference + // Operations this operator supports for this type + OperationsSupported []Operation +} + +type CRDReference struct { + Group string + Version string + Kind string +} + +type Operation string + +const ( + OperationCreate Operation = "CREATE" + OperationRead Operation = "READ" + OperationUpdate Operation = "UPDATE" + OperationDelete Operation = "DELETE" + OperationDiscover Operation = "DISCOVER" // Level 3 only +) +``` + +### 3.2 Client โ€” DCM Control Plane Interface + +```go +// Client is the interface for communicating with the DCM control plane. +// The SDK creates and manages this internally โ€” operator developers +// use it only through the higher-level SDK methods. +type Client interface { + // Register sends the provider registration to DCM. + // Returns the DCM-assigned provider UUID on success. + Register(ctx context.Context, reg ProviderRegistration) (string, error) + + // ReportStatus sends a realized state payload to DCM. + ReportStatus(ctx context.Context, resourceID string, status RealizedState) error + + // ReportEvent sends a lifecycle event to DCM. + ReportEvent(ctx context.Context, resourceID string, event LifecycleEvent) error + + // ReportCapacity sends a capacity update to DCM. + // Required for Level 2+. + ReportCapacity(ctx context.Context, capacity CapacityReport) error + + // ConfirmDecommission acknowledges a decommission request from DCM. + // Required for Level 3. + ConfirmDecommission(ctx context.Context, resourceID string, confirmation DecommissionConfirmation) error +} +``` + +### 3.3 SDK โ€” Primary Interface + +```go +// SDK is the primary interface for the DCM Operator SDK. +// Operator developers interact with DCM through this interface. +type SDK interface { + // --- Lifecycle --- + + // Register sends the provider registration to DCM. + // Called during operator startup. Retries with exponential backoff. + // Does not block โ€” runs in background goroutine. + Register(ctx context.Context) + + // Shutdown gracefully deregisters the operator from DCM and + // stops background goroutines. + Shutdown(ctx context.Context) error + + // --- HTTP Server --- + + // StartServer starts the HTTP server with all DCM-required endpoints. + // Blocks until context is cancelled. + StartServer(ctx context.Context, addr string) error + + // Handler returns an http.Handler for use with an existing HTTP server. + // Alternative to StartServer when the operator already has an HTTP server. + Handler() http.Handler + + // --- Reconciliation Helpers --- + + // IsManagedResource returns true if the Kubernetes object + // carries DCM management labels. + IsManagedResource(obj client.Object) bool + + // IsUnsanctionedChange returns true if the object's spec has changed + // without a corresponding DCM request annotation. + // Used in reconciliation loops to detect drift. + IsUnsanctionedChange(obj client.Object) bool + + // DetectChangedFields returns the list of fields that changed + // relative to the last known DCM request state. + DetectChangedFields(obj client.Object) []FieldChange + + // InjectLabels adds DCM-required labels to a Kubernetes object + // before creation. Called before submitting a CR to Kubernetes. + InjectLabels(obj client.Object, req CreateRequest) client.Object + + // AnnotateRequest adds the DCM request ID annotation to a + // Kubernetes object. Used to mark changes as DCM-sanctioned. + AnnotateRequest(obj client.Object, requestID string) client.Object + + // --- Status Translation --- + + // TranslateStatus translates a Kubernetes object's status + // to a DCM RealizedState using the configured field mappings. + TranslateStatus(obj client.Object) (RealizedState, error) + + // ReportStatus translates and reports status to DCM in one call. + // Convenience wrapper for TranslateStatus + Client.ReportStatus. + ReportStatus(ctx context.Context, obj client.Object) error + + // --- Event Emission --- + + // ReportEvent sends a lifecycle event to DCM. + ReportEvent(ctx context.Context, obj client.Object, event LifecycleEventType, details EventDetails) error + + // ReportUnsanctionedChange is a convenience method for reporting + // an unsanctioned change event with the detected changed fields. + ReportUnsanctionedChange(ctx context.Context, obj client.Object, changes []FieldChange) error + + // ReportDegradation reports a DEGRADATION event to DCM. + ReportDegradation(ctx context.Context, obj client.Object, reason string) error + + // ReportHealthChange reports an ENTITY_HEALTH_CHANGE event. + ReportHealthChange(ctx context.Context, obj client.Object, healthy bool, reason string) error + + // --- Capacity --- + + // StartCapacityReporting starts the background capacity reporting + // goroutine. Required for Level 2+. Called automatically by StartServer. + StartCapacityReporting(ctx context.Context) + + // --- Discovery (Level 3) --- + + // BuildDiscoveryResponse queries Kubernetes for existing resources + // and returns them in DCM Realized State format. + // Used to implement the POST /discover endpoint. + BuildDiscoveryResponse(ctx context.Context, k8sClient client.Client, opts DiscoveryOptions) ([]RealizedState, error) +} +``` + +--- + +## 4. Field Mapping API + +```go +// FieldMapping declares how a DCM Resource Type maps to a Kubernetes CRD. +// Can be loaded from a YAML file or declared inline in Go. +type FieldMapping struct { + ServiceTypeName string + ServiceTypeUUID string + CRDReference CRDReference + + // DCM Requested State โ†’ Kubernetes CR spec (Naturalization) + DCMToCR []FieldMap + + // Kubernetes CR status โ†’ DCM Realized State (Denaturalization) + CRStatusToDCM []FieldMap + + // Kubernetes conditions โ†’ DCM lifecycle states + ConditionMappings []ConditionMapping + + // Kubernetes events โ†’ DCM lifecycle event types + LifecycleEventMappings []LifecycleEventMapping + + // Namespace strategy for this resource type + NamespaceStrategy NamespaceStrategy +} + +// FieldMap declares a single field translation +type FieldMap struct { + // Source field path โ€” dot-notation, supports array indexing + // e.g., "resources.cpu" or "nodes.controlPlane[0].cpu" + SourcePath string + + // Destination field path + DestPath string + + // Transform function name โ€” registered in the transform registry + // "none" for direct copy, or a named transform + Transform string + + // Required โ€” if true and source field is absent, returns error + Required bool + + // Default โ€” used when source field is absent and Required is false + Default interface{} +} + +// ConditionMapping maps a Kubernetes condition to a DCM lifecycle state +type ConditionMapping struct { + // Kubernetes condition expression โ€” e.g., "Ready=True" + // Supports AND: "Ready=False,Progressing=True" + KubernetesCondition string + + // DCM lifecycle state + DCMLifecycleState LifecycleState +} + +// LifecycleEventMapping maps a Kubernetes event to a DCM event type +type LifecycleEventMapping struct { + // "condition_change" | "spec_change_without_dcm_request" | "deletion" + KubernetesEvent string + + // Condition that triggers this mapping (for condition_change events) + Condition string + + // DCM event type + DCMEventType LifecycleEventType + + // Severity + Severity EventSeverity +} + +// Transform registry โ€” operator developers register custom transforms +type TransformRegistry interface { + // Register adds a named transform function + Register(name string, fn TransformFunc) error + + // Get retrieves a transform function by name + Get(name string) (TransformFunc, error) +} + +// TransformFunc transforms a value from source to destination format +type TransformFunc func(value interface{}) (interface{}, error) +``` + +--- + +## 5. Status and State Types + +```go +// LifecycleState represents the DCM lifecycle state of a resource +type LifecycleState string + +const ( + LifecycleStateProvisioning LifecycleState = "PROVISIONING" + LifecycleStateOperational LifecycleState = "OPERATIONAL" + LifecycleStateDegraded LifecycleState = "DEGRADED" + LifecycleStateSuspended LifecycleState = "SUSPENDED" + LifecycleStateFailed LifecycleState = "FAILED" + LifecycleStateDecommissioned LifecycleState = "DECOMMISSIONED" +) + +// RealizedState is the DCM Unified Data Model representation of +// a resource's realized state. This is what the operator sends +// to DCM after successful provisioning or status change. +type RealizedState struct { + // DCM resource ID (returned by DCM in the create request) + ResourceID string + + // DCM entity UUID + DCMEntityUUID string + + // Current lifecycle state + LifecycleState LifecycleState + + // Timestamp of this realization + RealizedTimestamp time.Time + + // All realized fields in DCM Unified Data Model format + Spec map[string]interface{} + + // Level 3 โ€” field-level provenance + FieldProvenance map[string]FieldProvenance + + // Kubernetes reference for correlation + KubernetesReference KubernetesReference + + // Relationships created during realization + Relationships []RelationshipRecord +} + +// KubernetesReference carries Kubernetes-specific identity for correlation +type KubernetesReference struct { + Namespace string + Name string + UID types.UID + ResourceVersion string + Generation int64 +} + +// FieldProvenance carries lineage for a single field (Level 3) +type FieldProvenance struct { + SourceType string // "provider" + SourceUUID string // operator provider UUID + Timestamp time.Time + Reason string +} +``` + +--- + +## 6. Event Types + +```go +// LifecycleEventType represents a DCM lifecycle event type +type LifecycleEventType string + +const ( + EventEntityHealthChange LifecycleEventType = "ENTITY_HEALTH_CHANGE" + EventDegradation LifecycleEventType = "DEGRADATION" + EventMaintenanceScheduled LifecycleEventType = "MAINTENANCE_SCHEDULED" + EventMaintenanceStarted LifecycleEventType = "MAINTENANCE_STARTED" + EventMaintenanceCompleted LifecycleEventType = "MAINTENANCE_COMPLETED" + EventUnsanctionedChange LifecycleEventType = "UNSANCTIONED_CHANGE" + EventCapacityChange LifecycleEventType = "CAPACITY_CHANGE" + EventDecommissionNotice LifecycleEventType = "DECOMMISSION_NOTICE" + EventProviderDegradation LifecycleEventType = "PROVIDER_DEGRADATION" +) + +// EventSeverity represents the severity of a lifecycle event +type EventSeverity string + +const ( + SeverityInfo EventSeverity = "INFO" + SeverityWarning EventSeverity = "WARNING" + SeverityCritical EventSeverity = "CRITICAL" +) + +// LifecycleEvent is the payload sent to DCM for a lifecycle event +type LifecycleEvent struct { + EventUUID string + EventType LifecycleEventType + ProviderID string + ResourceID string + DCMEntityUUID string + EventTimestamp time.Time + Severity EventSeverity + RequiresImmediateAction bool + Details EventDetails + KubernetesReference KubernetesReference +} + +// EventDetails carries event-specific detail data +type EventDetails struct { + // For UNSANCTIONED_CHANGE events + ChangedFields []FieldChange + + // For DEGRADATION events + DegradationReason string + AffectedComponents []string + + // For MAINTENANCE events + MaintenanceWindow *MaintenanceWindow + MaintenanceReason string + + // For CAPACITY_CHANGE events + PreviousCapacity *CapacityReport + CurrentCapacity *CapacityReport + + // Human-readable message for any event type + Message string +} + +// FieldChange describes a single field change in an unsanctioned change event +type FieldChange struct { + FieldPath string + PreviousValue interface{} + CurrentValue interface{} + ChangedBy string // Kubernetes user or service account + ChangedAt time.Time +} +``` + +--- + +## 7. Capacity Types + +```go +// CapacityReporter is the interface operator developers implement +// to report capacity data to DCM. The SDK calls this on schedule. +type CapacityReporter interface { + // GetCapacity returns the current capacity for all service types. + // Called by the SDK on the configured reporting schedule. + GetCapacity(ctx context.Context) (CapacityReport, error) +} + +// CapacityReport contains capacity data for all service types +type CapacityReport struct { + ProviderID string + ReportTimestamp time.Time + NextReportAt time.Time + CapacityByServiceType []ServiceTypeCapacity +} + +// ServiceTypeCapacity contains capacity for a single service type +type ServiceTypeCapacity struct { + ServiceTypeUUID string + AvailableUnits int + ReservedUnits int + CommittedUnits int + UnitDefinition string + KubernetesResources KubernetesResourceCapacity +} + +// KubernetesResourceCapacity contains raw Kubernetes resource availability +type KubernetesResourceCapacity struct { + AvailableCPUMillicores int64 + AvailableMemoryBytes int64 + AvailableStorageBytes int64 + NodeCount int +} +``` + +--- + +## 8. Constructor and Initialization + +```go +// New creates and initializes a new DCM SDK instance. +// Returns an error if the configuration is invalid or +// if required components for the declared conformance level +// are missing. +func New(config Config) (SDK, error) + +// NewWithClient creates a new SDK instance with a pre-configured +// DCM client. Used primarily for testing. +func NewWithClient(config Config, client Client) (SDK, error) + +// LoadFieldMappings loads field mapping declarations from YAML files. +// Accepts one or more file paths or glob patterns. +func LoadFieldMappings(paths ...string) ([]FieldMapping, error) + +// MustNew creates a new SDK instance and panics if initialization fails. +// Convenience function for use in main() where error handling via +// panic is acceptable. +func MustNew(config Config) SDK +``` + +--- + +## 9. Minimal Level 1 Example + +```go +package main + +import ( + "context" + "os" + + dcmsdk "github.com/dcm-project/operator-sdk" + ctrl "sigs.k8s.io/controller-runtime" +) + +func main() { + // Minimal Level 1 configuration + dcm, err := dcmsdk.New(dcmsdk.Config{ + ProviderName: "my-operator", + DisplayName: "My Operator DCM Provider", + DCMEndpoint: os.Getenv("DCM_ENDPOINT"), + OperatorEndpoint: os.Getenv("OPERATOR_ENDPOINT"), + ConformanceLevel: dcmsdk.Level1, + ServiceTypes: []dcmsdk.ServiceTypeConfig{ + { + ServiceTypeName: "Storage.Database", + ServiceTypeUUID: "dcm-registry-uuid-for-storage-database", + CRDReference: dcmsdk.CRDReference{ + Group: "postgresql.cnpg.io", + Version: "v1", + Kind: "Cluster", + }, + OperationsSupported: []dcmsdk.Operation{ + dcmsdk.OperationCreate, + dcmsdk.OperationRead, + dcmsdk.OperationDelete, + }, + }, + }, + FieldMappingFiles: []string{"dcm-mappings.yaml"}, + }) + if err != nil { + panic(err) + } + + ctx := ctrl.SetupSignalHandler() + + // Register with DCM in background โ€” does not block startup + dcm.Register(ctx) + + // Start HTTP server with health + DCM endpoints + go dcm.StartServer(ctx, ":8080") + + // Start operator manager (existing code unchanged) + mgr, _ := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{}) + mgr.Start(ctx) +} + +// In reconciliation loop โ€” minimal Level 1 additions +func (r *ClusterReconciler) Reconcile( + ctx context.Context, + req ctrl.Request, +) (ctrl.Result, error) { + + cluster := &cnpgv1.Cluster{} + if err := r.Get(ctx, req.NamespacedName, cluster); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + + // Only process DCM-managed resources + if !r.DCM.IsManagedResource(cluster) { + return ctrl.Result{}, nil + } + + // Existing reconciliation logic here... + + // Report status to DCM (SDK handles translation via field mappings) + r.DCM.ReportStatus(ctx, cluster) + + return ctrl.Result{}, nil +} +``` + +--- + +## 10. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the SDK support non-Go operator frameworks via a language-agnostic REST adapter? | Ecosystem breadth | โ“ Unresolved | +| 2 | How should the SDK handle DCM endpoint unavailability โ€” queue events locally or drop? | Reliability | โ“ Unresolved | +| 3 | Should field mappings support dynamic resolution โ€” a transform that queries external data? | Flexibility | โ“ Unresolved | +| 4 | Should the SDK provide a testing framework for unit testing operator-DCM integration? | Developer experience | โ“ Unresolved | +| 5 | Should the SDK expose metrics (Prometheus) for DCM registration status, event delivery success, etc.? | Observability | โ“ Unresolved | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From f65237560315fa7b38eddb8feb70133d0aece43f Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Thu, 26 Mar 2026 11:47:25 -0500 Subject: [PATCH 02/49] Added 2nd round of updates. Mostly policy and relationship mapping. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 524 +++++++++++++++++- .../docs/architecture/data-model/_index.md | 2 + .../data-model/entity-relationships.md | 466 +++++++++++++++- .../architecture/data-model/four-states.md | 59 +- .../data-model/layering-and-versioning.md | 227 +++++++- .../data-model/resource-service-entities.md | 71 ++- content/docs/architecture/overview.md | 34 +- 7 files changed, 1324 insertions(+), 59 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 707a8b1..a47d3b9 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -314,14 +314,16 @@ From lowest to highest precedence: - **Transformation** โ€” enriches or modifies the payload. Adds missing fields, applies standards. Recorded in provenance. - **GateKeeper** โ€” highest authority. Can override any field including consumer-declared values. Used for sovereignty constraints, security mandates, and hard compliance rules. All overrides recorded in provenance. -### 6.6 Assembly Process (Seven Steps) -1. **Intent Capture** โ€” Request Layer stored as Intent State in Intent Store. No modification. +### 6.6 Assembly Process (Nine Steps) +1. **Intent Capture** โ€” Request Layer stored as Intent State. No modification. 2. **Layer Resolution** โ€” Processor identifies applicable layers by Resource Type and organizational context. 3. **Layer Merge** โ€” Layers merged in precedence order. Each field records source layer UUID in provenance. -4. **Request Layer Application** โ€” Consumer values applied last in data layer merge. Overrides recorded in provenance. -5. **Transformation Policies** โ€” Enrich and modify payload. Provenance recorded per field. -6. **Validation Policies** โ€” Check payload. Failures reject request with reason. -7. **GateKeeper Policies** โ€” Hard overrides applied. Provenance recorded. Payload stored as Requested State. +4. **Request Layer Application** โ€” Consumer values applied. Overrides recorded in provenance. +5. **Pre-Placement Policies** (`placement_phase: pre`) โ€” Transformation โ†’ Validation โ†’ GateKeeper. Produces placement constraints. +6. **Placement Engine โ€” Placement Loop** โ€” Iterates candidate providers. Per candidate: Reserve Query (atomic: verify + metadata + hold) โ†’ Loop Policy Phase. Confirmed on first passing candidate. See Section 6.12. +7. **Post-Placement Policies** (`placement_phase: post`) โ€” Transformation โ†’ Validation โ†’ GateKeeper. Has access to placement block including selected provider and all returned metadata. +8. **Requested State Storage** โ€” Complete payload stored: resource fields + placement block + policy gap records + enrichment_status. +9. **Provider Dispatch** โ€” Dispatched to selected provider via API Gateway. Hold confirmed by dispatch. ### 6.7 Key Rules - Core Layers are type-agnostic โ€” applied to every request regardless of Resource Type @@ -433,6 +435,85 @@ override_matrix: **Actor extensibility:** Custom actors default to `deny`, require explicit grants, follow universal versioning and deprecation model. +### 6.12 Placement Engine and Placement Loop + +The **Placement Engine** is a distinct named control plane component โ€” a peer to the Policy Engine, not subordinate to it. It takes the policy-processed payload and placement constraints, builds a scored candidate list, and iterates until placement is confirmed or candidates are exhausted. + +**Input:** assembled payload + placement constraints + provider registry + topology data +**Output:** `selected_provider_uuid` + placement block written to Requested State + +**Placement loop per candidate:** +``` +Reserve Query โ†’ Loop Policy Phase + confirmed/partial โ†’ policies โ†’ pass/warn โ†’ PLACEMENT CONFIRMED + insufficient/refused โ†’ next candidate + policy reject_candidate โ†’ release hold, next candidate + policy gatekeep โ†’ release hold, ABORT, reject request +No candidates โ†’ on_exhaustion: reject | escalate | manual_placement +``` + +**Reserve Query โ€” single atomic call (primary placement query):** +- Verifies provider can satisfy placement constraints +- Returns all available metadata in one response +- Places a resource hold for `hold_ttl_seconds` +- Response: `confirmed | partial | insufficient | refused` +- `partial` = hold confirmed but some requested metadata unavailable + +**Non-hold queries (outside the loop):** +`capacity_query | metadata_query | constraint_verification` โ€” informational, no side effects + +### 6.13 Policy Placement Phase and Required Context + +**`placement_phase` on every policy:** +- `pre` โ€” steps 5 (before provider known) โ€” default +- `loop` โ€” step 6 (inside loop, evaluates reserve query response) +- `post` โ€” step 7 (after placement confirmed, provider known) +- `both` โ€” pre and post (not loop) + +**`required_context` for missing metadata:** +```yaml +policy: + placement_phase: loop + required_context: + - field: placement.provider_metadata.sovereignty_certifications + if_absent: gatekeep # block if this field is missing + - field: placement.provider_metadata.patch_level + if_absent: warn # proceed with warning + - field: placement.topology.rack + if_absent: skip # not applicable if absent +``` + +**Missing metadata behavior:** +| Situation | Behavior | Audit Record | +|-----------|---------|-------------| +| Field absent, `required_context: gatekeep` | Release hold, abort loop, reject request | Policy rejection with missing field detail | +| Field absent, `required_context: warn` | Record warning, proceed | Warning in Requested State | +| Field absent, `required_context: skip` | Not evaluated | Skipped in provenance | +| Field absent, no policy declares it | `implicit_approval` | `policy_gap_record` | + +### 6.14 Policy Gap Record and Implicit Approval + +When a field is absent and no active policy has declared `required_context` for it, the result is **implicit approval** โ€” not unknown, not unchecked, but explicitly recorded: + +```yaml +policy_gap_record: + request_uuid: + field: patch_level + field_value: null + evaluation_result: implicit_approval + reason: "No active policy declared required_context for this field." + provider_uuid: + recorded_at: + resolution_expected: +``` + +Provider is expected to complete missing metadata in: +1. **Realized payload** โ€” provider returns full metadata on realization +2. **Discovery loop** โ€” periodic discovery fills remaining gaps + +The realized entity's `enrichment_status: pending | partial | complete` tracks metadata completeness. + + --- ## SECTION 7 โ€” RESOURCE TYPE HIERARCHY AND SERVICE CATALOG @@ -738,6 +819,101 @@ All relationships form a traversable graph. Used for: rehydration (full graph tr ### 11.10 Supersedes Dependency Graph The Entity Relationship model unifies the previously separate dependency graph concept. The dependency graph IS the relationship graph at pre-realization time โ€” same structure, different lifecycle state. +### 11.11 Lifecycle Policy Conflict Resolution (Q57 resolved) +Lifecycle policy fields on relationships are **just fields**. They carry the same `override` metadata, the same provenance obligations, and resolve under the same Policy Engine authority hierarchy as any other DCM field. No special case โ€” minimum variance. + +**Priority schema governs conflicts within a tier.** Highest numeric priority runs first. First policy to set `override: immutable` on a lifecycle policy field locks it. `immutable_ceiling: absolute` applies for compliance mandates that must survive future policy changes. + +**Ingestion conflict detection applies.** Two policies declaring conflicting lifecycle policies without priority differentiation โ†’ CONFLICT ERROR at ingestion. + +**DCM System Policies:** +- `REL-008` โ€” A `constituent` relationship lifecycle policy may not be set to `ignore` for `on_related_destroy` +- `REL-009` โ€” Lifecycle policy conflicts between policies are resolved by the standard Policy Engine authority hierarchy โ€” no special case + +### 11.12 Relationship Type ร— Nature Matrix (valid combinations) + +The two relationship dimensions form an explicit matrix. Invalid combinations are rejected by the Policy Engine at request time (REL-013). + +| | `constituent` | `operational` | `informational` | +|---|---|---|---| +| **`requires`** | โœ… Core constituent | โœ… Hard operational dependency | โŒ Invalid | +| **`depends_on`** | โœ… Soft constituent | โœ… **Allocated resource cell** | โœ… Awareness | +| **`contains`** | โœ… Ownership container | โš ๏ธ Rare โ€” justify explicitly | โŒ Invalid | +| **`references`** | โŒ Invalid | โŒ Invalid | โœ… **Business context cell** | +| **`peer`** | โŒ Invalid | โœ… Operational peers | โœ… Informational peers | +| **`manages`** | โœ… Component management | โœ… Operational management | โœ… Audit management | + +- `operational` + `depends_on` = **allocated resource cell** โ€” cross-tenant allocations live here +- `informational` + `references` = **business context cell** โ€” Business Unit, Cost Center etc. + +### 11.13 Cross-Tenant Relationships (Q59 resolved) + +Relationship **nature** governs cross-tenant permissions: + +| Nature | Cross-Tenant? | Rule | +|--------|--------------|------| +| `constituent` | โŒ Never | REL-010 | +| `operational` | โœ… With dual authorization | REL-011 | +| `informational` | โœ… Unless deny_all | REL-012 | + +**Hard tenancy declaration** on Tenant entity: +```yaml +hard_tenancy: + cross_tenant_relationships: operational_only + # deny_all | operational_only | informational_only | allow_all +``` + +### 11.14 Allocated Resource Model (Q59 extension, Q61 partial) + +An **Allocated Resource** is a pre-defined discrete slice of a parent resource made available by the owning Tenant for consuming Tenants to claim. It becomes a **first-class entity** in the consuming Tenant's scope with its own UUID, lifecycle, and governance. + +**Relationship:** `depends_on` + `operational` + `cross_tenant: true` + `allocation_uuid` + +**Parent pre-defines** `available_allocations` โ€” consuming Tenant claims โ†’ DCM creates allocated entity + relationship โ†’ parent tracks in `active_allocations` with `notification_endpoint`. + +**Lifecycle events** propagate from parent to all active allocations per each allocation's `parent_lifecycle_policy`: +`on_parent_destroy | on_parent_suspend | on_parent_maintenance | on_parent_degrade | on_parent_capacity_change` + +**System policies:** REL-013 (invalid matrix combinations rejected), REL-014 (claim requires available allocation record) + +### 11.15 Shared Resource Model โ€” Same-Tenant (Q61 resolved) + +A **Shared Resource** is an entity within a single Tenant that has active relationships from multiple parent entities. DCM maintains `active_relationship_count` โ€” the number of active constituent/operational relationships. Informational relationships never count (REL-016). + +**`sharing_model` on entity:** +```yaml +sharing_model: + shareable: true + sharing_scope: tenant + active_relationship_count: 3 # DCM-maintained + minimum_relationship_count: 0 + on_last_relationship_released: +``` + +**`shareability` on Resource Type Specification:** +- `shareability.allowed: true` โ€” instances can have multiple active relationships +- `shareability.allowed: false` (e.g., `Compute.BootDisk`) โ€” second relationship rejected (REL-017) +- `max_active_relationships` โ€” optional cap (e.g., license seat limits) + +**Destruction deferral (REL-015):** Destructive lifecycle actions on shared resources are deferred until `active_relationship_count` reaches `minimum_relationship_count`. Deferred destruction is recorded in `deferred_destruction_record`. + +### 11.16 Lifecycle Action Hierarchy โ€” Save Overrides Destroy (REL-018) + +When multiple relationships produce different lifecycle action recommendations on a shared resource, the most conservative action wins: + +``` +retain > notify > suspend > detach > cascade > destroy +``` + +`retain` always beats `destroy` โ€” the save_overrides_destroy rule. Applies automatically per REL-018. + +**Conflict detection (REL-019):** When recommendations differ, a `lifecycle_conflict_record` is created: +- Severity `info` โ€” hierarchy resolved cleanly (e.g., retain beats destroy non-adjacent) +- Severity `warning` โ€” adjacent levels or `notify` is the winning action โ€” notifications sent +- Severity `critical` โ€” immutable lifecycle lock overridden by REL-018 โ€” platform admin notified + +**Unified model:** Same-tenant sharing (`active_relationship_count`) and cross-tenant allocation (`active_allocations`) are the same reference-counting concept at different scopes. Both defer destructive actions until the last relationship/allocation is released. + --- ## SECTION 12 โ€” INFORMATION PROVIDERS @@ -982,6 +1158,14 @@ Providers must honor a multi-dimensional contract: - Report Realized State completely and accurately - Handle drift detection requests +**Query Contract** +- Support `reserve_query` โ€” atomic: verify constraints + return metadata + place hold +- Support `capacity_query` (informational, no hold) +- Support `metadata_query` (informational, no hold) +- Declare which query types and metadata fields are supported in provider registration +- Complete missing metadata in realized payload or discovery loop +- `reserve_query` response must include `missing_metadata` for any requested fields not returned + **Trust Contract** *(validation mechanism โ€” to be detailed)* - Providers must be validated and certified to participate - Trust is established at onboarding @@ -1002,6 +1186,7 @@ Providers must honor a multi-dimensional contract: - **Atomic Providers** โ€” manage a single fundamental resource type (VM, IP, VLAN, container) - **Meta Providers** โ€” compose multiple providers as components of their own service - **Process Providers** โ€” purely process-based (no infrastructure resource, but a workflow or automation) +- **Policy Providers** โ€” supply policies from external authoritative sources; follow same base contract; three delivery modes (push/pull/webhook); three formats (dcm_native/opa_rego/external_schema); trust level governs max authority - **Real-world providers** are typically combinations of all three --- @@ -1063,6 +1248,23 @@ The Policy Engine exclusively sets `override_control` metadata. Three levels (se - Level 2 โ€” simple `override: allow|constrained|immutable` - Level 3 โ€” full `override_matrix` with per-actor permissions and trusted grants +### 17.9 Policy Placement Phase +Every policy declares when in the assembly process it executes: +- `pre` โ€” before placement (default) โ€” no provider context +- `loop` โ€” inside the placement loop โ€” has reserve query response data +- `post` โ€” after placement confirmed โ€” has full placement block +- `both` โ€” pre and post + +### 17.10 Policy Required Context +Policies declare what fields they need and what to do when fields are absent: +```yaml +required_context: + - field: placement.provider_metadata.sovereignty_certifications + if_absent: + if_absent_reason: +``` +If no policy declares `required_context` for an absent field โ†’ `implicit_approval` recorded in `policy_gap_record`. The system has no implicit opinion beyond what policies state. + --- ## SECTION 18 โ€” KEY USE CASES @@ -1082,11 +1284,24 @@ Enforce workload placement within specific regions or sovereignty constraints. C ### 18.5 Data Enrichment System enriches consumer requests with ancillary implementation details the consumer should not need to know. Consumer declares intent โ€” DCM fills in the details. -### 18.6 Greening the Brownfield -Bring existing unmanaged resources under DCM lifecycle management: -1. **Discovery** โ€” provider interrogates existing resources, creates Discovered State payload -2. **Enrichment** โ€” business data associated with discovered resources (owner, cost center, purpose) -3. **Lifecycle Ownership** โ€” Discovered State promoted to Realized State; DCM assumes lifecycle management +### 18.6 Greening the Brownfield โ€” Unified Ingestion Model +Bring existing unmanaged infrastructure under DCM lifecycle management using the unified ingestion model (see Section 20 and data model document 13-ingestion-model.md). The same three-step pattern applies to both brownfield ingestion and V1 migration: + +``` +1. INGEST โ€” bring the entity into DCM with whatever identity/metadata is available +2. ENRICH โ€” associate business data, ownership, Tenant assignment, relationships +3. PROMOTE โ€” transition from holding state to full DCM lifecycle ownership +``` + +**Brownfield flow:** Service Provider performs discovery โ†’ DCM identifies unmanaged Discovered State records โ†’ Entity stubs created (state: INGESTED, Tenant: `__transitional__`) โ†’ Business data and Tenant assigned โ†’ Promotion authorized โ†’ Discovered State promoted to Realized State โ†’ Drift detection active from this point. + +**V1 Migration flow:** V1 resources inventoried โ†’ Auto-assignment attempted via signals (resource groups, business unit, request history) โ†’ Auto-assignable resources assigned in bulk โ†’ Manually assignable resources surfaced in admin queue โ†’ Orphaned resources assigned to `__transitional__` โ†’ Enrichment and promotion โ†’ Migration complete when `__transitional__` Tenant is empty. + +**Key concepts:** +- `__transitional__` Tenant โ€” system-managed holding area for unassigned ingested entities. Cannot be deleted or used for new provisioning. Governance policy enforces max residency and escalation. +- `ingestion_record` โ€” provenance record on every ingested entity: source, confidence, assignment method, enrichment history, promotion timestamp +- Ingestion confidence: `high` (strong signal) | `medium` (inferred) | `low` (orphaned) +- Entities in `INGESTED` or `ENRICHING` state cannot be parents for allocated resource claims or hard dependencies for new requests --- @@ -1105,7 +1320,215 @@ DCM addresses four dimensions of digital sovereignty: --- -## SECTION 20 โ€” PERSONAS +## SECTION 20 โ€” INGESTION MODEL + +The Ingestion Model is the **unified mechanism for bringing entities that exist outside DCM's lifecycle control into DCM governance**. It covers V1 Migration, Brownfield Discovery, and Manual Import โ€” all follow the same pattern. + +### 20.1 Three-Step Pattern +``` +INGEST โ†’ ENRICH โ†’ PROMOTE โ†’ OPERATIONAL +``` + +### 20.2 Ingestion Lifecycle States + +| State | Tenant | New Requests? | Parent for Allocations? | New Relationships? | +|-------|--------|--------------|------------------------|-------------------| +| `INGESTED` | `__transitional__` or assigned | No | No | Informational only | +| `ENRICHING` | Assigned | No | No | Operational (read-only) | +| `PROMOTED` | Assigned | Yes | Yes | All types | + +### 20.3 The `__transitional__` Tenant +System-managed holding Tenant for unassigned ingested entities: +- Cannot be deleted, renamed, or used for new resource provisioning +- Governance policy enforces `max_residency_days` and escalation action +- Hard tenancy: `operational_only` by default +- `created_via: system` โ€” artifact metadata + +### 20.4 Ingestion Record +Every ingested entity carries an `ingestion_record` in provenance: +- `ingestion_source` โ€” `v1_migration | brownfield_discovery | manual_import` +- `assigned_tenant_uuid` โ€” real Tenant or null if still in `__transitional__` +- `assignment_method` โ€” `auto | manual | transitional` +- `assignment_signal` โ€” human-readable description of what drove auto-assignment +- `ingestion_confidence` โ€” `high | medium | low` +- `enrichment_status` โ€” `pending | partial | complete` +- `enrichment_history` โ€” append-only log of all enrichment actions +- `promoted_at` โ€” when entity reached PROMOTED state + +### 20.5 Auto-Assignment Signal Priority +DCM attempts auto-assignment in this order (configurable): +1. Explicit ownership metadata (high confidence) +2. Resource group membership (high confidence) +3. Request history (high confidence) +4. Network/location context (medium confidence) +5. Naming convention (medium confidence) +6. Provider context (medium confidence) +7. No signal โ†’ `__transitional__` (low confidence) + +### 20.6 V1 Migration (Q55 resolved) +V1 resources have no `tenant_uuid`. V2 requires one (TEN-001). Migration uses the ingestion model: +- Pre-migration analysis pass classifies all V1 resources: `auto_assignable | manually_assignable | orphaned` +- Auto-assignable โ†’ bulk Tenant assignment + ingestion record +- Manually assignable โ†’ admin queue for human review and assignment +- Orphaned โ†’ `__transitional__` Tenant + governance timer +- Migration complete when `__transitional__` Tenant is empty + +### 20.7 Brownfield Ingestion +Unmanaged discovered entities follow the same ingestion model: +- Service Provider discovery creates Discovered State records +- DCM identifies unmanaged Discovered State records (no matching Realized State) +- Entity stubs created (state: INGESTED, source: brownfield_discovery) +- Enriched โ†’ promoted โ†’ Discovered State becomes initial Realized State +- Drift detection active from promotion forward + +### 20.8 DCM System Policies for Ingestion + +| Policy | Rule | +|--------|------| +| `ING-001` | Every ingested entity must be assigned to one Tenant โ€” real or `__transitional__` โ€” before V2 eligibility | +| `ING-002` | Entities in `INGESTED` or `ENRICHING` state may not be parents for allocated resource claims | +| `ING-003` | `__transitional__` Tenant cannot be deleted, renamed, or used for new provisioning | +| `ING-004` | Every ingested entity must carry an `ingestion_record` in provenance | +| `ING-005` | Entities in `__transitional__` beyond `max_residency_days` must trigger configured escalation | +| `ING-006` | Brownfield entities may not be promoted without explicit actor authorization | +| `ING-007` | At brownfield promotion, Discovered State is promoted to Realized State โ€” DCM assumes lifecycle ownership | + +--- + +## SECTION 21 โ€” POLICY ORGANIZATION: GROUPS, PROFILES, AND POLICY PROVIDERS + +### 21.1 Three-Level Policy Organization +``` +Policy Profile โ€” complete use-case configuration (composed of groups) + โ”‚ +Policy Groups โ€” single-concern policy collections (composed of policies) + โ”‚ +Policies โ€” individual Transformation / Validation / GateKeeper rules + โ”‚ optionally sourced from +Policy Providers โ€” external authoritative policy sources (fifth provider type) +``` + +### 21.2 Policy Groups +A **Policy Group** is a cohesive collection of policies addressing a single identifiable concern. + +**Concern types:** `technology | compliance | sovereignty | business | operational | security` + +**Key fields:** `handle` (domain/group/name), `concern_type`, `concern_tags`, `extends` (inherits parent), `policies` (constituent policies), `activation_scope` (resource types, tenant tags, regions), `conflicts_with` (explicit conflict declarations), `source` (local or policy_provider) + +**DCM built-in groups include:** core-minimal, dev-defaults, ephemeral-resources, audit-basic, audit-compliance, data-classification, cost-governance, sla-enforcement, hard-tenancy, explicit-cross-tenant, zero-trust, encryption-baseline, pci-dss, gdpr-eu, nist-800-53, iso-27001, fsi-audit, lifecycle-ttl-enforcement, air-gap, kubevirt, openstack, vmware + +### 21.3 Policy Profiles +A **Policy Profile** is a complete DCM configuration for a specific use case composed of Policy Groups. + +**Six DCM built-in profiles (least to most restrictive):** + +| Profile | Tenancy | Enforcement | Cross-Tenant | Audit | +|---------|---------|-------------|-------------|-------| +| `minimal` | Optional โ€” auto-created | Advisory only | allow_all | None | +| `dev` | Recommended | Warn only | operational_only | Basic 90-day | +| `standard` | Required | Blocking | explicit_only | Compliance-grade | +| `prod` | Required | Blocking + SLA | explicit_only | Compliance-grade | +| `fsi` | Hard tenancy | Blocking | explicit_only | 7-year retention | +| `sovereign` | Hard tenancy | Blocking | deny_all | 10-year retention | + +**Profile inheritance chain:** sovereign extends fsi extends prod extends standard extends dev extends minimal + +**Profile activation levels (more specific wins):** +```yaml +installation_config: default_profile: minimal +platform_config: active_profile: prod; minimum_tenant_profile: dev +tenant_config: active_profile: fsi # must be >= minimum_tenant_profile +``` + +**Profile shadow validation:** proposed profiles run in shadow mode before activation โ€” same as proposed policies. + +### 21.4 Policy Provider (Fifth Provider Type) +A **Policy Provider** is a fifth DCM provider type โ€” an external authoritative source supplying policies into DCM or evaluating/enriching data via external logic. + +**Four delivery modes:** + +| Mode | Name | Logic Lives In | +|------|------|---------------| +| 1 | DCM Native Push/Pull | DCM Policy Engine | +| 2 | OPA/Rego Bundle | DCM Policy Engine (OPA) | +| 3 | External Schema (naturalization) | DCM Policy Engine (post-translation) | +| 4 | Black Box Query-Enrichment | External provider โ€” opaque to DCM | + +**Modes 1-3** deliver policy rules. **Mode 4** is a query-response interface โ€” DCM sends data, external system evaluates and/or enriches, returns structured result. + +**Mode 4 can:** +- **Evaluate** โ€” return pass/fail, score, or recommendation +- **Enrich** โ€” inject additional fields into the payload (risk scores, compliance citations, cost predictions, org context, case references) +- **Both** โ€” multi_factor result combining decision + enrichment + +**Mode 4 governance requirements:** +- Data sovereignty check before ANY query is sent (BBQ-001, BBQ-003) +- Data minimization โ€” only declared fields sent (BBQ-002) +- Full audit record per query-response cycle (BBQ-004) including `audit_token` for cross-system correlation +- Default failure behavior is `gatekeep` โ€” unknown is not safe (BBQ-005) +- Injected enrichment fields carry standard field-level provenance with `source_type: black_box_provider` + `audit_token` (BBQ-007) +- Override control applies to injected fields โ€” GateKeeper can refuse enrichment (BBQ-008) +- Mode 4 enrichment providers require minimum `transformation` trust level (BBQ-009) + +**Trust levels (all modes):** +- `trusted` โ†’ GateKeeper authority (dual approval elevation required) +- `verified` โ†’ Transformation/Validation only; Mode 4 enrichment minimum +- `untrusted` โ†’ advisory only + +**`on_update` (Modes 1-3):** `proposed` (shadow validation) | `active` (immediate โ€” trusted only) +**On provider failure:** policies deprecated with configurable sunset; Mode 4 โ†’ `on_unavailable` behavior fires + +### 21.5 Lifecycle Time Constraints +First-class field on any resource entity. Follow standard data model precedence and override control. + +**Two types:** +- `ttl` โ€” ISO 8601 duration relative to reference_point (created_at | realization_timestamp | last_modified) +- `expires_at` โ€” absolute ISO 8601 timestamp + +When both declared, earliest wins (LTC-004). GateKeeper can lock as `override: immutable`. + +**`on_expiry` actions:** `destroy | suspend | notify | review` + +Expiry enforcement is a DCM control plane function โ€” not a provider concern. Failed expiry action โ†’ `PENDING_EXPIRY_ACTION` state + escalation (LTC-005). + +### 21.6 Cross-Tenancy Authorization โ€” Explicit_Only Default +Default stance is now **`explicit_only`** โ€” informational sharing is NOT open by default. Every cross-tenant relationship of any nature requires an explicit `cross_tenant_authorization` record. + +**Authorization specifies who/what/when/where:** +```yaml +cross_tenant_authorization: + authorized_consumer_tenant_uuid: # WHO + permitted_fields: [field1, field2] # WHAT โ€” empty = all fields + valid_from/valid_until: # WHEN + permitted_in_regions: [eu-west] # WHERE + authorization_level: + # Hierarchy: field_specific > resource_specific > tenant_global (more specific wins) +``` + +All cross-tenant authorization decisions are policy-driven and DCM-enforced (XTA-004). + +### 21.7 Rehydration Tenancy Controls +Tenancy controls, sovereignty directives, and cross-tenant authorizations **always use current policies during rehydration โ€” cannot be pinned**. + +**`policy_version: pinned`** governs resource configuration policies only. Tenancy/sovereignty always current. + +When rehydration conflicts with current tenancy controls โ†’ entity enters **PENDING_REVIEW** state: +- Allocation not automatically released +- Notifications: entity owner, both Tenant admins, platform admin +- Resolution: re_authorize | release | escalate +- A policy may declare automatic resolution behavior (RHY-004) + +### 21.8 System Policy Summary + +**LTC:** LTC-001 through LTC-005 โ€” lifecycle time constraint enforcement +**XTA:** XTA-001 through XTA-005 โ€” cross-tenancy authorization model +**RHY:** RHY-001 through RHY-004 โ€” rehydration tenancy controls +**DEP:** DEP-001 through DEP-003 โ€” cross-tenant dependency rules +**BBQ:** BBQ-001 through BBQ-009 โ€” Mode 4 black box query-enrichment governance + +--- + +## SECTION 22 โ€” PERSONAS | Persona | Primary Concern | |---------|----------------| @@ -1122,7 +1545,7 @@ DCM addresses four dimensions of digital sovereignty: --- -## SECTION 21 โ€” TERMINOLOGY GLOSSARY +## SECTION 23 โ€” TERMINOLOGY GLOSSARY | Term | Definition | |------|-----------| @@ -1175,6 +1598,39 @@ DCM addresses four dimensions of digital sovereignty: | **Tenant Advocate** | DCM's role in protecting Tenant interests in all provider interactions | | **DCM System Policy** | Non-overridable policy built into DCM โ€” cannot be disabled or overridden by organizational policy | | **Webhook** | Push-based outbound notification from DCM to an external system triggered by a DCM event | +| **Mode 4 Policy Provider** | Black box query-enrichment policy provider โ€” DCM sends query, external system evaluates and/or enriches, returns structured result; logic is opaque to DCM | +| **Black Box Query-Enrichment** | Mode 4 operation where an external system simultaneously evaluates request data and injects enrichment fields into the payload | +| **audit_token** | Provider-issued reference in Mode 4 responses enabling cross-system audit correlation between DCM audit trail and provider's internal logs | +| **data_request_spec** | Mode 4 registration declaration of which fields the provider is authorized to receive, with classification ceiling per field | +| **Policy Naturalization** | Translation of external policy schemas (OSCAL, XCCDF, CIS JSON) into DCM policy format โ€” Mode 3 Policy Provider mechanism | +| **Policy Group** | Cohesive versioned collection of policies addressing a single identifiable concern โ€” the unit of policy reuse | +| **Policy Profile** | Complete DCM configuration for a specific use case โ€” composed of Policy Groups | +| **Policy Provider** | Fifth DCM provider type โ€” external authoritative source supplying policies into DCM | +| **Policy Naturalization** | Translation of external policy schemas (OSCAL, XCCDF, CIS JSON) into DCM policy format | +| **concern_type** | Policy Group classification: technology, compliance, sovereignty, business, operational, security | +| **minimal profile** | Least restrictive built-in profile โ€” advisory enforcement, auto-tenant, home lab / evaluation | +| **sovereign profile** | Most restrictive built-in profile โ€” hard tenancy, deny_all cross-tenant, maximum sovereignty | +| **Lifecycle Constraint Enforcer** | DCM control plane component monitoring realized entities against time constraints | +| **cross_tenant_authorization** | Explicit authorization record for cross-tenant relationship โ€” specifies who, what, when, where | +| **explicit_only** | Default cross-tenant hard tenancy setting โ€” ALL cross-tenant requires explicit authorization | +| **PENDING_REVIEW** | Entity state during paused rehydration tenancy conflict โ€” awaiting resolution | +| **Policy Gap Record** | Audit record for fields absent from reserve query with no applicable policy โ€” records implicit_approval | +| **Shared Resource** | An entity within a single Tenant with active relationships from multiple parent entities; governed by sharing_model and reference counting | +| **sharing_model** | Entity-level declaration of shareability, active_relationship_count, and on_last_relationship_released behavior | +| **active_relationship_count** | DCM-maintained count of active constituent/operational relationships on a shared resource | +| **save_overrides_destroy** | The lifecycle action hierarchy rule: retain > notify > suspend > detach > cascade > destroy; most conservative action always wins | +| **lifecycle_conflict_record** | Audit record created when multiple lifecycle action recommendations differ; carries severity, resolved action, and resolution rule | +| **deferred_destruction_record** | Audit record created when a destructive lifecycle action is deferred because active_relationship_count is above minimum | +| **Ingestion Model** | Unified DCM mechanism for bringing entities outside lifecycle control into DCM governance โ€” covers V1 migration, brownfield discovery, and manual import | +| **Ingestion Record** | Provenance record on every ingested entity โ€” source, confidence, assignment method, enrichment history, promotion timestamp | +| **`__transitional__` Tenant** | System-managed holding Tenant for unassigned ingested entities โ€” cannot be deleted, renamed, or used for new provisioning | +| **Ingestion Confidence** | `high | medium | low` โ€” quality signal for auto-assignment; reflects how reliable the Tenant assignment is | +| **Brownfield** | Existing infrastructure not yet under DCM lifecycle management โ€” brought in via brownfield ingestion | +| **Greening the Brownfield** | The progressive process of bringing unmanaged infrastructure under DCM lifecycle control via the ingestion model | +| **V1 Migration** | Migration of pre-Tenant DCM V1 entities to V2 using the ingestion model | +| **INGESTED state** | First ingestion lifecycle state โ€” entity in DCM, minimal metadata, Tenant may be __transitional__ | +| **ENRICHING state** | Second ingestion lifecycle state โ€” Tenant assigned, metadata and relationships being completed | +| **PROMOTED state** | Final ingestion lifecycle state โ€” all requirements met, DCM assumes full lifecycle ownership | | **DCM Event Type** | A versioned, typed event that DCM can emit โ€” follows universal versioning model | | **Event Type Registry** | DCM-maintained registry of standard event types โ€” extensible like the Resource Type Registry | | **Webhook Registration** | Declaration by a consumer, provider, or external system of which DCM events they want to receive and where | @@ -1244,7 +1700,7 @@ DCM addresses four dimensions of digital sovereignty: --- -## SECTION 22 โ€” OPEN QUESTIONS +## SECTION 24 โ€” OPEN QUESTIONS These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -1281,7 +1737,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl | 29 | How does SUSPENDED state interact with cost analysis โ€” is a suspended Entity still billable? | Entities | | 30 | How are dependency graphs versioned relative to catalog item versions? | Dependencies | | 31 | Should the dependency graph be stored as a separate entity or embedded in the request payload? | Dependencies | -| 32 | How are cross-tenant dependencies handled? | Dependencies | +| 32 | How are cross-tenant dependencies handled? | Dependencies | โœ… Resolved โ€” governed by REL-010/011/012; DEP-001/002/003 for dependency graph specifics; explicit_only default; cross_tenant_authorization required | | 33 | Should there be a maximum dependency graph depth? | Dependencies | | 34 | How does the dependency graph interact with the Meta Provider model? | Dependencies | | 35 | Should DCM maintain a registry of well-known custom group types? | Grouping | @@ -1303,14 +1759,14 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl | 51 | When immutable is set by a Global policy, can a higher-priority Global policy still override it? | Override Control | โœ… Resolved โ€” execution order makes default immutable effectively absolute; immutable_ceiling: absolute provides explicit forward-looking protection | | 52 | Should constraint_schema on a constrained field be visible to consumers in the Service Catalog UI? | Override Control | | 53 | Enhancement gaps: storage/networking bundling vs. dependency model โ€” V1 simplification or new concept needed? | Enhancements | -| 54 | Enhancement gaps: selected_provider as policy output vs. placement component concern | Enhancements | -| 55 | Enhancement gaps: migration path from V1 (no Tenant) to Tenant-mandatory | Enhancements | +| 54 | Enhancement gaps: selected_provider as policy output vs. placement component concern | Enhancements | โœ… Resolved โ€” Placement Engine is a distinct named component; nine-step assembly; reserve query; placement loop with policy phases; policy_gap_record for implicit approval; post-placement policy pass | +| 55 | Enhancement gaps: migration path from V1 (no Tenant) to Tenant-mandatory | Enhancements | โœ… Resolved โ€” unified ingestion model; __transitional__ Tenant; three-step ingest/enrich/promote; ING-001 through ING-007; also covers brownfield ingestion | | 56 | Enhancement gaps: should editable field concept from Catalog Item Schema be incorporated into Resource Type Spec? | Enhancements | -| 57 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Entity Relationships | +| 57 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Entity Relationships | โœ… Resolved โ€” standard Policy Engine authority hierarchy; lifecycle policy fields are just fields; no special case; REL-008 and REL-009 | | 58 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Entity Relationships | -| 59 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Entity Relationships | +| 59 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Entity Relationships | โœ… Resolved โ€” nature governs; constituent never; operational with dual auth; informational unless deny_all; REL-010/011/012; allocated resource model | | 60 | Should there be a maximum relationship graph depth? | Entity Relationships | -| 61 | How are shared entities represented โ€” an entity required by multiple parents? | Entity Relationships | +| 61 | How are shared entities represented โ€” an entity required by multiple parents? | Entity Relationships | โœ… Resolved โ€” sharing_model with active_relationship_count; save_overrides_destroy hierarchy; lifecycle_conflict_record; REL-015 through REL-019 | | 62 | How are conflicting Information Provider push events handled โ€” two providers claim authority for the same record? | Information Providers | | 63 | Should Information Providers support write-back โ€” DCM updating external records via the provider? | Information Providers | | 64 | How is the extended schema versioned when a provider adds or removes extended fields? | Information Providers | @@ -1341,7 +1797,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 23 โ€” DOCUMENTATION STRUCTURE +## SECTION 25 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -1389,7 +1845,7 @@ content/ --- -## SECTION 24 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 26 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: @@ -1433,6 +1889,30 @@ When working on this project, follow these instructions: 38. **Conflicts are resolved at ingestion, not assembly** โ€” all active layers in DCM are pre-validated conflict-free; the assembly process never encounters an ambiguous merge; if a conflict is found at ingestion, the PR is blocked until resolved 39. **Priority schema is advisory for categories, mandatory for ordering** โ€” the reference taxonomy (900=Compliance, 800=Security, etc.) is advisory and organizations may adapt it; however, the numeric comparison rule is always enforced and always deterministic 40. **Proposed status enables shadow validation** โ€” policy artifacts in proposed status execute in shadow mode against real traffic; output is captured in proposed_evaluation_record but never applied; this is the required validation step before activation +63. **Mode 4 Policy Providers are query-response interfaces** โ€” logic lives externally; DCM sends minimized data, receives decision and/or enrichment; data sovereignty check always runs before any query is dispatched; default failure behavior is gatekeep +64. **Mode 4 enrichment fields carry full provenance** โ€” source_type: black_box_provider, source_uuid, and audit_token; override control applies; a GateKeeper can refuse enrichment on sensitive fields; enrichment providers require transformation trust level minimum +57. **Policy Profiles are the primary configuration mechanism** โ€” most deployments activate a built-in profile and add organization-specific groups; do not configure individual policies from scratch when a profile covers the use case +58. **Policy Groups are the unit of reuse** โ€” when designing policies for a concern, package them as a group; groups can be shared across profiles and inherited by other groups +59. **Policy Providers are the fifth provider type** โ€” they follow the same base contract; trust level determines max policy authority; untrusted providers are advisory only; trusted requires dual approval elevation +60. **Cross-tenant default is explicit_only** โ€” informational sharing is NOT open by default; every cross-tenant relationship of any nature requires a cross_tenant_authorization record; this supersedes the earlier operational_only default +61. **Lifecycle time constraints are first-class fields** โ€” they follow standard precedence and override control; GateKeeper can lock them immutable; expiry enforcement is a DCM control plane function not a provider concern +62. **Rehydration cannot bypass tenancy or sovereignty** โ€” policy_version: pinned only governs resource configuration policies; tenancy and sovereignty always use current policies; conflicts produce PENDING_REVIEW state +53. **Shared resources use reference counting** โ€” DCM maintains active_relationship_count; destructive actions are deferred until the count reaches minimum_relationship_count per REL-015; informational relationships never count (REL-016) +54. **Save overrides destroy โ€” always** โ€” the lifecycle action hierarchy (retain > notify > suspend > detach > cascade > destroy) resolves all multi-parent lifecycle conflicts deterministically; retain always wins per REL-018; this is non-negotiable +55. **Lifecycle conflicts are recorded, not silently resolved** โ€” lifecycle_conflict_record created whenever multiple different action recommendations exist; warning/critical severity triggers notifications; info severity is logged only +56. **shareability.allowed: false blocks multiple relationships at type level** โ€” non-shareable resource types (e.g., boot disks) reject second constituent/operational relationships at request time per REL-017; check Resource Type Specification before designing multi-parent relationships +49. **Assembly is nine steps not seven** โ€” steps 1-4 (layers), step 5 (pre-placement policies), step 6 (Placement Engine loop), step 7 (post-placement policies), step 8 (Requested State storage), step 9 (dispatch); always use the correct step number when discussing assembly +50. **Reserve query is atomic** โ€” it simultaneously verifies constraints, returns metadata, and places a resource hold; it is the primary placement query inside the loop; non-hold queries (capacity, metadata, constraint_verification) are available outside the loop for informational purposes +51. **Missing metadata is a policy concern only** โ€” DCM has no built-in opinion about metadata sufficiency; if no policy declares required_context for an absent field, the result is implicit_approval; implicit approvals are recorded explicitly in policy_gap_records +52. **Placement Engine is a named component** โ€” it is a peer to the Policy Engine, not subordinate to it; it owns the placement loop, candidate scoring, reserve query dispatch, and hold management +45. **Ingestion model is the unified mechanism** โ€” V1 migration and brownfield ingestion are the same three-step pattern: ingest โ†’ enrich โ†’ promote; use the same ingestion_record structure, same __transitional__ Tenant, same governance policies regardless of source +46. **`__transitional__` Tenant is a system artifact** โ€” never design around it for normal operations; it exists only as a migration/ingestion holding area; entities there are governance liabilities to be resolved +47. **Ingested entities have capability restrictions** โ€” INGESTED and ENRICHING state entities cannot be parents for allocated resource claims or hard dependencies for new requests; always check ingestion state before designing dependencies +48. **Promotion is the lifecycle gate** โ€” an entity is not a full DCM citizen until it reaches PROMOTED state; before that it is in a holding state with restricted capabilities +41. **Lifecycle policy fields on relationships are just fields** โ€” they carry the same override metadata and resolve under the same Policy Engine authority hierarchy as any other DCM field; no special conflict resolution mechanism +42. **Relationship type ร— nature matrix is explicit and enforced** โ€” invalid combinations are rejected at request time per REL-013; the matrix is the authoritative source for valid relationship combinations +43. **Cross-tenant relationships are governed by nature** โ€” constituent never crosses tenant boundaries; operational requires dual authorization; informational is permitted unless deny_all; hard_tenancy declaration on the Tenant entity controls the boundary +44. **Allocated resources are first-class entities** โ€” a consuming Tenant gets its own UUID, lifecycle, and governance; the relationship is depends_on + operational + cross_tenant; the parent pre-defines available allocations; DCM tracks active allocations on the parent with notification endpoints โ€” they carry the same override metadata and resolve under the same Policy Engine authority hierarchy as any other DCM field; there is no special conflict resolution mechanism for lifecycle policies; REL-008 and REL-009 are the only relationship-specific system policies that add constraints beyond the standard model --- diff --git a/content/docs/architecture/data-model/_index.md b/content/docs/architecture/data-model/_index.md index 739ff75..5fd0171 100644 --- a/content/docs/architecture/data-model/_index.md +++ b/content/docs/architecture/data-model/_index.md @@ -25,4 +25,6 @@ realization, operation, and decommission. {{< card link="information-providers" title="Information Providers" icon="globe" subtitle="External data providers, information types, lookup model, and verification." >}} {{< card link="storage-providers" title="Storage Providers" icon="archive" subtitle="Storage provider contracts for GitOps stores, event streams, audit, and observability." >}} {{< card link="audit-provenance-observability" title="Audit, Provenance, and Observability" icon="eye" subtitle="The three distinct concerns of audit, data lineage, and operational observability." >}} + {{< card link="ingestion-model" title="Ingestion Model" icon="inbox-in" subtitle="Unified V1 migration and brownfield ingestion: ingest, enrich, promote. Transitional Tenant and ING system policies." >}} + {{< card link="policy-profiles" title="Policy Organization" icon="collection" subtitle="Policy Groups, Profiles, and Providers. Built-in profiles from minimal to sovereign. Lifecycle constraints. Cross-tenancy authorization." >}} {{< /cards >}} diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md index db25dd1..3e64ddd 100644 --- a/content/docs/architecture/data-model/entity-relationships.md +++ b/content/docs/architecture/data-model/entity-relationships.md @@ -12,7 +12,7 @@ weight: 8 **Document Status:** ๐Ÿ”„ In Progress -**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Resource Type Hierarchy](../resource-type-hierarchy/) | [Resource/Service Entities](../resource-service-entities/) | [Service Dependencies](../service-dependencies/) | [Resource Grouping](../resource-grouping/) | [Information Providers](../information-providers/) +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) | [Information Providers](10-information-providers.md) --- @@ -187,6 +187,206 @@ Nature describes the **structural character** of a relationship โ€” what it mean --- +## 6a. Relationship Type ร— Nature Matrix + +The two dimensions of every relationship โ€” type and nature โ€” form a matrix of valid combinations. This matrix makes explicit what each combination means semantically and what behavioral rules apply. Not all 18 combinations are valid. + +| | `constituent` | `operational` | `informational` | +|---|---|---|---| +| **`requires`** | โœ… **Core constituent** โ€” entity cannot function without this component; component is part of its definition. Lifecycle policy required. | โœ… **Hard operational dependency** โ€” entity cannot function without this but it is not a component. Lifecycle policy required. | โš ๏ธ **Invalid** โ€” if an entity truly requires something, it has an operational or constituent dependency, not merely informational context. | +| **`depends_on`** | โœ… **Soft constituent** โ€” entity degrades without this component but is not fully broken. Lifecycle policy required. | โœ… **Primary cell for allocated resources** โ€” soft operational dependency. Cross-tenant allocations live here. Lifecycle policy required. | โœ… **Awareness dependency** โ€” entity is aware of and tracks this entity but has no hard operational dependency. No lifecycle policy. | +| **`contains`** | โœ… **Ownership container** โ€” this entity logically owns and contains the related entity as a component. Lifecycle policy required. | โš ๏ธ **Rare** โ€” containing something operationally is unusual; most containment is constituent. Use with explicit justification. | โŒ **Invalid** โ€” containing something purely informational has no semantic meaning. | +| **`references`** | โŒ **Invalid** โ€” a reference implies no ownership or dependency; constituent implies the opposite. | โŒ **Invalid** โ€” if there is an operational dependency, use `depends_on`. A reference that creates operational coupling is mismodeled. | โœ… **Pure informational reference** โ€” primary cell for Business Unit, Cost Center, Product Owner relationships. No lifecycle policy. | +| **`peer`** | โŒ **Invalid** โ€” peers cannot be constituent components of each other. | โœ… **Operational peers** โ€” equal entities with mutual operational interdependency. Lifecycle policy on each side. | โœ… **Informational peers** โ€” equal entities that are aware of each other. No lifecycle policy. | +| **`manages`** | โœ… **Component management** โ€” this entity has lifecycle authority over a component it manages. Lifecycle policy required. | โœ… **Operational management** โ€” this entity manages the operations of another entity without owning it. Lifecycle policy required. | โœ… **Audit/reporting management** โ€” management relationship for visibility only. No lifecycle policy. | + +**Key behavioral rules derived from the matrix:** + +- Any `constituent` or `operational` relationship **must** declare a lifecycle policy (REL-004, REL-008) +- `constituent` + `requires` is the strongest possible relationship โ€” both the entity and its component are mutually dependent; cross-tenant is prohibited (REL-010) +- `operational` + `depends_on` is the **allocated resource cell** โ€” this is where cross-tenant allocations are modeled +- `informational` + `references` is the **business context cell** โ€” Business Unit, Cost Center, Person relationships live here +- `โŒ Invalid` combinations must be rejected by the Policy Engine at request time + +--- + +## 6b. Cross-Tenant Relationships + +### 6b.1 The Governing Principle + +The relationship **nature** determines whether a cross-tenant relationship is permitted: + +| Nature | Cross-Tenant Permitted? | Governing Rule | +|--------|------------------------|---------------| +| `constituent` | โŒ Never | REL-010 โ€” DCM System Policy | +| `operational` | โœ… With explicit dual authorization | REL-011 โ€” both Tenants must authorize | +| `informational` | โœ… Unless denied by hard tenancy | REL-012 โ€” blocked only by `deny_all` | + +### 6b.2 Hard Tenancy Declaration + +Tenants declare their cross-tenant relationship policy. This is enforced by the GateKeeper Policy Engine at request time: + +```yaml +tenant: + uuid: + hard_tenancy: + cross_tenant_relationships: explicit_only + # deny_all: no relationships of any nature may cross this boundary + # explicit_only: ALL cross-tenant must be explicitly authorized (DEFAULT) + # operational_permitted: operational cross-tenant permitted; informational requires explicit auth + # allow_all: all cross-tenant permitted โ€” requires justification +``` + +**Default is `explicit_only` โ€” informational sharing is not open by default.** Every cross-tenant relationship of any nature requires an explicit `cross_tenant_authorization` record. This closes the model โ€” cross-tenant access must be deliberately granted, not passively permitted. + +### 6b.3 DCM System Policies for Cross-Tenant Relationships + +| Policy | Rule | +|--------|------| +| `REL-010` | Constituent relationships may not cross Tenant boundaries | +| `REL-011` | Cross-tenant operational relationships require explicit authorization from both the owning Tenant and the consuming Tenant | +| `REL-012` | A Tenant with `hard_tenancy.cross_tenant_relationships: deny_all` may not participate in any cross-tenant relationship in any direction | + +--- + +## 6c. Allocated Resources โ€” Cross-Tenant Operational Model + +### 6c.1 Concept + +An **Allocated Resource** is a pre-defined, discrete slice of a parent resource โ€” provisioned by the owning Tenant and made available for consuming Tenants to claim. The allocated resource becomes a **first-class entity** in the consuming Tenant's scope with its own UUID, its own lifecycle, and its own governance โ€” while maintaining a formal `depends_on` + `operational` relationship to the parent resource across the Tenant boundary. + +This models real infrastructure practice: the network team pre-carves VLANs, the storage team pre-partitions pools, the platform team pre-defines availability zones. Consumers claim from what is available. + +The relationship type is `depends_on` + `operational` โ€” the allocated entity depends on the parent operationally but is not a constituent component of it. The allocation is the relationship; the entity itself is independently governed. + +### 6c.2 Parent Resource โ€” Available Allocations + +The owning Tenant pre-defines allocations on the parent resource: + +```yaml +parent_resource_entity: + uuid: + tenant_uuid: + + available_allocations: + - allocation_uuid: + allocation_type: Network.VLANRange + allocation_spec: + vlan_range: "100-199" + bandwidth: "10Gbps" + status: + claimable_by: + - tenant_uuid: + - tenant_uuid: + # Empty list = any authorized Tenant may claim + + active_allocations: + - allocation_uuid: + claimed_by_tenant_uuid: + claimed_entity_uuid: + claimed_at: + notification_endpoint: + # Parent uses this to notify Tenant A of lifecycle changes +``` + +### 6c.3 Allocated Entity โ€” In the Consuming Tenant + +When a consuming Tenant claims an available allocation, DCM creates a first-class entity in the consuming Tenant's scope: + +```yaml +allocated_entity: + uuid: + entity_type: allocated_resource + resource_type_uuid: + tenant_uuid: # Belongs to the consuming Tenant + + allocation_spec: + vlan_range: "100-199" + bandwidth: "10Gbps" + # The specific slice allocated to this Tenant + + parent_allocation: + parent_entity_uuid: + parent_tenant_uuid: + allocation_uuid: + + lifecycle_state: OPERATIONAL + + parent_lifecycle_policy: + on_parent_destroy: notify_then_detach + on_parent_suspend: suspend + on_parent_maintenance: notify + on_parent_degrade: notify + on_parent_capacity_change: notify + + relationships: + - relationship_uuid: + related_entity_uuid: + related_entity_type: internal + related_entity_tenant_uuid: + relationship_type: depends_on + nature: operational + cross_tenant: true + allocation_uuid: + authorized_by: + owning_tenant_policy_uuid: + consuming_tenant_policy_uuid: + + artifact_metadata: + +``` + +### 6c.4 Lifecycle Event Propagation + +When the parent resource changes state, DCM iterates all active allocations and propagates according to each allocation's `parent_lifecycle_policy`: + +``` +Parent resource enters MAINTENANCE + โ”‚ + โ–ผ +DCM iterates active_allocations + โ”‚ + For each active allocation: + โ”‚ Read parent_lifecycle_policy.on_parent_maintenance + โ”‚ โ†’ notify: dispatch lifecycle event to consuming Tenant + โ”‚ โ†’ suspend: transition allocated entity to SUSPENDED state + โ”‚ โ†’ detach: terminate relationship, allocated entity becomes independent + โ”‚ + Policy Engine evaluates each propagation: + โ”‚ SLA commitments that gate maintenance? + โ”‚ Override policies in consuming Tenant? + โ–ผ +Events dispatched via notification_endpoint on each active_allocation record +``` + +### 6c.5 Claiming Flow + +``` +Parent Tenant pre-defines available_allocations on parent resource + โ”‚ + โ–ผ +Consuming Tenant A submits claim request + โ”‚ Specifies: parent_entity_uuid, allocation_uuid + โ–ผ +Policy Engine evaluates: + โ”‚ Is allocation_uuid still available? + โ”‚ Is Tenant A in claimable_by list (or list is open)? + โ”‚ Does Tenant A's cross_tenant policy permit this? + โ”‚ Does Infrastructure Tenant's cross_tenant policy permit this? + โ–ผ +DCM creates: + โ”‚ Allocated Entity (owned by Tenant A) with UUID + โ”‚ depends_on / dependency_of relationship (bidirectional, cross_tenant: true) + โ”‚ Updates parent's available_allocation status: available โ†’ claimed + โ”‚ Adds record to parent's active_allocations + โ”‚ Provenance recorded on both entities + โ–ผ +Infrastructure Tenant owner notified of new claim + โ”‚ Via owned_by.notification_endpoint on the parent entity +``` + +--- + ## 7. Lifecycle Policies Lifecycle policies declare what happens to an entity when its related entity changes state. They apply to `constituent` and `operational` relationships only โ€” `informational` relationships have no lifecycle implications. @@ -203,7 +403,78 @@ Lifecycle policies declare what happens to an entity when its related entity cha | `cascade` | Cascade the change from the related entity to this entity | | `ignore` | Take no action โ€” the change to the related entity does not affect this entity | -### 7.2 Lifecycle Policy Authority Hierarchy +### 7.2 Lifecycle Action Hierarchy โ€” Save Overrides Destroy + +When a shared resource has multiple active relationships and a lifecycle event triggers, each relationship may produce a different action recommendation. DCM resolves conflicts using a deterministic hierarchy โ€” **the most conservative action always wins**: + +``` +retain โ† most conservative โ€” entity preserved unconditionally + โ”‚ +notify โ† inform and wait โ€” human decision required + โ”‚ +suspend โ† temporarily inactive โ€” reversible + โ”‚ +detach โ† relationship released โ€” entity becomes independent + โ”‚ +cascade โ† propagate state change from related entity + โ”‚ +destroy โ† least conservative โ€” entity terminated +``` + +**The save_overrides_destroy rule (REL-018):** If any active relationship recommends `retain`, the entity is retained regardless of what any other relationship recommends โ€” including relationships with `override: immutable` lifecycle policies. `retain` is the save. It always beats `destroy`. + +This rule applies automatically and silently when the hierarchy resolves cleanly (e.g., `retain` beats `destroy`). It is recorded in the `lifecycle_conflict_record` with severity `info` for audit purposes but requires no notification. + +### 7.3 Lifecycle Conflict Detection + +**Not all multi-recommendation scenarios are conflicts.** The hierarchy resolves most cases deterministically. A conflict worth surfacing occurs when: + +1. **Adjacent hierarchy levels** โ€” two relationships recommend actions that are one step apart (e.g., `notify` vs `suspend`) โ€” the hierarchy resolves it but the ambiguity is worth surfacing +2. **An immutable lifecycle lock couldn't be honored** โ€” a GateKeeper set `on_related_destroy: destroy` with `immutable_ceiling: absolute` but `retain` from another relationship won per REL-018 +3. **`notify` is the winning action** โ€” inherently means human decision required; the notification should include the full conflict picture + +**Conflict severity:** + +| Scenario | Severity | Action | +|----------|---------|--------| +| `retain` beats `destroy` โ€” non-adjacent levels | `info` | Logged only โ€” working as designed | +| All relationships agree | None | No record needed | +| Adjacent levels (e.g., `notify` vs `suspend`) | `warning` | Notify entity owner and affected policy owners | +| `notify` is the winning action | `warning` | Notify owner โ€” human decision required | +| Immutable lifecycle lock overridden by REL-018 | `critical` | Notify entity owner, policy owner, and platform admin | + +**Lifecycle conflict record:** + +```yaml +lifecycle_conflict_record: + entity_uuid: + event_trigger: + triggering_entity_uuid: + action_recommendations: + - relationship_uuid: + related_entity_uuid: + recommended_action: destroy + source: lifecycle_policy + - relationship_uuid: + related_entity_uuid: + recommended_action: retain + source: gatekeeper_policy + policy_uuid: + - relationship_uuid: + related_entity_uuid: + recommended_action: notify + source: lifecycle_policy + resolved_action: retain + resolution_rule: save_overrides_destroy + conflict_detected: true + conflict_severity: info + notifications_sent: + - recipient_uuid: + message: "Lifecycle conflict resolved: retain overrode destroy and notify." + recorded_at: +``` + +### 7.4 Lifecycle Policy Authority Hierarchy Lifecycle policies follow the same three-tier authority model as override control: @@ -224,6 +495,130 @@ DCM System Policy (non-overridable โ€” sovereignty and compliance mandates) --- +## 7a. Shared Resource Model โ€” Same-Tenant + +### 7a.1 Concept + +A **Shared Resource** is an entity within a single Tenant that has active relationships from multiple parent entities. Rather than being exclusively owned by one parent, it is referenced by N parents โ€” each with its own lifecycle relationship. + +This is the same-tenant counterpart to the cross-tenant Allocated Resource model. Both use reference counting to defer destructive actions. The sharing model applies within a Tenant; the allocation model applies across Tenant boundaries. + +**Examples:** Shared NFS volume mounted by multiple VMs. Shared database cluster used by multiple application services. Shared VLAN used by multiple VMs. Shared TLS certificate used by multiple services. + +### 7a.2 The `sharing_model` Declaration + +The Resource Type Specification declares whether instances of a type can be shared. Individual entities carry the runtime sharing state: + +```yaml +# On the Resource Type Specification +resource_type_spec: + fully_qualified_name: Storage.SharedVolume + shareability: + allowed: true + default_sharing_scope: tenant # tenant | cross_tenant + max_active_relationships: null # null = unlimited; integer = cap (e.g., license seats) + +# On the entity instance +entity: + uuid: + sharing_model: + shareable: true + sharing_scope: tenant + active_relationship_count: 3 # DCM maintains this โ€” do not set manually + minimum_relationship_count: 0 # below this, on_last_relationship_released fires + on_last_relationship_released: + # destroy: entity destroyed when last relationship is released + # retain: entity persists independently โ€” becomes unowned + # notify: notify owner, entity enters PENDING_DECISION +``` + +**`shareability.allowed: false`** on a Resource Type (e.g., `Compute.BootDisk`) means the Policy Engine rejects any attempt to create a second active constituent or operational relationship to an instance. Boot disks, primary network interfaces, and similar exclusively-owned resources are non-shareable by type definition (REL-017). + +### 7a.3 Reference Count Lifecycle + +DCM maintains `active_relationship_count` automatically: + +- **Relationship created** โ†’ `active_relationship_count` incremented +- **Relationship released** (parent decommissioned, relationship detached) โ†’ `active_relationship_count` decremented +- **Informational relationships** โ†’ never counted (REL-016) +- **Count reaches `minimum_relationship_count`** โ†’ `on_last_relationship_released` fires + +When a parent entity is destroyed and has a relationship to a shared resource: + +``` +Parent entity destroyed + โ”‚ + โ–ผ +DCM collects action recommendations from all active relationships on shared resource + โ”‚ Each relationship's lifecycle policy produces one recommendation + โ”‚ Informational relationships excluded + โ”‚ + โ–ผ +Action resolution โ€” save_overrides_destroy hierarchy (REL-018) + โ”‚ Most conservative recommendation wins + โ”‚ Lifecycle conflict record created if multiple recommendations differ + โ”‚ + โ–ผ +Execute winning action + โ”‚ retain โ†’ shared resource unaffected + โ”‚ notify โ†’ PENDING_DECISION state, notifications dispatched + โ”‚ suspend โ†’ shared resource suspended + โ”‚ detach โ†’ parent's relationship released, count decremented + โ”‚ destroy โ†’ only if count reaches minimum_relationship_count (REL-015) + โ”‚ + โ–ผ +Deferred destruction record created (if action was deferred) +``` + +### 7a.4 Deferred Destruction Records + +Every time a destructive action is deferred by the reference count mechanism: + +```yaml +deferred_destruction_record: + entity_uuid: + triggering_request_uuid: + triggering_relationship_uuid: + relationship_count_before: 3 + relationship_count_after: 2 + action_taken: deferred + reason: "active_relationship_count above minimum. Destruction deferred." + remaining_relationships: + - relationship_uuid: + related_entity_uuid: + relationship_type: required_by + - relationship_uuid: + related_entity_uuid: + relationship_type: dependency_of + recorded_at: +``` + +When the last relationship is released: + +```yaml +deferred_destruction_record: + relationship_count_before: 1 + relationship_count_after: 0 + action_taken: "on_last_relationship_released โ†’ destroy" + reason: "Last active relationship released. Executing on_last_relationship_released." + recorded_at: +``` + +### 7a.5 Unified with the Allocated Resource Model + +The same-tenant sharing model and the cross-tenant allocated resource model are the same concept at different scopes: + +| Dimension | Same-Tenant Sharing | Cross-Tenant Allocation | +|-----------|--------------------|-----------------------| +| Scope | Within one Tenant | Across Tenant boundaries | +| Pre-definition | Not required โ€” relationships declared at request time | Parent pre-defines `available_allocations` | +| Reference tracking | `active_relationship_count` on entity | `active_allocations` list on parent | +| Destruction deferral | Deferred until count reaches minimum | Deferred until last allocation released | +| Lifecycle events | `on_last_relationship_released` | `parent_lifecycle_policy` per allocation | +| Governed by | REL-015 through REL-019 | REL-011, REL-014 | + +--- + ## 8. Relationship Declarations โ€” Where They Live Relationship declarations exist at multiple levels, each building on the previous: @@ -434,12 +829,69 @@ The relationship graph exists across all four states: | `REL-001` | Every relationship must have a UUID | | `REL-002` | Every relationship must be recorded on both participating entities | | `REL-003` | Circular relationships are invalid and must be rejected | -| `REL-004` | A constituent relationship must declare a lifecycle policy | +| `REL-004` | A constituent or operational relationship must have a lifecycle policy declared somewhere in the authority chain before provider dispatch | | `REL-005` | External relationships must reference a registered Information Provider | | `REL-006` | Relationship types must be from the standard vocabulary | | `REL-007` | Consumer-declared binding types must be permitted by the Resource Type Specification | +| `REL-008` | A constituent relationship lifecycle policy may not be set to `ignore` for `on_related_destroy` | +| `REL-009` | Lifecycle policy conflicts between policies are resolved by the standard Policy Engine authority hierarchy โ€” no special case | +| `REL-010` | Constituent relationships may not cross Tenant boundaries | +| `REL-011` | Cross-tenant operational relationships require explicit authorization from both the owning Tenant and the consuming Tenant | +| `REL-012` | A Tenant with `hard_tenancy.cross_tenant_relationships: deny_all` may not participate in any cross-tenant relationship in any direction | +| `REL-013` | `โŒ Invalid` relationship type ร— nature combinations (per the matrix in Section 6a) must be rejected by the Policy Engine at request time | +| `REL-014` | An allocated resource claim requires a matching `available` allocation record on the parent entity | +| `REL-015` | A destructive lifecycle action on a shared resource entity (`shareable: true`) is deferred until `active_relationship_count` reaches `minimum_relationship_count` | +| `REL-016` | Informational relationships do not contribute to `active_relationship_count` on shared resource entities | +| `REL-017` | A Resource Type Specification with `shareability.allowed: false` must reject any attempt to create more than one active constituent or operational relationship to an instance of that type | +| `REL-018` | When a lifecycle event produces multiple action recommendations on a shared resource, the most conservative action wins per the hierarchy: `retain > notify > suspend > detach > cascade > destroy` (save_overrides_destroy) | +| `REL-019` | When lifecycle action recommendations conflict, a `lifecycle_conflict_record` is created. Conflicts at `warning` or `critical` severity trigger notifications to the entity owner and affected policy owners | + +### 11.2 Lifecycle Policy Conflict Resolution + +Lifecycle policy fields on relationships are fields. They carry the same `override` metadata, the same provenance obligations, and resolve under the same Policy Engine authority hierarchy as any other field in DCM. There is no special case โ€” minimum variance applies. + +**Authority chain for a relationship lifecycle policy field (lowest to highest):** + +``` +Resource Type Specification default + โ†’ Provider Catalog Item default + โ†’ Consumer declaration at request time + โ†’ Transformation Policy (may set override: constrained) + โ†’ Validation Policy (checks โ€” no modification) + โ†’ GateKeeper Policy (may set override: immutable) + โ†’ DCM System Policies REL-008, REL-009 (non-overridable) +``` + +**Within the Policy Engine**, the priority schema governs conflicts between policies at the same tier. Highest numeric priority value within a tier runs first. The first policy to set `override: immutable` on a lifecycle policy field locks it โ€” all subsequent policies in that execution find it locked and cannot modify it. + +**Conflict detection at ingestion** applies to lifecycle policy declarations in policies exactly as it does to layer fields: +- Two policies both declare `on_related_destroy` for the same relationship type without priority differentiation โ†’ CONFLICT ERROR at ingestion โ€” both owners notified +- One has higher priority value โ†’ Higher wins, documented in provenance +- Equal priority โ†’ CONFLICT ERROR + +**`immutable_ceiling: absolute` applies here.** A sovereign compliance mandate that storage must always be retained when a VM is destroyed โ€” `on_related_destroy: retain` with `immutable_ceiling: absolute` โ€” cannot be overridden by any future policy regardless of priority. + +**Example โ€” compliant lifecycle policy field with override control:** + +```yaml +lifecycle_policy: + on_related_destroy: + value: retain + metadata: + override: immutable + locked_by_policy_uuid: + locked_at_level: global + basis_for_value: "Compliance mandate โ€” storage must outlive VM for audit retention" + immutable_ceiling: absolute + provenance: + origin: + source_type: policy + source_uuid: + timestamp: + modifications: [] +``` -### 11.2 Relationship Versioning and Deprecation +### 11.3 Relationship Versioning and Deprecation Relationships follow the universal versioning and deprecation model. A relationship version changes when its lifecycle policy, nature, or role changes. Terminated relationships are retained in provenance permanently. @@ -449,11 +901,11 @@ Relationships follow the universal versioning and deprecation model. A relations | # | Question | Impact | Status | |---|----------|--------|--------| -| 1 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Policy model | โ“ Unresolved | +| 1 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Policy model | โœ… Resolved โ€” standard Policy Engine authority hierarchy; REL-008 and REL-009 | | 2 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Operational complexity | โ“ Unresolved | -| 3 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Multi-tenancy | โ“ Unresolved | +| 3 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Multi-tenancy | โœ… Resolved โ€” nature governs; constituent never; operational with dual auth; informational unless deny_all; REL-010/011/012 | | 4 | Should there be a maximum relationship graph depth to prevent runaway complexity? | Operational governance | โ“ Unresolved | -| 5 | How are shared entities represented in the relationship graph โ€” an entity required by multiple parents? | Graph model | โ“ Unresolved | +| 5 | How are shared entities represented in the relationship graph โ€” an entity required by multiple parents? | Graph model | โœ… Resolved โ€” sharing_model declaration; active_relationship_count; save_overrides_destroy hierarchy (REL-018); lifecycle_conflict_record; REL-015 through REL-019 | --- diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md index c993e3f..45266d8 100644 --- a/content/docs/architecture/data-model/four-states.md +++ b/content/docs/architecture/data-model/four-states.md @@ -12,7 +12,7 @@ weight: 2 **Document Status:** ๐Ÿ”„ In Progress -**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Entity Relationships](../entity-relationships/) | [Storage Providers](../storage-providers/) | [Audit, Provenance, and Observability](../audit-provenance-observability/) +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) --- @@ -120,7 +120,7 @@ Given an entity UUID, DCM can reconstruct the complete history of that entity ac DCM describes store **contracts**, not implementations. Each store is a Storage Provider โ€” a formal DCM provider type with registration, health check, and trust obligations. Implementors choose the technology that satisfies the contract. -See [Storage Providers](../storage-providers/) for the complete contract specifications. +See [Storage Providers](11-storage-providers.md) for the complete contract specifications. ### 4.1 GitOps Stores (Intent and Requested) @@ -300,7 +300,60 @@ Two independent axes โ€” placement and policy version โ€” produce four distinct Historical modes require elevated authorization. All modes run governance โ€” the difference is whether governance uses current or pinned policies. -### 5.5 Partial Resolution of Q54 โ€” Provider Selection +### 5.5 Rehydration Tenancy and Sovereignty Controls + +**Tenancy controls, sovereignty directives, and cross-tenant authorizations are always evaluated against current policies during rehydration โ€” they cannot be pinned to historical versions.** + +The `policy_version: pinned` setting governs resource configuration policies only. It does not apply to: +- Tenancy boundary enforcement +- Sovereignty constraints +- Cross-tenant authorization requirements + +```yaml +rehydration: + policy_version: pinned # governs resource configuration policies + # The following ALWAYS use current policies โ€” cannot be pinned: + tenancy_controls: always_current + sovereignty_controls: always_current + cross_tenant_authorizations: always_current +``` + +**When rehydration conflicts with current tenancy controls:** + +If the current policy environment produces a tenancy or sovereignty constraint that conflicts with a cross-tenant allocation valid at original request time โ€” for example, the consuming Tenant's authorization was revoked since the original request โ€” the rehydration is **paused**, not failed or silently bypassed: + +``` +Rehydration detects cross-tenant authorization conflict + โ”‚ + โ–ผ +Entity enters PENDING_REVIEW state + โ”‚ Allocation is not automatically released + โ”‚ Rehydration_tenancy_conflict_record created + โ–ผ +Notifications dispatched: + โ”‚ entity owner, owning Tenant admin, + โ”‚ consuming Tenant admin, platform admin + โ–ผ +Resolution options: + re_authorize โ†’ issue new cross_tenant_authorization for this allocation + release โ†’ release the allocation, entity decommissioned + escalate โ†’ refer to platform admin for manual decision + โ”‚ + โ””โ”€โ”€ A policy may declare automatic resolution: + "on rehydration conflict โ†’ re_authorize if consuming Tenant + still meets sovereignty requirements" +``` + +**System policies for rehydration tenancy:** + +| Policy | Rule | +|--------|------| +| `RHY-001` | Tenancy, sovereignty, and cross-tenant authorizations always use current policies during rehydration โ€” cannot be pinned | +| `RHY-002` | Rehydration that conflicts with current tenancy/sovereignty pauses and enters PENDING_REVIEW | +| `RHY-003` | A paused rehydration allocation is not automatically released โ€” requires explicit resolution | +| `RHY-004` | A policy may declare automatic resolution behavior for rehydration tenancy conflicts | + +### 5.6 Partial Resolution of Q54 โ€” Provider Selection The placement flag model clarifies the Q54 question (selected_provider as policy output vs placement component). The emerging answer: diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md index cbd4eeb..3d72794 100644 --- a/content/docs/architecture/data-model/layering-and-versioning.md +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -1059,17 +1059,177 @@ Layers are merged in precedence order (lowest to highest). For each field: ### Step 4 โ€” Request Layer Application The consumer's Request Layer is applied last in the data layer merge. Consumer-declared values override all data layer values. Each override is recorded in provenance. -### Step 5 โ€” Policy Processing -Policies are applied to the merged payload in order. Each policy step may read and set `override_control` metadata on fields โ€” see Section 5a for full detail. -1. **Transformation Policies** โ€” enrich and modify the payload. May set `override_preference: constrained` on fields, declaring that future overrides must satisfy a constraint schema. Each transformation records the policy UUID, operation type, reason, and any override control declarations in the affected fields' provenance. -2. **Validation Policies** โ€” check the payload against rules and verify that existing `override_preference` declarations have not been violated. Failures reject the request with a detailed reason. No field modification occurs. -3. **GateKeeper Policies** โ€” apply hard overrides and blocks. May set `override_preference: immutable` on fields, permanently locking them against further modification. GateKeeper overrides record the policy UUID, the overridden value, the new value, the lock type, and the reason in provenance. +### Step 5 โ€” Pre-Placement Policy Processing +Policies with `placement_phase: pre` (or `both`) are evaluated against the merged payload before any provider is known. Three policy types execute in order: -### Step 6 โ€” Requested State Storage -The fully assembled, policy-processed payload is stored as the **Requested State** in the Request Store. This is the complete, provider-ready payload with full provenance chain intact. +1. **Transformation Policies** โ€” enrich and modify the payload. May set `override: constrained` on fields. Each transformation records the policy UUID, operation type, reason, and any override control declarations in provenance. +2. **Validation Policies** โ€” check the payload against rules. Pass/fail only โ€” no field modification. Failures reject the request. +3. **GateKeeper Policies** โ€” apply hard overrides and blocks. May set `override: immutable`. All overrides recorded in provenance. -### Step 7 โ€” Provider Dispatch -The Requested State payload is dispatched to the appropriate Service Provider via the API Gateway. +Pre-placement policies produce **placement constraints** โ€” declarative requirements a provider must satisfy (sovereignty zone, hardware class, conformance level, etc.). These constraints are carried forward as inputs to the Placement Engine. + +### Step 6 โ€” Placement Engine โ€” Placement Loop + +The Placement Engine takes the policy-processed payload and placement constraints, builds a candidate provider list (filtered by constraints, ordered by scoring criteria), and iterates through candidates until placement is confirmed or all candidates are exhausted. + +**Placement loop governance** (configurable by policy): +```yaml +placement_loop_config: + max_iterations: 5 # maximum candidates to attempt + max_duration_seconds: 30 # timeout for entire loop + on_exhaustion: + hold_ttl_seconds: 300 # how long provider holds resources +``` + +**Per-candidate iteration:** + +``` +โ”€โ”€ RESERVE QUERY (single atomic call to provider) โ”€โ”€ + Request: constraints + resource spec + hold TTL + metadata_requested + Response status: + confirmed: resources held, constraints satisfied, metadata returned + partial: hold confirmed, some metadata unavailable + insufficient: provider lacks capacity โ€” skip to next candidate + refused: provider cannot satisfy constraints โ€” skip to next candidate + +โ”€โ”€ POLICY PHASE (placement_phase: loop) โ”€โ”€ + Policies evaluate: payload + constraints + reserve query response + For each field declared in policy required_context: + Field present: evaluate normally + Field absent, required_context declared: + if_absent: gatekeep โ†’ release hold, abort loop, REJECT REQUEST + if_absent: warn โ†’ record warning, continue + if_absent: skip โ†’ record as skipped, continue + Field absent, no policy declares required_context: + โ†’ record policy_gap_record (implicit_approval), continue + Policy outcomes: + gatekeep โ†’ release hold, abort loop, REJECT REQUEST + reject_candidate โ†’ release hold, skip to next candidate + pass / warn โ†’ PLACEMENT CONFIRMED โ€” exit loop +``` + +**Reserve query structure:** +```yaml +reserve_query_request: + request_uuid: + hold_uuid: + resource_type: + placement_constraints: + resource_spec: + cpu: 16 + ram_gb: 64 + storage_gb: 500 + hold_ttl_seconds: 300 + metadata_requested: + - capacity_available + - topology + - sovereignty_certifications + - patch_level + - maintenance_windows + +reserve_query_response: + hold_uuid: + provider_hold_reference: + hold_status: + hold_confirmed_spec: + cpu: 16 + ram_gb: 64 + storage_gb: 500 + zone: eu-west-1a + rack: rack-07 + metadata: + topology: + zone: eu-west-1a + rack: rack-07 + network_segment: vlan-142 + available_ips: ["10.20.4.0/24"] + sovereignty_certifications: + - cert: ISO-27001 + valid_until: "2027-06-30" + missing_metadata: + - field: patch_level + reason: "Provider does not track patch metadata at this conformance level" +``` + +**Non-hold queries** (available outside the placement loop for capacity checks, provider health, cost estimation, and pre-filtering): + +| Query Type | Hold? | Purpose | +|-----------|-------|---------| +| `reserve_query` | Yes โ€” atomic | Primary placement loop query | +| `capacity_query` | No | Pre-loop filtering, dashboard, cost estimation | +| `metadata_query` | No | Provider health checks, audit, policy pre-evaluation | +| `constraint_verification` | No | Rapid pre-filter before entering the loop | + +**Policy gap records** โ€” when a field is absent and no policy declares `required_context` for it: +```yaml +policy_gap_record: + request_uuid: + field: patch_level + field_value: null + evaluation_result: implicit_approval + reason: > + No active policy declared required_context for this field. + Field was absent in reserve query response. + Request proceeded without policy evaluation of this field. + provider_uuid: + recorded_at: + resolution_expected: realized_payload + # Provider expected to supply this field in the realized payload or discovery +``` + +**Provider metadata completeness โ€” eventual consistency:** +Fields missing from the reserve query response are expected to be completed in: +1. **Realized payload** (primary) โ€” provider returns full metadata when confirming realization +2. **Discovery loop** (fallback) โ€” periodic discovery fills remaining gaps + +The realized entity carries `enrichment_status: pending | partial | complete` reflecting how complete its metadata is. This is the same pattern as the ingestion model. + +### Step 7 โ€” Post-Placement Policy Processing +Policies with `placement_phase: post` (or `both`) execute after the Placement Engine has confirmed a provider selection. These policies have full access to the `placement` block of the payload including the provider selection, hold confirmation, and all returned metadata. + +1. **Transformation Policies** โ€” provider-aware enrichment. Inject zone-specific configuration, provider-specific defaults, topology-derived values that are only knowable after provider selection. +2. **Validation Policies** โ€” post-placement checks. Verify the selected provider meets requirements that couldn't be expressed as pre-placement constraints. +3. **GateKeeper Policies** โ€” post-placement hard overrides. May inject mandatory fields triggered by the specific provider selected (e.g., additional data handling requirements for a provider in a specific jurisdiction). + +**Policy `placement_phase` values:** +```yaml +policy: + placement_phase:
+  # pre:  steps 5 โ€” before provider known (default)
+  # loop: step 6 โ€” inside placement loop, evaluates reserve query response
+  # post: step 7 โ€” after placement confirmed, provider known
+  # both: pre and post (not loop)
+```
+
+**Policy `required_context` for missing metadata:**
+```yaml
+policy:
+  placement_phase: loop
+  required_context:
+    - field: placement.provider_metadata.sovereignty_certifications
+      if_absent: gatekeep
+      if_absent_reason: >
+        Cannot evaluate sovereignty compliance without provider
+        certification data. Blocking request. Provider must register
+        this metadata to participate in sovereignty-scoped requests.
+    - field: placement.provider_metadata.patch_level
+      if_absent: warn
+      if_absent_reason: >
+        Patch level not available. Proceeding with warning.
+        Provider notified to register patch metadata.
+```
+
+### Step 8 โ€” Requested State Storage
+The fully assembled, policy-processed, placement-confirmed payload is stored as the **Requested State** in the Request Store. The Requested State includes:
+- All assembled resource fields with full provenance chain
+- Complete `placement` block: selected provider, hold UUID, all reserve query responses per iteration, all policy evaluations per iteration, placement constraints applied, alternatives considered
+- All `policy_gap_record` entries for implicit approvals
+- `enrichment_status` reflecting metadata completeness at dispatch time
+
+### Step 9 โ€” Provider Dispatch
+The Requested State payload is dispatched to the selected Service Provider via the API Gateway. The resource hold placed during the Placement Loop is confirmed by dispatch. The provider uses the hold reference to fulfill the request against the reserved resources.
+
+---
 
 ---
 
@@ -1080,12 +1240,12 @@ Consumer Request
       โ”‚
       โ–ผ
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
-โ”‚  REQUEST LAYER  โ”‚  โ† Consumer declared intent โ†’ stored as INTENT STATE
+โ”‚  REQUEST LAYER  โ”‚  โ† Consumer declared intent โ†’ stored as INTENT STATE (Step 1)
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
          โ”‚
          โ–ผ
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
-โ”‚                  LAYER RESOLUTION                        โ”‚
+โ”‚             LAYER RESOLUTION + MERGE (Steps 2-4)         โ”‚
 โ”‚                                                          โ”‚
 โ”‚  Base Layer          (lowest precedence)                 โ”‚
 โ”‚       โ†“                                                  โ”‚
@@ -1101,22 +1261,55 @@ Consumer Request
          โ”‚  Merged payload with full provenance
          โ–ผ
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
-โ”‚                  POLICY PROCESSING                       โ”‚
+โ”‚          PRE-PLACEMENT POLICY PROCESSING (Step 5)        โ”‚
 โ”‚                                                          โ”‚
 โ”‚  Transformation Policies  (enrich / modify)              โ”‚
 โ”‚       โ†“                                                  โ”‚
 โ”‚  Validation Policies      (pass / fail check)            โ”‚
 โ”‚       โ†“                                                  โ”‚
 โ”‚  GateKeeper Policies      (override / block)             โ”‚
+โ”‚       โ†“ outputs: placement constraints                   โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+         โ”‚  Policy-processed payload + placement constraints
+         โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚              PLACEMENT ENGINE โ€” LOOP (Step 6)            โ”‚
+โ”‚                                                          โ”‚
+โ”‚  For each candidate provider (filtered + scored):        โ”‚
+โ”‚    โ”‚                                                     โ”‚
+โ”‚    โ”œโ”€โ”€ Reserve Query (atomic: verify + metadata + hold)  โ”‚
+โ”‚    โ”‚     confirmed / partial โ†’ policy phase              โ”‚
+โ”‚    โ”‚     insufficient / refused โ†’ next candidate         โ”‚
+โ”‚    โ”‚                                                     โ”‚
+โ”‚    โ””โ”€โ”€ Loop Policy Phase (placement_phase: loop)         โ”‚
+โ”‚          Field present โ†’ evaluate normally               โ”‚
+โ”‚          Field absent + required_context โ†’ if_absent     โ”‚
+โ”‚          Field absent + no policy โ†’ implicit_approval    โ”‚
+โ”‚          pass/warn โ†’ PLACEMENT CONFIRMED                 โ”‚
+โ”‚          reject_candidate โ†’ release hold, next           โ”‚
+โ”‚          gatekeep โ†’ release hold, REJECT REQUEST         โ”‚
+โ”‚                                                          โ”‚
+โ”‚  No candidates remain โ†’ on_exhaustion behavior           โ”‚
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
-         โ”‚  Complete, validated, policy-processed payload
+         โ”‚  selected_provider_uuid + placement block
+         โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚         POST-PLACEMENT POLICY PROCESSING (Step 7)        โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Transformation Policies  (provider-aware enrichment)    โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  Validation Policies      (post-placement checks)        โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  GateKeeper Policies      (provider-triggered overrides) โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+         โ”‚  Complete, validated, placement-confirmed payload
          โ–ผ
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
-โ”‚ REQUESTED STATE โ”‚  โ† Stored in Request Store
-โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
-         โ”‚
+โ”‚ REQUESTED STATE โ”‚  โ† Stored in Request Store (Step 8)
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    includes: placement block, hold records,
+         โ”‚             policy gap records, enrichment_status
          โ–ผ
-   Service Provider
+   Service Provider  (Step 9 โ€” dispatch, hold confirmed)
 ```
 
 ---
diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md
index 81c1707..239e366 100644
--- a/content/docs/architecture/data-model/resource-service-entities.md
+++ b/content/docs/architecture/data-model/resource-service-entities.md
@@ -12,7 +12,7 @@ weight: 5
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Layering and Versioning](../layering-and-versioning/) | [Resource Type Hierarchy](../resource-type-hierarchy/) | [Service Dependencies](../service-dependencies/) | [Resource Grouping](../resource-grouping/)
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md)
 
 ---
 
@@ -410,7 +410,7 @@ dcm_capacity_rating:
 
 Every Resource/Service Entity carries a `relationships` section declaring its relationships to other entities โ€” internal DCM entities, external data entities, and business context entities. The relationship model is universal โ€” the same structure is used for all relationship types.
 
-See [Entity Relationships](../entity-relationships/) for the complete relationship model.
+See [Entity Relationships](09-entity-relationships.md) for the complete relationship model.
 
 ```yaml
 resource_service_entity:
@@ -452,6 +452,73 @@ The following are **non-overridable DCM System Policies** that apply to all Reso
 
 ---
 
+## 9a. Lifecycle Time Constraints
+
+### 9a.1 Concept
+
+**Lifecycle time constraints** declare when a resource should cease to exist or trigger a lifecycle action. They are a first-class field on any resource entity โ€” governed, provenance-tracked, and subject to the standard override control model.
+
+Any source in the data model precedence chain can declare a time constraint: a consumer request, a Core Layer, a Service Layer, or a policy. The Policy Engine has full authority over constraints โ€” a GateKeeper can lock a TTL immutable or set `immutable_ceiling: absolute` on an expiry date.
+
+### 9a.2 Constraint Structure
+
+```yaml
+lifecycle_constraints:
+  ttl:
+    duration: P14D                            # ISO 8601 duration
+    reference_point: realization_timestamp    # created_at | realization_timestamp | last_modified
+    on_expiry: 
+    metadata:
+      override: allow                         # standard override control
+      basis_for_value: "Consumer declared ephemeral โ€” 14-day lab resource"
+
+  expires_at:
+    timestamp: "2026-06-30T23:59:59Z"         # absolute calendar date
+    on_expiry: notify
+    metadata:
+      override: immutable
+      locked_by_policy_uuid: 
+      basis_for_value: "Project deadline โ€” resource must not persist beyond Q2"
+
+  enforcement:
+    warn_before_expiry: P1D                   # warn 1 day before expiry
+    grace_period: PT1H                        # 1 hour grace after expiry before action
+    on_grace_period_expiry: 
+```
+
+When both `ttl` and `expires_at` are declared, the earliest expiry wins (LTC-004).
+
+### 9a.3 Precedence
+
+Time constraints follow the same precedence as all other resource fields:
+
+```
+Base Layer (lowest โ€” e.g., no TTL by default)
+  โ†“  Core Layer (e.g., all dev resources: TTL 90 days)
+  โ†“  Service Layer (e.g., ephemeral compute: TTL 7 days)
+  โ†“  Request Layer (consumer declared)
+  โ†“  Transformation Policy (enrich from business context)
+  โ†“  GateKeeper Policy (highest โ€” may lock immutable)
+```
+
+### 9a.4 Expiry Enforcement
+
+The **Lifecycle Constraint Enforcer** is a DCM control plane component โ€” not a provider concern. It monitors realized entities, fires `on_expiry` actions when constraints are reached, and records all enforcement in provenance and the Audit Store.
+
+Entities whose `on_expiry` action fails to execute enter `PENDING_EXPIRY_ACTION` state and trigger an escalation (LTC-005).
+
+### 9a.5 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `LTC-001` | Lifecycle time constraints follow standard data model precedence |
+| `LTC-002` | GateKeeper policies may lock lifecycle constraints as immutable |
+| `LTC-003` | Expiry enforcement is a DCM control plane function |
+| `LTC-004` | When multiple time constraints exist, the earliest expiry wins |
+| `LTC-005` | Failed expiry action execution triggers `PENDING_EXPIRY_ACTION` state and escalation |
+
+---
+
 ## 10. Open Questions
 
 | # | Question | Impact | Status |
diff --git a/content/docs/architecture/overview.md b/content/docs/architecture/overview.md
index 1b2a136..ce7921b 100644
--- a/content/docs/architecture/overview.md
+++ b/content/docs/architecture/overview.md
@@ -232,20 +232,38 @@ Intent State captured (immutable consumer declaration)
   โ”‚  CI: policy pre-validation, cost estimate, sovereignty check
   โ”‚  Human review via PR (if policy requires)
   โ–ผ  [PR merged โ€” CD pipeline fires]
-Request Payload Processor
-  โ”‚  Layer assembly (Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request)
-  โ”‚  Policy Engine (Transformation โ†’ Validation โ†’ GateKeeper)
+Request Payload Processor โ€” Nine-Step Assembly
+  โ”‚
+  โ”‚  Steps 1-4: Layer assembly
+  โ”‚    Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request Layer
+  โ”‚
+  โ”‚  Step 5: Pre-Placement Policies
+  โ”‚    Transformation โ†’ Validation โ†’ GateKeeper
+  โ”‚    Outputs: placement constraints
+  โ”‚
+  โ”‚  Step 6: Placement Engine โ€” Placement Loop
+  โ”‚    For each candidate provider:
+  โ”‚      Reserve Query (atomic: verify + metadata + hold)
+  โ”‚      Loop Policy Phase (evaluates reserve query response)
+  โ”‚        pass โ†’ Placement confirmed
+  โ”‚        reject_candidate โ†’ next candidate
+  โ”‚        gatekeep โ†’ request rejected
+  โ”‚
+  โ”‚  Step 7: Post-Placement Policies
+  โ”‚    Transformation โ†’ Validation โ†’ GateKeeper
+  โ”‚    Provider-aware enrichment and validation
+  โ”‚
   โ–ผ
 Requested State committed to Git (full provenance chain)
-  โ”‚  Provider selected by placement component
+  โ”‚  Includes: placement block, hold records, policy gap records
   โ–ผ
-Provider dispatch via API Gateway
+Provider dispatch via API Gateway (hold confirmed)
   โ”‚  Naturalization: DCM format โ†’ provider native format
-  โ”‚  Provider realizes resource
+  โ”‚  Provider realizes resource, returns full metadata
   โ”‚  Denaturalization: provider native โ†’ DCM format
   โ–ผ
 Realized State (event stream, provider-confirmed)
-  โ”‚
+  โ”‚  enrichment_status updated as metadata arrives
   โ–ผ  [Continuous]
 Drift Detection: Discovered State vs Realized State
   โ”‚  Unsanctioned changes โ†’ Policy Engine response
@@ -256,6 +274,6 @@ Drift Detection: Discovered State vs Realized State
 
 ## Related Documents
 
-- [Data Model](data-model/) โ€” Complete data model documentation
+- [Data Model](data-model/) โ€” Complete data model documentation including the Ingestion Model for V1 migration and brownfield ingestion
 - [Specifications](specifications/) โ€” Operator Interface Specification, Kubernetes compatibility, SDK API, CNCF strategy
 - [Enhancements](../enhancements/) โ€” Enhancement proposals for the DCM project

From 21b8d311048f8465173ac3083a18d8f402691967 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 13:29:15 -0500
Subject: [PATCH 03/49] Added auditing specifications, deployment
 specifications

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 328 +++++++++++++++---
 .../docs/architecture/data-model/_index.md    |   4 +
 .../data-model/entity-relationships.md        |  13 +
 .../data-model/layering-and-versioning.md     |   4 +-
 .../data-model/resource-grouping.md           |   4 +-
 5 files changed, 303 insertions(+), 50 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index a47d3b9..e0b4a28 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -697,34 +697,67 @@ Rehydration uses **Intent State** โ€” not Realized State โ€” to reconstruct the
 
 ---
 
-## SECTION 10 โ€” RESOURCE GROUPING AND TENANCY
-
-### 10.1 DCM Tenant โ€” Mandatory First-Class Ownership
-
-Every Resource/Service Entity must belong to exactly one DCM Tenant. This is a **non-overridable DCM System Policy**.
-
-Tenant provides: ownership, isolation, cost attribution, policy scope, drift detection scope, rehydration scope, audit scope, sovereignty boundary.
-
-**Ownership vs Consumption:** A resource belongs to one Tenant (owner) but can be consumed by multiple Tenants via the Service Catalog. Consumption is tracked through service requests โ€” not Tenant membership.
-
-### 10.2 Resource Groups โ€” Flexible Composable Grouping
-
-Resource Groups function like **structured tags** โ€” a resource accumulates group memberships describing its context from multiple dimensions simultaneously.
+## SECTION 10 โ€” UNIVERSAL GROUP MODEL AND TENANCY
+
+> The Universal Group Model (document 15-universal-groups.md) supersedes the separate Tenant and Resource Group models for new implementations. All existing constructs map 1:1 to `group_class` values. Existing UUIDs and API surfaces are preserved.
+
+### 10.1 Universal Group Structure
+Every grouping construct in DCM is a `DCMGroup` with:
+- `group_class` โ€” determines system behavior (see 10.2)
+- `group_subclass` โ€” advisory label, no system behavior
+- `member_types_permitted` โ€” what can be a member
+- `exclusivity` โ€” one or many groups of this class per member
+- `enforcement_model` โ€” advisory | enforced | mandatory (tenant_boundary: profile-governed)
+- `lifecycle_policy` โ€” on_group_destroy: **detach (default)** | notify | cascade | retain
+- `former_group_membership` records retained permanently after detach
+- `group_destruction_record` retained in Audit Store permanently
+- Time-bounded membership: `valid_from / valid_until` on every membership
+
+### 10.2 Group Classes
+
+| group_class | Purpose | enforcement |
+|-------------|---------|------------|
+| `tenant_boundary` | Ownership/isolation boundary (replaces Tenant) | profile-governed |
+| `resource_grouping` | Flexible entity tagging (replaces Resource Groups) | advisory |
+| `policy_collection` | Policy cohesion unit (replaces Policy Group) | advisory |
+| `policy_profile` | Deployment configuration (replaces Policy Profile) | enforced |
+| `layer_grouping` | Related layers for a context | advisory |
+| `composite` | Cross-type organizational unit | configurable |
+| `federation` | Peer-group association (federated Tenants) | advisory |
+
+### 10.3 DCM Tenant โ€” tenant_boundary group_class
+Every resource entity must belong to exactly one `tenant_boundary` group (GRP-001). Provides: ownership, isolation, cost attribution, policy scope, drift detection scope, rehydration scope, audit scope, sovereignty boundary.
+
+**Structurally locked invariants** (cannot be overridden by any policy):
+- One tenant_boundary group per resource โ€” always
+- Constituent relationships never cross tenant_boundary boundaries โ€” at any nesting level
+
+**Profile-governed enforcement:** `minimal` profile โ†’ advisory; `standard`/`prod`/`fsi`/`sovereign` โ†’ mandatory. A GateKeeper policy fires when advisory tenancy detected in prod/fsi/sovereign deployment (GRP-011).
+
+### 10.4 Nested Tenants
+A `tenant_boundary` group can have `parent_group_uuid` pointing to another `tenant_boundary` group. The parent-child relationship is a governance and cost relationship โ€” NOT ownership transfer.
+
+**Parent can:** aggregate cost, apply governance overlay, query aggregate audit, declare child lifecycle policies  
+**Parent cannot:** own child resources, cross child isolation boundaries, override more-restrictive child policies
+
+**Governance inheritance โ€” most restrictive wins (GRP-009):**
+```
+Most restrictive policy at ANY nesting level wins
+  Child policies that are more restrictive than parent โ†’ child wins
+  Parent policies cascade where child has no policy or is less restrictive
+  Platform policies govern all tenant_boundary groups
+```
 
-**Two classes โ€” equal capability:**
-- **DCM Default Resource Group** โ€” built-in, standard grouping mechanism
-- **Custom Resource Group** โ€” implementor-defined, tied to business structures (CostCenter, BusinessUnit, RegulatoryScope, etc.)
+### 10.5 Federated Tenants
+A `federation` group contains peer `tenant_boundary` groups. Enables shared policy application, cross-federation visibility, and consolidated reporting. Does NOT grant governance authority โ€” member Tenants remain independent (GRP-010).
 
-Both implement the same **Resource Group Interface**.
+### 10.6 Composite Groups
+A `composite` group permits all member types โ€” resource entities, policies, layers, and other groups. Enables "everything about Payments" as one organizational unit.
 
-### 10.3 Multi-Group Membership
-A resource can belong to multiple groups across all classes. Membership constraints are configurable per group definition:
-- `exclusive: true` โ€” resource can only belong to one group of this type at a time
-- `exclusive: false` โ€” resource can belong to multiple groups of this type
-- Organizational policies can further restrict multi-group membership
+Policy targeting composite groups defaults to all member types. Declare `member_type_filter` to narrow. Policy linting warns if composite is targeted without filter (GRP-012).
 
-### 10.4 Nesting
-Groups that declare `nesting: true` can contain other groups as members. Nesting is configurable per group definition. Circular nesting is invalid. Child groups inherit policy scope from parent groups.
+### 10.7 GRP System Policies
+GRP-001 through GRP-014. Key: GRP-001 (one tenant_boundary per resource), GRP-003 (no circular nesting), GRP-005 (detach is default on destroy), GRP-006 (resource in leaf Tenant always), GRP-009 (most restrictive wins in nesting), GRP-011 (advisory tenancy in prod triggers notification), GRP-013 (former_group_membership permanent), GRP-014 (destruction record permanent).
 
 ### 10.5 DCM System Policies for Grouping
 
@@ -1422,16 +1455,16 @@ A **Policy Profile** is a complete DCM configuration for a specific use case com
 
 **Six DCM built-in profiles (least to most restrictive):**
 
-| Profile | Tenancy | Enforcement | Cross-Tenant | Audit |
-|---------|---------|-------------|-------------|-------|
-| `minimal` | Optional โ€” auto-created | Advisory only | allow_all | None |
-| `dev` | Recommended | Warn only | operational_only | Basic 90-day |
-| `standard` | Required | Blocking | explicit_only | Compliance-grade |
-| `prod` | Required | Blocking + SLA | explicit_only | Compliance-grade |
-| `fsi` | Hard tenancy | Blocking | explicit_only | 7-year retention |
-| `sovereign` | Hard tenancy | Blocking | deny_all | 10-year retention |
+| Profile | Handle | Tenancy | Enforcement | Cross-Tenant | Audit |
+|---------|--------|---------|-------------|-------------|-------|
+| `minimal` | `system/profile/minimal` | Optional โ€” auto-created | Advisory only | allow_all | None |
+| `dev` | `system/profile/dev` | Recommended | Warn only | operational_only | Basic 90-day |
+| `standard` | `system/profile/standard` | Required | Blocking | explicit_only | Compliance-grade |
+| `prod` | `system/profile/prod` | Required | Blocking + SLA | explicit_only | Compliance-grade |
+| `fsi` | `system/profile/fsi` | Hard tenancy | Blocking | explicit_only | 7-year retention |
+| `sovereign` | `system/profile/sovereign` | Hard tenancy | Blocking | deny_all | 10-year retention |
 
-**Profile inheritance chain:** sovereign extends fsi extends prod extends standard extends dev extends minimal
+**Profile inheritance chain:** `system/profile/sovereign` extends `system/profile/fsi` extends `system/profile/prod` extends `system/profile/standard` extends `system/profile/dev` extends `system/profile/minimal`
 
 **Profile activation levels (more specific wins):**
 ```yaml
@@ -1447,12 +1480,12 @@ A **Policy Provider** is a fifth DCM provider type โ€” an external authoritative
 
 **Four delivery modes:**
 
-| Mode | Name | Logic Lives In |
-|------|------|---------------|
-| 1 | DCM Native Push/Pull | DCM Policy Engine |
-| 2 | OPA/Rego Bundle | DCM Policy Engine (OPA) |
-| 3 | External Schema (naturalization) | DCM Policy Engine (post-translation) |
-| 4 | Black Box Query-Enrichment | External provider โ€” opaque to DCM |
+| Mode | Name | `delivery.mode` value | Logic Lives In |
+|------|------|----------------------|---------------|
+| 1 | DCM Native Push/Pull | `push` / `pull` / `webhook` | DCM Policy Engine |
+| 2 | OPA/Rego Bundle | `opa_bundle` | DCM Policy Engine (OPA) |
+| 3 | External Schema (naturalization) | `external_schema` | DCM Policy Engine (post-translation) |
+| 4 | Black Box Query-Enrichment | `black_box_query` | External provider โ€” opaque to DCM |
 
 **Modes 1-3** deliver policy rules. **Mode 4** is a query-response interface โ€” DCM sends data, external system evaluates and/or enriches, returns structured result.
 
@@ -1528,7 +1561,169 @@ When rehydration conflicts with current tenancy controls โ†’ entity enters **PEN
 
 ---
 
-## SECTION 22 โ€” PERSONAS
+## SECTION 22 โ€” UNIVERSAL GROUP MODEL
+
+DCM collapses all grouping constructs into a single **DCMGroup** entity with `group_class` metadata. One mental model, one API, one registry. See `15-universal-groups.md` for the complete model.
+
+### 22.1 Group Classes
+
+| group_class | Replaces | member_types_permitted | exclusivity |
+|-------------|---------|----------------------|------------|
+| `tenant_boundary` | Tenant | resource_entity, group | one (structural lock) |
+| `resource_grouping` | Resource Group | resource_entity | many |
+| `policy_collection` | Policy Group | policy | many |
+| `policy_profile` | Policy Profile | group | many |
+| `layer_grouping` | Layer grouping | layer | many |
+| `provider_grouping` | Provider collections | provider | many |
+| `composite` | (new) | all types | many |
+| `federation` | (new) | group (tenant_boundary) | many |
+
+### 22.2 Structural Invariants (non-overridable)
+- `GRP-INV-001` โ€” resource_entity belongs to exactly one tenant_boundary group
+- `GRP-INV-002` โ€” constituent relationships cannot cross tenant_boundary boundaries
+- `GRP-INV-003` โ€” destroying parent tenant_boundary requires explicit resolution of all children first
+- `GRP-INV-004` โ€” resource in child tenant_boundary belongs to child โ€” never parent
+- `GRP-INV-005` โ€” circular group membership invalid
+- `GRP-INV-006` โ€” group cannot be a member of itself
+
+### 22.3 Composite Groups
+`member_types_permitted: [resource_entity, policy, layer, group, provider]`. Policies targeting composite groups apply to all member types by default. Narrow with `member_type_filter`.
+
+### 22.4 Nested Tenants
+`tenant_boundary` group with `parent_group_uuid`. Parent has governance overlay and cost rollup โ€” not ownership. Policy inheritance direction is profile-governed (opt_in for minimal/dev/fsi/sovereign; opt_out for standard/prod).
+
+### 22.5 Federated Tenants
+`federation` group containing `tenant_boundary` groups as peers. Members remain fully independent. Enables shared governance, consolidated reporting, scoped cross-member visibility.
+
+### 22.6 API Backward Compatibility
+`GET /tenants` โ†’ `GET /groups?group_class=tenant_boundary`. All existing UUIDs and API endpoints preserved.
+
+---
+
+## SECTION 23 โ€” UNIVERSAL AUDIT MODEL
+
+Every modification to every DCM artifact produces an audit record. No exceptions. See `16-universal-audit.md` for the complete model.
+
+### 23.1 Four Required Fields
+**Date/time** (ISO 8601 microseconds) | **Who** (composite actor chain) | **What** (subject entity) | **Action** (closed vocabulary)
+
+### 23.2 Two-Stage Audit โ€” Synchronous Commit + Async Enrichment
+
+**Stage 1 (synchronous, < 1ms, in critical path):**
+```yaml
+commit_log_entry:
+  entry_uuid: 
+  sequence:            # monotonically increasing
+  timestamp:    # AUTHORITATIVE audit timestamp
+  entity_uuid / entity_type / action / actor_uuid / tenant_uuid
+  change_fingerprint: 
+  status: pending_forward
+```
+Written using Raft consensus โ€” confirmed when quorum (2/3 or 3/5) of Commit Log replicas acknowledge. Operation returns success after Stage 1. Stage 1 timestamp is the authoritative audit timestamp (AUD-013).
+
+**Stage 2 (asynchronous, out of critical path):**
+Audit Forward Service enriches minimal Commit Log entry โ†’ full audit_record โ†’ hash chain computed โ†’ written to Audit Store with retry. Full record visible seconds to minutes after Stage 1.
+
+### 23.3 Composite Actor Chain โ€” The "Who"
+```yaml
+actor:
+  immediate:
+    type: 
+    uuid / display_name
+  authorized_by:
+    uuid / authorization_method: 
+  request_uuid / policy_uuid / correlation_id
+```
+
+### 23.4 Action Vocabulary (closed โ€” AUD-007)
+`CREATE | MODIFY | STATE_TRANSITION | DELETE | ACTIVATE | DEACTIVATE | DEPRECATE | RETIRE | MEMBER_ADD | MEMBER_REMOVE | RELATIONSHIP_CREATE | RELATIONSHIP_RELEASE | AUTHORIZE | REVOKE | EVALUATE | ENRICH | LOCK | HOLD_PLACE | HOLD_CONFIRM | HOLD_RELEASE | DRIFT_DETECT | DRIFT_RESOLVE | INGEST | PROMOTE | EXPIRE | REHYDRATE | QUERY | DISCOVER | LOGIN | LOGOUT | CONFIG_CHANGE`
+
+### 23.5 Retention โ€” Reference-Based
+- `retention_status: live` โ€” any referenced entity non-retired โ†’ retain unconditionally
+- `retention_status: all_retired` โ†’ apply governing policy
+- Post-lifecycle defaults: dev=P90D, standard=P3Y, prod/fsi=P7Y (DEFAULT), sovereign=P10Y
+
+### 23.6 Tamper-Evidence โ€” Hash Chain
+`record_hash` (SHA-256 of record) + `previous_record_hash` (preceding record for this entity) + `chain_sequence`. Chain breaks detectable and trigger security alerts (AUD-010).
+
+### 23.7 Recoverability
+- DCM crash after Stage 1 โ†’ Audit Forward Service replays `pending_forward` entries on restart (AUD-011)
+- Audit Store unavailable โ†’ Commit Log accumulates; Audit Forward Service retries when recovered
+- Commit Log quorum unavailable โ†’ operation aborted โ€” no silent change
+
+### 23.8 System Policies
+- `AUD-001` โ€” Every modification produces Commit Log entry synchronously; Commit Log write failure aborts operation
+- `AUD-002` โ€” Audit records append-only and immutable while retention obligations apply
+- `AUD-003` โ€” Audit records survive at least as long as any referenced entity is live
+- `AUD-004` โ€” Post-lifecycle retention governed by policy; default P7Y
+- `AUD-005` โ€” Actor field must identify immediate actor + authorized_by chain
+- `AUD-006` โ€” record_hash + previous_record_hash hash chain required
+- `AUD-007` โ€” Action field must use closed vocabulary
+- `AUD-008` โ€” Audit Store must support queries by entity_uuid, actor_uuid, action, timestamp range, tenant_uuid, request_uuid, retention_status
+- `AUD-009` โ€” Audit Forward Service delivers with exponential backoff retry; cleared only after Audit Store confirmation AND retention window
+- `AUD-010` โ€” Hash chain verification first-class; chain breaks trigger immediate security alerts
+- `AUD-011` โ€” On restart, Audit Forward Service replays all pending_forward entries before accepting new operations
+- `AUD-012` โ€” Commit Log uses Raft consensus with quorum writes
+- `AUD-013` โ€” Stage 1 Commit Log timestamp is authoritative audit timestamp
+
+---
+
+## SECTION 24 โ€” DEPLOYMENT AND REDUNDANCY
+
+Every DCM component and store is designed for redundancy by default. Everything containerized. Profile-governed. Self-hosting. See `17-deployment-redundancy.md` for the complete model.
+
+### 24.1 Core Principles
+- **Redundant by default** โ€” every component and store has a redundancy model
+- **Everything containerized** โ€” all components run as Kubernetes pods; no bare-metal DCM
+- **Profile-governed** โ€” replica counts and quorum set by active Profile; not per-component
+- **Stateless control plane** โ€” all state in external stores; any pod can fail and be replaced
+- **Self-hosting** โ€” DCM's own deployment is a DCM resource; DCM manages itself
+
+### 24.2 Redundancy Matrix by Profile
+
+| Profile | CP Replicas | Store Replicas | Write Quorum | Zone Spread | Geo-Replication |
+|---------|------------|---------------|-------------|------------|----------------|
+| `minimal` | 1 | 1 | No | No | No |
+| `dev` | 1 | 1 | No | No | No |
+| `standard` | 3 | 3 | 2/3 | Preferred | No |
+| `prod` | 3 | 3 | 2/3 | Required | Yes |
+| `fsi` | 5 | 5 | 3/5 | Required | Yes |
+| `sovereign` | 5 | 5 | 3/5 | Required | Within boundary |
+
+### 24.3 Store Redundancy Model
+
+| Store | Implementation | Write Quorum | Notes |
+|-------|---------------|-------------|-------|
+| Commit Log | etcd (Raft) | 2/3 | Stage 1 audit; < 1ms write |
+| GitOps Store | Gitea/equivalent | 2/3 | Intent, Requested, Layers, Policies |
+| Event Stream | Kafka/equivalent | 2/3 | Realized, Discovered, Audit events |
+| Audit Store | Elasticsearch/equivalent | 2/3 | Indexed, queryable, compliance-grade |
+| Search Index | Elasticsearch/equivalent | 1 | Non-authoritative โ€” rebuildable from Git |
+
+### 24.4 Pod Security Model (all components)
+`run_as_non_root: true` | `run_as_user: 65534` | `read_only_root_filesystem: true` | `allow_privilege_escalation: false` | `capabilities.drop: [ALL]` | mTLS for all inter-component communication (RED-009)
+
+### 24.5 Self-Hosting
+DCM's own deployment is declared as a `dcm_deployment` DCM resource in Git. DCM runs drift detection on its own components. Bootstrap sequence: bootstrap installer โ†’ reads `dcm_deployment` from Git โ†’ provisions to target redundant state โ†’ hands off.
+
+### 24.6 The Repave Scenario
+DCM lost entirely โ†’ bootstrap installer on new cluster โ†’ reads `dcm_deployment` from Git โ†’ provisions itself โ†’ rehydrates customer workloads in dependency order โ†’ drift detection validates. Recovery bounded by infrastructure provisioning speed โ€” not backup restoration.
+
+### 24.7 System Policies
+- `RED-001` โ€” All DCM components run as containers in Kubernetes pods
+- `RED-002` โ€” All control plane components stateless
+- `RED-003` โ€” Profiles above `minimal`: replicas >= 3 with anti-affinity
+- `RED-004` โ€” Profiles above `minimal`: quorum writes with write_quorum >= 2
+- `RED-005` โ€” Commit Log uses Raft consensus with quorum writes
+- `RED-006` โ€” DCM deployment declared as DCM resource in Git
+- `RED-007` โ€” DCM runs drift detection on its own components
+- `RED-008` โ€” Rolling updates must not reduce replicas below min_available
+- `RED-009` โ€” All component communication uses mTLS
+- `RED-010` โ€” Bootstrap manifest is the only DCM config outside DCM's management scope
+
+---
+
+## SECTION 25 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -1545,7 +1740,7 @@ When rehydration conflicts with current tenancy controls โ†’ entity enters **PEN
 
 ---
 
-## SECTION 23 โ€” TERMINOLOGY GLOSSARY
+## SECTION 26 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -1598,6 +1793,33 @@ When rehydration conflicts with current tenancy controls โ†’ entity enters **PEN
 | **Tenant Advocate** | DCM's role in protecting Tenant interests in all provider interactions |
 | **DCM System Policy** | Non-overridable policy built into DCM โ€” cannot be disabled or overridden by organizational policy |
 | **Webhook** | Push-based outbound notification from DCM to an external system triggered by a DCM event |
+| **Commit Log** | Stage 1 audit store โ€” minimal record, Raft consensus quorum write, sub-millisecond; Audit Forward Service reads from it to produce full audit records |
+| **Audit Forward Service** | DCM component that enriches Commit Log entries into full audit_record structures and delivers them to the Audit Store asynchronously with retry |
+| **Self-Hosting** | DCM's own deployment is a DCM resource; DCM manages itself through the same model used for customer infrastructure |
+| **dcm_deployment** | The DCM resource declaring DCM's own deployment โ€” profile, replica counts, store implementations, redundancy configuration |
+| **Bootstrap manifest** | The minimal configuration outside DCM's management scope used to bootstrap DCM before it can manage itself |
+| **Redundancy by Default** | DCM architectural principle: every component and store has a redundancy model; `minimal` profile sets replicas: 1; all others set replicas >= 3 |
+| **Quorum Write** | Write confirmed durable only when a majority of replicas acknowledge it; used by Commit Log and all durable stores in standard+ profiles |
+| **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
+| **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
+| **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **group_subclass** | Advisory label on a DCMGroup โ€” no system behavior; used for organization-specific semantics (e.g., cost_center, business_unit) |
+| **composite group** | DCMGroup with group_class: composite โ€” permits cross-type membership (resources, policies, layers, groups) |
+| **federation group** | DCMGroup with group_class: federation โ€” peer association of tenant_boundary groups; enables shared policies and consolidated reporting |
+| **nested Tenant** | A tenant_boundary group with parent_group_uuid pointing to another tenant_boundary group |
+| **federated Tenant** | A tenant_boundary group that is a member of a federation group |
+| **former_group_membership** | Permanent provenance record retained by a member after group destruction or membership expiry |
+| **group_destruction_record** | Permanent Audit Store record of a destroyed group including its full member list at destruction time |
+| **member_type_filter** | Policy targeting declaration narrowing scope within a composite group to specific member types |
+| **most_restrictive_wins** | Governance inheritance principle for nested Tenants โ€” most restrictive policy at any level in the hierarchy applies |
+| **Nested Tenant** | tenant_boundary group with parent_group_uuid โ€” child maintains isolation; parent has governance overlay and cost rollup |
+| **GRP-INV** | Universal group structural invariants โ€” non-overridable regardless of enforcement_model or Profile |
+| **Universal Audit Record** | Uniform audit record produced by every DCM component for every change โ€” date/time, who, what, action |
+| **Composite Actor Chain** | The who in an audit record โ€” immediate actor + authorized_by human chain + originating request/policy |
+| **Action Vocabulary** | Closed set of audit record action values โ€” free text rejected at write time (AUD-007) |
+| **Reference-Based Retention** | Audit records retained while any referenced entity is live โ€” not fixed time schedule |
+| **Write-Ahead Log (WAL)** | Local audit delivery buffer โ€” change + audit record written atomically; Audit Store delivery async with retry; WAL cleared after Audit Store confirms |
+| **Hash Chain** | Per-entity tamper-evident chain: record_hash + previous_record_hash; chain breaks detectable and trigger security alerts |
 | **Mode 4 Policy Provider** | Black box query-enrichment policy provider โ€” DCM sends query, external system evaluates and/or enriches, returns structured result; logic is opaque to DCM |
 | **Black Box Query-Enrichment** | Mode 4 operation where an external system simultaneously evaluates request data and injects enrichment fields into the payload |
 | **audit_token** | Provider-issued reference in Mode 4 responses enabling cross-system audit correlation between DCM audit trail and provider's internal logs |
@@ -1606,7 +1828,6 @@ When rehydration conflicts with current tenancy controls โ†’ entity enters **PEN
 | **Policy Group** | Cohesive versioned collection of policies addressing a single identifiable concern โ€” the unit of policy reuse |
 | **Policy Profile** | Complete DCM configuration for a specific use case โ€” composed of Policy Groups |
 | **Policy Provider** | Fifth DCM provider type โ€” external authoritative source supplying policies into DCM |
-| **Policy Naturalization** | Translation of external policy schemas (OSCAL, XCCDF, CIS JSON) into DCM policy format |
 | **concern_type** | Policy Group classification: technology, compliance, sovereignty, business, operational, security |
 | **minimal profile** | Least restrictive built-in profile โ€” advisory enforcement, auto-tenant, home lab / evaluation |
 | **sovereign profile** | Most restrictive built-in profile โ€” hard tenancy, deny_all cross-tenant, maximum sovereignty |
@@ -1626,7 +1847,6 @@ When rehydration conflicts with current tenancy controls โ†’ entity enters **PEN
 | **`__transitional__` Tenant** | System-managed holding Tenant for unassigned ingested entities โ€” cannot be deleted, renamed, or used for new provisioning |
 | **Ingestion Confidence** | `high | medium | low` โ€” quality signal for auto-assignment; reflects how reliable the Tenant assignment is |
 | **Brownfield** | Existing infrastructure not yet under DCM lifecycle management โ€” brought in via brownfield ingestion |
-| **Greening the Brownfield** | The progressive process of bringing unmanaged infrastructure under DCM lifecycle control via the ingestion model |
 | **V1 Migration** | Migration of pre-Tenant DCM V1 entities to V2 using the ingestion model |
 | **INGESTED state** | First ingestion lifecycle state โ€” entity in DCM, minimal metadata, Tenant may be __transitional__ |
 | **ENRICHING state** | Second ingestion lifecycle state โ€” Tenant assigned, metadata and relationships being completed |
@@ -1682,7 +1902,6 @@ When rehydration conflicts with current tenancy controls โ†’ entity enters **PEN
 | **Audit Component** | Separate DCM component aggregating provenance events from all stores โ€” compliance-grade, long-retention |
 | **Observability Store** | Time-series metrics, traces, and logs โ€” operational, not compliance-grade |
 | **Third Rail** | Direct API ingress path โ€” bypasses PR workflow, never bypasses governance |
-| **Unsanctioned Change** | A resource modification not traceable to a DCM request โ€” triggers UNSANCTIONED_CHANGE event |
 | **Layer Domain** | Organizational and architectural home of a layer โ€” system, platform, tenant, service, provider |
 | **Layer Handle** | Human-readable stable identifier for a layer โ€” format: domain/layer_type/name |
 | **Priority Schema** | Hierarchical dotted-notation priority system for deterministic layer conflict resolution |
@@ -1700,7 +1919,7 @@ When rehydration conflicts with current tenancy controls โ†’ entity enters **PEN
 
 ---
 
-## SECTION 24 โ€” OPEN QUESTIONS
+## SECTION 27 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -1797,7 +2016,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 25 โ€” DOCUMENTATION STRUCTURE
+## SECTION 28 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -1845,7 +2064,7 @@ content/
 
 ---
 
-## SECTION 26 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 29 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -1889,6 +2108,21 @@ When working on this project, follow these instructions:
 38. **Conflicts are resolved at ingestion, not assembly** โ€” all active layers in DCM are pre-validated conflict-free; the assembly process never encounters an ambiguous merge; if a conflict is found at ingestion, the PR is blocked until resolved
 39. **Priority schema is advisory for categories, mandatory for ordering** โ€” the reference taxonomy (900=Compliance, 800=Security, etc.) is advisory and organizations may adapt it; however, the numeric comparison rule is always enforced and always deterministic
 40. **Proposed status enables shadow validation** โ€” policy artifacts in proposed status execute in shadow mode against real traffic; output is captured in proposed_evaluation_record but never applied; this is the required validation step before activation
+65. **Universal Group Model supersedes separate grouping constructs** โ€” all Tenants, Resource Groups, Policy Groups, and Profiles are DCMGroup entities with group_class; use the universal model for new implementations; existing UUIDs and APIs are preserved
+66. **group_class drives system behavior, group_subclass is advisory** โ€” never design system behavior around group_subclass; only the built-in group_class values produce system-enforced behavior
+67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
+68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
+69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+71. **Two-stage audit: Stage 1 is the durability guarantee** โ€” the Commit Log quorum write confirms the change is audited; Stage 2 enrichment is asynchronous; Stage 1 timestamp is the authoritative audit timestamp (AUD-013)
+72. **Redundancy is profile-governed โ€” not per-component** โ€” never configure replica counts individually; activate the appropriate Profile and it configures redundancy for the entire deployment
+73. **DCM is self-hosting** โ€” DCM's own deployment is a DCM resource; DCM manages itself through the same four-state model, policy engine, and audit trail used for customer workloads
+74. **Everything is containerized** โ€” no bare-metal DCM components; all components run as Kubernetes pods following the standard pod security model; state is always in external stores (stateless control plane)
+65. **All DCM grouping uses DCMGroup with group_class** โ€” there is no separate Tenant entity, Resource Group entity, or Policy Group entity; they are all group_class values; use the class-filtered API views for backward compatibility
+66. **Composite groups apply to all member types by default** โ€” always use member_type_filter when targeting a composite group with a policy that should apply only to specific member types
+67. **Nested Tenants inherit governance from parent โ€” not ownership** โ€” a resource always belongs to its leaf tenant_boundary group; the parent has governance overlay and cost rollup only; GRP-INV-004 is non-overridable
+68. **Every change produces an audit record โ€” no exceptions** โ€” the WAL guarantees delivery; WAL write failure aborts the change; no silent unaudited changes are possible
+69. **Audit retention is reference-based โ€” not time-based** โ€” a 7-year retention policy means 7 years AFTER all referenced entities retire; a record created 20 years ago is retained unconditionally if any referenced entity is still operational
+70. **The audit hash chain is tamper-evident** โ€” any insertion, modification, or deletion of a historical record breaks the chain; verification is a first-class DCM operation; chain breaks trigger security alerts
 63. **Mode 4 Policy Providers are query-response interfaces** โ€” logic lives externally; DCM sends minimized data, receives decision and/or enrichment; data sovereignty check always runs before any query is dispatched; default failure behavior is gatekeep
 64. **Mode 4 enrichment fields carry full provenance** โ€” source_type: black_box_provider, source_uuid, and audit_token; override control applies; a GateKeeper can refuse enrichment on sensitive fields; enrichment providers require transformation trust level minimum
 57. **Policy Profiles are the primary configuration mechanism** โ€” most deployments activate a built-in profile and add organization-specific groups; do not configure individual policies from scratch when a profile covers the use case
diff --git a/content/docs/architecture/data-model/_index.md b/content/docs/architecture/data-model/_index.md
index 5fd0171..2ea55dd 100644
--- a/content/docs/architecture/data-model/_index.md
+++ b/content/docs/architecture/data-model/_index.md
@@ -27,4 +27,8 @@ realization, operation, and decommission.
   {{< card link="audit-provenance-observability" title="Audit, Provenance, and Observability" icon="eye" subtitle="The three distinct concerns of audit, data lineage, and operational observability." >}}
   {{< card link="ingestion-model" title="Ingestion Model" icon="inbox-in" subtitle="Unified V1 migration and brownfield ingestion: ingest, enrich, promote. Transitional Tenant and ING system policies." >}}
   {{< card link="policy-profiles" title="Policy Organization" icon="collection" subtitle="Policy Groups, Profiles, and Providers. Built-in profiles from minimal to sovereign. Lifecycle constraints. Cross-tenancy authorization." >}}
+  {{< card link="universal-groups" title="Universal Group Model" icon="user-group" subtitle="One DCMGroup with group_class. Composite groups, nested Tenants, federated Tenants. All grouping constructs unified." >}}
+  {{< card link="universal-audit" title="Universal Audit Model" icon="shield-check" subtitle="Two-stage audit: synchronous Commit Log + async Audit Store. Reference-based retention. Tamper-evident hash chains. AUD-001 through AUD-013." >}}
+  {{< card link="deployment-redundancy" title="Deployment and Redundancy" icon="server" subtitle="Redundant by default. Everything containerized. Profile-governed replicas. Self-hosting. Quorum writes. RED-001 through RED-010." >}}
+  {{< card link="universal-groups" title="Universal Group Model" icon="view-grid" subtitle="Unified grouping: Tenants, Resource Groups, Policy Groups as one model. Composite groups, nested and federated Tenants, permanent membership history." >}}
 {{< /cards >}}
diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md
index 3e64ddd..d7590ef 100644
--- a/content/docs/architecture/data-model/entity-relationships.md
+++ b/content/docs/architecture/data-model/entity-relationships.md
@@ -246,6 +246,11 @@ tenant:
 | `REL-010` | Constituent relationships may not cross Tenant boundaries |
 | `REL-011` | Cross-tenant operational relationships require explicit authorization from both the owning Tenant and the consuming Tenant |
 | `REL-012` | A Tenant with `hard_tenancy.cross_tenant_relationships: deny_all` may not participate in any cross-tenant relationship in any direction |
+| `XTA-001` | Cross-tenant information sharing is closed by default โ€” explicit authorization required for all cross-tenant relationships of any nature (see Policy Organization document Section 6) |
+| `XTA-002` | Cross-tenant authorizations must specify who, what, when, and where |
+| `XTA-003` | More specific authorizations take precedence: field_specific > resource_specific > tenant_global |
+| `XTA-004` | All cross-tenant authorization decisions are policy-driven and DCM-enforced |
+| `XTA-005` | Sovereignty constraints declared by either Tenant must be honored by all cross-tenant relationships |
 
 ---
 
@@ -891,6 +896,14 @@ lifecycle_policy:
       modifications: []
 ```
 
+### 11.2a Cross-Tenant Dependency System Policies
+
+| Policy | Rule |
+|--------|------|
+| `DEP-001` | Cross-tenant constituent dependencies are prohibited โ€” a dependency that would produce a constituent cross-tenant relationship is rejected at dependency graph construction time |
+| `DEP-002` | Cross-tenant operational dependencies require a valid available allocation record on the target resource โ€” failure returns `CROSS_TENANT_DEPENDENCY_UNAVAILABLE` |
+| `DEP-003` | A Resource Type Specification may only declare cross-tenant dependencies if explicitly marked `cross_tenant: permitted` โ€” default is `cross_tenant: not_permitted` |
+
 ### 11.3 Relationship Versioning and Deprecation
 
 Relationships follow the universal versioning and deprecation model. A relationship version changes when its lifecycle policy, nature, or role changes. Terminated relationships are retained in provenance permanently.
diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md
index 3d72794..297c397 100644
--- a/content/docs/architecture/data-model/layering-and-versioning.md
+++ b/content/docs/architecture/data-model/layering-and-versioning.md
@@ -255,7 +255,7 @@ The **Priority Schema** is the deterministic ordering mechanism for resolving co
 
 **Format:** `{integer}.{integer}.{integer}...` โ€” unlimited depth
 
-**Comparison:** Left-to-right, segment by segment. **Higher numeric value = higher priority.** No ceiling โ€” you can always go higher.
+**Comparison:** Left-to-right, segment by segment. **Higher numeric value = higher priority** (higher value = higher priority). No ceiling โ€” you can always go higher.
 
 ```
 900.10    beats    800.10    (900 > 800 at segment 1)
@@ -1036,7 +1036,7 @@ field_name:
 
 ---
 
-The Request Payload Processor assembles the final payload by executing the following steps in order. Each step is recorded in the payload's provenance chain.
+The Request Payload Processor assembles the final payload by executing the following **nine steps** in order. Each step is recorded in the payload's provenance chain.
 
 ### Step 1 โ€” Intent Capture
 The consumer's Request Layer is received and stored as the **Intent State** in the Intent Store. No modification occurs at this step. The Intent State is the immutable record of what the consumer asked for.
diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md
index d90db24..5c7f31f 100644
--- a/content/docs/architecture/data-model/resource-grouping.md
+++ b/content/docs/architecture/data-model/resource-grouping.md
@@ -11,8 +11,10 @@ weight: 7
 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
+> **Universal Group Model:** The constructs defined in this document (DCM Tenant, Resource Groups) are superseded by the [Universal Group Model](15-universal-groups.md) for new implementations. Existing constructs map 1:1 to `group_class` values in the universal model โ€” UUIDs, handles, and API references are preserved. This document remains valid as a reference for pre-universal implementations and for understanding the migration path.
+
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Resource/Service Entities](../resource-service-entities/) | [Service Dependencies](../service-dependencies/)
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md)
 
 ---
 

From 8911368799f05401f9e95eb3e51c317a7819b35e Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 14:18:09 -0500
Subject: [PATCH 04/49] Added auth providers and webhooks

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 150 +++++++++++++++++-
 .../docs/architecture/data-model/_index.md    |   2 +
 2 files changed, 144 insertions(+), 8 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index e0b4a28..d9a26c8 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -1218,9 +1218,12 @@ Providers must honor a multi-dimensional contract:
 ### 16.4 Provider Types
 - **Atomic Providers** โ€” manage a single fundamental resource type (VM, IP, VLAN, container)
 - **Meta Providers** โ€” compose multiple providers as components of their own service
-- **Process Providers** โ€” purely process-based (no infrastructure resource, but a workflow or automation)
-- **Policy Providers** โ€” supply policies from external authoritative sources; follow same base contract; three delivery modes (push/pull/webhook); three formats (dcm_native/opa_rego/external_schema); trust level governs max authority
-- **Real-world providers** are typically combinations of all three
+- **Process Providers** โ€” purely process-based workflow or automation
+- **Policy Providers** โ€” supply policies; Mode 4 evaluates/enriches via black box query
+- **Message Bus Providers** โ€” bidirectional bridge to external message buses (Kafka, AMQP, NATS, etc.)
+- **Credential Providers** โ€” resolve secrets from external stores (Vault, AWS SM, Azure KV, CyberArk, etc.)
+- **Auth Providers** โ€” authenticate identities and resolve permissions (OIDC, LDAP, AD, FreeIPA, etc.)
+- **Real-world providers** are typically combinations of the above
 
 ---
 
@@ -1723,7 +1726,132 @@ DCM lost entirely โ†’ bootstrap installer on new cluster โ†’ reads `dcm_deployme
 
 ---
 
-## SECTION 25 โ€” PERSONAS
+## SECTION 25 โ€” WEBHOOKS, MESSAGING, AND EXTERNAL INTEGRATION
+
+### 25.1 The Three Integration Mechanisms
+- **Outbound Webhooks** โ€” DCM pushes event notifications to external HTTP endpoints
+- **Inbound Webhooks** โ€” External systems push requests, queries, and events to DCM
+- **Message Bus Provider** โ€” Persistent bidirectional event streaming with external message buses
+
+All three are authenticated, authorized, and audited identically to any other DCM API call. No privileged back-channel.
+
+### 25.2 Universal Ingress/Egress Actor Model
+
+Every request carries an immutable `ingress` block set by the DCM ingress layer:
+```yaml
+ingress:
+  surface: 
+  protocol: 
+  authenticated_via: 
+  actor:
+    uuid / type / display_name / identity_source
+    auth_provider_uuid / auth_provider_type
+    roles / tenant_scope / groups / permissions
+    authorized_by: {method, authorizing_entity_uuid, expiry}
+    session_uuid / mfa_verified
+    external_identity: {provider, subject, claims}
+  webhook_registration_uuid / message_bus_provider_uuid / parent_request_uuid
+  source_ip  # audit only โ€” never used for authorization
+```
+The ingress block is immutable โ€” policies may read but never modify it. Carried verbatim into audit records. Policies can act on any ingress field (surface, actor.roles, auth_provider_type, mfa_verified, etc.).
+
+Egress calls carry DCM's authenticated identity via the `egress` block: component, authenticated_via, credential_ref, originating_request_uuid.
+
+### 25.3 Outbound Webhooks
+Optional and policy-governed. Profile sets defaults โ€” fsi/sovereign may require via Policy Group.
+
+Key properties:
+- **Schema adapters** โ€” consumer pins to a schema version; DCM transforms forever; 90-day deprecation notice
+- **Managed secret rotation** โ€” automatic with transition window; consumer notified via signed event
+- **Endpoint health** โ€” suspend-not-delete on failure; full config retained for reactivation
+- **Versioned registrations** โ€” standard DCM artifact lifecycle; Git-managed
+- **Sovereignty-aware** โ€” delivery blocked if endpoint jurisdiction incompatible with Tenant sovereignty
+
+Delivery: at-least-once; per-entity ordering guaranteed; cross-entity ordering not guaranteed; `event_uuid` is idempotency key.
+
+### 25.4 Inbound Webhooks
+DCM exposes typed authenticated endpoints:
+- `POST /webhooks/inbound/request` โ€” submit service request
+- `POST /webhooks/inbound/query` โ€” query entity state/catalog
+- `POST /webhooks/inbound/event` โ€” push provider state change / CI/CD signal
+- `POST /webhooks/inbound/ingestion` โ€” push brownfield ingestion data
+- `POST /webhooks/inbound/data` โ€” push enrichment or information data
+
+Callers must be registered as **webhook actors** with role, tenant_scope, permitted_operations, and rate_limit. Full Policy Engine evaluation โ€” same as any other API call. Returns 202 Accepted + request_uuid for async operations.
+
+### 25.5 Message Bus Provider (Sixth Provider Type)
+Persistent bidirectional event streaming. Supports: kafka, amqp, nats, mqtt, azure_service_bus, aws_eventbridge, gcp_pubsub, rabbitmq, custom.
+
+Inbound messages processed as authenticated API calls via registered webhook actor identity. Same Policy Engine evaluation as inbound webhooks.
+
+Architecture: internal Message Bus โ†’ Message Bus Bridge Service โ†” external message bus.
+
+### 25.6 Webhook System Policies
+WHK-001 through WHK-014 โ€” see doc 18. Key: ING-008 (ingress block immutable), ING-009 (full actor context required), ING-010 (egress authenticated), ING-011 (no anonymous access), ING-012 (webhook/message bus always authenticated).
+
+---
+
+## SECTION 26 โ€” AUTHENTICATION, AUTHORIZATION, AND AUTH PROVIDERS
+
+### 26.1 Auth Provider โ€” The Eighth Provider Type
+An **Auth Provider** answers two questions: (1) is this identity who they claim to be? and (2) what are they permitted to do? Every authentication mode is an Auth Provider implementation.
+
+**Authentication is always required โ€” no anonymous access in any profile.** The difference between profiles is how much effort setup requires.
+
+### 26.2 Built-In Auth Provider (zero configuration)
+Always registered, cannot be deregistered. Supports:
+- **Static API key** โ€” generated at bootstrap, shown once, 30 seconds to start
+- **Local users** โ€” `dcm user create --username admin --role platform_admin`
+- **GitHub/GitLab OAuth** โ€” opt-in, requires client_id + secret
+
+### 26.3 Auth Modes by Profile
+
+| Profile | Auth Modes | Setup Effort |
+|---------|-----------|-------------|
+| `minimal` | Static API key, Local user/password | 30 seconds โ€“ 2 min |
+| `dev` | + GitHub/GitLab OAuth, FreeIPA/AD direct bind | 5โ€“15 minutes |
+| `standard` | + OIDC via broker, AD/FreeIPA direct | 30โ€“60 minutes |
+| `prod` | + OIDC direct, MFA configurable | 1โ€“2 hours |
+| `fsi` | + mTLS required, MFA required | 4โ€“8 hours |
+| `sovereign` | + Air-gapped OIDC/mTLS | 1โ€“2 days |
+
+No anonymous access in any profile. No static API key in standard+. mTLS required in fsi/sovereign.
+
+### 26.4 LDAP / FreeIPA / Active Directory
+FreeIPA: direct LDAP bind with optional Kerberos SSO and HBAC enforcement. Ideal for Red Hat / Linux-first environments.
+
+Active Directory: LDAP bind with `LDAP_MATCHING_RULE_IN_CHAIN` (OID 1.2.840.113556.1.4.1941) for nested group resolution. `sAMAccountName` or UPN for user lookup. Automatic DC failover.
+
+Both support: group_role_map (external groups โ†’ DCM roles), tenant_mapping (external groups โ†’ DCM Tenants), group_sync (interval-based re-sync), multiple domain controllers for failover.
+
+### 26.5 Multiple Auth Providers and Routing
+Multiple providers registered simultaneously. Ingress layer routes based on authentication signal (mtls_client_cert โ†’ mtls provider; bearer_token โ†’ OIDC or API key provider; basic_auth โ†’ LDAP; hmac_signature โ†’ webhook provider; none โ†’ reject).
+
+Auth providers can be chained: authentication (LDAP bind) โ†’ enrichment (LDAP groups) โ†’ augmentation (OIDC userinfo for rich claims like department, cost_center).
+
+### 26.6 Credential Provider (Seventh Provider Type)
+Cross-cutting dependency for all secret resolution. Supports: hashicorp_vault, aws_secrets_manager, azure_key_vault, gcp_secret_manager, kubernetes_secrets, cyberark, delinea, external_api, dcm_internal.
+
+All provider registrations, webhook configurations, and Auth Provider connections reference credentials via:
+```yaml
+secret_ref:
+  credential_provider_uuid: 
+  secret_path: "dcm/path/to/secret"
+  version: latest
+```
+Credentials never stored in Git. Never appear in audit record values (only secret_path logged). Cached in memory with configurable TTL.
+
+### 26.7 Auth Provider Health
+On unhealthy: existing sessions remain valid until TTL expiry; new auth attempts route to fallback_provider_uuid or are rejected. On_unhealthy options: alert, fallback_to_next, block_new_sessions.
+
+### 26.8 System Policies
+AUTH-001 through AUTH-010 โ€” see doc 19. Key: AUTH-008 (no anonymous access in any profile), AUTH-009 (webhook/message bus always authenticated), AUTH-007 (credentials always via Credential Provider).
+
+---
+
+## SECTION 27 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -1740,7 +1868,7 @@ DCM lost entirely โ†’ bootstrap installer on new cluster โ†’ reads `dcm_deployme
 
 ---
 
-## SECTION 26 โ€” TERMINOLOGY GLOSSARY
+## SECTION 28 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -1919,7 +2047,7 @@ DCM lost entirely โ†’ bootstrap installer on new cluster โ†’ reads `dcm_deployme
 
 ---
 
-## SECTION 27 โ€” OPEN QUESTIONS
+## SECTION 29 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2016,7 +2144,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 28 โ€” DOCUMENTATION STRUCTURE
+## SECTION 30 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2064,7 +2192,7 @@ content/
 
 ---
 
-## SECTION 29 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 31 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2113,6 +2241,12 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+75. **Eight provider types โ€” not five** โ€” Message Bus Provider (6), Credential Provider (7), and Auth Provider (8) complete the ecosystem; all follow the same base contract
+76. **The ingress block is the policy surface for all access control** โ€” every request carries surface, protocol, authenticated_via, actor.roles, actor.auth_provider_type, mfa_verified, and external_identity claims; GateKeeper policies act on all of these
+77. **No anonymous access in any profile** โ€” minimal profile uses static API key (30 seconds to set up); the authentication ladder is about setup effort, not whether auth exists; AUTH-008 is non-negotiable
+78. **Credentials always via Credential Provider** โ€” no plaintext credentials anywhere in DCM: not in Git, not in audit records, not in logs; always reference a Credential Provider secret_path
+79. **Auth Providers are versioned artifacts** โ€” role_mapping and tenant_mapping changes go through proposed โ†’ active validation; Auth Provider config changes are audited; multiple providers can be registered simultaneously with signal-based routing
+80. **Webhook registrations are versioned artifacts** โ€” Git-managed, lifecycle-governed, schema adapters for long-lived compatibility; inbound callers must be registered as webhook actors with explicit permissions
 71. **Two-stage audit: Stage 1 is the durability guarantee** โ€” the Commit Log quorum write confirms the change is audited; Stage 2 enrichment is asynchronous; Stage 1 timestamp is the authoritative audit timestamp (AUD-013)
 72. **Redundancy is profile-governed โ€” not per-component** โ€” never configure replica counts individually; activate the appropriate Profile and it configures redundancy for the entire deployment
 73. **DCM is self-hosting** โ€” DCM's own deployment is a DCM resource; DCM manages itself through the same four-state model, policy engine, and audit trail used for customer workloads
diff --git a/content/docs/architecture/data-model/_index.md b/content/docs/architecture/data-model/_index.md
index 2ea55dd..19ef264 100644
--- a/content/docs/architecture/data-model/_index.md
+++ b/content/docs/architecture/data-model/_index.md
@@ -30,5 +30,7 @@ realization, operation, and decommission.
   {{< card link="universal-groups" title="Universal Group Model" icon="user-group" subtitle="One DCMGroup with group_class. Composite groups, nested Tenants, federated Tenants. All grouping constructs unified." >}}
   {{< card link="universal-audit" title="Universal Audit Model" icon="shield-check" subtitle="Two-stage audit: synchronous Commit Log + async Audit Store. Reference-based retention. Tamper-evident hash chains. AUD-001 through AUD-013." >}}
   {{< card link="deployment-redundancy" title="Deployment and Redundancy" icon="server" subtitle="Redundant by default. Everything containerized. Profile-governed replicas. Self-hosting. Quorum writes. RED-001 through RED-010." >}}
+  {{< card link="webhooks-messaging" title="Webhooks and Messaging" icon="arrows-expand" subtitle="Inbound and outbound webhooks. Message Bus Provider. Universal ingress/egress actor model. Credential Provider. WHK and ING policies." >}}
+  {{< card link="auth-providers" title="Auth Providers" icon="lock-closed" subtitle="Eight provider types. Auth ladder from API key to air-gapped OIDC. LDAP, FreeIPA, AD, OIDC, mTLS. No anonymous access. AUTH policies." >}}
   {{< card link="universal-groups" title="Universal Group Model" icon="view-grid" subtitle="Unified grouping: Tenants, Resource Groups, Policy Groups as one model. Composite groups, nested and federated Tenants, permanent membership history." >}}
 {{< /cards >}}

From 44ecdd31756e43b76da6018c4e99aa4d2db908d4 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 15:54:21 -0500
Subject: [PATCH 05/49] Added git pr ingestion, additional detail on auth
 providers, webhooks, layers, sovereignty controls.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 473 +++++++++---------
 .../docs/architecture/data-model/_index.md    |   1 +
 .../data-model/layering-and-versioning.md     | 395 +++++++++++++--
 .../data-model/storage-providers.md           | 386 +++++++++++++-
 4 files changed, 983 insertions(+), 272 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index d9a26c8..bfe5dd7 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -271,250 +271,72 @@ All DCM capabilities โ€” catalog, requests, entities, policies, audit, observabi
 
 ## SECTION 6 โ€” DATA LAYERS AND THE ASSEMBLY PROCESS
 
-### 6.1 What is a Layer?
-A Layer is a **declarative, immutable, versioned unit of data** that contributes fields to a merged payload. Layers do not execute โ€” they declare. Every layer has a UUID, follows universal versioning, is immutable once published, carries a parent entity reference, and contributes field-level provenance metadata for every field it sets.
-
-### 6.2 Layer Types
-
-| Layer Type | Scope | Ownership | Purpose |
-|------------|-------|-----------|---------|
-| **Base Layer** | Type-agnostic or type-scoped | DCM platform / implementor | Foundation entity โ€” minimum required fields and defaults. Every chain starts here. |
-| **Core Layers** | Type-agnostic by default | Infrastructure teams / implementors | Organizational, infrastructure, and contextual data applicable across all resource types (DC, Zone, Rack, Region, Environment) |
-| **Intermediate / Customization Layers** | Type-agnostic or type-scoped | Organizational teams / domain owners | Organizational hierarchy and deployment context overrides stacked between Core and Service Layers |
-| **Service Layers** | **Must be type-scoped** | Service Providers / service domain teams | Service-specific configuration for a specific Resource Type. Invalid without declared type scope. |
-| **Request Layer** | Scoped to requested Resource Type | Consumer | Consumer's declared intent. Becomes Intent State on submission. Has higher precedence than all data layers. |
-| **Policy Layers** | Scoped by policy domain | Policy creators / security / compliance | Governance layer โ€” operates on the assembled payload after data layers are merged |
-
-### 6.3 Service Layer Type Scope
-Service Layers must declare their Resource Type scope and scope inheritance behavior:
-```yaml
-type_scope:
-  resource_type_uuid: 
-  resource_type_fully_qualified_name: 
-  scope_inheritance: 
-  # exact: applies only to the declared Resource Type
-  # descendants: applies to the declared type and all child types via inheritance
-```
-
-### 6.4 Precedence Order
-From lowest to highest precedence:
-```
-1. Base Layer                    (foundation defaults)
-2. Core Layers                   (organizational and infrastructure context)
-3. Intermediate/Customization    (organizational hierarchy overrides)
-4. Service Layers                (service-specific configuration)
-5. Request Layer                 (consumer intent โ€” overrides all data layers)
-6. Transformation Policies       (enrich / modify โ€” additive)
-7. Validation Policies           (pass/fail โ€” no field modification)
-8. GateKeeper Policies           (highest authority โ€” overrides everything including consumer input)
-```
-
-### 6.5 Policy Layer Behavior
-- **Validation** โ€” checks data against rules, does not modify. Pass/fail. Failure rejects the request.
-- **Transformation** โ€” enriches or modifies the payload. Adds missing fields, applies standards. Recorded in provenance.
-- **GateKeeper** โ€” highest authority. Can override any field including consumer-declared values. Used for sovereignty constraints, security mandates, and hard compliance rules. All overrides recorded in provenance.
-
-### 6.6 Assembly Process (Nine Steps)
-1. **Intent Capture** โ€” Request Layer stored as Intent State. No modification.
-2. **Layer Resolution** โ€” Processor identifies applicable layers by Resource Type and organizational context.
-3. **Layer Merge** โ€” Layers merged in precedence order. Each field records source layer UUID in provenance.
-4. **Request Layer Application** โ€” Consumer values applied. Overrides recorded in provenance.
-5. **Pre-Placement Policies** (`placement_phase: pre`) โ€” Transformation โ†’ Validation โ†’ GateKeeper. Produces placement constraints.
-6. **Placement Engine โ€” Placement Loop** โ€” Iterates candidate providers. Per candidate: Reserve Query (atomic: verify + metadata + hold) โ†’ Loop Policy Phase. Confirmed on first passing candidate. See Section 6.12.
-7. **Post-Placement Policies** (`placement_phase: post`) โ€” Transformation โ†’ Validation โ†’ GateKeeper. Has access to placement block including selected provider and all returned metadata.
-8. **Requested State Storage** โ€” Complete payload stored: resource fields + placement block + policy gap records + enrichment_status.
-9. **Provider Dispatch** โ€” Dispatched to selected provider via API Gateway. Hold confirmed by dispatch.
-
-### 6.7 Key Rules
-- Core Layers are type-agnostic โ€” applied to every request regardless of Resource Type
-- Service Layers must be type-scoped โ€” only applied when request Resource Type matches declared scope
-- A Service Layer without a declared type scope is invalid and must be rejected
-- Conflicting fields at same precedence: resolved by priority if declared; CONFLICT ERROR if not declared or equal
-- Conflicts detected at **ingestion time** โ€” not assembly time โ€” all active layers are pre-validated conflict-free
-- All layer modifications are recorded in field-level provenance
-
-### 6.8 Layer Identity โ€” Domain, Handle, Priority
-
-**Layer Domain** โ€” organizational home and authorization:
-
-| Domain | Scope | Can Override |
-|--------|-------|-------------|
-| `system` | DCM built-in | Nothing above |
-| `platform` | All tenants | tenant, service, provider |
-| `tenant` | Single tenant | service, provider within tenant |
-| `service` | Service Provider | provider |
-| `provider` | Catalog Item | Nothing above |
-
-**Layer Handle** โ€” `{domain}/{layer_type}/{name}` โ€” human-readable stable ID, unique in DCM, mirrors Git path.
-Example: `platform/core/security-cpu-limits` โ†’ `dcm-layers/platform/core/security-cpu-limits/v1.2.0.yaml`
-
-**Priority Schema** โ€” deterministic conflict resolution:
-- Format: `{int}.{int}.{int}...` โ€” unlimited depth, higher value = higher priority
-- `900.10` beats `800.10`; `900.20` beats `900.10`; no ceiling โ€” infinite upward insertability
-- Priority label: semantic context only, does not affect ordering
-- Reference taxonomy (advisory, not enforced): 900=Compliance, 800=Security, 700=Sovereignty, 600=Operations, 500=Platform, 400=Service, 300=Organization, 200=Site, 100=Custom
-- Organizations needing authority above all standard categories use `1000.*` or higher โ€” no renumbering required
-
-**Immutable ceiling model (Q51 resolved):**
-- `override: immutable` (default) โ€” protected by execution order; first policy to lock wins during this execution
-- `override: immutable` + `immutable_ceiling: absolute` โ€” protected against all future policies; attempted overrides are rejected and logged in audit
-
-### 6.9 Artifact Metadata Standard
-
-**Every DCM artifact** carries a universal artifact metadata block. Universal โ€” not optional, not per-artifact.
-
-**Five artifact statuses:**
-
-| Status | Applied? | Shadow? | Key Behavior |
-|--------|---------|---------|-------------|
-| `developing` | No | No | Dev mode only |
-| `proposed` | No | Yes (policies) | Shadow output captured for validation |
-| `active` | Yes | Yes (audit) | Live and governing |
-| `deprecated` | Yes (warning) | Yes | Replacement available |
-| `retired` | No | No | Terminal โ€” cannot be used |
-
-**Key fields:** `created_by` (audit โ€” who submitted), `owned_by` (accountability โ€” who gets notified), `created_via` (pr/api/migration/system โ€” audit quality), `modifications` (append-only history)
-
-**Contact โ€” two modes:** UUID+display_name when IdP registered; display_name+email in standalone/air-gapped mode. Both fully supported.
-
-**Proposed shadow (policies):** Shadow output captured per request as `proposed_evaluation_record` โ€” what the policy would have done, never applied. Feeds Validation Dashboard.
-
-**Notifications to `owned_by.notification_endpoint`:** layer conflict, deprecation, provider deregistered, policy violation, drift, high-impact shadow, approaching sunset.
-
-### 6.10 Field Override Control โ€” Two Categories
-
-**Structural Rules (Request Payload Processor โ€” non-overridable DCM System behavior):**
-- Layer immutability โ€” a published version cannot be modified
-- A child layer cannot remove a parent field โ€” only override its value
-- Layer precedence order is fixed โ€” Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request โ†’ Policy
-- Circular references and typeless Service Layers are always rejected
-
-**Business Rules (Policy Engine โ€” configurable):**
-The Policy Engine is the **sole authority** for field override control. It sets override control metadata on fields using the standard policy mechanism. Data layers and the Request Payload Processor never set override control.
-
-### 6.11 Field Override Control โ€” Three Levels
+### 6.1 Layers vs Policies โ€” The Clear Distinction
 
-**Design Principle: Simple by default, powerful when needed. Use only the level you need.**
+**Layers are data.** They carry static configuration, defaults, metadata, and context assembled into the request payload. A layer answers: "what values should these fields have?" Layers are passive โ€” they declare values but do not execute logic. They come first in assembly (Steps 1-4).
 
-**Level 1 โ€” No declaration (default)**
-Field is fully overridable by any actor. Zero configuration. Covers the majority of fields.
+**Policies are logic.** They evaluate the assembled payload and enforce rules, inject derived values, and make decisions. A policy answers: "given this data, is it valid? what should change? should this proceed?" Policies execute โ€” they run code. They come after layers (Steps 5-9).
 
-**Level 2 โ€” Simple declaration**
-Single `override` property โ€” sufficient for most governed fields:
-- `override: allow` โ€” explicit allow (same as default, self-documenting)
-- `override: constrained` โ€” any actor can override within `constraint_schema`
-- `override: immutable` โ€” no actor can override at any level
+**The flow is strictly unidirectional:**
+```
+Steps 1-4: LAYERS assembled โ†’ merged payload produced (data)
+Steps 5-9: POLICIES execute โ†’ payload evaluated and acted upon (logic)
+```
 
-**Level 3 โ€” Matrix declaration**
-Full per-actor permission matrix for fields requiring nuanced governance:
+**The decision rule:** Value that should appear in payload โ†’ Layer. Rule about whether payload is correct โ†’ Policy. Value derived by evaluating payload โ†’ Policy (Transformation type).
 
-```yaml
-override_matrix:
-  default: allow
-  inheritance: restrict_only
-  actors:
-    - actor: policy.global    # can_expand: true
-    - actor: policy.tenant    # can_expand: true (within global ceiling)
-    - actor: policy.user      # can_expand: false
-    - actor: consumer_request # can_expand: false
-    - actor: process_resource # permission: deny by default
-    - actor: provider         # can_expand: false
-    - actor: sre_override     # can_expand: false
-    - actor: admin_override   # can_expand: true (within scope)
-  trusted_grants:
-    - granted_to_uuid: 
-      actor_type: process_resource
-      permission: allow
-      granted_by_policy_uuid: 
-      expires: 
-```
+**What belongs in layers:** infrastructure defaults, organizational context, service configuration defaults, compliance metadata, business context labels.
 
-**Where declared:** Resource Type Specification sets the ceiling. Catalog Item can only restrict further. Policy Engine applies at runtime within those bounds.
+**What belongs in policies:** validation rules, compliance enforcement, derived value injection, placement constraints, approval gates, security enforcement.
 
-**Expansion rules:** `policy.global`, `policy.tenant`, `admin_override` can grant expansion. `policy.user`, `consumer_request`, `provider` can never expand. `process_resource` and `sre_override` denied by default โ€” require trusted grant.
+A policy that repeatedly injects the same static value into every request โ†’ that value belongs in a layer. A layer that contains conditional logic โ†’ that logic belongs in a policy.
 
-**Actor extensibility:** Custom actors default to `deny`, require explicit grants, follow universal versioning and deprecation model.
+### 6.2 Layer Domain Model (mirrors Policy domain)
 
-### 6.12 Placement Engine and Placement Loop
+| Domain | Authority | Can Override |
+|--------|----------|-------------|
+| `system` | DCM built-in โ€” highest | Nothing above system |
+| `platform` | Platform team | tenant, service, provider |
+| `tenant` | Tenant Admin | service, provider within Tenant |
+| `service` | Service Provider | provider |
+| `provider` | Provider owner | Nothing above provider |
+| `request` | Consumer โ€” lowest | Nothing above request |
 
-The **Placement Engine** is a distinct named control plane component โ€” a peer to the Policy Engine, not subordinate to it. It takes the policy-processed payload and placement constraints, builds a scored candidate list, and iterates until placement is confirmed or candidates are exhausted.
+### 6.3 The Full Layer Structure
 
-**Input:** assembled payload + placement constraints + provider registry + topology data
-**Output:** `selected_provider_uuid` + placement block written to Requested State
+Every layer carries: artifact_metadata (standard), domain + priority (authority), concern_tags (discoverability), compatibility (resource_types, versions, profile_constraints), activation_condition (Q23 โ€” conditional inclusion), fields with per-field override metadata (override: allow/constrained/immutable, basis_for_value), and usage context (description, applies_when, excludes_when, conflicts_with).
 
-**Placement loop per candidate:**
-```
-Reserve Query โ†’ Loop Policy Phase
-  confirmed/partial โ†’ policies โ†’ pass/warn โ†’ PLACEMENT CONFIRMED
-  insufficient/refused โ†’ next candidate
-  policy reject_candidate โ†’ release hold, next candidate
-  policy gatekeep โ†’ release hold, ABORT, reject request
-No candidates โ†’ on_exhaustion: reject | escalate | manual_placement
-```
+**activation_condition** โ€” layer only included if condition evaluates true during Step 2. Conditions reference: request fields, tenant attributes, resource type fields, resolved core layer fields, ingress fields. Enables role-specific layers, GPU-only layers, PCI-scope-only layers.
 
-**Reserve Query โ€” single atomic call (primary placement query):**
-- Verifies provider can satisfy placement constraints
-- Returns all available metadata in one response
-- Places a resource hold for `hold_ttl_seconds`
-- Response: `confirmed | partial | insufficient | refused`
-- `partial` = hold confirmed but some requested metadata unavailable
+### 6.4 Layer Groups
 
-**Non-hold queries (outside the loop):**
-`capacity_query | metadata_query | constraint_verification` โ€” informational, no side effects
+Layer Groups are `DCMGroup` with `group_class: layer_grouping` โ€” cohesive collections of related layers. Same model as Policy Groups. Enables discovery ("show me all PCI compliance layers"), composition, and governance.
 
-### 6.13 Policy Placement Phase and Required Context
+### 6.5 Consumer Layer Exclusion (Q21)
 
-**`placement_phase` on every policy:**
-- `pre` โ€” steps 5 (before provider known) โ€” default
-- `loop` โ€” step 6 (inside loop, evaluates reserve query response)
-- `post` โ€” step 7 (after placement confirmed, provider known)
-- `both` โ€” pre and post (not loop)
+Consumers declare `layer_exclusions` with mandatory reason. Excluded layers removed in Step 2, produce no fields, cannot satisfy validation requirements. GateKeeper policies may declare layers non-excludable (LAY-001).
 
-**`required_context` for missing metadata:**
-```yaml
-policy:
-  placement_phase: loop
-  required_context:
-    - field: placement.provider_metadata.sovereignty_certifications
-      if_absent: gatekeep     # block if this field is missing
-    - field: placement.provider_metadata.patch_level
-      if_absent: warn         # proceed with warning
-    - field: placement.topology.rack
-      if_absent: skip         # not applicable if absent
-```
+### 6.6 Service Layer Versioning (Q22)
 
-**Missing metadata behavior:**
-| Situation | Behavior | Audit Record |
-|-----------|---------|-------------|
-| Field absent, `required_context: gatekeep` | Release hold, abort loop, reject request | Policy rejection with missing field detail |
-| Field absent, `required_context: warn` | Record warning, proceed | Warning in Requested State |
-| Field absent, `required_context: skip` | Not evaluated | Skipped in provenance |
-| Field absent, no policy declares it | `implicit_approval` | `policy_gap_record` |
+Service Layers independently versioned. Providers declare semver compatibility constraints (`^1.0.0`, `~1.2`). Cache entries carry version โ€” invalidated when registered version changes (LAY-002).
 
-### 6.14 Policy Gap Record and Implicit Approval
+### 6.7 Conditional Layer Inclusion (Q23)
 
-When a field is absent and no active policy has declared `required_context` for it, the result is **implicit approval** โ€” not unknown, not unchecked, but explicitly recorded:
+`activation_condition` on layer evaluated in Step 2. False โ†’ layer excluded. Conditions reference request, tenant, resource type, core layer, and ingress fields. Recorded in assembly provenance (LAY-003).
 
-```yaml
-policy_gap_record:
-  request_uuid: 
-  field: patch_level
-  field_value: null
-  evaluation_result: implicit_approval
-  reason: "No active policy declared required_context for this field."
-  provider_uuid: 
-  recorded_at: 
-  resolution_expected: 
-```
+### 6.8 Dependency Layer Chains (Q24)
 
-Provider is expected to complete missing metadata in:
-1. **Realized payload** โ€” provider returns full metadata on realization
-2. **Discovery loop** โ€” periodic discovery fills remaining gaps
+Each service dependency has its own independent layer chain. Inherits parent's resolved placement fields (read-only). Does NOT inherit parent consumer declarations or type-specific layers. Layer exclusions declarable per-dependency (LAY-004).
 
-The realized entity's `enrichment_status: pending | partial | complete` tracks metadata completeness.
+### 6.9 The Nine-Step Assembly Process
 
+Step 1 (Intent Capture) โ†’ Step 2 (Layer Resolution โ€” with exclusions and activation_conditions) โ†’ Step 3 (Layer Merge โ€” priority ordering, field-level provenance) โ†’ Step 4 (Request Layer Application) โ†’ Step 5 (Pre-Placement Policies: Transformation โ†’ Validation โ†’ GateKeeper) โ†’ Step 6 (Placement Engine Loop: reserve query + loop policy phase per candidate) โ†’ Step 7 (Post-Placement Policies) โ†’ Step 8 (Requested State Storage) โ†’ Step 9 (Provider Dispatch)
 
----
+### 6.10 Layer System Policies
+- `LAY-001` โ€” Consumer layer exclusions with mandatory reason; GateKeeper can lock layers as non-excludable
+- `LAY-002` โ€” Service Layers independently versioned; semver compatibility on provider; cache invalidation on version change
+- `LAY-003` โ€” activation_condition on layers evaluated in Step 2; results recorded in provenance
+- `LAY-004` โ€” Each dependency has own layer chain; inherits parent resolved placement; no consumer declaration inheritance
 
 ## SECTION 7 โ€” RESOURCE TYPE HIERARCHY AND SERVICE CATALOG
 
@@ -1851,7 +1673,198 @@ AUTH-001 through AUTH-010 โ€” see doc 19. Key: AUTH-008 (no anonymous access in
 
 ---
 
-## SECTION 27 โ€” PERSONAS
+## SECTION 27 โ€” REGISTRY GOVERNANCE
+
+### 27.1 The Three-Tier Registry
+
+| Tier | Name | Maintained By | Contains |
+|------|------|--------------|---------|
+| 1 | DCM Core | DCM Project team | Universal types (Compute.VirtualMachine, Network.VLAN, etc.) |
+| 2 | Verified Community | Named community maintainers | Technology-specific types (OpenStack.HeatStack, KubeVirt.VirtualMachine) |
+| 3 | Organization | Deploying organization | Organization-specific/proprietary types |
+
+### 27.2 The Federated Registry Model
+Not centralized, not fully distributed โ€” federated:
+```
+DCM Project Registry (origin) โ†’ Organization Registry (local mirror) โ†’ Air-gapped Registry (offline copy)
+```
+Every DCM deployment has exactly one active **Registry Provider** (sub-type of Information Provider). Air-gapped deployments use signed bundles verified against the organization's public key โ€” no external connectivity required.
+
+### 27.3 PR-Based Proposal Workflow (Q9)
+Resource Type proposals are Pull Requests against the registry repository. Automated gates before review: schema validation, FQN conflict check, dependency resolution, breaking change detection, test case coverage. Shadow validation in `proposed` status is mandatory before `active` promotion.
+
+**Review periods by change type:** Revision=3 days, Minor/Tier2=7 days, Tier1=14 days, Breaking=21 days, Deprecation=30 days, Emergency=waived (7-day shadow minimum).
+
+### 27.4 Deprecation Lifecycle โ€” Default Policies (Q11)
+Deprecation lifecycle is governed by **default DCM system policies** (REG-DP-001 through REG-DP-007), overridable via standard policy priority. FSI/sovereign profiles lock sunset periods as immutable.
+
+| Policy | Default | Overridable? |
+|--------|---------|-------------|
+| `REG-DP-001` | 30-day notification before deprecation | Yes |
+| `REG-DP-002` | Sunset: Tier 1=P12M, Tier 2=P6M | Yes (locked in fsi/sovereign) |
+| `REG-DP-003` | Migration window: P90D after retirement | Yes |
+| `REG-DP-004` | Successor type required in deprecation notice | Yes |
+| `REG-DP-005` | Retired types reject new requests | **No โ€” structural** |
+| `REG-DP-006` | Existing realizations โ†’ DEPRECATED_RUNTIME | Yes |
+| `REG-DP-007` | Emergency migration floor: P30D | **No โ€” floor** |
+
+DEPRECATED_RUNTIME: eligible for modify/decommission; not eligible for rehydration using deprecated type; drift detection continues.
+
+### 27.5 Version Resolution Policy (Q12)
+Strictly enforced โ€” no silent resolution to different version. DCM never auto-upgrades across major versions.
+
+`version_policy` options: `exact` | `compatible` (^major) | `latest_minor` (~minor) | `latest`
+
+Profile defaults: minimal=latest, dev/standard/prod=compatible, fsi/sovereign=exact.
+
+### 27.6 Provider Tie-Breaking Hierarchy (Q13)
+When multiple providers satisfy all placement criteria equally:
+1. **Policy preference** โ€” Transformation policy injected preference_score or preferred_provider_uuid
+2. **Provider priority** โ€” numeric field on registration (default: 50; higher = preferred)
+3. **Tenant affinity** โ€” Policy Group declares preferred providers for resource types
+4. **Cost analysis** โ€” if Cost Analysis has current data AND cost is determinable and comparable (skip if not)
+5. **Least loaded** โ€” capacity utilization from reserve_query (skip if data unavailable)
+6. **Consistent hash** โ€” SHA-256(request_uuid + resource_type + sorted_candidate_uuids); deterministic, never round-robin
+
+Cost ranks above operational load because it is a business decision. 5% threshold โ€” candidates within 5% cost are treated as equal.
+
+### 27.7 Registry Provider โ€” Policy Governed (Q14)
+The Registry Provider is fully policy-governed. Policies act on registry sync, activation, bundle import, and version upgrades. Profile-appropriate registry policy groups activated by default:
+
+| Group | Profile | Behavior |
+|-------|---------|---------|
+| `system/group/registry-minimal` | minimal | Advisory; pull everything; warn only |
+| `system/group/registry-dev` | dev | Warn on unverified sources; Tier 1+2 |
+| `system/group/registry-standard` | standard | Block unverified; sovereignty filter |
+| `system/group/registry-prod` | prod | Vendor allowlist; audit all syncs; major version manual approval |
+| `system/group/registry-fsi` | fsi | Exact pinning; immutable sunset; dual-approval syncs |
+| `system/group/registry-sovereign` | sovereign | Signed bundles only; offline; no external connectivity |
+
+### 27.8 System Policies
+REG-001 through REG-007 and REG-DP-001 through REG-DP-007 โ€” see doc 20.
+
+---
+
+## SECTION 28 โ€” STORAGE ARCHITECTURE
+
+### 28.1 Git Repository Structure (Q79)
+Handle-based directory structure. Four repos: Intent, Requested, Layers, Policies. Minimal/dev may use monorepo; standard+ use separate repos. `main` is authoritative. Tenant isolation under `{tenant-uuid}` directories. DCM service account handles all Git reads/writes โ€” no direct Tenant Git access.
+
+```
+dcm-intent/tenants/{tenant-uuid}/requests/{request-uuid}/intent.yaml
+dcm-requested/tenants/{tenant-uuid}/requests/{request-uuid}/requested-payload.yaml
+dcm-layers/{domain}/{type}/{name}/v{Major}.{Minor}.{Revision}.yaml
+dcm-policies/{domain}/{type}/{name}/v{Major}.{Minor}.{Revision}.yaml
+```
+
+### 28.2 Multi-Region Replication (Q80)
+Declared capability on Storage Provider registration. Active Profile determines minimum requirements:
+- minimal/dev: 1 replica, no multi-region
+- standard: 3 replicas, strong/bounded consistency
+- prod/fsi/sovereign: 3-5 replicas, strong consistency, geo-replicated
+- sovereign: multi-region required but within sovereignty boundary only
+
+(STO-001)
+
+### 28.3 Storage Provider Failure Handling (Q81)
+Per store type โ€” policy-governed:
+- **Commit Log:** quorum unavailable โ†’ abort operation (no silent changes)
+- **GitOps Stores:** unavailable โ†’ queue writes locally (max size + max age); explicit reject on exhaustion
+- **Event Stream:** producer queues locally; consumer resumes from last offset on recovery
+- **Audit Store:** two-stage model โ€” accumulates in Commit Log; operations not blocked
+- **Search Index:** non-authoritative; degrades gracefully; full rebuild on recovery
+
+(STO-002)
+
+### 28.4 Search Index โ€” Separate Sub-Type (Q82)
+Separate Storage Provider sub-type โ€” distinct from GitOps stores. Non-authoritative, rebuildable from authoritative stores. Consistency lag declared (e.g., PT5M). API queries may specify `freshness: authoritative` to bypass index. (STO-003)
+
+### 28.5 Audit Store โ€” Specialized Sub-Type (Q83)
+Specialized Storage Provider sub-type โ€” NOT the same as Event Stream. Properties: append-only with immutability enforcement, hash chain integrity, reference-based retention tracking, compliance-grade multi-dimensional queries. Event Stream is the delivery channel; Audit Store is the compliance destination. (STO-004)
+
+---
+
+## SECTION 29 โ€” PROVIDER SOVEREIGNTY DECLARATIONS
+
+### 29.1 Obligation
+Every provider registration (Service, Information, Message Bus, Policy, Auth Provider) MUST include a `sovereignty_declaration` block. Contractual obligation โ€” not optional metadata.
+
+### 29.2 What Sovereignty Declaration Covers
+- **operating_jurisdictions** โ€” countries and legal jurisdictions where provider physically operates
+- **legal_frameworks** โ€” applicable frameworks (GDPR, HIPAA, FedRAMP, ITAR, etc.)
+- **data_residency_guarantee** โ€” data never leaves declared jurisdictions (true/false)
+- **data_transit_jurisdictions** โ€” jurisdictions data transits through during operations
+- **external_dependencies** โ€” external connectivity requirements, air_gap_capable flag, external services with data sharing details
+- **sub_processors** โ€” third-party sub-processors with jurisdiction and data handled
+- **government_access_risk** โ€” which governments can legally compel access
+- **certifications** โ€” current certifications with validity periods (ISO-27001, SOC2, PCI-DSS, FedRAMP)
+- **audit_rights** โ€” customer audit rights and notice periods
+- **change_notification** โ€” mandatory notification events and SLA (e.g., PT24H)
+
+### 29.3 Change Notification and DCM Response
+Provider MUST notify DCM when any sovereignty data changes. DCM treats sovereignty changes as discovered drift โ†’ Policy Engine re-evaluation:
+- **No violations:** update record, emit webhook event, notify Tenants (informational)
+- **Violations found:** for each affected resource, policy declares action:
+  - `notify_only` โ€” inform Tenant; no automatic action
+  - `pause` โ€” suspend resource; Tenant must act
+  - `migrate` โ€” Provider-Portable Rehydration to compliant provider (sequential)
+  - `emergency_migrate` โ€” parallel provisioning before decommission
+
+Sovereignty violation record created in Audit Store. Notifications: Tenant owner, platform admin, data_protection_officer.
+
+### 29.4 Auto-Migration
+Policy declares `migrate` or `emergency_migrate` โ†’ DCM uses Provider-Portable Rehydration. Non-compliant provider excluded from placement candidate set. Full audit trail linking violation record to migration request. (SOV-001 through SOV-005)
+
+---
+
+## SECTION 30 โ€” GIT PR INGRESS
+
+### 30.1 Concept
+DCM supports `git_pr_merge` and `git_pr_open` as ingress surfaces. Teams submit standard DCM resource definition YAML as Pull Requests. DCM's Git Request Watcher monitors designated repositories.
+
+### 30.2 Git Actor Identity Resolution
+**DCM trusts the Git server's authentication assertion โ€” not user-declared Git configuration.** Git `user.email` self-declaration is ignored โ€” spoofing vector.
+
+Resolution methods (all go through registered Auth Provider):
+- `oidc_subject_lookup` โ€” Git server OAuth subject โ†’ OIDC Auth Provider โ†’ DCM actor
+- `ldap_username_lookup` โ€” Git server username โ†’ LDAP/AD Auth Provider โ†’ DCM actor
+- `ssh_key_fingerprint` โ€” key fingerprint โ†’ DCM SSH key registry โ†’ DCM actor
+- `webhook_service_account` โ€” CI/CD service account โ†’ registered webhook actor
+
+**The resolved actor has IDENTICAL roles, groups, and tenant scope to the same user logging in via web UI.** Git PR ingress does not grant different permissions than any other surface. Same Auth Provider, same group mappings, same tenant scope enforcement.
+
+**Unresolvable identity โ†’ explicit PR rejection comment** with actionable guidance. Never silently ignored.
+
+### 30.3 PR Lifecycle
+1. PR opened โ†’ DCM resolves author โ†’ Auth Provider โ†’ shadow policy evaluation posted as PR comments
+2. Human review + Git branch protection approvals
+3. PR merged โ†’ actor re-verified at merge time (not assumed from PR open) โ†’ full nine-step assembly โ†’ realization result posted as PR comment
+4. Realized state committed to `realized/` directory (optional)
+
+### 30.4 The git_context in ingress block
+```yaml
+ingress:
+  surface: git_pr_merge
+  actor: 
+  git_context:
+    repository / pr_number / pr_url / merge_commit
+    pr_author / pr_reviewers / pr_approved_by
+    # pr_approved_by: DCM resolves reviewer Git identities via same Auth Provider
+```
+
+### 30.5 Policy Use Cases
+- Require specific approvers in pr_approved_by before processing
+- Require MFA for Git PR merges in prod Tenants
+- Restrict resource types submittable via Git PR
+- Require actor to be in authorized Tenant group
+- Post shadow evaluation results as PR comments
+
+### 30.6 System Policies
+GIT-001 through GIT-008 โ€” see doc 18. AUTH-011 โ€” Git identity resolution uses registered Auth Provider; same role/group/tenant scope as any other ingress.
+
+---
+
+## SECTION 31 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -1868,7 +1881,7 @@ AUTH-001 through AUTH-010 โ€” see doc 19. Key: AUTH-008 (no anonymous access in
 
 ---
 
-## SECTION 28 โ€” TERMINOLOGY GLOSSARY
+## SECTION 32 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2047,7 +2060,7 @@ AUTH-001 through AUTH-010 โ€” see doc 19. Key: AUTH-008 (no anonymous access in
 
 ---
 
-## SECTION 29 โ€” OPEN QUESTIONS
+## SECTION 33 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2144,7 +2157,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 30 โ€” DOCUMENTATION STRUCTURE
+## SECTION 34 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2192,7 +2205,7 @@ content/
 
 ---
 
-## SECTION 31 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 35 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2241,6 +2254,18 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+89. **Provider sovereignty is a contractual obligation** โ€” every provider registration requires sovereignty_declaration; changes must be notified within declared SLA; DCM treats sovereignty changes as drift and re-evaluates placement; auto-migration available via Provider-Portable Rehydration
+90. **Git PR ingress actors resolve through the same Auth Provider as all other users** โ€” DCM trusts the Git server's authentication assertion; git config user.email is ignored (spoofing vector); the resolved actor has IDENTICAL roles/groups/tenant scope to web UI login for the same user; unresolvable identities are always rejected with an actionable PR comment
+91. **Storage Provider sub-types are distinct** โ€” Search Index (non-authoritative, rebuildable, consistency lag declared) and Audit Store (append-only, hash chain, reference-based retention, compliance queries) are separate sub-types; never treat them as interchangeable
+92. **GitOps stores use handle-based directory structure** โ€” deterministic path from artifact identity; main is authoritative; monorepo acceptable for minimal/dev; separate repos for standard+
+85. **Layers are data, policies are logic โ€” never conflate them** โ€” if a policy repeatedly injects the same static value, that value belongs in a layer; if a layer contains conditional evaluation logic, that logic belongs in a policy; the flow is strictly unidirectional (layers Steps 1-4, policies Steps 5-9)
+86. **Layer domains mirror policy domains** โ€” system > platform > tenant > service > provider > request; same authority model, same override precedence; lower cannot override higher
+87. **Layer Groups use DCMGroup group_class: layer_grouping** โ€” same universal group model as policy_collection; enables discovery, composition, and governance of related layers
+88. **activation_condition enables conditional layer inclusion** โ€” evaluated in Step 2; can reference request fields, tenant attributes, resource type, resolved core layer fields, and ingress fields; condition false = layer excluded = recorded in provenance
+81. **Registry governance is PR-based and policy-governed** โ€” proposals are Pull Requests with automated validation gates; shadow validation in proposed status is mandatory before active; the Registry Provider is fully policy-governed with profile-appropriate policy groups
+82. **Deprecation defaults are policies โ€” not hard-coded values** โ€” REG-DP-001 through REG-DP-007 are overridable via standard priority; FSI/sovereign profiles lock sunset periods as immutable; REG-DP-005 (retired rejects new requests) is structural and never overridable
+83. **Version constraints are strictly enforced โ€” no silent upgrades** โ€” DCM never auto-upgrades across major versions; version_policy governs flexibility within that constraint; profile sets the default policy
+84. **Cost analysis ranks above least-loaded in tie-breaking** โ€” cost is a business decision; but only if Cost Analysis has current data and cost is determinable; skip silently if not; consistent hash is always the final deterministic tiebreaker
 75. **Eight provider types โ€” not five** โ€” Message Bus Provider (6), Credential Provider (7), and Auth Provider (8) complete the ecosystem; all follow the same base contract
 76. **The ingress block is the policy surface for all access control** โ€” every request carries surface, protocol, authenticated_via, actor.roles, actor.auth_provider_type, mfa_verified, and external_identity claims; GateKeeper policies act on all of these
 77. **No anonymous access in any profile** โ€” minimal profile uses static API key (30 seconds to set up); the authentication ladder is about setup effort, not whether auth exists; AUTH-008 is non-negotiable
diff --git a/content/docs/architecture/data-model/_index.md b/content/docs/architecture/data-model/_index.md
index 19ef264..3dcd177 100644
--- a/content/docs/architecture/data-model/_index.md
+++ b/content/docs/architecture/data-model/_index.md
@@ -32,5 +32,6 @@ realization, operation, and decommission.
   {{< card link="deployment-redundancy" title="Deployment and Redundancy" icon="server" subtitle="Redundant by default. Everything containerized. Profile-governed replicas. Self-hosting. Quorum writes. RED-001 through RED-010." >}}
   {{< card link="webhooks-messaging" title="Webhooks and Messaging" icon="arrows-expand" subtitle="Inbound and outbound webhooks. Message Bus Provider. Universal ingress/egress actor model. Credential Provider. WHK and ING policies." >}}
   {{< card link="auth-providers" title="Auth Providers" icon="lock-closed" subtitle="Eight provider types. Auth ladder from API key to air-gapped OIDC. LDAP, FreeIPA, AD, OIDC, mTLS. No anonymous access. AUTH policies." >}}
+  {{< card link="registry-governance" title="Registry Governance" icon="collection" subtitle="Three-tier registry. PR-based proposals. Default deprecation policies. Cost-aware placement. Federated with air-gap signed bundles." >}}
   {{< card link="universal-groups" title="Universal Group Model" icon="view-grid" subtitle="Unified grouping: Tenants, Resource Groups, Policy Groups as one model. Composite groups, nested and federated Tenants, permanent membership history." >}}
 {{< /cards >}}
diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md
index 297c397..a721882 100644
--- a/content/docs/architecture/data-model/layering-and-versioning.md
+++ b/content/docs/architecture/data-model/layering-and-versioning.md
@@ -31,6 +31,76 @@ The layering model enables:
 
 ---
 
+## 1a. Layers vs Policies โ€” The Clear Distinction
+
+Layers and policies are the two foundational mechanisms of DCM's assembly process. They are complementary and distinct โ€” understanding the difference is critical to using DCM correctly.
+
+### Layers Are Data
+
+A layer is a **declarative, immutable, versioned unit of data**. It carries static configuration values, organizational defaults, compliance metadata, and contextual information. A layer answers the question: **"what values should these fields have?"**
+
+Layers are **passive** โ€” they declare values but do not execute logic. They do not evaluate the payload, make branching decisions, or enforce rules. The assembly process merges them in priority order. Layers come first.
+
+**What belongs in a layer:**
+- Infrastructure defaults (DNS servers, NTP servers, MTU values)
+- Organizational context (data center location, rack assignment, environment tier)
+- Service-specific configuration defaults (VM sizing defaults, storage class preferences)
+- Compliance metadata (data classification labels, retention tags, jurisdiction markers)
+- Provider-specific configuration (provider default settings, tooling parameters)
+- Business context (cost center defaults, environment labels, team tags)
+
+### Policies Are Logic
+
+A policy is an **executable rule** that evaluates the assembled payload and takes action. A policy answers the question: **"given this data, is it valid? what should change? should this proceed?"**
+
+Policies **execute** โ€” they run logic (OPA Rego, DCM native rules, Mode 4 black box calls). They can read every layer-provided value, validate correctness, transform fields, inject derived values, and gate requests. Policies come after layers โ€” they operate on the assembled result.
+
+**What belongs in a policy:**
+- Validation rules ("this field must be present and within these bounds")
+- Compliance enforcement ("all resources must have a classification label")
+- Derived value injection ("inject cost center from OIDC claims")
+- Placement constraints ("must be in EU sovereignty zone")
+- Approval gates ("resources above X size require manager approval")
+- Security enforcement ("encryption must be enabled โ€” if not, enable it or reject")
+
+### The Flow Is Strictly Unidirectional
+
+```
+Steps 1-4: LAYERS assembled โ†’ merged payload produced
+  โ”‚  Layers contribute field values
+  โ”‚  Higher priority layers override lower priority
+  โ”‚  Immutable fields locked at this stage
+  โ”‚
+  โ–ผ
+Steps 5-9: POLICIES execute โ†’ payload evaluated and acted upon
+  โ”‚  Policies read assembled payload
+  โ”‚  Transformation: modify/inject derived fields
+  โ”‚  Validation: verify correctness
+  โ”‚  GateKeeper: approve or reject
+  โ”‚
+  โ–ผ
+Provider-ready payload dispatched
+```
+
+Policies cannot set static configuration โ€” that is a layer's job. A policy that finds itself repeatedly injecting the same static value into every request should be refactored: that value belongs in a layer.
+
+Layers cannot enforce rules โ€” that is a policy's job. A layer that contains conditional logic or rule evaluation is being misused โ€” that logic belongs in a policy.
+
+### The Decision Rule for Practitioners
+
+> "Is this a **value** that should appear in the payload? โ†’ **Layer**  
+> Is this a **rule** about whether the payload is correct? โ†’ **Policy**  
+> Is this a **value derived by evaluating** the payload? โ†’ **Policy** (Transformation type)"
+
+### The Analogy
+
+- Layers are the **ingredients** โ€” pre-measured, pre-arranged, versioned
+- Policies are the **chef** โ€” decides what to do with the ingredients, can add derived elements, makes judgment calls, can reject the dish entirely
+
+Both are necessary. Neither replaces the other.
+
+---
+
 ## 2. What is a Layer?
 
 A Layer is a **declarative, immutable, versioned unit of data** that contributes some or all of its fields to a merged payload. Layers do not execute โ€” they declare. The assembly process is what merges them.
@@ -206,23 +276,139 @@ Every layer has a formal identity model with three components that together make
 
 ### 4.1 Layer Domain
 
-The **Layer Domain** is the organizational and architectural home of a layer. It declares ownership, storage location, and authorization scope โ€” who can create and modify this layer, and which other layers it can override.
+The **Layer Domain** mirrors the Policy domain model exactly. It declares ownership, storage location, and authorization scope. The same domain hierarchy, the same authority model, the same override precedence.
 
 | Domain | Meaning | Authorization | Can Override |
 |--------|---------|--------------|-------------|
-| `system` | DCM built-in layers, shipped with DCM | DCM maintainers only | Nothing above system |
-| `platform` | Platform team layers, apply across all tenants | Platform team | tenant, service, provider |
-| `tenant` | Tenant-specific layers | Tenant Admin | service, provider within tenant |
+| `system` | DCM built-in layers โ€” ship with DCM | DCM maintainers only | Nothing above system |
+| `platform` | Platform team layers โ€” apply across all Tenants | Platform team | tenant, service, provider |
+| `tenant` | Tenant-specific layers โ€” scoped to one Tenant | Tenant Admin | service, provider within Tenant |
 | `service` | Service Provider contributed layers | Service Provider owner | provider |
 | `provider` | Provider Catalog Item layers | Provider owner | Nothing above provider |
+| `request` | Consumer-declared values in the request itself | Consumer | Nothing above request โ€” lowest authority |
 
 A lower-domain layer cannot override a higher-domain layer. A `tenant` layer cannot override a `platform` layer. This is enforced at ingestion โ€” the conflict detection pipeline checks domain authority before allowing a merge.
 
-### 4.2 Layer Handle
+**Domain mirrors policy authority:** Just as system-domain policies have highest authority in the Policy Engine, system-domain layers have highest authority in the assembly process. The same mental model applies to both.
+
+### 4.2 Layer Groups โ€” DCMGroup with group_class: layer_grouping
 
-The **Layer Handle** is the human-readable, stable identifier for a layer within DCM. It complements the UUID (machine-meaningful) with a reference that humans can use in conversation, documentation, policy declarations, and audit records.
+Just as Policy Groups organize policies into cohesive concern-based collections, **Layer Groups** organize layers. A Layer Group is a `DCMGroup` with `group_class: layer_grouping` โ€” a versioned, audited, GitOps-managed collection of related layers.
 
-**Format:** `{domain}/{layer_type}/{name}`
+Layer Groups enable:
+- **Discovery** โ€” "show me all layers related to PCI compliance"
+- **Composition** โ€” include a group in a profile rather than listing individual layers
+- **Governance** โ€” activate or deactivate a concern's worth of layers in one operation
+
+```yaml
+# A Layer Group โ€” DCMGroup with group_class: layer_grouping
+dcm_group:
+  artifact_metadata:
+    uuid: 
+    handle: "platform/layer-groups/pci-network-standards"
+    version: "1.0.0"
+    status: active
+  group_class: layer_grouping
+  concern_tags: [pci-dss, networking, standards]
+  members:
+    - member_uuid: 
+      member_type: layer
+      member_role: network_segmentation_defaults
+    - member_uuid: 
+      member_type: layer
+      member_role: firewall_baseline
+    - member_uuid: 
+      member_type: layer
+      member_role: tls_minimum_version
+```
+
+### 4.3 The Full Layer Structure
+
+Every layer carries: identity, domain and authority, compatibility metadata, per-field override metadata, and usage context. This mirrors the richness of a Policy registration.
+
+```yaml
+layer:
+  artifact_metadata:
+    uuid: 
+    handle: "platform/core/default-dns-config"
+    version: "1.2.0"
+    status: active
+    owned_by:
+      display_name: "Platform Infrastructure Team"
+      notification_endpoint: 
+    created_via: pr   # pr | api | migration | system
+
+  # DOMAIN AND AUTHORITY
+  domain: platform
+  priority:
+    value: "500.20.0"
+    label: "platform.networking.dns"
+    category: platform
+    rationale: "Platform DNS infrastructure โ€” primary and secondary resolvers"
+
+  # CONCERN TAGS โ€” for discoverability and grouping
+  concern_tags: [networking, dns, platform-defaults]
+
+  # COMPATIBILITY METADATA โ€” what this layer applies to
+  compatibility:
+    resource_types: [Compute.VirtualMachine, Compute.Container]
+    resource_type_versions: "^1.0.0"
+    provider_types: []              # empty = all providers
+    profile_constraints: []         # empty = all profiles; or: [standard, prod, fsi]
+    domains_applicable: [platform, tenant, service, provider]  # which domains may use this
+
+  # CONDITIONAL INCLUSION (Q23) โ€” activation condition
+  activation_condition:
+    # Layer only included if this condition evaluates true during Step 2 (Layer Resolution)
+    field: tenant.tags
+    operator: not_contains          # equals|not_equals|exists|not_exists|contains|in|not_in
+    value: custom-dns
+    # Compound conditions:
+    # conditions:
+    #   operator: and   # and | or
+    #   rules:
+    #     - field: request.gpu_requested
+    #       operator: equals
+    #       value: true
+    #     - field: ingress.actor.roles
+    #       operator: contains
+    #       value: developer
+
+  # FIELDS โ€” with per-field override metadata
+  fields:
+    dns_servers:
+      value: [10.0.0.53, 10.0.0.54]
+      metadata:
+        override: allow             # allow | constrained | immutable
+        basis_for_value: "Platform DNS infrastructure โ€” primary and secondary"
+        provenance_note: "Set by platform infrastructure team per INFRA-2024-089"
+    dns_search_domain:
+      value: corp.example.com
+      metadata:
+        override: immutable         # lower layers cannot override this field
+        locked_by_policy_uuid: 
+        basis_for_value: "Corporate domain โ€” cannot be customized per SECURITY-2024-034"
+
+  # USAGE CONTEXT โ€” human documentation embedded in the artifact
+  usage:
+    description: "Default DNS configuration for all platform VMs and containers"
+    applies_when: "All requests unless consumer declares layer exclusion or tenant has custom-dns tag"
+    excludes_when: "Tenant has custom_dns tag; consumer declares explicit layer exclusion"
+    supersedes: []                  # handles of layers this replaces
+    conflicts_with: []              # handles of layers this conflicts with โ€” detected at ingestion
+
+  # SOURCE OF TRUTH
+  scm_location:
+    repository: https://git.corp.example.com/dcm-layers
+    path: platform/core/default-dns-config/v1.2.0.yaml
+    commit: 
+```
+
+### 4.4 Layer Handle
+
+The **Layer Handle** is the human-readable, stable identifier for a layer within DCM.
+
+**Format:** `{domain}/{concern_or_type}/{name}`
 
 **Examples:**
 ```
@@ -234,24 +420,18 @@ service/service/kubevirt-vm-defaults
 provider/service/cloudnativepg-database-config
 ```
 
-**Rules:**
-- Unique within DCM โ€” enforced at ingestion
-- Stable across versions โ€” the handle does not change when a new version is published
-- URL-safe characters only โ€” lowercase, hyphens, forward slashes
-- The Git file path mirrors the handle structure exactly
-
 **Git path from handle:**
 ```
-{layer_store_root}/{domain}/{layer_type}/{name}/v{Major}.{Minor}.{Revision}.yaml
+{layer_store_root}/{domain}/{concern_or_type}/{name}/v{Major}.{Minor}.{Revision}.yaml
 
 # Example:
 dcm-layers/platform/core/security-cpu-limits/v1.2.0.yaml
 dcm-layers/tenant/{tenant-uuid}/service/payments-vm-standards/v1.0.0.yaml
 ```
 
-### 4.3 Priority Schema
+### 4.5 Priority Schema
 
-The **Priority Schema** is the deterministic ordering mechanism for resolving conflicts between layers of the same type and scope. It uses a hierarchical dotted-notation system that supports infinite differentiation โ€” new priority values can always be inserted between any two existing values.
+The **Priority Schema** is the deterministic ordering mechanism for resolving conflicts between layers of the same type and scope.
 
 **Format:** `{integer}.{integer}.{integer}...` โ€” unlimited depth
 
@@ -260,19 +440,12 @@ The **Priority Schema** is the deterministic ordering mechanism for resolving co
 ```
 900.10    beats    800.10    (900 > 800 at segment 1)
 900.20    beats    900.10    (20 > 10 at segment 2)
-900.10.5  beats    900.10    (longer path with matching prefix โ€” 5 at segment 3 > nothing)
+900.10.5  beats    900.10    (longer path with matching prefix)
 900.10.10 beats    900.10.5  (10 > 5 at segment 3)
 ```
 
-**Infinite insertability โ€” symmetric in both directions:**
-Between `900.10` and `900.20` insert `900.15`. Between existing values or above the current maximum โ€” there is no ceiling. You can always go higher. This is the key advantage over a lower-wins model which would have a hard floor at 1.
-
-**Priority Label:** Semantic context for the numeric value โ€” human-readable, does not affect ordering.
-
 **Reference Priority Taxonomy (advisory โ€” not enforced by DCM):**
 
-DCM ships a reference taxonomy as documentation and starter templates. Organizations adopt, adapt, or ignore it โ€” DCM resolves conflicts purely by numeric comparison. The taxonomy is a suggested convention, not a system constraint.
-
 | Suggested Range | Category | Rationale |
 |-----------------|----------|-----------|
 | `900.*` | Compliance | Regulatory mandates โ€” highest authority |
@@ -285,20 +458,7 @@ DCM ships a reference taxonomy as documentation and starter templates. Organizat
 | `200.*` | Site | Location-specific overrides |
 | `100.*` | Custom | Implementor-defined โ€” lowest standard category |
 
-Higher number = higher priority. An organization that needs a mandate above all standard categories simply uses `1000.*` or above โ€” no renumbering required. An organization that prioritizes sovereignty above compliance would simply swap their `700.*` and `900.*` ranges.
-
-**Priority in a layer definition:**
-
-```yaml
-priority:
-  value: "800.30.10"
-  label: "security.container.cpu_limits"
-  category: security
-  rationale: >
-    CPU limit enforcement for container workloads per
-    CISO mandate SEC-2024-047. Overrides platform defaults
-    to ensure no container can exceed approved limits.
-```
+Higher number = higher priority. Organizations adopt, adapt, or ignore this taxonomy โ€” DCM resolves conflicts purely by numeric comparison.
 
 ---
 
@@ -1421,16 +1581,165 @@ The layer chain of a Realized Entity is always traceable โ€” given a Realized St
 
 ---
 
-## 13. Open Questions
+## 13. Layer Gaps โ€” Q21 through Q24
+
+### 13.1 Consumer Layer Exclusion (Q21)
+
+Consumers may explicitly exclude specific layers from their request. Each exclusion carries a mandatory human-readable reason recorded in provenance and the audit trail.
+
+```yaml
+request:
+  resource_type: Compute.VirtualMachine
+  layer_exclusions:
+    - layer_handle: "platform/networking/default-dns-config"
+      reason: "This VM uses custom DNS โ€” default config conflicts with application requirements"
+    - layer_uuid: 
+      reason: "Dev environment โ€” monitoring layer not required"
+```
+
+**Exclusion mechanics:**
+- Excluded layers are removed from the candidate set during **Step 2 (Layer Resolution)** before priority ordering
+- Excluded layers produce no fields in the assembled payload
+- If a validation policy requires a field that would have been injected by an excluded layer, the validation fails with a clear message identifying the excluded layer
+- Exclusion is different from override โ€” exclusion removes the entire layer; override changes specific field values
+
+**Policy enforcement:** GateKeeper policies may declare specific layers non-excludable:
+
+```yaml
+policy:
+  type: gatekeeper
+  rule: >
+    If request.layer_exclusions CONTAINS layer.concern_tags CONTAINS "security-baseline"
+    THEN gatekeep: "Security baseline layers cannot be excluded"
+  immutable_ceiling: absolute
+```
+
+### 13.2 Service Layer Versioning (Q22)
+
+Service Layers are **independently versioned artifacts** โ€” not coupled to Service Provider versions. Service Providers declare semver-compatible version constraints for the layers they use.
+
+```yaml
+# Service Provider registration โ€” layer compatibility declarations
+provider_registration:
+  layer_compatibility:
+    - layer_handle: "layers/vm-compute-defaults"
+      compatible_versions: "^1.0.0"    # any 1.x version
+    - layer_handle: "layers/vm-networking-config"
+      compatible_versions: "~1.2"      # any 1.2.x revision
+```
+
+**Version lifecycle:** Service Layers follow the standard five-status artifact lifecycle. A deprecated Service Layer continues to work for existing realizations until retired. If a provider bumps to a new major version and updates its compatibility declaration, the old layer version is no longer used for new requests via that provider but continues to work for existing realizations.
+
+**Cache invalidation:** Service Layer Cache entries carry the layer version. When the registered layer version increments, the cache entry is invalidated and refreshed before the next assembly.
+
+### 13.3 Conditional Layer Inclusion (Q23)
+
+Layers may declare an `activation_condition` โ€” a field comparison evaluated during **Step 2 (Layer Resolution)**. Layers whose condition evaluates false are excluded from the candidate set.
+
+```yaml
+layer:
+  handle: "platform/compute/gpu-config"
+  activation_condition:
+    field: request.gpu_requested
+    operator: equals
+    value: true
+```
+
+**Compound conditions:**
+```yaml
+activation_condition:
+  conditions:
+    operator: and   # and | or
+    rules:
+      - field: request.gpu_requested
+        operator: equals
+        value: true
+      - field: request.resource_class
+        operator: in
+        value: [ml-training, gpu-compute]
+```
+
+**Condition field scope** โ€” activation conditions may reference:
+- Request fields (`request.gpu_requested`)
+- Tenant attributes (`tenant.tags`, `tenant.profile`)
+- Resource type fields (`resource_type.version`)
+- Core Layer fields already resolved in Step 1 (`core_layers.location_region`)
+- Ingress fields (`ingress.actor.roles`) โ€” enabling role-specific layers
+
+**Condition vs consumer exclusion:** Conditional inclusion is declared by the layer author and evaluated automatically. Consumer exclusion (Q21) is declared at request time by the consumer. Both result in the layer being absent from the candidate set โ€” but for different reasons, recorded differently in provenance.
+
+### 13.4 Layer Chain and Service Dependencies (Q24)
+
+Each service dependency executes its **own independent layer chain** during assembly. Dependencies do not share the parent request's layer chain.
+
+**Dependencies inherit from parent (read-only context):**
+- Tenant UUID and sovereignty context
+- Parent's resolved placement fields (declared by Resource Type Specification as `propagated_to_dependencies`)
+- Parent's resolved identity fields (hostname, etc.)
+- Active Profile
+
+**Dependencies do NOT inherit:**
+- Parent consumer declarations
+- Resource-type-specific layers (each type has its own)
+- Provider-specific layers (each provider has its own)
+
+**Dependency assembly flow:**
+```
+Parent request: Compute.VirtualMachine
+  โ”‚
+  โ–ผ  Steps 1-4: Parent layer chain โ†’ parent_assembled_payload
+  โ”‚
+  โ–ผ  Step 5: Pre-placement policies on parent
+  โ”‚
+  โ–ผ  Step 6: Placement loop โ€” parent provider selected
+  โ”‚           Also identifies required dependency providers
+  โ”‚
+  โ–ผ  For each dependency (parallel where ordering allows):
+  โ”‚  โ”œโ”€โ”€ Network.IPAddress โ†’ own layer chain (Steps 1-4)
+  โ”‚  โ”‚     Context: inherits parent resolved placement fields
+  โ”‚  โ”œโ”€โ”€ Network.Port โ†’ own layer chain (Steps 1-4)
+  โ”‚  โ”‚     Context: inherits parent + IP resolution result
+  โ”‚  โ””โ”€โ”€ DNS.Record โ†’ own layer chain (Steps 1-4)
+  โ”‚         Context: inherits parent + IP + Port results
+  โ”‚
+  โ–ผ  Steps 7-9: Post-placement, storage, dispatch
+       Parent + all dependency payloads dispatched together
+```
+
+**Layer exclusions on dependencies** โ€” consumers may declare per-dependency exclusions:
+```yaml
+request:
+  resource_type: Compute.VirtualMachine
+  dependencies:
+    - resource_type: Network.IPAddress
+      layer_exclusions:
+        - layer_handle: "layers/ip-default-ttl-config"
+          reason: "Custom TTL required โ€” excluding default"
+```
+
+---
+
+## 13a. Layer System Policies
+
+| Policy | Rule |
+|--------|------|
+| `LAY-001` | Consumers may declare `layer_exclusions` in their request. Each exclusion must carry a human-readable reason recorded in provenance. GateKeeper policies may declare specific layers non-excludable. Excluded layers produce no fields in the assembled payload. |
+| `LAY-002` | Service Layers are independently versioned artifacts. Service Providers declare layer compatibility using semver constraints. Service Layer Cache entries carry the layer version and are invalidated when the registered version changes. |
+| `LAY-003` | Service Layers may declare `activation_condition` evaluated during Step 2 (Layer Resolution). Layers whose conditions evaluate false are excluded from the candidate set. Condition evaluation results are recorded in the assembly provenance. Conditions may reference request fields, tenant attributes, resource type fields, resolved core layer fields, and ingress fields. |
+| `LAY-004` | Each service dependency executes its own independent layer chain during assembly. Dependencies inherit the parent's resolved placement and identity fields as declared by the Resource Type Specification. Dependencies do not inherit parent consumer declarations, resource-type-specific layers, or provider-specific layers. Layer exclusions may be declared per-dependency. |
+
+---
+
+## 14. Open Questions
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
 | 1 | How are conflicting Service Layers at the same precedence level resolved? | Assembly determinism | โœ… Resolved โ€” priority schema + conflict detection at ingestion |
 | 2 | Should Core Layers be ordered within their precedence level? | Merge determinism | โœ… Resolved โ€” priority schema provides deterministic ordering |
-| 3 | Can a consumer explicitly exclude a layer from their request? | Consumer control vs. standardization | โ“ Unresolved |
-| 4 | How are Service Layers registered and versioned relative to Service Provider registration? | Provider contract | โ“ Unresolved |
-| 5 | Should assembly support conditional layer inclusion? | Assembly flexibility | โ“ Unresolved |
-| 6 | How does the layer chain interact with service dependencies? | Dependency model | โ“ Unresolved |
+| 3 | Can a consumer explicitly exclude a layer from their request? | Consumer control vs. standardization | โœ… Resolved โ€” layer_exclusions with mandatory reason; GateKeeper can lock layers as non-excludable (LAY-001) |
+| 4 | How are Service Layers registered and versioned relative to Service Provider registration? | Provider contract | โœ… Resolved โ€” independently versioned; provider declares semver compatibility; cache invalidation on version change (LAY-002) |
+| 5 | Should assembly support conditional layer inclusion? | Assembly flexibility | โœ… Resolved โ€” activation_condition on layers; evaluated in Step 2; references request, tenant, resource type, core layer, and ingress fields (LAY-003) |
+| 6 | How does the layer chain interact with service dependencies? | Dependency model | โœ… Resolved โ€” each dependency has its own layer chain; inherits parent resolved placement context; no consumer declaration inheritance (LAY-004) |
 | 7 | Should `override_preference` be declarable in layer definitions as a hint to the Policy Engine? | Override control | โ“ Unresolved |
 | 8 | When `override_preference: immutable` is set by a Global policy, can a higher-priority Global policy still override it? | Override control precedence | โ“ Unresolved |
 | 9 | Should the `constraint_schema` on a constrained field be visible to consumers in the Service Catalog UI? | Consumer experience | โ“ Unresolved |
diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md
index c22e8b2..6143b73 100644
--- a/content/docs/architecture/data-model/storage-providers.md
+++ b/content/docs/architecture/data-model/storage-providers.md
@@ -12,7 +12,7 @@ weight: 10
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Four States](../four-states/) | [Audit, Provenance, and Observability](../audit-provenance-observability/) | [Information Providers](../information-providers/)
+**Related Documents:** [Four States](02-four-states.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Information Providers](10-information-providers.md)
 
 ---
 
@@ -296,10 +296,386 @@ DCM may maintain internal performance caches between components and stores. Thes
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | Should Storage Providers support multi-region replication as a declared capability? | Sovereignty | โ“ Unresolved |
-| 2 | How are Storage Provider failures handled โ€” failover, queuing, or rejection? | Reliability | โ“ Unresolved |
-| 3 | Should the Search Index be a separate registered Storage Provider or bundled with the GitOps store? | Architecture | โ“ Unresolved |
-| 4 | How does the Storage Provider model interact with air-gapped environments? | Sovereignty | โ“ Unresolved |
+| 1 | Should Storage Providers support multi-region replication as a declared capability? | Sovereignty | โœ… Resolved โ€” declared capability in registration; Profile determines minimum (STO-001) |
+| 2 | How are Storage Provider failures handled โ€” failover, queuing, or rejection? | Reliability | โœ… Resolved โ€” per store type: Commit Log aborts; GitOps queues; Event Stream queues; Audit accumulates; Search degrades (STO-002) |
+| 3 | Should the Search Index be a separate registered Storage Provider or bundled with the GitOps store? | Architecture | โœ… Resolved โ€” separate sub-type; non-authoritative; rebuildable (STO-003) |
+| 4 | How does the Storage Provider model interact with air-gapped environments? | Sovereignty | โœ… Resolved โ€” sovereignty_declaration on all providers; air_gap_capable flag; offline registry; signed bundles (SOV-001) |
+
+---
+
+
+## 10. Storage Architecture โ€” Q79 through Q83
+
+### 10.1 Git Repository Structure โ€” Intent and Requested Stores (Q79)
+
+GitOps stores use a deterministic handle-based directory structure. Every artifact lives at a path derivable from its identity โ€” human-navigable, diff-readable, and independently verifiable without DCM tooling.
+
+```
+dcm-intent/                              โ† Intent Store repository
+  tenants/
+    {tenant-uuid}/
+      requests/
+        {request-uuid}/
+          intent.yaml                    โ† Consumer's original submission
+          metadata.yaml                  โ† Timestamp, actor, ingress block
+
+dcm-requested/                           โ† Requested Store repository
+  tenants/
+    {tenant-uuid}/
+      requests/
+        {request-uuid}/
+          requested-payload.yaml         โ† Fully assembled, policy-processed payload
+          assembly-provenance.yaml       โ† Layer chain, policy evaluation results
+          placement.yaml                 โ† Selected provider, placement constraints
+          dependencies/
+            {dependency-uuid}/
+              requested-payload.yaml     โ† Each dependency's assembled payload
+
+dcm-layers/                              โ† Layer Store repository
+  {domain}/
+    {concern-or-type}/
+      {name}/
+        v{Major}.{Minor}.{Revision}.yaml
+
+dcm-policies/                            โ† Policy Store repository
+  {domain}/
+    {concern-or-type}/
+      {name}/
+        v{Major}.{Minor}.{Revision}.yaml
+```
+
+**Tenant isolation:** Each Tenant's requests live under their `{tenant-uuid}` directory. Access control is enforced at the DCM API layer โ€” the Git repository uses DCM's service account for all reads/writes. Individual Tenants never have direct Git access.
+
+**Branching model:** `main` is the authoritative branch. No feature branches for operational stores โ€” all writes go directly to `main` via the DCM service account. The Git history IS the audit trail for the GitOps stores.
+
+**Repository count:**
+- `minimal` and `dev` profiles: monorepo (all four stores in one repository โ€” simpler, single backup target)
+- `standard` and above: separate repositories per store type (cleaner governance boundaries, independent scaling, independent access control)
+
+**System policy (STO-005):** GitOps stores use a handle-based directory structure. `main` is authoritative. Minimal/dev may use monorepo; standard+ should use separate repos.
+
+### 10.2 Multi-Region Replication Capability Declaration (Q80)
+
+Storage Providers declare their replication capabilities in their registration. The active Profile determines the minimum replication requirement.
+
+```yaml
+storage_provider_registration:
+  capabilities:
+    replication:
+      multi_region: true
+      supported_regions: [eu-west-1, eu-west-2, us-east-1]
+      replication_modes: [active_active, active_passive]
+      consistency_model: 
+    redundancy:
+      replicas: 3
+      write_quorum: 2
+      zone_spread: required
+    backup:
+      automated: true
+      schedule: "0 */6 * * *"
+      retention: P30D
+      cross_region: true
+```
+
+**Profile minimum replication requirements:**
+
+| Profile | Multi-Region | Min Replicas | Consistency |
+|---------|-------------|-------------|------------|
+| `minimal` | No | 1 | Any |
+| `dev` | No | 1 | Any |
+| `standard` | No | 3 | Strong or bounded |
+| `prod` | Yes | 3 | Strong |
+| `fsi` | Yes | 5 | Strong |
+| `sovereign` | Yes (within boundary) | 5 | Strong |
+
+For `sovereign` profile: `multi_region: true` is required but all regions must be within the declared sovereignty boundary. Cross-boundary replication is blocked by sovereign policy groups.
+
+**System policy (STO-001):** Storage Providers must declare replication capabilities. Active Profile determines minimum requirements. Providers not meeting Profile minimum cannot be activated for that Profile's stores.
+
+### 10.3 Storage Provider Failure Handling (Q81)
+
+Failure behavior is declared per store type and governed by the active Profile.
+
+```yaml
+storage_failure_policy:
+  commit_log:
+    on_quorum_unavailable: abort_operation      # hard โ€” no silent changes
+    on_minority_failure: continue               # transparent via Raft
+  gitops_store:
+    on_unavailable: queue_writes                # local buffer; serve reads from cache
+    max_queue_size: 10000
+    max_queue_age: PT1H                         # reject if queued > 1 hour
+    on_queue_exhausted: reject                  # explicit rejection โ€” not silent drop
+  event_stream:
+    on_unavailable: queue_locally               # producer-side queuing
+    consumer_behavior: resume_from_offset       # no data loss on recovery
+  audit_store:
+    on_unavailable: accumulate_in_commit_log    # two-stage audit handles this
+    max_accumulation_age: P7D                   # alert if pending > 7 days
+  search_index:
+    on_unavailable: serve_degraded              # warn + direct to authoritative
+    on_recovery: rebuild_from_authoritative     # full index rebuild
+```
+
+**By store type:**
+- **Commit Log:** Quorum unavailable โ†’ operation aborted. Minority failure โ†’ continues via Raft reelection.
+- **GitOps Stores:** Unavailable โ†’ writes queue locally; reads serve from cache. Queue exhausted โ†’ explicit rejection.
+- **Event Stream:** Producer queues locally. Consumer resumes from last committed offset on recovery. No data loss.
+- **Audit Store:** Two-stage model โ€” Commit Log accumulates `pending_forward` entries. Operations not blocked.
+- **Search Index:** Non-authoritative. Unavailable โ†’ degraded response + reference to authoritative store. Recovery โ†’ full index rebuild.
+
+`fsi` and `sovereign` profiles tighten GitOps failure policy: write buffer age limit drops to PT15M; queue exhaustion triggers platform alert and operator notification.
+
+**System policy (STO-002):** Storage Provider failure behavior declared per store type and governed by Profile. GitOps unavailability queues writes โ€” does not silently drop. Commit Log quorum loss aborts operation. Audit Store unavailability accumulates in Commit Log. Search Index unavailability degrades queries without impacting writes.
+
+### 10.4 Search Index โ€” Separate Storage Provider Sub-Type (Q82)
+
+The Search Index is a **separate Storage Provider sub-type** distinct from GitOps stores. Treating them as the same type would obscure the critical distinction: GitOps stores are authoritative and cannot be lost; the Search Index is non-authoritative and rebuildable.
+
+```yaml
+search_index_provider:
+  provider_type: search_index              # distinct sub-type of storage_provider
+  implementation: elasticsearch           # or: opensearch
+  authoritative: false                    # explicitly non-authoritative
+  rebuildable_from: [gitops_store, event_stream]
+  rebuild_trigger: 
+  rebuild_schedule: "0 3 * * 0"          # weekly full rebuild
+  consistency_lag: PT5M                   # acceptable lag behind authoritative stores
+```
+
+**Indexing model:** DCM control plane emits index update events on every authoritative store write. Search Index Storage Provider consumes these events and updates incrementally. On failure, rebuilds from authoritative stores.
+
+**API freshness:** Queries may specify `freshness: authoritative` to bypass the index and query the GitOps store directly for guaranteed-current results.
+
+**System policy (STO-003):** Search Index is a separate Storage Provider sub-type โ€” non-authoritative, rebuildable, distinct backend from GitOps. API queries may specify `freshness: authoritative` to bypass the index.
+
+### 10.5 Audit Store โ€” Specialized Storage Provider Sub-Type (Q83)
+
+The Audit Store is a **specialized Storage Provider sub-type** with compliance properties no general Event Stream Store satisfies:
+
+- **Append-only with immutability enforcement** โ€” records cannot be modified or deleted while retention obligations apply
+- **Hash chain integrity** โ€” maintains and verifies the per-entity hash chain (AUD-006)
+- **Reference-based retention tracking** โ€” tracks entity lifecycle states for retention eligibility (AUD-003)
+- **Compliance-grade query** โ€” multi-dimensional queries by entity_uuid, actor_uuid, action, timestamp range, tenant_uuid
+
+The Event Stream (Kafka) is the **delivery channel** to the Audit Store โ€” transient, cleared after Audit Store confirms receipt. The Audit Store is the **compliance destination** โ€” permanent for the duration of retention obligations.
+
+```yaml
+audit_store_provider:
+  provider_type: audit_store             # specialized sub-type of storage_provider
+  implementation: elasticsearch          # or: opensearch
+  authoritative: true
+  append_only_enforced: true
+  hash_chain_verification: true
+  retention_tracking: reference_based
+  query_capabilities:
+    - entity_uuid
+    - actor_uuid
+    - action
+    - timestamp_range
+    - tenant_uuid
+    - request_uuid
+    - retention_status
+    - ingress_surface
+    - auth_provider_type
+  compliance_certifications: [SOC2, ISO-27001, PCI-DSS]
+```
+
+```
+Commit Log โ†’ Audit Forward Service โ†’ Event Stream โ†’ Audit Store
+(Stage 1)    (enrichment + hash)     (delivery)     (compliance storage)
+```
+
+**System policy (STO-004):** Audit Store is a specialized Storage Provider sub-type โ€” append-only, hash chain integrity, reference-based retention, compliance-grade queries. Event Stream is the delivery channel only.
+
+---
+
+## 11. Provider Sovereignty Declaration
+
+### 11.1 Obligation
+
+Every provider registration โ€” Service Provider, Information Provider, Message Bus Provider, Policy Provider, and Auth Provider โ€” must include a `sovereignty_declaration` block. This is a contractual obligation, not optional metadata. DCM uses sovereignty declarations to make placement decisions, enforce Tenant sovereignty requirements, and detect drift between declared and actual posture.
+
+### 11.2 Sovereignty Declaration Structure
+
+```yaml
+sovereignty_declaration:
+  # JURISDICTIONAL DATA
+  operating_jurisdictions:
+    - country: DE
+      legal_system: eu_gdpr
+      data_center_location: Frankfurt
+    - country: FR
+      legal_system: eu_gdpr
+      data_center_location: Paris
+  # Does data ever transit through other jurisdictions?
+  data_transit_jurisdictions: []          # empty = data stays in declared jurisdictions
+  data_residency_guarantee: true          # data never leaves declared jurisdictions
+  
+  # LEGAL FRAMEWORKS
+  legal_frameworks: [eu_gdpr, eu_nis2]
+  excluded_frameworks: []                 # frameworks this provider explicitly cannot support
+
+  # EXTERNAL DEPENDENCIES โ€” does the provider require external connectivity?
+  external_dependencies:
+    air_gap_capable: false               # true = can operate without external connectivity
+    external_services:
+      - service: licensing_server
+        jurisdiction: US
+        data_shared: [license_key, hostname]
+      - service: telemetry_endpoint
+        jurisdiction: US
+        data_shared: [usage_metrics]
+    opt_out_available:
+      telemetry: true                    # telemetry can be disabled
+
+  # THIRD-PARTY SUB-PROCESSORS
+  sub_processors:
+    - name: "Acme Cloud Storage"
+      jurisdiction: US
+      data_handled: [vm_disk_images]
+      gdpr_dpa_in_place: true
+
+  # GOVERNMENT ACCESS RISK
+  government_access_risk:
+    jurisdictions_with_compelled_access: [US]
+    # US CLOUD Act, FISA Section 702, etc.
+    legal_challenge_policy: notify_customer_where_legally_permitted
+
+  # CERTIFICATIONS โ€” with validity periods
+  certifications:
+    - name: ISO-27001
+      issuer: BSI
+      valid_from: "2024-03-01"
+      valid_until: "2027-03-01"
+      scope: "Cloud Infrastructure Operations"
+      certificate_ref:
+        credential_provider_uuid: 
+        path: "dcm/providers/kubevirt/certs/iso27001"
+    - name: SOC2-Type-II
+      issuer: Deloitte
+      valid_from: "2025-01-01"
+      valid_until: "2026-01-01"
+      scope: "Infrastructure as a Service"
+
+  # AUDIT RIGHTS
+  audit_rights:
+    customer_audit_right: true
+    audit_notice_days: 30
+    third_party_audit_accepted: true
+
+  # CHANGE NOTIFICATION OBLIGATION
+  change_notification:
+    # Provider MUST notify DCM when any sovereignty data changes
+    notification_endpoint: 
+    # Changes that MUST be notified:
+    mandatory_notification_events:
+      - certification_expiry
+      - new_jurisdiction_added
+      - jurisdiction_removed
+      - new_sub_processor
+      - sub_processor_removed
+      - new_external_dependency
+      - government_access_event
+    notification_sla: PT24H              # must notify within 24 hours of change
+```
+
+### 11.3 Change Notification and DCM Response
+
+When a provider notifies DCM of a sovereignty change (or DCM discovers one via periodic verification):
+
+```
+Provider sovereignty change detected
+  โ”‚
+  โ–ผ
+Policy Engine evaluates: does the change violate any Tenant's
+sovereignty requirements for resources currently placed with this provider?
+  โ”‚
+  โ”œโ”€โ”€ No violations:
+  โ”‚     Update sovereignty record
+  โ”‚     Emit: provider.sovereignty_changed webhook event
+  โ”‚     Notify: affected Tenants (informational)
+  โ”‚
+  โ””โ”€โ”€ Violations found โ€” for each affected resource:
+        Policy determines action:
+          notify_only       โ€” inform Tenant; no automatic action
+          pause             โ€” suspend resource; Tenant must act
+          migrate           โ€” Provider-Portable Rehydration to compliant provider
+          emergency_migrate โ€” immediate parallel provisioning; decommission after
+        Record: sovereignty_violation_record (in Audit Store)
+        Notify: Tenant owner, platform admin, data_protection_officer (if declared)
+```
+
+### 11.4 Sovereignty Violation Record
+
+```yaml
+sovereignty_violation_record:
+  record_uuid: 
+  detected_at: 
+  detection_method: 
+  provider_uuid: 
+  change_type: 
+  previous_value: 
+  new_value: 
+  affected_resources:
+    - entity_uuid: 
+      tenant_uuid: 
+      sovereignty_requirement_violated: "Data must not transit US jurisdiction"
+      policy_action: migrate
+      migration_request_uuid:    # if migrate or emergency_migrate
+  notifications_sent:
+    - recipient: tenant_owner
+    - recipient: platform_admin
+    - recipient: data_protection_officer
+```
+
+### 11.5 Auto-Migration
+
+Auto-migration (`migrate` or `emergency_migrate` policy action) uses Provider-Portable Rehydration:
+
+```
+Sovereignty violation detected โ†’ policy declares: migrate
+  โ”‚
+  โ–ผ
+DCM assembles migration request:
+  โ”‚  Same entity declaration โ€” new placement constraints
+  โ”‚  Placement engine excludes non-compliant provider from candidate set
+  โ”‚  Selects compliant alternative provider
+  โ”‚
+  โ–ผ
+emergency_migrate: parallel provisioning
+  โ”‚  New resource provisioned BEFORE old one decommissioned
+  โ”‚  Traffic/workload cutover coordinated with Tenant
+  โ”‚
+standard migrate: sequential
+  โ”‚  Old resource suspended โ†’ new resource provisioned โ†’ old decommissioned
+  โ”‚
+  โ–ผ
+Full audit trail: sovereignty_violation_record links to migration request
+```
+
+### 11.6 System Policies โ€” Provider Sovereignty
+
+| Policy | Rule |
+|--------|------|
+| `SOV-001` | All provider registrations must include a `sovereignty_declaration` block covering operating_jurisdictions, legal_frameworks, data_residency_guarantees, external_dependencies, certifications with validity periods, and government_access_risk. |
+| `SOV-002` | Providers must notify DCM when any declared sovereignty data changes. Sovereignty change notifications are treated as discovered drift and trigger Policy Engine re-evaluation. Notification SLA is declared in the provider registration. |
+| `SOV-003` | When a provider sovereignty change violates a Tenant's sovereignty requirements, the Policy Engine evaluates affected resources and applies the declared action: notify_only, pause, migrate, or emergency_migrate. |
+| `SOV-004` | Auto-migration triggered by SOV-003 uses Provider-Portable Rehydration. The non-compliant provider is excluded from the placement candidate set. The migration is a first-class DCM operation with full audit trail. |
+| `SOV-005` | Certification validity periods are tracked by DCM. Certifications expiring within P30D trigger a warning notification to the provider and affected Tenants. Expired certifications trigger SOV-003 re-evaluation. |
+
+---
+
+## 12. System Policies โ€” Storage Architecture
+
+| Policy | Rule |
+|--------|------|
+| `STO-001` | Storage Providers must declare replication capabilities. Active Profile determines minimum replication requirements. Providers not meeting Profile minimum cannot be activated for that Profile's stores. |
+| `STO-002` | Storage Provider failure behavior is declared per store type and governed by the active Profile. GitOps unavailability queues writes locally โ€” does not silently drop. Commit Log quorum loss aborts the triggering operation. Audit Store unavailability accumulates entries in the Commit Log. Search Index unavailability degrades query responses without impacting write operations. |
+| `STO-003` | The Search Index is a separate Storage Provider sub-type โ€” non-authoritative and rebuildable. API queries may specify `freshness: authoritative` to bypass the index. |
+| `STO-004` | The Audit Store is a specialized Storage Provider sub-type โ€” append-only, hash chain integrity, reference-based retention, compliance-grade queries. The Event Stream is the delivery channel only. |
+| `STO-005` | GitOps stores use a handle-based directory structure. The main branch is authoritative. Minimal and dev profiles may use a monorepo; standard and above should use separate repositories per store type. |
+
 
 ---
 

From af3eb2b51f865a0332e3742c06a75129cda704a0 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 16:25:32 -0500
Subject: [PATCH 06/49] Update service dependency model, resource entity
 details.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        |  68 ++++++-
 .../data-model/resource-service-entities.md   | 166 +++++++++++++++++-
 .../data-model/service-dependencies.md        | 127 +++++++++++++-
 3 files changed, 342 insertions(+), 19 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index bfe5dd7..8743de7 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -1864,7 +1864,61 @@ GIT-001 through GIT-008 โ€” see doc 18. AUTH-011 โ€” Git identity resolution use
 
 ---
 
-## SECTION 31 โ€” PERSONAS
+## SECTION 31 โ€” ENTITY AND DEPENDENCY GAPS
+
+### 31.1 Ownership Transfers (Q25)
+Ownership transfers are **unlimited by default**. Each transfer is immutably recorded with a monotonically incrementing `transfer_number` and mandatory reason field. Policy may declare a maximum per resource type via GateKeeper. ENT-001.
+
+### 31.2 Bare Metal Indivisibility (Q26)
+`Compute.BareMetal` declares `allocation_model: whole_unit` and `shareability.allowed: false` (structural lock). Placement holds are exclusive โ€” no concurrent holds on the same server. Provider must report full physical identity (serial_number, hardware_profile) in realized payload and notify DCM of any sharing attempt. ENT-002.
+
+### 31.3 Capacity Confidence Actions (Q27)
+Confidence ratings trigger policy-governed automatic actions:
+- `HIGH` โ†’ proceed (all profiles)
+- `MEDIUM` โ†’ proceed_with_warning (minimal/dev) or refresh_before_placement (prod/fsi/sovereign)
+- `LOW` โ†’ proceed_with_warning (minimal), refresh_before_placement (dev/standard), reject (prod/fsi/sovereign)
+
+LOW confidence triggers a Mode 1 Information Provider query before finalizing placement in standard+ profiles. Policy Group overrides per resource type. ENT-003.
+
+### 31.4 Process Resource Execution Time (Q28)
+`max_execution_time` is **mandatory** on Process Resource entities. Enforced by the Lifecycle Constraint Enforcer as a standard TTL. Profile governs default `on_max_exceeded`:
+- minimal/dev: `notify`
+- standard/prod: `escalate`
+- fsi/sovereign: `terminate`
+
+ENT-004.
+
+### 31.5 SUSPENDED State Billing (Q29)
+`billing_state` is a first-class field on all entities: `billable | non_billable | reduced_rate`. Policy injects `billing_state` and `billing_metadata` (rate_multiplier, billable_components) during state transitions. Cost Analysis component consumes the field โ€” DCM carries the billing signal, policy decides the billing model. ENT-005.
+
+### 31.6 Dependency Graph Versioning (Q30)
+Dependency graphs versioned as part of their parent catalog item โ€” not independently. New required dependency or removed dependency = **major (breaking) version bump**. New optional dependency = minor bump. Constraint change = revision bump. Dependency graph version captured in assembly provenance. ENT-006.
+
+### 31.7 Dependency Graph Storage (Q31)
+Not a separate entity. Three levels:
+- Declared graph: embedded in Resource Type Specification (GitOps)
+- Resolved graph: embedded in `placement.yaml` in Requested State
+- Realized graph: Realized State events per dependency
+
+ENT-007.
+
+### 31.8 Dependency Graph Depth (Q33)
+Profile-governed maximum: minimal=20, dev=15, standard/prod=10, fsi/sovereign=7. Requests exceeding max depth rejected with clear error. Circular dependency detection always enforced regardless of depth configuration. ENT-008.
+
+### 31.9 Meta Provider Composition Visibility (Q34)
+Meta Providers declare `composition_visibility`:
+- `opaque` โ€” consumer sees only top-level service; sub-resources not in DCM; drift on realized payload only
+- `transparent` โ€” all sub-resources registered as DCM entities; full drift detection
+- `selective` โ€” provider declares which sub-resources are DCM-visible
+
+ENT-009.
+
+### 31.10 System Policies
+ENT-001 through ENT-009 โ€” see docs 06 and 07.
+
+---
+
+## SECTION 32 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -1881,7 +1935,7 @@ GIT-001 through GIT-008 โ€” see doc 18. AUTH-011 โ€” Git identity resolution use
 
 ---
 
-## SECTION 32 โ€” TERMINOLOGY GLOSSARY
+## SECTION 33 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2060,7 +2114,7 @@ GIT-001 through GIT-008 โ€” see doc 18. AUTH-011 โ€” Git identity resolution use
 
 ---
 
-## SECTION 33 โ€” OPEN QUESTIONS
+## SECTION 34 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2157,7 +2211,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 34 โ€” DOCUMENTATION STRUCTURE
+## SECTION 35 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2205,7 +2259,7 @@ content/
 
 ---
 
-## SECTION 35 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 36 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2254,6 +2308,10 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+93. **Process Resource max_execution_time is mandatory** โ€” it is not optional metadata; enforced by the Lifecycle Constraint Enforcer; profile governs the default on_max_exceeded action (notify/escalate/terminate)
+94. **Dependency graphs are embedded, not separate entities** โ€” declared graph in Resource Type Specification; resolved graph in placement.yaml (Requested State); realized graph in Realized State events; no separate dependency graph artifact needed
+95. **Billing state is first-class โ€” not metadata** โ€” DCM carries the billing_state field; policy determines the billing model per resource type and state; Cost Analysis consumes it; organizations decide what is billable
+96. **Meta Provider composition_visibility governs DCM's view** โ€” opaque means DCM only sees what the provider reports; transparent means all sub-resources are full DCM entities with drift detection; selective is the middle ground
 89. **Provider sovereignty is a contractual obligation** โ€” every provider registration requires sovereignty_declaration; changes must be notified within declared SLA; DCM treats sovereignty changes as drift and re-evaluates placement; auto-migration available via Provider-Portable Rehydration
 90. **Git PR ingress actors resolve through the same Auth Provider as all other users** โ€” DCM trusts the Git server's authentication assertion; git config user.email is ignored (spoofing vector); the resolved actor has IDENTICAL roles/groups/tenant scope to web UI login for the same user; unresolvable identities are always rejected with an actionable PR comment
 91. **Storage Provider sub-types are distinct** โ€” Search Index (non-authoritative, rebuildable, consistency lag declared) and Audit Store (append-only, hash chain, reference-based retention, compliance queries) are separate sub-types; never treat them as interchangeable
diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md
index 239e366..d406190 100644
--- a/content/docs/architecture/data-model/resource-service-entities.md
+++ b/content/docs/architecture/data-model/resource-service-entities.md
@@ -519,19 +519,173 @@ Entities whose `on_expiry` action fails to execute enter `PENDING_EXPIRY_ACTION`
 
 ---
 
+## 9a. Lifecycle Time Constraints โ€” Process Resources (Q28)
+
+Process Resource entities must declare a maximum execution time. This is a mandatory field โ€” not optional. A Process Resource with no execution time limit creates operational blindness (DCM cannot know if it is hung).
+
+```yaml
+process_resource_entity:
+  resource_type: Process.AnsiblePlaybook
+  execution_constraints:
+    max_execution_time: PT2H          # mandatory โ€” ISO 8601 duration
+    expected_completion: PT30M        # advisory โ€” when we expect completion
+    grace_period: PT15M               # grace period after max before action fires
+    on_max_exceeded: 
+    # escalate:  notify platform admin and provider; human decides
+    # terminate: DCM instructs provider to terminate the process
+    # notify:    notify consumer and wait; no automatic action
+    escalation_recipient: 
+```
+
+The Lifecycle Constraint Enforcer handles this โ€” process execution time is a `lifecycle_constraint.ttl` with `reference_point: realization_timestamp`. The `on_max_exceeded` action maps to the standard `on_expiry` lifecycle action vocabulary.
+
+**Profile-governed default `on_max_exceeded`:**
+
+| Profile | Default Action |
+|---------|---------------|
+| `minimal` | `notify` |
+| `dev` | `notify` |
+| `standard` | `escalate` |
+| `prod` | `escalate` |
+| `fsi` | `terminate` |
+| `sovereign` | `terminate` |
+
+---
+
+## 9b. Billing State and SUSPENDED Entities (Q29)
+
+DCM carries billing state as a first-class field โ€” the Cost Analysis component consumes it. Organizations declare billing behavior via policy โ€” DCM does not decide what is billable.
+
+```yaml
+entity:
+  lifecycle_state: SUSPENDED
+  billing_state: 
+  billing_metadata:
+    billing_rate_multiplier: 0.3       # 30% of normal rate if reduced_rate
+    billing_reason: "Reserved capacity โ€” suspended but resources held"
+    billing_policy_uuid:         # policy that determined this billing state
+    billable_components: [storage, ip_address]   # which sub-resources are billed
+    non_billable_components: [compute]
+```
+
+**Three billing models for SUSPENDED:**
+- **`billable`** โ€” resources reserved and capacity held (stopped VM still consuming reserved IP and storage)
+- **`non_billable`** โ€” resources fully released on suspension (spot/ephemeral resource)
+- **`reduced_rate`** โ€” partial resources held (storage retained, compute released)
+
+Policy injects `billing_state` and `billing_metadata` during state transitions. A GateKeeper can declare: "all suspended VMs in the payments Tenant are billed at 30% โ€” compute released but storage and IP retained."
+
+---
+
+## 9c. Bare Metal Indivisibility (Q26)
+
+Bare metal Whole Allocation uses the same `shareability.allowed: false` mechanism as any non-shareable resource (REL-017), plus an explicit `allocation_model` declaration:
+
+```yaml
+resource_type_spec:
+  fully_qualified_name: Compute.BareMetal
+  allocation_model: whole_unit         # whole_unit | fractional | pooled
+  shareability:
+    allowed: false                     # structural lock โ€” cannot be changed by policy
+    indivisibility_reason: "Physical hardware โ€” cannot be partitioned"
+  capacity:
+    unit: server
+    minimum_allocation: 1
+    maximum_allocation: 1              # whole unit only
+
+# Provider contract obligations for bare metal:
+provider_contract_obligations:
+  - Report full physical identity in realized payload (serial_number, hardware_profile)
+  - Exclusive placement hold during reserve_query โ€” no concurrent holds on same server
+  - Notify DCM immediately if any sharing attempt is detected (drift trigger)
+```
+
+---
+
+## 9d. Capacity Confidence โ€” Automatic Actions (Q27)
+
+Capacity confidence ratings trigger policy-governed automatic actions. Policy determines the action per confidence level; the active Profile sets defaults.
+
+```yaml
+capacity_confidence_policy:
+  HIGH:
+    action: proceed
+    max_data_age: PT5M
+  MEDIUM:
+    action: proceed_with_warning      # default โ€” overridable by policy
+    max_data_age: PT30M
+  LOW:
+    action: refresh_before_placement  # default โ€” trigger Mode 1 query
+    max_data_age: PT1H
+    trigger_mode1_query: true
+```
+
+**Profile-governed defaults:**
+
+| Profile | HIGH | MEDIUM | LOW |
+|---------|------|--------|-----|
+| `minimal` | proceed | proceed | proceed_with_warning |
+| `dev` | proceed | proceed | refresh_before_placement |
+| `standard` | proceed | proceed_with_warning | refresh_before_placement |
+| `prod` | proceed | refresh_before_placement | reject |
+| `fsi` | proceed | refresh_before_placement | reject |
+| `sovereign` | proceed | refresh_before_placement | reject |
+
+---
+
+## 9e. Ownership Transfer Count (Q25)
+
+Ownership transfers are unlimited by default. Each transfer is immutably recorded with a monotonically incrementing `transfer_number`. Policy may declare a maximum per resource type.
+
+```yaml
+ownership_transfer_record:
+  transfer_uuid: 
+  transfer_number: 3              # monotonically incrementing โ€” never resets
+  from_tenant_uuid: 
+  to_tenant_uuid: 
+  authorized_by: 
+  transfer_timestamp: 
+  reason: 
+  policy_uuid: 
+```
+
+Policy-governed maximum when needed:
+```yaml
+policy:
+  type: gatekeeper
+  rule: >
+    If resource.ownership_transfer_count > 5
+    AND resource_type == Compute.VirtualMachine
+    THEN gatekeep: "VM has exceeded 5 ownership transfers โ€” manual review required"
+```
+
+---
+
 ## 10. Open Questions
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | For Hybrid Transfer โ€” what is the maximum number of ownership transfers allowed, or is it unlimited? | Operational complexity | โ“ Unresolved |
-| 2 | For Whole Allocation of bare metal โ€” how is the indivisibility enforced at the provider level? | Provider contract | โ“ Unresolved |
-| 3 | Should capacity confidence ratings trigger automatic actions (e.g., LOW confidence triggers a Mode 1 query)? | Capacity model | โ“ Unresolved |
-| 4 | For Process Resources โ€” should there be a maximum execution time after which DCM escalates? | Operational governance | โ“ Unresolved |
-| 5 | How does the SUSPENDED state interact with cost analysis โ€” is a suspended Entity still billable? | Cost model | โ“ Unresolved |
+| 1 | For Hybrid Transfer โ€” what is the maximum number of ownership transfers allowed, or is it unlimited? | Operational complexity | โœ… Resolved โ€” unlimited by default; policy may declare maximum; monotonically incrementing transfer_number (ENT-001) |
+| 2 | For Whole Allocation of bare metal โ€” how is the indivisibility enforced at the provider level? | Provider contract | โœ… Resolved โ€” allocation_model: whole_unit; shareability.allowed: false; exclusive hold; provider reports physical identity (ENT-002) |
+| 3 | Should capacity confidence ratings trigger automatic actions? | Capacity model | โœ… Resolved โ€” policy-governed actions per confidence level; LOW triggers Mode 1 query by default in standard+; profile-governed (ENT-003) |
+| 4 | For Process Resources โ€” should there be a maximum execution time? | Operational governance | โœ… Resolved โ€” mandatory max_execution_time; enforced by Lifecycle Constraint Enforcer; profile-governed on_max_exceeded (ENT-004) |
+| 5 | How does the SUSPENDED state interact with cost analysis? | Cost model | โœ… Resolved โ€” billing_state field (billable/non_billable/reduced_rate); policy injects on state transition; Cost Analysis consumes (ENT-005) |
 
 ---
 
-## 11. Related Concepts
+## 11. DCM System Policies โ€” Entity and Dependency Gaps
+
+| Policy | Rule |
+|--------|------|
+| `ENT-001` | Ownership transfer count is unlimited by default. Policy may declare a maximum transfer count per resource type. Each transfer is immutably recorded with a monotonically incrementing transfer_number and mandatory reason field. |
+| `ENT-002` | Bare metal resources declare `allocation_model: whole_unit` and `shareability.allowed: false`. Placement holds are exclusive. Providers must report the server's physical identity in the realized payload and notify DCM of any sharing attempt. |
+| `ENT-003` | Capacity confidence ratings trigger policy-governed automatic actions. LOW confidence triggers a Mode 1 Information Provider query by default in standard+ profiles. Profile determines the default action per confidence level. |
+| `ENT-004` | Process Resource entities must declare `max_execution_time`. This field is mandatory. Execution time is enforced by the Lifecycle Constraint Enforcer. Profile governs the default `on_max_exceeded` action. |
+| `ENT-005` | Entity `billing_state` (billable, non_billable, or reduced_rate) is a first-class field injected by policy during state transitions. The Cost Analysis component consumes `billing_state` for cost attribution. DCM does not decide billing policy โ€” it carries the billing signal. |
+
+---
+
+
 
 - **DCM Tenant** โ€” the mandatory ownership boundary for all Resource/Service Entities
 - **Four States** โ€” Intent, Requested, Realized, Discovered โ€” the state lifecycle of a Resource/Service Request and Entity
diff --git a/content/docs/architecture/data-model/service-dependencies.md b/content/docs/architecture/data-model/service-dependencies.md
index f989ce7..5e72930 100644
--- a/content/docs/architecture/data-model/service-dependencies.md
+++ b/content/docs/architecture/data-model/service-dependencies.md
@@ -12,9 +12,9 @@ weight: 6
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Entity Relationships](../entity-relationships/) | [Resource Type Hierarchy](../resource-type-hierarchy/) | [Resource/Service Entities](../resource-service-entities/)
+**Related Documents:** [Entity Relationships](09-entity-relationships.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md)
 
-> **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](../entity-relationships/). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document.
+> **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](09-entity-relationships.md). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document.
 
 ---
 
@@ -351,19 +351,130 @@ catalog_item:
 
 ---
 
+## 11a. Dependency Graph Versioning (Q30)
+
+Dependency graphs are versioned as properties of their parent catalog item โ€” not as independent artifacts. When the dependency graph changes, the catalog item version increments following standard semver semantics:
+
+| Change | Semver Impact | Reason |
+|--------|--------------|--------|
+| Dependency version constraint tightened | Revision bump | Compatible โ€” narrower constraint |
+| New optional dependency added | Minor bump | Compatible โ€” additive |
+| New required dependency added | **Major bump** | Breaking โ€” consumers must update |
+| Required dependency removed | **Major bump** | Breaking โ€” consumers may depend on it |
+| Dependency type changed | **Major bump** | Breaking โ€” structural change |
+
+**At request time:** The catalog item version determines the dependency graph. A consumer pinning to `catalog_item_version: "1.5.3"` gets exactly the dependency graph declared in that version.
+
+**For existing realizations:** The dependency graph version is captured in the Requested State assembly provenance. Rehydration with `re_evaluate: false` replays from the Requested State. Rehydration with `re_evaluate: true` uses the current dependency graph for the selected version.
+
+---
+
+## 11b. Dependency Graph Storage (Q31)
+
+The dependency graph is embedded in assembly provenance โ€” not a separate entity.
+
+| Level | What is stored | Where |
+|-------|---------------|-------|
+| Declared dependency graph | Part of Resource Type Specification | GitOps Layer/Policy Store |
+| Resolved dependency graph | `placement.yaml` in Requested State | GitOps Requested Store |
+| Realized dependency graph | Realized State events per dependency | Event Stream / Realized Store |
+
+```yaml
+# In placement.yaml โ€” resolved dependency graph
+dependency_resolution:
+  - dependency_role: storage
+    resource_type: Storage.Block
+    resolved_provider_uuid: 
+    resolved_catalog_item_version: "1.2.0"
+    reserved_entity_uuid: 
+    reservation_hold_uuid: 
+  - dependency_role: networking
+    resource_type: Network.IPAddress
+    resolved_provider_uuid: 
+    reserved_entity_uuid: 
+```
+
+The full dependency chain is always traceable from the Requested State record โ€” no separate entity needed.
+
+---
+
+## 11c. Dependency Graph Depth (Q33)
+
+Dependency graph depth is limited to a profile-governed maximum. Circular dependency detection is always enforced regardless of depth configuration.
+
+```yaml
+dependency_depth_policy:
+  max_depth: 10                  # configurable via Policy Group
+  on_max_exceeded: reject        # reject with clear error identifying depth + chain
+  cycle_detection: always        # non-configurable โ€” always enforced
+```
+
+**Profile-governed defaults:**
+
+| Profile | Default Max Depth | Rationale |
+|---------|-----------------|-----------|
+| `minimal` | 20 | Home lab โ€” free composition |
+| `dev` | 15 | Development โ€” generous |
+| `standard` | 10 | Production baseline |
+| `prod` | 10 | Production |
+| `fsi` | 7 | Tight โ€” complex dependencies harder to audit |
+| `sovereign` | 7 | Maximum control |
+
+In practice, well-designed service compositions rarely exceed 5-6 levels. Depth 10 provides headroom without allowing pathological compositions.
+
+---
+
+## 11d. Meta Provider Composition Visibility (Q34)
+
+Meta Providers declare how their internal composition is exposed to DCM. This determines whether sub-resources are DCM entities subject to standard lifecycle management, or opaque to DCM.
+
+```yaml
+meta_provider_registration:
+  composition_visibility:
+    mode: 
+    # opaque:      Consumer sees only top-level service entity
+    #              Sub-resources not visible in DCM
+    # transparent: All sub-resources registered as DCM entities
+    #              Full dependency graph visible; drift detection on all
+    # selective:   Provider declares which sub-resources are DCM-visible
+    dcm_visible_sub_resources:    # if selective
+      - resource_type: Compute.VirtualMachine
+        role: control_plane_node
+      - resource_type: Network.LoadBalancer
+        role: api_endpoint
+```
+
+**Drift detection interaction:**
+- `opaque` โ€” drift detection only on what the Meta Provider reports via realized payload; sub-resources are provider's responsibility
+- `transparent` โ€” drift detection on all sub-resources as full DCM entities
+- `selective` โ€” drift detection on declared DCM-visible sub-resources only
+
+---
+
 ## 12. Open Questions
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | How are dependency graphs versioned โ€” does a new version of a catalog item invalidate existing dependency graphs? | Versioning model | โ“ Unresolved |
-| 2 | Should the dependency graph be stored as a separate entity or embedded in the request payload? | Data model structure | โ“ Unresolved |
-| 3 | How are cross-tenant dependencies handled โ€” where one tenant's resource depends on another tenant's resource? | Multi-tenancy | โ“ Unresolved |
-| 4 | Should there be a maximum dependency graph depth to prevent runaway transitive dependencies? | Operational complexity | โ“ Unresolved |
-| 5 | How does the dependency graph interact with the Meta Provider model โ€” where one provider orchestrates others? | Provider model | โ“ Unresolved |
+| 1 | How are dependency graphs versioned โ€” does a new version of a catalog item invalidate existing dependency graphs? | Versioning model | โœ… Resolved โ€” versioned as part of catalog item; semver semantics; captured in assembly provenance (ENT-006) |
+| 2 | Should the dependency graph be stored as a separate entity or embedded in the request payload? | Data model structure | โœ… Resolved โ€” embedded in assembly provenance; declared in Resource Type Spec; resolved in placement.yaml (ENT-007) |
+| 3 | How are cross-tenant dependencies handled? | Multi-tenancy | โœ… Resolved โ€” governed by REL-010/011/012 and DEP-001/002/003; see Entity Relationships doc |
+| 4 | Should there be a maximum dependency graph depth? | Operational complexity | โœ… Resolved โ€” profile-governed max (10 standard/prod, 7 fsi/sovereign); circular detection always enforced (ENT-008) |
+| 5 | How does the dependency graph interact with the Meta Provider model? | Provider model | โœ… Resolved โ€” composition_visibility (opaque/transparent/selective); transparent/selective registers sub-resources as DCM entities (ENT-009) |
 
 ---
 
-## 13. Related Concepts
+## 13. DCM System Policies โ€” Dependency Gaps
+
+| Policy | Rule |
+|--------|------|
+| `ENT-006` | Dependency graphs are versioned as properties of their parent catalog item. New required dependency or removed dependency is a major (breaking) version bump. The dependency graph version used in a realization is captured in assembly provenance. |
+| `ENT-007` | The declared dependency graph is embedded in the Resource Type Specification. The resolved dependency graph is embedded in the Requested State assembly provenance (placement.yaml). No separate dependency graph entity is required. |
+| `ENT-008` | Dependency graph depth is limited to a profile-governed maximum (default: 10 for standard/prod; 7 for fsi/sovereign). Requests exceeding the maximum depth are rejected with a clear error. Circular dependency detection is always enforced regardless of depth configuration. |
+| `ENT-009` | Meta Providers declare composition_visibility as opaque, transparent, or selective. Transparent and selective modes register sub-resources as DCM entities subject to standard lifecycle management and drift detection. Opaque mode delegates sub-resource management entirely to the provider. |
+
+---
+
+
 
 - **Resource Type Specification** โ€” declares type-level dependencies for a Resource Type
 - **Provider Catalog Item** โ€” declares provider-specific additional dependencies

From 463ee18bc7677960af469b6768a5410cf3445be7 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 20:14:24 -0500
Subject: [PATCH 07/49] Added some advanced information provider details, add
 DCM federation.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 105 +++++++++++++++++-
 .../docs/architecture/data-model/_index.md    |   2 +
 2 files changed, 102 insertions(+), 5 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index 8743de7..1835b15 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -1918,7 +1918,98 @@ ENT-001 through ENT-009 โ€” see docs 06 and 07.
 
 ---
 
-## SECTION 32 โ€” PERSONAS
+## SECTION 32 โ€” INFORMATION PROVIDER CONFIDENCE SCORING AND AUTHORITY
+
+### 32.1 Confidence Scoring โ€” 0 to 100
+Every Information Provider field value carries a confidence score (0-100). DCM computes scores โ€” providers do not self-declare. Score bands for policy use: very_high (81-100), high (61-80), medium (41-60), low (21-40), very_low (0-20).
+
+**Formula:**
+```
+confidence_score = min(100, base_score ร— freshness_multiplier ร— corroboration_multiplier ร— authority_multiplier)
+```
+
+| Factor | Values |
+|--------|--------|
+| Base score | primary_authoritative=90, secondary=70, discovered=60, advisory=50, self_reported=40, inferred=30 |
+| Freshness | <1h=1.00, <1d=0.95, <7d=0.85, <30d=0.70, >30d=0.50 |
+| Corroboration | 1 source=1.00, 2 agree=1.10, 3+ agree=1.15, disagree=0.60 |
+| Authority | primary=1.00, secondary=0.85, advisory=0.70 |
+
+### 32.2 Authority as Layer Data
+Authority scope and priority for Information Providers are declared in **platform or system domain layers** โ€” not just policies. This is static organizational knowledge ("our CMDB is authoritative for business unit data"). Layer-defined authority establishes the default; policies act on confidence scores at runtime.
+
+### 32.3 Ingestion-Time Conflict Detection
+Conflict detection at ingestion time (7-step flow): schema validation โ†’ authority scope check โ†’ confidence score computation โ†’ conflict detection โ†’ resolution policy โ†’ entity record update โ†’ INGEST audit record.
+
+**Resolution strategies:** `higher_authority_wins` | `higher_confidence_wins` | `higher_priority_wins` | `escalate` | `merge` (array fields only)
+
+Authority scope conflicts detected at **registration time** โ€” two providers claiming primary authority for the same field cannot both go active without explicit resolution.
+
+### 32.4 Write-Back (Q63)
+Optional declared capability. Policy triggers write-back โ€” never automatic. Produces ENRICH audit records. Credentials via Credential Provider. (INF-002)
+
+### 32.5 Extended Schema Versioning (Q64)
+Semver semantics: field removal/type change = major (breaking); new optional field = minor; constraint change = revision. Migration plan required for major bumps. (INF-003)
+
+### 32.6 Well-Known Provider Registry (Q65)
+Three-tier registry (Core/Community/Organization) โ€” same governance model as Resource Type Registry. Separate registries, shared infrastructure. (INF-004)
+
+### 32.7 Air-Gapped Verification (Q66)
+Three modes: pre-verified signed bundle, internal mTLS (for internal providers), periodic online re-verification with cached tokens. Profile governs cache expiry behavior. (INF-005)
+
+### 32.8 System Policies
+INF-001 through INF-008 โ€” see docs 10 and 21.
+
+---
+
+## SECTION 33 โ€” DCM FEDERATION AND CROSS-INSTANCE COORDINATION
+
+### 33.1 Three Relationship Types
+- **Peer DCM** โ€” same organizational level; share resources/information
+- **Parent-Child DCM** โ€” hierarchical; parent has governance overlay; does not own child resources
+- **Hub DCM** โ€” specialized parent as resource allocation clearinghouse
+
+All use the Universal Group Model: federation group (peers) or tenant_boundary nesting (parent-child).
+
+### 33.2 Provider Federation Eligibility
+Every provider registration declares `federation_eligibility`:
+- `mode: none` โ€” cannot participate in any federation (sovereign/classified providers)
+- `mode: selective` โ€” only with explicitly declared partners
+- `mode: open` โ€” any trusted DCM peer (sovereignty checks always apply)
+
+**Layer-defined defaults** in `platform` domain layer. Individual registrations may be **more restrictive** โ€” never more permissive without GateKeeper approval.
+
+**Federation scope declares:** permitted resource types + operations, data sharing permissions, max concurrent allocations. Remote DCMs CANNOT decommission local resources through a tunnel.
+
+**Storage providers default to `mode: none`** โ€” data sovereignty prohibits storage federation unless explicitly authorized.
+
+### 33.3 The DCM Provider โ€” Ninth Provider Type
+Wraps another DCM instance's API. Always mTLS (non-configurable). Sovereignty checks mandatory before tunnel establishment. Local DCM policies govern ALL resources from any tunnel.
+
+**Non-negotiable primary concerns on all tunnels:**
+- Sovereignty: verified before establishment; data classification checked per egress
+- Authentication: always mTLS โ€” no API key or bearer token
+- Authorization: local policies govern; remote policies cannot override
+- Audit: records in BOTH DCM instances; shared correlation_id
+- Observability: cross-DCM allocation visible in both instances
+
+### 33.4 Cross-DCM Confidence Scoring
+```
+cross_dcm_confidence = source_resource_confidence ร— (tunnel_trust_score / 100)
+```
+Federation trust score (0-100): factors include identity verification, sovereignty compatibility, certifications currency, audit trail integrity, uptime, compliance.
+
+### 33.5 DCM Export/Import
+Signed export package: tenants, layers, policies, provider registrations (not credentials), entity intent/requested states, groups, audit records with hash chain. Never export credentials.
+
+Import trust score (0-100): source verification + sovereignty compatibility + data completeness + schema compatibility + audit trail integrity. Low score โ†’ reject or escalate.
+
+### 33.6 System Policies
+DCM-001 through DCM-008 โ€” see doc 22.
+
+---
+
+## SECTION 34 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -1935,7 +2026,7 @@ ENT-001 through ENT-009 โ€” see docs 06 and 07.
 
 ---
 
-## SECTION 33 โ€” TERMINOLOGY GLOSSARY
+## SECTION 35 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2114,7 +2205,7 @@ ENT-001 through ENT-009 โ€” see docs 06 and 07.
 
 ---
 
-## SECTION 34 โ€” OPEN QUESTIONS
+## SECTION 36 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2211,7 +2302,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 35 โ€” DOCUMENTATION STRUCTURE
+## SECTION 37 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2259,7 +2350,7 @@ content/
 
 ---
 
-## SECTION 36 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 38 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2308,6 +2399,10 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+97. **Confidence scoring is computed by DCM โ€” never self-declared** โ€” the formula (base_score ร— freshness ร— corroboration ร— authority) is standardized and auditable; policies work on bands (very_high/high/medium/low/very_low) not raw values
+98. **Information Provider authority is layer-defined** โ€” static organizational knowledge ("our CMDB is authoritative for business unit") belongs in a platform domain layer; conflict detection happens at ingestion time; policy governs automated resolution
+99. **DCM Provider is the ninth provider type** โ€” always mTLS (non-configurable); sovereignty checks mandatory; local policies govern ALL federated resources; audit records in both DCM instances with shared correlation_id
+100. **Provider federation eligibility is layer-defined with policy enforcement** โ€” platform layer sets defaults per provider type; individual registrations may be more restrictive; storage providers default to mode: none; remote DCMs cannot decommission local resources through tunnels
 93. **Process Resource max_execution_time is mandatory** โ€” it is not optional metadata; enforced by the Lifecycle Constraint Enforcer; profile governs the default on_max_exceeded action (notify/escalate/terminate)
 94. **Dependency graphs are embedded, not separate entities** โ€” declared graph in Resource Type Specification; resolved graph in placement.yaml (Requested State); realized graph in Realized State events; no separate dependency graph artifact needed
 95. **Billing state is first-class โ€” not metadata** โ€” DCM carries the billing_state field; policy determines the billing model per resource type and state; Cost Analysis consumes it; organizations decide what is billable
diff --git a/content/docs/architecture/data-model/_index.md b/content/docs/architecture/data-model/_index.md
index 3dcd177..39efdfe 100644
--- a/content/docs/architecture/data-model/_index.md
+++ b/content/docs/architecture/data-model/_index.md
@@ -33,5 +33,7 @@ realization, operation, and decommission.
   {{< card link="webhooks-messaging" title="Webhooks and Messaging" icon="arrows-expand" subtitle="Inbound and outbound webhooks. Message Bus Provider. Universal ingress/egress actor model. Credential Provider. WHK and ING policies." >}}
   {{< card link="auth-providers" title="Auth Providers" icon="lock-closed" subtitle="Eight provider types. Auth ladder from API key to air-gapped OIDC. LDAP, FreeIPA, AD, OIDC, mTLS. No anonymous access. AUTH policies." >}}
   {{< card link="registry-governance" title="Registry Governance" icon="collection" subtitle="Three-tier registry. PR-based proposals. Default deprecation policies. Cost-aware placement. Federated with air-gap signed bundles." >}}
+  {{< card link="information-providers-advanced" title="Information Provider Confidence" icon="chart-bar" subtitle="0-100 confidence scoring. Authority as layer data. Ingestion-time conflict detection. Write-back. Schema versioning. INF-001 through INF-008." >}}
+  {{< card link="dcm-federation" title="DCM Federation" icon="globe" subtitle="Peer, parent-child, hub relationships. Provider federation eligibility. DCM Provider (ninth type). Always mTLS. Cross-DCM confidence scoring. DCM-001 through DCM-008." >}}
   {{< card link="universal-groups" title="Universal Group Model" icon="view-grid" subtitle="Unified grouping: Tenants, Resource Groups, Policy Groups as one model. Composite groups, nested and federated Tenants, permanent membership history." >}}
 {{< /cards >}}

From 21e58cc33aab954e1407909f07b1045f0512a729 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 20:53:22 -0500
Subject: [PATCH 08/49] Provenanage models, validation, policy review. Two
 dimensional profile model, posture groups, compliance domain groups

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 178 +++++++++++++++++-
 .../data-model/layering-and-versioning.md     | 137 ++++++++++++++
 2 files changed, 310 insertions(+), 5 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index 1835b15..9a23a99 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2009,7 +2009,168 @@ DCM-001 through DCM-008 โ€” see doc 22.
 
 ---
 
-## SECTION 34 โ€” PERSONAS
+## SECTION 34 โ€” OPERATIONAL AND PERFORMANCE GAPS
+
+### 34.1 Field-Level Provenance Models (Q7, Q8)
+
+**Three configurable models โ€” organization chooses; profile provides default:**
+
+**Model A โ€” Full Inline**
+All provenance stored on entity record. Simplest queries, highest storage cost, no tooling required.
+- โœ… Auditors read one record โ€” regulatory clarity
+- โœ… No dependency on layer chain store
+- โŒ Very high storage volume at scale
+- โŒ Write amplification
+
+**Model B โ€” Deduplicated (Content-Addressed) โ† RECOMMENDED**
+Layer chain is the deduplication key. Classical content-addressed dedup (like Git objects, Docker layers). Only delta fields store unique provenance. 95-99% storage reduction for standardized deployments. Lossless because layer chains are immutable.
+- โœ… Dramatic storage reduction
+- โœ… Full audit reconstruction always possible
+- โœ… Write performance highest (layer-matching fields free)
+- โŒ Chain traversal tooling required for queries
+- โŒ Layer chain must be retained while any entity references it
+
+**Model C โ€” Tiered Archive**
+Hot (full detail) โ†’ warm (change events) โ†’ cold (hash anchors). Degrades gracefully.
+- โœ… Balances cost and access speed
+- โœ… Compliant for long retention
+- โŒ Cross-tier queries for long time ranges
+- โŒ Cold tier requires full records from warm/hot for reconstruction
+
+**Model B+C โ€” Combined**
+Maximum efficiency: content-addressed dedup + tiered archival of chains and deltas.
+
+**Profile defaults:**
+
+| Profile | Provenance Group | Rationale |
+|---------|----------------|-----------|
+| minimal, dev | `system/group/provenance-full-inline` | Simplicity; scale not a concern |
+| standard, prod | `system/group/provenance-deduplicated` | Scale matters; tooling justified |
+| fsi, sovereign | `system/group/provenance-full-inline` | Regulatory clarity; self-contained |
+
+Organizations override by swapping the active provenance Policy Group.
+
+**Audit completeness guarantee (OPS-002):** Regardless of model, full provenance is always reconstructable from entity record + layer chain store + Audit Store combined.
+
+### 34.2 Background Conflict Validation (Q85)
+Event-triggered (primary) on layer ingestion/update โ€” async, non-blocking. Scheduled weekly sweep as safety net. Both triggers produce same conflict record format and audit trail. (OPS-003)
+
+### 34.3 Policy Minimum Review Periods (Q86)
+Change-type minimum periods: GateKeeper=14d, Validation=7d, Transformation=3d. Profile multipliers: minimal=0ร—, dev=0.5ร—, standard=1ร—, prod=1.5ร—, fsi/sovereign=2ร—. DCM enforces โ€” not bypassable except emergency activation with dual-approval audit. (OPS-004)
+
+### 34.4 Shadow Evaluation Store (Q87)
+Dedicated **Validation Store** (not Audit Store). Queryable and modifiable. Links to Audit Store EVALUATE events via audit_record_uuid. Default retention P90D after policy promotion/retirement. (OPS-005)
+
+### 34.5 Artifact Status Extensions (Q88)
+Five standard statuses (developing/proposed/active/deprecated/retired) are invariant โ€” no custom additions. Organizations use status_metadata for workflow state (purely informational, no system behavior). Policy gates status transitions based on status_metadata field values. (OPS-006)
+
+### 34.6 System Policies
+OPS-001 through OPS-006 โ€” see docs 03 and 06.
+
+---
+
+## SECTION 35 โ€” PROFILE COMPOSITION โ€” POSTURE AND COMPLIANCE DOMAINS
+
+### 35.1 The Two-Dimensional Profile Model
+
+**Profiles compose two orthogonal dimensions:**
+
+```
+Complete Profile = Deployment Posture Group + Compliance Domain Group(s)
+```
+
+**Dimension 1 โ€” Deployment Posture** (vertical axis): How DCM infrastructure behaves โ€” redundancy, enforcement strictness, audit retention, tenancy model.
+
+| Posture Group | Key Behaviors |
+|--------------|--------------|
+| `system/group/posture-minimal` | Advisory; single instance; no redundancy |
+| `system/group/posture-dev` | Warn-not-block; basic logging |
+| `system/group/posture-standard` | Full enforcement; 3-replica; explicit cross-tenant |
+| `system/group/posture-prod` | Full enforcement + SLA; geo-replicated |
+| `system/group/posture-hardened` | 5-replica; 7-year audit; dual approval |
+| `system/group/posture-sovereign` | Air-gap; deny_all; 10-year audit; signed bundles |
+
+**Dimension 2 โ€” Compliance Domain** (horizontal): Which regulatory frameworks govern data and resources.
+
+| Compliance Group | Domain |
+|----------------|--------|
+| `system/group/compliance-fsi` | Financial Services โ€” Basel III, SOX, Dodd-Frank |
+| `system/group/compliance-pci-dss` | Payment Card Industry โ€” PCI-DSS v4 |
+| `system/group/compliance-hipaa` | Healthcare โ€” HIPAA/HITECH PHI |
+| `system/group/compliance-fedramp-moderate` | US Federal Moderate โ€” NIST 800-53 Moderate |
+| `system/group/compliance-fedramp-high` | US Federal High โ€” NIST 800-53 High |
+| `system/group/compliance-dod-il2` through `il6` | DoD Impact Levels |
+| `system/group/compliance-government` | Government/public sector |
+| `system/group/compliance-gdpr` | EU GDPR data protection |
+| `system/group/compliance-iso27001` | ISO 27001 information security |
+| `system/group/compliance-nist-800-53` | NIST 800-53 security framework |
+| `system/group/compliance-soc2` | SOC 2 service organization controls |
+| `system/group/compliance-nerc-cip` | Critical infrastructure energy/utilities |
+| `system/group/compliance-sovereign` | Sovereign/classified โ€” air-gap, HSM, signed bundles |
+
+### 35.2 Built-In Profile Compositions
+
+The six core profiles are posture+compliance compositions:
+
+```
+minimal = posture-minimal
+dev = posture-dev
+standard = posture-standard
+prod = posture-prod
+fsi = posture-hardened + compliance-fsi + compliance-pci-dss + compliance-iso27001
+sovereign = posture-sovereign + compliance-sovereign
+```
+
+**Extended built-in profiles:**
+
+| Profile | Extends | Compliance Groups Added |
+|---------|---------|------------------------|
+| `system/profile/hipaa-prod` | prod | compliance-hipaa, compliance-iso27001 |
+| `system/profile/hipaa-sovereign` | sovereign | compliance-hipaa |
+| `system/profile/fedramp-moderate` | prod | compliance-fedramp-moderate, compliance-nist-800-53 |
+| `system/profile/fedramp-high` | sovereign | compliance-fedramp-high, compliance-nist-800-53 |
+| `system/profile/government` | prod | compliance-government, compliance-nist-800-53 |
+| `system/profile/dod-il4` | sovereign | compliance-dod-il4, compliance-fedramp-high, compliance-nist-800-53 |
+| `system/profile/dod-il5` | dod-il4 | compliance-dod-il5 |
+| `system/profile/dod-il6` | dod-il5 | compliance-dod-il6, compliance-sovereign |
+
+### 35.3 HIPAA Compliance Group Key Controls
+- PHI field classification enforcement (phi: true tag required)
+- PHI access control (phi_authorized role required)
+- Audit retention: P6Y minimum
+- AES-256 at rest, TLS 1.3 in transit for PHI
+- Breach notification workflow via sovereignty_violation_record
+- BAA tracking: providers declare baa_in_place in sovereignty_declaration
+- Minimum Necessary standard on Mode 4 data_request_spec
+
+### 35.4 Government/DoD Key Controls
+- Data classification mandatory on all resources
+- Cross-boundary controls for classification levels
+- Audit retention: P10Y minimum
+- DoD IL4+: CUI handling markers; foreign sub-processor exclusion
+- DoD IL5+: sovereign posture within US boundary
+- DoD IL6: classified + HSM required for key management
+
+### 35.5 Tenant-Level Compliance Overlay
+**One DCM deployment, multiple compliance postures per Tenant:**
+```yaml
+tenant_config:
+  active_profile: system/profile/prod     # posture from platform
+  compliance_groups:
+    - system/group/compliance-hipaa        # this Tenant handles PHI
+    - system/group/compliance-pci-dss      # this Tenant processes payments
+```
+Clinical Tenants (HIPAA) + Billing Tenants (HIPAA + PCI-DSS) + Admin Tenants (standard) โ€” all on same DCM platform.
+
+### 35.6 System Policies
+- `PROF-001` โ€” Profiles compose posture + compliance domain groups
+- `PROF-002` โ€” Compliance groups apply at platform or Tenant level; additive not replacing
+- `PROF-003` โ€” DCM ships built-in compliance groups for all major domains
+- `PROF-004` โ€” implementation_posture groups (provenance model etc.) are independent of compliance domain
+
+---
+
+## SECTION 36 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2026,7 +2187,7 @@ DCM-001 through DCM-008 โ€” see doc 22.
 
 ---
 
-## SECTION 35 โ€” TERMINOLOGY GLOSSARY
+## SECTION 37 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2205,7 +2366,7 @@ DCM-001 through DCM-008 โ€” see doc 22.
 
 ---
 
-## SECTION 36 โ€” OPEN QUESTIONS
+## SECTION 38 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2302,7 +2463,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 37 โ€” DOCUMENTATION STRUCTURE
+## SECTION 39 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2350,7 +2511,7 @@ content/
 
 ---
 
-## SECTION 38 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 40 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2399,6 +2560,13 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+105. **Profiles have two independent dimensions** โ€” posture (how DCM infrastructure behaves) and compliance domain (which regulatory frameworks apply); compose them freely; a hospital uses hipaa-prod = posture-prod + compliance-hipaa; a defense contractor uses dod-il4 = posture-sovereign + compliance-dod-il4 + compliance-fedramp-high + compliance-nist-800-53
+106. **Compliance domain groups apply at Tenant level** โ€” one DCM deployment can host clinical Tenants (HIPAA), billing Tenants (HIPAA + PCI-DSS), and admin Tenants (standard) simultaneously; compliance_groups in tenant_config are additive to the platform profile
+107. **HIPAA group enforces PHI minimum necessary** โ€” Mode 4 data_request_spec is limited to minimum PHI fields; providers handling PHI must declare baa_in_place in sovereignty_declaration; audit retention is P6Y regardless of profile default; breach notification triggers via sovereignty_violation_record
+101. **Provenance model is configurable โ€” Model B is recommended for standard+** โ€” organizations choose full_inline (simple, high storage), deduplicated/Model B (content-addressed dedup, lossless, 95-99% storage reduction), tiered (hot/warm/cold), or combined; swap the active provenance Policy Group to change
+102. **Shadow evaluation records go to Validation Store โ€” not Audit Store** โ€” Validation Store is queryable and modifiable (records marked reviewed); links to Audit Store via audit_record_uuid; P90D retention after policy promotion/retirement
+103. **Policy minimum review periods are DCM-enforced** โ€” not guidelines; GateKeeper=14d, Validation=7d, Transformation=3d ร— profile multiplier; emergency bypass requires dual-approval audit
+104. **Artifact status extensions are not permitted** โ€” the five statuses are invariant; use status_metadata for workflow state (no system behavior); policy gates transitions based on status_metadata values
 97. **Confidence scoring is computed by DCM โ€” never self-declared** โ€” the formula (base_score ร— freshness ร— corroboration ร— authority) is standardized and auditable; policies work on bands (very_high/high/medium/low/very_low) not raw values
 98. **Information Provider authority is layer-defined** โ€” static organizational knowledge ("our CMDB is authoritative for business unit") belongs in a platform domain layer; conflict detection happens at ingestion time; policy governs automated resolution
 99. **DCM Provider is the ninth provider type** โ€” always mTLS (non-configurable); sovereignty checks mandatory; local policies govern ALL federated resources; audit records in both DCM instances with shared correlation_id
diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md
index a721882..097076c 100644
--- a/content/docs/architecture/data-model/layering-and-versioning.md
+++ b/content/docs/architecture/data-model/layering-and-versioning.md
@@ -118,6 +118,143 @@ Layers are stored in Git following GitOps practices. They are the configuration
 
 ---
 
+## 3a. Provenance Model Configuration
+
+### 3a.1 The Three Provenance Models
+
+Field-level provenance tracks which layer set each field, which policy modified it, and the full change history. DCM supports three configurable models โ€” organizations choose based on their scale, compliance requirements, and operational preferences. The active Profile provides a recommended default via its activated Policy Group.
+
+**Model A โ€” Full Inline**
+All provenance stored explicitly on every entity record. Every field carries its complete provenance inline: source layer, modifying policies, previous values, timestamps, actor chain.
+
+| Aspect | Detail |
+|--------|--------|
+| Storage cost | Very high โ€” scales with entities ร— fields ร— changes |
+| Query simplicity | Highest โ€” all provenance in one record, no traversal |
+| Write performance | Lowest โ€” every field change requires provenance write |
+| Audit clarity | Highest โ€” regulators see everything in one record |
+| Tooling required | Minimal |
+| Best for | Small deployments; FSI/sovereign (regulatory clarity); home lab |
+
+**Model B โ€” Deduplicated (Content-Addressed) โ† RECOMMENDED**
+Classical content-addressed deduplication applied to provenance. The layer chain is the deduplication key โ€” every entity sharing the same configuration references the same chain rather than storing a copy. Only fields deviating from the chain store unique delta records.
+
+```
+Full provenance = layer chain content (deduplicated, shared) + entity deltas (unique per entity)
+```
+
+**Why lossless:** Layer chains are immutable. A reference to `layer-chain-abc123` always resolves to exactly the same data โ€” no cache invalidation, no drift. This is what makes the deduplication lossless for audit. The reference always reconstructs the original.
+
+**Storage reduction:** 95-99% for standardized deployments (many entities, few unique chains). 36 layer definitions serving 40,000 VMs produces 36 chain references, not 8 million field provenance entries.
+
+| Aspect | Detail |
+|--------|--------|
+| Storage cost | Low โ€” scales with unique configurations, not entity count |
+| Query simplicity | Medium โ€” chain traversal required for layer-set fields |
+| Write performance | Highest โ€” only deltas write; chain-matching fields are free |
+| Audit clarity | Complete โ€” full reconstruction always possible |
+| Tooling required | Moderate โ€” chain traversal tooling |
+| Best for | Standard and prod deployments; large-scale environments |
+
+**Analogous to:** Git content-addressed objects, Docker image layers, ZFS block deduplication โ€” all content-addressed, deduplicated, lossless.
+
+**Model C โ€” Tiered Archive**
+Hot/warm/cold storage tiers with decreasing detail. Recent provenance at full detail and fast access; older provenance compressed to change events; oldest compressed to hash anchors only (tamper-evidence without full reconstruction).
+
+| Aspect | Detail |
+|--------|--------|
+| Storage cost | Medium โ€” time-dependent, degrades gracefully |
+| Query simplicity | Medium โ€” cross-tier joins for long time ranges |
+| Write performance | Medium |
+| Audit clarity | Full detail in hot tier; change events in warm; anchors in cold |
+| Tooling required | Moderate โ€” tier promotion jobs, consistency checks |
+| Best for | Large deployments with long retention requirements |
+
+**Models B and C are orthogonal** โ€” combine them for maximum efficiency: deduplicate at the entity level (Model B) AND tier the storage of chains and deltas (Model C). This is the highest-efficiency option for very large-scale deployments with long retention requirements.
+
+### 3a.2 Configurable Provenance Model
+
+The provenance model is declared in the DCM deployment configuration and activated via a Policy Group:
+
+```yaml
+provenance_config:
+  model: 
+
+  # Model A โ€” Full Inline
+  full_inline:
+    include_previous_values: true
+    include_actor_chain: true
+    include_policy_rationale: true
+
+  # Model B โ€” Deduplicated (Content-Addressed)
+  layer_chain_ref:
+    store_layer_derivable: false      # do not store fields matching chain default
+    delta_detail_level: 
+    history_document_retention: P7Y
+    chain_store_retention: P7Y       # chains retained while any entity references them
+
+  # Model C โ€” Tiered Archive
+  tiered:
+    hot_tier_duration: P30D          # full detail, fast access
+    warm_tier_duration: P365D        # change events only
+    cold_tier_duration: P10Y         # hash anchors only
+    warm_tier_detail: 
+
+  # Model B + C โ€” Deduplicated + Tiered (maximum efficiency)
+  layer_chain_ref_tiered:
+    chain_store_hot: P365D           # chains fast for 1 year
+    chain_store_warm: P7Y            # chains slower for 7 years
+    delta_store_hot: P90D            # deltas fast for 90 days
+    delta_store_warm: P7Y            # deltas slower for 7 years
+```
+
+### 3a.3 Profile-Appropriate Provenance Policy Groups
+
+DCM ships four provenance Policy Groups. The active Profile activates the appropriate group by default. Organizations override by swapping the active group.
+
+| Group Handle | Model | Profile Default | Concern Type |
+|-------------|-------|----------------|-------------|
+| `system/group/provenance-full-inline` | A โ€” Full Inline | minimal, dev, fsi, sovereign | implementation_posture |
+| `system/group/provenance-deduplicated` | B โ€” Deduplicated | standard, prod | implementation_posture |
+| `system/group/provenance-tiered-archive` | C โ€” Tiered | (available โ€” not default) | implementation_posture |
+| `system/group/provenance-deduplicated-tiered` | B+C โ€” Combined | (available for large-scale) | implementation_posture |
+
+**To change provenance model:**
+```yaml
+# Override profile default โ€” swap the active provenance group
+tenant_config:
+  policy_group_overrides:
+    replace:
+      - from: system/group/provenance-full-inline
+        to: system/group/provenance-deduplicated
+        reason: "Deploying at scale โ€” switching to deduplicated model"
+```
+
+### 3a.4 The Audit Completeness Guarantee
+
+Regardless of provenance model, full provenance must always be reconstructable:
+
+```
+OPS-002  Regardless of provenance model, full provenance must always be
+         reconstructable for any entity from the combination of: entity
+         record, layer chain store, and Audit Store. The provenance model
+         governs where data is stored and how it is accessed โ€” not whether
+         it is available.
+```
+
+For Model B: chain reference + entity deltas โ†’ full provenance (lossless, immutable source)
+For Model C: hot tier (full) OR warm tier (events) + cold tier (anchors prove integrity)
+For Model A: entity record alone is sufficient
+
+### 3a.5 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `OPS-001` | Field-level provenance model is configurable: full_inline, layer_chain_ref (deduplicated), tiered, or layer_chain_ref_tiered. Profile activates the appropriate Policy Group as default. Organizations override by replacing the active provenance group. Model B (layer_chain_ref) is the recommended default for standard+ profiles. |
+| `OPS-002` | Regardless of provenance model, full provenance must always be reconstructable from the combination of entity record, layer chain store, and Audit Store. The provenance model governs storage location and access pattern โ€” not data availability. |
+
+---
+
 ## 3. Layer Types
 
 DCM defines six layer types. Each has a distinct purpose, scope, ownership model, and position in the assembly precedence chain.

From 45aa37577a426f24275843684bbff2952c1d15f2 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 21:06:33 -0500
Subject: [PATCH 09/49] resolve group and relationship gaps

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 60 ++++++++++++++--
 .../data-model/entity-relationships.md        | 68 ++++++++++++++++++-
 2 files changed, 122 insertions(+), 6 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index 9a23a99..c1de069 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2170,7 +2170,54 @@ Clinical Tenants (HIPAA) + Billing Tenants (HIPAA + PCI-DSS) + Admin Tenants (st
 
 ---
 
-## SECTION 36 โ€” PERSONAS
+## SECTION 36 โ€” GROUPING AND RELATIONSHIP GAPS
+
+### 36.1 Group Subclass Registry (Q35)
+No separate registry needed. `group_class` is the closed system-behavior set. `group_subclass` is open and advisory โ€” freely declared, never validated. DCM ships a community subclass catalog as a non-authoritative reference (same infrastructure as well-known provider registry). GRP-011.
+
+### 36.2 Group Sovereignty Interaction (Q36)
+Class-specific sovereignty rules:
+- `tenant_boundary` โ€” NEVER cross-sovereignty (structural, not configurable)
+- `resource_grouping` โ€” permitted by default; policy may restrict for classified resources
+- `policy_collection` / `layer_grouping` โ€” always permitted (governance artifacts, no data)
+- `composite` โ€” governed by most restrictive member type
+- `federation` โ€” permitted with DCM federation rules (DCM-003)
+
+GRP-012.
+
+### 36.3 Tenant Decommission Lifecycle (Q37)
+Mandatory four-phase staged decommission:
+1. **Pre-decommission validation** (blocking): resource state, cross-tenant relationships, compliance holds, rehydration leases, child groups resolved first
+2. **Resource decommission**: cascade (default) / retain (ORPHANED state) / notify (PENDING_DECOMMISSION)
+3. **Group membership cleanup**: remove from all memberships; empty federation groups โ†’ EMPTY state
+4. **Audit record archival**: all records enter post-lifecycle retention โ€” NEVER destroyed
+
+Child tenant_boundary groups must be resolved BEFORE parent decommission (GRP-INV-003). GRP-013.
+
+### 36.4 Time-Bounded Group Membership (Q38)
+Already in Universal Group Model via `valid_from` / `valid_until` on every membership. Lifecycle Constraint Enforcer handles expiry. `on_expiry` actions: `remove` / `notify` (default) / `suspend_member`. `warn_before_expiry: P7D` standard. Expiry produces `MEMBER_REMOVE` audit record with `reason: membership_ttl_expired`. GRP-014.
+
+### 36.5 Group Policy Inheritance (Q39)
+Class-specific defaults, all profile-governed:
+- `tenant_boundary`: `opt_out` (standard/prod) โ€” parent cascades unless child excludes; `opt_in` (minimal/dev/fsi/sovereign)
+- `federation`: always `opt_in` โ€” peer consent required, not configurable
+- `composite`: `opt_out` by default, configurable
+- `resource_grouping` / `policy_collection`: not applicable
+
+GRP-015.
+
+### 36.6 Relationship Role Validation (Q58)
+Advisory by default. Resource Type Spec may declare `permitted_relationship_roles` with `role_validation: advisory | enforced`. Advisory โ†’ assembly warning for unknown roles. Enforced โ†’ unknown roles rejected at request time. DCM ships community role catalog as non-authoritative reference. REL-020.
+
+### 36.7 Relationship Graph Depth (Q60)
+Profile-governed max: minimal=25, dev=20, standard/prod=15, fsi/sovereign=10. Circular detection always enforced. Depth = graph traversal distance between any two entities (NOT count of relationships on one entity). REL-021.
+
+### 36.8 System Policies
+GRP-011 through GRP-015 โ€” see doc 15. REL-020, REL-021 โ€” see doc 09.
+
+---
+
+## SECTION 37 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2187,7 +2234,7 @@ Clinical Tenants (HIPAA) + Billing Tenants (HIPAA + PCI-DSS) + Admin Tenants (st
 
 ---
 
-## SECTION 37 โ€” TERMINOLOGY GLOSSARY
+## SECTION 38 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2366,7 +2413,7 @@ Clinical Tenants (HIPAA) + Billing Tenants (HIPAA + PCI-DSS) + Admin Tenants (st
 
 ---
 
-## SECTION 38 โ€” OPEN QUESTIONS
+## SECTION 39 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2463,7 +2510,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 39 โ€” DOCUMENTATION STRUCTURE
+## SECTION 40 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2511,7 +2558,7 @@ content/
 
 ---
 
-## SECTION 40 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 41 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2560,6 +2607,9 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+108. **Tenant decommission is four-phase and never silent** โ€” pre-decommission validation blocks the operation until all resources, cross-tenant relationships, compliance holds, and child groups are resolved; audit records enter post-lifecycle retention and are never destroyed
+109. **Group policy inheritance is class-specific** โ€” tenant_boundary uses opt_out for standard/prod (parent cascades) and opt_in for fsi/sovereign; federation always opt_in; resource_grouping and policy_collection are not applicable
+110. **Relationship graph depth differs from dependency depth** โ€” depth is graph traversal distance between any two entities, not relationship count; circular detection always enforced; profile-governed max 15 (standard/prod) or 10 (fsi/sovereign)
 105. **Profiles have two independent dimensions** โ€” posture (how DCM infrastructure behaves) and compliance domain (which regulatory frameworks apply); compose them freely; a hospital uses hipaa-prod = posture-prod + compliance-hipaa; a defense contractor uses dod-il4 = posture-sovereign + compliance-dod-il4 + compliance-fedramp-high + compliance-nist-800-53
 106. **Compliance domain groups apply at Tenant level** โ€” one DCM deployment can host clinical Tenants (HIPAA), billing Tenants (HIPAA + PCI-DSS), and admin Tenants (standard) simultaneously; compliance_groups in tenant_config are additive to the platform profile
 107. **HIPAA group enforces PHI minimum necessary** โ€” Mode 4 data_request_spec is limited to minimum PHI fields; providers handling PHI must declare baa_in_place in sovereignty_declaration; audit retention is P6Y regardless of profile default; breach notification triggers via sovereignty_violation_record
diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md
index d7590ef..de6642b 100644
--- a/content/docs/architecture/data-model/entity-relationships.md
+++ b/content/docs/architecture/data-model/entity-relationships.md
@@ -932,6 +932,72 @@ Relationships follow the universal versioning and deprecation model. A relations
 - **Resource Type Specification** โ€” declares possible relationships for a resource type
 - **External Entity Reference** โ€” stable pointer to data owned by an external system
 
+
+## 12. Relationship Gap Resolutions โ€” Q58 and Q60
+
+### 12.1 Relationship Role Validation (Q58)
+
+Relationship roles are semantic labels โ€” human-readable identifiers for the function a member plays in a relationship. By default, role validation is advisory. Resource Type Specifications may declare a closed set of permitted roles with enforced validation.
+
+```yaml
+resource_type_spec:
+  fully_qualified_name: Compute.VirtualMachine
+  permitted_relationship_roles:
+    - role: storage
+      relationship_types: [requires]
+      permitted_related_types: [Storage.Block, Storage.File]
+    - role: networking
+      relationship_types: [requires]
+      permitted_related_types: [Network.IPAddress, Network.Port]
+    - role: dns
+      relationship_types: [depends_on]
+      permitted_related_types: [DNS.Record]
+    - role: load_balancer
+      relationship_types: [depends_on]
+      permitted_related_types: [Network.LoadBalancer]
+  role_validation: advisory   # advisory | enforced
+  # advisory: unknown roles produce a warning in assembly provenance
+  # enforced: unknown roles are rejected at request time
+```
+
+**Community role catalog:** DCM ships a non-authoritative reference list of commonly-used roles. Organizations freely declare roles not in the catalog when role_validation is advisory.
+
+### 12.2 Maximum Relationship Graph Depth (Q60)
+
+Relationship graph depth is limited to a profile-governed maximum. Circular relationship detection is always enforced regardless of depth configuration.
+
+```yaml
+relationship_depth_policy:
+  max_depth: 15                  # configurable via Policy Group
+  on_max_exceeded: reject        # reject with clear error
+  cycle_detection: always        # non-configurable โ€” always enforced
+  # Depth = maximum traversal distance between any two entities
+  # NOT the count of relationships on one entity
+```
+
+**Profile-governed defaults:**
+
+| Profile | Max Depth | Rationale |
+|---------|----------|-----------|
+| `minimal` | 25 | Home lab โ€” free composition |
+| `dev` | 20 | Development โ€” generous |
+| `standard` | 15 | Production baseline |
+| `prod` | 15 | Production |
+| `fsi` | 10 | Tighter โ€” complex graphs harder to audit |
+| `sovereign` | 10 | Maximum control |
+
+**Note:** Relationship depth differs from dependency depth (ENT-008). Dependency depth counts the provisioning chain. Relationship depth counts the graph traversal distance between any two entities. A VM with 50 IP address relationships has depth 1, not 50.
+
+---
+
+## 13. System Policies โ€” Relationship Gaps
+
+| Policy | Rule |
+|--------|------|
+| `REL-020` | Relationship roles are semantic labels. Resource Type Specifications may declare permitted_relationship_roles with advisory or enforced validation. Advisory produces assembly warnings for unknown roles. Enforced rejects unknown roles at request time. DCM maintains a community role catalog as a non-authoritative reference. |
+| `REL-021` | Relationship graph depth is limited to a profile-governed maximum (default: 15 for standard/prod; 10 for fsi/sovereign). Circular relationship detection is always enforced regardless of depth configuration. Depth is measured as the maximum traversal distance between any two entities in the relationship graph. |
+
+
 ---
 
-*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
\ No newline at end of file

From 133954c24b42d7b1a520e90e9445021b7ad317ee Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 21:26:35 -0500
Subject: [PATCH 10/49] Reframe confidence scoring model, add observability
 stream.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md | 85 ++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 5 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index c1de069..68200d8 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2217,7 +2217,72 @@ GRP-011 through GRP-015 โ€” see doc 15. REL-020, REL-021 โ€” see doc 09.
 
 ---
 
-## SECTION 37 โ€” PERSONAS
+## SECTION 37 โ€” AUDIT AND OBSERVABILITY GAPS
+
+### 37.1 Information Provider Trust Score Validation (Q15)
+
+Dual-trigger model โ€” same pattern as conflict validation:
+
+**Event-triggered (primary):** push fails schema validation โ†’ degraded; push conflicts with primary authority โ†’ degraded; health check fails โ†’ degraded; sovereignty declaration change โ†’ re-evaluated; registration update โ†’ re-verification triggered.
+
+**Scheduled (safety net):** daily health check; weekly full re-verification (identity, sovereignty, certifications, schema). fsi/sovereign: daily full re-verification.
+
+**Trust score โ†’ source_trust mapping:**
+- Score โ‰ฅ 80: `verified` โ†’ confidence multiplier 1.00
+- Score 60-79: `degraded` โ†’ confidence multiplier 0.75
+- Score < 60: `suspended` โ†’ no new pushes accepted; score = 0
+
+(INF-009)
+
+### 37.2 Confidence Scoring โ€” The Hybrid Descriptor Model (Q15 extended)
+
+**Three-tier model:**
+- **Confidence Descriptor** (primary โ€” stored): `authority_level` + `corroboration` + `source_trust` + `last_updated_at`
+- **Derived Score** (0-100, computed on demand): mathematical composition for placement and conflict resolution
+- **Derived Band** (very_high/high/medium/low/very_low, computed on demand): what humans and policies use
+
+**Who sets each field:**
+- `authority_level`: set at registration from authority declaration layer (static per field per provider)
+- `corroboration`: computed at ingestion (confirmed/single_source/contested based on multi-provider comparison)
+- `source_trust`: maintained by trust scoring system (event-triggered + scheduled)
+- `last_updated_at`: set at each push event
+
+**Score formula:** `min(100, base(authority_level) ร— freshness_mult ร— corroboration_mult ร— trust_mult)`
+
+**Freshness is computed at query time** from `now - last_updated_at` โ€” never stored (avoids staleness). Score and band computed at query time for the same reason.
+
+**Audit reconstruction:** authority_level (from registration) + corroboration (from ingestion event) + source_trust (from trust audit) + last_updated_at (from push event) โ†’ score and band fully reconstructable from stored facts.
+
+**Configurable derivation:** base scores, freshness thresholds, and band thresholds configurable via Policy Group โ€” stored as versioned policy artifacts.
+
+### 37.3 Audit vs Observability โ€” Definitively Separate (Q16)
+
+| | Audit | Observability |
+|--|-------|--------------|
+| Purpose | WHAT HAPPENED + WHO authorized | SYSTEM HEALTH + PERFORMANCE |
+| Consumers | Auditors, compliance, legal | SREs, operators, dashboards |
+| Write rate | Low (per action) | Very high (per second) |
+| Retention | P7Y+ | Days to months |
+| Mutability | Never โ€” append-only | Downsampling acceptable |
+| Accuracy | 100% required | Statistical sampling OK |
+| Failure | Missing = compliance violation | Missing = operational inconvenience |
+
+They cannot be combined without violating one contract or the other. Observability may reference audit record UUIDs for correlation. AUD-013.
+
+### 37.4 Curated Observability Event Stream (Q17)
+
+DCM publishes a curated event stream via Message Bus โ€” NOT raw metrics. Policy governs what is published, subscriber roles, and redaction. Published by default: component.health_changed, resource.state_transition, capacity.threshold_crossed, drift.detected, security.gatekeeper_triggered, provider.confidence_changed. NOT published by default: metrics.raw (explicit policy opt-in required).
+
+Observability events on Message Bus do NOT replace audit records. OBS-001.
+
+### 37.5 System Policies
+- `INF-009` โ€” dual-trigger trust score; degraded/suspended states; policy governs thresholds
+- `AUD-013` โ€” audit and observability definitively separate; different contracts/consumers
+- `OBS-001` โ€” curated observability event stream via Message Bus; policy-filtered; raw metrics opt-in only
+
+---
+
+## SECTION 38 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2234,7 +2299,7 @@ GRP-011 through GRP-015 โ€” see doc 15. REL-020, REL-021 โ€” see doc 09.
 
 ---
 
-## SECTION 38 โ€” TERMINOLOGY GLOSSARY
+## SECTION 39 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2297,6 +2362,12 @@ GRP-011 through GRP-015 โ€” see doc 15. REL-020, REL-021 โ€” see doc 09.
 | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
 | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
 | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **confidence_descriptor** | Primary confidence data model: authority_level (from registration), corroboration (from ingestion), source_trust (from trust system), last_updated_at (from push) โ€” stored fields |
+| **freshness** | Derived confidence field computed at query time from (now - last_updated_at) vs thresholds: high (<1h), medium (<1d), low (<7d), stale (>7d) |
+| **corroboration** | Confidence descriptor field: confirmed (2+ sources agree), single_source, contested (sources disagree) |
+| **source_trust** | Confidence descriptor field maintained by trust scoring system: verified (score โ‰ฅ80), degraded (60-79), suspended (<60) |
+| **OBS-001** | Policy: DCM publishes curated Observability Event Stream via Message Bus; policy-filtered; raw metrics opt-in; does not replace audit records |
+| **AUD-013** | Policy: Audit and Observability are definitively separate components โ€” opposite trade-offs; cannot be combined |
 | **group_subclass** | Advisory label on a DCMGroup โ€” no system behavior; used for organization-specific semantics (e.g., cost_center, business_unit) |
 | **composite group** | DCMGroup with group_class: composite โ€” permits cross-type membership (resources, policies, layers, groups) |
 | **federation group** | DCMGroup with group_class: federation โ€” peer association of tenant_boundary groups; enables shared policies and consolidated reporting |
@@ -2413,7 +2484,7 @@ GRP-011 through GRP-015 โ€” see doc 15. REL-020, REL-021 โ€” see doc 09.
 
 ---
 
-## SECTION 39 โ€” OPEN QUESTIONS
+## SECTION 40 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2510,7 +2581,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 40 โ€” DOCUMENTATION STRUCTURE
+## SECTION 41 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2558,7 +2629,7 @@ content/
 
 ---
 
-## SECTION 41 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 42 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2607,6 +2678,10 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+111. **Confidence descriptor is primary โ€” score and band are derived** โ€” authority_level/corroboration/source_trust/last_updated_at are stored facts; freshness/score/band computed at query time from stored facts; never stored as primary (avoids staleness); all reconstructable from audit records
+112. **source_trust drives the confidence trust_multiplier** โ€” verified=1.00, degraded=0.75, suspended=0.00; maintained by dual-trigger trust scoring (event-triggered + weekly scheduled); push failures and schema errors degrade trust automatically
+113. **Audit and Observability answer different questions** โ€” Audit: what happened and who authorized it; Observability: is the system healthy; different consumers, opposite storage trade-offs; Observability may reference audit record UUIDs but lives in a separate store
+114. **Curated Observability Event Stream is policy-filtered** โ€” not raw metrics; policy governs what is published and subscriber role requirements; raw metrics.raw requires explicit policy opt-in; published events do not replace audit records
 108. **Tenant decommission is four-phase and never silent** โ€” pre-decommission validation blocks the operation until all resources, cross-tenant relationships, compliance holds, and child groups are resolved; audit records enter post-lifecycle retention and are never destroyed
 109. **Group policy inheritance is class-specific** โ€” tenant_boundary uses opt_out for standard/prod (parent cascades) and opt_in for fsi/sovereign; federation always opt_in; resource_grouping and policy_collection are not applicable
 110. **Relationship graph depth differs from dependency depth** โ€” depth is graph traversal distance between any two entities, not relationship count; circular detection always enforced; profile-governed max 15 (standard/prod) or 10 (fsi/sovereign)

From 4d620adb89b987cd0e4d1590d2bf55827d7620f1 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 21:36:55 -0500
Subject: [PATCH 11/49] Resolve override control, Constraint schema and
 post-realization editability.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 101 +++++++++++++++++-
 .../data-model/layering-and-versioning.md     |  94 +++++++++++++++-
 2 files changed, 187 insertions(+), 8 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index 68200d8..ba184e9 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2282,7 +2282,88 @@ Observability events on Message Bus do NOT replace audit records. OBS-001.
 
 ---
 
-## SECTION 38 โ€” PERSONAS
+## SECTION 38 โ€” OVERRIDE CONTROL AND ENHANCEMENT GAPS
+
+### 38.1 The Complete Field Lifecycle Contract
+
+Three questions together define the full field governance model across a resource's lifecycle:
+
+```
+ASSEMBLY TIME (Q50 โ€” override_preference):
+  Which layers can set this field?
+  immutable โ†’ only this layer and higher-domain layers
+  constrained โ†’ any layer within declared bounds
+  allow โ†’ any layer
+
+CATALOG PRESENTATION (Q52 โ€” constraint_visibility):
+  What does the consumer see about this field's constraints?
+  full โ†’ constraint + bounds + reason + suggestions
+  summary โ†’ bounds only
+  hidden โ†’ enforced silently
+
+POST-REALIZATION (Q56 โ€” editability):
+  Can the consumer change this field after provisioning?
+  editable: false โ†’ requires reprovisioning
+  editable: true โ†’ targeted delta update permitted (within edit_constraints)
+```
+
+### 38.2 Override Preference Enforcement (Q50)
+
+`override: allow | constrained | immutable` on layer fields is enforced by the Request Payload Processor during assembly **Step 3 (Layer Merge)**. No separate GateKeeper policy needed.
+
+**Authority rule:** `immutable` prevents overrides from lower-authority domains only. A platform domain `immutable` field blocks tenant/service/provider/request โ€” but system domain can still override. Higher authority always wins.
+
+**GateKeeper escalation:** A GateKeeper policy may additionally lock an `allow` or `constrained` field at runtime โ€” for compliance mandates the layer author didn't anticipate.
+
+**Enforcement:** If a lower-priority layer or consumer sets an `immutable` field โ†’ assembly halts with clear error identifying the conflicting layer and locking layer.
+
+LAY-005.
+
+### 38.3 Constraint Schema Visibility (Q52)
+
+Constrained fields expose their constraint schema to consumers in the Service Catalog UI and Consumer API at a policy-governed disclosure level.
+
+**Disclosure levels:** `full` (constraint + bounds + reason + suggestions), `summary` (bounds only), `hidden` (silently enforced)
+
+**Profile defaults:** minimal/dev/standard โ†’ full; prod/fsi โ†’ summary; sovereign โ†’ hidden
+
+**API:** `GET /api/v1/catalog/items/{id}/schema` returns field schemas at the declared visibility level for the authenticated consumer's Tenant profile.
+
+Policy may override per field or resource type.
+
+LAY-006.
+
+### 38.4 Post-Realization Field Editability (Q56)
+
+**Editability is orthogonal to override_preference:**
+- `override` governs assembly time (which layers can set the field during request construction)
+- `editable` governs post-realization (can the consumer update the field on a running resource)
+
+**Declared on Resource Type Specification:**
+```yaml
+fields:
+  cpu_count:    editable: true; edit_constraints: {range: 1-32}; requires_restart: true
+  hostname:     editable: false; non_editable_reason: "Requires reprovisioning"
+  dns_servers:  editable: true; requires_restart: false
+  region:       editable: false; non_editable_reason: "Region immutable post-realization"
+```
+
+**Update request flow:** validate editable โ†’ validate edit_constraints โ†’ evaluate edit_policy โ†’ produce delta Requested State โ†’ dispatch delta to provider โ†’ update Realized State
+
+**Critical:** Updates are targeted deltas โ€” **layers do NOT re-run**. Only changed fields validated and dispatched. Layer chain NOT re-assembled for updates.
+
+Editable fields and edit_constraints visible in Service Catalog at same constraint_visibility level as constraint schemas.
+
+ENT-010.
+
+### 38.5 System Policies
+- `LAY-005` โ€” override: allow/constrained/immutable enforced at Step 3; immutable = lower-authority only; GateKeeper may additionally lock
+- `LAY-006` โ€” constraint schema visible at full/summary/hidden level; profile-governed; API endpoint
+- `ENT-010` โ€” editability first-class on Resource Type Spec; independent of override_preference; updates = targeted deltas; layers do not re-run
+
+---
+
+## SECTION 39 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2299,7 +2380,7 @@ Observability events on Message Bus do NOT replace audit records. OBS-001.
 
 ---
 
-## SECTION 39 โ€” TERMINOLOGY GLOSSARY
+## SECTION 40 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2362,6 +2443,13 @@ Observability events on Message Bus do NOT replace audit records. OBS-001.
 | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
 | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
 | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **override: allow/constrained/immutable** | Layer field metadata declaring override intent; enforced by Request Payload Processor at Step 3; immutable prevents lower-authority overrides only; GateKeeper may additionally lock |
+| **constraint_visibility** | Policy-governed disclosure level for constrained fields: full (constraint+bounds+reason+suggestions), summary (bounds only), hidden (silently enforced) |
+| **editable** | Resource Type Spec field declaration: can this field be modified post-realization via a targeted delta update (true) or only via reprovisioning (false) |
+| **edit_constraints** | Bounds declared on editable fields: range, list, enum; validated at update time; same constraint types as assembly-time constraints |
+| **requires_restart** | Editable field flag indicating whether a provider restart action is needed to apply the update |
+| **targeted delta** | Update mechanism for editable fields: applies only changed fields to Realized State; does NOT re-run the layer assembly chain |
+| **non_editable_reason** | Human-readable explanation on non-editable fields explaining why reprovisioning is required to change them |
 | **confidence_descriptor** | Primary confidence data model: authority_level (from registration), corroboration (from ingestion), source_trust (from trust system), last_updated_at (from push) โ€” stored fields |
 | **freshness** | Derived confidence field computed at query time from (now - last_updated_at) vs thresholds: high (<1h), medium (<1d), low (<7d), stale (>7d) |
 | **corroboration** | Confidence descriptor field: confirmed (2+ sources agree), single_source, contested (sources disagree) |
@@ -2484,7 +2572,7 @@ Observability events on Message Bus do NOT replace audit records. OBS-001.
 
 ---
 
-## SECTION 40 โ€” OPEN QUESTIONS
+## SECTION 41 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2581,7 +2669,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 41 โ€” DOCUMENTATION STRUCTURE
+## SECTION 42 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2629,7 +2717,7 @@ content/
 
 ---
 
-## SECTION 42 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 43 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
@@ -2678,6 +2766,9 @@ When working on this project, follow these instructions:
 67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously
 68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control
 69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations
+115. **Three independent field governance mechanisms cover the full lifecycle** โ€” override_preference (assembly time: which layers can set), constraint_visibility (catalog: what consumers see), editability (post-realization: what consumers can change); all three are orthogonal and compose
+116. **Updates are targeted deltas โ€” layers never re-run on updates** โ€” PATCH requests validate editable fields and edit_constraints then dispatch a delta; the layer assembly chain is not re-invoked; this preserves original assembly integrity while allowing operational changes
+117. **immutable override only blocks lower-authority domains** โ€” a platform domain immutable field blocks tenant/service/provider/request; it does not block system domain; higher authority always wins; GateKeeper can additionally lock allow/constrained fields for compliance mandates
 111. **Confidence descriptor is primary โ€” score and band are derived** โ€” authority_level/corroboration/source_trust/last_updated_at are stored facts; freshness/score/band computed at query time from stored facts; never stored as primary (avoids staleness); all reconstructable from audit records
 112. **source_trust drives the confidence trust_multiplier** โ€” verified=1.00, degraded=0.75, suspended=0.00; maintained by dual-trigger trust scoring (event-triggered + weekly scheduled); push failures and schema errors degrade trust automatically
 113. **Audit and Observability answer different questions** โ€” Audit: what happened and who authorized it; Observability: is the system healthy; different consumers, opposite storage trade-offs; Observability may reference audit record UUIDs but lives in a separate store
diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md
index 097076c..d32c006 100644
--- a/content/docs/architecture/data-model/layering-and-versioning.md
+++ b/content/docs/architecture/data-model/layering-and-versioning.md
@@ -1856,6 +1856,94 @@ request:
 
 ---
 
+## 13b. Override Control and Constraint Visibility Gaps
+
+### 13b.1 Override Preference Enforcement (Q50)
+
+Layer fields declare override intent using three values. The Request Payload Processor enforces this during assembly Step 3 (Layer Merge) โ€” no separate GateKeeper policy required.
+
+```yaml
+fields:
+  dns_servers:
+    value: [10.0.0.53, 10.0.0.54]
+    metadata:
+      override: allow          # lower layers and consumers may change this
+
+  encryption_at_rest:
+    value: true
+    metadata:
+      override: immutable      # no lower-domain layer or consumer may change this
+      lock_reason: "CISO mandate SEC-2024-047 โ€” encryption always required"
+
+  cpu_count:
+    value: 4
+    metadata:
+      override: constrained    # may change within declared bounds
+      constraint:
+        type: range
+        min: 1
+        max: 32
+        step: 1
+      constraint_reason: "Platform capacity planning bounds"
+```
+
+**Authority rule:** `immutable` prevents overrides only from *lower-authority domains*. A `platform` domain field marked `immutable` cannot be changed by `tenant`, `service`, `provider`, or `request` layers โ€” but a `system` domain layer above it can still override it. Higher authority always wins.
+
+**GateKeeper escalation:** A GateKeeper policy at a higher authority level may additionally lock a field that a layer marked `allow` โ€” this is the compliance escape hatch for mandates the layer author did not anticipate.
+
+**Enforcement point:** Step 3 (Layer Merge) โ€” if a lower-priority layer or consumer request sets a field marked `immutable`, assembly halts with a clear error identifying the conflicting layer and the locking layer.
+
+### 13b.2 Constraint Schema Visibility (Q52)
+
+Constrained fields expose their constraint schema to consumers in the Service Catalog UI and Consumer API at a policy-governed disclosure level.
+
+```yaml
+constraint_visibility:
+  level: 
+  # full:    Show constraint type, bounds, constraint_reason, suggested values
+  # summary: Show bounds only โ€” no reason, no suggestions
+  # hidden:  Field appears free-form; constraint silently enforced at submission
+```
+
+**Profile defaults:**
+
+| Profile | Default Level | Rationale |
+|---------|--------------|-----------|
+| `minimal` | `full` | All context helpful |
+| `dev` | `full` | Developers benefit from full schema |
+| `standard` | `full` | Good developer experience |
+| `prod` | `summary` | Bounds visible; reasons may be sensitive |
+| `fsi` | `summary` | Regulatory constraints may not need full exposure |
+| `sovereign` | `hidden` | Constraint details may be operationally sensitive |
+
+**UI rendering (full mode):**
+```
+VM Size โ€” CPU Count
+  Enter a value between 1 and 32 (whole numbers)
+  Suggested: 2, 4, 8, 16
+  Reason: Platform capacity planning bounds
+```
+
+**API endpoint:** `GET /api/v1/catalog/items/{id}/schema` returns field schemas at the declared visibility level for the authenticated consumer's Tenant profile.
+
+Policy may override the profile default per field or resource type:
+```yaml
+policy:
+  type: transformation
+  rule: >
+    If resource_type == Compute.VirtualMachine
+    AND field.name == cpu_count
+    THEN set: constraint_visibility.level = full
+```
+
+### 13b.3 System Policies โ€” Override Control
+
+| Policy | Rule |
+|--------|------|
+| `LAY-005` | Layer fields declare override intent as allow, constrained, or immutable. The Request Payload Processor enforces override declarations during assembly Step 3. immutable prevents overrides from lower-authority domains only โ€” higher-domain layers may always override. GateKeeper policies may additionally lock allow or constrained fields at runtime. |
+| `LAY-006` | Constraint schemas on constrained fields are visible to consumers in the Service Catalog UI and Consumer API at a policy-governed disclosure level: full (constraint, bounds, reason, suggestions), summary (bounds only), or hidden (enforced but not displayed). Profile sets the default. Policy may override per field or resource type. |
+
+
 ## 13a. Layer System Policies
 
 | Policy | Rule |
@@ -1877,9 +1965,9 @@ request:
 | 4 | How are Service Layers registered and versioned relative to Service Provider registration? | Provider contract | โœ… Resolved โ€” independently versioned; provider declares semver compatibility; cache invalidation on version change (LAY-002) |
 | 5 | Should assembly support conditional layer inclusion? | Assembly flexibility | โœ… Resolved โ€” activation_condition on layers; evaluated in Step 2; references request, tenant, resource type, core layer, and ingress fields (LAY-003) |
 | 6 | How does the layer chain interact with service dependencies? | Dependency model | โœ… Resolved โ€” each dependency has its own layer chain; inherits parent resolved placement context; no consumer declaration inheritance (LAY-004) |
-| 7 | Should `override_preference` be declarable in layer definitions as a hint to the Policy Engine? | Override control | โ“ Unresolved |
-| 8 | When `override_preference: immutable` is set by a Global policy, can a higher-priority Global policy still override it? | Override control precedence | โ“ Unresolved |
-| 9 | Should the `constraint_schema` on a constrained field be visible to consumers in the Service Catalog UI? | Consumer experience | โ“ Unresolved |
+| 7 | Should `override_preference` be declarable in layer definitions as a hint to the Policy Engine? | Override control | โœ… Resolved โ€” override: allow/constrained/immutable enforced by Request Payload Processor at Step 3; GateKeeper may additionally lock (LAY-005) |
+| 8 | When `override_preference: immutable` is set โ€” can a higher-priority policy still override it? | Override control precedence | โœ… Resolved โ€” immutable prevents lower-authority overrides only; higher-domain layers always win; GateKeeper can additionally lock (LAY-005) |
+| 9 | Should the `constraint_schema` on a constrained field be visible to consumers in the Service Catalog UI? | Consumer experience | โœ… Resolved โ€” full/summary/hidden disclosure levels; profile-governed defaults; API endpoint returns schema at declared visibility (LAY-006) |
 | 10 | Should the background validation job for detecting post-ingestion conflicts run on a schedule or be event-triggered? | Operational | โ“ Unresolved |
 | 11 | What is the minimum validation review period for a proposed policy before it can be activated? | Policy governance | โ“ Unresolved |
 

From 8505e507e162b59243dc81a4850472620c6ac1c1 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 21:50:22 -0500
Subject: [PATCH 12/49] Updated states document, UUID preservation on
 rehydration, pinned auth level, concurrent rehydration, discovered state
 retention.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        |  89 +++++++++-
 .../architecture/data-model/four-states.md    | 160 +++++++++++++++++-
 2 files changed, 237 insertions(+), 12 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index ba184e9..aa92011 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2363,7 +2363,79 @@ ENT-010.
 
 ---
 
-## SECTION 39 โ€” PERSONAS
+## SECTION 39 โ€” FOUR STATES OPERATIONAL GAPS
+
+### 39.1 Entity UUID Preservation on Rehydration (Q75)
+
+Entity UUIDs are **preserved on rehydration** โ€” UUID is the stable logical identity across all provider migrations, sovereignty changes, and lifecycle events. All external references (CMDB, cost attribution, audit trails, relationships, dependencies) use UUID. Generating a new UUID would silently break all references.
+
+What changes: the **provider-side identifier** (actual VM ID, container name, resource handle). Recorded in `rehydration_history`:
+```yaml
+entity:
+  uuid:     # PRESERVED
+  rehydration_history:
+    - rehydration_uuid: 
+      from_realized_entity_id: "vm-12345"   # no longer valid
+      to_realized_entity_id: "vm-67890"     # new provider ID
+      trigger / from_provider / to_provider / rehydrated_by
+      intent_state_ref / previous_requested_state_ref / new_requested_state_ref
+```
+
+**Rehydration is transactional** โ€” failure preserves pre-rehydration state completely; no UUID change, no partial state. RHY-005.
+
+### 39.2 Pinned Authentication Level for Rehydration (Q76)
+
+Entities may declare `rehydration_constraints.min_auth_level` โ€” a minimum floor the rehydrating actor must meet. Prevents privilege escalation through the rehydration mechanism.
+
+Auth levels (ascending): `api_key โ†’ ldap_password โ†’ oidc โ†’ oidc_mfa โ†’ hardware_token โ†’ hardware_token_mfa`
+
+**Profile enforcement:** minimal/dev = not enforced; standard = advisory warn; prod = enforced reject; fsi = enforced + dual approval on mismatch; sovereign = dual approval always.
+
+**Automated rehydration** (DCM service account for provider migration): requires `allow_delegated_rehydration: true` OR platform admin manual authorization โ†’ full audit trail preserving accountability. RHY-006.
+
+### 39.3 Concurrent Rehydration Handling (Q77)
+
+**Exclusive rehydration lease per entity** โ€” only one rehydration active at a time.
+
+```yaml
+rehydration_lease:
+  entity_uuid / lease_uuid / acquired_by / acquired_at
+  lease_ttl: PT2H    # expires if rehydration hangs
+  trigger / status: active|completed|failed|expired
+```
+
+**Concurrent request:**
+- Active lease + higher priority incoming โ†’ escalate to platform admin; queue
+- Active lease + same/lower priority โ†’ reject with retry guidance; REHYDRATION_BLOCKED audit
+
+**Priority (1=highest):** security/compliance emergency โ†’ manual platform admin โ†’ automated sovereignty migration โ†’ provider decommission โ†’ manual consumer request
+
+**TTL expiry:** marks rehydration `failed`; releases lease; triggers drift detection for partial completion assessment. RHY-007.
+
+### 39.4 Discovered State Retention (Q78)
+
+Ephemeral operational data โ€” NOT the source of truth (Realized State is). Three modes:
+
+| Mode | Behavior |
+|------|---------|
+| `rolling_window` | Keep last N days; useful for trending |
+| `event_driven` | Retain until drift_resolved; ensures investigation has snapshot |
+| `hybrid` (recommended) | min_retention + retain_until_drift_resolved + max_retention ceiling |
+
+**Profile defaults:**
+- minimal: rolling P3D
+- dev: rolling P7D
+- standard/prod: hybrid P24-48H min / P30D max
+- fsi/sovereign: hybrid P7D min / P90D max
+
+**Audit relationship:** Discovered State records are NOT in the Audit Store (too high-volume, too ephemeral). Drift events triggered by Discovered State ARE in the Audit Store with discovery snapshot UUID reference. After snapshot expires: audit record preserved; snapshot no longer available. RHY-008.
+
+### 39.5 Complete Rehydration Policy Set
+RHY-001 through RHY-008 โ€” see doc 02.
+
+---
+
+## SECTION 40 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2380,7 +2452,7 @@ ENT-010.
 
 ---
 
-## SECTION 40 โ€” TERMINOLOGY GLOSSARY
+## SECTION 41 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2443,6 +2515,13 @@ ENT-010.
 | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
 | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
 | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **rehydration_history** | Immutable record on entity of all rehydration events: trigger, from/to provider, from/to provider-side IDs, actor, state refs |
+| **rehydration_lease** | Exclusive time-bounded lock per entity during rehydration; prevents concurrent rehydrations; TTL prevents orphans |
+| **min_auth_level** | Entity rehydration constraint declaring minimum actor authentication level required; profile governs enforcement |
+| **allow_delegated_rehydration** | Entity flag permitting DCM service accounts to rehydrate automatically; requires platform admin authorization audit trail |
+| **REHYDRATION_BLOCKED** | Audit event recorded when a concurrent rehydration attempt is rejected due to active lease |
+| **hybrid retention mode** | Discovered State retention: minimum window + retain until drift resolved + hard maximum ceiling |
+| **rolling_window retention** | Discovered State retention: keep last N days regardless of drift status |
 | **override: allow/constrained/immutable** | Layer field metadata declaring override intent; enforced by Request Payload Processor at Step 3; immutable prevents lower-authority overrides only; GateKeeper may additionally lock |
 | **constraint_visibility** | Policy-governed disclosure level for constrained fields: full (constraint+bounds+reason+suggestions), summary (bounds only), hidden (silently enforced) |
 | **editable** | Resource Type Spec field declaration: can this field be modified post-realization via a targeted delta update (true) or only via reprovisioning (false) |
@@ -2572,7 +2651,7 @@ ENT-010.
 
 ---
 
-## SECTION 41 โ€” OPEN QUESTIONS
+## SECTION 42 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2669,7 +2748,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 42 โ€” DOCUMENTATION STRUCTURE
+## SECTION 43 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2717,7 +2796,7 @@ content/
 
 ---
 
-## SECTION 43 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 44 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md
index 45266d8..f3d0276 100644
--- a/content/docs/architecture/data-model/four-states.md
+++ b/content/docs/architecture/data-model/four-states.md
@@ -1,5 +1,5 @@
 ---
-title: "Four States"
+title: "The Four States and Storage Model"
 type: docs
 weight: 2
 ---
@@ -473,16 +473,162 @@ The three ingress paths โ€” PR workflow, direct API, and programmatic (Terraform
 
 ---
 
+## 7a. Four States Operational Gaps โ€” Q75 through Q78
+
+### 7a.1 Entity UUID Preservation on Rehydration (Q75)
+
+Entity UUIDs are **preserved on rehydration**. The UUID represents the stable logical identity of the resource across provider migrations, sovereignty changes, and lifecycle events. All external references โ€” CMDB records, cost attribution, audit trails, cross-tenant relationships, dependency declarations โ€” reference the entity by UUID. Generating a new UUID on rehydration would silently break all of those references.
+
+What changes on rehydration is the **provider-side identifier** โ€” the actual VM ID, container name, or resource handle at the provider. These are recorded in the rehydration history:
+
+```yaml
+entity:
+  uuid:               # PRESERVED across all rehydrations
+  rehydration_history:
+    - rehydration_uuid: 
+      rehydrated_at: 
+      trigger: 
+      from_provider_uuid: 
+      to_provider_uuid: 
+      from_realized_entity_id: "vm-12345"   # provider's ID โ€” no longer valid
+      to_realized_entity_id: "vm-67890"     # new provider's ID after rehydration
+      rehydrated_by: 
+      intent_state_ref: 
+      previous_requested_state_ref: 
+      new_requested_state_ref: 
+```
+
+**Rehydration is transactional:** If the target provider cannot accept the entity (capacity unavailable, sovereignty mismatch discovered mid-rehydration), the original entity remains in its current state with no UUID change and no partial state. Failure preserves the pre-rehydration state completely.
+
+### 7a.2 Pinned Authentication Level for Rehydration (Q76)
+
+Entities may declare a minimum authentication level required to rehydrate them. This prevents escalation of privilege through the rehydration mechanism โ€” a resource provisioned with hardware-token MFA authorization should not be re-instantiatable by a simple API key.
+
+```yaml
+entity:
+  rehydration_constraints:
+    min_auth_level: hardware_token_mfa
+    # Ascending levels: api_key | ldap_password | oidc | oidc_mfa |
+    #                   hardware_token | hardware_token_mfa
+    auth_level_source: 
+    allow_delegated_rehydration: false
+    # true = DCM service accounts may rehydrate if explicitly authorized
+```
+
+**Profile-governed enforcement:**
+
+| Profile | Enforcement |
+|---------|------------|
+| `minimal` | Not enforced โ€” any auth level may rehydrate |
+| `dev` | Not enforced |
+| `standard` | Advisory โ€” warn if rehydrating actor has lower auth |
+| `prod` | Enforced โ€” reject if rehydrating actor has lower auth |
+| `fsi` | Enforced โ€” dual approval required if auth level mismatch |
+| `sovereign` | Enforced โ€” dual approval always; logged in classified audit |
+
+**Automated rehydration:** When DCM triggers rehydration automatically (sovereignty violation, provider decommission), the rehydration uses DCM's internal service account. This requires `allow_delegated_rehydration: true` OR a platform admin must manually authorize the operation. Authorization produces an audit record preserving accountability even when the action is automated.
+
+### 7a.3 Concurrent Rehydration Handling (Q77)
+
+Rehydration requests acquire an **exclusive rehydration lease** per entity. Only one rehydration may be active per entity at any time.
+
+```yaml
+rehydration_lease:
+  entity_uuid: 
+  lease_uuid: 
+  acquired_by: 
+  acquired_at: 
+  lease_ttl: PT2H                     # expires after 2 hours if not released
+  trigger: 
+  status: 
+```
+
+**Concurrent request handling:**
+
+```
+Second rehydration attempt arrives for entity 
+  โ”‚
+  โ”œโ”€โ”€ No active lease โ†’ acquire lease; proceed
+  โ”‚
+  โ””โ”€โ”€ Active lease exists:
+        Priority higher than active โ†’ escalate to platform admin; queue
+        Same or lower priority โ†’ reject:
+          "Rehydration in progress โ€” lease held since ; retry after PT2H"
+        REHYDRATION_BLOCKED audit event recorded
+```
+
+**Priority ordering:**
+1. Security/compliance emergency (sovereignty violation at fsi/sovereign)
+2. Manual platform admin rehydration
+3. Automated sovereignty migration
+4. Provider decommission migration
+5. Manual consumer rehydration request
+
+**Lease TTL expiry:** If rehydration hangs or crashes, the lease expires after TTL. DCM marks the rehydration `failed` in rehydration_history, releases the lease, and triggers drift detection to assess partial completion at the provider.
+
+### 7a.4 Discovered State Retention (Q78)
+
+Discovered State is ephemeral operational data โ€” not the authoritative source of truth (Realized State is). It is a snapshot used for drift detection. Three retention modes, all profile-governed:
+
+```yaml
+discovered_state_retention:
+  mode:    # hybrid recommended
+
+  rolling_window:
+    retention: P7D              # keep last 7 days; useful for trending
+
+  event_driven:
+    retain_until: drift_resolved  # keep until associated drift record resolved
+    # Ensures drift investigation has the discovery snapshot that triggered it
+
+  hybrid:                         # recommended โ€” combines both
+    minimum_retention: P24H
+    retain_until: drift_resolved  # extend beyond minimum until drift resolved
+    maximum_retention: P30D       # hard ceiling regardless of drift status
+```
+
+**Profile-governed defaults:**
+
+| Profile | Mode | Min Retention | Max Retention |
+|---------|------|--------------|--------------|
+| `minimal` | `rolling_window` | โ€” | P3D |
+| `dev` | `rolling_window` | โ€” | P7D |
+| `standard` | `hybrid` | P24H | P30D |
+| `prod` | `hybrid` | P48H | P30D |
+| `fsi` | `hybrid` | P7D | P90D |
+| `sovereign` | `hybrid` | P7D | P90D |
+
+**Discovered State and the Audit Store:**
+
+Discovered State records are **NOT** stored in the Audit Store โ€” they are too high-volume and too ephemeral for compliance-grade storage. However, drift events triggered by Discovered State ARE recorded in the Audit Store with a reference to the discovery snapshot UUID. After the Discovered State expires, the audit record still exists โ€” it cannot link to the full snapshot, but the drift event itself is preserved.
+
+---
+
+## 7b. Rehydration System Policies โ€” Complete Set
+
+| Policy | Rule |
+|--------|------|
+| `RHY-001` | Tenancy and sovereignty are always current on rehydration โ€” they cannot be pinned to historical state. |
+| `RHY-002` | Sovereignty conflicts discovered during rehydration place the entity in PENDING_REVIEW state. |
+| `RHY-003` | Resource allocations are not automatically released on rehydration. |
+| `RHY-004` | Rehydration leases have TTL to prevent orphaned lease states. |
+| `RHY-005` | Entity UUIDs are preserved on rehydration. The UUID represents stable logical identity across provider migrations. Provider-side identifiers change on rehydration and are recorded in rehydration_history. Rehydration is transactional โ€” failure preserves pre-rehydration state without UUID change. |
+| `RHY-006` | Entities may declare min_auth_level for rehydration. Profile governs enforcement. Automated rehydration by DCM service accounts requires allow_delegated_rehydration: true OR platform admin manual authorization with full audit trail. |
+| `RHY-007` | Rehydration requests acquire an exclusive lease per entity before proceeding. Only one rehydration may be active per entity. Concurrent requests are queued (higher priority) or rejected (same/lower). Lease TTL prevents indefinite blocking. Expiry triggers drift detection for partial completion assessment. |
+| `RHY-008` | Discovered State retention is profile-governed: rolling_window, event_driven, or hybrid. Discovered State is never stored in the Audit Store. Drift events triggered by Discovered State are recorded in the Audit Store with discovery snapshot UUID reference. Maximum retention: P30D for standard/prod; P90D for fsi/sovereign. |
+
+---
+
 ## 8. Open Questions
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | Git repository structure for Intent and Requested stores โ€” deferred pending Q54 resolution | Store design | โ“ Unresolved |
-| 2 | Should the entity UUID be preserved or regenerated on rehydration? | Entity identity | โ“ Unresolved |
-| 3 | For pinned policy version rehydration โ€” what is the minimum authorization level required? | Security | โ“ Unresolved |
-| 4 | How are concurrent rehydration requests for the same entity handled โ€” serialized or rejected? | Concurrency | โ“ Unresolved |
-| 5 | Should the Discovered Store retain full history or only a configurable window? | Retention | โ“ Unresolved |
-| 6 | How does the Search Index handle Git store unavailability โ€” serve stale results or fail? | Reliability | โ“ Unresolved |
+| 1 | Git repository structure for Intent and Requested stores | Store design | โœ… Resolved โ€” handle-based directory structure; 4 repos; tenant isolation (STO-005) |
+| 2 | Should the entity UUID be preserved or regenerated on rehydration? | Entity identity | โœ… Resolved โ€” UUID preserved; rehydration_history records provider-side ID changes; transactional (RHY-005) |
+| 3 | For pinned policy version rehydration โ€” what is the minimum authorization level required? | Security | โœ… Resolved โ€” min_auth_level on entity; profile-governed enforcement; delegated rehydration requires explicit authorization (RHY-006) |
+| 4 | How are concurrent rehydration requests for the same entity handled? | Concurrency | โœ… Resolved โ€” exclusive rehydration lease; priority ordering; TTL expiry triggers drift detection (RHY-007) |
+| 5 | Should the Discovered Store retain full history or only a configurable window? | Retention | โœ… Resolved โ€” hybrid mode recommended; profile-governed min/max; event-driven until drift resolved; max P30-90D (RHY-008) |
+| 6 | How does the Search Index handle Git store unavailability? | Reliability | โœ… Resolved โ€” serve degraded (warn + direct to authoritative); rebuild on recovery (STO-002) |
 
 ---
 

From 5182be0324ad4499bf07fc11b3556388d94bf0c2 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 22:12:52 -0500
Subject: [PATCH 13/49] Terminology replacement, cache model update, signal
 priority updates, deployment bootstrap info.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        | 80 +++++++++++++++++--
 .../data-model/context-and-purpose.md         | 16 ++--
 2 files changed, 82 insertions(+), 14 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index aa92011..c0df9bb 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2435,7 +2435,66 @@ RHY-001 through RHY-008 โ€” see doc 02.
 
 ---
 
-## SECTION 40 โ€” PERSONAS
+## SECTION 40 โ€” ARCHITECTURE GAPS: CACHE MODEL, INGESTION, DEPLOYMENT
+
+### 40.1 DCM Deployment Topology โ€” Hub/Regional/Sovereign
+
+The Ship/Shore/Enclave terminology from defense IT contexts has been replaced throughout with universally understood terms aligned to DCM's federation model:
+
+| Former Term | Replacement | Meaning |
+|-------------|-------------|---------|
+| Shore | **Hub DCM** | Central/global instance; governance origin; authoritative registry |
+| Ship | **Regional DCM** | Distributed regional instance; manages resources in its region |
+| Enclave | **Sovereign DCM** | Air-gapped/compliance-isolated; signed bundle updates only |
+
+These map directly onto the DCM federation model (DCM-001 through DCM-008, doc 22).
+
+### 40.2 Cache Placement (Q1)
+Caches placed closest to consumption point, subject to sovereignty constraints:
+- **Layer + Catalog caches**: Regional DCM (closest to assembly); Hub DCM is authoritative origin
+- **Information Provider caches**: co-located with source; never cross sovereignty boundary
+- **Search Index**: Hub DCM primary; Regional DCM optional mirror; Sovereign DCM local index
+- **Sovereign DCM caches**: always local static; populated from signed bundles; no live sync
+
+CACHE-001.
+
+### 40.3 Cache Synchronization (Q2)
+**Hybrid push-pull model:** Pull on profile-governed schedule (minimal=P4H through fsi/sovereign=P5M); push invalidation via Message Bus for time-sensitive events (layer_updated, policy_activated, sovereignty_change). Pull failure โ†’ serve cached. Push failure โ†’ queue+retry within PT1H. Sovereign DCM: signed bundle import only. CACHE-002.
+
+### 40.4 Cache Authoritativeness (Q3)
+GitOps stores and Event Streams always authoritative. Caches are derived projections โ€” never authoritative. Cache divergence โ†’ rebuild from nearest authoritative store โ†’ CACHE_DIVERGENCE_DETECTED Audit event. Regional DCMs send cache state hash in heartbeats to Hub DCM; mismatch โ†’ forced refresh. CACHE-003, CACHE-004.
+
+### 40.5 Native Passthrough (Q5)
+Core data model does not embed technology-specific data. `native_passthrough` field sanctioned for genuinely untranslatable provider-specific data. Always audit-logged (content if transparent; hash if opaque). Opaque passthrough blocked in fsi/sovereign profiles by default. DATA-001.
+
+### 40.6 Physical State Representation (Q6)
+Intent/Requested โ†’ YAML in Git. Realized โ†’ Event Stream โ†’ Realized Store. Discovered โ†’ Discovered Store (ephemeral). Resolved by STO-001 through STO-005.
+
+### 40.7 Ingestion Model Gaps
+
+**Signal priority configurable (Q1):** Platform domain layer declares priority order. explicit_tenant_tag always first; default_tenant always last; middle signals reorderable. ING-012.
+
+**Bulk promotion (Q2):** Supported with profile-governed limits (minimal=unlimited โ†’ sovereign=25 per batch). Preview required. PT24H rollback. BULK_PROMOTE audit record. ING-013.
+
+**Max ingestion sources per entity (Q3):** Profile-governed (5 standard/prod; 3 fsi/sovereign). Warn or reject on exceed. ING-014.
+
+**Ingestion โ†’ Service Catalog (Q4):** Ingested entities promotable to catalog items. Bidirectional drift detection between entity and catalog item. ING-015.
+
+### 40.8 Deployment Redundancy Gaps
+
+**Bootstrap manifest verification (Q1):** GitOps stored + hash-verified at every startup. Tampering prevents startup. Operator-signed. RED-011.
+
+**Sovereign Kubernetes upgrades (Q2):** Pre-staged images via signed bundles. DCM maintenance mode during upgrade. Startup verification before resuming queued requests. RED-012.
+
+**Non-Kubernetes runtimes (Q3):** Kubernetes required for production. Podman/Docker Compose for dev/community only. DCM Operator is Kubernetes-native. RED-013.
+
+**Minimum hardware specs (Q4):** Declared as DCM Resource definitions per profile; enforced by placement engine. minimal=2cpu/4Gi/20Gi/1 replica โ†’ fsi/sovereign=32cpu/64Gi/500Gi/5 replicas. RED-014.
+
+**DCM self-hosted drift detection (Q5):** DCM is a DCM-managed resource โ€” same drift detection. Bootstrap hash provides independent Operator verification. Audit hash chain breaks externally detectable. RED-015.
+
+---
+
+## SECTION 41 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2452,7 +2511,7 @@ RHY-001 through RHY-008 โ€” see doc 02.
 
 ---
 
-## SECTION 41 โ€” TERMINOLOGY GLOSSARY
+## SECTION 42 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2515,6 +2574,15 @@ RHY-001 through RHY-008 โ€” see doc 02.
 | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
 | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
 | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **Hub DCM** | Central/global DCM instance; authoritative registry origin; governance authority; replaces "Shore" terminology |
+| **Regional DCM** | Distributed regional DCM instance; manages resources in its region; caches from Hub DCM; replaces "Ship" terminology |
+| **Sovereign DCM** | Air-gapped or compliance-isolated DCM instance; local static caches from signed bundles; replaces "Enclave" terminology |
+| **native_passthrough** | Sanctioned field for provider-specific data that cannot be expressed in the unified model; always audit-logged; opaque mode blocked in fsi/sovereign |
+| **BULK_PROMOTE** | Audit action for bulk entity promotion; single audit record with full member list; requires preview + approval in prod+ profiles |
+| **CACHE policies** | CACHE-001 through CACHE-004 โ€” cache placement, sync, authoritativeness, consistency |
+| **DATA-001** | Policy: core data model does not embed technology-specific data; native_passthrough sanctioned with governance |
+| **RED-011 through RED-015** | Deployment redundancy gap policies: bootstrap verification, K8s upgrades, runtime support, hardware specs, self-hosted drift |
+| **ING-012 through ING-015** | Ingestion gap policies: signal priority configurable, bulk promotion, max sources, catalog promotion |
 | **rehydration_history** | Immutable record on entity of all rehydration events: trigger, from/to provider, from/to provider-side IDs, actor, state refs |
 | **rehydration_lease** | Exclusive time-bounded lock per entity during rehydration; prevents concurrent rehydrations; TTL prevents orphans |
 | **min_auth_level** | Entity rehydration constraint declaring minimum actor authentication level required; profile governs enforcement |
@@ -2651,13 +2719,13 @@ RHY-001 through RHY-008 โ€” see doc 02.
 
 ---
 
-## SECTION 42 โ€” OPEN QUESTIONS
+## SECTION 43 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
 | # | Question | Area |
 |---|----------|------|
-| 1 | Where should data caches live? (Shore, Ship, Enclave, all?) | Data Model |
+| 1 | Where should data caches live? (Hub DCM, Regional DCM, Sovereign DCM, all?) | Data Model |
 | 2 | Should cache synchronization be push, pull, or both? | Data Model |
 | 3 | Which cache is authoritative when caches diverge? | Data Model |
 | 4 | What mechanism maintains consistency across distributed caches? | Data Model |
@@ -2748,7 +2816,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 43 โ€” DOCUMENTATION STRUCTURE
+## SECTION 44 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2796,7 +2864,7 @@ content/
 
 ---
 
-## SECTION 44 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 45 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md
index 8b10ffe..2413af8 100644
--- a/content/docs/architecture/data-model/context-and-purpose.md
+++ b/content/docs/architecture/data-model/context-and-purpose.md
@@ -12,7 +12,7 @@ weight: 1
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Entity Types](01-entity-types.md) | [Four States](../four-states/) | [Layering and Versioning](../layering-and-versioning/) | [Examples](04-examples.md)
+**Related Documents:** [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md)
 
 ---
 
@@ -241,7 +241,7 @@ All DCM artifacts follow a five-status lifecycle:
 
 **Proposed shadow execution:** Policy artifacts in `proposed` status execute in shadow mode against real traffic โ€” output is captured and reported but never applied. Enables safe validation before activation.
 
-See [Data Layers and Assembly โ€” Section 4b](../layering-and-versioning/) for the complete artifact metadata structure and all field definitions.
+See [Data Layers and Assembly โ€” Section 4b](03-layering-and-versioning.md) for the complete artifact metadata structure and all field definitions.
 
 ---
 
@@ -349,12 +349,12 @@ The following questions remain unresolved and require decisions before the data
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | Where should data caches live? (Shore, Ship, Enclave, all?) | Cache architecture, latency, sovereignty | โ“ Unresolved |
-| 2 | Should cache synchronization be push, pull, or both? | Consistency model, network requirements | โ“ Unresolved |
-| 3 | Which cache is authoritative when caches diverge? | Conflict resolution, drift detection | โ“ Unresolved |
-| 4 | What mechanism maintains consistency across distributed caches? | Data integrity, operational complexity | โ“ Unresolved |
-| 5 | Should the data model allow embedded target-technology-specific data bundles? | Portability vs. practicality | โ“ Unresolved |
-| 6 | How are the four states represented physically? (fields, documents, separate stores?) | Implementation architecture | โ“ Unresolved |
+| 1 | Where should data caches live? | Cache architecture, latency, sovereignty | โœ… Resolved โ€” caches at Regional DCM level; Info Provider caches co-located with source; Sovereign DCM uses local static from signed bundles (CACHE-001) |
+| 2 | Should cache synchronization be push, pull, or both? | Consistency model | โœ… Resolved โ€” hybrid push-pull; pull on schedule; push for time-sensitive events via Message Bus (CACHE-002) |
+| 3 | Which cache is authoritative when caches diverge? | Conflict resolution | โœ… Resolved โ€” GitOps stores always authoritative; caches are projections; divergence triggers rebuild from authoritative store (CACHE-003) |
+| 4 | What mechanism maintains consistency across distributed caches? | Data integrity | โœ… Resolved โ€” hash-based heartbeat divergence detection + push invalidation; PT2H staleness alert; signed bundles for Sovereign DCM (CACHE-004) |
+| 5 | Should the data model allow embedded target-technology-specific data bundles? | Portability | โœ… Resolved โ€” native_passthrough field sanctioned; always audit-logged; opaque mode blocked in fsi/sovereign (DATA-001) |
+| 6 | How are the four states represented physically? | Physical model | โœ… Resolved โ€” Intent/Requested in Git; Realized in Event Stream; Discovered in Discovered Store (STO-005) |res?) | Implementation architecture | โ“ Unresolved |
 | 7 | What is the performance impact of field-level provenance at scale? What optimization strategies are acceptable? | Scalability, storage cost | โ“ Unresolved |
 | 8 | Should provenance metadata be stored inline with field data or in a linked provenance document? | Data model structure, query performance | โ“ Unresolved |
 

From 83863a42719526c0def4a7533f4fe3faf62f7a02 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 22:24:13 -0500
Subject: [PATCH 14/49] consolidate answers into other relevant sections.
 Provenance model, validat, policy review, governance, grouping, relationship
 role validation, information providers.

Signed-off-by: Chris Roadfeldt 
---
 .../data-model/context-and-purpose.md            |  4 ++--
 .../data-model/entity-relationships.md           |  4 ++--
 .../data-model/information-providers.md          | 12 ++++++------
 .../data-model/layering-and-versioning.md        |  4 ++--
 .../architecture/data-model/resource-grouping.md | 12 ++++++------
 .../data-model/resource-type-hierarchy.md        | 16 ++++++++--------
 6 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md
index 2413af8..50c7ceb 100644
--- a/content/docs/architecture/data-model/context-and-purpose.md
+++ b/content/docs/architecture/data-model/context-and-purpose.md
@@ -355,8 +355,8 @@ The following questions remain unresolved and require decisions before the data
 | 4 | What mechanism maintains consistency across distributed caches? | Data integrity | โœ… Resolved โ€” hash-based heartbeat divergence detection + push invalidation; PT2H staleness alert; signed bundles for Sovereign DCM (CACHE-004) |
 | 5 | Should the data model allow embedded target-technology-specific data bundles? | Portability | โœ… Resolved โ€” native_passthrough field sanctioned; always audit-logged; opaque mode blocked in fsi/sovereign (DATA-001) |
 | 6 | How are the four states represented physically? | Physical model | โœ… Resolved โ€” Intent/Requested in Git; Realized in Event Stream; Discovered in Discovered Store (STO-005) |res?) | Implementation architecture | โ“ Unresolved |
-| 7 | What is the performance impact of field-level provenance at scale? What optimization strategies are acceptable? | Scalability, storage cost | โ“ Unresolved |
-| 8 | Should provenance metadata be stored inline with field data or in a linked provenance document? | Data model structure, query performance | โ“ Unresolved |
+| 7 | What is the performance impact of field-level provenance at scale? What optimization strategies are acceptable? | Scalability, storage cost | โœ… Resolved โ€” three configurable provenance models: full_inline, deduplicated (Model B recommended), tiered; profile-appropriate Policy Groups; see docs 03 and 06 (OPS-001) |
+| 8 | Should provenance metadata be stored inline with field data or in a linked provenance document? | Data model structure, query performance | โœ… Resolved โ€” three-level structure: implicit chain ref, inline delta, linked history document; all reconstructable from stored facts; see doc 03 (OPS-002) |
 
 ---
 
diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md
index de6642b..ee7ce8c 100644
--- a/content/docs/architecture/data-model/entity-relationships.md
+++ b/content/docs/architecture/data-model/entity-relationships.md
@@ -915,9 +915,9 @@ Relationships follow the universal versioning and deprecation model. A relations
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
 | 1 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Policy model | โœ… Resolved โ€” standard Policy Engine authority hierarchy; REL-008 and REL-009 |
-| 2 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Operational complexity | โ“ Unresolved |
+| 2 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Operational complexity | โœ… Resolved โ€” advisory default; Resource Type Spec may declare permitted_relationship_roles with role_validation: advisory/enforced; community role catalog; see doc 09 Section 12 (REL-020) |
 | 3 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Multi-tenancy | โœ… Resolved โ€” nature governs; constituent never; operational with dual auth; informational unless deny_all; REL-010/011/012 |
-| 4 | Should there be a maximum relationship graph depth to prevent runaway complexity? | Operational governance | โ“ Unresolved |
+| 4 | Should there be a maximum relationship graph depth to prevent runaway complexity? | Operational governance | โœ… Resolved โ€” profile-governed max depth: 15 standard/prod, 10 fsi/sovereign; circular detection always enforced; depth = traversal distance; see doc 09 Section 12 (REL-021) |
 | 5 | How are shared entities represented in the relationship graph โ€” an entity required by multiple parents? | Graph model | โœ… Resolved โ€” sharing_model declaration; active_relationship_count; save_overrides_destroy hierarchy (REL-018); lifecycle_conflict_record; REL-015 through REL-019 |
 
 ---
diff --git a/content/docs/architecture/data-model/information-providers.md b/content/docs/architecture/data-model/information-providers.md
index 0bcf996..8dfadea 100644
--- a/content/docs/architecture/data-model/information-providers.md
+++ b/content/docs/architecture/data-model/information-providers.md
@@ -12,7 +12,7 @@ weight: 9
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Resource Type Hierarchy](../resource-type-hierarchy/) | [Resource/Service Entities](../resource-service-entities/) | [Entity Relationships](../entity-relationships/)
+**Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md)
 
 ---
 
@@ -419,11 +419,11 @@ This means an organization can start with an external Information Provider refer
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | How are conflicting provider push events handled โ€” two Information Providers claim authority for the same record? | Data integrity | โ“ Unresolved |
-| 2 | Should Information Providers support write-back โ€” DCM updating external records via the provider? | Scope expansion | โ“ Unresolved |
-| 3 | How is the extended schema versioned โ€” if a provider adds or removes extended fields, how are existing references affected? | Versioning | โ“ Unresolved |
-| 4 | Should DCM maintain a registry of well-known Information Providers (HR systems, finance systems) to simplify onboarding? | Adoption | โ“ Unresolved |
-| 5 | How does the verification model interact with air-gapped environments where Information Providers may be unreachable? | Sovereignty | โ“ Unresolved |
+| 1 | How are conflicting provider push events handled โ€” two Information Providers claim authority for the same record? | Data integrity | โœ… Resolved โ€” authority_level (primary/secondary/advisory) + authority_scope; conflict_resolution strategies; ingestion-time conflict detection; conflict records; see doc 21 (INF-001) |
+| 2 | Should Information Providers support write-back โ€” DCM updating external records via the provider? | Scope expansion | โœ… Resolved โ€” optional declared capability; policy-triggered write-back; audit records produced; credentials via Credential Provider; see doc 21 (INF-002) |
+| 3 | How is the extended schema versioned โ€” if a provider adds or removes extended fields, how are existing references affected? | Versioning | โœ… Resolved โ€” semver semantics on extended schema; field removal/type change = major; new optional field = minor; migration plan required for major bumps; see doc 21 (INF-003) |
+| 4 | Should DCM maintain a registry of well-known Information Providers (HR systems, finance systems) to simplify onboarding? | Adoption | โœ… Resolved โ€” three-tier Information Provider Registry (Core/Community/Organization); same governance model as Resource Type Registry; separate registries; see doc 21 (INF-004) |
+| 5 | How does the verification model interact with air-gapped environments where Information Providers may be unreachable? | Sovereignty | โœ… Resolved โ€” three air-gap modes: pre-verified signed bundle, internal mTLS, periodic online re-verification with cached tokens; profile-governed cache expiry (prod/fsi/sovereign=suspend on expiry); see doc 21 (INF-005) |
 
 ---
 
diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md
index d32c006..e6d2e7a 100644
--- a/content/docs/architecture/data-model/layering-and-versioning.md
+++ b/content/docs/architecture/data-model/layering-and-versioning.md
@@ -1968,8 +1968,8 @@ policy:
 | 7 | Should `override_preference` be declarable in layer definitions as a hint to the Policy Engine? | Override control | โœ… Resolved โ€” override: allow/constrained/immutable enforced by Request Payload Processor at Step 3; GateKeeper may additionally lock (LAY-005) |
 | 8 | When `override_preference: immutable` is set โ€” can a higher-priority policy still override it? | Override control precedence | โœ… Resolved โ€” immutable prevents lower-authority overrides only; higher-domain layers always win; GateKeeper can additionally lock (LAY-005) |
 | 9 | Should the `constraint_schema` on a constrained field be visible to consumers in the Service Catalog UI? | Consumer experience | โœ… Resolved โ€” full/summary/hidden disclosure levels; profile-governed defaults; API endpoint returns schema at declared visibility (LAY-006) |
-| 10 | Should the background validation job for detecting post-ingestion conflicts run on a schedule or be event-triggered? | Operational | โ“ Unresolved |
-| 11 | What is the minimum validation review period for a proposed policy before it can be activated? | Policy governance | โ“ Unresolved |
+| 10 | Should the background validation job for detecting post-ingestion conflicts run on a schedule or be event-triggered? | Operational | โœ… Resolved โ€” event-triggered primary (on layer ingestion/update) + weekly scheduled sweep safety net; async non-blocking; both produce same conflict record format (OPS-003) |
+| 11 | What is the minimum validation review period for a proposed policy before it can be activated? | Policy governance | โœ… Resolved โ€” GateKeeper=14d, Validation=7d, Transformation=3d ร— profile multiplier (minimal=0ร—, dev=0.5ร—, standard=1ร—, prod=1.5ร—, fsi/sovereign=2ร—); DCM enforces; emergency bypass requires dual-approval audit (OPS-004) |
 
 ---
 
diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md
index 5c7f31f..32a4895 100644
--- a/content/docs/architecture/data-model/resource-grouping.md
+++ b/content/docs/architecture/data-model/resource-grouping.md
@@ -1,5 +1,5 @@
 ---
-title: "Resource Grouping"
+title: "Resource Grouping and Tenancy"
 type: docs
 weight: 7
 ---
@@ -278,11 +278,11 @@ custom_group_type_registration:
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | Should there be a DCM-maintained registry of well-known custom group types to encourage standardization? | Interoperability | โ“ Unresolved |
-| 2 | How does group membership interact with sovereignty โ€” can a group span sovereignty boundaries? | Sovereignty model | โ“ Unresolved |
-| 3 | When a Tenant is decommissioned, what happens to its resources and group memberships? | Lifecycle management | โ“ Unresolved |
-| 4 | Should Resource Groups support time-bounded membership โ€” a resource belongs to a group for a defined period? | Operational flexibility | โ“ Unresolved |
-| 5 | How are group-level policies inherited by nested child groups โ€” is inheritance opt-in or opt-out? | Policy model | โ“ Unresolved |
+| 1 | Should there be a DCM-maintained registry of well-known custom group types to encourage standardization? | Interoperability | โœ… Resolved โ€” group_subclass open and advisory; community subclass catalog as non-authoritative reference; no validation or enforcement; see doc 15 (GRP-011) |
+| 2 | How does group membership interact with sovereignty โ€” can a group span sovereignty boundaries? | Sovereignty model | โœ… Resolved โ€” class-specific: tenant_boundary never cross-sovereignty (structural); resource_grouping permitted+policy restriction; policy_collection always permitted; composite governed by most restrictive member; see doc 15 (GRP-012) |
+| 3 | When a Tenant is decommissioned, what happens to its resources and group memberships? | Lifecycle management | โœ… Resolved โ€” four-phase staged decommission: pre-validation โ†’ resource decommission โ†’ membership cleanup โ†’ audit archival; child groups must be resolved first; audit records never destroyed; see doc 15 (GRP-013) |
+| 4 | Should Resource Groups support time-bounded membership โ€” a resource belongs to a group for a defined period? | Operational flexibility | โœ… Resolved โ€” valid_from/valid_until already in Universal Group Model; on_expiry (remove/notify/suspend_member); Lifecycle Constraint Enforcer handles; MEMBER_REMOVE audit record; see doc 15 (GRP-014) |
+| 5 | How are group-level policies inherited by nested child groups โ€” is inheritance opt-in or opt-out? | Policy model | โœ… Resolved โ€” class-specific defaults profile-governed; tenant_boundary: opt_out (standard/prod), opt_in (fsi/sovereign); federation always opt_in; composite opt_out; see doc 15 (GRP-015) |
 
 ---
 
diff --git a/content/docs/architecture/data-model/resource-type-hierarchy.md b/content/docs/architecture/data-model/resource-type-hierarchy.md
index 091bfaa..c30bacb 100644
--- a/content/docs/architecture/data-model/resource-type-hierarchy.md
+++ b/content/docs/architecture/data-model/resource-type-hierarchy.md
@@ -1,5 +1,5 @@
 ---
-title: "Resource Type Hierarchy"
+title: "Resource Type Hierarchy and Service Catalog"
 type: docs
 weight: 4
 ---
@@ -12,7 +12,7 @@ weight: 4
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Context and Purpose](../context-and-purpose/) | [Entity Types](01-entity-types.md) | [Four States](../four-states/) | [Layering and Versioning](../layering-and-versioning/) | [Examples](04-examples.md)
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md)
 
 ---
 
@@ -425,12 +425,12 @@ Once a version is published it is immutable. Any change โ€” even a documentation
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | What is the governance model for proposing and approving new Resource Types to the DCM registry? | Community adoption, quality control | โ“ Unresolved |
-| 2 | Should the registry support a formal review/approval workflow before a Resource Type becomes `active`? | Registry integrity, community trust | โ“ Unresolved |
-| 3 | What is the minimum sunset period for deprecated definitions? | Migration planning, operational stability | โ“ Unresolved |
-| 4 | Should version constraints in requests be strictly enforced or advisory? | Operational flexibility vs. predictability | โ“ Unresolved |
-| 5 | How are conflicts resolved when multiple providers satisfy all narrowing criteria equally? | Request resolution determinism | โ“ Unresolved |
-| 6 | Should the registry be distributed or centralized? How does this interact with sovereignty requirements? | Registry availability, sovereignty | โ“ Unresolved |
+| 1 | What is the governance model for proposing and approving new Resource Types to the DCM registry? | Community adoption, quality control | โœ… Resolved โ€” three-tier registry (DCM Core / Verified Community / Organization); PR-based proposals with automated validation gates; shadow validation period before active promotion; see doc 20 (REG-001, REG-002) |
+| 2 | Should the registry support a formal review/approval workflow before a Resource Type becomes `active`? | Registry integrity, community trust | โœ… Resolved โ€” PR-based workflow with automated gates (schema, FQN conflict, dependency resolution) and mandatory shadow validation before active; review periods by change type; see doc 20 (REG-002) |
+| 3 | What is the minimum sunset period for deprecated definitions? | Migration planning, operational stability | โœ… Resolved โ€” default sunset policies REG-DP-002: Tier 1=P12M, Tier 2=P6M; overridable via standard policy priority; locked as immutable in fsi/sovereign profiles; see doc 20 |
+| 4 | Should version constraints in requests be strictly enforced or advisory? | Operational flexibility vs. predictability | โœ… Resolved โ€” strictly enforced; version_policy options: exact/compatible/latest_minor/latest; DCM never auto-upgrades across major versions; profile-governed defaults (fsi/sovereign=exact); see doc 20 (REG-004) |
+| 5 | How are conflicts resolved when multiple providers satisfy all narrowing criteria equally? | Request resolution determinism | โœ… Resolved โ€” six-step tie-breaking: policy preference โ†’ provider priority โ†’ tenant affinity โ†’ cost analysis (if available) โ†’ least loaded โ†’ consistent hash on request_uuid; see doc 20 (REG-005) |
+| 6 | Should the registry be distributed or centralized? How does this interact with sovereignty requirements? | Registry availability, sovereignty | โœ… Resolved โ€” federated model: DCM Project registry โ†’ Organization mirror โ†’ Sovereign DCM (offline/signed bundles); air-gap via signed bundle import; see doc 20 (REG-006) |
 
 ---
 

From 3178a5edc03a2c36e7d5544508a72d8a9d05e6fc Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 22:40:46 -0500
Subject: [PATCH 15/49] Add SCIM, failover chain, MFA, pluggable storage, Hash
 chain verification, commit log capacity, system initiated records,
 distributed hash chains.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md | 93 ++++++++++++++++++++++++--
 1 file changed, 87 insertions(+), 6 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index c0df9bb..aad91b6 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2443,7 +2443,15 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th
 
 | Former Term | Replacement | Meaning |
 |-------------|-------------|---------|
-| Shore | **Hub DCM** | Central/global instance; governance origin; authoritative registry |
+| Shore | **SCIM 2.0** | System for Cross-domain Identity Management; optional Auth Provider capability for automated actor provisioning from enterprise IdPs; provisions actors and group memberships; roles not SCIM-provisioned |
+| **step-up MFA** | Additional MFA challenge at sensitive operations within an already-authenticated session; declared per operation by policy; step-up token TTL PT10M |
+| **actor.type** | Audit record field: human / service_account / system; enables filtering between human-initiated and automated lifecycle operations in queries and dashboards |
+| **system_actor** | Audit record block on system-initiated records: identifies DCM component, trigger event, and authorizing policy UUID |
+| **Merkle root proof** | Federation-level audit integrity mechanism: Hub DCM computes Merkle root of all Regional DCM chain tips daily; any chain break detectable against stored root |
+| **per-instance hash chain** | Each DCM instance (Hub/Regional/Sovereign) maintains its own independent hash chain; not merged cross-instance; cross-referenced via correlation_id |
+| **AUTH-012 through AUTH-015** | Auth Provider gap policies: SCIM provisioning, failover handling, two-tier MFA, pluggable user store |
+| **AUD-014 through AUD-017** | Universal Audit gap policies: hash chain verification modes, Commit Log capacity, system-initiated records, distributed hash chains |
+| **Hub DCM** | Central/global instance; governance origin; authoritative registry |
 | Ship | **Regional DCM** | Distributed regional instance; manages resources in its region |
 | Enclave | **Sovereign DCM** | Air-gapped/compliance-isolated; signed bundle updates only |
 
@@ -2494,7 +2502,72 @@ Intent/Requested โ†’ YAML in Git. Realized โ†’ Event Stream โ†’ Realized Store.
 
 ---
 
-## SECTION 41 โ€” PERSONAS
+## SECTION 41 โ€” SECURITY, AUTH, AND AUDIT REFINEMENTS
+
+### 41.1 SCIM 2.0 User Provisioning (Auth Q1)
+
+SCIM 2.0 is an optional Auth Provider capability for enterprise deployments. Automates actor lifecycle from enterprise IdPs (Okta, Azure AD, Ping, JumpCloud).
+
+**What SCIM manages:** DCM actor records (create/update/deactivate), DCM group memberships from IdP groups.
+
+**What SCIM does NOT manage:** Roles โ€” they require explicit DCM policy authorization. This prevents privilege escalation through the SCIM channel.
+
+**Deprovisioning:** `suspend` by default (reversible; sessions terminated; leases released; in-flight requests complete first). AUTH-012.
+
+### 41.2 Auth Provider Failover โ€” In-Flight Requests (Auth Q2)
+
+**In-flight requests (already authenticated):** Continue to completion โ€” session token carries resolved roles/groups/tenant scope; Auth Provider not needed for assembly.
+
+**New requests, provider down:** Follow declared failover chain. Sessions valid for declared TTL (PT8H default) regardless of provider availability.
+
+**Session expiry during outage:** Requires re-auth via available failover provider. All providers unavailable โ†’ reject new authentication with clear error.
+
+Failover chain: primary LDAP โ†’ OIDC backup โ†’ local users (last resort). AUTH-013.
+
+### 41.3 Two-Tier MFA โ€” Per-Session and Step-Up (Auth Q3)
+
+**Per-session MFA:** Validated at login; captured in `ingress.actor.mfa_verified`.
+
+**Step-up MFA:** Additional challenge at sensitive operations even within a valid MFA session. Policy declares which operations require step-up:
+- platform_policy_activate, provider_decommission, tenant_decommission
+- sovereignty_zone_change, auth_provider_update, manual_rehydration (if entity requires hardware_token_mfa)
+
+Step-up token TTL: PT10M. Profile defaults: minimal/dev = no MFA; standard = recommended; prod = per-session required + destructive ops step-up; fsi = per-session + all policy changes; sovereign = hardware token + all admin ops. AUTH-014.
+
+### 41.4 Built-In Auth Provider Storage Backend (Auth Q4)
+
+Pluggable storage backend following the Storage Provider model. SQLite (minimal/dev) โ†’ PostgreSQL (standard+). FSI/sovereign require encryption at rest. Local store should contain bootstrap users, service accounts, API keys only โ€” not enterprise users. AUTH-015.
+
+### 41.5 Hash Chain Verification Modes (Audit Q1)
+
+Three independent levels:
+- **Continuous write:** Hash computed on every write โ€” this IS chain construction; always active
+- **Scheduled sweep:** Background verification: standard=weekly, prod=daily, fsi/sovereign=every 6 hours
+- **On-demand:** Operator-triggered for any time range (max P365D per run)
+
+Failure: security alert + integrity incident; new writes continue (halting writes is itself a security risk). AUD-014.
+
+### 41.6 Commit Log Capacity and Overflow (Audit Q2)
+
+Configurable max capacity (default 10Gi) with declared overflow policy:
+- `alert_and_continue` (minimal/dev/standard/prod) โ€” availability priority
+- `reject_new_ops` (fsi/sovereign) โ€” audit completeness priority; operating unaudited is a compliance violation
+
+Backpressure: alert at 75%, urgent at 90%. P7D max age triggers escalation regardless of capacity. AUD-015.
+
+### 41.7 System-Initiated Audit Records (Audit Q3)
+
+`actor.type` field on all audit records: `human | service_account | system`
+
+System records include `system_actor` block: component + trigger + authorizing_policy_uuid. Full audit records โ€” appear in all queries and compliance reports. Enables dashboard filtering: "show only human-initiated changes" vs "show only automated lifecycle operations". AUD-016.
+
+### 41.8 Distributed Hash Chain Integrity (Audit Q4)
+
+Per-instance hash chains โ€” each DCM instance (Hub, Regional, Sovereign) has its own independent chain. Federation-level integrity via daily Merkle root proof at Hub DCM. Cross-instance queries: parallel chains with cross-references via `correlation_id` โ€” not merged. Per-instance verification always local; federation verification requires Hub DCM connectivity. AUD-017.
+
+---
+
+## SECTION 42 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2511,7 +2584,7 @@ Intent/Requested โ†’ YAML in Git. Realized โ†’ Event Stream โ†’ Realized Store.
 
 ---
 
-## SECTION 42 โ€” TERMINOLOGY GLOSSARY
+## SECTION 43 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2574,6 +2647,14 @@ Intent/Requested โ†’ YAML in Git. Realized โ†’ Event Stream โ†’ Realized Store.
 | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
 | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
 | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **SCIM 2.0** | System for Cross-domain Identity Management; optional Auth Provider capability for automated actor provisioning from enterprise IdPs; provisions actors and group memberships; roles not SCIM-provisioned |
+| **step-up MFA** | Additional MFA challenge at sensitive operations within an already-authenticated session; declared per operation by policy; step-up token TTL PT10M |
+| **actor.type** | Audit record field: human / service_account / system; enables filtering between human-initiated and automated lifecycle operations in queries and dashboards |
+| **system_actor** | Audit record block on system-initiated records: identifies DCM component, trigger event, and authorizing policy UUID |
+| **Merkle root proof** | Federation-level audit integrity mechanism: Hub DCM computes Merkle root of all Regional DCM chain tips daily; any chain break detectable against stored root |
+| **per-instance hash chain** | Each DCM instance (Hub/Regional/Sovereign) maintains its own independent hash chain; not merged cross-instance; cross-referenced via correlation_id |
+| **AUTH-012 through AUTH-015** | Auth Provider gap policies: SCIM provisioning, failover handling, two-tier MFA, pluggable user store |
+| **AUD-014 through AUD-017** | Universal Audit gap policies: hash chain verification modes, Commit Log capacity, system-initiated records, distributed hash chains |
 | **Hub DCM** | Central/global DCM instance; authoritative registry origin; governance authority; replaces "Shore" terminology |
 | **Regional DCM** | Distributed regional DCM instance; manages resources in its region; caches from Hub DCM; replaces "Ship" terminology |
 | **Sovereign DCM** | Air-gapped or compliance-isolated DCM instance; local static caches from signed bundles; replaces "Enclave" terminology |
@@ -2719,7 +2800,7 @@ Intent/Requested โ†’ YAML in Git. Realized โ†’ Event Stream โ†’ Realized Store.
 
 ---
 
-## SECTION 43 โ€” OPEN QUESTIONS
+## SECTION 44 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2816,7 +2897,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 44 โ€” DOCUMENTATION STRUCTURE
+## SECTION 45 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2864,7 +2945,7 @@ content/
 
 ---
 
-## SECTION 45 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 46 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 

From 489f85c9255cc21d12cdf4764818096f050512fe Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 22:52:09 -0500
Subject: [PATCH 16/49] Updated policy profile registry and submission
 lifecycle, certification metadata, operational analysis.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md | 68 +++++++++++++++++++++++---
 1 file changed, 62 insertions(+), 6 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index aad91b6..e3c1d03 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2443,7 +2443,12 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th
 
 | Former Term | Replacement | Meaning |
 |-------------|-------------|---------|
-| Shore | **SCIM 2.0** | System for Cross-domain Identity Management; optional Auth Provider capability for automated actor provisioning from enterprise IdPs; provisions actors and group memberships; roles not SCIM-provisioned |
+| Shore | **certified profile** | DCM profile carrying formal third-party certification metadata (HIPAA assessor, FedRAMP JAB, PCI QSA); promoted to Tier 1; applies to artifact not deployment |
+| **POLICY_PROVIDER_ELEVATED** | Audit action recorded when a Policy Provider's mode level is elevated; always produced regardless of profile |
+| **tier_certifications** | Certification metadata on Resource Type Specs or profiles from recognized certifying bodies; filter criterion, not structural tier boundary |
+| **tier_3_to_tier_2_promotion** | PR-based pathway for organizations to promote internal Tier 3 Resource Types to Verified Community (Tier 2); requires production deployment, OSS license, named maintainer, migration path |
+| **independent operation mode** | Registry Provider state when upstream registry is permanently unavailable; existing types continue; new community type adoption requires governance decision |
+| **SCIM 2.0** | System for Cross-domain Identity Management; optional Auth Provider capability for automated actor provisioning from enterprise IdPs; provisions actors and group memberships; roles not SCIM-provisioned |
 | **step-up MFA** | Additional MFA challenge at sensitive operations within an already-authenticated session; declared per operation by policy; step-up token TTL PT10M |
 | **actor.type** | Audit record field: human / service_account / system; enables filtering between human-initiated and automated lifecycle operations in queries and dashboards |
 | **system_actor** | Audit record block on system-initiated records: identifies DCM component, trigger event, and authorizing policy UUID |
@@ -2567,7 +2572,53 @@ Per-instance hash chains โ€” each DCM instance (Hub, Regional, Sovereign) has it
 
 ---
 
-## SECTION 42 โ€” PERSONAS
+## SECTION 42 โ€” POLICY AND REGISTRY REFINEMENTS
+
+### 42.1 Community Profile and Group Submissions (Policy Q1)
+
+Organizations submit custom profiles and policy groups to the DCM community registry via same PR-based workflow as Resource Types โ€” Tier 2. Requirements: documented use case, at least one production deployment reference, test results, named maintainer. Same lifecycle as Resource Types (shadow validation before active, deprecation, sunset). PROF-005.
+
+### 42.2 Certified Profile Program (Policy Q2)
+
+Certified profiles carry third-party certification metadata (HIPAA assessor, FedRAMP JAB, PCI QSA). Certified profiles promoted to Tier 1 (DCM Core). Certification applies to the profile artifact only โ€” not to any specific deployment. Community-contributed Tier 2 profiles that obtain certification can be promoted to Tier 1. PROF-006.
+
+### 42.3 Policy Provider Trust Elevation Approval (Policy Q3)
+
+Trust elevation (increasing mode level) requires formal approval workflow. Profile-governed approver requirements:
+- standard: 1 platform_admin
+- prod: platform_admin + security_owner (min 2)
+- fsi: platform_admin + security_owner + compliance_officer (dual approval)
+- sovereign: 3 approvers + change control ticket
+
+P7D shadow period after elevation before outputs become binding. POLICY_PROVIDER_ELEVATED audit record. PROF-007.
+
+### 42.4 Dev Profile Resource TTL Configurability (Policy Q4)
+
+Default TTL declared in system domain layer; overridable at platform domain level. Per-resource-type overrides supported (VMs: P7D, Storage: P14D, DNS: P3D). on_expiry action configurable: notify (consumers can extend) vs destroy. PROF-008.
+
+### 42.5 Air-Gapped Policy Provider Delivery (Policy Q5)
+
+Signed bundle model โ€” same as registry bundles. Mode 3 bundles include OPA Rego files. Mode 4 in sovereign profiles: endpoint must be within sovereignty boundary โ€” external AI service calls blocked by BBQ-001 sovereignty check. PROF-009.
+
+### 42.6 No Fourth Registry Tier (Registry Q1)
+
+Certification metadata within existing tier structure โ€” no structural fourth tier. filter: tier_certifications provides equivalent discovery. REG-008.
+
+### 42.7 Tier 3 to Tier 2 Promotion (Registry Q2)
+
+PR-based promotion pathway with additional requirements: production deployment + OSS-compatible license + named community maintainer + documented migration path from Tier 3 handle. Current Tier 3 users notified. REG-009.
+
+### 42.8 Upstream Registry Permanently Unavailable (Registry Q3)
+
+Organization Registry mirror is self-sufficient โ€” upstream loss is a governance decision, not an operational crisis. Three options: designate community mirror as new upstream / fork the registry / continue as independent installation. Existing operations never interrupted. REG-010.
+
+### 42.9 Provider Cost Metadata Source (Registry Q4)
+
+Static declaration or dynamic Cost Analysis sourcing; hybrid recommended (Cost Analysis preferred, static fallback, fallback_max_age: PT24H). Placement engine cost analysis step uses freshest available source โ€” no changes to tie-breaking model. REG-011.
+
+---
+
+## SECTION 43 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2584,7 +2635,7 @@ Per-instance hash chains โ€” each DCM instance (Hub, Regional, Sovereign) has it
 
 ---
 
-## SECTION 43 โ€” TERMINOLOGY GLOSSARY
+## SECTION 44 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2647,6 +2698,11 @@ Per-instance hash chains โ€” each DCM instance (Hub, Regional, Sovereign) has it
 | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
 | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
 | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **certified profile** | DCM profile carrying formal third-party certification metadata (HIPAA assessor, FedRAMP JAB, PCI QSA); promoted to Tier 1; applies to artifact not deployment |
+| **POLICY_PROVIDER_ELEVATED** | Audit action recorded when a Policy Provider's mode level is elevated; always produced regardless of profile |
+| **tier_certifications** | Certification metadata on Resource Type Specs or profiles from recognized certifying bodies; filter criterion, not structural tier boundary |
+| **tier_3_to_tier_2_promotion** | PR-based pathway for organizations to promote internal Tier 3 Resource Types to Verified Community (Tier 2); requires production deployment, OSS license, named maintainer, migration path |
+| **independent operation mode** | Registry Provider state when upstream registry is permanently unavailable; existing types continue; new community type adoption requires governance decision |
 | **SCIM 2.0** | System for Cross-domain Identity Management; optional Auth Provider capability for automated actor provisioning from enterprise IdPs; provisions actors and group memberships; roles not SCIM-provisioned |
 | **step-up MFA** | Additional MFA challenge at sensitive operations within an already-authenticated session; declared per operation by policy; step-up token TTL PT10M |
 | **actor.type** | Audit record field: human / service_account / system; enables filtering between human-initiated and automated lifecycle operations in queries and dashboards |
@@ -2800,7 +2856,7 @@ Per-instance hash chains โ€” each DCM instance (Hub, Regional, Sovereign) has it
 
 ---
 
-## SECTION 44 โ€” OPEN QUESTIONS
+## SECTION 45 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2897,7 +2953,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 45 โ€” DOCUMENTATION STRUCTURE
+## SECTION 46 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -2945,7 +3001,7 @@ content/
 
 ---
 
-## SECTION 46 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 47 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 

From fb0235e76d2ef228e1e52bfee5a25ff7bca07e65 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Thu, 26 Mar 2026 23:08:18 -0500
Subject: [PATCH 17/49] Updated DCM Feneration, cert rotation, request routing
 flow, sovereignty pre-filter, audit provenance, universal groups, information
 providers.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        |  82 ++++++++++++--
 .../audit-provenance-observability.md         | 100 ++++++++++++++++--
 2 files changed, 170 insertions(+), 12 deletions(-)

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index e3c1d03..4eaa52e 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2443,7 +2443,13 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th
 
 | Former Term | Replacement | Meaning |
 |-------------|-------------|---------|
-| Shore | **certified profile** | DCM profile carrying formal third-party certification metadata (HIPAA assessor, FedRAMP JAB, PCI QSA); promoted to Tier 1; applies to artifact not deployment |
+| Shore | **federation routing** | Hub DCM applies placement engine logic at the DCM instance level; Regional DCMs are DCM Provider instances; sovereignty is a hard pre-filter; same tie-breaking hierarchy as provider selection |
+| **independent_with_overlap** | Certificate rotation model: old cert valid P30D after new cert issued; allows peers to update trust stores without coordinated downtime |
+| **alert_and_hold** | Federated drift detection response when peer DCM is unavailable: do not assume drift; hold state; escalate to platform admin after PT24H |
+| **AUDIT_STORE_UNAVAILABLE** | Gap record inserted in Audit Store hash chain after recovery from Audit Store failure; timestamps the exact outage window; makes gap explicit and auditable |
+| **confidence aggregation** | Per-entity endpoint computing overall confidence band (= lowest field band); identifies contested and stale fields; computed on demand never stored |
+| **explicit_no_filter** | Composite group declaration suppressing the no member_type filter linting warning; confirms broad targeting is intentional |
+| **certified profile** | DCM profile carrying formal third-party certification metadata (HIPAA assessor, FedRAMP JAB, PCI QSA); promoted to Tier 1; applies to artifact not deployment |
 | **POLICY_PROVIDER_ELEVATED** | Audit action recorded when a Policy Provider's mode level is elevated; always produced regardless of profile |
 | **tier_certifications** | Certification metadata on Resource Type Specs or profiles from recognized certifying bodies; filter criterion, not structural tier boundary |
 | **tier_3_to_tier_2_promotion** | PR-based pathway for organizations to promote internal Tier 3 Resource Types to Verified Community (Tier 2); requires production deployment, OSS license, named maintainer, migration path |
@@ -2618,7 +2624,65 @@ Static declaration or dynamic Cost Analysis sourcing; hybrid recommended (Cost A
 
 ---
 
-## SECTION 43 โ€” PERSONAS
+## SECTION 43 โ€” FEDERATION, OBSERVABILITY, AND FINAL REFINEMENTS
+
+### 43.1 DCM-to-DCM Certificate Rotation (Federation Q1)
+
+Independent rotation per instance with P30D overlap period. Peers notified via Message Bus 60 days before expiry. Automatic renewal triggers 90 days before expiry. Overlap allows peers to update trust stores without coordinated downtime. DCM-009.
+
+### 43.2 Federation Routing โ€” Full Placement Engine at DCM Level (Federation Q2 โ€” Extended)
+
+**Hub DCM federation routing follows the same placement engine logic as provider selection.** Regional DCMs are treated as DCM Provider instances.
+
+**Sovereignty is a hard pre-filter โ€” not a tie-breaker:**
+- Filter eligible Regional DCMs by sovereignty compatibility before the placement loop
+- No eligible Regional DCMs after filter โ†’ reject with clear error
+
+**Tie-breaking hierarchy at the DCM instance level (same as provider selection):**
+1. Policy preference (policy declares preferred Regional DCM)
+2. Federation priority (numeric priority on DCM Provider registration)
+3. Tenant affinity (Tenant's resources prefer a specific Regional DCM)
+4. Sovereignty match quality (exact over partial match)
+5. Geographic affinity (closest regional to consumer)
+6. Least loaded (capacity utilization)
+7. Consistent hash (deterministic tiebreaker)
+
+**Sub-regional routing:** Regional DCM acts as Hub for its children โ€” same logic recursive within federation depth limit. Load balancing is step 6 (least loaded) โ€” not a primary strategy.
+
+DCM-010.
+
+### 43.3 Federated Drift Detection Ownership (Federation Q3)
+
+Provider-side DCM discovers; consumer-side DCM compares against its Requested State. Discovered State events published via federation Message Bus with correlation_id + consumer_dcm_uuid tag. Peer DCM unavailable = alert-and-hold (not assumed drift); max hold PT24H then escalate. DCM-011.
+
+### 43.4 Cross-DCM Audit Correlation (Federation Q4)
+
+No full synchronization. correlation_id reference model โ€” each DCM keeps its own authoritative audit trail. On-demand pull for compliance investigations requires platform admin auth + sovereignty check + peer DCM authorization. DCM-012.
+
+### 43.5 Maximum Federation Depth (Federation Q5)
+
+Profile-governed: minimal/dev=5, standard/prod=3, fsi/sovereign=2. Measured as hops from deepest instance to Hub DCM. Depth 3 covers Hub โ†’ Regional โ†’ Sub-Regional โ†’ Edge. DCM-013.
+
+### 43.6 Audit Provenance Scattered Resolutions
+
+- **Q1 (Audit Store architecture):** Already resolved as STO-004 โ€” specialized Storage Provider sub-type; see doc 11.
+- **Q2 (Air-gapped replication):** Live sync for Regional DCMs; signed bundle export for Sovereign DCMs; sovereignty check required; hash chain preserved; AUD-018.
+- **Q3 (Default dashboard):** Grafana bundled for minimal/dev/standard; enterprise integration recommended for prod; required for fsi; local-only for sovereign; OBS-002.
+- **Q4 (Failing Storage Provider):** Two-stage model handles it โ€” Commit Log (etcd) independent of Storage Providers; AUDIT_STORE_UNAVAILABLE gap record on recovery; hash chain makes gap explicit; AUD-019.
+
+### 43.7 Universal Groups โ€” Composite Linting (Q1)
+
+Linting warning (not error) when composite group policy targeting has no member_type filter. Operator may suppress with explicit_no_filter: true. GRP-016.
+
+### 43.8 Information Provider โ€” Confidence Aggregation (Q2) and Override Notifications (Q3)
+
+**Confidence aggregation API:** GET /api/v1/entities/{uuid}/confidence โ€” overall band = lowest field band (conservative); computed on demand; identifies contested and stale fields. INF-010.
+
+**Override notifications:** Provider opt-in via conflict_notification in registration; webhook or Message Bus; overriding value may be redacted by policy for confidentiality. INF-011.
+
+---
+
+## SECTION 44 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2635,7 +2699,7 @@ Static declaration or dynamic Cost Analysis sourcing; hybrid recommended (Cost A
 
 ---
 
-## SECTION 44 โ€” TERMINOLOGY GLOSSARY
+## SECTION 45 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2698,6 +2762,12 @@ Static declaration or dynamic Cost Analysis sourcing; hybrid recommended (Cost A
 | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail |
 | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class |
 | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation |
+| **federation routing** | Hub DCM applies placement engine logic at the DCM instance level; Regional DCMs are DCM Provider instances; sovereignty is a hard pre-filter; same tie-breaking hierarchy as provider selection |
+| **independent_with_overlap** | Certificate rotation model: old cert valid P30D after new cert issued; allows peers to update trust stores without coordinated downtime |
+| **alert_and_hold** | Federated drift detection response when peer DCM is unavailable: do not assume drift; hold state; escalate to platform admin after PT24H |
+| **AUDIT_STORE_UNAVAILABLE** | Gap record inserted in Audit Store hash chain after recovery from Audit Store failure; timestamps the exact outage window; makes gap explicit and auditable |
+| **confidence aggregation** | Per-entity endpoint computing overall confidence band (= lowest field band); identifies contested and stale fields; computed on demand never stored |
+| **explicit_no_filter** | Composite group declaration suppressing the no member_type filter linting warning; confirms broad targeting is intentional |
 | **certified profile** | DCM profile carrying formal third-party certification metadata (HIPAA assessor, FedRAMP JAB, PCI QSA); promoted to Tier 1; applies to artifact not deployment |
 | **POLICY_PROVIDER_ELEVATED** | Audit action recorded when a Policy Provider's mode level is elevated; always produced regardless of profile |
 | **tier_certifications** | Certification metadata on Resource Type Specs or profiles from recognized certifying bodies; filter criterion, not structural tier boundary |
@@ -2856,7 +2926,7 @@ Static declaration or dynamic Cost Analysis sourcing; hybrid recommended (Cost A
 
 ---
 
-## SECTION 45 โ€” OPEN QUESTIONS
+## SECTION 46 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -2953,7 +3023,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 46 โ€” DOCUMENTATION STRUCTURE
+## SECTION 47 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -3001,7 +3071,7 @@ content/
 
 ---
 
-## SECTION 47 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 48 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
diff --git a/content/docs/architecture/data-model/audit-provenance-observability.md b/content/docs/architecture/data-model/audit-provenance-observability.md
index 9bbec19..87c5790 100644
--- a/content/docs/architecture/data-model/audit-provenance-observability.md
+++ b/content/docs/architecture/data-model/audit-provenance-observability.md
@@ -12,7 +12,7 @@ weight: 11
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
-**Related Documents:** [Four States](../four-states/) | [Storage Providers](../storage-providers/) | [Context and Purpose](../context-and-purpose/)
+**Related Documents:** [Four States](02-four-states.md) | [Storage Providers](11-storage-providers.md) | [Context and Purpose](00-context-and-purpose.md)
 
 ---
 
@@ -40,7 +40,7 @@ Audit queries provenance to answer its questions. Observability does not use pro
 
 ### 2.2 Provenance Structure
 
-See [Context and Purpose โ€” Section 4.4](../context-and-purpose/) for the complete field-level provenance structure. The key elements:
+See [Context and Purpose โ€” Section 4.4](00-context-and-purpose.md) for the complete field-level provenance structure. The key elements:
 
 ```yaml
 field_name:
@@ -346,10 +346,10 @@ Persona-based access control is enforced at the API Gateway level for all endpoi
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | Should the Audit Store be a specialized Storage Provider or can a general Event Stream Store satisfy the audit contract? | Architecture | โ“ Unresolved |
-| 2 | How are audit records replicated across sites in air-gapped or geographically distributed deployments? | Sovereignty | โ“ Unresolved |
-| 3 | Should DCM provide a default observability dashboard or only the telemetry? | Deployment | โ“ Unresolved |
-| 4 | How does the Audit component handle provenance events from a Storage Provider that has been deregistered? | Operational | โ“ Unresolved |
+| 1 | Should the Audit Store be a specialized Storage Provider or can a general Event Stream Store satisfy the audit contract? | Architecture | โœ… Resolved โ€” specialized Storage Provider sub-type; append-only; hash chain integrity; reference-based retention; compliance queries; see doc 11 (STO-004) |
+| 2 | How are audit records replicated across sites in air-gapped or geographically distributed deployments? | Sovereignty | โœ… Resolved โ€” live sync for Regional DCMs; signed bundle for Sovereign DCMs; sovereignty check required; hash chain preserved across transport (AUD-018) |
+| 3 | Should DCM provide a default observability dashboard or only the telemetry? | Deployment | โœ… Resolved โ€” default Grafana dashboard for minimal/dev/standard; enterprise integration recommended for prod; required for fsi; local-only for sovereign (OBS-002) |
+| 4 | How does the Audit component handle provenance events from a Storage Provider that has been deregistered? | Operational | โœ… Resolved โ€” two-stage model handles this; Commit Log independent of Storage Providers; gap record inserted on Audit Store recovery; chain makes gap explicit (AUD-019) |
 
 ---
 
@@ -363,6 +363,94 @@ Persona-based access control is enforced at the API Gateway level for all endpoi
 - **Drift Detection** โ€” uses discovered vs realized state comparison; drift events feed the Audit component
 - **Unsanctioned Change** โ€” a specific audit event type triggered by unauthorized resource modification
 
+
+## 7. Audit Provenance Observability Gap Resolutions
+
+### 7.1 Audit Store Architecture (Q1)
+
+Resolved as STO-004 in doc 11 (Storage Providers). The Audit Store is a specialized Storage Provider sub-type โ€” append-only with immutability enforcement, hash chain integrity, reference-based retention tracking, and compliance-grade multi-dimensional queries. The Event Stream is the delivery channel only, not the compliance destination.
+
+### 7.2 Audit Record Replication Across Sites (Q2)
+
+Each DCM instance maintains its own Audit Store. Replication uses live sync (Regional DCMs with connectivity) or signed bundle export (Sovereign DCMs without connectivity).
+
+```yaml
+audit_replication:
+  model: 
+
+  live_sync:                        # Regional DCMs with Hub connectivity
+    direction: regional_to_hub      # Regional pushes aggregated view to Hub
+    filters:
+      include: [SECURITY, GATEKEEPER_TRIGGERED, SOVEREIGNTY_VIOLATION]
+    sovereignty_check: required     # before any replication
+
+  signed_bundle:                    # Sovereign DCMs
+    export_on: [scheduled, connectivity_window, on_demand]
+    schedule: "0 0 * * 0"           # weekly during connectivity window
+    encryption: required
+    hash_chain_preserved: true      # chain integrity maintained across transport
+    import_at: hub_dcm_audit_store
+
+  per_instance_only:                # fully isolated Sovereign DCMs
+    export_on_request: via_signed_bundle_manual_transfer
+```
+
+### 7.3 Default Observability Dashboard (Q3)
+
+DCM ships a default Grafana-based observability dashboard for minimal/dev/standard profiles.
+
+```yaml
+default_observability_dashboard:
+  implementation: grafana
+  pre_built_dashboards:
+    - dcm_overview              # request throughput, error rates, component health
+    - resource_lifecycle        # entity state transitions, rehydration activity
+    - policy_evaluation         # GateKeeper triggers, shadow results, validation failures
+    - provider_health           # provider availability, capacity confidence, trust scores
+    - audit_integrity           # hash chain status, pending forwards, chain breaks
+    - federation_status         # federation tunnel health, cross-DCM traffic
+  profile_behavior:
+    minimal: included
+    dev: included
+    standard: included_optional         # shipped; organizations may substitute
+    prod: integration_recommended       # integrate with enterprise observability
+    fsi: integration_required
+    sovereign: local_only               # local Grafana; no external connections
+  export_formats: [prometheus, opentelemetry, json]
+```
+
+### 7.4 Audit Component Handling Failing Storage Provider (Q4)
+
+The two-stage audit model handles this by design โ€” the Stage 1 Commit Log (etcd) has no dependency on any Storage Provider.
+
+```
+Storage Provider failure detected
+  โ”‚
+  โ–ผ Stage 1 Commit Log (etcd) โ€” independent of Storage Provider
+  โ”‚   Records: STORAGE_PROVIDER_FAILURE event immediately
+  โ”‚
+  โ–ผ Stage 2 Audit Forward Service โ€” async, after recovery
+  โ”‚   Forwards accumulated events including the failure event itself
+  โ”‚
+  โ–ผ For Audit Store self-failure specifically:
+      Commit Log accumulates events as pending_forward
+      On recovery: queue drains in order
+      AUDIT_STORE_UNAVAILABLE gap record inserted with exact outage timestamps
+      Hash chain makes the gap explicitly visible โ€” not hidden
+```
+
+The gap record is not a failure โ€” it is evidence of correct behavior. Auditors can see exactly when the Audit Store was unavailable and that no records were lost (all arrived after recovery).
+
+### 7.5 System Policies โ€” Audit Provenance Gaps
+
+| Policy | Rule |
+|--------|------|
+| `STO-004` | The Audit Store is a specialized Storage Provider sub-type โ€” append-only, hash chain integrity, reference-based retention, compliance-grade queries. Event Stream is the delivery channel only. (See doc 11) |
+| `AUD-018` | Audit records are replicated using live sync (Regional DCMs) or signed bundle export (Sovereign DCMs). Sovereignty checks required before any replication. Hash chain integrity preserved across transport. Fully isolated Sovereign DCMs maintain local-only audit stores with manual export. |
+| `OBS-002` | DCM ships a default Grafana-based observability dashboard for minimal/dev/standard profiles. Standard+ profiles may substitute enterprise platforms. FSI requires enterprise observability. Sovereign DCMs use local dashboard only with no external connections. |
+| `AUD-019` | Storage Provider failures are recorded via the Stage 1 Commit Log (etcd), which is independent of all Storage Providers. Audit Store self-failures produce pending_forward records. On recovery, a gap record (AUDIT_STORE_UNAVAILABLE) is inserted with the outage window timestamps. The hash chain gap is explicit and auditable. |
+
+
 ---
 
 *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*

From 8005c7d63fdf8bdff7af13fb9bf6582d54142ba1 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Fri, 27 Mar 2026 09:21:33 -0500
Subject: [PATCH 18/49] Added taxonomy, initial capabilities matrix site,
 standardized on Service Providers vs producers

Signed-off-by: Chris Roadfeldt 
---
 content/_index.md                      |  46 +++++++++--
 content/docs/architecture/ai-prompt.md | 103 +++++++++++++++++++++++--
 content/docs/enhancements/_index.md    |   6 +-
 3 files changed, 140 insertions(+), 15 deletions(-)

diff --git a/content/_index.md b/content/_index.md
index 4585ceb..799d564 100644
--- a/content/_index.md
+++ b/content/_index.md
@@ -1,14 +1,44 @@
 ---
-title: Documentation
-type: docs
-sidebar:
-  open: true
+title: DCM Project
+toc: false
 ---
 
-# DCM Documentation
+> **โš ๏ธ Active Development** โ€” The DCM project is under active development. Architecture, data model, and specifications are work in progress. [Contribute on GitHub](https://github.com/dcm-project).
 
-Documentation for the DCM project.
+# Data Center Management
 
-## Documentation Sections
+Hyperscaler-like cloud experience for
+enterprise on-premises and sovereign cloud infrastructure
 
-- **[Enhancements](enhancements/)** - Enhancement proposals documenting new features, architectural decisions, and significant changes to the DCM project.
+{{< button href="/docs" >}}Get Started{{< /button >}}
+{{< button href="https://github.com/dcm-project" >}}View on GitHub{{< /button >}}
+
+## About DCM
+
+The Data Center Management (DCM) project provides a hyperscaler-like cloud experience
+for enterprise on-premises and sovereign cloud infrastructure. DCM is an
+**API-first, technology-agnostic framework** that relies on a declarative data model
+where the desired state of infrastructure is defined as code.
+
+The overarching goal is to define the configuration of an entire data center in
+version-controlled code, enabling rapid deployment and full environment recovery.
+
+## Core Principles
+
+{{< cards >}}
+  {{< card title="API-First Design" icon="plug" subtitle="Technology-agnostic framework built on modern API principles for maximum flexibility." >}}
+  {{< card title="Cloud-Native Approach" icon="cloud" subtitle="Moving away from monolithic management toward distributed, cloud-native principles." >}}
+  {{< card title="Service Catalog" icon="book-open" subtitle="Comprehensive catalog system for managing services and infrastructure offerings." >}}
+  {{< card title="Rules Engine" icon="adjustments" subtitle="Built-in governance and policy enforcement through a flexible rules engine." >}}
+{{< /cards >}}
+
+---
+
+## Explore More
+
+{{< cards >}}
+  {{< card link="docs" title="Documentation" icon="document-text" subtitle="Guides, tutorials, and technical documentation." >}}
+  {{< card link="docs/enhancements" title="Enhancements" icon="light-bulb" subtitle="Enhancement proposals documenting new features and changes." >}}
+  {{< card link="blog" title="Blog & Updates" icon="rss" subtitle="Latest news, releases, and articles." >}}
+  {{< card link="docs/architecture" title="Architecture" icon="template" subtitle="High-level design, data model, specifications, and Kubernetes integration." >}}
+{{< /cards >}}
diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index 4eaa52e..8738823 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -2682,7 +2682,100 @@ Linting warning (not error) when composite group policy targeting has no member_
 
 ---
 
-## SECTION 44 โ€” PERSONAS
+## SECTION 44 โ€” FOUNDATIONAL CAPABILITIES MATRIX
+
+### 44.1 Overview
+
+DCM has 95 foundational capabilities across 15 domains. Each capability has a unique ID (domain prefix + sequence number), three perspectives (Consumer, Producer, Platform/Admin), and declared dependencies. This matrix drives Jira ticket creation and implementation planning.
+
+### 44.2 Minimum Viable Capability Set (21 capabilities for end-to-end demo)
+
+IAM-001 โ†’ IAM-002 โ†’ IAM-003 โ†’ IAM-007 โ†’ CAT-001 โ†’ REQ-001 โ†’ REQ-002 โ†’ REQ-003 โ†’ REQ-004 โ†’ REQ-005 โ†’ REQ-006 โ†’ REQ-007 โ†’ PRV-001 โ†’ PRV-002 โ†’ PRV-003 โ†’ PRV-004 โ†’ PRV-005 โ†’ LCM-001 โ†’ DRF-001 โ†’ DRF-002 โ†’ AUD-001
+
+### 44.3 Domain Summary
+
+| Prefix | Domain | Count |
+|--------|--------|-------|
+| IAM | Identity and Access Management | 7 |
+| CAT | Service Catalog | 7 |
+| REQ | Request Lifecycle Management | 10 |
+| PRV | Provider Contract and Realization | 9 |
+| LCM | Resource Lifecycle Management | 7 |
+| DRF | Drift Detection and Remediation | 5 |
+| POL | Policy Management | 7 |
+| LAY | Data Layer Management | 5 |
+| INF | Information and Data Integration | 6 |
+| ING | Ingestion and Brownfield Management | 4 |
+| AUD | Audit and Compliance | 5 |
+| OBS | Observability and Operations | 5 |
+| STO | Storage and State Management | 6 |
+| FED | DCM Federation and Multi-Instance | 5 |
+| GOV | Platform Governance and Administration | 7 |
+| **Total** | | **95** |
+
+### 44.4 Perspectives
+
+- **Consumer** โ€” what the end user / application team experiences or can do
+- **Producer** โ€” what the Service Provider or platform component must implement
+- **Platform/Admin** โ€” what the platform engineer or SRE must configure or operate
+
+Empty perspective = that capability does not have a direct touchpoint for that role.
+
+### 44.5 Key Dependency Chain
+
+```
+IAM-001 (Auth) โ†’ IAM-003 (RBAC) โ†’ CAT-001 (Catalog) โ†’ REQ-001 (Submit)
+  โ†’ REQ-003 (Layers) โ†’ REQ-004 (Policy) โ†’ REQ-005 (Placement) โ†’ REQ-007 (Dispatch)
+    โ†’ PRV-001 (Provider Reg) โ†’ PRV-003 (Realization) โ†’ PRV-005 (Realized State)
+      โ†’ LCM-001 (State Transitions) โ†’ DRF-001 (Discovery) โ†’ DRF-002 (Drift)
+```
+
+### 44.6 Resources
+- Interactive map: DCM-Capabilities-Map.html
+- CSV for Jira import: DCM-Capabilities-Matrix.csv
+- Markdown reference: DCM-Capabilities-Matrix.md
+- Taxonomy: DCM-Taxonomy.md
+
+---
+
+## SECTION 45 โ€” DCM TAXONOMY
+
+The DCM Taxonomy is the authoritative vocabulary for all DCM work โ€” code, documentation, Jira tickets, design discussions. Four parts:
+
+### 45.1 Core Vocabulary (key terms)
+- **Service Provider** โ€” provisions/configures/manages infrastructure; implements naturalization, realization, denaturalization, capacity reporting, sovereignty maintenance. *NOT "producer."*
+- **Hub DCM** โ€” central/global instance; authoritative registry origin. *NOT "Shore."*
+- **Regional DCM** โ€” distributed regional instance; treated as DCM Provider by Hub placement engine. *NOT "Ship."*
+- **Sovereign DCM** โ€” air-gapped/compliance-isolated; signed bundle updates only. *NOT "Enclave."*
+- **Layer** โ€” passive data (what values should fields have); distinct from Policy (executable logic)
+- **Policy** โ€” executable rule evaluating assembled payload; distinct from Layer
+- **Confidence Descriptor** โ€” primary data model for Information Provider confidence: authority_level + corroboration + source_trust + last_updated_at (stored); score + band derived at query time
+- **Rehydration** โ€” replaying Intent State to new provider; UUID always preserved
+- **Targeted Delta** โ€” post-realization field update; does not re-run layer assembly chain
+- **Fulfillment** โ€” complete process from consumer submission to Service Provider realization
+- **Reserve Query** โ€” placement engine asking providers "can you fulfill this right now?"
+
+### 45.2 Anti-Vocabulary (terms to avoid)
+| Avoid | Use Instead |
+|-------|-------------|
+| Producer | **Service Provider** |
+| Shore / Ship / Enclave | **Hub DCM** / **Regional DCM** / **Sovereign DCM** |
+| Realize / Realization | **Provision** / **Install** / **Fulfill** |
+| Widgets | Specific resource type name |
+| Data Center | **Region** / **Zone** |
+| User (generic) | **Developer**, **Application Owner**, **Platform Engineer** |
+| Service (unqualified) | **Catalog Item** / **Resource Type** / **Service Provider** |
+| Manage (unqualified) | **Provision** / **Configure** / **Monitor** / **Decommission** |
+
+### 45.3 Roles and Personas
+Developer/Application Owner (consumer), Platform Engineer (platform ops), Policy Owner (governance), Platform Admin (highest-privilege ops), SRE (operational health), Tenant Admin (tenant management), Service Provider Team (provider integration).
+
+### 45.4 Capability Domain Prefixes
+IAM, CAT, REQ, PRV, LCM, DRF, POL, LAY, INF, ING, AUD, OBS, STO, FED, GOV โ€” see Section 44.
+
+---
+
+## SECTION 46 โ€” PERSONAS
 
 | Persona | Primary Concern |
 |---------|----------------|
@@ -2699,7 +2792,7 @@ Linting warning (not error) when composite group policy targeting has no member_
 
 ---
 
-## SECTION 45 โ€” TERMINOLOGY GLOSSARY
+## SECTION 47 โ€” TERMINOLOGY GLOSSARY
 
 | Term | Definition |
 |------|-----------|
@@ -2926,7 +3019,7 @@ Linting warning (not error) when composite group policy targeting has no member_
 
 ---
 
-## SECTION 46 โ€” OPEN QUESTIONS
+## SECTION 48 โ€” OPEN QUESTIONS
 
 These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance.
 
@@ -3023,7 +3116,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl
 
 ---
 
-## SECTION 47 โ€” DOCUMENTATION STRUCTURE
+## SECTION 49 โ€” DOCUMENTATION STRUCTURE
 
 DCM documentation follows a hierarchical structure:
 
@@ -3071,7 +3164,7 @@ content/
 
 ---
 
-## SECTION 48 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 50 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, follow these instructions:
 
diff --git a/content/docs/enhancements/_index.md b/content/docs/enhancements/_index.md
index 3bce142..3a38db7 100644
--- a/content/docs/enhancements/_index.md
+++ b/content/docs/enhancements/_index.md
@@ -1,12 +1,14 @@
 ---
 title: Enhancements
 type: docs
+weight: 2
 sidebar:
   open: true
 ---
 
 # DCM Enhancements
 
-Enhancement proposals for the DCM project. Each enhancement documents new features, significant changes, or architectural decisions.
+Enhancement proposals for the DCM project. Each enhancement documents new features,
+significant changes, or architectural decisions.
 
-View all enhancements in the [DCM Enhancements Repository](https://github.com/dcm-project/enhancements).
\ No newline at end of file
+View all enhancements in the [DCM Enhancements Repository](https://github.com/dcm-project/enhancements).

From 42ae5366c60e1e0947e055919eedb54269dc0843 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Fri, 27 Mar 2026 09:28:04 -0500
Subject: [PATCH 19/49] Added specific note about WIP for K8S operator items.

Signed-off-by: Chris Roadfeldt 
---
 .../kubernetes-compatibility.md               | 28 +++++++++++++------
 .../specifications/operator-interface-spec.md | 24 ++++++++++++----
 .../specifications/operator-sdk-api.md        | 26 ++++++++++++-----
 3 files changed, 57 insertions(+), 21 deletions(-)

diff --git a/content/docs/architecture/specifications/kubernetes-compatibility.md b/content/docs/architecture/specifications/kubernetes-compatibility.md
index d1eb6de..6e67eb6 100644
--- a/content/docs/architecture/specifications/kubernetes-compatibility.md
+++ b/content/docs/architecture/specifications/kubernetes-compatibility.md
@@ -1,19 +1,31 @@
 ---
-title: "Kubernetes Compatibility"
+title: "Kubernetes Compatibility and Concept Mappings"
 type: docs
-weight: 2
+weight: 1
 ---
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+> ## โš ๏ธ Work in Progress โ€” Kubernetes Operator Integration
+>
+> **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.**
+>
+> The Kubernetes operator integration layer โ€” including the Operator Interface Specification, Operator SDK API, and Kubernetes compatibility mappings โ€” represents design intent that has not yet been validated against implementation. Specific interface contracts, API signatures, SDK method names, and CRD structures **will change** as implementation work begins.
+>
+> **Do not build against these specifications yet.** They are published to share design direction and invite feedback, not as stable contracts.
+>
+> Known gaps and open items for this section:
+> - Operator Interface Specification: reconciliation hook signatures are provisional
+> - Operator SDK API: Go module structure and dependency model not yet finalized
+> - Kubernetes Compatibility Mappings: some concept mappings remain under discussion
+> - SDK code examples are illustrative only โ€” not yet tested against a real implementation
+>
+> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues).
+
 
 
 **Document Status:** ๐Ÿ”„ In Progress  
 **Document Type:** Architecture Reference  
-**Related Documents:** [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](../../specifications/operator-interface-spec/)
+**Related Documents:** [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md)
 
 ---
 
diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md
index 44d7cc9..2c5bc80 100644
--- a/content/docs/architecture/specifications/operator-interface-spec.md
+++ b/content/docs/architecture/specifications/operator-interface-spec.md
@@ -1,14 +1,26 @@
 ---
 title: "DCM Operator Interface Specification"
 type: docs
-weight: 1
+weight: 2
 ---
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+> ## โš ๏ธ Work in Progress โ€” Kubernetes Operator Integration
+>
+> **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.**
+>
+> The Kubernetes operator integration layer โ€” including the Operator Interface Specification, Operator SDK API, and Kubernetes compatibility mappings โ€” represents design intent that has not yet been validated against implementation. Specific interface contracts, API signatures, SDK method names, and CRD structures **will change** as implementation work begins.
+>
+> **Do not build against these specifications yet.** They are published to share design direction and invite feedback, not as stable contracts.
+>
+> Known gaps and open items for this section:
+> - Operator Interface Specification: reconciliation hook signatures are provisional
+> - Operator SDK API: Go module structure and dependency model not yet finalized
+> - Kubernetes Compatibility Mappings: some concept mappings remain under discussion
+> - SDK code examples are illustrative only โ€” not yet tested against a real implementation
+>
+> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues).
+
 
 
 **Version:** 0.1.0-draft  
diff --git a/content/docs/architecture/specifications/operator-sdk-api.md b/content/docs/architecture/specifications/operator-sdk-api.md
index b2ab69b..851d86b 100644
--- a/content/docs/architecture/specifications/operator-sdk-api.md
+++ b/content/docs/architecture/specifications/operator-sdk-api.md
@@ -1,14 +1,26 @@
 ---
-title: "DCM Operator SDK API"
+title: "DCM Operator SDK โ€” API Design"
 type: docs
 weight: 3
 ---
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+> ## โš ๏ธ Work in Progress โ€” Kubernetes Operator Integration
+>
+> **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.**
+>
+> The Kubernetes operator integration layer โ€” including the Operator Interface Specification, Operator SDK API, and Kubernetes compatibility mappings โ€” represents design intent that has not yet been validated against implementation. Specific interface contracts, API signatures, SDK method names, and CRD structures **will change** as implementation work begins.
+>
+> **Do not build against these specifications yet.** They are published to share design direction and invite feedback, not as stable contracts.
+>
+> Known gaps and open items for this section:
+> - Operator Interface Specification: reconciliation hook signatures are provisional
+> - Operator SDK API: Go module structure and dependency model not yet finalized
+> - Kubernetes Compatibility Mappings: some concept mappings remain under discussion
+> - SDK code examples are illustrative only โ€” not yet tested against a real implementation
+>
+> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues).
+
 
 
 **Version:** 0.1.0-draft  
@@ -16,7 +28,7 @@ weight: 3
 **Document Type:** Technical Design  
 **Language:** Go  
 **Repository:** https://github.com/dcm-project/operator-sdk  
-**Related Documents:** [DCM Operator Interface Specification](../../specifications/operator-interface-spec/) | [Kubernetes Compatibility](../../specifications/kubernetes-compatibility/)
+**Related Documents:** [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Kubernetes Compatibility](11-kubernetes-compatibility.md)
 
 ---
 

From 18d666641ecc8454657616d65b98f80d08ac4554 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Fri, 27 Mar 2026 09:56:26 -0500
Subject: [PATCH 20/49] Added missing items...

Signed-off-by: Chris Roadfeldt 
---
 content/docs/_index.md                        |   23 +
 .../architecture/data-model/auth-providers.md |  676 ++++++++
 .../architecture/data-model/dcm-federation.md |  585 +++++++
 .../data-model/deployment-redundancy.md       |  593 +++++++
 .../information-providers-advanced.md         |  694 ++++++++
 .../data-model/ingestion-model.md             |  471 ++++++
 .../data-model/policy-profiles.md             | 1500 +++++++++++++++++
 .../data-model/registry-governance.md         |  614 +++++++
 .../data-model/universal-audit.md             |  608 +++++++
 .../data-model/universal-groups.md            |  588 +++++++
 .../data-model/webhooks-messaging.md          |  726 ++++++++
 content/docs/capabilities-map.html            |  824 +++++++++
 content/docs/capabilities-matrix.md           |  271 +++
 content/docs/capabilities.md                  |   67 +
 content/docs/data-model/_index.md             |   27 +
 .../audit-provenance-observability.md         |  361 ++++
 .../docs/data-model/context-and-purpose.md    |  367 ++++
 .../docs/data-model/entity-relationships.md   |  465 +++++
 content/docs/data-model/four-states.md        |  442 +++++
 .../docs/data-model/information-providers.md  |  434 +++++
 .../data-model/layering-and-versioning.md     | 1212 +++++++++++++
 content/docs/data-model/resource-grouping.md  |  292 ++++
 .../data-model/resource-service-entities.md   |  472 ++++++
 .../data-model/resource-type-hierarchy.md     |  443 +++++
 .../docs/data-model/service-dependencies.md   |  371 ++++
 content/docs/data-model/storage-providers.md  |  299 ++++
 content/docs/specifications/_index.md         |   19 +
 content/docs/specifications/cncf-strategy.md  |  227 +++
 .../kubernetes-compatibility.md               |  347 ++++
 .../specifications/operator-interface-spec.md |  830 +++++++++
 .../docs/specifications/operator-sdk-api.md   |  638 +++++++
 content/docs/taxonomy.md                      |  279 +++
 32 files changed, 15765 insertions(+)
 create mode 100644 content/docs/_index.md
 create mode 100644 content/docs/architecture/data-model/auth-providers.md
 create mode 100644 content/docs/architecture/data-model/dcm-federation.md
 create mode 100644 content/docs/architecture/data-model/deployment-redundancy.md
 create mode 100644 content/docs/architecture/data-model/information-providers-advanced.md
 create mode 100644 content/docs/architecture/data-model/ingestion-model.md
 create mode 100644 content/docs/architecture/data-model/policy-profiles.md
 create mode 100644 content/docs/architecture/data-model/registry-governance.md
 create mode 100644 content/docs/architecture/data-model/universal-audit.md
 create mode 100644 content/docs/architecture/data-model/universal-groups.md
 create mode 100644 content/docs/architecture/data-model/webhooks-messaging.md
 create mode 100644 content/docs/capabilities-map.html
 create mode 100644 content/docs/capabilities-matrix.md
 create mode 100644 content/docs/capabilities.md
 create mode 100644 content/docs/data-model/_index.md
 create mode 100644 content/docs/data-model/audit-provenance-observability.md
 create mode 100644 content/docs/data-model/context-and-purpose.md
 create mode 100644 content/docs/data-model/entity-relationships.md
 create mode 100644 content/docs/data-model/four-states.md
 create mode 100644 content/docs/data-model/information-providers.md
 create mode 100644 content/docs/data-model/layering-and-versioning.md
 create mode 100644 content/docs/data-model/resource-grouping.md
 create mode 100644 content/docs/data-model/resource-service-entities.md
 create mode 100644 content/docs/data-model/resource-type-hierarchy.md
 create mode 100644 content/docs/data-model/service-dependencies.md
 create mode 100644 content/docs/data-model/storage-providers.md
 create mode 100644 content/docs/specifications/_index.md
 create mode 100644 content/docs/specifications/cncf-strategy.md
 create mode 100644 content/docs/specifications/kubernetes-compatibility.md
 create mode 100644 content/docs/specifications/operator-interface-spec.md
 create mode 100644 content/docs/specifications/operator-sdk-api.md
 create mode 100644 content/docs/taxonomy.md

diff --git a/content/docs/_index.md b/content/docs/_index.md
new file mode 100644
index 0000000..1cde65a
--- /dev/null
+++ b/content/docs/_index.md
@@ -0,0 +1,23 @@
+---
+title: Documentation
+type: docs
+sidebar:
+  open: false
+---
+
+> **โš ๏ธ Active Development** โ€” DCM documentation is actively being developed. Content represents work in progress and is subject to change as design decisions are finalized.
+
+# DCM Documentation
+
+Documentation for the DCM (Data Center Management) project โ€” a governing framework
+that provides a hyperscaler-like cloud experience for enterprise on-premises and
+sovereign cloud infrastructure.
+
+## Documentation Sections
+
+{{< cards >}}
+  {{< card link="architecture" title="Architecture" icon="template" subtitle="High-level design, data model, specifications, and Kubernetes integration." >}}
+  {{< card link="capabilities" title="Capabilities Map" icon="view-grid" subtitle="Interactive map of 95 foundational capabilities across 15 domains. Filter by domain, perspective, or search. Includes CSV export for Jira import." >}}
+  {{< card link="taxonomy" title="Taxonomy" icon="book-open" subtitle="Authoritative vocabulary reference for all DCM architecture and implementation work. Core vocabulary, anti-vocabulary, roles, and capability prefixes." >}}
+  {{< card link="enhancements" title="Enhancements" icon="document-text" subtitle="Enhancement proposals documenting new features, architectural decisions, and significant changes." >}}
+{{< /cards >}}
diff --git a/content/docs/architecture/data-model/auth-providers.md b/content/docs/architecture/data-model/auth-providers.md
new file mode 100644
index 0000000..1bf04a0
--- /dev/null
+++ b/content/docs/architecture/data-model/auth-providers.md
@@ -0,0 +1,676 @@
+---
+title: "Authentication, Authorization, and Auth Providers"
+type: docs
+weight: 18
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).*
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Webhooks and Messaging](18-webhooks-messaging.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md)
+
+---
+
+## 1. Purpose
+
+DCM authenticates and authorizes every request โ€” inbound and outbound โ€” through a unified **Auth Provider** model. An Auth Provider is the eighth DCM provider type: an external (or built-in) system that answers two questions:
+
+1. **Authentication** โ€” is this identity who they claim to be?
+2. **Authorization** โ€” what is this identity permitted to do?
+
+Every authentication mode DCM supports โ€” static API key, local users, GitHub OAuth, LDAP, FreeIPA, Active Directory, OIDC, mTLS โ€” is an Auth Provider implementation. The built-in Auth Provider ships with DCM and requires zero external configuration, enabling immediate home lab and evaluation use. External Auth Providers are registered artifacts, versioned, GitOps-managed, and audited.
+
+**Authentication is always required โ€” there is no anonymous access in any DCM profile.** The difference between profiles is how much effort authentication setup requires, not whether it exists.
+
+---
+
+## 2. The Eight Provider Types
+
+Auth Provider completes the DCM provider ecosystem:
+
+| # | Type | Purpose |
+|---|------|---------|
+| 1 | **Service Provider** | Realizes resources |
+| 2 | **Information Provider** | Serves authoritative external data |
+| 3 | **Meta Provider** | Composes multiple providers |
+| 4 | **Storage Provider** | Persists DCM state |
+| 5 | **Policy Provider** | Supplies and evaluates policies |
+| 6 | **Message Bus Provider** | Bridges internal/external event streams |
+| 7 | **Credential Provider** | Resolves secrets from external stores |
+| 8 | **Auth Provider** | Authenticates identities and resolves permissions |
+
+---
+
+## 3. Auth Provider Registration
+
+```yaml
+auth_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/auth/corporate-freeipa"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Platform Team"
+
+  name: "Corporate FreeIPA"
+  description: "Primary enterprise directory โ€” FreeIPA with Kerberos"
+
+  # Capabilities
+  capabilities:
+    authentication: true
+    authorization: true
+    mfa: false                    # does this provider enforce MFA?
+    session_management: true
+    group_sync: true
+
+  # Provider type
+  provider_type: 
+
+  # What actor types this provider can authenticate
+  authenticates: [human, service_account, webhook_service_account]
+
+  # Trust level
+  trust_level: 
+  # authoritative: DCM accepts all decisions without re-evaluation
+  # verified:      DCM accepts with additional Policy Engine checks
+  # advisory:      DCM treats decisions as input โ€” full re-evaluation always
+
+  # Connection credentials
+  connection_credentials_ref:
+    credential_provider_uuid: 
+    secret_path: "dcm/auth/freeipa/bind-password"
+
+  # Health check
+  health_check:
+    interval_seconds: 30
+    on_unhealthy: 
+    fallback_provider_uuid: 
+
+  # Session configuration
+  session:
+    token_ttl: PT8H
+    refresh_enabled: true
+    refresh_ttl: P7D
+    concurrent_sessions: 3
+
+  # Role mapping โ€” external groups โ†’ DCM roles
+  role_mapping:
+    default_role: consumer
+    group_role_map:
+      - external_group: "cn=dcm-admins,cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+        dcm_role: platform_admin
+      - external_group: "cn=dcm-sre,cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+        dcm_role: sre
+      - external_group: "cn=dcm-consumers,cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+        dcm_role: consumer
+
+  # Tenant mapping โ€” external groups โ†’ DCM Tenants
+  tenant_mapping:
+    strategy: 
+    group_tenant_map:
+      - external_group: "cn=payments-team,cn=groups,..."
+        tenant_uuid: 
+      - external_group: "cn=platform-team,cn=groups,..."
+        tenant_scope: [all]
+
+  # Config changes go through shadow validation
+  on_config_change: proposed
+```
+
+---
+
+## 4. Authentication Modes
+
+### 4.1 Built-In Auth Provider (zero configuration)
+
+Ships with DCM. Always registered. Cannot be deregistered โ€” only deprioritized.
+
+```yaml
+built_in_auth_provider:
+  handle: "providers/auth/dcm-builtin"
+  provider_type: built_in
+  modes:
+    static_api_key:
+      enabled: true             # generated at bootstrap โ€” shown once
+    local_users:
+      enabled: true             # managed via: dcm user create
+    github_oauth:
+      enabled: false            # opt-in: requires client_id + secret
+    gitlab_oauth:
+      enabled: false            # opt-in: requires client_id + secret
+```
+
+**Static API Key** โ€” generated at bootstrap, shown once:
+```
+โœ“ DCM is ready.
+Your bootstrap API key (shown once โ€” store it securely):
+  dcm_sk_a1b2c3...f7g8
+
+curl -H "Authorization: Bearer dcm_sk_a1b2c3...f7g8" https://localhost:8443/api/v1/catalog
+```
+
+**Local Users** โ€” managed via CLI:
+```bash
+dcm user create --username admin --role platform_admin
+dcm user create --username developer --role consumer --tenant payments
+```
+
+### 4.2 GitHub / GitLab OAuth
+
+```yaml
+auth_provider:
+  provider_type: github_oauth
+  config:
+    client_id: 
+    client_secret_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/github/client-secret"
+    role_mapping:
+      default_role: consumer
+      org_role_map:
+        - github_org: "my-lab-org"
+          dcm_role: platform_admin
+```
+
+### 4.3 LDAP / FreeIPA
+
+```yaml
+auth_provider:
+  provider_type: freeipa          # or: ldap
+  config:
+    server: ldaps://freeipa.corp.example.com:636
+    tls:
+      mode: ldaps                 # ldaps | starttls
+      ca_cert_ref:
+        credential_provider: dcm_internal
+        path: "dcm/auth/freeipa/ca-cert"
+    bind_dn: "uid=dcm-service,cn=users,cn=accounts,dc=corp,dc=example,dc=com"
+    bind_password_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/freeipa/bind-password"
+
+    user_search:
+      base_dn: "cn=users,cn=accounts,dc=corp,dc=example,dc=com"
+      filter: "(uid={username})"
+      attributes:
+        username: uid
+        email: mail
+        display_name: cn
+
+    group_search:
+      base_dn: "cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+      filter: "(member={user_dn})"
+      attributes:
+        group_name: cn
+
+    # FreeIPA-specific integrations
+    kerberos:
+      enabled: true               # SSO for Linux CLI users
+      keytab_ref:
+        credential_provider: dcm_internal
+        path: "dcm/auth/freeipa/dcm.keytab"
+      service_principal: "HTTP/dcm.corp.example.com@CORP.EXAMPLE.COM"
+    hbac:
+      enforce: true               # Honor FreeIPA Host-Based Access Control
+    ca:
+      trust_freeipa_ca: true      # Trust FreeIPA CA for mTLS
+
+    group_sync:
+      enabled: true
+      interval_seconds: 300
+      on_group_change: reauthorize
+```
+
+### 4.4 Active Directory
+
+```yaml
+auth_provider:
+  provider_type: active_directory
+  config:
+    domain_controllers:
+      - ldaps://dc01.corp.example.com:636
+      - ldaps://dc02.corp.example.com:636   # automatic failover
+    tls:
+      mode: ldaps
+      ca_cert_ref:
+        credential_provider: dcm_internal
+        path: "dcm/auth/ad/ca-cert"
+    bind_dn: "CN=DCM Service,OU=Service Accounts,DC=corp,DC=example,DC=com"
+    bind_password_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/ad/bind-password"
+
+    user_search:
+      base_dn: "DC=corp,DC=example,DC=com"
+      filter: "(sAMAccountName={username})"
+      # UPN alternative: "(userPrincipalName={username}@corp.example.com)"
+      attributes:
+        username: sAMAccountName
+        email: userPrincipalName
+        display_name: displayName
+        sid: objectSid            # AD Security Identifier โ€” for audit
+
+    group_search:
+      base_dn: "DC=corp,DC=example,DC=com"
+      # LDAP_MATCHING_RULE_IN_CHAIN โ€” resolves nested AD group membership
+      filter: "(&(objectClass=group)(member:1.2.840.113556.1.4.1941:={user_dn}))"
+      attributes:
+        group_name: cn
+        group_dn: distinguishedName
+```
+
+### 4.5 OIDC
+
+```yaml
+auth_provider:
+  provider_type: oidc
+  config:
+    issuer: https://accounts.google.com      # or: Okta, Azure AD, Keycloak, Dex
+    client_id: dcm-production
+    client_secret_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/oidc/client-secret"
+    scopes: [openid, profile, email, groups]
+    claims_mapping:
+      username: preferred_username
+      email: email
+      display_name: name
+      groups: groups
+      department: department          # custom claims
+      cost_center: cost_center
+```
+
+### 4.6 mTLS
+
+```yaml
+auth_provider:
+  provider_type: mtls
+  config:
+    ca_cert_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/mtls/ca-cert"
+    # Client certificate CN โ†’ DCM actor mapping
+    cn_actor_mapping:
+      - cn_pattern: "service-account-*"
+        actor_type: service_account
+        default_role: consumer
+      - cn_pattern: "provider-*"
+        actor_type: provider
+```
+
+---
+
+## 5. Multiple Auth Providers โ€” Priority and Routing
+
+DCM routes to the appropriate Auth Provider based on the authentication signal present in the request:
+
+```yaml
+auth_provider_resolution:
+  resolution_order:
+    - signal: mtls_client_cert
+      provider_uuid: 
+    - signal: bearer_token_oidc
+      provider_uuid: 
+    - signal: bearer_token_apikey
+      provider_uuid: 
+    - signal: basic_auth
+      provider_uuid: 
+    - signal: hmac_signature
+      provider_uuid: 
+    - signal: none
+      action: reject               # always โ€” no anonymous access
+```
+
+### 5.1 Auth Provider Chain
+
+Authentication and authorization enrichment can be chained:
+
+```yaml
+auth_provider_chain:
+  authentication:
+    provider_uuid:    # fast LDAP bind
+  enrichment:
+    provider_uuid:    # LDAP group membership
+  augmentation:
+    provider_uuid:  # OIDC userinfo for rich claims
+    # (department, cost_center, project codes from HR system)
+```
+
+---
+
+## 6. Credential Provider
+
+A **Credential Provider** is the seventh DCM provider type โ€” a cross-cutting dependency that any DCM component or provider registration references for secret resolution. DCM never stores credentials directly.
+
+```yaml
+credential_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/credentials/hashicorp-vault-prod"
+    status: active
+
+  name: "HashiCorp Vault Production"
+  backend_type: 
+
+  connection:
+    endpoint: https://vault.corp.example.com:8200
+    auth_method: 
+    namespace: 
+
+  credential_types: [hmac_secret, api_key, certificate, connection_string,
+                     bearer_token, private_key, username_password, ldap_bind]
+
+  health_check:
+    interval_seconds: 60
+    on_unhealthy: 
+    # suspend_dependents: suspend all components using this provider
+    # fail_open: continue using cached credentials (risk โ€” use cautiously)
+
+  caching:
+    enabled: true
+    ttl_seconds: 300             # refresh from vault every 5 minutes
+```
+
+**Credential references** โ€” used everywhere a secret is needed:
+
+```yaml
+# In webhook authentication
+secret_ref:
+  credential_provider_uuid: 
+  secret_path: "dcm/webhooks/payments/hmac-secret"
+  version: latest
+
+# In Auth Provider connection
+bind_password_ref:
+  credential_provider_uuid: 
+  secret_path: "dcm/auth/freeipa/bind-password"
+
+# In Service Provider registration
+credentials_ref:
+  credential_provider_uuid: 
+  secret_path: "dcm/providers/kubevirt/service-account"
+```
+
+Credentials are cached in memory per the configured TTL. On cache miss, DCM fetches from the Credential Provider. Credentials never appear in audit records (only the `secret_path` is recorded), never in Git, never in logs.
+
+---
+
+## 7. The Authentication Ladder
+
+Every rung is authenticated. The ladder is about setup effort โ€” not whether authentication exists.
+
+| Profile | Auth Modes Available | Setup Effort | Notes |
+|---------|---------------------|-------------|-------|
+| `minimal` | Static API key, Local user/password | 30 seconds โ€“ 2 minutes | Generated at bootstrap; zero external config |
+| `dev` | + GitHub/GitLab OAuth, FreeIPA/AD (direct bind) | 5โ€“15 minutes | OAuth requires app registration; LDAP requires server config |
+| `standard` | + OIDC via broker (Dex/Keycloak), AD/FreeIPA direct | 30โ€“60 minutes | Enterprise directory or IdP integration |
+| `prod` | + OIDC direct, MFA | 1โ€“2 hours | Full enterprise IdP; MFA configurable |
+| `fsi` | + mTLS required, MFA required | 4โ€“8 hours | Certificate infrastructure required |
+| `sovereign` | + Air-gapped OIDC/mTLS | 1โ€“2 days | No external auth dependencies |
+
+### 7.1 First-Run Setup
+
+```
+DCM First Run Setup
+โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
+
+Welcome to DCM. Choose an authentication mode:
+
+  [1] Static API Key    โ€” Instant start. One key, full access.
+                          Best for: solo home lab, quick evaluation.
+
+  [2] Local Users       โ€” Create usernames and passwords.
+                          Best for: small team, dev environment.
+
+  [3] GitHub OAuth      โ€” Login with GitHub accounts.
+                          Best for: dev teams using GitHub.
+
+  [4] Configure later   โ€” Start with API key, switch to OIDC/LDAP later.
+
+Choice [1]:
+```
+
+### 7.2 The Upgrade Path
+
+Authentication configuration is a DCM artifact โ€” versioned, stored in Git, upgradeable via standard lifecycle:
+
+```bash
+# Upgrade from static API key to FreeIPA LDAP
+dcm auth configure \
+  --provider-type freeipa \
+  --server ldaps://freeipa.corp.example.com:636 \
+  --bind-dn "uid=dcm-service,..." \
+  --bind-password-ref "dcm/auth/freeipa/bind-password"
+# DCM validates, runs in shadow mode, cuts over โ€” API key deprecated on schedule
+```
+
+---
+
+## 8. Profile-Governed Enforcement
+
+| Feature | minimal | dev | standard | prod | fsi | sovereign |
+|---------|---------|-----|---------|------|-----|----------|
+| Static API key | โœ… | โœ… | โŒ | โŒ | โŒ | โŒ |
+| Local user/password | โœ… | โœ… | โŒ | โŒ | โŒ | โŒ |
+| GitHub/GitLab OAuth | โœ… | โœ… | โœ… | โŒ | โŒ | โŒ |
+| LDAP direct bind | โœ… | โœ… | โœ… | โœ… | โœ… | โœ… |
+| OIDC (any provider) | โœ… | โœ… | โœ… | โœ… | โœ… | โœ… |
+| mTLS | โŒ | โŒ | Optional | Recommended | Required | Required |
+| MFA | โŒ | โŒ | Optional | Configurable | Required | Required |
+| Air-gapped OIDC | โŒ | โŒ | โŒ | โŒ | Optional | Required |
+| Anonymous access | โŒ | โŒ | โŒ | โŒ | โŒ | โŒ |
+
+---
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `AUTH-001` | All DCM authentication must be handled through a registered Auth Provider. The built-in Auth Provider is always available and cannot be deregistered. |
+| `AUTH-002` | Multiple Auth Providers may be registered simultaneously. The ingress layer routes to the appropriate provider based on the authentication signal in the request. |
+| `AUTH-003` | Auth Provider trust level governs how DCM treats decisions: authoritative (accepted as-is), verified (with Policy Engine augmentation), advisory (full re-evaluation). |
+| `AUTH-004` | Auth Provider role and tenant mappings are versioned artifacts subject to standard DCM artifact lifecycle. Changes go through proposed โ†’ active validation. |
+| `AUTH-005` | If an Auth Provider becomes unhealthy, existing sessions remain valid until TTL expiry. New authentication attempts route to the configured fallback provider or are rejected. |
+| `AUTH-006` | The Auth Provider used to authenticate a request is recorded in the ingress block and carried into the audit record. Policies may act on auth_provider_uuid and provider_type. |
+| `AUTH-007` | Auth Provider configuration credentials must reference a registered Credential Provider. Plaintext credentials are rejected. |
+| `AUTH-008` | There is no anonymous access in any DCM profile. Minimal and dev profiles support lightweight authenticated modes requiring minimal setup. |
+| `AUTH-009` | Webhook and message bus inbound surfaces always require authentication regardless of active Profile. Anonymous actors are never permitted on these surfaces. |
+| `AUTH-010` | Rate limiting is enforced per authenticated actor. Limits are declared on the Auth Provider or webhook actor registration. |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should DCM support SCIM for automated user provisioning from enterprise IdPs? | Enterprise integration | โœ… Resolved โ€” SCIM 2.0 optional capability; provisions actors and group memberships; roles not SCIM-provisioned; suspend on deprovision default (AUTH-012) |
+| 2 | How does Auth Provider failover interact with in-flight requests during the transition? | Reliability | โœ… Resolved โ€” in-flight requests complete on cached tokens; failover chain for new auth; session TTL respected during outage; all providers unavailable โ†’ reject (AUTH-013) |
+| 3 | Should MFA enforcement be per-operation (step-up MFA) or per-session? | Security UX | โœ… Resolved โ€” two-tier MFA: per-session + step-up; policy declares which operations require step-up; PT10M step-up token TTL; profile-governed defaults (AUTH-014) |
+| 4 | Should the built-in Auth Provider's local user store be backed by a pluggable database? | Architecture | โœ… Resolved โ€” pluggable storage backend; SQLite for minimal/dev; PostgreSQL for standard+; encryption required for fsi/sovereign; local store for bootstrap/service accounts only (AUTH-015) |
+
+---
+
+## 11. Related Concepts
+
+- **Webhooks and Messaging** (doc 18) โ€” ingress/egress actor model; webhook actor registration
+- **Policy Organization** (doc 14) โ€” policies act on auth_provider_type and ingress fields
+- **Universal Audit Model** (doc 16) โ€” auth provider and ingress context in every audit record
+- **Credential Provider** โ€” resolves all Auth Provider connection secrets
+- **Universal Group Model** (doc 15) โ€” group memberships resolved via Auth Provider group sync
+
+## 10. Git Identity Resolution
+
+When DCM processes Git PR ingress, it must resolve the Git server's verified actor identity to a DCM actor with full role, group, and tenant scope context โ€” identical to web UI or API login for the same user.
+
+### 10.1 The Trust Model
+
+DCM trusts the **Git server's authentication assertion** โ€” not user-declared Git configuration. The Git server has already authenticated the user (via SSH key, OAuth token, or LDAP password). DCM receives the Git server's verified identity from the PR merge webhook and resolves it through the registered Auth Provider.
+
+```
+Git server authenticates user โ†’ PR merge webhook โ†’ DCM Auth Provider resolution โ†’ DCM actor
+```
+
+### 10.2 Resolution Methods
+
+| Method | When Used | Auth Provider |
+|--------|----------|--------------|
+| `oidc_subject_lookup` | Git server uses same OIDC/OAuth IdP as DCM | OIDC Auth Provider |
+| `ldap_username_lookup` | Git server authenticates via LDAP/AD | LDAP/AD Auth Provider |
+| `ssh_key_fingerprint` | SSH key-authenticated Git workflows | DCM SSH key registry |
+| `webhook_service_account` | Automated CI/CD Git workflows | Registered webhook actor |
+
+### 10.3 The Resolved Actor
+
+The resolved Git actor carries **identical** role, group, and tenant scope to the same user authenticating via web UI:
+
+```yaml
+# A user who logs into the web UI via FreeIPA LDAP gets:
+# roles: [sre], tenant_scope: [payments-uuid], groups: [payments-team-uuid]
+
+# The same user merging a Git PR via their Git account gets:
+# roles: [sre], tenant_scope: [payments-uuid], groups: [payments-team-uuid]
+# โ€” identical โ€” because both resolve through the same FreeIPA Auth Provider
+```
+
+This is the key invariant: **Git PR ingress does not grant different permissions than any other ingress surface.** The same Auth Provider, the same group mappings, the same tenant scope enforcement.
+
+### 10.4 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `AUTH-011` | Git PR actor identity resolution must use the registered Auth Provider. DCM trusts the Git server's verified identity assertion โ€” not user-declared Git configuration. The resolved actor carries the same role, group, and tenant scope as any other user authenticated via the same Auth Provider. |
+
+
+## 11. Auth Provider Gap Resolutions
+
+### 11.1 SCIM 2.0 User Provisioning (Q1)
+
+DCM supports SCIM 2.0 as an optional Auth Provider capability for enterprise deployments. SCIM automates actor lifecycle management โ€” provisioning, attribute updates, and deprovisioning โ€” from enterprise IdPs (Okta, Azure AD, Ping Identity, JumpCloud).
+
+```yaml
+scim_provider_config:
+  enabled: true
+  scim_version: "2.0"
+  endpoint: https://dcm.corp.example.com/scim/v2
+  auth:
+    mode: bearer_token
+    token_ref:
+      credential_provider_uuid: 
+      path: "dcm/auth/scim/bearer-token"
+
+  provisioned_resources:
+    dcm_actors: true             # create/update/deactivate DCM actor records
+    group_memberships: true      # manage DCM group memberships from IdP groups
+    role_assignments: false      # roles managed by DCM policy โ€” not SCIM
+
+  attribute_mapping:
+    idp_userName: actor.username
+    idp_email: actor.email
+    idp_displayName: actor.display_name
+    idp_department: actor.status_metadata.department
+    idp_groups: actor.groups     # IdP groups โ†’ DCM group memberships (where mapped)
+
+  on_user_deprovisioned:
+    action: suspend              # suspend | deactivate | archive
+    # suspend: reversible; sessions terminated; leases released
+    in_flight_request_handling: complete_then_suspend
+```
+
+**What SCIM does NOT manage:** Roles are not SCIM-provisioned โ€” they require explicit DCM policy authorization. This prevents privilege escalation through the SCIM channel.
+
+### 11.2 Auth Provider Failover and In-Flight Requests (Q2)
+
+In-flight requests authenticated before Auth Provider failure continue to completion using cached session tokens. New requests follow the declared failover chain.
+
+```yaml
+auth_failover_config:
+  primary_provider_uuid: 
+  failover_chain:
+    - provider_uuid: 
+      promotion_delay: PT30S          # wait 30s before promoting failover
+    - provider_uuid: 
+  session_cache:
+    enabled: true
+    ttl: PT8H                         # valid sessions remain valid during outage
+```
+
+**Three scenarios:**
+- **Mid-assembly request (already authenticated):** Continues to completion โ€” session token carries resolved roles/groups/tenant scope; Auth Provider not needed for assembly
+- **New request, Auth Provider down:** Follows failover chain; served from session cache if session still valid
+- **Session expiry during outage:** Requires re-authentication via available failover provider; if all unavailable โ†’ reject with clear error
+
+### 11.3 Step-Up MFA (Q3)
+
+MFA enforcement is two-tier: per-session (validated at login, captured in `mfa_verified` field) and step-up (additional challenge at sensitive operations within an already-authenticated session).
+
+```yaml
+step_up_mfa_config:
+  step_up_required_for:
+    - platform_policy_activate
+    - provider_decommission
+    - tenant_decommission
+    - sovereignty_zone_change
+    - auth_provider_update
+    - manual_rehydration          # if entity min_auth_level = hardware_token_mfa
+  step_up_method: 
+  step_up_token_ttl: PT10M
+  step_up_challenge_max_age: PT5M
+```
+
+**Profile defaults:**
+
+| Profile | Per-Session MFA | Step-Up Required |
+|---------|----------------|-----------------|
+| minimal | No | No |
+| dev | No | No |
+| standard | Recommended | Optional |
+| prod | Required | Destructive operations |
+| fsi | Required | All policy changes |
+| sovereign | Required (hardware token) | All administrative operations |
+
+### 11.4 Built-In Auth Provider Storage Backend (Q4)
+
+The built-in Auth Provider's local user store uses a pluggable storage backend following the Storage Provider model.
+
+```yaml
+builtin_auth_provider_config:
+  user_store:
+    profile_defaults:
+      minimal: sqlite            # zero infrastructure; single-file
+      dev: sqlite
+      standard: postgresql       # durable; concurrent; backupable
+      prod: postgresql
+      fsi: postgresql            # encrypted storage (TDE required)
+      sovereign: postgresql      # HSM-backed encryption required
+    encryption_at_rest:
+      required_profiles: [fsi, sovereign]
+      key_ref:
+        credential_provider_uuid: 
+        path: "dcm/auth/builtin/encryption-key"
+```
+
+**The local user store should only contain:** bootstrap users, service accounts, and API key holders. Enterprise users belong in external Auth Providers (LDAP, OIDC, SCIM).
+
+### 11.5 System Policies โ€” Auth Provider Gaps
+
+| Policy | Rule |
+|--------|------|
+| `AUTH-012` | DCM supports SCIM 2.0 as an optional Auth Provider capability. SCIM provisions and deprovisions DCM actors and group memberships. Roles are not SCIM-provisioned โ€” they require explicit DCM policy authorization. SCIM deprovisioning suspends actors by default; in-flight requests complete before suspension. |
+| `AUTH-013` | In-flight requests authenticated before Auth Provider failure continue using cached session tokens. New requests follow the declared failover chain. Sessions remain valid for their declared TTL during outages. Session expiry during outage requires re-authentication via available failover provider. All providers unavailable โ†’ new authentication rejected. |
+| `AUTH-014` | MFA enforcement is two-tier: per-session MFA (captured in mfa_verified field) and step-up MFA (additional challenge at sensitive operations). Policy declares which operations require step-up regardless of session MFA status. Step-up tokens are short-lived (PT10M default). Profile governs default requirements. |
+| `AUTH-015` | The built-in Auth Provider uses a pluggable storage backend. SQLite is the default for minimal/dev profiles. PostgreSQL is the default for standard+ profiles. FSI and sovereign profiles require encryption at rest. The local user store should only contain bootstrap users, service accounts, and API key holders. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/dcm-federation.md b/content/docs/architecture/data-model/dcm-federation.md
new file mode 100644
index 0000000..6cdff51
--- /dev/null
+++ b/content/docs/architecture/data-model/dcm-federation.md
@@ -0,0 +1,585 @@
+---
+title: "DCM Federation and Cross-Instance Coordination"
+type: docs
+weight: 21
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md)
+
+---
+
+## 1. Purpose
+
+DCM instances do not operate in isolation. Organizations with multiple data centers, regions, or organizational boundaries may run multiple DCM instances that need to coordinate, share resources, and maintain consistent governance. This document defines how DCM instances relate to each other โ€” as peers, in parent-child hierarchies, or as hub-and-spoke configurations โ€” and the mechanisms for cross-instance resource sharing, data export/import, and provider federation eligibility.
+
+---
+
+## 2. DCM-to-DCM Relationship Types
+
+### 2.1 Three Relationship Types
+
+**Peer DCM** โ€” two DCM instances at the same organizational level that share resources or information. Regional DCM instances sharing VLAN allocations. Campus DCM instances sharing compute capacity across departments.
+
+**Parent-Child DCM** โ€” hierarchical relationship where the parent has governance overlay authority over child instances. Corporate DCM with regional children. Service provider DCM with customer children. Parent does not own child resources โ€” it has governance visibility and policy overlay authority (same model as nested Tenants in the Universal Group Model).
+
+**Hub DCM** โ€” specialized Parent DCM acting as a clearinghouse for resource allocation across multiple children. The Hub holds the master resource inventory; children request allocations from the Hub.
+
+### 2.2 Relationship Mapping to Universal Group Model
+
+DCM-to-DCM relationships use the existing Universal Group Model:
+
+```yaml
+# Peer relationship โ€” federation group
+dcm_group:
+  group_class: federation
+  name: "EU-US Regional Federation"
+  members:
+    - member_uuid: 
+      member_type: dcm_peer
+      member_role: eu_region_dcm
+    - member_uuid: 
+      member_type: dcm_peer
+      member_role: us_region_dcm
+  federation_config:
+    shared_policy_inheritance: opt_in
+    cross_member_visibility: audit_only
+
+# Parent-Child relationship โ€” nesting
+dcm_group:
+  group_class: tenant_boundary
+  name: "Corporate DCM"
+  child_groups:
+    - 
+    - 
+  policy_inheritance: opt_out      # parent policies cascade unless child excludes
+```
+
+---
+
+## 3. Provider Federation Eligibility
+
+### 3.1 Concept
+
+Every provider registration carries a `federation_eligibility` declaration โ€” whether the provider can participate in cross-DCM federation, with whom, and under what conditions. This is **layer-defined** (static organizational knowledge) and **policy-enforced** (runtime governance).
+
+### 3.2 Federation Eligibility on Provider Registration
+
+```yaml
+provider_registration:
+  handle: "providers/service/eu-compute-primary"
+
+  federation_eligibility:
+    mode: 
+    # none:      Provider cannot participate in any DCM federation
+    #            (sovereign, classified, or compliance-restricted providers)
+    # selective: Federation permitted only with explicitly declared partners
+    # open:      Federation permitted with any trusted DCM peer
+    #            (sovereignty checks always apply regardless)
+
+    permitted_partners:
+      - partner_type: 
+        dcm_instance_uuids: []          # specific instances
+        dcm_instance_tags: [region-eu, internal]  # tag-based matching
+        dcm_certification_required: [ISO-27001, GDPR-compliant]
+        relationship_requires_approval: true   # bilateral approval required
+
+    federation_scope:
+      permitted_resource_types:
+        - resource_type: Compute.VirtualMachine
+          operations: [allocate, query_capacity]
+          # NOT: decommission โ€” remote DCMs cannot decommission local resources
+        - resource_type: Network.VLAN
+          operations: [allocate, query_capacity, release]
+      data_sharing:
+        capacity_data: true
+        realized_state: false           # do not share realized state details
+        pricing_data: true
+        sovereignty_declaration: true   # always share โ€” required for federation
+      max_allocations_per_partner: 100
+      max_concurrent_allocations: 500
+
+    override_reason: null               # populated when overriding layer default
+```
+
+### 3.3 Layer-Defined Federation Defaults
+
+Federation eligibility defaults live in a `platform` domain layer โ€” static organizational knowledge inherited by all providers unless overridden. Individual provider registrations may be **more restrictive** than the layer default (always permitted); **less restrictive** requires GateKeeper policy approval.
+
+```yaml
+layer:
+  handle: "platform/federation/provider-federation-defaults"
+  domain: platform
+  priority: 600.0.0
+  fields:
+    provider_federation_defaults:
+      compute_providers:
+        federation_eligibility:
+          mode: selective
+          permitted_partners:
+            - partner_type: dcm_peer
+              dcm_instance_tags: [internal, eu-region]
+              dcm_certification_required: [ISO-27001]
+          federation_scope:
+            permitted_resource_types:
+              - resource_type: Compute.VirtualMachine
+                operations: [allocate, query_capacity]
+      storage_providers:
+        federation_eligibility:
+          mode: none                    # storage never federated โ€” data sovereignty
+      network_providers:
+        federation_eligibility:
+          mode: selective
+          permitted_partners:
+            - partner_type: dcm_peer
+              dcm_instance_tags: [internal]
+      information_providers:
+        federation_eligibility:
+          mode: selective
+          data_sharing_restrictions:
+            max_classification: internal   # never share confidential/restricted
+```
+
+### 3.4 Policy Enforcement on Federation
+
+Policies act on federation eligibility at three enforcement points:
+
+**At tunnel establishment:**
+```yaml
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_establishment
+  rule: >
+    If provider.federation_eligibility.mode == none
+    THEN gatekeep: "Provider is not eligible for federation"
+
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_establishment
+  rule: >
+    If remote_dcm.sovereignty_zone NOT IN permitted_sovereignty_zones
+    THEN gatekeep: "Remote DCM sovereignty zone incompatible with local requirements"
+
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_establishment
+  rule: >
+    If remote_dcm.certifications NOT CONTAINS
+       provider.federation_eligibility.permitted_partners.dcm_certification_required
+    THEN gatekeep: "Remote DCM does not hold required certifications"
+```
+
+**At allocation time:**
+```yaml
+policy:
+  type: gatekeeper
+  target: cross_dcm_allocation
+  rule: >
+    If resource_type NOT IN provider.federation_eligibility.federation_scope.permitted_resource_types
+    THEN gatekeep: "Resource type not permitted through this federation tunnel"
+
+policy:
+  type: gatekeeper
+  target: cross_dcm_allocation
+  rule: >
+    If cross_dcm_allocations_active > provider.federation_eligibility.max_concurrent_allocations
+    THEN gatekeep: "Maximum concurrent federation allocations exceeded"
+```
+
+**At data egress:**
+```yaml
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_data_egress
+  rule: >
+    If data.classification > remote_dcm.max_data_classification_receivable
+    THEN gatekeep: "Data classification exceeds remote DCM authorization"
+```
+
+---
+
+## 4. The DCM Provider โ€” Cross-Instance Tunneling
+
+### 4.1 Concept
+
+A **DCM Provider** is a ninth provider type that wraps another DCM instance's API, enabling one DCM to consume resources managed by another DCM as if they were local providers.
+
+| # | Type | Purpose |
+|---|------|---------|
+| 1 | Service Provider | Realizes resources |
+| 2 | Information Provider | Serves authoritative external data |
+| 3 | Meta Provider | Composes multiple providers |
+| 4 | Storage Provider | Persists DCM state |
+| 5 | Policy Provider | Supplies and evaluates policies |
+| 6 | Message Bus Provider | Bridges internal/external event streams |
+| 7 | Credential Provider | Resolves secrets |
+| 8 | Auth Provider | Authenticates identities |
+| 9 | **DCM Provider** | Wraps another DCM instance's API |
+
+### 4.2 DCM Provider Registration
+
+```yaml
+dcm_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/dcm/region-eu-dcm"
+    status: active
+
+  provider_type: dcm_provider
+  relationship_type: 
+
+  remote_dcm:
+    instance_uuid: 
+    endpoint: https://dcm.region-eu.corp.example.com
+    dcm_version: "2.1.0"          # minimum compatible version
+    sovereignty_declaration_ref:   # verified at registration time
+
+  # Authentication โ€” always mTLS for DCM-to-DCM
+  auth:
+    mode: mtls
+    client_cert_ref:
+      credential_provider_uuid: 
+      path: "dcm/dcm-providers/region-eu/client-cert"
+    server_ca_ref:
+      credential_provider_uuid: 
+      path: "dcm/dcm-providers/region-eu/server-ca"
+
+  # Tunnel configuration
+  tunnel_config:
+    encrypted: true                # always โ€” not configurable
+    sovereignty_boundary_check: true  # always โ€” not configurable
+    permitted_resource_types: [Compute.VirtualMachine, Network.VLAN]
+    max_allocation_per_request: 10
+    audit_forwarding: true         # forward audit records to local Audit Store
+    observability_forwarding: true
+
+  # Sovereignty โ€” must be compatible with local requirements
+  sovereignty_declaration:
+    remote_jurisdiction: eu-west
+    data_residency_guarantee: true
+    certifications: [ISO-27001, GDPR-compliant]
+
+  # Health check
+  health_check:
+    endpoint: /api/v1/health
+    interval_seconds: 60
+    on_unhealthy: suspend_allocations
+```
+
+### 4.3 Primary Concerns on All DCM Tunnels
+
+These are non-negotiable on every DCM-to-DCM connection:
+
+| Concern | Enforcement |
+|---------|-----------|
+| **Sovereignty** | Sovereignty_declaration verified before tunnel establishment; data classification checked before every egress |
+| **Authentication** | Always mTLS โ€” no API key, no bearer token โ€” mutual certificate authentication between DCM instances |
+| **Authorization** | Local DCM policies govern ALL resources obtained through a tunnel; remote DCM's policies do not override local |
+| **Audit** | All cross-DCM operations produce audit records in BOTH DCM instances; shared `correlation_id` links the two trails |
+| **Observability** | Cross-DCM resource allocation visible in both instances' observability stores |
+| **Governance** | Local GateKeeper policies apply to all resources from any tunnel source |
+
+---
+
+## 5. Cross-DCM Confidence Scoring
+
+Resources obtained through a DCM tunnel carry compound confidence scores โ€” the resource's confidence in its source DCM, degraded by the tunnel trust score:
+
+```
+cross_dcm_confidence = source_resource_confidence ร— (tunnel_trust_score / 100)
+```
+
+A resource with confidence 90 in source DCM, through a tunnel with trust score 85: `90 ร— 85/100 = 76.5` โ†’ **77**
+
+### 5.1 Federation Trust Score
+
+```yaml
+dcm_federation_trust_score:
+  remote_dcm_uuid: 
+  score: 84                        # 0-100
+  scored_at: 
+  decay_rate: per_30_days
+  factors:
+    identity_verified: true        # mTLS certificate chain verified
+    sovereignty_compatible: true
+    certifications_current: true
+    audit_trail_integrity: true    # audit hash chains verified on sample
+    uptime_score: 0.98
+    compliance_score: 0.90         # policy compliance in recent operations
+    data_completeness: 0.92
+  action_on_score_below:
+    threshold: 60
+    action: 
+```
+
+---
+
+## 6. DCM Export and Import
+
+### 6.1 Export Package
+
+DCM state is fully exportable as a signed package โ€” for disaster recovery, migration, cross-DCM sharing, and Hub DCM onboarding.
+
+```yaml
+dcm_export_package:
+  package_uuid: 
+  exported_at: 
+  exported_by: 
+  dcm_version: 
+  signed_by: 
+
+  scope:
+    tenants: [, ...]
+    resource_types: all
+    layers: [platform, tenant]
+    policies: [platform, tenant]
+    providers: registrations_only   # not credentials โ€” never export credentials
+    entities: [intent_state, requested_state]   # not realized (that's provider state)
+    groups: all
+    audit_records:
+      date_range: [, ]
+      include_hash_chain: true      # for audit trail verification on import
+
+  sovereignty:
+    classification: internal
+    permitted_import_jurisdictions: [eu-west]
+    signed: true
+    encryption: aes256_gcm
+```
+
+### 6.2 Import Trust Score
+
+When importing from another DCM instance, each imported resource carries a trust score:
+
+```yaml
+import_trust_score:
+  score: 78                        # 0-100
+  factors:
+    source_dcm_verified: true      # source DCM identity verified
+    sovereignty_compatible: true
+    data_completeness: 0.92
+    schema_compatibility: 1.00     # source schema matches current version
+    audit_trail_complete: true     # audit records included and hash chain valid
+    certifications_current: true
+  action_on_low_score: 
+  threshold: 70                    # reject if below
+```
+
+### 6.3 Scoring in Resource Definition and Allocation
+
+Resources imported from or allocated through peer DCMs carry their compound confidence score throughout their lifecycle in the importing DCM. The score is visible to the placement engine, Cost Analysis, and the Policy Engine โ€” enabling policies that prefer locally-managed resources over federated resources when scores are comparable.
+
+---
+
+## 7. Profile-Appropriate Federation Policy Groups
+
+DCM ships built-in federation policy groups activated by default per profile:
+
+| Group | Profile | Behavior |
+|-------|---------|---------|
+| `system/group/federation-minimal` | minimal | No federation โ€” single instance only |
+| `system/group/federation-dev` | dev | Peer federation permitted; advisory only |
+| `system/group/federation-standard` | standard | Peer federation with certification requirements |
+| `system/group/federation-prod` | prod | Selective federation; bilateral approval; audit forwarding |
+| `system/group/federation-fsi` | fsi | Strict federation; within-jurisdiction only; full audit; no storage federation |
+| `system/group/federation-sovereign` | sovereign | No external federation; internal peer federation within sovereignty boundary only |
+
+---
+
+## 8. DCM System Policies โ€” Federation
+
+| Policy | Rule |
+|--------|------|
+| `DCM-001` | DCM instances may establish peer, parent-child, or hub relationships using the Universal Group Model federation and nesting constructs. |
+| `DCM-002` | All DCM-to-DCM communication uses mTLS. No API key or bearer token. Sovereignty checks are mandatory before tunnel establishment. These requirements are non-configurable. |
+| `DCM-003` | Local DCM policies govern all resources obtained through DCM tunnels. Cross-DCM operations produce audit records in both DCM instances with a shared correlation_id. |
+| `DCM-004` | DCM state is exportable as a signed package. Imported packages carry a trust score (0-100) computed from source verification, sovereignty compatibility, data completeness, schema compatibility, and audit trail integrity. |
+| `DCM-005` | Resources obtained through DCM tunnels carry compound confidence scores: source_resource_confidence ร— (tunnel_trust_score / 100). |
+| `DCM-006` | Every provider registration must declare federation_eligibility (mode: none, selective, or open). Federation eligibility defaults are declared in platform domain layers. Individual provider registrations may be more restrictive โ€” never more permissive without GateKeeper policy approval. |
+| `DCM-007` | Provider federation scope declares: permitted resource types, permitted operations per type, data sharing permissions, and allocation limits. Remote DCMs cannot decommission local resources through a federation tunnel. |
+| `DCM-008` | Storage providers default to federation_eligibility.mode: none. Data sovereignty constraints prohibit storage federation unless explicitly authorized by sovereign policy with full justification. |
+
+---
+
+## 9. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | How are DCM-to-DCM certificate rotation and renewal handled โ€” coordinated or independent? | Operations | โœ… Resolved โ€” independent rotation with P30D overlap; peer notification 60 days before expiry via Message Bus; auto-renewal at 90 days (DCM-009) |
+| 2 | Should Hub DCM relationships support automatic load balancing across child DCMs? | Architecture | โœ… Resolved โ€” full placement engine logic at DCM instance level; sovereignty as hard pre-filter; tie-breaking hierarchy same as provider selection; sub-regional routing recursive (DCM-010) |
+| 3 | How does drift detection work for resources allocated from a peer DCM โ€” who is responsible for discovery? | Operational | โœ… Resolved โ€” provider-side DCM discovers; consumer-side DCM compares; events via federation Message Bus; peer unavailable = alert-and-hold (DCM-011) |
+| 4 | Should cross-DCM audit records be synchronized โ€” so each DCM has the other's audit records? | Compliance | โœ… Resolved โ€” correlation_id reference model; no full sync; on-demand pull with platform admin auth + sovereignty check (DCM-012) |
+| 5 | What is the maximum supported federation depth (peer of peer of peer)? | Architecture | โœ… Resolved โ€” profile-governed max depth: minimal/dev=5, standard/prod=3, fsi/sovereign=2; measured as hops from deepest to Hub (DCM-013) |
+
+
+## 11. Federation Gap Resolutions
+
+### 11.1 Certificate Rotation and Renewal (Q1)
+
+DCM-to-DCM mTLS certificates rotate independently per instance with a coordinated notification model. Coordinated simultaneous rotation would create a single point of failure.
+
+```yaml
+dcm_federation_cert_rotation:
+  rotation_model: independent_with_overlap
+  overlap_period: P30D              # old cert valid for 30 days after new cert issued
+  notification:
+    notify_peers_at: P60D_before_expiry
+    notification_channel: message_bus
+    notification_payload:
+      new_cert_public_key: 
+      new_cert_valid_from: 
+      old_cert_valid_until: 
+  automatic_renewal:
+    trigger_at: P90D_before_expiry
+    requires_approval: false          # renewal is automatic; elevation requires approval
+```
+
+The P30D overlap allows peers to update their trust stores at their own pace without service interruption.
+
+### 11.2 Federation Routing โ€” Placement Engine at the DCM Level (Q2)
+
+**Hub DCM federation routing follows the same placement engine logic as provider selection.** Regional DCMs are treated as DCM Provider instances in the placement engine.
+
+**Sovereignty is a hard pre-filter โ€” not a preference:**
+
+```
+Before placement loop:
+  Filter eligible Regional DCMs where:
+    sovereignty_declaration satisfies request constraints
+    operating_jurisdictions includes required jurisdictions
+    sovereignty_zone matches tenant.sovereignty_zone
+  
+  If no eligible Regional DCMs โ†’ Reject with clear error
+  Only eligible DCMs enter the placement loop
+```
+
+**The full federation routing flow:**
+
+```
+Request arrives at Hub DCM
+  โ”‚
+  โ–ผ Steps 1-5: Standard nine-step assembly (layers, policies, placement constraints)
+  โ”‚  Pre-placement policies may declare federation routing constraints:
+  โ”‚    "This resource must be in a Regional DCM with EU sovereignty"
+  โ”‚    "This Tenant's resources must stay in Regional DCM-EU-West"
+  โ”‚
+  โ–ผ Step 6: Placement loop โ€” at the DCM instance level
+  โ”‚  Reserve query to eligible Regional DCMs:
+  โ”‚    capacity available? sovereignty compatible? trust score adequate?
+  โ”‚
+  โ–ผ Tie-breaking (same hierarchy as provider selection):
+  โ”‚  1. Policy preference (policy declares preferred Regional DCM)
+  โ”‚  2. Federation priority (numeric priority on DCM Provider registration)
+  โ”‚  3. Tenant affinity (Tenant's resources prefer a specific Regional DCM)
+  โ”‚  4. Sovereignty match quality (exact match over partial match)
+  โ”‚  5. Geographic affinity (closest regional to consumer)
+  โ”‚  6. Least loaded (capacity utilization across instances)
+  โ”‚  7. Consistent hash (deterministic tiebreaker)
+  โ”‚
+  โ–ผ Selected Regional DCM receives assembled request payload
+  โ”‚  Runs its own local assembly and placement (regional layers, regional providers)
+  โ”‚  Returns realization result to Hub DCM โ†’ forwarded to consumer
+  โ”‚
+  โ–ผ Sub-regional routing: Regional DCM acts as Hub for its children
+    Same logic applies recursively within federation depth limit (DCM-013)
+```
+
+**Load balancing is the least-loaded step in the hierarchy** โ€” not a primary strategy. Sovereignty, policy, and tenant affinity all take precedence. Optional `hub_dcm_load_balancing` configuration:
+
+```yaml
+hub_dcm_load_balancing:
+  enabled: true                     # default: true
+  sovereignty_override: true        # always โ€” sovereignty is a hard pre-filter
+  fallback_on_regional_unavailable: route_to_next_eligible
+```
+
+### 11.3 Federated Drift Detection Ownership (Q3)
+
+Provider-side DCM is responsible for discovery; consumer-side DCM is responsible for drift comparison.
+
+```yaml
+federated_drift_detection:
+  discovery_responsibility: provider_side_dcm
+  comparison_responsibility: consumer_side_dcm
+  mechanism:
+    provider_dcm:
+      - Run standard discovery against its providers
+      - Publish Discovered State events to federation Message Bus
+      - Tagged with: entity_uuid + consumer_dcm_uuid + correlation_id
+    consumer_dcm:
+      - Subscribe to Discovered State events for its federated entities
+      - Compare against its Requested State
+      - Trigger drift response policy if drift detected
+  on_peer_dcm_unavailable:
+    action: alert_and_hold            # not assumed drift
+    max_hold_period: PT24H
+    on_hold_exceeded: escalate_to_platform_admin
+```
+
+### 11.4 Cross-DCM Audit Record Correlation (Q4)
+
+No full synchronization. Each DCM keeps its own authoritative audit trail. Cross-DCM correlation uses correlation_id references and on-demand pull.
+
+```yaml
+cross_dcm_audit_correlation:
+  model: correlation_id_reference
+  local_audit_record:
+    action: ALLOCATE_FROM_PEER
+    correlation_id: 
+    peer_dcm_uuid: 
+    peer_audit_record_uuid:     # reference โ€” not a copy
+  on_demand_pull:
+    endpoint: GET /api/v1/audit/cross-dcm/{correlation_id}
+    requires: platform_admin + peer_dcm_authorization + sovereignty_check
+```
+
+Full synchronization is not required โ€” auditors follow correlation_id to the peer DCM on demand.
+
+### 11.5 Maximum Federation Depth (Q5)
+
+```yaml
+federation_depth_policy:
+  max_depth: 3                        # profile-governed
+  on_max_exceeded: reject_federation_establishment
+  profile_defaults:
+    minimal: 5
+    dev: 5
+    standard: 3
+    prod: 3
+    fsi: 2
+    sovereign: 2
+```
+
+Depth is measured as hops from the deepest instance to the Hub DCM. Depth 3 covers Hub โ†’ Regional โ†’ Sub-Regional โ†’ Edge โ€” sufficient for most real-world architectures.
+
+### 11.6 System Policies โ€” Federation Gaps
+
+| Policy | Rule |
+|--------|------|
+| `DCM-009` | DCM-to-DCM mTLS certificates rotate independently per instance with a P30D overlap period. Peers are notified 60 days before expiry via Message Bus. Automatic renewal triggers 90 days before expiry. The overlap period allows peers to update trust stores without coordinated downtime. |
+| `DCM-010` | Hub DCM federation routing follows the same placement engine logic as provider selection. Sovereignty is a hard pre-filter โ€” only Regional DCMs satisfying all sovereignty constraints enter the placement loop. The tie-breaking hierarchy applies at the DCM instance level: policy preference โ†’ federation priority โ†’ tenant affinity โ†’ sovereignty match quality โ†’ geographic affinity โ†’ least loaded โ†’ consistent hash. Regional DCMs are treated as DCM Provider instances. Sub-regional routing applies the same logic recursively within the federation depth limit. |
+| `DCM-011` | For resources allocated from peer DCMs, the provider-side DCM is responsible for discovery. The consumer-side DCM is responsible for drift comparison. Discovered State events are published via federation Message Bus with correlation_id. Peer DCM unavailability triggers alert-and-hold โ€” not assumed drift. |
+| `DCM-012` | Cross-DCM audit records are referenced via correlation_id โ€” not fully synchronized. Each DCM keeps its own authoritative audit trail. Cross-DCM correlation uses on-demand pull with platform admin authorization and sovereignty check. |
+| `DCM-013` | Federation depth is limited to a profile-governed maximum (default: 3 for standard/prod; 2 for fsi/sovereign; 5 for minimal/dev). Requests to establish federation beyond the maximum depth are rejected. Depth is measured as hops from the deepest instance to the Hub DCM. |
+
+
+---
+
+## 10. Related Concepts
+
+- **Universal Group Model** (doc 15) โ€” federation and nesting group classes
+- **Storage Providers** (doc 11) โ€” storage never federated by default
+- **Auth Providers** (doc 19) โ€” mTLS for DCM-to-DCM authentication
+- **Universal Audit Model** (doc 16) โ€” audit records in both DCM instances; correlation_id
+- **Registry Governance** (doc 20) โ€” signed bundles for air-gapped registry updates
+- **Information Providers Advanced** (doc 21) โ€” confidence scoring used in cross-DCM context
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/deployment-redundancy.md b/content/docs/architecture/data-model/deployment-redundancy.md
new file mode 100644
index 0000000..47e5c1f
--- /dev/null
+++ b/content/docs/architecture/data-model/deployment-redundancy.md
@@ -0,0 +1,593 @@
+---
+title: "Deployment and Redundancy"
+type: docs
+weight: 16
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit Model](16-universal-audit.md) | [Policy Organization](14-policy-profiles.md)
+
+---
+
+## 1. Purpose
+
+Every DCM component, every data store, and every capability is designed for redundancy by default. Redundancy is not an add-on or an advanced configuration โ€” it is the baseline operational posture for all profiles above `minimal`.
+
+**The minimal profile** provides single-instance deployment for home lab and evaluation simplicity. All other profiles assume redundancy as the floor. The transition from `minimal` to `dev` is the transition from "works on my laptop" to "survives a node failure."
+
+**Everything in DCM runs as a container in a pod.** No bare-metal DCM components. No special-case deployment paths. Every DCM component follows the same container lifecycle, the same health check model, the same rolling update pattern. DCM runs on Kubernetes โ€” and manages Kubernetes.
+
+**DCM is self-hosting.** DCM's own deployment is expressible as DCM resources. DCM can manage its own lifecycle, detect drift in its own components, and rehydrate its own deployment from Git. This is the ultimate expression of the data center repave use case โ€” DCM restoring itself.
+
+---
+
+## 2. Design Principles
+
+**Redundant by default.** Every component, store, and capability has a redundancy model. The `minimal` profile sets `replicas: 1`. Every other profile sets `replicas: >= 3` with quorum writes and anti-affinity scheduling.
+
+**Everything containerized.** All DCM components run as containers in Kubernetes pods. No exceptions. This gives a consistent deployment model, rolling updates, health checks, and self-healing as first-class properties.
+
+**Profile-governed redundancy.** Replica counts, quorum thresholds, geo-replication, and anti-affinity requirements are declared by the active Profile โ€” not by per-component configuration. Activating a Profile configures redundancy for the entire deployment.
+
+**Self-hosting.** DCM's own deployment is a DCM resource. DCM manages itself through the same model it uses to manage customer infrastructure.
+
+**Stateless control plane.** All DCM control plane components are stateless โ€” all state lives in external stores. Any component instance can fail and be replaced without data loss. State recovery means restarting a pod โ€” not restoring a database.
+
+**Quorum writes for durability.** All durable stores use quorum writes โ€” a write is confirmed only when a majority of replicas acknowledge it. This ensures durability even if a minority of replicas fail simultaneously.
+
+---
+
+## 3. Component Redundancy Model
+
+### 3.1 Control Plane Components
+
+All control plane components are stateless, horizontally scalable, and deployed as Kubernetes Deployments with configurable replica counts.
+
+```yaml
+component_redundancy:
+  component: request_payload_processor   # same model for all components
+  deployment_type: kubernetes_deployment
+  replicas: 3                    # set by active Profile
+  affinity:
+    anti_affinity: required      # pods spread across nodes
+    zone_spread: preferred       # prefer spreading across availability zones
+  disruption_budget:
+    min_available: 2             # always keep 2 running during rolling updates
+  health_check:
+    liveness:
+      path: /healthz
+      interval_seconds: 10
+      failure_threshold: 3
+    readiness:
+      path: /readyz
+      interval_seconds: 5
+      failure_threshold: 2
+  rolling_update:
+    strategy: RollingUpdate
+    max_unavailable: 0           # never take a pod down before replacement is ready
+    max_surge: 1
+```
+
+**Control plane components:**
+
+| Component | Stateless? | Replica Model |
+|-----------|-----------|--------------|
+| API Gateway | Yes | Deployment + HorizontalPodAutoscaler |
+| Request Payload Processor | Yes | Deployment |
+| Policy Engine (OPA) | Yes | Deployment + PolicyBundle sidecar |
+| Placement Engine | Yes | Deployment |
+| Service Catalog | Yes | Deployment |
+| IDM / IAM | Yes | Deployment (external IdP recommended) |
+| Audit Forward Service | Yes | Deployment (1 active + 1 standby) |
+| Lifecycle Constraint Enforcer | Yes | Deployment (leader election) |
+| Drift Detection | Yes | Deployment (leader election for scheduling) |
+| Resource Discovery | Yes | Deployment (leader election) |
+| Message Bus Router | Yes | Deployment |
+
+**Leader election** for scheduler-type components (Lifecycle Constraint Enforcer, Drift Detection, Resource Discovery): multiple replicas run but only one holds the leader lease at a time. On leader failure, a replica acquires the lease within seconds.
+
+### 3.2 Data Store Redundancy
+
+All DCM data stores run as containers. Each store type has a declared replication and quorum model.
+
+#### Commit Log
+
+```yaml
+commit_log:
+  replicas: 3
+  write_quorum: 2               # confirmed durable when 2/3 replicas acknowledge
+  read_quorum: 1                # any replica can serve reads
+  affinity:
+    zone_spread: required       # replicas MUST span availability zones
+  implementation: etcd          # or equivalent consensus store
+  # etcd is purpose-built for this pattern: Raft consensus, quorum writes,
+  # sub-millisecond local writes, proven in Kubernetes itself
+```
+
+The Commit Log uses consensus protocol (Raft/equivalent). A write is confirmed when the quorum acknowledges โ€” ensuring durability even if a minority of replicas fail simultaneously.
+
+#### GitOps Store (Intent, Requested, Layers, Policies)
+
+```yaml
+gitops_store:
+  implementation: gitea          # or equivalent self-hosted Git
+  replicas: 3
+  replication_mode: active_active  # any node can accept writes
+  write_quorum: 2
+  backup:
+    enabled: true
+    schedule: "0 */6 * * *"    # every 6 hours
+    retention: 30d
+```
+
+#### Event Stream Store (Realized, Discovered, Audit Events)
+
+```yaml
+event_stream:
+  implementation: kafka          # or equivalent
+  brokers: 3
+  replication_factor: 3
+  min_insync_replicas: 2        # minimum replicas that must acknowledge a write
+  partitions: 12                # enables parallel consumption
+  retention:
+    bytes: -1                   # unlimited โ€” retention governed by policy
+    ms: -1                      # unlimited
+```
+
+#### Audit Store
+
+```yaml
+audit_store:
+  implementation: elasticsearch  # or equivalent โ€” optimized for queryable retention
+  replicas: 3
+  primary_shards: 5
+  replica_shards: 1             # each shard has 1 replica = 2 copies total
+  geo_replicated: true          # in prod/fsi/sovereign profiles
+  append_only_enforced: true    # storage layer enforces immutability
+```
+
+#### Search Index (Non-Authoritative)
+
+```yaml
+search_index:
+  implementation: elasticsearch
+  replicas: 2                   # lower redundancy โ€” can rebuild from Git
+  rebuild_from_git: true        # on data loss, rebuild from authoritative stores
+```
+
+### 3.3 Container Specification
+
+Every DCM component pod follows a common security and resource model:
+
+```yaml
+pod_spec:
+  security_context:
+    run_as_non_root: true
+    run_as_user: 65534           # nobody
+    run_as_group: 65534
+    fs_group: 65534
+    seccomp_profile:
+      type: RuntimeDefault
+    capabilities:
+      drop: [ALL]
+
+  containers:
+    - name: 
+      image: ghcr.io/dcm-project/:
+      image_pull_policy: IfNotPresent
+      security_context:
+        read_only_root_filesystem: true
+        allow_privilege_escalation: false
+      resources:
+        requests:
+          cpu: 500m
+          memory: 512Mi
+        limits:
+          cpu: 2000m
+          memory: 2Gi
+      liveness_probe: 
+      readiness_probe: 
+      volume_mounts:
+        - name: tmp
+          mount_path: /tmp       # writable temp โ€” no other writable paths
+
+  volumes:
+    - name: tmp
+      empty_dir: {}
+```
+
+---
+
+## 4. Redundancy by Profile
+
+Profile activation configures redundancy for the entire deployment. Organizations do not configure replica counts individually โ€” they activate a profile.
+
+```yaml
+# Redundancy matrix per profile
+redundancy_matrix:
+  minimal:
+    control_plane_replicas: 1
+    store_replicas: 1
+    write_quorum: false
+    zone_spread: false
+    geo_replication: false
+    anti_affinity: false
+    note: "Single-instance. No redundancy. Home lab and evaluation only."
+
+  dev:
+    control_plane_replicas: 1
+    store_replicas: 1
+    write_quorum: false
+    zone_spread: false
+    geo_replication: false
+    anti_affinity: false
+    note: "Single-instance with backup. Basic resilience for dev environments."
+
+  standard:
+    control_plane_replicas: 3
+    store_replicas: 3
+    write_quorum: 2             # 2 of 3
+    zone_spread: preferred
+    geo_replication: false
+    anti_affinity: required
+    note: "Production baseline. Survives single node or zone failure."
+
+  prod:
+    control_plane_replicas: 3
+    store_replicas: 3
+    write_quorum: 2
+    zone_spread: required
+    geo_replication: true
+    anti_affinity: required
+    sla_disruption_budget: "always 2 replicas available"
+    note: "Production with SLA. Geo-replicated stores."
+
+  fsi:
+    control_plane_replicas: 5
+    store_replicas: 5
+    write_quorum: 3             # 3 of 5
+    zone_spread: required
+    geo_replication: true
+    anti_affinity: required
+    audit_store_replicas: 5
+    note: "FSI-grade. Higher quorum threshold. Compliance-grade audit."
+
+  sovereign:
+    control_plane_replicas: 5
+    store_replicas: 5
+    write_quorum: 3
+    zone_spread: required
+    geo_replication: true        # within sovereignty boundary only
+    anti_affinity: required
+    air_gap_backup: true
+    sovereignty_boundary_enforced: true
+    note: "Maximum. All geo-replication within sovereignty boundary."
+```
+
+---
+
+## 5. The DCM Deployment Specification
+
+The DCM deployment itself is a DCM resource โ€” declared in YAML, stored in Git, governed by Policy, subject to the same four-state lifecycle as any other resource.
+
+```yaml
+dcm_deployment:
+  artifact_metadata:
+    uuid: 
+    handle: "deployments/primary/dcm-control-plane"
+    version: "1.0.0"
+    status: active
+
+  profile: system/profile/standard
+  kubernetes_namespace: dcm-system
+
+  # Redundancy โ€” set by active Profile, overridable per component
+  redundancy:
+    control_plane:
+      replicas: 3
+      affinity:
+        anti_affinity: required
+        zone_spread: preferred
+      disruption_budget:
+        min_available: 2
+
+    stores:
+      commit_log:
+        replicas: 3
+        write_quorum: 2
+        implementation: etcd
+      gitops_store:
+        replicas: 3
+        write_quorum: 2
+        implementation: gitea
+      event_stream:
+        brokers: 3
+        replication_factor: 3
+        min_insync_replicas: 2
+        implementation: kafka
+      audit_store:
+        replicas: 3
+        geo_replicated: false     # standard profile default
+        implementation: elasticsearch
+      search_index:
+        replicas: 2
+        implementation: elasticsearch
+
+  # Provider health configuration
+  providers:
+    health_check_interval_seconds: 30
+    unhealthy_threshold: 3
+    automatic_failover: true
+
+  # Self-hosting: DCM manages its own deployment
+  self_managed: true
+  drift_detection_enabled: true
+  rehydration_enabled: true
+  # DCM detects if its own components drift from declared spec
+  # and can rehydrate (redeploy) from this declaration
+```
+
+---
+
+## 6. Self-Hosting โ€” DCM Managing Itself
+
+DCM's own deployment is managed through the same model it uses to manage customer infrastructure. This is the **self-hosting principle** โ€” DCM eats its own cooking.
+
+### 6.1 What Self-Hosting Means
+
+- DCM control plane components are defined as Resource Entities in DCM
+- DCM data stores are defined as Storage Provider resources in DCM
+- DCM's own Policy Groups govern DCM's own deployment constraints
+- DCM runs drift detection on its own components โ€” a component running the wrong image version is drift
+- DCM can rehydrate its own deployment from the `dcm_deployment` declaration in Git
+
+### 6.2 The Bootstrap Problem
+
+DCM cannot manage itself before it exists. The bootstrap sequence:
+
+```
+1. Bootstrap installer deploys minimal DCM (single instance, no redundancy)
+   from a declarative bootstrap manifest
+   โ”‚
+2. Bootstrap DCM reads the target dcm_deployment declaration from Git
+   โ”‚
+3. Bootstrap DCM provisions itself to the target state:
+   - Scales from 1 to N replicas
+   - Provisions redundant stores
+   - Configures quorum
+   โ”‚
+4. Bootstrap instance hands off to the now-redundant DCM
+   โ”‚
+5. DCM manages its own lifecycle from this point forward
+```
+
+The bootstrap manifest is the only thing that exists outside DCM's management scope. It is minimal by design โ€” just enough to get DCM running.
+
+### 6.3 Self-Hosted Drift Detection
+
+DCM continuously compares its own running state against the `dcm_deployment` declaration:
+
+| Drift Type | Example | Response |
+|-----------|---------|---------|
+| Wrong image version | Component running v1.1.0, declared v1.2.0 | Rolling update triggered |
+| Wrong replica count | 2 replicas running, declared 3 | Scale-up triggered |
+| Wrong resource limits | Component using more than declared limits | Alert + potential eviction |
+| Store replication mismatch | Store has 2 replicas, declared 3 | Replication repair triggered |
+
+### 6.4 The Repave Scenario
+
+The ultimate test of self-hosting: DCM is lost entirely (ransomware, catastrophic failure). Recovery:
+
+```
+1. Deploy bootstrap installer to new Kubernetes cluster
+   โ”‚
+2. Bootstrap DCM reads dcm_deployment declaration from Git backup
+   โ”‚
+3. DCM provisions itself โ€” full redundant deployment
+   โ”‚
+4. DCM reads all resource declarations from Git
+   โ”‚
+5. DCM rehydrates customer workloads in dependency order
+   โ”‚
+6. Drift detection validates recovered state matches declared state
+```
+
+The recovery time is bounded by infrastructure provisioning speed โ€” not by backup restoration or manual configuration. Everything is code. Everything is declarative. Everything is in Git.
+
+---
+
+## 7. Commit Log Redundancy โ€” Two-Stage Audit Integration
+
+The Commit Log is the synchronous component of the two-stage audit model. In a distributed deployment, "synchronous durable write" means quorum acknowledgment:
+
+```
+DCM component initiates change
+  โ”‚
+  โ–ผ
+Write to Commit Log (Raft consensus)
+  โ”‚  Propose to leader
+  โ”‚  Leader replicates to followers
+  โ”‚  Write confirmed when quorum (2/3 or 3/5) acknowledge
+  โ”‚
+  โ”œโ”€โ”€ Replica 1 (local node)     โ†’ ACK โ”€โ”
+  โ”œโ”€โ”€ Replica 2 (different node) โ†’ ACK โ”€โ”ค quorum reached
+  โ””โ”€โ”€ Replica 3 (different zone) โ†’ ACK โ”€โ”˜
+  โ”‚
+  โ–ผ  Operation returns success (< 1ms typical with NVMe)
+  โ”‚
+  โ–ผ  [async โ€” Audit Forward Service]
+Read from any surviving Commit Log replica
+  โ”‚  Enrich โ†’ write to Audit Store
+  โ””โ”€โ”€ Clear Commit Log entry after Audit Store confirms
+```
+
+**Failure scenarios and recovery:**
+
+| Failure | During Stage 1 | Effect |
+|---------|---------------|--------|
+| Single replica fails before quorum | Quorum still achievable | No impact |
+| Majority fail before quorum | Commit Log unavailable | Operation aborted โ€” no silent change |
+| Leader fails after quorum | New leader elected (seconds) | In-flight writes complete on new leader |
+| All replicas fail after quorum | Audit Forward Service reads from backup | Full recovery on restart |
+| Audit Store unavailable | Commit Log accumulates | Forward resumes when Audit Store recovers |
+
+---
+
+## 8. Network Architecture
+
+### 8.1 Service Mesh
+
+All DCM component-to-component communication uses a service mesh (Istio or equivalent):
+- mTLS everywhere โ€” no plaintext internal communication
+- Traffic policies enforced at mesh level
+- Observability: traces, metrics, logs for all inter-component calls
+- Circuit breaking: prevent cascade failures
+
+### 8.2 Ingress
+
+External traffic enters through a redundant Ingress layer:
+
+```
+External clients
+  โ”‚
+  โ–ผ
+Load Balancer (external โ€” cloud or on-premises)
+  โ”‚
+  โ–ผ
+Ingress Controller (replicated โ€” 2+ instances)
+  โ”‚
+  โ–ผ
+API Gateway pods (3+ instances, anti-affinity)
+  โ”‚
+  โ–ผ
+Internal service mesh
+```
+
+### 8.3 DNS and Service Discovery
+
+All DCM components address each other via Kubernetes Service DNS. No hardcoded IPs. Service discovery is automatic โ€” a new pod replica is immediately addressable.
+
+---
+
+## 9. DCM System Policies โ€” Redundancy
+
+| Policy | Rule |
+|--------|------|
+| `RED-001` | All DCM control plane components must run as containers in Kubernetes pods |
+| `RED-002` | All control plane components must be stateless โ€” all persistent state in external stores |
+| `RED-003` | In profiles above `minimal`, all control plane components must have `replicas >= 3` with anti-affinity |
+| `RED-004` | All durable stores in profiles above `minimal` must use quorum writes with `write_quorum >= 2` |
+| `RED-005` | The Commit Log must use consensus protocol (Raft or equivalent) with quorum writes |
+| `RED-006` | The DCM deployment must be declared as a DCM resource in Git โ€” self-hosting required |
+| `RED-007` | DCM must run drift detection on its own components โ€” version drift is treated as resource drift |
+| `RED-008` | A rolling update of any DCM component must not reduce available replicas below `min_available` |
+| `RED-009` | All DCM component communication must use mTLS โ€” no plaintext internal communication |
+| `RED-010` | The bootstrap manifest is the only DCM configuration outside DCM's management scope |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should the bootstrap manifest be version-controlled and verifiable? | Bootstrap integrity | โœ… Resolved โ€” GitOps store + hash verification at every startup; tampering prevents start; operator-signed (RED-011) |
+| 2 | How does DCM handle Kubernetes cluster upgrades in sovereign deployments? | Operational | โœ… Resolved โ€” pre-staged images via signed bundles; maintenance mode during upgrade; startup verification before resume (RED-012) |
+| 3 | Should DCM support non-Kubernetes container runtimes? | Portability | โœ… Resolved โ€” Kubernetes primary/required for production; Podman/Docker Compose for dev/community only (RED-013) |
+| 4 | What is the minimum hardware specification per profile? | Implementation | โœ… Resolved โ€” declared as DCM Resource definitions; enforced by placement engine; table documented (RED-014) |
+| 5 | How does DCM's self-hosted drift detection handle DCM drifting from its own state? | Self-hosting | โœ… Resolved โ€” DCM is DCM-managed resource; Operator reconciles; bootstrap hash provides independent check; audit hash chain externally verifiable (RED-015) |
+
+---
+
+## 11. Related Concepts
+
+- **Universal Audit Model** (doc 16) โ€” two-stage audit; Commit Log quorum model
+- **Policy Organization** (doc 14) โ€” Profile-governed redundancy configuration
+- **Storage Providers** (doc 11) โ€” Store contracts include replication requirements
+- **Four States** (doc 02) โ€” all state stores are redundant per this model
+- **Ingestion Model** (doc 13) โ€” DCM's own deployment recovery uses the repave/rehydration pattern
+
+
+## 8. Deployment Redundancy Gap Resolutions
+
+### 8.1 Bootstrap Manifest Verification (Q1)
+
+The bootstrap manifest is stored in the GitOps store, hash-verified at DCM startup and every restart. Tampering prevents DCM from starting.
+
+```yaml
+bootstrap_manifest:
+  version: "1.0.0"
+  manifest_uuid: 
+  manifest_hash:           # computed at creation; verified at every startup
+  signed_by:    # signed by deploying operator
+```
+
+DCM startup sequence: verify manifest hash โ†’ verify manifest signature โ†’ proceed with initialization. On failure: refuse to start; emit security alert; notify platform admin.
+
+### 8.2 Kubernetes Cluster Upgrades in Sovereign Deployments (Q2)
+
+Sovereign DCM (air-gapped) uses pre-staged container images and the maintenance mode pattern:
+
+```
+Pre-upgrade:
+  1. Pre-stage all DCM container images in local registry (signed bundles)
+  2. DCM enters maintenance mode โ€” new requests queued; in-flight complete
+
+During upgrade:
+  3. Kubernetes upgrade proceeds using pre-staged images
+  4. DCM components restart against new cluster version
+
+Post-upgrade:
+  5. Startup verification (bootstrap manifest hash check)
+  6. DCM exits maintenance mode โ€” queued requests resume
+```
+
+Same signed bundle model as registry updates โ€” no new pattern needed.
+
+### 8.3 Non-Kubernetes Container Runtime Support (Q3)
+
+Kubernetes is DCM's primary and recommended container runtime. Non-Kubernetes runtimes (Podman, Docker Compose) are supported for development and community extension only. The DCM Operator is Kubernetes-native and not supported on other runtimes. Production deployments must use Kubernetes.
+
+### 8.4 Minimum Hardware Specifications (Q4)
+
+Expressed as DCM Resource definitions per profile โ€” machine-readable and enforced by the placement engine during self-deployment.
+
+| Profile | CPU | Memory | Storage | Replicas |
+|---------|-----|--------|---------|---------|
+| minimal | 2 cores | 4 Gi | 20 Gi | 1 |
+| dev | 4 cores | 8 Gi | 50 Gi | 1 |
+| standard | 8 cores | 16 Gi | 100 Gi | 3 |
+| prod | 16 cores | 32 Gi | 200 Gi | 3 |
+| fsi | 32 cores | 64 Gi | 500 Gi | 5 |
+| sovereign | 32 cores | 64 Gi | 500 Gi | 5 |
+
+These are minimums. Production workloads may require significantly more based on managed resource count.
+
+### 8.5 DCM Self-Hosted Drift Detection (Q5)
+
+DCM's own deployment is a DCM-managed resource subject to the same drift detection as any other resource. The DCM Operator continuously reconciles running components against the declared deployment manifest.
+
+**The bootstrap paradox โ€” who watches the watchman:**
+- DCM Operator drifts โ†’ bootstrap manifest hash verification (RED-011) detects independently
+- Audit component compromised โ†’ Audit Store hash chain break is detectable by external verification
+- Full DCM compromise โ†’ signed bundle verification at import time provides external trust anchor
+
+### 8.6 System Policies โ€” Deployment Redundancy Gaps
+
+| Policy | Rule |
+|--------|------|
+| `RED-011` | The bootstrap manifest is version-controlled in the GitOps store, hash-verified at DCM startup and every restart. Bootstrap manifest tampering prevents DCM from starting and triggers a security alert. |
+| `RED-012` | Kubernetes cluster upgrades in Sovereign DCM deployments use pre-staged container images from the local signed bundle registry. DCM enters maintenance mode during upgrade. In-flight operations complete before upgrade. DCM exits maintenance mode on successful startup verification. |
+| `RED-013` | Kubernetes is DCM's primary and recommended container runtime. Non-Kubernetes runtimes are supported for development and community extension purposes only. Production deployments must use Kubernetes. |
+| `RED-014` | Minimum hardware specifications are expressed as DCM Resource definitions per profile and enforced by the placement engine during DCM self-deployment. |
+| `RED-015` | DCM's own deployment is a DCM-managed resource subject to the same drift detection as any other resource. Bootstrap manifest hash verification (RED-011) provides independent verification. Audit Store hash chain breaks are detectable externally. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/information-providers-advanced.md b/content/docs/architecture/data-model/information-providers-advanced.md
new file mode 100644
index 0000000..3d6baf7
--- /dev/null
+++ b/content/docs/architecture/data-model/information-providers-advanced.md
@@ -0,0 +1,694 @@
+---
+title: "Information Providers: Confidence Scoring and Authority"
+type: docs
+weight: 20
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Information Providers](10-information-providers.md) | [Policy Organization](14-policy-profiles.md) | [Universal Audit Model](16-universal-audit.md) | [DCM Federation](22-dcm-federation.md)
+
+---
+
+## 1. Purpose
+
+This document extends the base Information Provider model with the advanced concepts required for enterprise-grade information governance: confidence scoring for all provider-supplied data, authority and priority declarations as layer-defined organizational knowledge, ingestion-time conflict detection with policy-driven resolution, write-back capabilities, schema versioning, the well-known Information Provider Registry, and air-gapped verification.
+
+---
+
+## 2. Confidence Scoring โ€” The Hybrid Descriptor Model
+
+### 2.1 Purpose and Design Goals
+
+Every field value supplied by an Information Provider carries a confidence descriptor. DCM aggregates data from multiple external sources โ€” CMDB, HR systems, IPAM, asset management, monitoring tools โ€” each with different freshness, authority, and reliability. The confidence model answers: **how much should you trust this field value?**
+
+Three goals drive the design:
+- **Accuracy** โ€” each dimension of confidence is independently meaningful and auditable
+- **Reliability** โ€” a derived numeric score enables mathematical composition for placement decisions and conflict resolution
+- **Ease of use** โ€” a derived band (very_high through very_low) is what humans and policies work with day to day
+
+### 2.2 The Primary Data Model โ€” Confidence Descriptor
+
+The **confidence_descriptor** is the primary data model. The score and band are derived from it โ€” not the other way around. This separation removes false precision: the score is explicitly a convenience derivation, not an independent measurement.
+
+```yaml
+field_confidence:
+  # PRIMARY โ€” stored; set at specific lifecycle points
+  authority_level: primary          # set at provider registration from authority layer
+  corroboration: single_source      # set at ingestion; updated on subsequent pushes
+  source_trust: verified            # maintained by trust scoring system (INF-009)
+  last_updated_at:        # set at each push event
+  source_provider_uuid:       # set at ingestion
+
+  # DERIVED โ€” computed on demand; never stored as primary
+  freshness: high                   # computed: (now - last_updated_at) vs thresholds
+  data_age_minutes: 87              # computed: now - last_updated_at
+  score: 86                         # computed: from descriptor components
+  band: high                        # computed: from score vs band thresholds
+```
+
+### 2.3 Who Sets Each Descriptor Field
+
+| Field | Set By | When | How |
+|-------|--------|------|-----|
+| `authority_level` | Authority declaration layer | Provider registration | Organizational knowledge โ€” static per field per provider |
+| `corroboration` | DCM ingestion pipeline | Each push event | Compared against existing values from other providers |
+| `source_trust` | DCM trust scoring system | Event-triggered + weekly | Push failures, schema errors, health check, re-verification |
+| `last_updated_at` | DCM ingestion pipeline | Each push event | Timestamp of the push event |
+| `freshness` | DCM โ€” derived | Query time | Computed from `now - last_updated_at` vs thresholds |
+| `score` | DCM โ€” derived | Query time | Computed from descriptor components |
+| `band` | DCM โ€” derived | Query time | Computed from score vs band thresholds |
+
+**DCM computes all derived values โ€” providers never self-declare confidence.**
+
+### 2.4 Descriptor Component Values
+
+**`authority_level`** โ€” from the authority declaration layer:
+
+| Value | Meaning |
+|-------|---------|
+| `primary` | Declared primary authoritative source for this field |
+| `secondary` | Corroborating source; used if primary unavailable |
+| `advisory` | Context only; never used for decisions |
+| `discovered` | Value found via active interrogation |
+| `self_reported` | Entity reported its own value |
+| `inferred` | Value inferred from other data |
+
+**`corroboration`** โ€” computed at ingestion time:
+
+| Value | Condition | Confidence Effect |
+|-------|-----------|-----------------|
+| `confirmed` | 2+ providers agree on this value | Increases confidence |
+| `single_source` | Only one provider has asserted this value | Neutral |
+| `contested` | 2+ providers disagree on this value | Reduces confidence |
+
+**`source_trust`** โ€” maintained by trust scoring system:
+
+| Value | Condition | Confidence Effect |
+|-------|-----------|-----------------|
+| `verified` | Provider identity, sovereignty, certs all current | Full confidence |
+| `degraded` | Provider has elevated error/conflict rate | Reduced confidence |
+| `suspended` | Provider below trust threshold; pushes stopped | No new data |
+
+**`freshness`** โ€” computed from `data_age_minutes`:
+
+| Value | Age Threshold |
+|-------|-------------|
+| `high` | < 1 hour |
+| `medium` | 1 hour โ€“ 1 day |
+| `low` | 1 day โ€“ 7 days |
+| `stale` | > 7 days |
+
+### 2.5 The Score Derivation Formula
+
+The score is a convenience number derived deterministically from the descriptor. It enables mathematical composition (cross-DCM scoring, conflict resolution ordering) where a single number is needed.
+
+```
+score = min(100, base(authority_level)
+              ร— freshness_multiplier(freshness)
+              ร— corroboration_multiplier(corroboration)
+              ร— trust_multiplier(source_trust))
+```
+
+**Base values by authority_level:**
+
+| authority_level | Base Score |
+|----------------|-----------|
+| `primary` | 90 |
+| `secondary` | 70 |
+| `discovered` | 60 |
+| `advisory` | 50 |
+| `self_reported` | 40 |
+| `inferred` | 30 |
+
+**Freshness multipliers:**
+
+| freshness | Multiplier |
+|-----------|-----------|
+| `high` | 1.00 |
+| `medium` | 0.95 |
+| `low` | 0.85 |
+| `stale` | 0.50 |
+
+**Corroboration multipliers:**
+
+| corroboration | Multiplier |
+|--------------|-----------|
+| `confirmed` | 1.15 |
+| `single_source` | 1.00 |
+| `contested` | 0.60 |
+
+**Trust multipliers:**
+
+| source_trust | Multiplier |
+|-------------|-----------|
+| `verified` | 1.00 |
+| `degraded` | 0.75 |
+| `suspended` | 0.00 |
+
+**Example:** Primary authority, fresh data (30 min old), single source, verified:
+`min(100, 90 ร— 1.00 ร— 1.00 ร— 1.00)` = **90**
+
+**Example:** Primary authority, medium freshness (4 hours), two sources agree, verified:
+`min(100, 90 ร— 0.95 ร— 1.15 ร— 1.00)` = 98.3 โ†’ **98**
+
+**Example:** Secondary authority, stale data (10 days), contested, degraded:
+`min(100, 70 ร— 0.50 ร— 0.60 ร— 0.75)` = 15.75 โ†’ **16**
+
+### 2.6 Score Bands โ€” For Policy Use
+
+Policies use bands, not raw scores. This avoids the brittleness of threshold values like "reject if score < 73":
+
+| Band | Score Range | Policy Label |
+|------|------------|-------------|
+| Very High | 81-100 | `very_high` |
+| High | 61-80 | `high` |
+| Medium | 41-60 | `medium` |
+| Low | 21-40 | `low` |
+| Very Low | 0-20 | `very_low` |
+
+```yaml
+# Policy using band โ€” clear and maintainable
+policy:
+  type: gatekeeper
+  rule: >
+    If field.owner_business_unit.band IN [very_low, low]
+    THEN gatekeep: "Business unit confidence insufficient โ€” manual verification required"
+
+# Policy using individual descriptor dimensions โ€” most precise
+policy:
+  type: gatekeeper
+  rule: >
+    If field.cost_center.corroboration == contested
+    THEN gatekeep: "Cost center is contested between providers โ€” resolve before provisioning"
+
+# Policy using score โ€” for mathematical thresholds
+policy:
+  type: gatekeeper
+  rule: >
+    If field.cost_center.score < 60
+    THEN gatekeep: "Cost center confidence below required threshold"
+```
+
+### 2.7 Derivation Chain Summary
+
+```
+STORED (authoritative):
+  authority_level + corroboration + source_trust + last_updated_at
+
+DERIVED AT QUERY TIME (deterministic from stored):
+  freshness โ† (now - last_updated_at) vs thresholds
+  score     โ† base(authority_level) ร— freshness_mult ร— corroboration_mult ร— trust_mult
+  band      โ† score vs band thresholds
+
+AUDIT RECORD (what auditors can reconstruct from):
+  authority_level (from registration)
+  corroboration (from ingestion event)
+  source_trust (from trust audit at that time)
+  last_updated_at (from push event timestamp)
+  โ†’ score and band fully reconstructable from these four stored fields
+```
+
+### 2.8 Configurable Derivation
+
+Organizations may configure the base scores, multipliers, and band thresholds via Policy Group. This allows domain-specific calibration without changing the underlying descriptor model:
+
+```yaml
+confidence_derivation_config:
+  # Override defaults for this deployment
+  base_scores:
+    primary: 90           # default โ€” can increase to 95 for high-trust environments
+    secondary: 70
+  freshness_thresholds:
+    high_max_minutes: 60  # default 60; can tighten to 15 for real-time requirements
+    medium_max_minutes: 1440
+    low_max_minutes: 10080
+  band_thresholds:
+    very_high_min: 81     # default; adjust as needed
+    high_min: 61
+```
+
+Adjusted derivation configs are stored as Policy Group artifacts โ€” versioned, auditable, and profile-governed.
+
+## 3. Authority and Priority โ€” Layer-Defined
+
+### 3.1 Authority as Layer Data
+
+Information Provider authority scope and priority are **layer-defined** โ€” not just policy-driven. They represent static organizational knowledge about information architecture ("our CMDB is the authoritative source for business unit data"). This knowledge belongs in a `platform` domain layer โ€” versioned, GitOps-managed, and inherited by all requests.
+
+```yaml
+layer:
+  handle: "platform/information-authority/cmdb-authority"
+  domain: platform
+  priority: 600.0.0
+  concern_tags: [information-authority, cmdb, organizational-data]
+  fields:
+    information_authority:
+      primary_sources:
+        - provider_uuid: 
+          fields: [owner_business_unit, cost_center, cmdb_id, cmdb_location]
+          authority_level: primary
+          priority: 900.0.0
+        - provider_uuid: 
+          fields: [asset_tag, purchase_date, warranty_expiry, serial_number]
+          authority_level: primary
+          priority: 900.0.0
+      secondary_sources:
+        - provider_uuid: 
+          fields: [owner_business_unit, employee_id]
+          authority_level: secondary
+          priority: 500.0.0
+          # Secondary: corroborates primary; used if primary unavailable
+      advisory_sources:
+        - provider_uuid: 
+          fields: [reported_hostname, reported_ip]
+          authority_level: advisory
+          priority: 200.0.0
+          # Advisory: context only; never used for decisions
+```
+
+### 3.2 Priority Within Authority Level
+
+When multiple providers have the same `authority_level`, the `priority` field (using the same numeric priority schema as layers and policies) determines which value wins:
+
+```
+Higher priority value โ†’ wins when authority levels are equal
+Authority level hierarchy: primary > secondary > advisory
+Within same authority level: higher priority number wins
+```
+
+### 3.3 Policy Acting on Authority
+
+Policies can act on authority metadata at runtime:
+
+```yaml
+# Transformation: enrich payload with confidence-weighted values
+policy:
+  type: transformation
+  rule: >
+    If field.owner_business_unit.confidence_score < 60
+    AND field.owner_business_unit.authority_level != primary
+    THEN inject: request_flags.requires_manual_business_unit_verification = true
+
+# GateKeeper: require high confidence for financial operations
+policy:
+  type: gatekeeper
+  rule: >
+    If resource_type == Compute.VirtualMachine
+    AND field.cost_center.confidence_band IN [very_low, low]
+    THEN gatekeep: "Cost center assignment confidence insufficient for VM provisioning"
+```
+
+---
+
+## 4. Ingestion-Time Conflict Detection and Resolution
+
+### 4.1 Conflict Detection at Ingestion
+
+Conflict detection occurs at ingestion time โ€” when DCM receives a push event from an Information Provider. This is the correct architectural moment: before the data enters the entity record, not after.
+
+```
+Information Provider push event received
+  โ”‚
+  โ–ผ 1. Schema validation
+  โ”‚   Validate against provider's declared schema version
+  โ”‚   Reject on violation (strict) or warn (lenient per policy)
+  โ”‚
+  โ–ผ 2. Authority scope check
+  โ”‚   Is this provider authorized to assert values for these fields on this entity?
+  โ”‚   Reject unauthorized field assertions (INF-001)
+  โ”‚
+  โ–ผ 3. Confidence score computation
+  โ”‚   Compute per-field score using standard formula (Section 2.2)
+  โ”‚   Factor: source authority level, data freshness, corroboration
+  โ”‚
+  โ–ผ 4. Conflict detection
+  โ”‚   For each field: does an existing value exist from another provider?
+  โ”‚   Same value โ†’ corroboration (confidence increases, multiplier applied)
+  โ”‚   Different value โ†’ conflict record created
+  โ”‚   No existing value โ†’ new assertion (accept)
+  โ”‚
+  โ–ผ 5. Conflict resolution policy
+  โ”‚   Apply declared resolution strategy:
+  โ”‚   higher_authority_wins โ†’ use higher authority_level value
+  โ”‚   higher_confidence_wins โ†’ use higher confidence score
+  โ”‚   higher_priority_wins โ†’ use value from higher-priority provider
+  โ”‚   escalate โ†’ create conflict record; human resolves; existing value retained
+  โ”‚   merge โ†’ combine values (array/set fields only)
+  โ”‚
+  โ–ผ 6. Entity record update
+  โ”‚   Write accepted values with full field provenance
+  โ”‚
+  โ–ผ 7. Audit record โ€” INGEST action
+     All field changes, conflicts detected/resolved, confidence scores,
+     authority assertions recorded in Audit Store
+```
+
+### 4.2 The Conflict Record
+
+```yaml
+information_provider_conflict_record:
+  record_uuid: 
+  detected_at: 
+  field_path: owner_business_unit
+  subject_entity_uuid: 
+  conflicting_values:
+    - provider_uuid: 
+      authority_level: primary
+      priority: 900.0.0
+      value: "Payments Team"
+      confidence_score: 86
+      received_at: 
+    - provider_uuid: 
+      authority_level: secondary
+      priority: 500.0.0
+      value: "Infrastructure Team"
+      confidence_score: 72
+      received_at: 
+  resolution_strategy: higher_authority_wins
+  resolution_status: 
+  auto_resolved_value: "Payments Team"
+  auto_resolved_reason: "primary authority_level wins over secondary"
+  # If escalated:
+  resolved_by: 
+  resolved_value: "Payments Team"
+  resolution_reason: "CMDB is authoritative for business unit โ€” HR system has stale data"
+  resolution_audit_record_uuid: 
+```
+
+### 4.3 Authority Scope Conflicts at Registration
+
+When a new provider registers and declares authority over a field already claimed by an existing provider at the same or higher authority level, DCM detects the conflict at registration time and requires explicit resolution before the provider becomes active.
+
+```
+New provider registers authority_scope: [owner_business_unit, primary]
+  โ”‚
+  โ–ผ DCM checks: existing primary authority for owner_business_unit?
+  โ”‚
+  โ”œโ”€โ”€ No existing primary โ†’ register; no conflict
+  โ”‚
+  โ””โ”€โ”€ Existing primary provider found:
+        Create: authority_scope_conflict_record
+        Action required:
+          - Demote new provider to secondary, or
+          - Demote existing provider to secondary, or
+          - Declare explicit resolution strategy for this field
+        Provider registration blocked until resolved
+```
+
+---
+
+## 5. Write-Back Capability (Q63)
+
+Information Providers may optionally support write-back โ€” DCM updating external records when entity state changes. Write-back is triggered by policy, never automatic.
+
+```yaml
+information_provider_registration:
+  capabilities:
+    read: true                       # always required
+    write_back: true                 # optional
+    write_back_operations:
+      - operation: create
+        resource_types: [Compute.VirtualMachine]
+        fields: [hostname, ip_address, owner_business_unit, lifecycle_state]
+      - operation: update
+        resource_types: [Compute.VirtualMachine]
+      - operation: delete
+        resource_types: [Compute.VirtualMachine]
+    write_back_authentication:
+      mode: api_key
+      key_ref:
+        credential_provider_uuid: 
+        secret_path: "dcm/providers/info/cmdb/write-key"
+```
+
+**Policy triggers write-back:**
+
+```yaml
+policy:
+  type: transformation
+  placement_phase: post
+  rule: >
+    If action IN [CREATE, STATE_TRANSITION, DELETE]
+    AND resource_type == Compute.VirtualMachine
+    THEN trigger_write_back:
+      provider_uuid: 
+      operation: update
+      fields: [hostname, ip_address, lifecycle_state, owner_business_unit]
+```
+
+Write-back produces an audit record with `ENRICH` action and `source_type: information_provider_write_back`.
+
+---
+
+## 6. Extended Schema Versioning (Q64)
+
+Information Provider extended schemas follow semver semantics โ€” the same model as Resource Type Specifications.
+
+```yaml
+information_provider_registration:
+  extended_schema:
+    version: "2.1.0"
+    fields:
+      - name: cmdb_id
+        type: string
+        required: false
+      - name: cmdb_ci_class
+        type: string
+        required: false
+      - name: cmdb_location
+        type: object
+        required: false
+    changelog:
+      "2.0.0": "Removed deprecated cmdb_legacy_id field (major โ€” breaking)"
+      "2.1.0": "Added cmdb_location optional field (minor โ€” compatible)"
+    migration_plan:              # required for major version bumps
+      from_version: "1.x"
+      migration_script_ref: "git://cmdb-provider/migrations/v1-to-v2.yaml"
+      migration_window: P30D
+```
+
+**Semver semantics for extended schemas:**
+
+| Change | Version Bump | Reason |
+|--------|-------------|--------|
+| Field removed | **Major** | Breaking โ€” consumers may depend on it |
+| Field type changed | **Major** | Breaking โ€” consumers must update |
+| New optional field added | **Minor** | Compatible โ€” additive |
+| Description or constraint changed | **Revision** | Compatible โ€” no structural change |
+
+DCM validates incoming push data against the declared schema version. Major version bumps require a declared migration plan before the new schema version activates.
+
+---
+
+## 7. Well-Known Information Provider Registry (Q65)
+
+DCM maintains a three-tier Information Provider Registry following the same governance model as the Resource Type Registry.
+
+| Tier | Name | Contains | Examples |
+|------|------|---------|---------|
+| 1 | DCM Core | Universal integration patterns | Generic CMDB, Generic IPAM, Generic DNS |
+| 2 | Verified Community | Specific platform integrations | ServiceNow, Infoblox, NetBox, FreeIPA, AD, HashiCorp Vault |
+| 3 | Organization | Internal/proprietary | Acme ERP, Corp Asset Database |
+
+Well-known provider registrations include:
+- Pre-configured authority scope declarations
+- Pre-built extended schema definitions
+- Pre-configured write-back operation mappings
+- Connection templates with documented credential requirements
+- Example Policy Group activations for common use cases
+- Health check endpoint patterns
+
+The Information Provider Registry is **distinct from the Resource Type Registry** โ€” separate governance, separate GitOps repositories โ€” but shares the same infrastructure pattern: federated, PR-based proposals, automated validation, shadow validation period, and signed bundles for air-gapped import.
+
+---
+
+## 8. Air-Gapped Verification (Q66)
+
+Three modes for Information Provider verification in air-gapped environments:
+
+### 8.1 Mode 1 โ€” Pre-Verified Signed Bundle (Recommended)
+
+```yaml
+air_gapped_provider_bundle:
+  bundle_uuid: 
+  bundle_type: information_provider
+  signed_at: 
+  signing_key_ref: 
+  provider_registration: 
+  tls_certificate_chain: 
+  schema_definitions: 
+  verification_token: 
+  valid_until:         # bundles have expiry
+```
+
+### 8.2 Mode 2 โ€” Internal mTLS (Internal Providers)
+
+Providers that are themselves internal (internal CMDB, internal IPAM) register with `air_gap_mode: internal_only` and verify using internal mTLS certificates issued by the organization's internal CA (FreeIPA CA or equivalent).
+
+### 8.3 Mode 3 โ€” Periodic Online Re-Verification
+
+For environments air-gapped most of the time but with occasional connectivity windows:
+
+```yaml
+provider_verification:
+  mode: periodic_online
+  cache_ttl: P30D                # how long cached verification is valid
+  on_cache_expiry:
+    minimal: continue            # continue without re-verification
+    dev: alert                   # warn but continue
+    standard: alert              # warn but continue
+    prod: suspend                # suspend until re-verified
+    fsi: suspend
+    sovereign: suspend
+```
+
+---
+
+## 8a. Information Provider Trust Score Validation (Q15)
+
+### 8a.1 Dual-Trigger Model
+
+Information Provider trust scores are maintained per provider using the same dual-trigger pattern as conflict validation: event-triggered updates (primary) with scheduled re-verification (safety net).
+
+**Event-triggered updates:**
+- Provider push fails schema validation โ†’ `source_trust` degraded
+- Provider push conflicts with primary authority source โ†’ `source_trust` degraded
+- Health check fails โ†’ `source_trust` degraded
+- Sovereignty declaration change โ†’ trust re-evaluated against current Tenant requirements
+- Provider registration update (new credentials, endpoint change) โ†’ re-verification triggered
+
+**Scheduled re-verification:**
+- Daily: health check against all active Information Providers
+- Weekly: full re-verification (identity, sovereignty, certifications, schema compatibility)
+- `fsi` / `sovereign` profiles: daily full re-verification
+
+### 8a.2 Trust Score Structure
+
+```yaml
+information_provider_trust_score:
+  provider_uuid: 
+  score: 87                        # 0-100; contributes to source_trust field
+  scored_at: 
+  components:
+    identity_verified: true        # mTLS cert chain valid; re-checked weekly
+    endpoint_reachable: true       # health check; re-checked daily
+    schema_current: true           # schema version matches registered
+    sovereignty_compatible: true   # sovereignty matches Tenant requirements
+    certifications_current: true   # certifications not expired
+    push_error_rate:
+      rate: 0.02                   # 2% of pushes had schema/auth errors (7-day rolling)
+      weight: 0.15                 # contributes 15% to score degradation
+    conflict_rate:
+      rate: 0.05                   # 5% of pushes had value conflicts (7-day rolling)
+      weight: 0.10
+  decay_rate: per_7_days           # score decays if not re-verified
+  current_source_trust: verified   # verified | degraded | suspended
+  action_on_score_below:
+    threshold: 60
+    action: 
+    # suspend:       stop accepting pushes from this provider
+    # alert:         notify platform admin; continue with degraded trust
+    # reduce_weight: automatically degrade source_trust to degraded
+```
+
+### 8a.3 Trust Score to source_trust Mapping
+
+| Trust Score | source_trust | Effect on Confidence |
+|------------|-------------|---------------------|
+| โ‰ฅ 80 | `verified` | Full confidence multiplier (1.00) |
+| 60-79 | `degraded` | Reduced confidence multiplier (0.75) |
+| < 60 | `suspended` | No new data accepted; score = 0 |
+
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `INF-001` | Information Providers declare authority_level (primary, secondary, advisory) and authority_scope (resource types and fields). Conflicting authority scope declarations are detected at registration time. Conflicting field values from different providers at ingestion time are resolved per declared strategy. All conflicts produce audit records. |
+| `INF-002` | Information Providers may declare write_back capability with specific operations (create, update, delete) and resource types. Write-back is triggered by policy only โ€” never automatic. Write-back produces audit records with ENRICH action. Credentials reference a registered Credential Provider. |
+| `INF-003` | Information Provider extended schemas are versioned using semver. Removing a field or changing a field type is a major (breaking) version bump requiring a declared migration plan. Adding an optional field is a minor bump. DCM validates incoming push data against the declared schema version. |
+| `INF-004` | DCM maintains a three-tier Information Provider Registry (Core, Verified Community, Organization) following the same governance model as the Resource Type Registry. |
+| `INF-009` | Information Provider trust scores (0-100) are maintained per provider with event-triggered updates (push failure, schema mismatch, conflict, health check) and scheduled weekly re-verification. Trust score degradation transitions source_trust to degraded (reduced confidence multiplier). Suspension stops accepting pushes. Policy governs thresholds and actions per provider. |
+| `INF-005` | In air-gapped environments, Information Providers verify via pre-verified signed bundles, internal mTLS, or periodic online re-verification with cached tokens. Profile governs cache expiry behavior. |
+| `INF-006` | Information Provider field values carry a confidence score (0-100) computed from: source authority level, data freshness, and corroboration. DCM computes scores โ€” providers do not self-declare confidence. Scores decay with data age. |
+| `INF-007` | Authority scope and priority for Information Providers are declared in platform or system domain layers. Policy acts on confidence scores and bands โ€” gating, filtering, and escalating based on threshold declarations. |
+| `INF-008` | Conflict detection occurs at ingestion time. Policy governs automated resolution strategy. All conflicts โ€” detected, auto-resolved, and escalated โ€” produce INGEST audit records with the full conflict detail. |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should confidence score decay be linear or exponential? | Scoring model | โœ… Resolved โ€” freshness is discrete threshold bands not decay curve; stale multiplier 0.50 provides the cliff effect |
+| 2 | Should DCM expose confidence score aggregation APIs โ€” e.g., "average confidence across all fields for this entity"? | Consumer experience | โœ… Resolved โ€” per-entity confidence aggregation endpoint; overall band = lowest field band; computed on demand; identifies contested and stale fields (INF-010) |
+| 3 | Should conflicting providers receive notification when their value is overridden? | Provider relationship | โœ… Resolved โ€” provider opt-in override notifications; webhook or Message Bus; payload policy-governed; overriding value may be redacted (INF-011) |
+
+
+## 11. Information Provider Advanced Gap Resolutions
+
+### 11.1 Confidence Score Aggregation API (Q2)
+
+DCM exposes a per-entity confidence aggregation endpoint. The overall band reflects the lowest (most conservative) field band โ€” preventing high-scoring fields from masking problematic ones. Aggregations are computed on demand โ€” never stored (freshness changes continuously).
+
+```yaml
+confidence_aggregation_api:
+  endpoint: GET /api/v1/entities/{uuid}/confidence
+  response:
+    entity_uuid: 
+    overall_band: high              # lowest band across all fields (conservative)
+    field_summaries:
+      - field: owner_business_unit
+        band: high
+        score: 86
+        authority_level: primary
+        last_updated_at: 
+      - field: cost_center
+        band: medium
+        score: 54
+        corroboration: contested    # two providers disagree
+    lowest_confidence_fields:
+      - field: cost_center
+        reason: contested
+      - field: asset_tag
+        reason: stale               # data age > 7 days
+    computed_at: 
+```
+
+### 11.2 Conflicting Provider Override Notifications (Q3)
+
+Information Providers may opt in to override notifications. Not a universal default โ€” read-only public registries have no use for notifications; internal CMDBs may want to investigate discrepancies.
+
+```yaml
+information_provider_registration:
+  conflict_notification:
+    enabled: true                   # provider opts in
+    notification_channel: webhook   # or: message_bus
+    notification_endpoint: https://cmdb.corp.example.com/dcm-notifications
+    notify_on: [value_overridden, value_contested, authority_superseded]
+    notification_payload:
+      field_path: true
+      overriding_provider_uuid: true
+      overriding_value: false       # may be redacted for confidentiality
+      conflict_record_uuid: true
+```
+
+**Privacy:** The overriding value may be confidential (from a classified source). Policy governs what is included in the notification payload. GateKeeper can redact the overriding value if classified.
+
+### 11.3 System Policies โ€” Information Provider Advanced Gaps
+
+| Policy | Rule |
+|--------|------|
+| `INF-010` | DCM exposes a confidence aggregation endpoint per entity (GET /api/v1/entities/{uuid}/confidence). The overall confidence band reflects the lowest (most conservative) field band. Aggregations are computed on demand โ€” never stored. The response identifies contested and stale fields requiring attention. |
+| `INF-011` | Information Providers may opt in to override notifications by declaring conflict_notification in their registration. Notifications sent via webhook or Message Bus. The notification payload is policy-governed โ€” the overriding value may be redacted for confidentiality reasons. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/ingestion-model.md b/content/docs/architecture/data-model/ingestion-model.md
new file mode 100644
index 0000000..8ddb033
--- /dev/null
+++ b/content/docs/architecture/data-model/ingestion-model.md
@@ -0,0 +1,471 @@
+---
+title: "Ingestion Model"
+type: docs
+weight: 12
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md)
+
+---
+
+## 1. Purpose
+
+The DCM Ingestion Model is the **unified mechanism for bringing entities that exist outside DCM's lifecycle control into DCM's governance model**. It applies to three distinct sources:
+
+- **V1 Migration** โ€” entities from a DCM V1 deployment that predate the mandatory Tenant model
+- **Brownfield Discovery** โ€” entities discovered by a Service Provider that already exist in the infrastructure but are unknown to DCM
+- **Manual Import** โ€” entities imported from external systems (CMDBs, spreadsheets, legacy records) during onboarding
+
+All three sources follow the same pattern: ingest, enrich, and promote. The same data structures, the same governance policies, the same audit trail, and the same transitional holding mechanism apply regardless of source.
+
+**The three-step pattern:**
+
+```
+1. INGEST   โ€” bring the entity into DCM with whatever identity and metadata is available
+2. ENRICH   โ€” associate business data, ownership, Tenant assignment, and relationships
+3. PROMOTE  โ€” transition from holding state to full DCM lifecycle ownership
+```
+
+---
+
+## 2. Design Principles
+
+**Unified model โ€” minimum variance.** V1 migration and brownfield ingestion are the same fundamental operation. One model, one audit record structure, one set of governance policies.
+
+**Non-blocking.** Entities that cannot be immediately assigned a Tenant do not block migration or discovery. They land in the `__transitional__` Tenant and are resolved progressively. Migration does not require every entity to be assigned before any entity can proceed.
+
+**Provenance transparency.** Ingested entities are honest about their provenance depth. `created_via: migration` or `created_via: discovery` on the artifact metadata signals that the chain has limited depth. The ingestion record carries the confidence level.
+
+**Promotion gates governance.** An entity in a holding state cannot be the parent of a new allocated resource claim, cannot be used as a hard dependency by new requests, and cannot receive new operational relationships until promoted. Informational relationships are permitted โ€” entities can be referenced during enrichment.
+
+**Audit completeness.** Every ingested entity carries an `ingestion_record` in its provenance. Every Tenant assignment, enrichment action, and promotion event is recorded with actor, timestamp, and reason.
+
+---
+
+## 3. Ingestion Lifecycle States
+
+Entities going through ingestion follow a distinct mini-lifecycle before entering the standard entity lifecycle:
+
+```
+INGESTED
+  โ”‚  Entity exists in DCM. Minimal metadata. Tenant may be __transitional__.
+  โ”‚  Action: enrich โ€” add business data, assign relationships, assign real Tenant
+  โ–ผ
+ENRICHING
+  โ”‚  Tenant assigned. Metadata being completed. Relationships being established.
+  โ”‚  Action: complete enrichment, satisfy governance requirements
+  โ–ผ
+PROMOTED
+  โ”‚  All required fields present. Governance satisfied. Full DCM lifecycle assumed.
+  โ–ผ
+OPERATIONAL  (standard entity lifecycle from here)
+```
+
+### 3.1 State Behavior
+
+| State | Tenant | New Requests Can Use? | Parent for Allocations? | New Relationships? |
+|-------|--------|----------------------|------------------------|-------------------|
+| `INGESTED` | `__transitional__` or assigned | No | No | Informational only |
+| `ENRICHING` | Assigned | No | No | Operational (read-only) |
+| `PROMOTED` | Assigned | Yes | Yes | All types |
+| `OPERATIONAL` | Assigned | Yes | Yes | All types |
+
+### 3.2 Promotion Requirements
+
+Before an entity can be promoted, the following must be satisfied:
+
+- Assigned to a real Tenant (not `__transitional__`)
+- All `universal` fields on the Resource Type Specification are populated
+- All `constituent` relationships declared on the Resource Type Specification are resolved
+- At least one actor has reviewed and authorized the promotion
+- `ingestion_record.enrichment_status` is `complete`
+
+---
+
+## 4. The `__transitional__` Tenant
+
+The `__transitional__` Tenant is a DCM System artifact โ€” a system-managed holding area for entities that have been ingested but not yet assigned to a real Tenant.
+
+```yaml
+tenant:
+  uuid: 
+  handle: "__transitional__"
+  type: system_managed
+  purpose: ingestion_holding
+  governance:
+    max_residency_days: 90          # configurable per deployment
+    on_max_residency: escalate      # escalate | block | alert
+    escalation_endpoint: 
+  hard_tenancy:
+    cross_tenant_relationships: operational_only
+  artifact_metadata:
+    created_by:
+      display_name: "DCM Ingestion System"
+    created_via: system
+    status: active
+```
+
+**Properties:**
+- Cannot be deleted
+- Cannot be renamed
+- Cannot be used for new resource provisioning โ€” only ingestion assignment
+- Entities in `__transitional__` are fully auditable and visible in DCM
+- Governance policy enforces maximum residency and escalation
+
+---
+
+## 5. The Ingestion Record
+
+Every ingested entity carries an `ingestion_record` in its provenance chain. This is the audit record of how the entity entered DCM.
+
+```yaml
+ingestion_record:
+  ingestion_uuid: 
+  resource_entity_uuid: 
+  ingestion_timestamp: 
+
+  ingestion_source: 
+
+  # V1 migration fields (when ingestion_source: v1_migration)
+  v1_identifier: 
+  v1_metadata_snapshot: 
+
+  # Brownfield discovery fields (when ingestion_source: brownfield_discovery)
+  discovered_state_uuid: 
+  discovery_provider_uuid: 
+  discovery_timestamp: 
+
+  # Manual import fields (when ingestion_source: manual_import)
+  import_source_system: 
+  import_reference: 
+
+  # Common fields
+  assigned_tenant_uuid: 
+  assignment_method: 
+  assignment_signal: >
+    Human-readable description of what drove auto-assignment.
+    e.g., "Resource group membership: payments-group โ†’ Payments Tenant"
+    e.g., "Business unit metadata: BU-PAY โ†’ Payments Tenant"
+    e.g., "No signal found โ€” assigned to __transitional__"
+  assigned_by:
+    uuid: 
+    display_name: 
+    timestamp: 
+
+  ingestion_confidence: 
+  # high:   strong unambiguous signal โ€” auto-assignment reliable
+  # medium: inferred from metadata โ€” reasonable confidence, human review recommended
+  # low:    orphaned or conflicting signals โ€” assigned to __transitional__
+
+  enrichment_status: 
+  enrichment_history:
+    - sequence: 1
+      action: 
+      performed_by:
+        display_name: 
+      timestamp: 
+      detail: 
+
+  promoted_at: 
+  promoted_by:
+    display_name: 
+```
+
+---
+
+## 6. Auto-Assignment Signals
+
+When DCM ingests an entity, it attempts auto-assignment to a real Tenant using the following signals in priority order:
+
+| Signal | Confidence | Description |
+|--------|-----------|-------------|
+| Explicit ownership metadata | High | Business unit, cost center, or team tag on the resource maps unambiguously to a Tenant |
+| Resource group membership | High | Resource belongs to a group that maps to a known Tenant |
+| Request history | High | V1 request record identifies the requesting team, which maps to a Tenant |
+| Network / location context | Medium | Resource's location, VLAN, or network segment maps to a Tenant by convention |
+| Naming convention | Medium | Resource name matches a known Tenant naming pattern |
+| Provider context | Medium | Resource was provisioned by a known provider associated with a Tenant |
+| No signal found | Low | No auto-assignment possible โ€” entity goes to `__transitional__` |
+
+Multiple signals can be combined. If signals conflict, the higher-confidence signal wins and the conflict is recorded in the ingestion record with `ingestion_confidence: medium` regardless of individual signal strengths.
+
+---
+
+## 7. V1 Migration
+
+### 7.1 Overview
+
+V1 resources have no `tenant_uuid`. V2 requires one (`TEN-001`). The V1 migration process uses the ingestion model to assign every V1 resource a Tenant before it can participate in V2 operations.
+
+### 7.2 Resource Categories
+
+| Category | Description | Assignment Path |
+|----------|-------------|----------------|
+| **Auto-assignable** | Clear ownership signals โ€” resource group, business unit, request history | Auto-assigned during migration analysis pass |
+| **Manually assignable** | Ambiguous signals โ€” multiple possible owners, or medium-confidence signals only | Surfaced in admin assignment queue |
+| **Orphaned** | No signals โ€” no ownership data available | Assigned to `__transitional__` |
+
+### 7.3 Migration Flow
+
+```
+V1 estate
+  โ”‚
+  โ–ผ  Step 1 โ€” Pre-migration analysis pass
+  โ”‚  Inventory all V1 resources
+  โ”‚  Attempt auto-assignment via signals (Section 6)
+  โ”‚  Classify each resource: auto_assignable | manually_assignable | orphaned
+  โ”‚  Produce migration readiness report
+  โ”‚
+  โ–ผ  Step 2 โ€” Auto-assignment
+  โ”‚  Create or map to existing V2 Tenants
+  โ”‚  Assign auto_assignable resources in bulk
+  โ”‚  Create ingestion_record per resource (ingestion_source: v1_migration)
+  โ”‚  State: INGESTED โ†’ ENRICHING (for auto-assigned)
+  โ”‚
+  โ–ผ  Step 3 โ€” Manual assignment queue
+  โ”‚  manually_assignable resources surfaced in admin UI
+  โ”‚  Administrators review and assign Tenants
+  โ”‚  Each assignment recorded in enrichment_history
+  โ”‚
+  โ–ผ  Step 4 โ€” Transitional fallback
+  โ”‚  orphaned resources โ†’ __transitional__ Tenant
+  โ”‚  ingestion_record.assignment_method: transitional
+  โ”‚  ingestion_record.ingestion_confidence: low
+  โ”‚  Governance timer starts
+  โ”‚
+  โ–ผ  Step 5 โ€” Enrichment and promotion
+  โ”‚  Relationships established, missing fields populated
+  โ”‚  Each entity reviewed and promoted when complete
+  โ”‚  State: ENRICHING โ†’ PROMOTED โ†’ OPERATIONAL
+  โ”‚
+  โ–ผ  Migration complete when __transitional__ Tenant is empty
+```
+
+### 7.4 Migration System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ING-001` | Every entity ingested into V2 from V1 must be assigned to exactly one Tenant โ€” either a real Tenant or `__transitional__` โ€” before it is eligible for new V2 requests |
+| `ING-002` | Entities in `INGESTED` or `ENRICHING` state may not be the parent resource for a new allocated resource claim |
+| `ING-003` | The `__transitional__` Tenant is system-managed and cannot be deleted, renamed, or used for new resource provisioning |
+| `ING-004` | Every ingested entity must carry an `ingestion_record` in its provenance chain |
+| `ING-005` | Entities in `__transitional__` for longer than `max_residency_days` must trigger the configured escalation action |
+
+---
+
+## 8. Brownfield Ingestion
+
+### 8.1 Overview
+
+Brownfield ingestion brings infrastructure that already exists in the real world โ€” but is unknown to DCM โ€” under DCM lifecycle management. The source is the **Discovered State**: a Service Provider interrogates existing infrastructure and creates Discovered State records for everything it finds.
+
+This is the "greening the brownfield" use case โ€” taking an unmanaged estate and progressively bringing it under DCM governance without requiring a big-bang cutover.
+
+### 8.2 Brownfield Flow
+
+```
+Service Provider performs discovery scan
+  โ”‚  Interrogates existing infrastructure
+  โ”‚  Creates Discovered State records for all found entities
+  โ”‚
+  โ–ผ  DCM identifies "unmanaged" discovered entities
+  โ”‚  Discovered State records with no matching Realized State = unmanaged
+  โ”‚  These are brownfield candidates
+  โ”‚
+  โ–ผ  Ingestion initiation
+  โ”‚  Platform admin or automated policy initiates ingestion
+  โ”‚  DCM creates entity stubs with:
+  โ”‚    - New UUID (DCM-assigned)
+  โ”‚    - ingestion_source: brownfield_discovery
+  โ”‚    - State: INGESTED
+  โ”‚    - Tenant: __transitional__ (pending enrichment)
+  โ”‚    - ingestion_record linking to Discovered State UUID
+  โ”‚
+  โ–ผ  Enrichment
+  โ”‚  Business data associated (owner, cost center, purpose)
+  โ”‚  Tenant assigned based on auto-assignment signals
+  โ”‚  Relationships established to other entities
+  โ”‚  Missing fields populated from discovery data
+  โ”‚
+  โ–ผ  Promotion
+  โ”‚  Review and authorization by responsible actor
+  โ”‚  State: ENRICHING โ†’ PROMOTED
+  โ”‚  DCM assumes lifecycle ownership:
+  โ”‚    - Discovered State record becomes the initial Realized State
+  โ”‚    - Entity enters standard DCM lifecycle (OPERATIONAL)
+  โ”‚    - Drift detection active from this point forward
+  โ”‚
+  โ–ผ  OPERATIONAL
+     DCM now manages the full lifecycle of this previously unmanaged entity
+```
+
+### 8.3 Discovered โ†’ Realized Promotion
+
+When a brownfield entity is promoted, its Discovered State record is promoted to become the initial Realized State. This is the moment DCM assumes lifecycle authority:
+
+```yaml
+realized_state_record:
+  entity_uuid: 
+  source: brownfield_promotion
+  ingestion_uuid: 
+  discovered_state_uuid: 
+  promoted_at: 
+  promoted_by:
+    display_name: 
+  initial_realized_payload: 
+  provenance:
+    origin:
+      source_type: brownfield_discovery
+      source_uuid: 
+      timestamp: 
+```
+
+From this point, the standard drift detection cycle runs: future discoveries are compared against the Realized State and any deviations are flagged as drift.
+
+---
+
+## 9. Relationship to the Four States
+
+Ingestion interacts with the Four States model as follows:
+
+| Ingestion Source | States Involved | Flow |
+|-----------------|----------------|------|
+| V1 Migration | Intent โ†’ Requested โ†’ (no Realized yet) | V1 records treated as incomplete Requested State; migration creates minimal Realized State |
+| Brownfield Discovery | Discovered โ†’ Realized | Discovered State is promoted to Realized State at promotion |
+| Manual Import | None initially | Entity stub created; no prior state records; Realized State created at promotion from import data |
+
+In all cases: once an entity reaches `PROMOTED`, it has a Realized State record and full Four States tracking begins.
+
+---
+
+## 10. DCM System Policies โ€” Full List
+
+| Policy | Rule |
+|--------|------|
+| `ING-001` | Every entity ingested into DCM must be assigned to exactly one Tenant โ€” either a real Tenant or `__transitional__` โ€” before it is eligible for new requests |
+| `ING-002` | Entities in `INGESTED` or `ENRICHING` state may not be the parent resource for a new allocated resource claim |
+| `ING-003` | The `__transitional__` Tenant is system-managed โ€” cannot be deleted, renamed, or used for new resource provisioning |
+| `ING-004` | Every ingested entity must carry an `ingestion_record` in its provenance chain |
+| `ING-005` | Entities in `__transitional__` beyond `max_residency_days` must trigger the configured escalation action |
+| `ING-006` | A brownfield entity may not be promoted to `PROMOTED` state without explicit actor authorization |
+| `ING-007` | At promotion, the Discovered State record must be promoted to Realized State โ€” this is the moment DCM assumes lifecycle ownership |
+
+---
+
+## 11. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should the auto-assignment signal priority order be configurable per deployment? | Migration flexibility | โœ… Resolved โ€” platform domain layer declares priority; explicit_tenant_tag fixed first; default_tenant fixed last; middle signals configurable (ING-012) |
+| 2 | Can multiple entities be promoted in bulk? | Operational efficiency | โœ… Resolved โ€” bulk promotion supported; profile-governed max batch sizes; preview required; PT24H rollback; BULK_PROMOTE audit (ING-013) |
+| 3 | Should there be a maximum number of ingestion sources per entity? | Data integrity | โœ… Resolved โ€” profile-governed max (5 standard/prod, 3 fsi/sovereign); warn or reject on exceed (ING-014) |
+| 4 | How does ingestion interact with the Service Catalog? | Catalog model | โœ… Resolved โ€” ingested entities promotable to catalog items; bidirectional drift detection (ING-015) |
+
+---
+
+## 12. Related Concepts
+
+- **`__transitional__` Tenant** โ€” system-managed holding Tenant for unassigned ingested entities
+- **Ingestion Record** โ€” provenance record carried by every ingested entity
+- **Four States** โ€” Discovered State is the entry point for brownfield ingestion; Realized State is the output of promotion
+- **Brownfield** โ€” existing infrastructure not yet under DCM lifecycle management
+- **Drift Detection** โ€” begins for brownfield entities at the moment of promotion
+- **V1 Migration** โ€” migration of pre-Tenant DCM V1 entities to V2 using the ingestion model
+- **Greening the Brownfield** โ€” the progressive process of bringing unmanaged infrastructure under DCM lifecycle control
+
+
+## 8. Ingestion Gap Resolutions
+
+### 8.1 Configurable Signal Priority Order (Q1)
+
+The ingestion signal priority order is declared in a platform-domain layer and configurable per deployment. `explicit_tenant_tag` always has highest priority; `default_tenant` always has lowest. The middle signals may be reordered.
+
+```yaml
+layer:
+  handle: "platform/ingestion/signal-priority"
+  domain: platform
+  fields:
+    ingestion_signal_priority:
+      - explicit_tenant_tag         # fixed: always first
+      - provider_declared_tenant    # configurable order
+      - network_segment_mapping     # configurable order
+      - hardware_class_mapping      # configurable order
+      - geographic_location         # configurable order
+      - default_tenant              # fixed: always last
+```
+
+### 8.2 Bulk Entity Promotion (Q2)
+
+Bulk promotion is supported with profile-governed limits and approval requirements.
+
+```yaml
+bulk_promotion_config:
+  max_entities_per_bulk: 500        # configurable per profile
+  requires_approval: true
+  preview_required: true            # must review bulk preview before confirming
+  rollback_window: PT24H
+  audit_record: BULK_PROMOTE        # single audit event with full member list
+```
+
+| Profile | Max per Bulk | Approval Required |
+|---------|-------------|-----------------|
+| minimal | Unlimited | No |
+| dev | 1000 | No |
+| standard | 500 | Recommended |
+| prod | 100 | Yes |
+| fsi | 50 | Yes + dual approval |
+| sovereign | 25 | Yes + dual approval |
+
+### 8.3 Maximum Ingestion Sources per Entity (Q3)
+
+Profile-governed maximum to encourage clear data ownership and manageable conflict resolution.
+
+```yaml
+ingestion_source_limits:
+  max_sources_per_entity: 5         # default for standard/prod
+  on_max_exceeded: 
+  profile_defaults:
+    minimal: unlimited
+    dev: 10
+    standard: 5
+    prod: 5
+    fsi: 3
+    sovereign: 3
+```
+
+### 8.4 Ingestion to Service Catalog Promotion (Q4)
+
+Ingested entities may be promoted to Service Catalog items โ€” the pathway from brownfield discovery to catalog-driven management.
+
+```
+Ingested entity
+  โ†’ Operator associates entity with Resource Type Specification
+  โ†’ Fields validated against spec
+  โ†’ Service Catalog item created from entity's configuration
+  โ†’ Entity becomes template ("golden example") for this catalog item
+  โ†’ Future requests use catalog item
+  โ†’ Drift detection bidirectional:
+      entity drifts from catalog item โ†’ drift event
+      catalog item updated โ†’ entity flagged for review
+```
+
+### 8.5 System Policies โ€” Ingestion Gaps
+
+| Policy | Rule |
+|--------|------|
+| `ING-012` | Ingestion signal priority order is declared in a platform domain layer and configurable per deployment. explicit_tenant_tag always has highest priority. default_tenant always has lowest priority. Middle signals are reorderable. |
+| `ING-013` | Bulk entity promotion is supported with profile-governed maximum batch sizes and approval requirements. Preview required before confirmation. Rollback window PT24H. Single BULK_PROMOTE audit record with full member list. |
+| `ING-014` | Maximum ingestion sources per entity is profile-governed (default: 5 for standard/prod; 3 for fsi/sovereign). Exceeding the maximum triggers warn or reject per policy. |
+| `ING-015` | Ingested entities may be associated with Resource Type Specifications and promoted to Service Catalog items. Drift detection operates bidirectionally between the ingested entity and its associated catalog item. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md
new file mode 100644
index 0000000..a906013
--- /dev/null
+++ b/content/docs/architecture/data-model/policy-profiles.md
@@ -0,0 +1,1500 @@
+---
+title: "Policy Organization: Groups, Profiles, and Providers"
+type: docs
+weight: 13
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+> **Universal Group Model:** Policy Groups (`group_class: policy_collection`) and Policy Profiles (`group_class: policy_profile`) are expressions of the [Universal Group Model](15-universal-groups.md). The structures defined in this document remain authoritative for policy-specific behavior; the universal model adds composability, cross-type membership, and the ability to include policy groups within composite groups.
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Data Layers and Assembly](03-layering-and-versioning.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md)
+
+---
+
+## 1. Purpose
+
+DCM's Policy Engine is powerful โ€” but power without usability is a barrier to adoption. This document defines the **policy organization model**: the structures that make DCM easy to configure correctly for any use case, from a home lab evaluation to a sovereign financial services deployment.
+
+Three concepts work together:
+
+- **Policy Groups** โ€” cohesive collections of policies addressing a single identifiable concern (a technology, a compliance standard, a sovereignty requirement, a business process)
+- **Policy Profiles** โ€” complete DCM configurations for a specific use case, composed of Policy Groups
+- **Policy Providers** โ€” external authoritative sources that supply policies directly into DCM, extending the provider model to its fifth type
+
+The relationship is compositional:
+
+```
+Policy Profile     โ€” complete use-case configuration
+  โ”‚  composed of
+  โ–ผ
+Policy Groups      โ€” single-concern policy collections
+  โ”‚  composed of
+  โ–ผ
+Policies           โ€” individual Transformation / Validation / GateKeeper rules
+  โ”‚  optionally sourced from
+  โ–ผ
+Policy Providers   โ€” external authoritative policy sources
+```
+
+---
+
+## 2. Policy Groups
+## 1a. Two-Dimensional Profile Model
+
+### 1a.1 The Gap in the Original Model
+
+The original six profiles (minimal โ†’ sovereign) are organized around **deployment posture** โ€” how strict, how redundant, how governed. But organizations need compliance governance that is orthogonal to posture. A healthcare organization needs HIPAA controls regardless of whether they deploy at `standard` or `sovereign` posture. A payment processor needs PCI-DSS regardless of their redundancy profile.
+
+The new model makes compliance a first-class dimension that composes with posture:
+
+```
+Complete Profile = Deployment Posture Group + Compliance Domain Group(s)
+```
+
+### 1a.2 Dimension 1 โ€” Deployment Posture Groups
+
+Posture groups govern how the DCM infrastructure itself behaves โ€” redundancy, enforcement strictness, audit retention, tenancy model, cross-tenant defaults. These are the vertical axis from least to most governed.
+
+| Group Handle | Posture | Key Behaviors |
+|-------------|---------|--------------|
+| `system/group/posture-minimal` | Minimal | Advisory only; single instance; no redundancy |
+| `system/group/posture-dev` | Development | Warn-not-block; basic logging; ephemeral defaults |
+| `system/group/posture-standard` | Standard | Full enforcement; 3-replica; explicit cross-tenant |
+| `system/group/posture-prod` | Production | Full enforcement + SLA; geo-replicated; cost governance |
+| `system/group/posture-hardened` | Enterprise Hardened | 5-replica; 7-year audit; hard tenancy; dual approval |
+| `system/group/posture-sovereign` | Sovereign | Air-gap; deny_all cross-tenant; 10-year audit; signed bundles |
+
+### 1a.3 Dimension 2 โ€” Compliance Domain Groups
+
+Compliance domain groups govern what data handling, audit, and control requirements apply to resources managed by DCM. Multiple compliance domains can apply simultaneously.
+
+| Group Handle | Domain | Key Controls |
+|-------------|--------|-------------|
+| `system/group/compliance-fsi` | Financial Services | Basel III, SOX, Dodd-Frank financial controls |
+| `system/group/compliance-pci-dss` | Payment Card | PCI-DSS v4 โ€” payment card industry |
+| `system/group/compliance-hipaa` | Healthcare | HIPAA/HITECH PHI handling and audit |
+| `system/group/compliance-fedramp-moderate` | US Federal Moderate | FedRAMP Moderate โ€” NIST 800-53 Moderate baseline |
+| `system/group/compliance-fedramp-high` | US Federal High | FedRAMP High โ€” NIST 800-53 High baseline |
+| `system/group/compliance-dod-il2` | DoD IL2 | Public/unclassified defense data |
+| `system/group/compliance-dod-il4` | DoD IL4 | Controlled Unclassified Information (CUI) |
+| `system/group/compliance-dod-il5` | DoD IL5 | National Security Systems non-classified |
+| `system/group/compliance-dod-il6` | DoD IL6 | Classified โ€” maximum sovereign posture |
+| `system/group/compliance-government` | Government General | General government/public sector controls |
+| `system/group/compliance-gdpr` | EU Data Protection | GDPR data residency and rights |
+| `system/group/compliance-iso27001` | ISO 27001 | Information security management |
+| `system/group/compliance-nist-800-53` | NIST 800-53 | NIST security control framework |
+| `system/group/compliance-soc2` | SOC 2 | Service organization controls |
+| `system/group/compliance-nerc-cip` | Critical Infrastructure | Energy/utilities NERC CIP |
+| `system/group/compliance-sovereign` | Sovereign/Classified | Sovereign deployment, air-gap, classified |
+
+### 1a.4 Compliance Domain Group Contents
+
+#### `system/group/compliance-hipaa`
+
+HIPAA/HITECH controls for Protected Health Information (PHI):
+- PHI field classification enforcement (fields containing PHI must be tagged `phi: true`)
+- PHI access control (only roles with `phi_authorized: true` may access PHI-tagged fields)
+- Audit retention: P6Y minimum (HIPAA requires 6 years from creation or last use)
+- Encryption at rest: AES-256 required for all PHI storage
+- Transmission security: TLS 1.3 minimum for PHI in transit
+- Breach notification workflow: sovereignty_violation_record triggers HIPAA breach assessment
+- Business Associate Agreement (BAA) tracking: providers handling PHI must declare `baa_in_place: true` in sovereignty_declaration
+- Right to Access: consumer data export capability required for PHI entities
+- Minimum Necessary standard: data_request_spec on Mode 4 providers limited to minimum PHI fields
+
+#### `system/group/compliance-government`
+
+General government and public sector controls:
+- Data classification mandatory on all resources (classification_level field required)
+- Cross-boundary controls: data cannot cross classification levels without explicit policy
+- Audit retention: P10Y minimum
+- Air-gap capability required for Sensitive compartments
+- Actor authentication must declare clearance level in external_identity claims
+- All provider sovereignty_declarations must declare government_access_risk
+
+#### `system/group/compliance-fedramp-moderate`
+
+FedRAMP Moderate authorization controls:
+- NIST 800-53 Rev 5 Moderate baseline implemented as policy group
+- FedRAMP-authorized provider preference injected as placement constraint
+- Continuous monitoring: drift detection mandatory; P24H maximum drift resolution window
+- Incident response: webhook required for security events (audit.chain_break, drift.escalated)
+- POA&M tracking: `poam_status` field in status_metadata on all policy artifacts
+- Boundary protection: explicit ingress/egress control documentation
+
+#### `system/group/compliance-dod-il4`
+
+DoD Impact Level 4 โ€” Controlled Unclassified Information:
+- Inherits compliance-fedramp-moderate
+- CUI handling markers on all data fields containing controlled information
+- Sovereign posture within US jurisdiction boundary
+- Provider sovereignty_declaration must exclude foreign sub-processors
+- CMMC Level 2 cyber hygiene controls
+
+#### `system/group/compliance-sovereign`
+
+Sovereign and classified deployment controls:
+- All data must remain within declared sovereignty boundary
+- Air-gap capability: provider air_gap_capable: true required
+- Signed bundle import only โ€” no live registry connectivity
+- deny_all cross-tenant cross-boundary data flows
+- Hardware security module (HSM) required for key management
+- Audit records: 10-year retention; cryptographic signing required
+
+### 1a.5 Profile Composition Model
+
+The six built-in profiles become explicit posture+compliance compositions:
+
+```yaml
+system/profile/minimal:
+  policy_groups: [system/group/posture-minimal]
+
+system/profile/dev:
+  extends: system/profile/minimal
+  policy_groups: [system/group/posture-dev]
+
+system/profile/standard:
+  extends: system/profile/dev
+  policy_groups: [system/group/posture-standard]
+
+system/profile/prod:
+  extends: system/profile/standard
+  policy_groups: [system/group/posture-prod]
+
+system/profile/fsi:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/posture-hardened
+    - system/group/compliance-fsi
+    - system/group/compliance-pci-dss
+    - system/group/compliance-iso27001
+
+system/profile/sovereign:
+  extends: system/profile/fsi
+  policy_groups:
+    - system/group/posture-sovereign
+    - system/group/compliance-sovereign
+```
+
+### 1a.6 DCM Built-In Extended Profiles
+
+In addition to the six core profiles, DCM ships extended profiles for common compliance domains:
+
+```yaml
+system/profile/hipaa-prod:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/compliance-hipaa
+    - system/group/compliance-iso27001
+  description: "Production infrastructure with HIPAA/HITECH compliance"
+
+system/profile/hipaa-sovereign:
+  extends: system/profile/sovereign
+  policy_groups:
+    - system/group/compliance-hipaa
+  description: "Sovereign deployment with HIPAA/HITECH compliance โ€” highest healthcare posture"
+
+system/profile/fedramp-moderate:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/compliance-fedramp-moderate
+    - system/group/compliance-nist-800-53
+  description: "FedRAMP Moderate authorized deployment"
+
+system/profile/fedramp-high:
+  extends: system/profile/sovereign
+  policy_groups:
+    - system/group/compliance-fedramp-high
+    - system/group/compliance-nist-800-53
+  description: "FedRAMP High authorized deployment"
+
+system/profile/government:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/compliance-government
+    - system/group/compliance-nist-800-53
+  description: "General government and public sector deployment"
+
+system/profile/dod-il4:
+  extends: system/profile/sovereign
+  policy_groups:
+    - system/group/compliance-dod-il4
+    - system/group/compliance-fedramp-high
+    - system/group/compliance-nist-800-53
+  description: "DoD Impact Level 4 โ€” Controlled Unclassified Information"
+
+system/profile/dod-il5:
+  extends: system/profile/dod-il4
+  policy_groups:
+    - system/group/compliance-dod-il5
+  description: "DoD Impact Level 5 โ€” National Security Systems non-classified"
+
+system/profile/dod-il6:
+  extends: system/profile/dod-il5
+  policy_groups:
+    - system/group/compliance-dod-il6
+    - system/group/compliance-sovereign
+  description: "DoD Impact Level 6 โ€” Classified"
+```
+
+### 1a.7 Organization Custom Profiles โ€” Composition Examples
+
+```yaml
+# Healthcare with federal cloud authorization
+org/profile/hipaa-fedramp:
+  extends: system/profile/fedramp-moderate
+  policy_groups:
+    - system/group/compliance-hipaa
+  description: "Healthcare workloads on FedRAMP Moderate platform"
+
+# Multi-compliance financial + healthcare
+org/profile/fsi-hipaa:
+  extends: system/profile/fsi
+  policy_groups:
+    - system/group/compliance-hipaa
+  description: "FSI-grade deployment for organizations managing both financial and health data"
+
+# Tenant-level compliance override โ€” platform is standard; this Tenant is PCI-scoped
+# (declared in tenant_config โ€” not profile)
+tenant_compliance_overlay:
+  active_profile: system/profile/standard    # platform posture
+  compliance_groups:
+    - system/group/compliance-pci-dss        # this Tenant handles payment cards
+    - system/group/compliance-hipaa          # this Tenant also handles PHI
+  # Other Tenants on same platform have standard posture, no compliance overlay
+```
+
+### 1a.8 Compliance at Tenant Level
+
+Compliance domain groups may apply at Tenant level โ€” different Tenants on the same DCM deployment can have different compliance domains:
+
+```yaml
+tenant_config:
+  active_profile: system/profile/prod        # posture from platform
+  compliance_groups:
+    - system/group/compliance-hipaa          # this Tenant handles PHI
+    - system/group/compliance-pci-dss        # this Tenant processes payments
+```
+
+This is the critical capability: **one DCM deployment, multiple compliance postures per Tenant**. A hospital system can run a single DCM with: clinical Tenants (HIPAA), billing Tenants (HIPAA + PCI-DSS), and administrative Tenants (standard) โ€” all on the same platform profile.
+
+### 1a.9 System Policies โ€” Profile Composition
+
+| Policy | Rule |
+|--------|------|
+| `PROF-001` | Profiles compose a Deployment Posture Group with zero or more Compliance Domain Groups. Posture groups govern DCM infrastructure behavior. Compliance domain groups govern data handling, audit, and control requirements. |
+| `PROF-002` | Compliance Domain Groups may be applied at platform level (all Tenants) or Tenant level (specific Tenants). Tenant-level compliance groups are additive โ€” they do not replace platform-level groups. |
+| `PROF-003` | DCM ships built-in Compliance Domain Groups for: FSI, PCI-DSS, HIPAA/HITECH, FedRAMP Moderate, FedRAMP High, DoD IL2-IL6, Government, GDPR, ISO 27001, NIST 800-53, SOC2, NERC-CIP, and Sovereign/Classified. Organizations extend these groups or compose them into custom profiles. |
+| `PROF-004` | The `implementation_posture` concern_type Policy Groups (provenance model, auth simplicity, deployment complexity) are independent of compliance domain โ€” organizations select their implementation posture separately from their compliance requirements. |
+
+---
+
+
+### 2.1 Definition
+
+A **Policy Group** is a versioned, cohesive collection of policies that together address a **single identifiable concern**. The group is the unit of reuse โ€” activate a group to enable a concern, not individual policies.
+
+```yaml
+policy_group:
+  artifact_metadata:
+    uuid: 
+    handle: "system/group/pci-dss"
+    version: "1.2.0"
+    status: active
+    owned_by:
+      display_name: "DCM Project Team"
+      notification_endpoint: 
+
+  name: "PCI-DSS v4"
+  description: >
+    Policy group implementing PCI-DSS v4 controls relevant to
+    DCM-managed infrastructure. Enforces encryption standards,
+    network segmentation, access control, and audit requirements
+    for resources in PCI scope.
+
+  concern_type: compliance
+  concern_tags: [pci-dss, financial, encryption, network-segmentation]
+  extends: null   # or another group handle โ€” inherits all parent policies
+
+  # Source โ€” locally authored or from a Policy Provider
+  source:
+    type: 
+    provider_uuid: 
+    provider_group_reference: 
+    on_provider_update: 
+    # proposed: provider updates require local review before activation
+    # active:   provider updates activate immediately (trusted providers only)
+
+  # Constituent policies
+  policies:
+    - policy_uuid: 
+      handle: "system/gatekeeper/pci-encryption-aes256"
+      description: "Enforce AES-256 on all PCI-scoped storage"
+      placement_phase: pre
+    - policy_uuid: 
+      handle: "system/validation/pci-network-segmentation"
+      description: "Validate network segment isolation for PCI resources"
+      placement_phase: pre
+    - policy_uuid: 
+      handle: "system/transformation/pci-classification-inject"
+      description: "Auto-inject PCI classification on scoped resources"
+      placement_phase: pre
+    - policy_uuid: 
+      handle: "system/gatekeeper/pci-audit-retention"
+      description: "Enforce 10-year audit retention for PCI evidence"
+      placement_phase: pre
+
+  # Activation scope โ€” surgical application within a profile
+  activation_scope:
+    resource_types: []          # empty = all resource types
+    tenant_tags: [pci-scope]    # only Tenants tagged pci-scope
+    regions: []                 # empty = all regions
+
+  # Conflict declarations
+  conflicts_with:
+    - group_handle: "system/group/dev-defaults"
+      reason: "PCI requires blocking enforcement; dev-defaults uses warn-only"
+      resolution: this_group_wins
+```
+
+### 2.2 Concern Types
+
+| Type | Description | Examples |
+|------|-------------|---------|
+| `technology` | Policies specific to a technology or provider | kubevirt, openstack, kubernetes, vmware |
+| `compliance` | Regulatory or standards compliance | pci-dss, iso-27001, nist-800-53, fedramp |
+| `sovereignty` | Data residency, jurisdictional, air-gap | gdpr-eu, air-gap, data-residency-uk |
+| `business` | Business process, cost, lifecycle governance | cost-governance, ephemeral-resources, chargeback |
+| `operational` | Operational posture, defaults, SLAs | dev-defaults, hard-tenancy, sla-enforcement |
+| `security` | Security controls and posture | data-classification, zero-trust, encryption-baseline |
+| `implementation_posture` | Implementation complexity vs capability trade-offs | provenance-full-inline, provenance-deduplicated, single-instance-deployment, advisory-policies-only |
+
+### 2.3 Group Inheritance
+
+A Policy Group may extend another group โ€” inheriting all its policies and overriding or adding to them:
+
+```yaml
+policy_group:
+  handle: "org/group/pci-dss-extended"
+  extends: "system/group/pci-dss"
+  # Inherits all system/group/pci-dss policies
+  # Additional policies added below are on top of the parent
+  policies:
+    - policy_uuid: 
+      handle: "org/gatekeeper/our-pci-additional-control"
+```
+
+### 2.4 DCM Built-In Policy Groups
+
+DCM ships the following policy groups as part of its standard distribution:
+
+| Handle | Concern | Description |
+|--------|---------|-------------|
+| `system/group/core-minimal` | operational | Absolute minimum โ€” UUID requirements, basic well-formedness |
+| `system/group/dev-defaults` | operational | Warn-not-block, 90-day TTL defaults, single-auth cross-tenant |
+| `system/group/ephemeral-resources` | business | TTL enforcement, auto-expiry, short-lived resource defaults |
+| `system/group/audit-basic` | operational | Basic audit logging, 90-day retention |
+| `system/group/audit-compliance` | compliance | Compliance-grade audit, configurable retention |
+| `system/group/data-classification` | security | Data classification tagging and handling rules |
+| `system/group/cost-governance` | business | Budget enforcement, cost attribution, TTL governance |
+| `system/group/sla-enforcement` | operational | SLA tracking, availability commitments |
+| `system/group/hard-tenancy` | operational | Full tenant isolation, deny_all cross-tenant default |
+| `system/group/explicit-cross-tenant` | operational | Explicit cross-tenant authorization requirement (XTA-001 through XTA-005) |
+| `system/group/zero-trust` | security | Zero-trust network and identity enforcement |
+| `system/group/encryption-baseline` | security | AES-256 minimum, TLS 1.3, key rotation |
+| `system/group/pci-dss` | compliance | PCI-DSS v4 controls |
+| `system/group/gdpr-eu` | sovereignty | GDPR data residency and handling |
+| `system/group/nist-800-53` | compliance | NIST 800-53 control implementation |
+| `system/group/iso-27001` | compliance | ISO 27001 controls |
+| `system/group/fedramp-moderate` | compliance | FedRAMP Moderate baseline |
+| `system/group/air-gap` | sovereignty | Air-gapped deployment constraints |
+| `system/group/fsi-audit` | compliance | Financial services audit retention (7-year minimum) |
+| `system/group/lifecycle-ttl-enforcement` | operational | Lifecycle time constraint enforcement (LTC-001 through LTC-004) |
+| `system/group/kubevirt` | technology | KubeVirt provider-specific policies |
+| `system/group/openstack` | technology | OpenStack provider-specific policies |
+| `system/group/vmware` | technology | VMware provider-specific policies |
+| `system/group/provenance-full-inline` | implementation_posture | Model A โ€” all provenance inline on entity records; simplest; highest storage cost |
+| `system/group/provenance-deduplicated` | implementation_posture | Model B โ€” content-addressed deduplication; recommended; 95-99% storage reduction; lossless |
+| `system/group/provenance-tiered-archive` | implementation_posture | Model C โ€” hot/warm/cold tiers; balances cost and access speed |
+| `system/group/provenance-deduplicated-tiered` | implementation_posture | Model B+C โ€” maximum efficiency for very large-scale deployments |
+
+---
+
+## 3. Policy Profiles
+
+### 3.1 Definition
+
+A **Policy Profile** is a named, versioned, curated composition of Policy Groups that together configure DCM for a specific use case. Activating a profile is the primary configuration mechanism โ€” most deployments should activate a profile and then add organization-specific groups on top rather than configuring policies individually.
+
+```yaml
+policy_profile:
+  artifact_metadata:
+    uuid: 
+    handle: "system/profile/fsi"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "DCM Project Team"
+
+  name: "FSI Production"
+  description: >
+    Policy profile for Financial Services production deployments.
+    Enforces hard tenancy, regulatory-grade audit retention,
+    full sovereignty controls, and explicit cross-tenant authorization.
+
+  target_use_case: fsi_production
+  extends: "system/profile/prod"   # inherits all prod groups + overrides
+
+  enforcement_summary:
+    tenancy: hard_tenancy_required
+    audit_retention_years: 7
+    sovereignty: full
+    cross_tenant_default: explicit_only
+    policy_enforcement: blocking
+    immutable_fields: [sovereignty_zone, classification_level, audit_retention]
+    policy_version_pinning: permitted_with_elevation
+
+  policy_groups:
+    - group_handle: "system/group/fsi-audit"
+    - group_handle: "system/group/pci-dss"
+    - group_handle: "system/group/gdpr-eu"
+    - group_handle: "system/group/hard-tenancy"
+    - group_handle: "system/group/explicit-cross-tenant"
+    - group_handle: "system/group/encryption-baseline"
+```
+
+### 3.2 DCM Built-In Profiles
+
+DCM ships six profiles covering the spectrum from minimal to sovereign:
+
+#### `system/profile/minimal` โ€” Home Lab / Evaluation
+
+```yaml
+handle: "system/profile/minimal"
+name: "Minimal"
+extends: null
+description: >
+  Minimal configuration for home lab, local testing, and evaluation.
+  Most controls advisory only. Single Tenant auto-created on first use.
+  No audit requirements. No sovereignty enforcement.
+
+enforcement_summary:
+  tenancy: optional
+  audit_retention: none
+  sovereignty: none
+  cross_tenant_default: allow_all
+  policy_enforcement: advisory     # policies warn but do not block
+  time_constraints: optional
+
+auto_tenant:
+  enabled: true
+  default_tenant_handle: "default"
+  # TEN-001 satisfied silently โ€” no explicit Tenant declaration required
+
+policy_groups:
+  - group_handle: "system/group/core-minimal"
+```
+
+#### `system/profile/dev` โ€” Development Environments
+
+```yaml
+handle: "system/profile/dev"
+name: "Development"
+extends: "system/profile/minimal"
+description: >
+  Development environment profile. Tenancy recommended but not blocking.
+  Basic logging. Ephemeral resource defaults. Warn-not-block enforcement.
+
+enforcement_summary:
+  tenancy: recommended
+  audit_retention: basic
+  sovereignty: none
+  cross_tenant_default: operational_only
+  policy_enforcement: warn_only
+  default_ttl: P90D              # dev resources default to 90-day TTL
+
+policy_groups:
+  - group_handle: "system/group/dev-defaults"
+  - group_handle: "system/group/ephemeral-resources"
+  - group_handle: "system/group/audit-basic"
+```
+
+#### `system/profile/standard` โ€” General Enterprise Production
+
+```yaml
+handle: "system/profile/standard"
+name: "Standard"
+extends: "system/profile/dev"
+description: >
+  General enterprise production profile. Full policy enforcement.
+  Tenancy required. Explicit cross-tenant authorization. Basic
+  data classification. Compliance-grade audit.
+
+enforcement_summary:
+  tenancy: required
+  audit_retention: compliance_grade
+  sovereignty: configurable
+  cross_tenant_default: explicit_only
+  policy_enforcement: blocking
+
+policy_groups:
+  - group_handle: "system/group/data-classification"
+  - group_handle: "system/group/audit-compliance"
+  - group_handle: "system/group/explicit-cross-tenant"
+  - group_handle: "system/group/encryption-baseline"
+```
+
+#### `system/profile/prod` โ€” Production with SLA Requirements
+
+```yaml
+handle: "system/profile/prod"
+name: "Production"
+extends: "system/profile/standard"
+description: >
+  Production profile with SLA enforcement, cost governance, and
+  full lifecycle constraint enforcement.
+
+enforcement_summary:
+  tenancy: required
+  audit_retention: compliance_grade
+  sovereignty: configurable
+  cross_tenant_default: explicit_only
+  policy_enforcement: blocking
+
+policy_groups:
+  - group_handle: "system/group/cost-governance"
+  - group_handle: "system/group/sla-enforcement"
+  - group_handle: "system/group/lifecycle-ttl-enforcement"
+```
+
+#### `system/profile/fsi` โ€” Financial Services Production
+
+```yaml
+handle: "system/profile/fsi"
+name: "FSI Production"
+extends: "system/profile/prod"
+description: >
+  Financial services production. Hard tenancy. 7-year audit retention.
+  Full sovereignty enforcement. PCI-DSS and GDPR compliance.
+
+enforcement_summary:
+  tenancy: hard_tenancy_required
+  audit_retention_years: 7
+  sovereignty: full
+  cross_tenant_default: explicit_only
+  policy_enforcement: blocking
+
+policy_groups:
+  - group_handle: "system/group/fsi-audit"
+  - group_handle: "system/group/pci-dss"
+  - group_handle: "system/group/gdpr-eu"
+  - group_handle: "system/group/hard-tenancy"
+```
+
+#### `system/profile/sovereign` โ€” Air-Gapped / Sovereign Deployments
+
+```yaml
+handle: "system/profile/sovereign"
+name: "Sovereign"
+extends: "system/profile/fsi"
+description: >
+  Maximum control profile for air-gapped, sovereign, and highest-security
+  deployments. Complete tenant isolation. Zero external dependencies.
+  Maximum audit and sovereignty enforcement.
+
+enforcement_summary:
+  tenancy: hard_tenancy_required
+  audit_retention_years: 10
+  sovereignty: maximum
+  cross_tenant_default: deny_all
+  policy_enforcement: blocking
+  immutable_ceiling_on_all_sovereignty_fields: true
+
+policy_groups:
+  - group_handle: "system/group/air-gap"
+  - group_handle: "system/group/zero-trust"
+```
+
+### 3.3 Profile Inheritance Chain
+
+```
+system/profile/sovereign
+  extends: system/profile/fsi
+    extends: system/profile/prod
+      extends: system/profile/standard
+        extends: system/profile/dev
+          extends: system/profile/minimal
+            extends: null (base)
+```
+
+Each level adds groups without replacing parent groups. An organization extending a DCM profile only needs to declare what differs:
+
+```yaml
+# Organization custom profile
+org/profile/my-prod:
+  extends: system/profile/prod
+  policy_groups:
+    - group_handle: "org/group/our-naming-conventions"
+    - group_handle: "org/group/our-cost-centers"
+    - group_handle: "system/group/iso-27001"    # add a DCM compliance group
+```
+
+### 3.4 Profile Activation
+
+Profiles activate at three levels โ€” more specific takes precedence:
+
+```yaml
+# DCM installation default
+installation_config:
+  default_profile: "system/profile/minimal"
+
+# Platform-level (applies to all Tenants)
+platform_config:
+  active_profile: "system/profile/prod"
+  minimum_tenant_profile: "system/profile/dev"   # Tenants cannot go below this
+  maximum_tenant_profile: null                    # null = no ceiling
+
+# Tenant-level override
+tenant_config:
+  active_profile: "system/profile/fsi"           # must be >= minimum_tenant_profile
+```
+
+**A Tenant cannot activate a profile less restrictive than the platform minimum.** A sovereign deployment can set `minimum_tenant_profile: system/profile/sovereign` โ€” no Tenant can drop below that level.
+
+### 3.5 Profile Conflict Resolution
+
+When a profile is activated, DCM runs conflict detection across all constituent group policies โ€” same ingestion conflict detection that layers use. Conflicts must be resolved before a profile is marked `active`.
+
+Conflict resolution order:
+1. **Explicit `conflicts_with` declarations** on groups โ€” use declared resolution rule
+2. **Priority schema** โ€” higher numeric priority wins
+3. **Domain authority** โ€” `system` beats `platform` beats `tenant`
+4. **Unresolved** โ€” profile activation fails with detailed conflict report
+
+### 3.6 Profile Shadow Validation
+
+When a profile is in `proposed` status, all its constituent policies run in shadow mode โ€” the same proposed policy shadow execution model. The Validation Dashboard shows the aggregate impact across all policies in the profile before activation. This enables safe preview of what a profile upgrade would do to an existing deployment.
+
+---
+
+## 4. Policy Providers
+
+### 4.1 The Fifth Provider Type
+
+A **Policy Provider** is a fifth DCM provider type โ€” an external authoritative source that supplies policies directly into DCM or evaluates and enriches DCM data through an external logic engine.
+
+| Type | Purpose | DCM Owns Result? |
+|------|---------|-----------------|
+| **Service Provider** | Realizes resources | Yes |
+| **Information Provider** | Serves authoritative external data | No |
+| **Meta Provider** | Composes multiple providers | Yes |
+| **Storage Provider** | Persists DCM state | Yes |
+| **Policy Provider** | Supplies policies from external authoritative sources, or evaluates and enriches data via external logic | Policies and enrichment data become DCM-owned on import |
+
+**Why Policy Providers?** Organizations should not need to manually translate regulatory controls, security benchmarks, or vendor-specific policies into DCM format. A Policy Provider is the authoritative source โ€” DCM subscribes to it and receives updates automatically. For Mode 4, the external system is the authoritative logic engine โ€” DCM queries it and acts on the result.
+
+### 4.2 The Four Policy Provider Delivery Modes
+
+| Mode | Name | How it works | Logic lives in |
+|------|------|-------------|---------------|
+| **Mode 1** | DCM Native Push/Pull | Provider delivers DCM-format policy artifacts | DCM Policy Engine |
+| **Mode 2** | OPA/Rego Bundle | Provider delivers OPA Rego bundles | DCM Policy Engine (OPA) |
+| **Mode 3** | External Schema | Provider delivers external-format policies requiring naturalization | DCM Policy Engine (post-translation) |
+| **Mode 4** | Black Box Query-Enrichment | DCM sends a query, provider evaluates and/or enriches, returns structured result | External provider โ€” logic is opaque to DCM |
+
+Modes 1-3 are **policy delivery** modes โ€” the provider sends rules, DCM stores and executes them. Mode 4 is fundamentally different: the policy logic lives in the external system. DCM sends data, receives a result, and acts on it.
+
+### 4.3 Policy Provider Contract โ€” Modes 1-3
+
+Policy Providers in Modes 1-3 follow the same base contract as all providers: registration, health check, trust, and provenance emission.
+
+```yaml
+policy_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/policy/grc-platform-001"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Security Team"
+      notification_endpoint: 
+
+  name: "Enterprise GRC Platform"
+  description: "Publishes compliance controls from GRC platform to DCM"
+
+  delivery:
+    mode: 
+    pull_schedule: "0 2 * * *"    # cron โ€” if mode: pull
+    endpoint: 
+
+  policy_format: 
+
+  managed_domains:
+    concern_types: [compliance, sovereignty]
+    concern_tags: [pci-dss, gdpr, iso-27001]
+
+  trust_level: 
+  max_policy_authority: gatekeeper
+
+  on_update: proposed
+  on_provider_failure:
+    action: 
+    sunset_days: 30
+```
+
+### 4.4 Policy Naturalization (Mode 3)
+
+When a Policy Provider delivers policies in an external format, DCM applies **Policy Naturalization** โ€” translating external policy schemas into DCM policy format.
+
+```
+External Policy Format (OSCAL, XACML, CIS JSON, STIG XCCDF)
+  โ”‚
+  โ–ผ  Policy Naturalization (translator component)
+  โ”‚
+  โ–ผ
+DCM Policy Format (standard DCM policy artifact)
+  โ”‚
+  โ–ผ  Trust validation + conflict detection
+  โ”‚
+  โ–ผ
+DCM Policy Engine
+```
+
+```yaml
+naturalization:
+  source_schema: oscal           # oscal | xccdf | cis-json | xacml | custom
+  translator_uuid: 
+  validation_on_import: strict   # strict | lenient
+```
+
+### 4.5 Policy Provider and Policy Groups
+
+A Modes 1-3 Policy Provider can deliver at three levels of granularity:
+
+- **Individual policies** โ€” administrator manually assigns to groups
+- **Complete Policy Groups** โ€” provider supplies group definitions alongside policies
+- **Complete Policy Profiles** โ€” provider supplies a full deployment profile for one-step activation
+
+```yaml
+policy_group:
+  handle: "org/group/grc-pci-dss"
+  source:
+    type: policy_provider
+    provider_uuid: 
+    provider_group_reference: "pci-dss-v4-full"
+    on_provider_update: proposed
+    last_synced: 
+    provider_version: "4.0.1"
+```
+
+### 4.6 Trust Levels and Policy Authority
+
+| Trust Level | Max Policy Authority | Requires | Use Case |
+|-------------|---------------------|----------|---------|
+| `trusted` | GateKeeper | Manual elevation + dual approval | Regulatory body, certified compliance package |
+| `verified` | Validation | Registration + health check | Security vendor, GRC platform |
+| `untrusted` | Advisory (logged but not executed) | Registration only | Evaluation, new providers |
+
+Trust elevation requires explicit authorization from both a platform admin and a security owner.
+
+### 4.7 Policy Provider Health and Lifecycle
+
+- **Healthy** โ€” policies current, delivery working normally
+- **Degraded** โ€” delivery delayed or partial โ€” warnings emitted
+- **Unhealthy** โ€” policies move toward `deprecated` per `on_provider_failure` declaration
+- **Deregistered** โ€” policies deprecated with configured sunset
+
+---
+
+### 4.8 Mode 4 โ€” Black Box Query-Enrichment
+
+#### 4.8.1 Concept
+
+A **Mode 4 Policy Provider** is an external system that DCM queries during the assembly process to evaluate request data, return a decision, enrich the payload with additional fields, or do both simultaneously.
+
+**The key distinction from Modes 1-3:** The policy logic lives in the external system and is opaque to DCM. DCM does not receive or store the rules โ€” it sends a query and receives a structured result. The external system is the authoritative evaluator and enricher.
+
+**Mode 4 providers can:**
+- **Evaluate** โ€” return a pass/fail, score, or recommendation based on the query
+- **Enrich** โ€” inject additional fields into the payload (risk scores, compliance citations, cost predictions, organizational context, case references)
+- **Do both** โ€” evaluate and enrich in a single atomic query-response cycle
+
+**Examples:**
+- AI/ML risk scoring engine โ€” returns risk score AND injects mitigation recommendations
+- Compliance oracle โ€” returns pass/fail AND injects compliance citations
+- FinOps cost predictor โ€” returns predicted cost AND injects cost allocation metadata
+- Sovereignty verification service โ€” returns jurisdiction compliance AND injects residency certificates
+- Fraud detection system โ€” returns anomaly score AND injects case reference number
+- Identity enrichment service โ€” returns authorization AND injects organizational context (business unit, cost center, project codes)
+- Privileged access management system โ€” returns allow/deny AND injects access justification record
+
+#### 4.8.2 Governance Concerns
+
+Mode 4 introduces governance concerns that Modes 1-3 do not:
+
+**Data sovereignty on outbound data:** DCM is sending request payload data โ€” potentially sensitive โ€” to an external system. Before any data is sent, DCM must verify the provider is authorized to receive the data classifications present in the query. A sovereign deployment must prevent any data from leaving its boundary without explicit authorization.
+
+**Result integrity:** The black box returns a result that DCM acts on. The logic is opaque โ€” DCM cannot inspect it. The provenance chain must record the full query-response cycle: what was sent, what was returned, what action was taken, and the provider's audit token for cross-system correlation.
+
+**Enrichment governance:** Fields injected by a Mode 4 provider carry the same provenance obligations as fields injected by a Transformation Policy. The override control model applies โ€” a GateKeeper can refuse black box enrichment on sovereignty-sensitive fields. Enrichment output may itself carry classification implications and must be governed accordingly.
+
+**Failure behavior:** The black box is external and can be unavailable, slow, or malformed. Failure behavior must be explicitly declared โ€” the default is `gatekeep` (unknown is not safe).
+
+**Non-determinism:** A Mode 4 provider may return different results for the same input at different times (e.g., a risk model updated overnight). Result caching must be declared โ€” and cached results carry a validity period.
+
+#### 4.8.3 Registration โ€” Mode 4 Specific Fields
+
+```yaml
+policy_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/policy/risk-scoring-engine"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Security Engineering Team"
+      notification_endpoint: 
+
+  name: "Enterprise Risk Scoring Engine"
+  description: >
+    ML-based risk scoring and enrichment for infrastructure requests.
+    Returns risk score and injects mitigation recommendations.
+
+  delivery:
+    mode: black_box_query           # Mode 4
+    endpoint: 
+    query_protocol: 
+    timeout_seconds: 30
+    on_timeout: gatekeep            # gatekeep | allow | escalate
+    on_error: gatekeep
+    on_unavailable: gatekeep
+
+  # What data this provider is authorized to receive
+  data_request_spec:
+    fields_requested:
+      - field: resource_type
+        classification_ceiling: unclassified
+      - field: placement.selected_provider_uuid
+        classification_ceiling: internal
+      - field: requester.tenant_uuid
+        classification_ceiling: internal
+      - field: lifecycle_constraints.ttl
+        classification_ceiling: unclassified
+    # Fields NOT declared here are NEVER sent โ€” DCM enforces data minimization
+    # classification_ceiling: maximum classification level this provider may receive
+
+  # Where the provider operates โ€” sovereignty gating
+  operational_sovereignty:
+    jurisdiction: eu-west
+    certifications: [ISO-27001, GDPR-compliant, SOC2-Type2]
+    # DCM checks these against Tenant sovereignty requirements
+    # before authorizing any query
+
+  # Result and enrichment capabilities
+  result_capabilities:
+    result_type: 
+    # pass_fail:    decision only
+    # score:        numeric score with optional threshold
+    # recommendation: structured recommendation
+    # enrichment:   data injection only โ€” no decision
+    # multi_factor: decision + enrichment combined
+
+    # Decision component schema (if applicable)
+    decision_schema:
+      outcome_field: outcome        # field name in response
+      score_field: score            # field name for numeric score
+      confidence_field: confidence
+      citations_field: citations
+      valid_until_field: valid_until
+      audit_token_field: audit_token
+
+    # Enrichment component schema (if applicable)
+    enrichment_schema:
+      fields_injected:
+        - field: risk_score
+          type: float
+          classification: internal
+        - field: risk_citations
+          type: array
+          classification: internal
+        - field: recommended_mitigations
+          type: array
+          classification: internal
+
+  # Result caching
+  result_caching:
+    enabled: true
+    ttl_seconds: 300
+    cache_key_fields: [resource_type, requester.tenant_uuid]
+
+  trust_level: verified
+  max_policy_authority: transformation   # enrichment = transformation authority
+  # A Mode 4 provider that only evaluates can have validation or gatekeeper authority
+  # A Mode 4 provider that enriches requires at minimum transformation authority
+```
+
+#### 4.8.4 Data Sovereignty Governance โ€” Pre-Query Evaluation
+
+Before DCM sends any data to a Mode 4 provider, the Policy Engine evaluates:
+
+```
+Query to Mode 4 provider proposed
+  โ”‚
+  โ–ผ
+Data classification check (BBQ-001)
+  โ”‚  What classification levels are in the query payload fields?
+  โ”‚  Is each field's classification โ‰ค provider's declared ceiling?
+  โ”‚  โ†’ Any field exceeds ceiling: strip field or reject query
+  โ”‚
+  โ–ผ
+Sovereignty check (BBQ-003)
+  โ”‚  Does the provider's operational_sovereignty.jurisdiction
+  โ”‚  satisfy the requesting Tenant's sovereignty requirements?
+  โ”‚  โ†’ Incompatible: block query, apply on_sovereign_mismatch behavior
+  โ”‚
+  โ–ผ
+Data minimization (BBQ-002)
+  โ”‚  Strip all fields not in provider's data_request_spec
+  โ”‚  Apply field-level filtering per cross_tenant_authorization if applicable
+  โ”‚
+  โ–ผ
+Authorized โ†’ send minimized query
+Unauthorized โ†’ apply on_unavailable behavior (typically gatekeep)
+```
+
+#### 4.8.5 Assembly Process Integration
+
+Mode 4 providers participate in any assembly phase โ€” most usefully inside the placement loop where provider-specific data is available:
+
+```yaml
+policy:
+  placement_phase: loop         # pre | loop | post | both
+  evaluation_type: black_box_query
+  black_box_provider_uuid: 
+
+  # What to send โ€” must be subset of provider's data_request_spec
+  query_fields: [resource_type, placement.selected_provider_uuid, requester.tenant_uuid]
+
+  # How to act on the decision component
+  on_decision:
+    pass: continue
+    fail: 
+    score_below_threshold:
+      threshold: 0.7
+      action: reject_candidate   # try next provider candidate
+    score_above_threshold:
+      threshold: 0.9
+      action: continue
+
+  # How to act on the enrichment component
+  on_enrichment:
+    inject_fields: true          # inject returned fields into payload
+    override_existing: false     # do not overwrite fields already set
+    # Each injected field carries source_type: black_box_provider + audit_token
+```
+
+#### 4.8.6 Result Schema and Provenance
+
+**Full result structure:**
+
+```yaml
+black_box_result:
+  # Decision component (optional)
+  decision:
+    outcome: 
+    score: 0.83
+    confidence: 
+    citations:
+      - "Provider certification ISO-27001 current as of 2026-01-15"
+      - "No open security incidents in region eu-west-1a"
+    valid_until: 
+    audit_token: "BB-2026-03-26-00847-A"  # provider's internal reference
+
+  # Enrichment component (optional)
+  enrichment:
+    fields_to_inject:
+      - field: risk_score
+        value: 0.83
+        provenance_note: "Returned by risk scoring engine v2.3"
+      - field: risk_citations
+        value: ["ISO-27001:A.12.1", "No active incidents"]
+        provenance_note: "Risk scoring engine evidence"
+      - field: recommended_mitigations
+        value: ["Enable MFA", "Restrict egress to known endpoints"]
+        provenance_note: "Risk scoring engine recommendations"
+```
+
+**Provenance on injected enrichment fields:**
+
+Each field injected by a Mode 4 provider carries standard field-level provenance:
+
+```yaml
+risk_score:
+  value: 0.83
+  metadata:
+    override: allow              # standard override control applies
+    basis_for_value: "ML risk scoring engine evaluation"
+  provenance:
+    origin:
+      source_type: black_box_provider
+      source_uuid: 
+      timestamp: 
+      audit_token: "BB-2026-03-26-00847-A"
+      query_uuid: 
+```
+
+#### 4.8.7 Audit Record
+
+Every Mode 4 query-response cycle produces a `black_box_evaluation_record` in the Audit Store regardless of outcome:
+
+```yaml
+black_box_evaluation_record:
+  record_uuid: 
+  policy_uuid: 
+  request_uuid: 
+  provider_uuid: 
+  evaluated_at: 
+  placement_phase: loop
+
+  query_sent:
+    fields_included: [resource_type, placement.selected_provider_uuid]
+    # Field NAMES only โ€” not raw values. Values stored in provider's system.
+    # Full correlation via audit_token.
+    data_minimization_applied: true
+    sovereignty_check: passed
+    classification_ceiling_honored: true
+
+  result_received:
+    result_type: multi_factor
+    decision:
+      outcome: pass
+      score: 0.83
+      confidence: high
+      valid_until: 
+      audit_token: "BB-2026-03-26-00847-A"
+    enrichment:
+      fields_injected: [risk_score, risk_citations, recommended_mitigations]
+      override_existing_applied: false
+
+  action_taken: continue_with_enrichment
+  cached_result: false
+  cache_stored: true
+  cache_expires_at: 
+```
+
+The `audit_token` is the **cross-system audit bridge** โ€” DCM's record references the provider's internal record. Auditors can correlate DCM's audit trail with the black box provider's own logs for full end-to-end traceability.
+
+#### 4.8.8 Failure and Fallback Behavior
+
+| Condition | Default Behavior | Rationale |
+|-----------|-----------------|-----------|
+| `on_timeout` | `gatekeep` | Unknown is not safe |
+| `on_error` | `gatekeep` | Malformed response is not safe |
+| `on_unavailable` | `gatekeep` | External unavailability cannot bypass governance |
+| `on_sovereign_mismatch` | `gatekeep` | Sovereignty cannot be bypassed |
+| `on_classification_exceeded` | strip field or `gatekeep` | Data cannot be sent to unauthorized recipient |
+
+All failure behaviors are configurable. `allow` is available for non-critical enrichment where the enrichment is additive and the request can proceed safely without it. Organizations must explicitly declare `allow` โ€” it is never the default.
+
+#### 4.8.9 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `BBQ-001` | Before sending any data to a Mode 4 provider, the Policy Engine must verify the provider is authorized to receive the data classification levels present in the query |
+| `BBQ-002` | Data sent to a Mode 4 provider must be minimized to only the fields declared in the provider's `data_request_spec` |
+| `BBQ-003` | A Mode 4 provider's `operational_sovereignty` must be compatible with the requesting Tenant's sovereignty requirements before any query is dispatched |
+| `BBQ-004` | All Mode 4 query-response cycles must produce a `black_box_evaluation_record` in the Audit Store |
+| `BBQ-005` | Mode 4 provider failure behavior (`on_timeout`, `on_error`, `on_unavailable`) must be explicitly declared โ€” default is `gatekeep` |
+| `BBQ-006` | Cached Mode 4 results must include the original query timestamp and validity period in provenance |
+| `BBQ-007` | Fields injected into the payload by a Mode 4 provider enrichment must carry standard field-level provenance: `source_type: black_box_provider`, `source_uuid`, and `audit_token` |
+| `BBQ-008` | The override control model applies to fields injected by Mode 4 enrichment โ€” a GateKeeper policy may restrict or refuse black box enrichment on specific fields |
+| `BBQ-009` | A Mode 4 provider that performs enrichment requires at minimum `transformation` trust level authority |
+
+---
+
+## 5. Lifecycle Time Constraints
+
+### 5.1 Concept
+
+Lifecycle time constraints declare **when a resource should cease to exist or trigger a lifecycle action**. They are a first-class field on any resource entity โ€” not metadata, not a tag, but a governed field that follows the standard data model precedence and override control.
+
+### 5.2 Constraint Types
+
+| Type | Format | Description |
+|------|--------|-------------|
+| `ttl` | ISO 8601 duration (e.g., `P14D`) | Relative โ€” expires N time after the reference point |
+| `expires_at` | ISO 8601 timestamp | Absolute โ€” expires at a specific calendar date/time |
+
+When both are declared, the **earliest expiry wins** (LTC-004).
+
+### 5.3 Data Model
+
+```yaml
+lifecycle_constraints:
+  ttl:
+    duration: P14D                       # ISO 8601 duration โ€” 14 days
+    reference_point: realization_timestamp  # created_at | realization_timestamp | last_modified
+    on_expiry: 
+    metadata:
+      override: allow                    # standard override control applies
+      basis_for_value: "Consumer declared ephemeral โ€” 14-day lab resource"
+
+  expires_at:
+    timestamp: "2026-06-30T23:59:59Z"
+    on_expiry: notify
+    metadata:
+      override: immutable
+      locked_by_policy_uuid: 
+      basis_for_value: "Project deadline โ€” resource must not persist beyond Q2"
+
+  # Enforcement behavior
+  enforcement:
+    warn_before_expiry: P1D              # warn 1 day before expiry
+    warn_notification_endpoint: 
+    grace_period: PT1H                  # 1 hour grace after expiry before action
+    on_grace_period_expiry: 
+```
+
+### 5.4 Precedence
+
+Lifecycle time constraints follow the standard data model precedence chain:
+
+```
+Base Layer default (lowest โ€” e.g., "no TTL by default")
+  โ†“
+Core Layer (e.g., "all dev environment resources: TTL 90 days")
+  โ†“
+Service Layer (e.g., "ephemeral compute: TTL 7 days")
+  โ†“
+Request Layer (consumer declared TTL)
+  โ†“
+Transformation Policy (enrich TTL from business context)
+  โ†“
+GateKeeper Policy (highest โ€” may lock TTL as immutable)
+```
+
+A consumer can declare `ttl: P14D` in their request. A GateKeeper policy can override this to `P7D` and lock it immutable if organizational policy mandates shorter maximum lifetimes. A Core Layer can set default TTLs for resource classes. The provenance chain records every modification.
+
+### 5.5 Expiry Enforcement
+
+The **Lifecycle Constraint Enforcer** is a DCM control plane component that:
+- Monitors all realized entities against their declared lifecycle constraints
+- Fires the configured `on_expiry` action when a constraint is reached
+- Records the enforcement action in provenance and the Audit Store
+- Emits expiry warnings `warn_before_expiry` duration before the deadline
+
+Expiry enforcement is a DCM concern โ€” not a provider concern. The provider does not need to know about or implement TTL logic.
+
+### 5.6 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `LTC-001` | Lifecycle time constraints follow standard data model precedence โ€” layers, request, policies |
+| `LTC-002` | GateKeeper policies may lock lifecycle constraints as `override: immutable` or `immutable_ceiling: absolute` |
+| `LTC-003` | Expiry enforcement is a DCM control plane function โ€” not a provider concern |
+| `LTC-004` | When multiple time constraints exist on an entity, the earliest expiry wins |
+| `LTC-005` | Expired entities that fail to execute their `on_expiry` action enter `PENDING_EXPIRY_ACTION` state and trigger an escalation |
+
+---
+
+## 6. Cross-Tenancy Authorization Model
+
+### 6.1 Default Stance โ€” Closed
+
+Cross-tenant information sharing is **closed by default**. No cross-tenant relationship of any nature is permitted unless explicitly authorized. This applies to both operational dependencies and informational relationships.
+
+The hard tenancy spectrum:
+
+| Setting | Meaning |
+|---------|---------|
+| `deny_all` | No cross-tenant relationships of any nature |
+| `explicit_only` | All cross-tenant must be explicitly authorized (DEFAULT) |
+| `operational_permitted` | Operational cross-tenant permitted; informational requires explicit auth |
+| `allow_all` | All cross-tenant permitted โ€” requires justification; not available in sovereign profile |
+
+The default shifts from the Q59 model's `operational_only` to `explicit_only`. Informational sharing is no longer implicitly open โ€” every informational cross-tenant relationship requires an explicit authorization record.
+
+### 6.2 Cross-Tenant Authorization Record
+
+```yaml
+cross_tenant_authorization:
+  artifact_metadata:
+    uuid: 
+    handle: "tenant-a/auth/shared-network-read"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Infrastructure Tenant Admin"
+
+  # WHO
+  authorized_consumer_tenant_uuid: 
+  authorized_actor_constraint:
+    roles: [service_account, automation]    # null = any actor in the tenant
+    specific_uuids: []                      # specific actor UUIDs if needed
+
+  # WHAT
+  resource_entity_uuid: 
+  resource_type_uuid: 
+  permitted_fields:
+    - field: network_segment
+    - field: vlan_id
+    # empty list = all fields permitted
+  permitted_relationship_natures: [informational, operational]
+
+  # WHEN
+  valid_from: 
+  valid_until: 
+
+  # WHERE
+  permitted_in_regions: [eu-west, eu-central]   # null = any region
+  sovereignty_constraints:
+    must_honor_consuming_tenant_sovereignty: true
+    must_honor_owning_tenant_sovereignty: true
+
+  # GOVERNANCE
+  authorized_by_policy_uuid: 
+  authorization_level: 
+  # Hierarchy: field_specific > resource_specific > tenant_global
+  # More specific = higher precedence
+```
+
+### 6.3 Authorization Hierarchy
+
+More specific authorizations take precedence over broader ones:
+
+```
+field_specific     โ† highest precedence โ€” only these exact fields on this entity
+  โ”‚
+resource_specific  โ† this entity โ€” all permitted fields
+  โ”‚
+tenant_global      โ† all entities in this Tenant โ€” broadest
+```
+
+If a tenant-global policy says "allow informational sharing with Tenant B" but a resource-specific policy says "this resource is not shareable with anyone," the resource-specific policy wins.
+
+### 6.4 System Policies โ€” Cross-Tenancy
+
+| Policy | Rule |
+|--------|------|
+| `XTA-001` | Cross-tenant information sharing is closed by default โ€” explicit authorization required |
+| `XTA-002` | Cross-tenant authorizations must specify who, what, when, and where |
+| `XTA-003` | More specific authorizations take precedence: field_specific > resource_specific > tenant_global |
+| `XTA-004` | All cross-tenant authorization decisions are policy-driven and DCM-enforced |
+| `XTA-005` | Sovereignty constraints declared by either Tenant must be honored by all cross-tenant relationships |
+
+---
+
+## 7. Rehydration Tenancy Controls
+
+### 7.1 Tenancy and Sovereignty Are Always Current
+
+Tenancy controls, sovereignty directives, and cross-tenant authorizations are **always evaluated against current policies during rehydration**. They cannot be pinned to historical versions.
+
+```yaml
+rehydration:
+  re_evaluate: true/false          # governs placement
+  policy_version: current/pinned   # governs resource configuration policies
+  # The following are ALWAYS current โ€” cannot be pinned:
+  tenancy_controls: always_current
+  sovereignty_controls: always_current
+  cross_tenant_authorizations: always_current
+```
+
+### 7.2 Rehydration Tenancy Conflict
+
+When rehydration produces a tenancy or sovereignty constraint that conflicts with an existing cross-tenant allocation:
+
+```yaml
+rehydration_tenancy_conflict_record:
+  rehydration_request_uuid: 
+  entity_uuid: 
+  conflict_type: cross_tenant_authorization_conflict
+  original_authorization_uuid: 
+  current_policy_violation:
+    policy_uuid: 
+    violation: "Consuming Tenant no longer has authorization for this allocation"
+  action_taken: paused
+  entity_state: PENDING_REVIEW
+  notifications_sent:
+    - entity_owner
+    - owning_tenant_admin
+    - consuming_tenant_admin
+    - platform_admin
+  resolution_options:
+    - re_authorize
+    - release
+    - escalate
+  policy_override_available: true
+```
+
+### 7.3 System Policies โ€” Rehydration
+
+| Policy | Rule |
+|--------|------|
+| `RHY-001` | Tenancy, sovereignty, and cross-tenant authorizations always use current policies during rehydration |
+| `RHY-002` | Rehydration that conflicts with current tenancy/sovereignty pauses and enters PENDING_REVIEW |
+| `RHY-003` | A paused rehydration allocation is not automatically released โ€” requires explicit resolution |
+| `RHY-004` | A policy may declare automatic resolution behavior for rehydration tenancy conflicts |
+
+---
+
+## 8. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should organizations be able to submit custom profiles and groups back to the DCM project registry? | Community | โœ… Resolved โ€” community submissions via PR-based workflow; Tier 2; documented use case + deployment reference + test results + named maintainer (PROF-005) |
+| 2 | Should there be a certified profile program โ€” profiles that have been validated against specific regulatory frameworks? | Compliance | โœ… Resolved โ€” certified profile program with third-party certification metadata; certified profiles promoted to Tier 1; applies to artifact not deployment (PROF-006) |
+| 3 | Should Policy Provider trust elevation require a formal approval workflow in DCM UI or is out-of-band approval sufficient? | Security | โœ… Resolved โ€” formal approval workflow; profile-governed approvers (1 standard โ†’ 3 sovereign); P7D shadow period; POLICY_PROVIDER_ELEVATED audit (PROF-007) |
+| 4 | Should the default TTL for dev profile resources be configurable at the platform level or only at the group level? | Configuration | โœ… Resolved โ€” overridable at platform domain layer; per-resource-type TTL overrides; on_expiry action configurable (PROF-008) |
+| 5 | How does Policy Provider delivery interact with air-gapped deployments โ€” pull from internal mirror? | Sovereignty | โœ… Resolved โ€” signed bundle model identical to registry; Mode 4 in sovereign restricted to within-boundary endpoints (PROF-009) |
+
+---
+
+## 9. Related Concepts
+
+- **Policy Engine** โ€” executes the policies organized by groups and profiles
+- **Policy Layers** โ€” the assembly process step where policies execute
+- **Artifact Metadata** โ€” universal metadata on all policy artifacts
+- **Five Artifact Statuses** โ€” developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired
+- **Shadow Execution** โ€” proposed policies and profiles run in shadow mode for validation
+- **Override Control** โ€” policies set override control on fields; immutable_ceiling: absolute for non-negotiables
+- **Ingestion Model** โ€” policy profile requirements may gate brownfield promotion
+- **Four States** โ€” rehydration tenancy controls govern how historical states are replayed
+
+
+## 8. Policy Profile Gap Resolutions
+
+### 8.1 Community Profile and Group Submissions (Q1)
+
+Organizations may submit custom profiles and policy groups to the DCM community registry following the same PR-based proposal workflow as Resource Types. Community contributions live in Tier 2 (Verified Community).
+
+```yaml
+community_profile_submission:
+  profile:
+    handle: "community/profile/hipaa-openstack"
+    extends: system/profile/hipaa-prod
+    description: "HIPAA production profile for OpenStack deployments"
+    policy_groups:
+      - system/group/compliance-hipaa
+      - community/group/openstack-security-baseline
+    contributed_by: "Healthcare IT Community Group"
+    tested_with: [OpenStack 2024.1, DCM 1.0]
+  required_for_submission:
+    - documented_use_case
+    - at_least_one_real_deployment_reference
+    - test_results_against_reference_implementation
+    - named_maintainer
+```
+
+Community profiles carry the same lifecycle as Resource Types โ€” shadow validation before active, deprecation policies, sunset periods. Organizations adopt them directly or extend them further.
+
+### 8.2 Certified Profile Program (Q2)
+
+DCM supports a certified profile program where profiles carry formal third-party certification metadata against compliance frameworks. Certified profiles are promoted to Tier 1 (DCM Core).
+
+```yaml
+profile_certification:
+  certifications:
+    - framework: HIPAA
+      certifying_body: "Coalfire Systems"
+      certification_date: "2025-11-01"
+      valid_until: "2027-11-01"
+      certification_scope: "PHI data lifecycle management via DCM"
+      certificate_ref:
+        credential_provider_uuid: 
+        path: "dcm/registry/certifications/hipaa-prod-2025"
+```
+
+**Important:** Profile certification applies to the profile artifact only โ€” it does not certify the deploying organization's compliance posture. A certified profile is evidence that the profile implements the required controls; it is not a compliance certification of any specific deployment.
+
+### 8.3 Policy Provider Trust Elevation Approval (Q3)
+
+Policy Provider trust elevation (increasing the mode level) requires a formal approval workflow. Approval requirements are profile-governed.
+
+```yaml
+policy_provider_trust_elevation:
+  elevation_request:
+    from_mode: 1
+    to_mode: 3
+    justification: "Need OPA Rego for complex placement constraints"
+
+  approval_requirements:
+    standard:
+      approvers: [platform_admin]
+      min_approvers: 1
+    prod:
+      approvers: [platform_admin, security_owner]
+      min_approvers: 2
+    fsi:
+      approvers: [platform_admin, security_owner, compliance_officer]
+      min_approvers: 2
+      dual_approval_required: true
+    sovereign:
+      approvers: [platform_admin, security_owner, compliance_officer]
+      min_approvers: 3
+      requires_change_control_ticket: true
+
+  shadow_period_after_elevation: P7D    # elevated mode runs in shadow before active
+  audit_record: POLICY_PROVIDER_ELEVATED
+```
+
+The P7D shadow period catches unintended consequences before elevated outputs become binding on production requests.
+
+### 8.4 Dev Profile Resource TTL Configurability (Q4)
+
+The default TTL for dev profile resources is declared in the system domain layer and overridable at the platform domain level.
+
+```yaml
+layer:
+  handle: "platform/dev-profile/resource-ttl-override"
+  domain: platform
+  fields:
+    dev_profile_resource_ttl:
+      default_ttl: P30D               # platform override: 30d instead of system default P7D
+      max_ttl: P90D                   # consumers cannot declare TTL > 90 days in dev
+      on_expiry: notify               # notify (consumers can extend) vs destroy
+      per_resource_type_overrides:
+        Compute.VirtualMachine: P7D
+        Storage.Block: P14D
+        DNS.Record: P3D
+```
+
+### 8.5 Air-Gapped Policy Provider Delivery (Q5)
+
+Policy Provider delivery in air-gapped deployments uses signed bundles โ€” same model as the registry bundle system.
+
+```yaml
+policy_provider_airgap:
+  delivery_mode: signed_bundle
+  bundle_contents:
+    - provider_registration_yaml
+    - policy_artifacts_zip
+    - mode_specific_package:
+        mode_3: opa_rego_bundle       # OPA Rego files + data
+        mode_4: endpoint_config       # endpoint declaration (must be within boundary)
+  signing_key_ref: 
+  valid_until: 
+```
+
+**Mode 4 sovereign constraint:** In sovereign profiles, Mode 4 Policy Providers may only call endpoints within the sovereignty boundary. External AI service calls are blocked by the BBQ-001 sovereignty check before any Mode 4 query.
+
+### 8.6 System Policies โ€” Policy Profile Gaps
+
+| Policy | Rule |
+|--------|------|
+| `PROF-005` | Organizations may submit custom profiles and policy groups to the DCM community registry via the same PR-based proposal workflow as Resource Types. Community contributions live in Tier 2. Submissions require documented use case, at least one production deployment reference, test results, and a named maintainer. |
+| `PROF-006` | DCM supports a certified profile program where profiles carry formal third-party certification metadata. Certified profiles are promoted to Tier 1. Profile certification applies to the artifact only โ€” it does not certify the deploying organization's compliance posture. |
+| `PROF-007` | Policy Provider trust elevation requires a formal approval workflow (standard: 1 platform admin; prod: platform admin + security owner; fsi/sovereign: dual approval + compliance officer). Elevated providers run in shadow mode for P7D before activation. All elevations produce a POLICY_PROVIDER_ELEVATED audit record. |
+| `PROF-008` | The default TTL for dev profile resources is declared in the system domain layer and overridable at the platform domain level. Per-resource-type TTL overrides are supported. The on_expiry action is configurable. |
+| `PROF-009` | Policy Provider delivery in air-gapped deployments uses signed bundles identical to the registry bundle model. Mode 4 providers in sovereign profiles may only call endpoints within the sovereignty boundary. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/registry-governance.md b/content/docs/architecture/data-model/registry-governance.md
new file mode 100644
index 0000000..8c1e1d9
--- /dev/null
+++ b/content/docs/architecture/data-model/registry-governance.md
@@ -0,0 +1,614 @@
+---
+title: "Registry Governance"
+type: docs
+weight: 19
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Auth Providers](19-auth-providers.md)
+
+---
+
+## 1. Purpose
+
+The Resource Type Registry is the authoritative catalog of Resource Type Specifications available to DCM deployments. It governs what resources can be requested, how they are defined, and how those definitions evolve over time. Registry governance defines how new types are proposed, reviewed, approved, versioned, deprecated, and distributed โ€” including in air-gapped and sovereign deployments.
+
+Registry governance follows the same principles as all other DCM governance: GitOps-managed, policy-driven, profile-governed for ease of use, and audited.
+
+---
+
+## 2. The Three-Tier Registry
+
+### 2.1 Registry Tiers
+
+| Tier | Name | Maintained By | Contains | Governed By |
+|------|------|--------------|---------|------------|
+| 1 | **DCM Core** | DCM Project team | Universal resource types | DCM maintainers + community |
+| 2 | **Verified Community** | Named community maintainers | Technology/platform-specific types | Named maintainer(s) + DCM oversight |
+| 3 | **Organization** | Deploying organization | Organization-specific/proprietary types | Organization's own process |
+
+**Tier 1 examples:** `Compute.VirtualMachine`, `Network.VLAN`, `Network.IPAddress`, `Storage.Block`, `Storage.File`, `Container.Pod`
+
+**Tier 2 examples:** `OpenStack.HeatStack`, `VMware.NSXSegment`, `KubeVirt.VirtualMachine`, `Ansible.Playbook`
+
+**Tier 3 examples:** `Acme.LegacyMainframeJob`, `Corp.ServiceNowTicket`, `Internal.ComplianceReport`
+
+### 2.2 The Federated Registry Model
+
+The registry uses a federated model โ€” not centralized, not fully distributed. This supports air-gapped and sovereign deployments without external dependencies.
+
+```
+DCM Project Registry (authoritative origin)
+  Published at: registry.dcm-project.github.io
+  Contains: Tier 1 Core + Tier 2 Verified Community
+  โ”‚
+  โ–ผ  Sync (scheduled pull)
+Organization Registry (local mirror)
+  Hosted internally by the deploying organization
+  Adds: Tier 3 Organization-specific types
+  Authoritative for: this organization's DCM deployments
+  Can operate offline: yes โ€” pulls during sync windows
+  โ”‚
+  โ–ผ  Signed bundle transfer (for air-gapped)
+Air-gapped Registry (offline copy)
+  No external connectivity required
+  Updated via signed bundles verified against org public key
+  Authoritative for: this sovereign/air-gapped deployment
+```
+
+---
+
+## 3. Proposal and Review Workflow
+
+### 3.1 The PR-Based Proposal Flow
+
+Resource Type proposals follow a GitOps PR-based workflow โ€” not form submissions or tickets. A proposal is a Pull Request against the registry repository.
+
+```
+1. Author creates Resource Type Specification draft
+   โ”œโ”€โ”€ Standard artifact format (uuid, handle, version, status: developing)
+   โ”œโ”€โ”€ Schema definition
+   โ”œโ”€โ”€ Lifecycle declarations
+   โ”œโ”€โ”€ Declared dependencies (must exist in registry)
+   โ””โ”€โ”€ At least one example request payload
+
+2. Author opens Pull Request
+   โ”œโ”€โ”€ PR template: use case justification, example provider implementation,
+   โ”‚   test cases, schema validation passing
+   โ””โ”€โ”€ Status automatically set to: proposed (on PR open)
+
+3. Automated validation gates (must all pass before review begins)
+   โ”œโ”€โ”€ Schema validator passes
+   โ”œโ”€โ”€ No FQN conflict with existing active entries
+   โ”œโ”€โ”€ All declared dependencies resolve
+   โ”œโ”€โ”€ Breaking change detector (if version > 1.0.0)
+   โ””โ”€โ”€ Test case coverage (at least one valid example payload)
+
+4. Community review period (see Section 3.2)
+
+5. Maintainer approval + merge
+   โ””โ”€โ”€ Status: proposed โ†’ enters shadow validation
+
+6. Shadow validation period (same duration as review period)
+   โ”œโ”€โ”€ Specification available to DCM deployments opted into proposed feed
+   โ”œโ”€โ”€ Issues reported back as PR comments
+   โ””โ”€โ”€ Must pass without critical issues before promotion
+
+7. Promotion to active
+   โ””โ”€โ”€ Status: active โ†’ available in standard registry feed
+```
+
+### 3.2 Review Periods by Change Type
+
+| Change Type | Min Review Period | Shadow Validation | Approvers Required |
+|-------------|-----------------|-------------------|-------------------|
+| New Tier 1 resource type | 14 days | 14 days | 2 DCM maintainers |
+| New Tier 2 resource type | 7 days | 7 days | 1 DCM maintainer + named tier maintainer |
+| Minor version (non-breaking) | 7 days | 7 days | 1 DCM maintainer |
+| Revision (config data only) | 3 days | 3 days | 1 DCM maintainer (or auto-approve if CI passes) |
+| Breaking change (major version) | 21 days | 21 days | 2 DCM maintainers + community comment period |
+| Deprecation | 30 days | N/A | 2 DCM maintainers + affected provider notification |
+| Emergency (security) | Waived | 7 days minimum | 2 DCM maintainers + immediate notification |
+
+---
+
+## 4. Versioning
+
+### 4.1 Version Schema
+
+Resource Type Specifications use semantic versioning: `Major.Minor.Revision`
+
+| Component | Meaning | Compatibility |
+|-----------|---------|--------------|
+| **Major** | Breaking change โ€” field removed, type changed, behavior incompatible | Not compatible with previous major |
+| **Minor** | Non-breaking addition โ€” new optional fields, new lifecycle states | Compatible within major |
+| **Revision** | Configuration data change โ€” no structural change | Compatible within minor |
+
+### 4.2 Version Resolution Policy
+
+Version constraints in requests are **strictly enforced** โ€” DCM never silently resolves to a different version than declared. The resolution policy governs how much flexibility a consumer has:
+
+```yaml
+resource_type_version_constraint:
+  resource_type: Compute.VirtualMachine
+  version_policy: 
+  # exact:        Must match โ€” "1.2.3" means only 1.2.3
+  # compatible:   Same major โ€” "^1.2.3" means >= 1.2.3 < 2.0.0
+  # latest_minor: Latest revision of specified minor โ€” "~1.2" means 1.2.x
+  # latest:       Always use the latest active version
+  pinned_version: "1.2.3"   # required if version_policy: exact
+```
+
+**DCM never automatically upgrades across major versions regardless of version_policy.** Moving from v1.x to v2.x always requires explicit consumer action.
+
+### 4.3 Profile-Governed Version Policy Defaults
+
+| Profile | Default Version Policy | Rationale |
+|---------|----------------------|-----------|
+| `minimal` | `latest` | Home lab โ€” always current, no pinning overhead |
+| `dev` | `compatible` | Dev โ€” tracks major version, picks up fixes automatically |
+| `standard` | `compatible` | Production โ€” stable within major version |
+| `prod` | `compatible` | Production โ€” explicit major version control |
+| `fsi` | `exact` | Regulatory โ€” version-controlled for auditability |
+| `sovereign` | `exact` | Maximum control โ€” exact versions for reproducibility |
+
+---
+
+## 5. Deprecation Lifecycle
+
+### 5.1 The Default Deprecation Policy
+
+Deprecation lifecycle is governed by **default DCM system policies** โ€” not hard-coded values. These defaults can be overridden using the standard policy priority mechanism. Higher-priority organizational policies can shorten, extend, or lock any of these values.
+
+```yaml
+# Default deprecation lifecycle policies (platform domain โ€” overridable)
+deprecation_lifecycle_policies:
+
+  REG-DP-001:
+    name: "Default deprecation notification period"
+    value: P30D           # 30 days notice before deprecation status applied
+    override: allow       # organizations may change this
+
+  REG-DP-002:
+    name: "Default sunset period by tier"
+    values:
+      tier_1: P12M        # 12 months for Core registry types
+      tier_2: P6M         # 6 months for Verified Community types
+      tier_3: organization_governed
+    override: allow
+    profile_locks:
+      fsi: immutable      # FSI profile locks sunset periods
+      sovereign: immutable
+
+  REG-DP-003:
+    name: "Default migration window after retirement"
+    value: P90D           # 90 days after retirement โ€” realizations enter DEPRECATED_RUNTIME
+    override: allow
+
+  REG-DP-004:
+    name: "Migration target declaration"
+    requirement: required_in_deprecation_notice
+    # Deprecation notice must declare: successor type or explicit migration guidance
+    override: allow
+
+  REG-DP-005:
+    name: "Behavior on retirement โ€” new requests"
+    value: reject         # retired types reject new requests (not warn โ€” reject)
+    override: not_permitted   # this is structural โ€” cannot be changed
+
+  REG-DP-006:
+    name: "Behavior on retirement โ€” existing realizations"
+    value: deprecated_runtime_state
+    # Existing realizations enter DEPRECATED_RUNTIME state:
+    # - Eligible for: modify, decommission, drift detection
+    # - Not eligible for: rehydration using deprecated type
+    # - Not automatically destroyed
+    override: allow
+
+  REG-DP-007:
+    name: "Emergency deprecation migration window"
+    value: P30D           # minimum 30 days even for security emergency
+    override: not_permitted   # floor cannot be removed
+```
+
+### 5.2 Deprecation Lifecycle Flow
+
+```
+Resource Type in active status
+  โ”‚
+  โ–ผ  Deprecation proposal (PR + 30 day review)
+Status: deprecated
+  โ”‚  Notification dispatched to:
+  โ”‚  - All registered providers implementing this type
+  โ”‚  - All organizations with active realizations
+  โ”‚  - All webhook registrations subscribed to registry events
+  โ”‚
+  โ–ผ  Sunset period (P12M Tier 1 / P6M Tier 2 โ€” per REG-DP-002)
+  โ”‚  During sunset:
+  โ”‚  - New requests: succeed with deprecation warning
+  โ”‚  - Existing realizations: unaffected
+  โ”‚  - Drift detection: continues
+  โ”‚  - Provider implementations: remain valid
+  โ”‚
+  โ–ผ  Retirement (status: retired)
+  โ”‚  Existing realizations โ†’ DEPRECATED_RUNTIME state
+  โ”‚  New requests โ†’ rejected (REG-DP-005)
+  โ”‚
+  โ–ผ  Migration window (P90D โ€” per REG-DP-003)
+  โ”‚  Organizations migrate realizations to successor type
+  โ”‚  DEPRECATED_RUNTIME entities can be decommissioned or migrated
+  โ”‚
+  โ–ผ  Post-migration window
+     DEPRECATED_RUNTIME entities remain operational but unsupported
+     Drift detection: continues but remediation is manual
+```
+
+### 5.3 Overriding Deprecation Defaults
+
+Organizations use standard policy priority to customize deprecation behavior:
+
+```yaml
+# Organizational policy: extend Tier 2 sunset to 12 months
+policy:
+  domain: platform
+  priority: 600.0.0
+  type: gatekeeper
+  rule: >
+    If registry.deprecation.tier == tier_2
+    THEN override: sunset_period = P12M
+    basis: "Our tooling requires longer migration windows"
+```
+
+```yaml
+# FSI profile lock: sunset periods immutable
+policy:
+  domain: system
+  priority: 900.0.0
+  immutable_ceiling: absolute
+  rule: >
+    If active_profile IN [fsi, sovereign]
+    THEN lock: REG-DP-002 as immutable
+    rationale: "Regulatory change control requirements"
+```
+
+---
+
+## 6. Provider Selection Tie-Breaking
+
+When the placement engine has multiple viable provider candidates that satisfy all constraints equally, the following hierarchy resolves the tie deterministically:
+
+### 6.1 Tie-Breaking Hierarchy
+
+```
+Priority  Factor                    Condition
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€    โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+1         Policy preference         A Transformation policy injected a
+                                    preference_score or preferred_provider_uuid
+
+2         Provider priority         Providers declare a numeric priority
+                                    Higher value = preferred (default: 50)
+
+3         Tenant affinity           Tenant's Policy Group declares preferred
+                                    providers for specific resource types
+
+4         Cost analysis             Cost Analysis component has current data
+                                    AND cost is determinable for candidates
+                                    Prefer lower total cost (CapEx + OpEx)
+                                    SKIP if cost data absent or incomparable
+
+5         Least loaded              Current capacity utilization from reserve_query
+                                    If utilization differs > 10%: prefer less loaded
+                                    SKIP if utilization data unavailable
+
+6         Consistent hash           SHA-256(request_uuid + resource_type + sorted_candidate_uuids)
+                                    Deterministic โ€” same request always resolves
+                                    to same provider in a stable cluster
+                                    Never round-robin
+```
+
+### 6.2 Cost Analysis Integration
+
+Cost analysis ranks above operational load because cost is a business decision. When cost data is available and comparable:
+
+- **CapEx consideration:** provider infrastructure cost allocation per resource type
+- **OpEx consideration:** operational overhead, licensing, support costs per resource unit
+- **Comparability requirement:** cost must be expressed in the same currency and time period; if not comparable (different currencies, missing data), skip to step 5
+
+Cost data is sourced from the **Cost Analysis** control plane component. If Cost Analysis is not deployed or does not have current data for the candidate providers, this step is skipped without blocking placement.
+
+```yaml
+# Cost analysis in placement loop
+placement_cost_evaluation:
+  enabled: true                    # false if Cost Analysis unavailable
+  data_freshness_max: PT1H         # reject cost data older than 1 hour
+  comparison_threshold: 0.05       # 5% cost difference to trigger preference
+  # If candidates are within 5% cost: skip cost as a tiebreaker
+  cost_components:
+    - capex_allocation_per_unit
+    - opex_per_unit_per_hour
+    - licensing_per_unit
+```
+
+### 6.3 Provider Priority Declaration
+
+```yaml
+provider_registration:
+  provider_priority: 100   # default: 50; higher = preferred when equal
+  cost_metadata:
+    capex_allocation_per_unit: 12.50    # USD per VM-month
+    opex_per_unit_per_hour: 0.08       # USD per VM-hour
+    currency: USD
+    last_updated: 
+```
+
+---
+
+## 7. The Registry Provider
+
+### 7.1 Concept
+
+The Registry Provider is a specialized sub-type of Information Provider โ€” the mechanism through which a DCM deployment accesses its authoritative Resource Type Registry. Every DCM deployment has exactly one active Registry Provider.
+
+### 7.2 Registration
+
+```yaml
+registry_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/registry/org-primary"
+    version: "1.0.0"
+    status: active
+
+  name: "Organization Primary Registry"
+  provider_type: registry              # sub-type of information_provider
+
+  # Registry source
+  registry_url: https://registry.corp.example.com
+  tier_1_source: https://registry.dcm-project.github.io   # upstream pull
+  tier_2_sources:
+    - https://registry.dcm-project.github.io
+    - https://registry.partner-org.example.com            # verified partner
+
+  # Sync configuration
+  sync:
+    schedule: "0 2 * * *"            # nightly pull from upstream
+    on_sync_failure: 
+    cache_ttl: P7D                   # use cached if upstream unavailable
+
+  # Air-gapped / sovereign configuration
+  offline_mode: false                # true: no external connectivity
+  signed_bundle_import: false        # true: updates via signed bundles only
+  bundle_signing_key_ref:
+    credential_provider_uuid: 
+    secret_path: "dcm/registry/bundle-verification-key"
+
+  # Sovereignty filtering
+  sovereignty_filter:
+    enabled: true
+    permitted_jurisdictions: [eu-west, eu-central]
+    # Only activate resource types flagged as compatible with these jurisdictions
+
+  # Vendor approval list
+  vendor_allowlist:
+    enabled: false                   # true in prod/fsi/sovereign
+    permitted_vendors: [dcm-project, vmware, redhat, hashicorp]
+    # Resource types from non-listed vendors are not activated
+```
+
+### 7.3 Signed Bundle Model (Air-Gapped Updates)
+
+```
+Online workstation (with registry access)
+  โ”‚
+  Pull registry delta since last sync
+  Sign with organization private key (via Credential Provider)
+  Package: registry-update-YYYY-MM-DD.bundle
+  โ”‚
+  Transfer via approved secure channel
+  โ”‚
+Air-gapped DCM deployment
+  โ”‚
+  Verify signature against organization public key
+  Import bundle โ†’ update local registry
+  Emit: registry.sync_completed audit event
+```
+
+### 7.4 Registry Provider Policies
+
+The Registry Provider is fully policy-governed โ€” policies act on registry operations at every stage:
+
+**Sovereignty enforcement:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_sync
+  rule: "If resource_type.jurisdiction_compatibility NOT CONTAINS tenant.sovereignty_zone THEN reject_activation"
+```
+
+**Vendor allowlist:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_activation
+  rule: "If resource_type.publisher NOT IN approved_vendor_list THEN gatekeep: require_manual_approval"
+```
+
+**Bundle verification:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_bundle_import
+  rule: "If bundle.signature_valid == false THEN reject: unsigned bundles not permitted"
+```
+
+**Version pinning in production:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_sync
+  rule: "If active_profile == prod AND resource_type.version_delta.type == major THEN gatekeep: major version upgrades require manual approval"
+```
+
+**Audit all syncs:**
+```yaml
+policy:
+  type: transformation
+  target: registry_sync
+  rule: "Always inject: sync_audit.required = true, sync_audit.reviewer = platform_admin"
+```
+
+### 7.5 Profile-Appropriate Registry Policy Groups
+
+DCM ships built-in registry policy groups โ€” one per profile, activated automatically:
+
+| Group Handle | Profile | Key Behaviors |
+|-------------|---------|--------------|
+| `system/group/registry-minimal` | minimal | Advisory only; pull everything; no restrictions; warn on unverified sources |
+| `system/group/registry-dev` | dev | Warn on unverified sources; pull Tier 1+2; no vendor restrictions |
+| `system/group/registry-standard` | standard | Block unverified sources; Tier 1+2 only; sovereignty filter enabled |
+| `system/group/registry-prod` | prod | Strict version pinning; approved vendor list; audit all syncs; major version manual approval |
+| `system/group/registry-fsi` | fsi | Exact version pinning; approved vendor list; immutable sunset periods; all syncs audited with dual approval |
+| `system/group/registry-sovereign` | sovereign | Signed bundles only; offline registry; no external connectivity; bundle verification required |
+
+Organizations can replace or extend these groups using standard Policy Group composition.
+
+---
+
+## 8. DCM System Policies
+
+| Policy | Rule |
+|--------|------|
+| `REG-001` | Resource Type proposals follow a PR-based GitOps workflow with automated validation gates (schema, FQN conflict, dependency resolution, breaking change detection) that must all pass before review begins. |
+| `REG-002` | All registry changes require a minimum review period by change type and a mandatory shadow validation period in `proposed` status before promotion to `active`. |
+| `REG-003` | Deprecation lifecycle is governed by default policies REG-DP-001 through REG-DP-007. These defaults are overridable via standard policy priority except where locked by active Profile. |
+| `REG-004` | Version constraints in requests are strictly enforced. DCM never automatically upgrades across major versions regardless of version_policy. Version resolution policy is profile-governed. |
+| `REG-005` | When multiple providers satisfy all placement criteria equally, the tie-breaking hierarchy applies: policy preference โ†’ provider priority โ†’ tenant affinity โ†’ cost analysis (if available) โ†’ least loaded โ†’ consistent hash on request_uuid. |
+| `REG-006` | The registry uses a federated model. Air-gapped and sovereign deployments use offline registries populated via signed bundles verified against the organization's public key. |
+| `REG-007` | The Registry Provider is policy-governed. Profile-appropriate registry policy groups are activated by default. Organizations may extend or replace these groups using standard Policy Group composition. |
+| `REG-DP-001` | Default deprecation notification period: P30D before deprecation status is applied. Overridable. |
+| `REG-DP-002` | Default sunset period: Tier 1 = P12M, Tier 2 = P6M. Overridable; locked as immutable in fsi and sovereign profiles. |
+| `REG-DP-003` | Default migration window after retirement: P90D. Overridable. |
+| `REG-DP-004` | Deprecation notices must declare a successor type or explicit migration guidance. Overridable. |
+| `REG-DP-005` | Retired resource types reject new requests. Not overridable โ€” structural. |
+| `REG-DP-006` | Existing realizations of retired types enter DEPRECATED_RUNTIME state โ€” eligible for modify and decommission, not rehydration. Overridable. |
+| `REG-DP-007` | Emergency deprecation minimum migration window: P30D. Not overridable โ€” floor cannot be removed. |
+
+---
+
+## 9. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should there be a certified registry tier between Tier 2 and DCM Core for formally audited types? | Ecosystem | โœ… Resolved โ€” no fourth tier; certification metadata within existing tier structure serves same purpose (REG-008) |
+| 2 | Should organizations be able to publish their Tier 3 types to the Verified Community registry? | Community | โœ… Resolved โ€” Tier 3 to Tier 2 promotion via PR pathway with additional requirements: production deployment + OSS license + named maintainer + migration path (REG-009) |
+| 3 | How does the Registry Provider handle a scenario where the upstream DCM Project Registry is permanently unavailable? | Resilience | โœ… Resolved โ€” Organization Registry mirror is self-sufficient; upstream loss is governance decision not operational crisis; three long-term options (REG-010) |
+| 4 | Should cost metadata on provider registrations be sourced from the Cost Analysis component or declared statically? | Architecture | โœ… Resolved โ€” static or Cost Analysis sourcing; hybrid with Cost Analysis preferred; placement engine uses freshest available (REG-011) |
+
+---
+
+## 10. Related Concepts
+
+- **Resource Type Hierarchy** (doc 05) โ€” the structure of Resource Type Specifications
+- **Policy Organization** (doc 14) โ€” Policy Groups governing registry behavior
+- **Deployment and Redundancy** (doc 17) โ€” registry sync and offline operation
+- **Auth Providers** (doc 19) โ€” authentication for registry access
+- **Universal Audit Model** (doc 16) โ€” all registry operations produce audit records
+
+
+## 11. Registry Governance Gap Resolutions
+
+### 11.1 No Fourth Registry Tier โ€” Certification Metadata Instead (Q1)
+
+A formal fourth registry tier is not introduced. Resource Type Specifications in any tier may carry certification metadata from recognized certifying bodies. Certification provides equivalent assurance to a separate tier without the governance complexity.
+
+```yaml
+resource_type_spec:
+  registry_tier: 2
+  tier_certifications:
+    - certifying_body: "OpenStack Foundation"
+      certification: "OpenStack Powered"
+      certified_versions: [">=2023.1"]
+      certificate_ref: 
+```
+
+Users seeking "formally audited types" filter on certification metadata โ€” same result as a separate tier, without the structural fragmentation.
+
+### 11.2 Tier 3 to Tier 2 Promotion Pathway (Q2)
+
+Organizations may promote Tier 3 Resource Type Specifications to Tier 2 (Verified Community) via the standard PR-based promotion pathway with additional requirements.
+
+```yaml
+tier_3_to_tier_2_promotion:
+  requirements:
+    - at_least_one_production_deployment: true
+    - documented_use_case: true
+    - open_source_license_compatible: true   # DCM is Apache 2.0
+    - named_community_maintainer: true
+    - test_suite_included: true
+    - migration_path_from_tier3: documented  # handles name changes, schema diffs
+  review_period: 14 days
+  existing_tier3_users_notified: true        # current users notified of promotion
+```
+
+The promotion pathway gives organizations a route from internal tooling to community contribution without requiring a ground-up rewrite. The migration path documentation ensures existing Tier 3 deployments can upgrade smoothly.
+
+### 11.3 Upstream Registry Permanently Unavailable (Q3)
+
+The Organization Registry mirror operates independently from the upstream DCM Project Registry. Permanent upstream loss is a governance decision, not an operational crisis.
+
+**Short-term:** Organization Registry mirror is self-sufficient for all operations. Existing types continue working normally.
+
+**Medium-term:** Registry Provider enters "independent operation" mode โ€” new Tier 1/2 types cannot be added (no upstream to sync from); existing types continue operating; Tier 3 unaffected (organization-governed).
+
+**Long-term governance options:**
+- **Option A:** Designate a community mirror as the new upstream (community self-governance)
+- **Option B:** Fork the registry โ€” organization takes ownership of their copy
+- **Option C:** Continue as independent installation (no new community types)
+
+The mirror's self-sufficiency means existing deployments never experience an operational outage due to upstream unavailability.
+
+### 11.4 Provider Cost Metadata Source (Q4)
+
+Provider cost metadata may be declared statically or sourced dynamically from the Cost Analysis component.
+
+```yaml
+provider_cost_metadata:
+  source: 
+
+  static:
+    capex_allocation_per_unit: 12.50   # USD per VM-month
+    opex_per_unit_per_hour: 0.08
+    currency: USD
+    last_updated: 
+
+  cost_analysis:
+    query_interval: PT1H               # refresh cost data hourly
+    fallback: static
+    fallback_max_age: PT24H            # use static if Cost Analysis data older than 24h
+
+  hybrid:
+    prefer: cost_analysis
+    static_for_unavailable: true
+```
+
+The placement engine's cost analysis step (tie-breaking step 4) uses whichever source is freshest and available โ€” Cost Analysis preferred, static as fallback. No changes required to the placement tie-breaking model.
+
+### 11.5 System Policies โ€” Registry Governance Gaps
+
+| Policy | Rule |
+|--------|------|
+| `REG-008` | A formal fourth registry tier is not introduced. Resource Type Specifications in any tier may carry certification metadata from recognized certifying bodies. Certification metadata is a filter criterion โ€” not a structural tier boundary. |
+| `REG-009` | Organizations may promote Tier 3 Resource Type Specifications to Tier 2 via the standard PR-based promotion pathway with additional requirements: at least one production deployment, OSS-compatible license, named community maintainer, and documented migration path from the Tier 3 handle. |
+| `REG-010` | The Organization Registry mirror operates independently from the upstream DCM Project Registry. Permanent upstream unavailability does not affect existing operations. New community type adoption requires a designated community mirror, organizational fork, or independent operation decision. |
+| `REG-011` | Provider cost metadata may be declared statically or sourced dynamically from the Cost Analysis component. Hybrid mode uses Cost Analysis when available and falls back to static. The placement engine uses whichever source is freshest and available. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/universal-audit.md b/content/docs/architecture/data-model/universal-audit.md
new file mode 100644
index 0000000..97da6b7
--- /dev/null
+++ b/content/docs/architecture/data-model/universal-audit.md
@@ -0,0 +1,608 @@
+---
+title: "Universal Audit Model"
+type: docs
+weight: 15
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Storage Providers](11-storage-providers.md) | [Universal Groups](15-universal-groups.md)
+
+---
+
+## 1. Purpose
+
+The Universal Audit Model defines the **unconditional obligation** for every DCM component to record every change to every artifact in a uniform, tamper-evident, retention-governed audit trail. No change is silent. No change is exempt.
+
+**The four required fields for every audit record:**
+- **Date and time** โ€” when the change occurred (ISO 8601 with milliseconds)
+- **Who** โ€” the complete actor chain (immediate actor + human authorization chain)
+- **What** โ€” the subject of the change (entity UUID, type, handle)
+- **Action** โ€” what happened (closed vocabulary โ€” not free text)
+
+**The retention requirement:** Audit records must survive at least as long as any referenced resource or group is live. Policy governs what happens after all affected parties reach terminal state.
+
+---
+
+## 2. Design Principles
+
+**Universal โ€” no exceptions.** Every mutation to every DCM artifact produces an audit record. Resources, policies, layers, groups, relationships, providers, configurations, authorizations, mode 4 queries, ingestion events, rehydration events, drift events, login events โ€” all covered.
+
+**Append-only โ€” tamper-evident.** Audit records are never modified or deleted while retention obligations apply. Each record carries a hash of its own content and a reference to the previous record's hash โ€” forming a tamper-evident chain per entity.
+
+**Guaranteed delivery โ€” not guaranteed synchrony.** Audit writes use a write-ahead log (WAL) pattern โ€” the change and its audit record are written to a local WAL first, then delivered to the Audit Store asynchronously with retry. A change is never silent โ€” it may be briefly buffered, but delivery is guaranteed before the WAL is cleared.
+
+**Reference-based retention โ€” not time-based.** Audit records are retained while any referenced entity is live. Fixed time schedules (7 years, 10 years) are applied only after all referenced entities reach terminal state. A record created 20 years ago is retained unconditionally if any entity it references is still operational.
+
+**Policy-governed post-lifecycle retention.** After all referenced entities reach terminal state, policy determines how long to keep the audit record. Default is 7 years post-retirement (FSI-aligned). Organizations configure per Profile.
+
+---
+
+## 3. The Universal Audit Record
+
+```yaml
+audit_record:
+  # IDENTITY โ€” immutable once written
+  record_uuid: 
+  record_timestamp: 
+  dcm_version: 
+
+  # WHO โ€” composite actor chain
+  actor:
+    # The immediate actor that performed the change
+    immediate:
+      type: 
+      uuid: 
+      display_name: 
+      session_uuid: 
+
+    # The human who ultimately authorized this action (traceable chain)
+    authorized_by:
+      uuid: 
+      display_name: 
+      authorization_method: 
+      # direct_action:      human directly performed this
+      # request_submission: human submitted the request that triggered this
+      # policy_activation:  human activated the policy that triggered this
+      # system_policy:      DCM System Policy โ€” no individual human
+      # scheduled:          scheduled job โ€” authorized by job owner
+
+    # Links to originating context
+    request_uuid: 
+    policy_uuid: 
+    policy_version: 
+    correlation_id: 
+
+  # WHAT โ€” the subject of the change
+  subject:
+    entity_uuid: 
+    entity_type: 
+    entity_handle: 
+    entity_version_before: 
+    entity_version_after: 
+
+  # ACTION โ€” closed vocabulary
+  action: 
+
+  # ACTION DETAIL โ€” structured per action type
+  action_detail:
+    # For MODIFY, ENRICH, LOCK
+    field_changes:
+      - field: 
+        previous_value: 
+        new_value: 
+        change_reason: 
+        locked_after: 
+
+    # For STATE_TRANSITION
+    state_transition:
+      from_state: 
+      to_state: 
+      transition_reason: 
+      triggered_by: 
+
+    # For RELATIONSHIP_CREATE, RELATIONSHIP_RELEASE
+    relationship_detail:
+      related_entity_uuid: 
+      related_entity_type: 
+      relationship_type: 
+      relationship_nature: 
+      cross_tenant: 
+
+    # For MEMBER_ADD, MEMBER_REMOVE
+    membership_detail:
+      group_uuid: 
+      group_class: 
+      member_role: 
+      time_bounded: 
+      valid_until: 
+
+    # For EVALUATE (policy evaluation)
+    evaluation_detail:
+      policy_uuid: 
+      policy_version: 
+      outcome: 
+      placement_phase: 
+      missing_fields: []
+
+    # For QUERY (Mode 4 black box)
+    query_detail:
+      provider_uuid: 
+      fields_queried: []
+      result_type: 
+      outcome: 
+      audit_token: 
+      cached_result: 
+
+    # For DRIFT_DETECT
+    drift_detail:
+      drifted_fields:
+        - field: 
+          realized_value: 
+          discovered_value: 
+      drift_severity: 
+      policy_response: 
+
+  # CONTEXT
+  context:
+    tenant_uuid: 
+    request_uuid: 
+    session_uuid: 
+    profile_active: 
+    tags: [...]   # arbitrary searchable tags
+
+  # RETENTION
+  retention:
+    referenced_entities:
+      - entity_uuid: 
+        entity_type: 
+        last_known_state: 
+    retention_status: 
+    # live:           at least one referenced entity is non-retired โ€” retain unconditionally
+    # all_retired:    all referenced entities have reached terminal state
+    # policy_governed: apply governing_policy after all_retired
+    governing_policy_uuid: 
+    retain_until: 
+
+  # INTEGRITY โ€” tamper-evident hash chain
+  integrity:
+    record_hash: 
+    previous_record_hash: 
+    # Forms a per-entity hash chain โ€” inserting, modifying, or deleting a
+    # historical record breaks the chain, detectable by verification
+    chain_sequence: 
+    signed_by: 
+    signature: 
+```
+
+---
+
+## 4. Action Vocabulary
+
+The `action` field uses a closed vocabulary. Free-text actions are invalid and rejected at write time (AUD-007).
+
+| Action | Applies To | Description |
+|--------|-----------|-------------|
+| `CREATE` | All | New artifact created |
+| `MODIFY` | All | Artifact field changed |
+| `STATE_TRANSITION` | Entities, groups | Lifecycle state changed |
+| `DELETE` | All | Artifact destroyed / decommissioned |
+| `ACTIVATE` | Policies, profiles, groups | Artifact made active |
+| `DEACTIVATE` | Policies, profiles, groups | Artifact made inactive |
+| `DEPRECATE` | All | Artifact deprecated |
+| `RETIRE` | All | Artifact retired |
+| `MEMBER_ADD` | Groups | Member added to group |
+| `MEMBER_REMOVE` | Groups | Member removed from group |
+| `RELATIONSHIP_CREATE` | Entities | Relationship established |
+| `RELATIONSHIP_RELEASE` | Entities | Relationship released |
+| `AUTHORIZE` | Cross-tenant, actors | Authorization granted |
+| `REVOKE` | Cross-tenant, actors | Authorization revoked |
+| `EVALUATE` | Policies | Policy evaluated (with outcome) |
+| `ENRICH` | Fields | Field enriched by policy, layer, or Mode 4 provider |
+| `LOCK` | Fields | Field locked (override: immutable set) |
+| `HOLD_PLACE` | Resources | Resource hold placed with provider |
+| `HOLD_CONFIRM` | Resources | Resource hold confirmed |
+| `HOLD_RELEASE` | Resources | Resource hold released |
+| `DRIFT_DETECT` | Entities | Drift detected between Realized and Discovered |
+| `DRIFT_RESOLVE` | Entities | Drift resolved |
+| `INGEST` | Entities | Entity ingested (brownfield or V1 migration) |
+| `PROMOTE` | Entities | Ingested entity promoted to full lifecycle |
+| `EXPIRE` | Entities | Lifecycle time constraint expiry action fired |
+| `REHYDRATE` | Entities | Rehydration requested |
+| `QUERY` | Mode 4 | Black box query sent and result received |
+| `DISCOVER` | Entities | Discovery cycle completed |
+| `LOGIN` | Actors | Actor authentication event |
+| `LOGOUT` | Actors | Actor session ended |
+| `CONFIG_CHANGE` | Platform | DCM configuration changed (profile activated, etc.) |
+
+---
+
+## 5. The "Who" โ€” Composite Actor Record
+
+The `who` in an audit record is not a single identity โ€” it is a **composite actor chain** tracing from the immediate action back to the human who ultimately authorized it.
+
+### 5.1 Actor Types
+
+| Type | Example | authorized_by |
+|------|---------|--------------|
+| `human` | Platform admin changes a policy | Self |
+| `system_component` | Lifecycle Constraint Enforcer fires expiry | Policy that set the constraint โ†’ human who activated policy |
+| `policy` | Transformation Policy enriches a field | Human who activated the policy |
+| `provider` | Service Provider updates Realized State | Dispatch that triggered it โ†’ human who submitted request |
+| `scheduled_job` | Discovery cycle runs | Owner of the scheduled job |
+| `mode4_provider` | Black box enriches a field | Policy that triggered the query โ†’ human who activated policy |
+
+### 5.2 System-Initiated Actions
+
+For system-initiated changes where there is no immediate human actor, the authorization chain traces back as far as possible:
+
+```yaml
+actor:
+  immediate:
+    type: system_component
+    uuid: 
+    display_name: "Lifecycle Constraint Enforcer"
+  authorized_by:
+    uuid: null   # no specific human โ€” system policy
+    display_name: "DCM System Policy LTC-003"
+    authorization_method: system_policy
+  policy_uuid: 
+  policy_version: "1.0.0"
+```
+
+---
+
+## 6. Retention Model
+
+### 6.1 Reference-Based Retention
+
+Audit records are retained based on the lifecycle state of all referenced entities โ€” not on a fixed time schedule.
+
+```
+Audit record created
+  โ”‚  retention_status: live (all referenced entities tracked)
+  โ”‚
+  โ–ผ  [continuous monitoring]
+  โ”‚
+  As referenced entities change state:
+  โ”‚  DCM updates last_known_state on each referenced_entity
+  โ”‚  When all reach terminal state โ†’ retention_status: all_retired
+  โ”‚
+  โ–ผ  retention_status: all_retired
+  โ”‚  Governing policy determines retain_until date
+  โ”‚  retention_status: policy_governed
+  โ”‚
+  โ–ผ  retain_until reached
+     Audit record eligible for destruction / archival
+```
+
+### 6.2 Post-Lifecycle Retention Options
+
+| Policy Setting | Meaning | Default Profile |
+|---------------|---------|----------------|
+| `destroy_immediately` | Destroy when last entity retires | (not available in standard+) |
+| `retain_for: P90D` | 90 days post-retirement | dev profile |
+| `retain_for: P3Y` | 3 years post-retirement | standard profile |
+| `retain_for: P7Y` | 7 years post-retirement | prod, fsi profiles (DEFAULT) |
+| `retain_for: P10Y` | 10 years post-retirement | sovereign profile |
+| `retain_indefinitely` | Never destroy | optional โ€” maximum compliance |
+| `archive_after: P1Y` | Move to cold storage 1 year post-retirement | configurable |
+
+### 6.3 Retention Shorter Than Referenced Entity Lifetime
+
+This cannot happen. While any referenced entity is live, `retention_status: live` and the record is retained unconditionally. The retention policy only applies **after** all referenced entities reach terminal state. A 90-day retention policy means "90 days after the last referenced entity is retired" โ€” not "90 days after creation."
+
+---
+
+## 7. Two-Stage Audit โ€” Synchronous Commit + Async Enrichment
+
+### 7.1 The Design
+
+DCM uses a **two-stage audit model** that provides synchronous durability guarantees without impacting request processing performance.
+
+```
+Stage 1 โ€” Commit Log (synchronous, in critical path, < 1ms)
+Stage 2 โ€” Audit Store (asynchronous, out of critical path, full record)
+```
+
+**Stage 1** writes a minimal Commit Log entry synchronously using consensus protocol (Raft). The write is confirmed when a quorum of Commit Log replicas acknowledges it. The operation returns success after Stage 1 confirms โ€” not after the Audit Store write.
+
+**Stage 2** runs asynchronously via the Audit Forward Service: enriches the minimal Commit Log entry into a full audit_record, computes the hash chain, and writes to the Audit Store with retry.
+
+### 7.2 Stage 1 โ€” Commit Log Entry (minimal, ultra-fast)
+
+```yaml
+commit_log_entry:
+  entry_uuid:             # links to full audit_record in Stage 2
+  sequence:            # monotonically increasing โ€” global ordering
+  timestamp:    # authoritative audit timestamp
+  entity_uuid: 
+  entity_type: 
+  action: 
+  actor_uuid:             # immediate actor only
+  request_uuid:           # if applicable
+  tenant_uuid: 
+  change_fingerprint: 
+  # change_fingerprint enables Stage 2 to verify full record matches Stage 1
+
+  status: 
+  forwarded_at:       # populated by Audit Forward Service
+  audit_record_uuid:      # UUID of full audit_record in Audit Store
+```
+
+**Stage 1 guarantees:** the change happened, at this exact time, this actor performed it, this entity was affected, this action was taken. Full detail follows in Stage 2.
+
+**Commit Log quorum write** (distributed deployment):
+```
+Write confirmed when quorum acknowledges:
+  โ”œโ”€โ”€ Replica 1 (local node)     โ†’ ACK โ”€โ”
+  โ”œโ”€โ”€ Replica 2 (different node) โ†’ ACK โ”€โ”ค quorum (2/3) โ€” write confirmed
+  โ””โ”€โ”€ Replica 3 (different zone) โ†’ (async best-effort)
+```
+
+### 7.3 Stage 2 โ€” Audit Forward Service
+
+```
+Audit Forward Service reads pending_forward Commit Log entries
+  โ”‚
+  โ”œโ”€โ”€ Retrieve full change context from DCM internal state
+  โ”‚   (field values before/after, complete actor chain, relationship detail)
+  โ”‚
+  โ”œโ”€โ”€ Construct complete audit_record (full structure per Section 3)
+  โ”‚   - Compute record_hash + previous_record_hash (hash chain)
+  โ”‚   - Set retention.referenced_entities
+  โ”‚
+  โ”œโ”€โ”€ Write to Audit Store
+  โ”‚   โ†’ Success: mark commit_log_entry status: forwarded
+  โ”‚   โ†’ Failure: retry with exponential backoff
+  โ”‚              N retries exhausted โ†’ status: forward_failed, alert admin
+  โ”‚
+  โ””โ”€โ”€ Commit Log entry eligible for cleanup after:
+      status: forwarded AND entry age > Commit Log retention window
+```
+
+### 7.4 Recoverability
+
+| Failure Scenario | Recovery |
+|-----------------|---------|
+| DCM crashes after Stage 1, before Stage 2 | On restart, Audit Forward Service replays all `pending_forward` entries |
+| Audit Store unavailable | Commit Log accumulates; Audit Forward Service retries when Audit Store recovers |
+| Stage 2 fails mid-enrichment | Commit Log entry remains `pending_forward`; retried from committed Stage 1 data |
+| Commit Log quorum unavailable | Stage 1 fails โ†’ operation aborted โ†’ no silent change |
+| All Commit Log replicas lost | Recovery from replica backup; forward_failed entries investigated |
+
+### 7.5 Performance Characteristics
+
+| Component | Latency | In Critical Path? |
+|-----------|---------|-----------------|
+| Stage 1 โ€” Commit Log quorum write | < 1ms (local NVMe + Raft) | Yes |
+| Stage 2 โ€” Audit Store write | 5โ€“50ms (network + indexing) | No |
+| Full audit record visible | Seconds to minutes after Stage 1 | No |
+
+**The Stage 1 timestamp is the authoritative audit timestamp.** Stage 2 write time is when the full record became queryable โ€” not when the change occurred.
+
+---
+
+---
+
+## 8. Tamper-Evidence โ€” Hash Chain
+
+Each audit record carries:
+- `record_hash` โ€” SHA-256 of the record's content
+- `previous_record_hash` โ€” hash of the immediately preceding audit record for this entity
+- `chain_sequence` โ€” monotonically increasing integer per entity
+
+Together these form a **per-entity hash chain**. To verify integrity:
+
+```
+For each entity:
+  Load all audit records ordered by chain_sequence
+  For each record:
+    Verify record_hash == SHA-256(record content)
+    Verify previous_record_hash == record_hash of sequence N-1
+  If any verification fails:
+    โ†’ Chain broken โ€” tampering detected
+    โ†’ Alert dispatched to security and platform admin
+    โ†’ Affected records flagged in audit dashboard
+```
+
+Inserting, modifying, or deleting any historical record breaks the chain at that point and all subsequent records for that entity. The breach is detectable at the next verification run.
+
+---
+
+## 9. DCM System Policies
+
+| Policy | Rule |
+|--------|------|
+| `AUD-001` | Every modification to any DCM artifact must produce a Commit Log entry synchronously before the operation returns success. Commit Log write failure aborts the operation โ€” no silent unaudited changes. |
+| `AUD-002` | Audit records are append-only and immutable. No audit record may be modified or deleted while retention_status is `live` or `policy_governed`. |
+| `AUD-003` | Audit records must survive at least as long as any referenced entity is in a non-retired/non-decommissioned state (retention_status: live). |
+| `AUD-004` | Post-lifecycle retention is governed by policy. Default is `retain_for: P7Y` after all referenced entities reach terminal state. |
+| `AUD-005` | The actor field must identify both the immediate actor and the authorized_by human actor chain to the extent traceable. |
+| `AUD-006` | Audit records must carry a `record_hash` and `previous_record_hash` forming a tamper-evident hash chain per entity. |
+| `AUD-007` | The action field must use the closed vocabulary โ€” free-text action fields are invalid and must be rejected at write time. |
+| `AUD-008` | Audit Store implementations must support queries by: entity_uuid, actor_uuid, action, timestamp range, tenant_uuid, request_uuid, and retention_status. |
+| `AUD-009` | The Audit Forward Service must deliver all Commit Log entries to the Audit Store with exponential backoff retry. Commit Log entries may only be cleared after both: (a) Audit Store confirms receipt AND (b) entry has aged beyond the Commit Log retention window. |
+| `AUD-010` | Hash chain verification must be available as a first-class DCM operation. Chain breaks must trigger immediate security alerts. |
+| `AUD-011` | On DCM restart, the Audit Forward Service must replay all `status: pending_forward` Commit Log entries before accepting new operations. |
+| `AUD-012` | The Commit Log must use consensus protocol (Raft or equivalent) with quorum writes. A write is confirmed durable only when a quorum of replicas acknowledges it. |
+| `AUD-013` | The Stage 1 timestamp in the Commit Log is the authoritative audit timestamp. Stage 2 enrichment timestamps record when the full audit record became queryable โ€” not when the change occurred. |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should hash chain verification run continuously or on-demand? | Security | โœ… Resolved โ€” three levels: continuous write (chain construction), scheduled sweep (weekly to 6-hourly per profile), on-demand (operator-triggered); failure โ†’ security alert + integrity incident (AUD-014) |
+| 2 | Should the WAL have a configurable maximum capacity, and what happens when it is reached? | Availability | โœ… Resolved โ€” configurable max capacity; alert_and_continue (standard/prod); reject_new_ops (fsi/sovereign); backpressure at 75%/90%; P7D max age escalation (AUD-015) |
+| 3 | Should audit records for system-initiated changes (no human actor) be flagged differently in the dashboard? | Operational | โœ… Resolved โ€” actor.type: human/service_account/system; system_actor block with component/trigger/policy; full audit records; enables filtering in queries and dashboards (AUD-016) |
+| 4 | How does hash chain verification interact with distributed DCM deployments where audit records may be written to multiple regional stores? | Architecture | โœ… Resolved โ€” per-instance hash chains; daily Merkle root federation integrity proof at Hub DCM; cross-instance queries via parallel chains + correlation_id (AUD-017) |
+
+---
+
+## 11. Related Concepts
+
+- **Audit, Provenance, and Observability** (doc 12) โ€” three distinct concerns; this document covers the audit concern in full
+- **Field-Level Provenance** โ€” data lineage embedded in every payload; separate from audit records
+- **Storage Providers** (doc 11) โ€” Audit Store contract: append-only, WAL delivery, hash chain, retention tracking
+- **Universal Groups** (doc 15) โ€” all group changes produce audit records per this model
+- **Policy Organization** (doc 14) โ€” policy activation, shadow evaluation, and Mode 4 queries all produce audit records
+
+
+## 10. Universal Audit Gap Resolutions
+
+### 10.1 Hash Chain Verification Modes (Q1)
+
+Hash chain verification operates at three independent levels:
+
+```yaml
+hash_chain_verification:
+  continuous_write: true              # always โ€” hash computed on every write (chain construction)
+
+  scheduled_sweep:
+    enabled: true
+    schedule:
+      standard: "0 2 * * 0"          # weekly
+      prod: "0 2 * * *"              # daily
+      fsi: "0 */6 * * *"            # every 6 hours
+      sovereign: "0 */6 * * *"
+
+  on_demand:
+    enabled: true                     # always available to platform admin
+    max_range: P365D                  # maximum time range per verification run
+
+  on_verification_failure:
+    action: alert_security_team
+    halt_new_writes: false            # do not halt โ€” alert and investigate
+    # Halting writes is itself a security risk; alerting is the correct response
+    create_integrity_incident: true
+```
+
+Continuous verification is part of chain construction (not a separate process). Scheduled sweep catches tampering between writes. On-demand is available for incident investigation, compliance audit, and pre-report verification.
+
+### 10.2 Commit Log Maximum Capacity (Q2)
+
+The Commit Log has a configurable maximum capacity with a declared overflow policy โ€” different profiles have different trade-offs between availability and audit completeness.
+
+```yaml
+commit_log_capacity:
+  max_size: 10Gi                      # configurable; profile-governed
+  max_age: P7D                        # records older than 7d escalate regardless
+  on_capacity_exceeded:
+    profile_defaults:
+      minimal: alert_and_continue     # availability priority
+      dev: alert_and_continue
+      standard: alert_and_continue
+      prod: alert_and_continue
+      fsi: reject_new_ops             # audit completeness priority
+      sovereign: reject_new_ops
+  warn_at_percent: 75                 # alert at 75% capacity
+  urgent_at_percent: 90              # urgent alert at 90%
+```
+
+**`reject_new_ops` for fsi/sovereign:** Operating without a functional audit trail is a compliance violation in regulated environments. Stopping operations is preferable to operating unaudited โ€” same principle as Commit Log quorum unavailability โ†’ abort operation (STO-002).
+
+### 10.3 System-Initiated Audit Records (Q3)
+
+Audit records for system-initiated changes declare `actor.type: system` with a `system_actor` block identifying the DCM component, trigger, and authorizing policy.
+
+```yaml
+audit_record:
+  action: REHYDRATE
+  actor:
+    uuid: 
+    type: system                      # human | service_account | system
+    system_actor:
+      component: lifecycle_constraint_enforcer
+      trigger: entity_ttl_expired
+      entity_uuid: 
+      policy_uuid: 
+    authorization: implicit           # implicit = authorized by DCM architecture
+                                      # explicit = authorized by named policy
+```
+
+System actor records are full audit records โ€” they appear in all queries and compliance reports. `actor.type` enables filtering:
+- `filter: actor.type = human` โ†’ all human-initiated changes
+- `filter: actor.type = system` โ†’ all automated lifecycle operations
+- `filter: actor.type = service_account` โ†’ all API/programmatic changes
+
+### 10.4 Distributed Hash Chain Integrity (Q4)
+
+In distributed DCM deployments (Hub + Regional + Sovereign DCMs), each instance maintains its own independent hash chain. Federation-level integrity is provided by daily Merkle root proofs.
+
+```yaml
+distributed_hash_chain:
+  model: per_instance               # each DCM instance has its own chain
+  instance_chain:
+    chain_id:    # chain scoped to this instance
+
+  federation_integrity_proof:
+    enabled: true
+    schedule: "0 0 * * *"           # daily
+    mechanism: merkle_root
+    # Hub DCM collects chain tip hashes from all Regional DCMs
+    # Computes Merkle root โ†’ stores as federation_integrity_record
+    # Any chain break in any instance is detectable against this root
+    stored_at: hub_dcm_audit_store
+    signed_by: hub_dcm_service_account
+```
+
+**Cross-instance queries:** Records from different chains are presented as parallel chains with cross-references via `correlation_id`. Not merged into a single chain โ€” each instance's chain remains independently verifiable. Federation-level verification requires Hub DCM connectivity; per-instance verification is always available locally.
+
+### 10.5 System Policies โ€” Universal Audit Gaps
+
+| Policy | Rule |
+|--------|------|
+| `AUD-014` | Hash chain verification operates at three levels: continuous (hash computed on every write), scheduled sweep (weekly to every 6 hours per profile), and on-demand (operator-triggered for any time range). Verification failure triggers a security alert and integrity incident. New audit writes continue โ€” halting writes is itself a security risk. |
+| `AUD-015` | The Commit Log has configurable maximum capacity with a declared overflow policy: alert_and_continue (standard/prod) or reject_new_ops (fsi/sovereign). Backpressure alerts fire at 75% and 90% capacity. Records older than P7D trigger escalation regardless of capacity. |
+| `AUD-016` | Audit records for system-initiated changes declare actor.type: system with a system_actor block identifying the DCM component, trigger, and authorizing policy. System actor records are full audit records appearing in all queries and compliance reports. actor.type enables filtering between human, service_account, and system-initiated changes. |
+| `AUD-017` | In distributed DCM deployments, each instance maintains its own independent hash chain scoped to that instance. Federation-level integrity is maintained via daily Merkle root proofs computed from all instance chain tips, stored at the Hub DCM. Cross-instance audit queries present parallel chains with cross-references via correlation_id. |
+
+
+## 9a. Audit vs Observability โ€” The Definitive Distinction (Q16)
+
+Audit and Observability are separate components with separate storage contracts, separate consumers, and opposite fundamental trade-offs. They cannot be combined without violating one contract or the other.
+
+### 9a.1 Comparison
+
+| Dimension | Audit | Observability |
+|-----------|-------|--------------|
+| **Purpose** | Immutable record of WHAT HAPPENED and WHO authorized it | Real-time visibility into SYSTEM HEALTH and PERFORMANCE |
+| **Primary consumers** | Auditors, compliance, security, legal, regulators | SREs, platform engineers, operators, dashboards |
+| **Write rate** | Low โ€” one record per action | Very high โ€” multiple per second per component |
+| **Retention** | Very long โ€” P7Y+ (compliance-driven) | Short โ€” days to months (operational) |
+| **Mutability** | Never โ€” append-only, hash-chained | Downsampling and aggregation acceptable |
+| **Accuracy** | 100% required โ€” no sampling | Statistical sampling acceptable |
+| **Compliance grade** | Required | Not required |
+| **Cost per event** | High โ€” hash chain computation | Low โ€” time series append |
+| **Failure behavior** | Missing audit = compliance violation | Missing observability = operational inconvenience |
+| **Query model** | Point-in-time, actor-based, compliance reports | Time-series, rate queries, anomaly detection |
+| **Data model** | Closed 30-action vocabulary, structured | Open schema โ€” any component emits any metric |
+| **Storage type** | Audit Store (specialized sub-type) | Time-series database (Prometheus, InfluxDB) |
+
+### 9a.2 The Relationship
+
+Observability data MAY reference audit record UUIDs for correlation โ€” a spike in error rate can link to audit records from that time window. But they live in separate stores with separate contracts.
+
+- **Audit** answers: "What happened and who authorized it?"
+- **Observability** answers: "Is the system healthy and how is it performing?"
+
+These are different questions requiring different storage architectures.
+
+### 9a.3 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `AUD-013` | Audit and Observability are separate components with separate storage contracts, consumers, and failure behaviors. Audit is compliance-grade, append-only, hash-chained, long-retention. Observability is operational, time-series, high-throughput, short-retention. They serve different consumers and cannot be combined without violating one contract or the other. Observability data may reference audit record UUIDs for correlation but is stored separately. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md
new file mode 100644
index 0000000..284d02f
--- /dev/null
+++ b/content/docs/architecture/data-model/universal-groups.md
@@ -0,0 +1,588 @@
+---
+title: "Universal Group Model"
+type: docs
+weight: 14
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Grouping](08-resource-grouping.md) | [Entity Relationships](09-entity-relationships.md) | [Policy Organization](14-policy-profiles.md)
+
+---
+
+## 1. Purpose
+
+DCM previously used eight distinct grouping constructs โ€” Tenant, Resource Groups, Custom Resource Groups, Policy Groups, Policy Profiles, Layer Domain, Activation Scope, and Cross-Tenant Authorization. Each had its own structure, its own API, its own documentation, and its own membership model. This fragmentation created cognitive overhead for operators and prevented natural organizational structures (such as "everything related to Payments") from being expressed as a single construct.
+
+The **Universal Group Model** collapses all grouping into a single `DCMGroup` entity distinguished by `group_class` metadata. One mental model. One API. One registry. The same UUID, versioning, lifecycle, policy targeting, and audit trail apply to every group regardless of its class.
+
+**What changes:**
+- All grouping constructs become `group_class` values on `DCMGroup`
+- Existing UUIDs, handles, and API references are preserved
+- Existing APIs become class-filtered views of the universal group store
+- Migration is additive โ€” no breaking changes
+
+**What does NOT change:**
+- The structural invariants of each construct (one Tenant per resource, no constituent cross-tenant, etc.)
+- Policy enforcement behavior โ€” governed by the active Profile, not per-group configuration
+- The hard-tenancy model โ€” tenant_boundary groups retain all isolation guarantees
+
+---
+
+## 2. The DCMGroup Entity
+
+### 2.1 Universal Structure
+
+```yaml
+dcm_group:
+  artifact_metadata:
+    uuid: 
+    handle: 
+    version: 
+    status: 
+    created_by: 
+    owned_by: 
+    created_via: 
+    modifications: 
+
+  # IDENTITY
+  name: 
+  description: 
+  concern_tags: [payments, pci-scope, eu-west]  # free tagging โ€” discoverability
+
+  # WHAT KIND OF GROUP
+  group_class: 
+  group_subclass: 
+  # group_subclass examples: cost_center, business_unit, compliance_scope, project
+
+  # MEMBERSHIP
+  member_types_permitted: [resource_entity, policy, layer, group, tenant]
+  # Determines what can be a member of this group
+  # Single-type groups declare one type (e.g., [policy] for policy_collection)
+  # Composite groups declare multiple types
+
+  exclusivity:
+    per_member: 
+    # one:  a member can belong to only one group of this class at a time
+    # many: a member can belong to multiple groups of this class simultaneously
+    per_group: 
+    cap: 
+
+  members:
+    - member_uuid: 
+      member_type: 
+      member_role: 
+      added_at: 
+      added_by: 
+      valid_from: 
+      valid_until: 
+      membership_status: 
+
+  # BEHAVIOR
+  enforcement_model: 
+  # advisory:   group is a tag โ€” no system behavior enforced by DCM
+  # enforced:   group drives policy scoping and system behavior
+  # mandatory:  group membership is non-optional (structural requirement)
+  # NOTE: For tenant_boundary groups, enforcement_model is profile-governed
+  # โ€” the active Profile sets the enforcement floor, not per-group configuration
+
+  cross_boundary:
+    tenant_spanning: 
+    sovereignty_spanning: 
+
+  lifecycle_coupling:
+    on_group_destroy: 
+    # detach (DEFAULT): destroying the group releases memberships but
+    #                   does NOT destroy members
+    # cascade: destroying the group destroys all members
+    # notify:  destroying the group notifies owners and waits for confirmation
+    # retain:  group cannot be destroyed while it has members
+    on_member_destroy: 
+
+  # INHERITANCE AND COMPOSITION
+  extends: 
+  includes_groups:
+    - group_uuid: 
+      member_type_filter: [resource_entity]  # optional โ€” only include this type
+      # If omitted: all member types from the included group are pulled in
+
+  # NESTING (for tenant_boundary groups)
+  parent_group_uuid: 
+  child_groups: [, ...]  # populated by DCM โ€” do not set manually
+
+  # POLICY TARGETING
+  # Any policy can target this group by UUID or handle โ€” no special declaration
+  # Policy targeting a composite group applies to all member types by default
+  # Policy can narrow with: member_type_filter: [resource_entity]
+```
+
+### 2.2 Group Classes
+
+| group_class | Replaces | member_types_permitted | exclusivity.per_member | enforcement_model |
+|-------------|---------|----------------------|----------------------|------------------|
+| `tenant_boundary` | Tenant | resource_entity, group | one (structural lock) | profile-governed |
+| `resource_grouping` | Resource Group, Custom Resource Group | resource_entity | many | advisory |
+| `policy_collection` | Policy Group | policy | many | enforced |
+| `policy_profile` | Policy Profile | group (policy_collection only) | many | enforced |
+| `layer_grouping` | Layer Domain grouping | layer | many | enforced |
+| `provider_grouping` | Provider collections | provider | many | advisory |
+| `composite` | (new) | all types | many | configurable |
+| `federation` | (new) | group (tenant_boundary) | many | advisory |
+
+### 2.3 Structural Invariants โ€” Non-Overridable
+
+Regardless of `enforcement_model`, `group_class`, or active Profile, the following structural invariants always hold:
+
+| Invariant | Applies To | Rule |
+|-----------|-----------|------|
+| `GRP-INV-001` | `tenant_boundary` | A resource_entity may belong to exactly one active tenant_boundary group |
+| `GRP-INV-002` | `tenant_boundary` | Constituent relationships may not cross tenant_boundary group boundaries |
+| `GRP-INV-003` | `tenant_boundary` | Destroying a parent tenant_boundary group requires explicit resolution of all child groups first โ€” no silent cascade |
+| `GRP-INV-004` | `tenant_boundary` | A resource in a child tenant_boundary group belongs to the child โ€” never the parent |
+| `GRP-INV-005` | All | Circular group membership is invalid |
+| `GRP-INV-006` | All | A group cannot be a member of itself |
+
+---
+
+## 3. Group Class Reference
+
+### 3.1 tenant_boundary
+
+**Replaces:** Tenant entity  
+**Purpose:** Ownership boundary, isolation enforcement, cost attribution, audit scope, sovereignty boundary
+
+```yaml
+dcm_group:
+  group_class: tenant_boundary
+  member_types_permitted: [resource_entity, group]
+  exclusivity:
+    per_member: one   # STRUCTURAL LOCK โ€” cannot be changed by policy
+  enforcement_model: mandatory   # set by active Profile โ€” not configurable per-group
+  cross_boundary:
+    tenant_spanning: false   # STRUCTURAL LOCK
+  lifecycle_coupling:
+    on_group_destroy: notify   # requires explicit resolution
+    on_member_destroy: remove_from_group
+
+  # Tenant-specific fields preserved from original model
+  tenant_config:
+    hard_tenancy:
+      cross_tenant_relationships: explicit_only
+    active_profile: system/profile/standard
+    minimum_child_profile: null
+```
+
+**Profile-governed enforcement:**
+- `minimal` profile โ†’ `enforcement_model: advisory` (tenancy optional)
+- `dev` profile โ†’ `enforcement_model: enforced` (tenancy recommended)
+- `standard` and above โ†’ `enforcement_model: mandatory` (tenancy required)
+
+### 3.2 resource_grouping
+
+**Replaces:** DCM Default Resource Group, Custom Resource Group  
+**Purpose:** Flexible composable grouping of resource entities โ€” structured tagging
+
+```yaml
+dcm_group:
+  group_class: resource_grouping
+  group_subclass: cost_center   # advisory โ€” CostCenter, BusinessUnit, Project, Team...
+  member_types_permitted: [resource_entity]
+  exclusivity:
+    per_member: many   # a resource can be in multiple resource groups
+  enforcement_model: advisory
+```
+
+### 3.3 policy_collection
+
+**Replaces:** Policy Group  
+**Purpose:** Cohesive collection of policies addressing a single concern
+
+```yaml
+dcm_group:
+  group_class: policy_collection
+  concern_tags: [pci-dss, encryption, network-segmentation]
+  member_types_permitted: [policy]
+  enforcement_model: enforced
+  # Source โ€” local or Policy Provider
+  source:
+    type: 
+    provider_uuid: 
+    on_provider_update: 
+```
+
+### 3.4 policy_profile
+
+**Replaces:** Policy Profile  
+**Purpose:** Complete DCM configuration for a use case, composed of policy_collection groups
+
+```yaml
+dcm_group:
+  group_class: policy_profile
+  member_types_permitted: [group]   # only policy_collection groups
+  extends:     # inherits all parent's groups
+  enforcement_model: enforced
+```
+
+### 3.5 composite
+
+**New concept:** A group whose members span multiple member types โ€” the organizational unit for a complete concern.
+
+```yaml
+dcm_group:
+  group_class: composite
+  name: "Payments Platform"
+  concern_tags: [payments, pci-scope]
+  member_types_permitted: [resource_entity, policy, layer, group, provider]
+  enforcement_model: advisory   # composite groups are organizational โ€” advisory default
+
+  members:
+    - member_uuid: 
+      member_type: resource_entity
+      member_role: compute
+    - member_uuid: 
+      member_type: group
+      member_role: compliance_governance
+    - member_uuid: 
+      member_type: group
+      member_role: resource_inventory
+    - member_uuid: 
+      member_type: layer
+      member_role: configuration
+```
+
+**Policy targeting composite groups:**
+```yaml
+policy:
+  target_groups:
+    - group_uuid: 
+      member_type_filter: [resource_entity]   # narrow to resources only
+      # Omit member_type_filter to apply to ALL member types (default)
+```
+
+### 3.6 federation
+
+**New concept:** A group of tenant_boundary groups that share governance, visibility, and resources while maintaining complete independence.
+
+```yaml
+dcm_group:
+  group_class: federation
+  name: "Global FSI Federation"
+  member_types_permitted: [group]   # tenant_boundary groups only
+  enforcement_model: advisory       # federation cannot override member Tenant isolation
+
+  members:
+    - member_uuid: 
+      member_type: group
+      member_role: member_tenant
+    - member_uuid: 
+      member_type: group
+      member_role: member_tenant
+    - member_uuid: 
+      member_type: group
+      member_role: shared_governance
+
+  federation_config:
+    shared_policy_inheritance: 
+    # opt_in:  member Tenants must explicitly adopt shared policies
+    # opt_out: shared policies apply to all members unless explicitly excluded
+    cross_member_visibility: 
+    consolidated_reporting: true
+```
+
+---
+
+## 4. Nested Tenants
+
+### 4.1 Concept
+
+A **Nested Tenant** is a `tenant_boundary` group that is a member of a parent `tenant_boundary` group. The child Tenant maintains complete isolation โ€” its resources belong to it, not the parent. The parent Tenant has governance overlay, cost rollup authority, and audit aggregation across all children.
+
+```
+corporate_tenant (tenant_boundary)
+  โ”‚  child_groups:
+  โ”œโ”€โ”€ business_unit_a_tenant (tenant_boundary)
+  โ”‚     โ””โ”€โ”€ resources, policies, layers owned by BU-A
+  โ””โ”€โ”€ business_unit_b_tenant (tenant_boundary)
+        โ””โ”€โ”€ resources, policies, layers owned by BU-B
+```
+
+### 4.2 Structural Invariants for Nested Tenants
+
+- A resource belongs to the **leaf** tenant_boundary group โ€” never the parent (GRP-INV-004)
+- Parent Tenant has **governance overlay** โ€” not ownership
+- Parent Tenant destruction requires all child Tenants to be resolved first (GRP-INV-003)
+- Constituent relationships cannot cross any tenant_boundary boundary โ€” including parent-child (GRP-INV-002)
+
+### 4.3 Policy Inheritance Direction
+
+Policy inheritance from parent to child Tenant is profile-governed:
+
+| Profile | Default | Meaning |
+|---------|---------|---------|
+| `minimal`, `dev` | `opt_in` | Child Tenants must explicitly adopt parent policies |
+| `standard`, `prod` | `opt_out` | Parent policies cascade to children unless child excludes |
+| `fsi`, `sovereign` | `opt_in` | Nothing crosses without consent |
+
+```yaml
+nested_tenant_config:
+  parent_group_uuid: 
+  policy_inheritance: opt_out   # governed by active Profile
+  parent_policy_exclusions:
+    - policy_uuid:    # explicitly excluded from cascading to this child
+  cost_rollup_to_parent: true
+  audit_visible_to_parent: true
+  sovereign_boundary: independent   # child sovereignty independent of parent
+```
+
+### 4.4 Nested Tenant Use Cases
+
+- **Enterprise structure:** Corporate โ†’ Business Unit โ†’ Team Tenants
+- **Multi-region deployment:** Global Tenant โ†’ Regional Tenants โ†’ Zone Tenants
+- **Multi-tier compliance:** Organization Tenant โ†’ PCI-scope Tenant โ†’ Payment-processing Tenant
+- **Partner/customer isolation:** Platform Tenant โ†’ Customer A Tenant โ†’ Customer B Tenant
+
+---
+
+## 5. Federated Tenants
+
+### 5.1 Concept
+
+A **Federated Tenant** structure is a `federation` group containing multiple independent `tenant_boundary` groups. Member Tenants maintain complete independence โ€” the federation provides shared governance, consolidated visibility, and mutual cross-tenant authorization within the federation scope.
+
+### 5.2 Federation Capabilities
+
+- **Shared policy application:** `policy_collection` groups included in the federation apply to all member Tenants (per `shared_policy_inheritance` setting)
+- **Cross-member visibility:** federation members can declare mutual `cross_tenant_authorization` scoped to federation membership โ€” without requiring separate bilateral authorizations
+- **Consolidated reporting:** cost, audit, and observability queries scoped to the federation group return aggregated results across all member Tenants
+- **Federation-level governance:** policies targeting the federation group apply to all member Tenants
+
+### 5.3 Federation vs Nesting
+
+| Dimension | Nested Tenants | Federated Tenants |
+|-----------|---------------|-----------------|
+| Relationship | Parent-child hierarchy | Peer membership |
+| Governance direction | Top-down from parent | Shared among peers |
+| Independence | Child subordinate to parent | Members fully independent |
+| Cost rollup | Mandatory to parent | Configurable |
+| Use case | Enterprise hierarchy | Multi-organization collaboration |
+
+---
+
+## 6. Group Registry and API
+
+### 6.1 Universal Registry
+
+All groups are stored in a single **Group Registry** โ€” a GitOps store following the standard Storage Provider contract. The registry is queryable by any combination of fields.
+
+### 6.2 Class-Filtered API Views
+
+The universal registry exposes class-filtered views that preserve backward compatibility with existing API consumers:
+
+| Endpoint | Equivalent Query |
+|----------|----------------|
+| `GET /tenants` | `GET /groups?group_class=tenant_boundary` |
+| `GET /resource-groups` | `GET /groups?group_class=resource_grouping` |
+| `GET /policy-groups` | `GET /groups?group_class=policy_collection` |
+| `GET /policy-profiles` | `GET /groups?group_class=policy_profile` |
+| `GET /federations` | `GET /groups?group_class=federation` |
+
+Existing API references continue to work unchanged. New API consumers can use the universal endpoint.
+
+---
+
+## 7. Migration from Current Constructs
+
+Existing constructs migrate to the universal model with UUID preservation:
+
+| Current Construct | Migration | UUID Preserved? |
+|------------------|-----------|----------------|
+| Tenant entity | `group_class: tenant_boundary` | Yes |
+| DCM Default Resource Group | `group_class: resource_grouping` | Yes |
+| Custom Resource Group | `group_class: resource_grouping, group_subclass: ` | Yes |
+| Policy Group | `group_class: policy_collection` | Yes |
+| Policy Profile | `group_class: policy_profile` | Yes |
+
+Migration uses the standard ingestion model โ€” existing constructs are ingested as `ingestion_source: migration` with `ingestion_confidence: high` (UUID preservation, no ambiguity).
+
+---
+
+## 8. DCM System Policies
+
+| Policy | Rule |
+|--------|------|
+| `GRP-INV-001` | A resource_entity may belong to exactly one active tenant_boundary group |
+| `GRP-INV-002` | Constituent relationships may not cross tenant_boundary group boundaries |
+| `GRP-INV-003` | Destroying a parent tenant_boundary group requires explicit resolution of all child groups first |
+| `GRP-INV-004` | A resource in a child tenant_boundary group belongs to the child โ€” never the parent |
+| `GRP-INV-005` | Circular group membership is invalid and must be rejected |
+| `GRP-INV-006` | A group cannot be a member of itself |
+| `GRP-007` | Composite group `on_group_destroy` default is `detach` โ€” destroying a group releases memberships but does not destroy members |
+| `GRP-008` | Policies targeting a composite group apply to all member types by default; `member_type_filter` narrows scope |
+| `GRP-009` | Federation groups cannot override member Tenant isolation boundaries |
+| `GRP-010` | Nested Tenant policy inheritance direction is governed by the active Profile โ€” not per-group configuration |
+
+---
+
+## 9. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should composite group policy targeting emit a linting warning when no member_type_filter is declared? | Operational safety | โœ… Resolved โ€” linting warning (not error) when composite policy targeting has no member_type filter; suppress with explicit_no_filter: true (GRP-016) |
+| 2 | Should there be a maximum nesting depth for tenant_boundary groups? | Operational governance | โ“ Unresolved โ€” future consideration |
+| 3 | How does group membership interact with the Search Index? | Performance | โ“ Unresolved โ€” future consideration |
+| 4 | Should time-bounded memberships (valid_until) trigger notifications before expiry? | Consumer experience | โœ… Resolved โ€” warn_before_expiry field on membership (GRP-014) |
+
+---
+
+## 9. Grouping and Relationship Gap Resolutions
+
+### 9.1 Community Subclass Catalog (Q35)
+
+The `group_class` set is closed โ€” system behavior is tied to declared classes only. `group_subclass` is open and advisory. DCM maintains a community subclass catalog as a non-authoritative reference shipped with the well-known Information Provider Registry:
+
+```yaml
+# Community subclass catalog (advisory โ€” not enforced, not validated)
+common_group_subclasses:
+  resource_grouping:
+    - subclass: cost_center
+      description: "Financial cost attribution grouping"
+    - subclass: business_unit
+      description: "Organizational business unit"
+    - subclass: project
+      description: "Project-scoped resource collection"
+    - subclass: environment
+      description: "Environment grouping (prod/staging/dev)"
+    - subclass: application
+      description: "Application component grouping"
+  policy_collection:
+    - subclass: compliance_framework
+      description: "Policies implementing a compliance framework"
+    - subclass: technology_baseline
+      description: "Technology-specific policy baseline"
+```
+
+Organizations freely declare subclasses not in the catalog โ€” there is no validation or enforcement on subclass values.
+
+### 9.2 Group Sovereignty Interaction (Q36)
+
+Sovereignty interaction is group_class-specific:
+
+| group_class | Cross-Sovereignty | Notes |
+|-------------|-----------------|-------|
+| `tenant_boundary` | **Never** | Structural โ€” not configurable |
+| `resource_grouping` | Permitted by default | Policy may restrict for classified resources |
+| `policy_collection` | Always permitted | Policies have no sovereignty โ€” governance artifacts |
+| `layer_grouping` | Always permitted | Layers have no sovereignty |
+| `composite` | Governed by most restrictive member | If contains cross-sovereignty resources, resource rules apply |
+| `federation` | Permitted with DCM federation rules | DCM-003 governs data flows |
+
+```yaml
+# Policy restricting cross-sovereignty resource group membership
+policy:
+  type: gatekeeper
+  rule: >
+    If group.group_class == resource_grouping
+    AND member.classification_level IN [confidential, restricted]
+    AND member.sovereignty_zone != group.primary_sovereignty_zone
+    THEN gatekeep: "Classified resources cannot join cross-sovereignty resource groups"
+```
+
+### 9.3 Tenant Decommission Lifecycle (Q37)
+
+Tenant decommission is the highest-stakes lifecycle operation in DCM. It requires mandatory pre-decommission validation and follows a staged sequence.
+
+**Phase 1 โ€” Pre-decommission validation (blocking):**
+- All resources in decommissionable state (not PROVISIONING or active incidents)
+- Cross-tenant operational relationships accounted for (consuming Tenants notified)
+- Allocated resources claimed by other Tenants addressed (returned or migrated)
+- Active rehydration leases released
+- Compliance holds reviewed (HIPAA/PCI records may need archival)
+- Child tenant_boundary groups resolved first (GRP-INV-003)
+
+**Phase 2 โ€” Resource decommission (per lifecycle policy):**
+```
+For each resource in the Tenant:
+  cascade โ†’ decommission resource (default for tenant_boundary)
+  retain  โ†’ resource enters ORPHANED state (operator must rehome or destroy)
+  notify  โ†’ alert owner; resource enters PENDING_DECOMMISSION
+```
+
+**Phase 3 โ€” Group membership cleanup:**
+- Remove Tenant from all group memberships
+- Empty federation groups enter EMPTY state
+- Orphaned child groups must have been resolved in Phase 1
+
+**Phase 4 โ€” Audit record archival:**
+All audit records enter `all_retired` retention_status. They are **never destroyed** as part of Tenant decommission. Post-lifecycle retention clock starts per governing policy.
+
+### 9.4 Time-Bounded Group Membership (Q38)
+
+Group memberships already support time-bounded validity via `valid_from` and `valid_until` in the Universal Group Model. The Lifecycle Constraint Enforcer handles expiry.
+
+```yaml
+member:
+  member_uuid: 
+  member_type: resource_entity
+  valid_from: "2026-01-01T00:00:00Z"
+  valid_until: "2026-12-31T23:59:59Z"
+  membership_status: 
+  on_expiry: 
+  # remove:         member silently removed from group on expiry
+  # notify:         notify group owner; member remains with expired status (default)
+  # suspend_member: transition the member entity to SUSPENDED state
+  warn_before_expiry: P7D          # notify 7 days before expiry
+```
+
+Membership expiry produces a `MEMBER_REMOVE` audit record with `reason: membership_ttl_expired`.
+
+### 9.5 Group Policy Inheritance โ€” Nested Groups (Q39)
+
+Policy inheritance for nested groups is group_class-specific and profile-governed:
+
+| group_class | Default | Profile Override |
+|-------------|---------|----------------|
+| `tenant_boundary` | `opt_out` (parent cascades unless child excludes) | `opt_in` for minimal/dev/fsi/sovereign |
+| `resource_grouping` | Not applicable | Resource groups are tags โ€” policies target them, not inherit through them |
+| `policy_collection` | Not applicable | Policy collections use `extends` for inheritance |
+| `composite` | `opt_out` | Configurable per group |
+| `federation` | `opt_in` | Peer consent always required โ€” not configurable |
+
+```yaml
+# Nested group policy inheritance declaration
+dcm_group:
+  group_class: tenant_boundary
+  parent_group_uuid: 
+  policy_inheritance: opt_out     # governed by active Profile
+  parent_policy_exclusions:
+    - policy_uuid:          # explicitly excluded from cascading to this child
+```
+
+---
+
+## 10. System Policies โ€” Grouping Gaps
+
+| Policy | Rule |
+|--------|------|
+| `GRP-011` | The group_class set is closed โ€” system behavior is tied to declared classes only. group_subclass is open and advisory. DCM maintains a community subclass catalog as a non-authoritative reference. No validation or enforcement on subclass values. |
+| `GRP-012` | Sovereignty interaction is group_class-specific. tenant_boundary groups never span sovereignty boundaries (structural). resource_grouping groups may span sovereignty boundaries by default โ€” policy may restrict for classified resources. policy_collection and layer_grouping groups always permitted cross-sovereignty. composite groups are governed by the sovereignty rules of their most restrictive member type. |
+| `GRP-013` | Tenant decommission requires pre-decommission validation (resource state, cross-tenant relationships, compliance holds, child group resolution). Resources follow declared lifecycle policy. Child tenant_boundary groups must be resolved before parent decommission. Audit records enter post-lifecycle retention โ€” never destroyed as part of Tenant decommission. |
+| `GRP-014` | Group memberships support time-bounded validity via valid_from and valid_until. Membership expiry is enforced by the Lifecycle Constraint Enforcer. Expiry produces a MEMBER_REMOVE audit record. on_expiry action (remove, notify, suspend_member) declared per membership. Default: notify. |
+| `GRP-015` | Group policy inheritance is group_class-specific and profile-governed. tenant_boundary: opt_out (standard/prod) or opt_in (minimal/dev/fsi/sovereign). federation: always opt_in โ€” peer consent required. composite: opt_out by default. resource_grouping and policy_collection: not applicable. |
+
+
+## 11. Related Concepts
+
+- **Resource Grouping** (doc 08) โ€” original resource grouping model, now implemented via `group_class: resource_grouping`
+- **Policy Organization** (doc 14) โ€” Policy Groups and Profiles, now implemented via `group_class: policy_collection` and `group_class: policy_profile`
+- **Entity Relationships** (doc 09) โ€” cross-tenant authorized relationships between groups
+- **Universal Audit Model** (doc 16) โ€” all group changes produce audit records
+- **Ingestion Model** (doc 13) โ€” migration of existing constructs to universal groups
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md
new file mode 100644
index 0000000..2855a08
--- /dev/null
+++ b/content/docs/architecture/data-model/webhooks-messaging.md
@@ -0,0 +1,726 @@
+---
+title: "Webhooks, Messaging, and External Integration"
+type: docs
+weight: 17
+---
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** ๐Ÿ”„ In Progress  
+**Related Documents:** [Universal Audit Model](16-universal-audit.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Authentication and Authorization](19-auth-providers.md) | [Policy Organization](14-policy-profiles.md)
+
+---
+
+## 1. Purpose
+
+DCM communicates with the outside world through three complementary mechanisms:
+
+- **Outbound Webhooks** โ€” DCM pushes event notifications to external HTTP endpoints
+- **Inbound Webhooks** โ€” External systems push requests, queries, and events to DCM HTTP endpoints
+- **Message Bus Providers** โ€” DCM integrates with external message buses for persistent, high-throughput bidirectional event streaming
+
+All three mechanisms are authenticated, authorized, and audited identically to any other DCM API call. There is no privileged back-channel. Every integration is a registered DCM actor subject to full Policy Engine evaluation.
+
+**Webhooks are optional and policy-governed.** The active Profile sets defaults โ€” `fsi` and `sovereign` profiles may require webhook or message bus coverage for audit events. `minimal` and `dev` profiles make them fully optional.
+
+---
+
+## 2. Ingress and Egress โ€” The Universal Actor Model
+
+### 2.1 The `ingress` Block
+
+Every request entering DCM carries an immutable `ingress` block set by the DCM ingress layer before Policy Engine evaluation. It is never consumer-declarable and never modifiable by policies โ€” policies may only read it.
+
+```yaml
+ingress:
+  # HOW it arrived
+  surface: 
+  protocol: 
+  authenticated_via: 
+  authorized_via: policy_engine
+
+  # WHO sent it โ€” fully resolved actor context
+  actor:
+    uuid: 
+    type: 
+    display_name: "Jane Smith"
+    identity_source: 
+    auth_provider_uuid:           # which Auth Provider authenticated
+    auth_provider_type: 
+
+    # Resolved DCM roles and scope (at authentication time)
+    roles: [sre]
+    tenant_scope: []
+    groups: []
+    permissions: [request.submit, query.entity_state]
+
+    # Authorization chain
+    authorized_by:
+      method: 
+      authorizing_entity_uuid: 
+      authorization_timestamp: 
+      expiry: 
+
+    # Session context (human actors)
+    session_uuid: 
+    session_started_at: 
+    mfa_verified: 
+
+    # External identity (federated actors)
+    external_identity:
+      provider: 
+      subject: "uid=jsmith,cn=users,cn=accounts,dc=corp,dc=example,dc=com"
+      claims:
+        email: jsmith@corp.example.com
+        display_name: "Jane Smith"
+        department: Engineering
+        cost_center: CC-1234
+        ldap_groups: [cn=dcm-sre,...]
+        sid: "S-1-5-21-..."        # AD Security Identifier
+
+    # Rate limit tracking
+    rate_limit_bucket: 
+
+  # Surface-specific detail
+  webhook_registration_uuid:    # if surface: webhook_inbound
+  message_bus_provider_uuid:    # if surface: message_bus_inbound
+  message_offset:             # if surface: message_bus_inbound
+  scheduler_job_uuid:           # if surface: scheduler
+  parent_request_uuid:          # if surface: policy_engine|rehydration
+  source_ip: 
+```
+
+### 2.2 The `egress` Block
+
+All outbound calls from DCM carry DCM's authenticated identity:
+
+```yaml
+egress:
+  surface: 
+  protocol: 
+  actor:
+    uuid: 
+    type: dcm_internal
+    component: 
+    authenticated_via: 
+    credential_ref:
+      credential_provider_uuid: 
+      secret_path: 
+  originating_request_uuid: 
+  originating_actor_uuid: 
+```
+
+### 2.3 Policy Engine Use Cases โ€” Ingress/Egress Fields
+
+The ingress block enables a rich class of governance rules:
+
+```yaml
+# Require specific auth for sensitive operations
+policy: "If action == decommission AND ingress.actor.mfa_verified == false THEN gatekeep"
+
+# Block legacy API keys from production Tenants
+policy: "If tenant.profile == prod AND ingress.actor.identity_source == static_api_key THEN gatekeep"
+
+# Require enterprise auth for security resources
+policy: "If resource_type IN [Network.FirewallRule] AND ingress.actor.auth_provider_type NOT IN [oidc, freeipa, active_directory] THEN gatekeep"
+
+# Enrich from OIDC claims
+policy: "If ingress.actor.external_identity.claims.department EXISTS THEN inject: business_context.department"
+
+# Block message bus inbound from non-service-accounts
+policy: "If ingress.surface == message_bus_inbound AND ingress.actor.type != webhook_service_account THEN gatekeep"
+
+# Sovereignty check on inbound message bus
+policy: "If ingress.surface == message_bus_inbound AND message_bus_provider.jurisdiction != tenant.sovereignty_zone THEN gatekeep"
+```
+
+### 2.4 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ING-008` | All DCM requests must carry an ingress block. The ingress block is set by the DCM ingress layer and is immutable โ€” policies may read but not modify it. |
+| `ING-009` | The ingress block must include a fully resolved actor context: uuid, type, identity_source, roles, tenant_scope, auth_provider_uuid, and authorized_by chain. |
+| `ING-010` | All egress calls from DCM must carry DCM's authenticated identity. Unauthenticated egress is rejected. |
+| `ING-011` | Authentication enforcement is profile-governed. Standard and above reject unauthenticated requests. Minimal and dev support lightweight authenticated modes. There is no anonymous access in any profile. |
+| `ING-012` | Webhook and message bus inbound surfaces always require authentication regardless of active Profile. |
+| `ING-013` | Rate limiting is enforced per registered actor. Exceeding rate limits returns 429 Too Many Requests. |
+
+---
+
+## 3. Outbound Webhooks
+
+### 3.1 Concept
+
+DCM pushes event notifications to registered external HTTP endpoints. Outbound webhooks are **optional and policy-governed** โ€” the active Profile and Policy Groups determine which events require external notification.
+
+### 3.2 Webhook Registration
+
+```yaml
+webhook_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "org/webhooks/payments-drift-alerts"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Payments Platform Team"
+      notification_endpoint: 
+
+  name: "Payments Platform Drift Alerts"
+  description: "Notifies payments team of drift detection events"
+
+  # SCOPE
+  scope:
+    type: 
+    tenant_uuid: 
+    cross_tenant_authorization_uuid:    # if cross_tenant
+
+  # EVENT SUBSCRIPTIONS
+  event_subscriptions:
+    - event_type: drift.detected
+      schema_version: "1.0"
+      adapter: true              # DCM transforms newer schemas to 1.0
+      filter:
+        tenant_uuid: 
+        resource_types: [Compute.VirtualMachine, Storage.Block]
+    - event_type: request.realized
+      schema_version: "1.0"
+    - event_type: entity.state_transition
+      schema_version: "1.0"
+      filter:
+        to_states: [DEGRADED, FAILED]
+
+  # ENDPOINT
+  endpoint:
+    url: https://alerts.payments.corp.example.com/dcm/events
+    sovereignty_check: true     # verify endpoint jurisdiction before delivery
+
+  # AUTHENTICATION
+  authentication:
+    mode: 
+    secret_ref:
+      credential_provider_uuid: 
+      secret_path: "dcm/webhooks/payments-drift/hmac-secret"
+    rotation_policy:
+      automatic: true
+      interval: P90D
+      transition_window: P7D
+      notify_before: P14D
+
+  # RELIABILITY
+  retry_policy:
+    max_attempts: 7
+    backoff: exponential
+    initial_interval: PT5S
+    max_interval: PT1H
+    timeout_per_attempt: PT10S
+    on_exhaustion: dead_letter   # dead_letter | discard | escalate
+
+  # HEALTH
+  health:
+    failure_threshold: 10
+    suspension_notification: true
+    auto_deactivate_after: P30D
+    status: active
+
+  # SCHEMA COMPATIBILITY
+  schema_adapter:
+    enabled: true
+    # DCM maintains forward-compatibility adapters per schema version
+    # Consumer stays on declared schema_version indefinitely
+    deprecation_notice_days: 90
+```
+
+### 3.3 Event Taxonomy
+
+The event taxonomy maps onto the Universal Audit action vocabulary. All are versioned registry entries:
+
+| Category | Events |
+|----------|--------|
+| Entity lifecycle | `entity.created`, `entity.modified`, `entity.state_transition`, `entity.deleted`, `entity.expired`, `entity.rehydrated` |
+| Group | `group.member_added`, `group.member_removed`, `group.created`, `group.deleted` |
+| Relationship | `relationship.created`, `relationship.released` |
+| Policy | `policy.activated`, `policy.deactivated`, `policy.evaluated` (fail/gatekeep only), `policy.shadow_result` |
+| Provider | `provider.healthy`, `provider.degraded`, `provider.unhealthy`, `provider.registered`, `provider.deregistered` |
+| Audit/security | `audit.chain_break`, `audit.forward_failed` |
+| Drift | `drift.detected`, `drift.resolved`, `drift.escalated` |
+| Request | `request.submitted`, `request.approved`, `request.rejected`, `request.realized`, `request.failed` |
+| Rehydration | `rehydration.started`, `rehydration.completed`, `rehydration.paused`, `rehydration.interrupted` |
+| Authorization | `authorization.granted`, `authorization.revoked` |
+| Webhook | `webhook.secret_rotated`, `webhook.suspended`, `webhook.schema_deprecated` |
+
+### 3.4 Payload Format
+
+```yaml
+webhook_payload:
+  # Envelope
+  event_uuid:               # idempotency key
+  event_type: drift.detected
+  event_schema_version: "1.0"
+  timestamp:            # from Stage 1 Commit Log โ€” authoritative
+  dcm_version: 
+
+  # Subject
+  subject:
+    entity_uuid: 
+    entity_type: resource_entity
+    entity_handle: 
+    tenant_uuid: 
+
+  # Delta
+  delta:
+    drifted_fields:
+      - field: cpu_count
+        realized_value: 4
+        discovered_value: 8
+    drift_severity: significant
+
+  # Links
+  links:
+    self: 
+    audit_record: 
+```
+
+### 3.5 Delivery Guarantees
+
+- **At-least-once** โ€” not exactly-once; consumers must be idempotent using `event_uuid`
+- **Per-entity ordering** โ€” events for a given `entity_uuid` delivered in Commit Log sequence order
+- **Cross-entity ordering** โ€” not guaranteed; use `timestamp` for actual occurrence time
+- **Sovereignty-aware** โ€” delivery blocked if endpoint jurisdiction incompatible with Tenant sovereignty (WHK-004)
+
+---
+
+## 4. Inbound Webhooks
+
+### 4.1 Concept
+
+DCM exposes authenticated HTTP endpoints that external systems call to submit requests, queries, and events. Inbound webhooks are subject to full Policy Engine evaluation โ€” identical to any other API call.
+
+### 4.2 Inbound Endpoints
+
+| Endpoint | Purpose |
+|----------|---------|
+| `POST /webhooks/inbound/request` | Submit a service request |
+| `POST /webhooks/inbound/query` | Query entity state or catalog |
+| `POST /webhooks/inbound/event` | Push an event (provider state change, CI/CD signal) |
+| `POST /webhooks/inbound/ingestion` | Push brownfield ingestion data |
+| `POST /webhooks/inbound/data` | Push enrichment or information data |
+
+### 4.3 Webhook Actor Registration
+
+Every inbound webhook caller must be registered as a **webhook actor** โ€” a service account in the DCM identity model:
+
+```yaml
+webhook_actor:
+  artifact_metadata:
+    uuid: 
+    handle: "actors/webhook/cicd-pipeline-prod"
+    status: active
+
+  name: "CI/CD Pipeline Production"
+  actor_type: webhook_service_account
+
+  # Authentication
+  authentication:
+    mode: hmac_sha256
+    secret_ref:
+      credential_provider_uuid: 
+      secret_path: "dcm/webhooks/inbound/cicd-pipeline/hmac"
+
+  # Authorization
+  role: consumer
+  tenant_scope: []
+  permitted_operations:
+    - request.submit
+    - query.entity_state
+    - query.catalog
+
+  # Rate limiting
+  rate_limit:
+    requests_per_minute: 60
+    burst: 10
+
+  # Audit identity
+  audit_identity:
+    display_name: "CI/CD Pipeline (Production)"
+    system: "jenkins-prod-01"
+```
+
+### 4.4 Response Model
+
+- **Queries** โ€” synchronous response with result
+- **Requests and events** โ€” `202 Accepted` + `request_uuid`; caller polls status or registers outbound webhook for completion notification
+
+---
+
+## 5. Message Bus Provider
+
+### 5.1 Concept
+
+A **Message Bus Provider** is the sixth DCM provider type โ€” a persistent, high-throughput integration with an external message bus for bidirectional event streaming. Where webhooks are point-to-point HTTP calls, a Message Bus Provider is a durable pub/sub connection.
+
+### 5.2 Registration
+
+```yaml
+message_bus_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/messagebus/corporate-kafka"
+    version: "1.0.0"
+    status: active
+
+  name: "Corporate Kafka Cluster"
+  provider_type: message_bus
+
+  # Direction
+  direction: 
+
+  # Protocol
+  protocol: 
+
+  # Connection
+  connection:
+    brokers: [kafka-1.corp:9093, kafka-2.corp:9093, kafka-3.corp:9093]
+    credentials_ref:
+      credential_provider_uuid: 
+      secret_path: "dcm/providers/messagebus/corporate-kafka/credentials"
+    tls:
+      mode: mtls
+      ca_cert_ref:
+        credential_provider_uuid: 
+        secret_path: "dcm/providers/messagebus/corporate-kafka/ca-cert"
+
+  # Outbound โ€” DCM publishes to external bus
+  outbound:
+    topic_mapping:
+      entity.state_transition: "dcm.entities.state"
+      drift.detected: "dcm.drift.alerts"
+      request.realized: "dcm.requests.completed"
+      audit.chain_break: "dcm.security.alerts"
+    schema_version: "1.0"
+    delivery_guarantee: at_least_once
+
+  # Inbound โ€” DCM consumes from external bus
+  inbound:
+    consumer_group: "dcm-inbound-prod"
+    topic_mapping:
+      "cicd.deployment.completed": request.submit
+      "cmdb.discovery.update": ingestion.push
+      "itsm.change.approved": request.approve
+    # Inbound messages processed as authenticated API calls
+    actor_identity_uuid: 
+    # Same Policy Engine evaluation as inbound webhooks
+
+  # Sovereignty
+  operational_sovereignty:
+    jurisdiction: eu-west
+    certifications: [ISO-27001, GDPR-compliant]
+
+  # Health
+  health_check:
+    interval_seconds: 30
+    on_unhealthy: alert
+```
+
+### 5.3 Architecture
+
+```
+DCM internal Message Bus (internal pub/sub backbone)
+  โ”‚
+  โ”œโ”€โ”€ Webhook Delivery Service โ”€โ”€โ”€โ”€โ”€โ”€โ†’ External HTTP endpoints (outbound webhooks)
+  โ”‚
+  โ””โ”€โ”€ Message Bus Bridge Service โ”€โ”€โ”€โ”€โ†’ External message bus (Message Bus Provider)
+                                 โ†โ”€โ”€โ”€ External message bus (inbound)
+```
+
+The internal Message Bus is never exposed directly. All external event integration goes through either the Webhook Delivery Service or the Message Bus Bridge Service โ€” both of which handle authentication, authorization, sovereignty checks, and schema transformation.
+
+---
+
+## 6. Git PR Ingress โ€” Distributed Git Request Mechanism
+
+### 6.1 Concept
+
+DCM supports **git_pr_merge** as a twelfth ingress surface โ€” enabling teams to submit DCM resource definitions as Pull Requests to a DCM-watched Git repository. This is the native workflow for infrastructure-as-code teams: open a PR, get it reviewed by humans and DCM's policy engine simultaneously, merge to execute.
+
+**Why Git PR ingress matters:**
+- GitOps teams work in Git โ€” their deployment workflow is already PR-based
+- Security and compliance teams review infrastructure changes the same way they review code
+- The PR itself is the human review record; DCM's audit trail captures the automated processing
+- Rollback is a Git revert โ€” natural and familiar
+- Multi-team approval workflows use existing Git branch protection rules
+- The PR diff shows exactly what changes โ€” field-level visibility
+
+### 6.2 The Git Request Watcher
+
+A dedicated control plane component โ€” the **Git Request Watcher** โ€” monitors designated repositories via webhooks (preferred) or polling. It is policy-governed: which repositories it watches, which branches trigger processing, and which resource types may be submitted via Git PR.
+
+### 6.3 Request Repository Structure
+
+```
+dcm-requests/                              โ† DCM-watched request repository
+  {tenant-uuid}/
+    pending/
+      {resource-handle}/
+        request.yaml                       โ† Standard DCM resource definition
+    realized/
+      {resource-handle}/
+        realized.yaml                      โ† DCM writes realized state here on success
+    failed/
+      {resource-handle}/
+        request.yaml                       โ† Moved here on failure with error detail
+```
+
+### 6.4 Git Actor Identity Resolution
+
+**Authentication is always required.** Git PR ingress actors must be resolved to DCM actors through the registered Auth Provider โ€” DCM trusts the Git server's authentication assertion, not user-declared Git configuration. Anyone can set their local `git config user.email` to anything; DCM ignores self-declared identity.
+
+**The trust chain:**
+```
+Git server authenticates user (SSH key, OAuth token, password)
+  โ”‚  Git server's authentication is trusted โ€” not user's claimed identity
+  โ–ผ
+DCM Git Request Watcher receives PR merge webhook
+  โ”‚  Webhook payload contains: actor.login, actor.auth_method, actor.external_id
+  โ”‚  All verified by the Git server
+  โ–ผ
+Auth Provider resolution (same path as web UI login for the same user):
+  โ”œโ”€โ”€ OIDC/OAuth: Git server OAuth subject โ†’ OIDC Auth Provider userinfo lookup
+  โ”œโ”€โ”€ LDAP/AD: Git server username โ†’ LDAP lookup โ†’ DCM actor
+  โ”œโ”€โ”€ SSH key: Git server key fingerprint โ†’ DCM SSH key registry โ†’ DCM actor
+  โ””โ”€โ”€ Service account: Git service account โ†’ registered webhook actor
+  โ–ผ
+Fully resolved DCM actor โ€” same roles, groups, tenant scope as any other user
+```
+
+**Resolution methods:**
+
+```yaml
+git_actor_resolution:
+  # Method 1: OIDC/OAuth (recommended โ€” Git server uses same IdP as DCM)
+  method: oidc_subject_lookup
+  auth_provider_uuid: 
+
+  # Method 2: LDAP/AD (enterprise โ€” Git server authenticates via corporate directory)
+  method: ldap_username_lookup
+  auth_provider_uuid: 
+
+  # Method 3: SSH key fingerprint
+  method: ssh_key_fingerprint
+  # Keys registered in DCM SSH key registry, linked to actor UUIDs
+
+  # Method 4: Service account (automated workflows)
+  method: webhook_service_account
+  # Git service account mapped to registered webhook actor
+```
+
+**Identity resolution failure โ€” explicit rejection:**
+
+When DCM cannot map the merge actor to a DCM actor, the PR is rejected with an actionable comment. Never silently ignored.
+
+```
+โŒ DCM Identity Resolution Failed
+
+DCM could not map the merge actor "jsmith" to a DCM actor identity.
+
+Possible causes:
+  โ€ข Your Git account is not linked to a DCM actor via the corporate Auth Provider
+  โ€ข Your DCM actor account has been suspended or deactivated
+
+To resolve:
+  โ€ข Contact your platform administrator: https://dcm.corp.example.com/actors/git-identity-setup
+
+This PR will not be processed until identity resolution succeeds.
+```
+
+### 6.5 The ingress Block for Git PR
+
+```yaml
+ingress:
+  surface: git_pr_merge               # or: git_pr_open (for shadow validation)
+  protocol: https
+  authenticated_via: oidc             # or: ldap_direct_bind, active_directory, ssh_key
+  actor:
+    uuid: 
+    type: human
+    display_name: "Jane Smith"
+    identity_source: oidc
+    auth_provider_uuid: 
+    roles: [consumer]
+    tenant_scope: []
+    groups: []
+    # Groups and tenant scope: SAME mappings as web UI login for this user
+    external_identity:
+      provider: github                # or: gitlab, gitea, freeipa, active_directory
+      subject:         # verified by Git server
+      git_username: jsmith
+      git_verified_email: jsmith@corp.com  # from Git server record โ€” not git config
+    mfa_verified: true                # from Auth Provider session record
+  git_context:
+    repository: https://git.corp.example.com/dcm-requests/payments
+    pr_number: 142
+    pr_url: https://git.corp.example.com/payments/pulls/142
+    merge_commit: 
+    base_branch: main
+    pr_author: jsmith
+    pr_reviewers: [platform-team, security-team]
+    pr_approved_by: [, ]
+    # Approved by: DCM resolves Git reviewer identities via same Auth Provider
+```
+
+### 6.6 PR Lifecycle โ€” DCM Processing Flow
+
+```
+1. Developer creates resource definition YAML (standard DCM request format)
+   โ”‚
+2. Opens PR against dcm-requests/{tenant-uuid}/pending/
+   โ”‚
+3. DCM Git Watcher detects PR (git_pr_open event)
+   โ”‚
+4. DCM resolves PR author โ†’ DCM actor via Auth Provider
+   โ”‚  Failure โ†’ post rejection comment; stop processing
+   โ”‚
+5. DCM validates actor tenant scope against target Tenant
+   โ”‚  Failure โ†’ post rejection comment; stop processing
+   โ”‚
+6. Shadow policy evaluation (same nine-step assembly โ€” dry run)
+   โ”‚  Results posted as PR review comments:
+   โ”‚  "โœ… Schema valid"
+   โ”‚  "โœ… All policies pass"
+   โ”‚  "โš ๏ธ  Will be placed in eu-west-1 per sovereignty policy"
+   โ”‚  "โŒ GateKeeper: VM size exceeds quota โ€” reduce cpu_count"
+   โ”‚
+7. Human review and approval (standard Git PR workflow)
+   โ”‚  Branch protection enforces required reviewers
+   โ”‚  Policy may declare: require DCM-defined approvers in git_context.pr_approved_by
+   โ”‚
+8. PR merged to main (git_pr_merge event)
+   โ”‚  ingress_surface: git_pr_merge
+   โ”‚  Actor re-verified at merge time (not assumed from PR open time)
+   โ”‚
+9. DCM processes as standard nine-step assembly (real โ€” not shadow)
+   โ”‚
+10. DCM posts realization result as PR comment + status check
+    โ”‚  "โœ… VM payments-prod-01 realized โ€” UUID: "
+    โ”‚  "โŒ Realization failed โ€” audit record: "
+    โ”‚
+11. DCM commits realized state to realized/ directory (optional)
+    Git history = full lifecycle record
+```
+
+### 6.7 Policy Engine Use Cases โ€” Git PR Ingress
+
+```yaml
+# Require PR approval before processing production resources
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND tenant.profile IN [prod, fsi, sovereign]
+    AND ingress.git_context.pr_approved_by NOT CONTAINS required_approvers
+    THEN gatekeep: "PR requires approval from platform admin and security owner"
+
+# Require MFA for Git PR merges in production Tenants
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND tenant.profile IN [prod, fsi, sovereign]
+    AND ingress.actor.mfa_verified == false
+    THEN gatekeep: "MFA required for Git PR merges in production Tenants"
+
+# Restrict resource types submittable via Git PR
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND resource_type NOT IN [Compute.VirtualMachine, Storage.Block]
+    THEN gatekeep: "Only compute and storage resources may be submitted via Git PR"
+
+# Require actor to be in authorized Git team for target Tenant
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND ingress.actor.groups NOT CONTAINS tenant.authorized_git_groups
+    THEN gatekeep: "Git PR author is not in an authorized group for this Tenant"
+
+# Post shadow results as PR comments (transformation)
+policy:
+  type: transformation
+  placement_phase: pre
+  rule: >
+    If ingress.surface IN [git_pr_merge, git_pr_open]
+    THEN inject: git_feedback.post_as_pr_comment = true
+```
+
+### 6.8 Updated Ingress Surface Taxonomy
+
+```yaml
+ingress_surface_taxonomy:
+  - web_ui                 # DCM's own web interface
+  - consumer_api           # Direct REST API call
+  - webhook_inbound        # Inbound webhook call
+  - message_bus_inbound    # Inbound message bus
+  - provider_callback      # Provider reporting back
+  - policy_engine          # Policy-generated sub-request
+  - scheduler              # Scheduled/timed trigger
+  - rehydration            # Rehydration-generated
+  - ingestion              # Ingestion pipeline
+  - dcm_internal           # DCM system-generated
+  - operator_cli           # Command line interface
+  - git_pr_merge           # Git PR merge โ†’ execute  โ† new
+  - git_pr_open            # Git PR open โ†’ shadow validation only  โ† new
+```
+
+### 6.9 System Policies โ€” Git PR Ingress
+
+| Policy | Rule |
+|--------|------|
+| `GIT-001` | DCM supports git_pr_merge and git_pr_open as ingress surfaces. Git PR ingress is subject to full Policy Engine evaluation identical to API ingress. |
+| `GIT-002` | DCM trusts the Git server's authentication assertion โ€” not user-declared Git configuration. DCM resolves the Git server's verified identity through the registered Auth Provider to produce a fully-resolved DCM actor with the same role, group, and tenant scope mappings as any other user authenticated via the same Auth Provider. |
+| `GIT-003` | Unresolvable Git actor identities are rejected with an actionable PR comment. PRs are never silently ignored. |
+| `GIT-004` | The resolved Git PR actor must have the target Tenant UUID in their tenant_scope. PRs targeting Tenants outside the actor's scope are rejected โ€” same enforcement as API tenant scope checks. |
+| `GIT-005` | DCM posts shadow policy evaluation results as PR review comments on git_pr_open. GateKeeper failures should be surfaced via repository branch protection integration. |
+| `GIT-006` | PR approval status may be declared as an authorization requirement by policy. DCM checks declared reviewer approvals against the PR's actual approval record before processing a merged PR. |
+| `GIT-007` | The Git Request Watcher component is policy-governed: which repositories it monitors, which branches trigger processing, and which resource types may be submitted via Git PR are declared via Policy Group. |
+| `GIT-008` | Actor identity is re-verified at merge time โ€” not assumed from PR open time. A user whose DCM actor is suspended between PR open and merge will be rejected at merge. |
+
+---
+
+
+
+| Policy | Rule |
+|--------|------|
+| `WHK-001` | Events for a given entity_uuid are delivered in Commit Log sequence order. Cross-entity ordering is not guaranteed. |
+| `WHK-002` | Webhook delivery uses at-least-once semantics. Consumers must be idempotent using event_uuid as the deduplication key. |
+| `WHK-003` | Outbound webhook authentication must be declared at registration. Supported: hmac_sha256 (default), mtls, bearer_token. Unauthenticated webhooks are rejected. |
+| `WHK-004` | Webhook endpoints that would deliver data outside a Tenant's sovereignty boundary are subject to sovereignty checks. |
+| `WHK-005` | Webhook governance is policy-driven. Profiles set defaults โ€” fsi/sovereign profiles may require webhook coverage for audit events via Policy Group. |
+| `WHK-006` | Webhook registrations must declare the event schema version expected. DCM supports current and N-1 schema versions simultaneously. |
+| `WHK-007` | Platform-scoped webhooks require Platform Admin role and are audit-logged as CONFIG_CHANGE. |
+| `WHK-008` | Cross-tenant webhooks require a valid cross_tenant_authorization record (XTA-001). |
+| `WHK-009` | Inbound webhook callers must be registered as webhook actors with explicit role, tenant scope, and permitted operations. Unregistered callers are rejected with 401. |
+| `WHK-010` | Inbound webhook calls are subject to full Policy Engine evaluation โ€” identical to any other API call. No bypass. |
+| `WHK-011` | All inbound webhook calls are recorded in the audit trail with the webhook actor as the immediate actor. |
+| `WHK-012` | Inbound webhook endpoints return 202 Accepted + request_uuid for async operations. |
+| `WHK-013` | Rate limiting is enforced per registered webhook actor. Exceeding rate limits returns 429 Too Many Requests. |
+| `WHK-014` | All credential references in webhook and message bus configurations must resolve through a registered Credential Provider. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/capabilities-map.html b/content/docs/capabilities-map.html
new file mode 100644
index 0000000..7ec144a
--- /dev/null
+++ b/content/docs/capabilities-map.html
@@ -0,0 +1,824 @@
+
+
+
+
+
+DCM Capabilities Matrix
+
+
+
+
+ +
+
DCM Project ยท Red Hat FlightPath
+

Foundational Capabilities Matrix

+

100 core operational capabilities required for DCM to perform lifecycle management. Each capability is mapped across consumer, producer, and platform/admin perspectives.

+
+
95
Capabilities
+
15
Domains
+
21
MVP Critical Path
+
3
Perspectives
+
+
+ + +
+ + +
+
+ โŒ• + +
+ +
+ + +
+
+ +
Showing 95 of 95 capabilities
+ + +
+
+ + + + + + + + + + + + +
IDCapabilityConsumerProducerPlatform / AdminDepends On
+
+
+ + + + + + +
+ + + + diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md new file mode 100644 index 0000000..aacf75b --- /dev/null +++ b/content/docs/capabilities-matrix.md @@ -0,0 +1,271 @@ +--- +title: "DCM Foundational Capabilities Matrix" +type: docs +weight: 10 +--- + +> **Purpose:** This document defines the core operational capabilities required for DCM to perform lifecycle management as defined by the data model. Each capability maps to a consumer/producer perspective and will be used to drive implementation work in Jira. +> +> **How to read this document:** +> - **Capability Domain** โ€” the architectural area the capability belongs to +> - **Capability** โ€” a discrete operational function; the smallest unit of independently implementable behavior +> - **Consumer perspective** โ€” what the end user / application team experiences +> - **Producer perspective** โ€” what the Service Provider or platform component must implement +> - **Platform/Admin perspective** โ€” what the platform engineer or SRE must configure or operate +> - **Depends on** โ€” other capabilities that must exist first + +--- + +## 1. Identity and Access Management + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| IAM-001 | Actor Authentication | Authenticate to DCM via configured IdP | โ€” | Register and configure Auth Providers; manage local user store | โ€” | +| IAM-002 | Session Token Management | Receive and use session tokens; token refresh | โ€” | Configure session TTL, failover chain | IAM-001 | +| IAM-003 | Role-Based Access Control | Receive role-appropriate service catalog and API responses | โ€” | Declare role mappings; assign roles to actors | IAM-001 | +| IAM-004 | Group Membership Resolution | Group memberships automatically applied from IdP | โ€” | Map IdP groups to DCM groups; declare group-role relationships | IAM-001, IAM-003 | +| IAM-005 | Multi-Factor Authentication | Satisfy per-session and step-up MFA challenges | โ€” | Configure MFA methods; declare step-up operations | IAM-001 | +| IAM-006 | SCIM Automated Provisioning | Actor created/updated/deprovisioned from IdP automatically | โ€” | Configure SCIM endpoint and attribute mappings | IAM-001 | +| IAM-007 | Tenant Scope Enforcement | Access restricted to authorized Tenants | โ€” | Declare Tenant membership; configure cross-tenant policies | IAM-003, IAM-004 | + +--- + +## 2. Service Catalog + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| CAT-001 | Service Catalog Presentation | Browse available services filtered by RBAC | Declare catalog items for offered resource types | Activate catalog items; configure catalog visibility policies | IAM-003, IAM-007 | +| CAT-002 | Service Schema Discovery | View field schemas, constraints, and edit constraints for a catalog item | Declare field schemas in Resource Type Spec | Configure constraint visibility level per profile | CAT-001 | +| CAT-003 | Catalog Item Search and Filter | Search catalog by keyword, resource type, tag | โ€” | Configure Search Index for catalog | CAT-001 | +| CAT-004 | Catalog Item Versioning | Request a specific version of a catalog item | Publish new catalog item versions following semver | Manage version lifecycle; enforce deprecation timelines | CAT-001 | +| CAT-005 | Cost Estimation | Receive estimated cost before submitting a request | Declare cost metadata on provider registration | Configure Cost Analysis component | CAT-001 | +| CAT-006 | Dependency Visualization | See required dependencies for a catalog item before requesting | Declare dependency graph in Resource Type Spec | โ€” | CAT-001 | +| CAT-007 | Catalog Item Deprecation | Receive deprecation warnings on deprecated catalog items | Declare successor types in deprecation notice | Manage deprecation lifecycle; notify consumers | CAT-004 | + +--- + +## 3. Request Lifecycle Management + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| REQ-001 | Submit Service Request | Submit a resource request via UI, API, or Git PR | โ€” | Configure request ingress surfaces | IAM-007, CAT-001 | +| REQ-002 | Intent State Capture | Request stored as versioned GitOps artifact before processing | โ€” | Configure Intent Store; manage Git repository structure | REQ-001 | +| REQ-003 | Layer Assembly | Request enriched with organizational defaults and context layers | Contribute Service Layers for resource types | Manage Core Layers; configure Layer Cache | REQ-002 | +| REQ-004 | Policy Evaluation | Request validated, transformed, and gated by applicable policies | Contribute provider-specific policies | Manage Policy Engine; configure Policy Groups and Profiles | REQ-003 | +| REQ-005 | Placement Engine Execution | Resource placed with the best available provider instance | Implement capacity reserve_query response | Configure placement constraints; manage provider priorities | REQ-004 | +| REQ-006 | Requested State Persistence | Assembled payload stored as authoritative GitOps record | โ€” | Configure Requested Store; manage storage redundancy | REQ-005 | +| REQ-007 | Provider Dispatch | Request payload delivered to selected provider | Implement Services API to receive DCM payloads | Configure API Gateway and egress | REQ-006 | +| REQ-008 | Request Status Tracking | Monitor request status from submitted through realized | Report realization status back to DCM | Configure observability for request tracking | REQ-007 | +| REQ-009 | Request Cancellation | Cancel a pending request before realization | Handle cancellation payloads | Configure cancellation policies | REQ-002 | +| REQ-010 | Git PR Request Ingress | Submit requests via Git Pull Request with policy dry-run feedback | โ€” | Configure Git Request Watcher; manage repository structure | REQ-001, IAM-001 | + +--- + +## 4. Provider Contract and Realization + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| PRV-001 | Provider Registration | โ€” | Register provider with DCM: declare type, capabilities, sovereignty, cost metadata | Configure Provider Registry; validate sovereignty declarations | IAM-001 | +| PRV-002 | Naturalization | โ€” | Convert DCM unified payload to provider-native format | โ€” | PRV-001, REQ-007 | +| PRV-003 | Realization | โ€” | Execute required actions to provision/configure/change resource | โ€” | PRV-002 | +| PRV-004 | Denaturalization | โ€” | Convert provider-native result back to DCM unified format | โ€” | PRV-003 | +| PRV-005 | Realized State Reporting | โ€” | Report realized payload and status to DCM API Gateway | Configure Realized State Store; manage Event Stream | PRV-004 | +| PRV-006 | Capacity Reporting | โ€” | Respond to reserve_query with current capacity and availability | Configure placement engine; manage capacity confidence | PRV-001 | +| PRV-007 | Provider Health Reporting | โ€” | Expose health check endpoint; report availability | Monitor provider health; configure trust score updates | PRV-001 | +| PRV-008 | Sovereignty Declaration Maintenance | โ€” | Notify DCM when sovereignty data changes within declared SLA | Monitor sovereignty changes; trigger re-evaluation | PRV-001 | +| PRV-009 | Meta Provider Orchestration | โ€” | Compose sub-providers to deliver higher-order services; manage composition visibility | Configure composite provider federation eligibility | PRV-001, PRV-003 | + +--- + +## 5. Resource Lifecycle Management + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| LCM-001 | Resource State Transitions | Trigger lifecycle actions: suspend, resume, decommission | Handle state transition payloads | Configure lifecycle policies; manage state machine | REQ-008 | +| LCM-002 | Post-Realization Field Updates | Update editable fields on realized resources (targeted delta) | Handle delta update payloads; apply partial changes | Configure editable field declarations; manage edit policies | PRV-005 | +| LCM-003 | Resource TTL Management | Declare and extend resource TTLs; receive expiry notifications | Handle TTL-triggered decommission payloads | Configure Lifecycle Constraint Enforcer; manage expiry policies | LCM-001 | +| LCM-004 | Ownership Transfer | Transfer resource ownership to a different Tenant | โ€” | Authorize and execute ownership transfers; record transfer history | IAM-007, LCM-001 | +| LCM-005 | Rehydration | Replay a resource's intent state to a new provider or context | Receive and execute rehydration payloads | Manage rehydration leases; configure auth level requirements | REQ-002, PRV-003 | +| LCM-006 | Billing State Management | โ€” | โ€” | Configure billing state policies; integrate with Cost Analysis | LCM-001 | +| LCM-007 | Resource Decommission | Decommission resources individually or as part of group decommission | Handle decommission payloads; release resources | Manage decommission workflows; coordinate dependency teardown | LCM-001 | + +--- + +## 6. Drift Detection and Remediation + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| DRF-001 | Active Discovery | โ€” | Expose discovery endpoint; respond to interrogation queries | Configure discovery schedules; manage Discovered Store | PRV-005 | +| DRF-002 | Drift Comparison | Receive drift notifications for owned resources | โ€” | Configure drift detection policies; manage comparison logic | DRF-001, PRV-005 | +| DRF-003 | Drift Notification | Receive actionable drift alerts with field-level detail | โ€” | Configure drift notification channels and escalation policies | DRF-002 | +| DRF-004 | Drift Remediation | Approve or reject automatic drift remediation | Execute remediation payloads | Configure remediation policies (revert/update/alert/escalate) | DRF-002, LCM-002 | +| DRF-005 | Unsanctioned Change Detection | Receive alerts on unauthorized resource modifications | Report all external state changes to DCM | Configure unsanctioned change policies | DRF-001 | + +--- + +## 7. Policy Management + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| POL-001 | Policy Authoring | โ€” | Contribute provider-specific policy rules | Author and manage policies in GitOps store | IAM-003 | +| POL-002 | Policy Validation and Shadow Mode | View shadow evaluation results on own requests | โ€” | Configure shadow mode; review shadow results in Validation Store | POL-001 | +| POL-003 | Policy Activation and Review | โ€” | โ€” | Manage policy review periods; authorize policy activation | POL-001, POL-002 | +| POL-004 | Policy Group Management | โ€” | โ€” | Compose Policy Groups; manage profile assignments | POL-003 | +| POL-005 | Profile Management | โ€” | โ€” | Configure deployment profiles; manage compliance domain groups | POL-004 | +| POL-006 | Policy Provider Registration | โ€” | Register Policy Providers; maintain provider in declared mode | Configure Policy Provider trust levels; manage trust elevation workflow | PRV-001, POL-001 | +| POL-007 | Policy Override and Constraint Visibility | View constraint details for service catalog fields | Declare constraint schemas on Resource Type Specs | Configure constraint visibility levels per profile | CAT-002, POL-003 | + +--- + +## 8. Data Layer Management + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| LAY-001 | Core Layer Authoring | โ€” | โ€” | Author and manage Core and Organizational Layers in GitOps | IAM-003 | +| LAY-002 | Service Layer Contribution | โ€” | Contribute Service Layers for offered resource types | Manage layer compatibility declarations | PRV-001, LAY-001 | +| LAY-003 | Layer Cache Management | โ€” | โ€” | Manage Layer Cache synchronization; handle cache invalidation | LAY-001, LAY-002 | +| LAY-004 | Layer Exclusion | Declare layer exclusions on specific requests | โ€” | Configure which layers may be excluded; manage non-excludable declarations | REQ-003 | +| LAY-005 | Layer Versioning and Lifecycle | โ€” | โ€” | Manage layer versions; handle deprecation; enforce immutability | LAY-001 | + +--- + +## 9. Information and Data Integration + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| INF-001 | Information Provider Registration | โ€” | Register Information Provider; declare authority scope and schema | Configure Information Provider Registry; manage authority layers | IAM-001 | +| INF-002 | Information Provider Push | โ€” | Push field value updates to DCM; respond to conflict notifications | Configure ingestion pipeline; manage conflict resolution policies | INF-001 | +| INF-003 | Information Provider Pull / Discovery | โ€” | Expose data query endpoint for DCM pull operations | Configure pull schedules; manage cache TTLs | INF-001 | +| INF-004 | Write-Back | โ€” | Implement write-back endpoint to receive DCM-initiated updates | Configure write-back triggers via policy | INF-001, INF-002 | +| INF-005 | Confidence Score Visibility | View confidence bands on entity field values; query confidence aggregation API | โ€” | Configure confidence scoring formula; manage trust score thresholds | INF-001 | +| INF-006 | Conflict Resolution Management | โ€” | โ€” | Review and resolve contested field values; manage conflict escalation | INF-002 | + +--- + +## 10. Ingestion and Brownfield Management + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ING-001 | Resource Discovery and Ingestion | โ€” | Expose discovery endpoints for brownfield resources | Configure ingestion pipeline; manage __transitional__ Tenant | DRF-001 | +| ING-002 | Ingested Entity Review | โ€” | โ€” | Review ingested entities; resolve conflicts; promote to active Tenants | ING-001 | +| ING-003 | Bulk Promotion | โ€” | โ€” | Execute bulk entity promotions with preview and rollback | ING-002 | +| ING-004 | Catalog Item Association | โ€” | โ€” | Associate ingested entities with Resource Type Specs; create catalog items | ING-002, CAT-001 | + +--- + +## 11. Audit and Compliance + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| AUD-001 | Audit Trail Access | Query audit records for own resources | โ€” | Configure Audit Store; manage retention policies | IAM-003 | +| AUD-002 | Compliance Reporting | โ€” | โ€” | Generate compliance reports; manage report schedules | AUD-001 | +| AUD-003 | Hash Chain Verification | โ€” | โ€” | Run scheduled and on-demand hash chain verification; manage integrity incidents | AUD-001 | +| AUD-004 | Cross-DCM Audit Correlation | โ€” | โ€” | Correlate audit records across DCM instances via correlation_id; authorize cross-DCM pulls | AUD-001, DCM-001 | +| AUD-005 | Audit Record Retention Management | โ€” | โ€” | Configure reference-based retention; manage post-lifecycle retention | AUD-001 | + +--- + +## 12. Observability and Operations + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| OBS-001 | Operational Dashboard | View health and status of own resources | โ€” | Configure and manage observability dashboard | โ€” | +| OBS-002 | Metrics and Telemetry Export | โ€” | Expose resource-level metrics to DCM | Configure observability export; integrate enterprise observability platform | โ€” | +| OBS-003 | Curated Event Stream Subscription | Subscribe to observability event types via Message Bus | โ€” | Configure event stream publication policies; manage subscriber roles | OBS-002 | +| OBS-004 | Alert and Notification Management | Receive resource and policy alerts via declared channels | โ€” | Configure alert routing; manage notification channels and escalation | OBS-001 | +| OBS-005 | Cost Analysis and Attribution | View cost estimates and actuals for owned resources | Provide cost metadata; report utilization | Configure Cost Analysis component; manage cost attribution policies | PRV-006 | + +--- + +## 13. Storage and State Management + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| STO-001 | GitOps Store Management | โ€” | โ€” | Configure and manage Intent and Requested Stores; manage Git repository structure | โ€” | +| STO-002 | Realized State Store Management | โ€” | โ€” | Configure Event Stream and Realized Store; manage retention | PRV-005 | +| STO-003 | Discovered State Store Management | โ€” | โ€” | Configure Discovered Store; manage retention policies per profile | DRF-001 | +| STO-004 | Search Index Management | Use entity and catalog search | โ€” | Configure Search Index; manage rebuild on failure | STO-001 | +| STO-005 | Backup and Recovery | โ€” | โ€” | Configure backup schedules; test recovery procedures | STO-001, STO-002 | +| STO-006 | Provenance Model Configuration | โ€” | โ€” | Select and configure provenance model (full_inline / deduplicated / tiered); manage tier transitions | STO-001 | + +--- + +## 14. DCM Federation and Multi-Instance + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| FED-001 | DCM Provider Registration | Submit requests that are routed to peer DCMs | Register as DCM Provider in peer instances | Configure DCM Provider registrations; manage federation trust | PRV-001, IAM-001 | +| FED-002 | Federation Routing | Requests automatically routed to appropriate Regional/Sovereign DCM | Respond to reserve queries from Hub DCM | Configure federation placement policies; manage sovereignty pre-filters | FED-001, REQ-005 | +| FED-003 | Federation Trust Management | โ€” | โ€” | Manage mTLS certificates; monitor federation trust scores; handle cert rotation | FED-001 | +| FED-004 | Cross-DCM Drift Detection | Receive drift alerts for federated resources | Publish Discovered State events to federation Message Bus | Configure federated drift detection; manage alert-and-hold policies | FED-001, DRF-002 | +| FED-005 | DCM Export and Import | โ€” | โ€” | Export and import DCM state packages; verify import trust scores | STO-001, STO-002 | + +--- + +## 15. Platform Governance and Administration + +| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| GOV-001 | Tenant Management | โ€” | โ€” | Create, configure, and decommission Tenants; manage compliance overlays | IAM-007 | +| GOV-002 | Group Management | โ€” | โ€” | Create and manage DCM Groups; configure sovereignty rules; manage time-bounded memberships | IAM-003 | +| GOV-003 | Registry Management | โ€” | Register and maintain Resource Type Specifications in organization registry | Manage registry sync; configure registry policies; manage Tier 3 types | PRV-001 | +| GOV-004 | Resource Type Lifecycle | โ€” | Manage deprecation notices; declare successor types; maintain migration guidance | Enforce deprecation timelines; manage sunset periods | GOV-003 | +| GOV-005 | Platform Configuration Management | โ€” | โ€” | Manage platform-wide layers; configure profiles; manage deployment manifest | LAY-001, POL-005 | +| GOV-006 | Bootstrap and Self-Hosting | โ€” | โ€” | Manage DCM self-deployment; verify bootstrap manifest; handle repave scenarios | STO-001 | +| GOV-007 | Sovereign Deployment Management | โ€” | โ€” | Manage air-gapped DCM instances; configure signed bundle import; manage offline registry | FED-001, STO-001 | + +--- + +## Capability Count Summary + +| Domain | Capabilities | +|--------|-------------| +| Identity and Access Management | 7 | +| Service Catalog | 7 | +| Request Lifecycle Management | 10 | +| Provider Contract and Realization | 9 | +| Resource Lifecycle Management | 7 | +| Drift Detection and Remediation | 5 | +| Policy Management | 7 | +| Data Layer Management | 5 | +| Information and Data Integration | 6 | +| Ingestion and Brownfield Management | 4 | +| Audit and Compliance | 5 | +| Observability and Operations | 5 | +| Storage and State Management | 6 | +| DCM Federation and Multi-Instance | 5 | +| Platform Governance and Administration | 7 | +| **Total** | **100** | + +--- + +## Dependency Map โ€” Critical Path Capabilities + +These capabilities block the most downstream work and should be implemented first: + +``` +IAM-001 (Actor Authentication) + โ””โ”€โ”€ IAM-002 (Session Tokens) โ†’ IAM-003 (RBAC) โ†’ IAM-007 (Tenant Scope) + โ””โ”€โ”€ CAT-001 (Service Catalog) + โ””โ”€โ”€ REQ-001 (Submit Request) + โ””โ”€โ”€ REQ-002 (Intent State) โ†’ REQ-003 (Layer Assembly) + โ””โ”€โ”€ REQ-004 (Policy Evaluation) โ†’ REQ-005 (Placement) + โ””โ”€โ”€ REQ-007 (Provider Dispatch) + โ””โ”€โ”€ PRV-003 (Realization) โ†’ PRV-005 (Realized State) + โ””โ”€โ”€ LCM-001 (State Transitions) + โ””โ”€โ”€ DRF-001 (Discovery) โ†’ DRF-002 (Drift) + +PRV-001 (Provider Registration) โ€” parallel critical path + โ””โ”€โ”€ PRV-002 (Naturalization) โ†’ PRV-003 (Realization) + โ””โ”€โ”€ PRV-006 (Capacity Reporting) โ†’ REQ-005 (Placement) +``` + +**Minimum viable DCM capability set (to demonstrate end-to-end lifecycle):** + +IAM-001 โ†’ IAM-002 โ†’ IAM-003 โ†’ IAM-007 โ†’ CAT-001 โ†’ REQ-001 โ†’ REQ-002 โ†’ REQ-003 โ†’ REQ-004 โ†’ REQ-005 โ†’ REQ-006 โ†’ REQ-007 โ†’ PRV-001 โ†’ PRV-002 โ†’ PRV-003 โ†’ PRV-004 โ†’ PRV-005 โ†’ LCM-001 โ†’ DRF-001 โ†’ DRF-002 โ†’ AUD-001 + +**21 capabilities for a functional end-to-end demonstration.** + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/capabilities.md b/content/docs/capabilities.md new file mode 100644 index 0000000..4bfe0d4 --- /dev/null +++ b/content/docs/capabilities.md @@ -0,0 +1,67 @@ +--- +title: "Capabilities Map" +type: docs +weight: 8 +--- + +The DCM Capabilities Map is an interactive reference of the 95 foundational capabilities required for DCM to perform lifecycle management. Each capability maps to three perspectives: **Consumer**, **Service Provider**, and **Platform/Admin**. + +## Using the Map + +- **Domain cards** โ€” click any of the 15 domain cards to filter to that capability area; click again to clear +- **Search** โ€” matches against capability IDs, names, descriptions, and dependencies; search `IAM-001` to find all capabilities that depend on authentication +- **Perspective filter** โ€” extract the implementation checklist for a specific role +- **Table / Cards** toggle โ€” dense comparison view or individual capability cards +- **MVP badges** โ€” the 21 capabilities on the minimum viable critical path +- **Dependency tags** โ€” clickable in table view; jumps to that capability +- **CSV download** โ€” built into the map; also available as a [direct download](/capabilities/DCM-Capabilities-Matrix.csv) + +## Capability Domains + +| Prefix | Domain | Capabilities | +|--------|--------|-------------| +| IAM | Identity and Access Management | 7 | +| CAT | Service Catalog | 7 | +| REQ | Request Lifecycle Management | 10 | +| PRV | Provider Contract and Realization | 9 | +| LCM | Resource Lifecycle Management | 7 | +| DRF | Drift Detection and Remediation | 5 | +| POL | Policy Management | 7 | +| LAY | Data Layer Management | 5 | +| INF | Information and Data Integration | 6 | +| ING | Ingestion and Brownfield Management | 4 | +| AUD | Audit and Compliance | 5 | +| OBS | Observability and Operations | 5 | +| STO | Storage and State Management | 6 | +| FED | DCM Federation and Multi-Instance | 5 | +| GOV | Platform Governance and Administration | 7 | + +## MVP Critical Path โ€” 21 Capabilities + +The minimum viable set for an end-to-end DCM lifecycle demonstration: + +``` +IAM-001 โ†’ IAM-002 โ†’ IAM-003 โ†’ IAM-007 โ†’ CAT-001 + โ†’ REQ-001 โ†’ REQ-002 โ†’ REQ-003 โ†’ REQ-004 โ†’ REQ-005 โ†’ REQ-006 โ†’ REQ-007 + โ†’ PRV-001 โ†’ PRV-002 โ†’ PRV-003 โ†’ PRV-004 โ†’ PRV-005 + โ†’ LCM-001 โ†’ DRF-001 โ†’ DRF-002 โ†’ AUD-001 +``` + +## Downloads + +| Format | Use | +|--------|-----| +| [CSV](/capabilities/DCM-Capabilities-Matrix.csv) | Import into Jira, Confluence, Notion, Airtable | +| [Standalone Map](/capabilities/map.html) | Full-page interactive map | +| [Markdown Reference](/docs/capabilities-matrix) | Full matrix with dependency map | + +## Interactive Map + +
+ +
diff --git a/content/docs/data-model/_index.md b/content/docs/data-model/_index.md new file mode 100644 index 0000000..8897ac6 --- /dev/null +++ b/content/docs/data-model/_index.md @@ -0,0 +1,27 @@ +--- +title: Data Model +type: docs +weight: 1 +sidebar: + open: true +--- + +# DCM Data Model + +The DCM Data Model is the foundational layer that governs how all data in DCM is +represented, versioned, assembled, and governed. It defines the structure of every +resource, relationship, policy, and state record across the entire DCM lifecycle. + +## Documents + +- **[Context and Purpose](context-and-purpose/)** โ€” Why the data model exists, core principles, provenance, and the artifact metadata standard. +- **[Four States](four-states/)** โ€” The four-state model (Intent, Requested, Realized, Discovered), rehydration, and CI/CD integration. +- **[Data Layers and Assembly](layering-and-versioning/)** โ€” Data layers, assembly process, override control, Layer Domain/Handle/Priority, and artifact metadata. +- **[Resource Type Hierarchy](resource-type-hierarchy/)** โ€” The Resource Type Registry, catalog model, and provider hierarchy. +- **[Resource/Service Entities](resource-service-entities/)** โ€” Entity definitions, ownership models, and lifecycle. +- **[Service Dependencies](service-dependencies/)** โ€” Rehydration ordering and failure handling. +- **[Resource Grouping](resource-grouping/)** โ€” Tenant model, Resource Groups, and grouping model. +- **[Entity Relationships](entity-relationships/)** โ€” Universal relationship model for all entity connections. +- **[Information Providers](information-providers/)** โ€” External data providers and the information type registry. +- **[Storage Providers](storage-providers/)** โ€” Storage provider contracts for all DCM stores. +- **[Audit, Provenance, and Observability](audit-provenance-observability/)** โ€” The three distinct concerns of audit, provenance, and observability. diff --git a/content/docs/data-model/audit-provenance-observability.md b/content/docs/data-model/audit-provenance-observability.md new file mode 100644 index 0000000..66f8c01 --- /dev/null +++ b/content/docs/data-model/audit-provenance-observability.md @@ -0,0 +1,361 @@ +--- +title: Audit, Provenance, and Observability +type: docs +weight: 11 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Four States](02-four-states.md) | [Storage Providers](11-storage-providers.md) | [Context and Purpose](00-context-and-purpose.md) + +--- + +## 1. Purpose + +Audit, Provenance, and Observability are three distinct but related capabilities in DCM. They are often conflated โ€” this document separates them precisely, defines their relationship, and establishes the architectural model for each. + +| Capability | Question Answered | Audience | Time Orientation | +|------------|------------------|----------|-----------------| +| **Provenance** | Where did this data come from and how did it change? | System โ€” embedded in data | Embedded in every payload | +| **Audit** | What happened, who authorized it, can you prove it? | Auditors, Compliance, Security | Backward-looking | +| **Observability** | Is the system healthy and performing within expectations? | SRE, Platform Engineers | Forward-looking, real-time | + +--- + +## 2. Provenance + +### 2.1 Definition + +Provenance is the structural data lineage mechanism embedded in every field of every DCM payload. It is not a separate system โ€” it is part of the data itself. Every field that can be created or modified by any DCM process carries provenance metadata alongside its value. + +Provenance answers: "where did this value come from, what modified it, and why?" + +Audit queries provenance to answer its questions. Observability does not use provenance directly โ€” it operates on event streams and metrics. + +### 2.2 Provenance Structure + +See [Context and Purpose โ€” Section 4.4](00-context-and-purpose.md) for the complete field-level provenance structure. The key elements: + +```yaml +field_name: + value: + metadata: + override: + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_uuid: + operation_type: + actor_uuid: + timestamp: + reason: +``` + +### 2.3 Provenance Obligations + +Every DCM component that modifies data carries a provenance obligation โ€” it must record its UUID, operation type, actor, timestamp, and reason for every field it touches. A component that modifies data without recording provenance violates the data model contract. + +| Component | Provenance Obligation | +|-----------|----------------------| +| Request Payload Processor | Record source UUID and type for every field assembled from layers | +| Policy Engine | Record policy UUID, level, operation type, and reason for every field modified or locked | +| Service Provider (Denaturalization) | Record provider UUID and timestamp for every field in the realized payload | +| Storage Provider | Emit provenance event to Audit component on every write | +| Resource Discovery | Record provider UUID, timestamp, and method for every discovered field | +| Rehydration Pipeline | Record source store, source record UUID, rehydration reason, and actor UUID | + +### 2.4 Provenance Across the Full Lifecycle + +The provenance chain for a single field may span multiple lifecycle stages: + +``` +Base Layer sets encryption_standard: AES-128 + origin: {source_type: base_layer, source_uuid: layer-uuid-001} + +Transformation Policy enriches to AES-256 + modification: {source_uuid: policy-uuid-001, operation: transformation, + reason: "Security standard requires AES-256 minimum"} + +GateKeeper Policy locks as immutable + modification: {source_uuid: policy-uuid-002, operation: lock, + reason: "CISO mandate โ€” encryption standard non-negotiable"} + +Provider reports realized value: AES-256 + modification: {source_uuid: provider-uuid-001, operation: denaturalization, + reason: "Provider confirmed encryption standard applied"} + +Drift detected: discovered value AES-128 + modification: {source_uuid: discovery-uuid-001, operation: discovery, + reason: "Direct modification detected outside DCM lifecycle"} +``` + +The complete chain tells the full story of that field across its entire existence. + +--- + +## 3. Audit + +### 3.1 Definition + +Audit is the compliance-grade, queryable record of all significant actions across the DCM lifecycle. It is backward-looking, human-readable, and access-controlled by persona. It answers: "what happened, who authorized it, can you prove it?" + +Audit is a **separate component** โ€” not a query against the GitOps stores, not a view into provenance directly. It aggregates and indexes provenance events from all stores and presents them through a structured query API surfaced by the DCM API Gateway. + +### 3.2 Architecture + +``` +All Storage Providers emit provenance events (contractual obligation) + โ”‚ + โ”‚ Events include: entity_uuid, operation, actor_uuid, + โ”‚ timestamp, payload_hash, store_reference + โ–ผ +Audit Component + โ”‚ Receives provenance events from all stores + โ”‚ Correlates events by entity_uuid across all stores + โ”‚ Indexes for structured query: by entity, tenant, actor, + โ”‚ time range, operation type, policy UUID + โ”‚ Maintains immutable records โ€” audit records are never modified + โ”‚ Enforces long retention (regulatory periods โ€” configurable, + โ”‚ minimum 7 years for FSI deployments) + โ”‚ Verifies payload hashes โ€” detects store tampering + โ–ผ +DCM API Gateway + โ”‚ Surfaces Audit query API with persona-based access control + โ”‚ Auditor: full access โ€” all entities, all tenants, all time + โ”‚ SRE: full access within operational scope + โ”‚ Admin: full access within administrative scope + โ”‚ Consumer: own entities and requests only + โ”‚ Provider: own provider's operations only +``` + +### 3.3 Audit API (via DCM API Gateway) + +``` +GET /api/v1/audit/entities/{uuid}/history + Returns: complete lifecycle history for an entity + Fields: all state transitions, all provenance events, all actor actions + +GET /api/v1/audit/requests/{uuid}/provenance + Returns: complete provenance chain for a specific request + Fields: intent, assembly, policy evaluation, provider dispatch, realization + +GET /api/v1/audit/policies/{uuid}/evaluations + Returns: all evaluations of a specific policy across all requests + Fields: when it ran, what it did, which entities it affected + +GET /api/v1/audit/actors/{uuid}/activity + Returns: all actions taken by a specific actor + Fields: requests submitted, approvals given, policy evaluations triggered + +GET /api/v1/audit/tenants/{uuid}/activity + Returns: all activity within a specific tenant + Fields: requests, realizations, drift events, policy violations + +POST /api/v1/audit/query + Body: structured audit query with field filters, time range, pagination + Returns: matching audit records +``` + +### 3.4 Audit Record Structure + +```yaml +audit_record: + audit_uuid: + entity_uuid: + tenant_uuid: + event_type: + timestamp: + actor_uuid: + actor_type: + + source_store: + store_type: + store_uuid: + store_reference: + payload_hash: + + provenance_summary: + + + policy_context: + policies_evaluated: [] + policies_applied: [] + policies_rejected: [] + override_control_changes: [] + + related_records: + intent_record_uuid: + requested_record_uuid: + realized_event_uuid: + rehydration_source_uuid: +``` + +### 3.5 Audit Integrity + +Audit records are immutable. The Audit component verifies payload hashes against the Storage Provider's stored values on every read โ€” if a hash mismatch is detected, the Audit component flags the record as potentially tampered and escalates to the Policy Engine. + +The Audit Store itself is a Storage Provider with the highest consistency and durability requirements โ€” linearizable consistency, synchronous replication, cryptographic payload hashing, and compliance-grade retention. + +--- + +## 4. Observability + +### 4.1 Definition + +Observability is real-time insight into the health, performance, and behavior of the DCM system. It is forward-looking, machine-readable, and aggregated. It answers: "is the system healthy, where are the bottlenecks, what is the error rate?" + +Observability is operationally oriented โ€” SREs and platform engineers use it to understand system behavior and respond to incidents. It does not carry the compliance obligations of Audit. + +### 4.2 The Three Pillars + +**Metrics** โ€” quantitative measurements of system state over time +- Request throughput: requests/second by resource type, tenant, provider +- Latency: assembly time, policy evaluation time, provider dispatch time, end-to-end time +- Error rates: policy rejection rate, provider failure rate, drift detection rate +- Capacity: provider utilization, store capacity, queue depth +- Cost: accumulated cost by tenant, resource type, provider + +**Traces** โ€” distributed traces of request execution across components +- Full request trace from Intent State creation through provider dispatch +- Policy evaluation trace โ€” which policies ran, in what order, how long each took +- Assembly trace โ€” which layers were applied, in what order, what each contributed + +**Logs** โ€” structured event logs from all DCM components +- Component startup and shutdown +- Registration events (provider registration, deregistration) +- Error conditions +- Drift detection events +- Unsanctioned change events + +### 4.3 Architecture + +``` +DCM components emit metrics, traces, and logs + โ”‚ + โ”‚ All telemetry in OpenTelemetry format + โ”‚ Standardized metric names, trace context propagation, + โ”‚ structured log format + โ–ผ +Observability Store + โ”‚ Time-series metrics store (Prometheus-compatible) + โ”‚ Distributed trace store (Jaeger/Zipkin compatible) + โ”‚ Log aggregation (structured, indexed) + โ”‚ Short-to-medium retention (configurable โ€” typically 90 days) + โ–ผ +DCM API Gateway + โ”‚ GET /api/v1/observability/metrics + โ”‚ GET /api/v1/observability/traces/{request_uuid} + โ”‚ GET /api/v1/observability/health + โ”‚ GET /api/v1/observability/providers/{uuid}/performance + โ–ผ +Dashboards and alerting (external tooling) + โ”‚ Grafana, DataDog, Splunk โ€” implementor choice + โ”‚ DCM provides OpenTelemetry-compatible telemetry + โ”‚ Dashboards are deployment artifacts, not DCM artifacts +``` + +### 4.4 Standard DCM Metrics + +``` +# Request lifecycle +dcm_requests_total{resource_type, tenant, status} +dcm_request_duration_seconds{resource_type, stage} +dcm_requests_in_flight{resource_type, tenant} + +# Policy Engine +dcm_policy_evaluations_total{policy_type, result} +dcm_policy_evaluation_duration_seconds{policy_type} +dcm_policy_rejections_total{policy_uuid, resource_type} + +# Provider +dcm_provider_requests_total{provider_uuid, resource_type, status} +dcm_provider_response_duration_seconds{provider_uuid} +dcm_provider_capacity_available{provider_uuid, resource_type} +dcm_provider_health_status{provider_uuid} + +# Drift +dcm_drift_detections_total{resource_type, severity} +dcm_unsanctioned_changes_total{resource_type, provider_uuid} +dcm_drift_resolution_duration_seconds{resolution_type} + +# Storage +dcm_store_write_duration_seconds{store_type, store_uuid} +dcm_store_read_duration_seconds{store_type, store_uuid} +dcm_store_health_status{store_type, store_uuid} + +# Rehydration +dcm_rehydrations_total{source_store, placement_mode, policy_version} +dcm_rehydration_duration_seconds{source_store} +``` + +### 4.5 Observability vs Audit โ€” The Key Distinctions + +| Dimension | Audit | Observability | +|-----------|-------|---------------| +| **Retention** | Regulatory period (years) | Operational window (days-months) | +| **Access control** | Strict persona-based | Operational teams | +| **Data volume** | Moderate โ€” per-entity events | High โ€” continuous time series | +| **Query model** | Structured, entity-centric | Aggregated, time-series | +| **Immutability** | Absolute โ€” records never modified | Aggregated data may be downsampled | +| **Compliance** | Compliance-grade โ€” hash-verified | Operational โ€” best effort | +| **Use case** | Prove what happened | Understand what is happening | + +--- + +## 5. The API Gateway โ€” Unified Access + +All three capabilities โ€” Provenance (embedded in data), Audit (structured history), and Observability (operational telemetry) โ€” are surfaced through the DCM API Gateway. There is no separate endpoint for audit or observability. All DCM capabilities live in a unified API hierarchy. + +``` +DCM API Gateway + โ”‚ + โ”œโ”€โ”€ /api/v1/catalog/ # Service Catalog + โ”œโ”€โ”€ /api/v1/requests/ # Request submission and management + โ”œโ”€โ”€ /api/v1/entities/ # Entity lifecycle management + โ”œโ”€โ”€ /api/v1/providers/ # Provider registration and management + โ”œโ”€โ”€ /api/v1/policies/ # Policy management + โ”œโ”€โ”€ /api/v1/audit/ # Audit queries + โ”œโ”€โ”€ /api/v1/observability/ # Operational metrics and traces + โ””โ”€โ”€ /api/v1/admin/ # Administrative functions +``` + +Persona-based access control is enforced at the API Gateway level for all endpoints. The same authentication and authorization model applies across the entire API surface. + +--- + +## 6. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the Audit Store be a specialized Storage Provider or can a general Event Stream Store satisfy the audit contract? | Architecture | โ“ Unresolved | +| 2 | How are audit records replicated across sites in air-gapped or geographically distributed deployments? | Sovereignty | โ“ Unresolved | +| 3 | Should DCM provide a default observability dashboard or only the telemetry? | Deployment | โ“ Unresolved | +| 4 | How does the Audit component handle provenance events from a Storage Provider that has been deregistered? | Operational | โ“ Unresolved | + +--- + +## 7. Related Concepts + +- **Provenance** โ€” field-level data lineage embedded in every DCM payload +- **Audit Store** โ€” compliance-grade, immutable store of all audit records +- **Observability Store** โ€” time-series metrics, traces, and logs +- **Storage Provider** โ€” formal provider type for all DCM stores +- **API Gateway** โ€” unified access point for all DCM capabilities including audit and observability +- **Drift Detection** โ€” uses discovered vs realized state comparison; drift events feed the Audit component +- **Unsanctioned Change** โ€” a specific audit event type triggered by unauthorized resource modification + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/context-and-purpose.md b/content/docs/data-model/context-and-purpose.md new file mode 100644 index 0000000..bce132d --- /dev/null +++ b/content/docs/data-model/context-and-purpose.md @@ -0,0 +1,367 @@ +--- +title: Context and Purpose +type: docs +weight: 1 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) + +--- + +## 1. Purpose + +The DCM Data Model is the foundational layer upon which the entire DCM framework operates. It is not a storage mechanism or a database schema โ€” it is the **lingua franca of DCM**. Every component in the DCM architecture communicates through the data model in some form, whether reading, writing, validating, enriching, transforming, or comparing data. + +The data model exists to solve a problem that is endemic to enterprise IT: **there is no single, trustworthy, consistent representation of infrastructure state**. Tools proliferate, CMDBs diverge, and the result is that no one knows with confidence what exists, what was requested, what was provisioned, or whether the current state matches the intended state. + +The DCM Data Model establishes a **unified, versioned, declarative single source of truth** for all infrastructure state across the full lifecycle of every resource DCM manages. + +--- + +## 2. Role in the DCM Architecture + +The data model is not owned by any single component โ€” it is the contract between all components. Every major DCM capability acts on data in a specific and well-defined way: + +| Component | Relationship to Data | +|-----------|---------------------| +| Request Payload Processor | Assembles and enriches data into a complete request payload | +| Policy Engine | Reads, validates, transforms, and gates data based on policy definitions | +| Service Provider | Consumes data (via Naturalization) and returns data (via Denaturalization) | +| Orchestration | Coordinates component interactions based on data state and dependencies | +| Audit | Records data at every state transition for compliance evidence | +| Drift Reconciliation | Compares versions of data across states to detect and remediate drift | +| Cost Analysis | Derives cost information from data throughout the resource lifecycle | +| Resource Discovery | Produces data representing the current discovered state of resources | +| IDM / IAM | Gates access to data and operations based on identity and role | +| Service Catalog | Exposes available services based on data definitions and RBAC policy | + +This means the data model is effectively the **API between all DCM components** โ€” even components that do not communicate directly are coupled through the data model. A well-designed data model makes every component easier to build, test, and evolve independently. + +--- + +## 3. Universal Identity Requirement + +Every data object in DCM must have a **UUID (Universally Unique Identifier)**. This is not optional โ€” it is a foundational requirement that applies to every entity in the data model without exception. + +UUIDs serve several critical functions: + +- **Unambiguous reference** โ€” any component, policy, layer, catalog item, or process that touches a data object can reference it precisely and without ambiguity +- **Provenance anchoring** โ€” every change recorded in a data object's lineage references the UUID of the entity that caused the change +- **Dependency mapping** โ€” relationships between resources, services, and components are expressed as UUID references, never by name alone +- **Audit fidelity** โ€” audit records reference UUIDs, ensuring that even if names or labels change, the audit trail remains accurate and traceable +- **Cross-state correlation** โ€” the same resource across Intent, Requested, Realized, and Discovered states can be correlated via UUID chains + +This applies to all entities including but not limited to: resource definitions, catalog items, data layers, policies, policy sets, components, service providers, consumers, and requests. + +--- + +## 4. Field-Level Provenance and Data Lineage + +One of the most critical requirements of the DCM Data Model is the ability to trace the complete lineage of any piece of data at any stage of the pipeline. This is not a logging concern โ€” it is a **structural requirement of the data model itself**. + +### 4.1 The Requirement + +At any point in the DCM pipeline, for any field in any data object, it must be possible to answer: + +- What is the current value of this field? +- Where did this value originate? (catalog item, base layer, intermediate layer, policy, consumer input, discovery) +- Has this value been modified since origination? +- If modified: + - What is the complete history of modifications? + - Which entity caused each modification? (identified by UUID) + - What type of entity caused it? (policy, layer, component, provider) + - When did each modification occur? + - What was the value before each modification? + - Why was the modification made? (enrichment, validation, transformation, gatekeeping) +- What is the complete chain of custody of this field from origin to current value? + +### 4.2 Why Field-Level Lineage Matters + +Document-level versioning alone is insufficient for DCM's requirements. Consider a resource request flowing through the pipeline: + +1. Consumer selects a catalog item โ€” catalog item UUID recorded +2. Base resource definition layer applied โ€” base layer UUID recorded, fields established +3. Intermediate layers applied โ€” each layer UUID recorded, field overrides recorded +4. Policy Engine validates โ€” policy UUID recorded, validation outcome recorded +5. Policy Engine enriches โ€” policy UUID recorded, enriched field values and their source recorded +6. Policy Engine transforms โ€” policy UUID recorded, transformation recorded with before/after values +7. Request payload submitted โ€” complete provenance chain intact across all fields + +Without field-level provenance, it is impossible to determine after the fact whether a specific field value came from a consumer request, a data layer, a security policy, or a business rule. This ambiguity is unacceptable in a governed, auditable system. + +### 4.3 Provenance as a Structural Element + +Field-level provenance must be carried within the data object itself โ€” not in an external log. This ensures that: + +- The data and its lineage are always co-located and cannot be separated +- Any consumer of the data can inspect its lineage without querying an external system +- Provenance survives data export, migration, and portability scenarios +- The audit capability reads provenance that is intrinsic to the data, not reconstructed from logs + +### 4.4 Provenance Metadata Structure + +Every field that can be created or modified by any DCM process carries provenance metadata alongside its value. The conceptual structure is: + +```yaml +field_name: + value: + metadata: + # Simple override declaration (Level 2) โ€” most fields only need this + override: + # OR matrix declaration (Level 3) โ€” for actor-specific governance + override_matrix: + default: + actors: + trusted_grants: + + # Always present regardless of level + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + constraint_schema: + + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_type: + source_uuid: + operation_type: + actor: + timestamp: + reason: +``` + +**Note:** The `metadata` block is set exclusively by the Policy Engine. Data layers and the Request Payload Processor never set override control. `operation_type: lock` is used when a GateKeeper policy sets `override: immutable`. `operation_type: grant` is used when a trusted_grant is issued. The three levels of override control are: Level 1 (no declaration โ€” fully overridable), Level 2 (simple `override:` property), Level 3 (full `override_matrix:` with per-actor permissions). See the Layering and Assembly document Section 5a for the complete model. + +### 4.5 Provenance Obligations + +Every DCM component that reads and modifies data carries a provenance obligation: + +| Component | Provenance Obligation | +|-----------|----------------------| +| Request Payload Processor | Record source UUID and type for every field assembled from layers and catalog items | +| Policy Engine | Record policy UUID, operation type, and reason for every field it enriches, transforms, or validates | +| Service Provider (Denaturalization) | Record provider UUID and timestamp for every field returned in the realized payload | +| Resource Discovery | Record provider UUID, discovery timestamp, and interrogation method for every field in the discovered payload | +| Data Layers | Each layer must declare its UUID so downstream provenance records can reference it | +| Catalog Items | Each catalog item must declare its UUID so downstream provenance records can reference it | + +Provenance recording is **not optional** for any component that modifies data. A component that modifies data without recording provenance violates the data model contract. + +### 4.6 Relationship to Audit + +The Audit capability in DCM reads provenance data that is intrinsic to every data object. This means: + +- Audit does not reconstruct history from logs โ€” it reads lineage that was recorded at the point of change +- Any data object can be audited at any time, in any state, by any authorized persona +- The audit trail is as durable and immutable as the data itself +- Compliance evidence is produced from the data, not from a separate audit system that could diverge from the data + +--- + +## 5. Foundational Constraints + +The DCM Data Model is governed by three foundational constraints that apply universally and without exception: + +### 3.1 Declarative + +Data in DCM describes **what something is or should be**, not how to achieve it. Every entity in the data model is a complete, self-describing statement of state. The procedures required to achieve that state are the concern of the Service Provider, not the data model. + +This means: +- A resource definition declares its desired configuration, not the steps to configure it +- A policy declares its conditions and outcomes, not its execution logic +- A layer declares its overrides, not the merge algorithm used to apply them + +### 3.2 Idempotent in Operation + +Applying the same data to the same system multiple times must always produce the same result. No operation on DCM data should have different outcomes based on how many times it has been applied. + +This is critical for: +- **Drift reconciliation** โ€” reapplying desired state to a drifted resource must produce correct results +- **DC rehydration** โ€” replaying the full set of declared states must reconstruct the environment correctly +- **Retry scenarios** โ€” failed operations can be safely retried without risk of inconsistent state +- **Audit and compliance** โ€” the same data, applied by anyone at any time, produces the same verifiable outcome + +### 3.3 Immutable if Versioned + +Once a version of any entity is published, it cannot be modified. If a change is required, a new version must be created. The previous version remains intact and accessible. + +This constraint is what makes the following capabilities trustworthy: +- **Audit trails** โ€” every state at every point in time is preserved and verifiable +- **Drift detection** โ€” comparison between states is meaningful because neither state can change retroactively +- **Intent portability** โ€” a previously declared intent can be replayed against current policies with confidence that the original intent is unchanged +- **Rollback** โ€” reverting to a previous version is always possible because previous versions are never destroyed +- **Chain of trust** โ€” the provenance of any configuration can be traced through an unbroken chain of immutable versions + +--- + +## 5a. Artifact Metadata Standard + +Every DCM artifact โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions, and all other defined or stored objects โ€” carries a universal **Artifact Metadata** block. This is a structural requirement that applies to all artifacts without exception. + +### The Purpose + +Artifact metadata answers: **who created this, when, who owns it, what changed, and how do we contact them?** It is the identity and accountability record for the artifact itself โ€” distinct from field-level provenance which tracks data value lineage. + +### The Five Artifact Statuses + +All DCM artifacts follow a five-status lifecycle: + +| Status | Meaning | Key Behavior | +|--------|---------|-------------| +| `developing` | In active development | Dev mode / dev pipeline only. Not applied in production. | +| `proposed` | Submitted for validation | Shadow execution for policies โ€” output captured, not applied. In PR review for data artifacts. | +| `active` | Live and governing | Applied to all relevant requests. | +| `deprecated` | Being phased out | Still works, replacement available, warning on use. | +| `retired` | End of life | Cannot be used. Terminal status. | + +### Key Design Decisions + +**created_by vs owned_by:** Deliberately separate. The creator is the audit record โ€” who physically submitted the artifact. The owner is the accountability record โ€” who is responsible and receives notifications for conflicts, deprecation warnings, and policy violations. + +**Contact info โ€” two modes:** When an Identity Provider is registered, the `uuid` field links to the IdP record and `display_name` is a non-authoritative display cache. In standalone/air-gapped mode, `uuid` is absent and `display_name` + `email` are the primary identity fields. Both modes are fully supported. + +**created_via:** Declares the ingestion path โ€” `pr` (full GitOps review history), `api` (direct submission), `migration` (imported, limited provenance), `system` (DCM-created). Makes audit quality transparent. + +**Proposed shadow execution:** Policy artifacts in `proposed` status execute in shadow mode against real traffic โ€” output is captured and reported but never applied. Enables safe validation before activation. + +See [Data Layers and Assembly โ€” Section 4b](03-layering-and-versioning.md) for the complete artifact metadata structure and all field definitions. + +--- + +## 6. The Four States + +DCM tracks the lifecycle of every resource through four distinct states. Together, these four states provide complete visibility into the gap between what was wanted, what was asked for, what was built, and what actually exists. + +### 6.1 Intent State + +The **Intent State** represents what a consumer wants to happen. It is the declared desire, captured at the moment a consumer initiates a request, before any processing, validation, or enrichment has occurred. + +- **When it is created:** When a consumer submits a request via the Web UI or Consumer API +- **Where it is stored:** Intent Store +- **Key characteristic:** Captures the consumer's raw intent โ€” what they asked for in their own terms +- **Primary use:** Source for Intent Portability โ€” replaying an intent through current policies to produce a new request for a different environment or provider + +### 6.2 Requested State + +The **Requested State** represents the fully processed, policy-validated, and enriched payload that has been submitted to a Service Provider for execution. It is the output of the Request Payload Processor after all policies have been applied and all data layers have been merged. + +- **When it is created:** When the Request Payload Processor completes processing and submits to the API Gateway +- **Where it is stored:** Request Store +- **Key characteristic:** Represents a complete, validated, provider-ready declaration of desired state +- **Primary use:** Record of what was formally requested; input to audit and drift processes + +### 6.3 Realized State + +The **Realized State** represents what was actually provisioned or executed by a Service Provider, returned to DCM in unified data model format via Denaturalization. It is the ground truth of what was built. + +- **When it is created:** When a Service Provider completes execution and returns the realized payload to the API Gateway +- **Where it is stored:** Realized Store +- **Key characteristic:** Must be a complete representation of the provisioned resource in DCM unified format โ€” not a status code, but a full state description +- **Primary use:** Baseline for drift detection; source of truth for audit and reporting; input to cost analysis + +### 6.4 Discovered State + +The **Discovered State** represents what actually exists in the environment as interrogated by a Service Provider during a discovery operation. It is an independent observation of reality, not derived from any previous DCM state. + +- **When it is created:** When a Service Provider completes a discovery cycle and returns the discovered payload to DCM +- **Where it is stored:** Discovered Store +- **Key characteristic:** Produced independently of the Realized State โ€” it is what is actually there, regardless of what was supposed to be there +- **Primary use:** Drift detection (compared against Realized State); brownfield ingestion (pathway to lifecycle ownership of unmanaged resources) + +### 6.5 State Relationships and Lifecycle Flow + +The four states relate to each other as follows: + +``` +Consumer Request + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ INTENT โ”‚ โ—„โ”€โ”€ What the consumer wants +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Policy Engine processes, enriches, validates + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REQUESTED โ”‚ โ—„โ”€โ”€ What was formally submitted to the provider +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Service Provider executes + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REALIZED โ”‚ โ—„โ”€โ”€ What was actually built +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ Compare โ—„โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”‚ DISCOVERED โ”‚ โ—„โ”€โ”€ What actually exists now + โ”‚ โ”‚ STATE โ”‚ + โ–ผ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + Drift Detection +``` + +**Key operations across states:** +- **Drift Detection:** Discovered State vs. Realized State +- **Request Validation:** Requested State vs. Policy definitions +- **Intent Portability:** Intent State โ†’ re-process through current policies โ†’ new Requested State +- **Brownfield Ingestion:** Discovered State โ†’ enrichment โ†’ Realized State (lifecycle ownership) + +--- + +## 7. Data as the Provider Contract Boundary + +The data model defines the boundary between DCM and its Service Providers. DCM is explicitly **not concerned with how a provider accomplishes its work** โ€” only with the data that crosses the boundary in both directions. + +This means: +- Providers are interchangeable as long as they honor the data contract +- New providers can be added without changing DCM's core data model +- Provider implementation can evolve independently of DCM +- The contract is enforced at the data level โ€” conformant data in, conformant data out + +The two mechanisms that enforce this boundary are: +- **Naturalization** โ€” the provider's responsibility to transform DCM unified data into its own tool-specific format for execution +- **Denaturalization** โ€” the provider's responsibility to transform its tool-specific result data back into DCM unified format for return to the control plane + +This separation of concerns is what makes DCM technology-agnostic while maintaining a consistent and trustworthy data model across all providers. + +--- + +## 8. Open Questions + +The following questions remain unresolved and require decisions before the data model specification can be considered complete: + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Where should data caches live? (Shore, Ship, Enclave, all?) | Cache architecture, latency, sovereignty | โ“ Unresolved | +| 2 | Should cache synchronization be push, pull, or both? | Consistency model, network requirements | โ“ Unresolved | +| 3 | Which cache is authoritative when caches diverge? | Conflict resolution, drift detection | โ“ Unresolved | +| 4 | What mechanism maintains consistency across distributed caches? | Data integrity, operational complexity | โ“ Unresolved | +| 5 | Should the data model allow embedded target-technology-specific data bundles? | Portability vs. practicality | โ“ Unresolved | +| 6 | How are the four states represented physically? (fields, documents, separate stores?) | Implementation architecture | โ“ Unresolved | +| 7 | What is the performance impact of field-level provenance at scale? What optimization strategies are acceptable? | Scalability, storage cost | โ“ Unresolved | +| 8 | Should provenance metadata be stored inline with field data or in a linked provenance document? | Data model structure, query performance | โ“ Unresolved | + +--- + +## 9. Related Concepts + +- **Sovereign Execution Posture** โ€” the target end state the data model enables by providing a verified, auditable chain of custody through the full resource lifecycle +- **CMDB Replacement** โ€” DCM's four-state model is intended to replace the fragmented multi-CMDB problem by becoming the singular resource domain +- **GitOps** โ€” all entities in the data model are stored in Git, enabling version control, change tracking, and standard software lifecycle practices +- **Data Lineage** โ€” the complete chain of custody of any field value from its origin through every modification, recorded within the data object itself +- **Field-Level Provenance** โ€” the structural mechanism by which data lineage is captured, carried, and made available for audit and compliance purposes +- **UUID** โ€” the universal identity mechanism that makes provenance references, dependency mapping, and cross-state correlation unambiguous and durable + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/entity-relationships.md b/content/docs/data-model/entity-relationships.md new file mode 100644 index 0000000..5db7abf --- /dev/null +++ b/content/docs/data-model/entity-relationships.md @@ -0,0 +1,465 @@ +--- +title: Entity Relationships +type: docs +weight: 8 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) | [Information Providers](10-information-providers.md) + +--- + +## 1. Purpose + +The DCM Entity Relationship model is the **universal mechanism for expressing relationships between any two entities in DCM** โ€” whether between two Resource/Service Entities, between an entity and external business data, or between entities at the service definition level. + +A single relationship model is used everywhere. There is no separate binding mechanism for storage, no separate dependency graph structure, no separate business data association mechanism. One model serves all relationship types across the full lifecycle โ€” from pre-realization planning through to post-realization management, drift detection, cost rollup, and rehydration. + +This document supersedes the dependency graph concept from the Service Dependencies document for data structure purposes. The Service Dependencies document retains content on rehydration ordering and failure handling, which operate on the relationship graph defined here. + +--- + +## 2. Design Principle + +**Single model. Minimum variance. Simple by default.** + +The worst outcome is a data model with different mechanisms for expressing similar concepts. Every relationship in DCM โ€” whether a VM requires storage, an application contains a web server, or a resource references a Business Unit โ€” is expressed using the same structure. The only things that vary are the relationship type, role, and nature โ€” all of which are declared fields, not structural differences. + +--- + +## 3. The Universal Relationship Structure + +Every relationship is a first-class data object with its own UUID. It is recorded **bidirectionally** โ€” on both participating entities. The same `relationship_uuid` appears on both sides, identifying the relationship itself. + +### 3.1 Relationship Record Structure + +```yaml +relationship: + relationship_uuid: + + # This entity's perspective + this_entity_uuid: + this_role: + + # The related entity + related_entity_uuid: + related_entity_type: + related_entity_role: + + # For external entities only + information_provider_uuid: + information_type: + lookup_method: + + # Relationship semantics + relationship_type: + nature: + + # Lifecycle policy โ€” for constituent and operational relationships only + lifecycle_policy: + on_related_destroy: + on_related_suspend: + on_related_modify: + + # Metadata + version: + status: + created_timestamp: + created_by_uuid: + + provenance: + +``` + +### 3.2 Bidirectional Recording + +Every relationship is recorded on both participating entities. The `relationship_uuid` is identical on both sides โ€” it identifies the relationship itself, not one side of it. + +**Example โ€” VM requires Storage:** + +```yaml +# On the VM Entity +relationships: + - relationship_uuid: "rel-uuid-001" + this_entity_uuid: "vm-uuid-001" + this_role: compute + related_entity_uuid: "storage-uuid-001" + related_entity_type: internal + related_entity_role: storage + relationship_type: requires + nature: constituent + lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + on_related_modify: notify + +# On the Storage Entity +relationships: + - relationship_uuid: "rel-uuid-001" + this_entity_uuid: "storage-uuid-001" + this_role: storage + related_entity_uuid: "vm-uuid-001" + related_entity_type: internal + related_entity_role: compute + relationship_type: required_by + nature: constituent + lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + on_related_modify: notify +``` + +--- + +## 4. Relationship Types + +Relationship types form a fixed standard vocabulary. Every type has an inverse โ€” when you record the relationship on both entities, the type is expressed from each entity's perspective. + +| Type | Inverse | Meaning | +|------|---------|---------| +| `requires` | `required_by` | This entity cannot function without the related entity | +| `depends_on` | `dependency_of` | This entity uses the related entity but can degrade without it | +| `contains` | `contained_by` | This entity is a logical container for the related entity | +| `references` | `referenced_by` | This entity references the related entity without owning or requiring it | +| `peer` | `peer` | Equal relationship โ€” neither owns, requires, or contains the other | +| `manages` | `managed_by` | This entity has lifecycle management authority over the related entity | + +--- + +## 5. Relationship Roles + +Roles describe the **function** a related entity serves in a relationship. They are semantic labels that carry meaning for humans and for policy evaluation โ€” they do not affect system behavior directly. + +### 5.1 Standard Roles (DCM-defined) + +| Role | Description | +|------|-------------| +| `compute` | Processing resource โ€” VM, container, bare metal | +| `storage` | Storage resource โ€” block, object, file | +| `networking` | Network resource โ€” IP, VLAN, subnet, port | +| `security` | Security resource โ€” firewall rule, certificate, HSM | +| `database` | Database resource โ€” relational, NoSQL, time-series | +| `web` | Web tier resource โ€” web server, reverse proxy, CDN | +| `app` | Application tier resource โ€” app server, runtime | +| `cache` | Caching resource โ€” in-memory cache, CDN layer | +| `queue` | Messaging resource โ€” message queue, event stream | +| `pipeline` | Pipeline resource โ€” CI/CD, data pipeline | +| `identity` | Identity resource โ€” service account, credential | +| `monitoring` | Monitoring resource โ€” metrics, logging, alerting | +| `business_unit` | Business Unit association | +| `cost_center` | Cost Center association | +| `product_owner` | Product Owner association | +| `regulatory_scope` | Regulatory or compliance scope association | + +### 5.2 Custom Roles (extensible) + +Organizations register custom roles for domain-specific relationship semantics. Custom roles are semantic labels only โ€” they do not change system behavior. DCM core ignores unknown custom roles in operational decisions but carries them in payloads for downstream consumers. + +```yaml +custom_role_registration: + uuid: + name: + description: + registered_by_tenant_uuid: + category: + version: + status: +``` + +--- + +## 6. Relationship Nature + +Nature describes the **structural character** of a relationship โ€” what it means for the entities involved. + +| Nature | Meaning | Lifecycle Policy | Example | +|--------|---------|-----------------|---------| +| `constituent` | The related entity is a required component of this entity's definition | Required โ€” declared on relationship | VM requires its boot disk | +| `operational` | The related entity is needed for operation but is not part of the definition | Required โ€” declared on relationship | Web server depends on load balancer | +| `informational` | The related entity provides context or reference only โ€” no operational dependency | Not applicable | Resource references its Business Unit | + +--- + +## 7. Lifecycle Policies + +Lifecycle policies declare what happens to an entity when its related entity changes state. They apply to `constituent` and `operational` relationships only โ€” `informational` relationships have no lifecycle implications. + +### 7.1 Policy Actions + +| Action | Meaning | +|--------|---------| +| `destroy` | Destroy this entity when the related entity is destroyed | +| `retain` | Keep this entity when the related entity is destroyed โ€” it becomes independent | +| `detach` | Detach this entity from the relationship โ€” relationship terminated, entity retained | +| `notify` | Notify appropriate personas and trigger Policy Engine evaluation โ€” no automatic action | +| `suspend` | Suspend this entity when the related entity is suspended | +| `cascade` | Cascade the change from the related entity to this entity | +| `ignore` | Take no action โ€” the change to the related entity does not affect this entity | + +### 7.2 Lifecycle Policy Authority Hierarchy + +Lifecycle policies follow the same three-tier authority model as override control: + +``` +Resource Type Specification default (lowest โ€” portable default) + โ”‚ + โ–ผ +Provider Catalog Item default (provider preference) + โ”‚ + โ–ผ +Consumer declaration (at request time โ€” within Resource Type bounds) + โ”‚ + โ–ผ +DCM System Policy (non-overridable โ€” sovereignty and compliance mandates) +``` + +**Example:** A DCM System Policy might declare that all storage entities in a PCI-DSS scope must `retain` when their parent VM is destroyed โ€” regardless of what the provider default or consumer declared. + +--- + +## 8. Relationship Declarations โ€” Where They Live + +Relationship declarations exist at multiple levels, each building on the previous: + +### 8.1 Resource Type Specification (structural ceiling) + +Declares what relationships are **possible** for a resource type. Sets the ceiling โ€” lower levels can only declare relationships within these bounds. + +```yaml +resource_type: Compute.VirtualMachine +possible_relationships: + - role: storage + relationship_type: requires + nature: constituent + permitted_related_types: + - Storage.Block + - Storage.File + default_lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + binding_types_permitted: [owned, referenced] + consumer_declarable: true + # Consumer can declare binding_type and lifecycle_policy override + + - role: networking + relationship_type: requires + nature: constituent + permitted_related_types: + - Network.IPAddress + default_lifecycle_policy: + on_related_destroy: destroy + consumer_declarable: false + # DCM manages this automatically โ€” consumer cannot override +``` + +### 8.2 Catalog Item (offering-specific) + +Declares the **actual relationships** for a specific curated offering. Can only be more restrictive than the Resource Type Specification. + +```yaml +catalog_item: Production VM +relationships: + - role: storage + relationship_type: requires + nature: constituent + related_catalog_item_uuid: + lifecycle_policy: + on_related_destroy: retain + # Overrides Resource Type default of destroy + # Storage persists even if VM is destroyed โ€” production data protection + binding_type: owned +``` + +### 8.3 Request Time (consumer-declared) + +The consumer declares relationships in their request. Bundled declarations (storage fields within a VM request) are automatically expanded into relationship records by the Request Payload Processor. + +```yaml +# Explicit relationship declaration in a request +request: + resource_type: Compute.VirtualMachine + # ... other fields ... + relationships: + - role: storage + relationship_type: requires + binding_type: referenced + related_entity_uuid: + # Consumer referencing existing storage โ€” not creating new + +# Bundled declaration โ€” expanded automatically +request: + resource_type: Compute.VirtualMachine + storage: + disks: + - name: boot + capacity: 100GB + # Processor expands this into a Storage Entity stub + # and a relationship record with binding_type: owned +``` + +### 8.4 External Data Relationships + +Relationships to external data entities follow the same structure with `related_entity_type: external`: + +```yaml +# On a VM Entity โ€” relationship to external Business Unit +relationships: + - relationship_uuid: + this_entity_uuid: + this_role: + related_entity_uuid: + related_entity_type: external + information_provider_uuid: + information_type: Business.BusinessUnit + relationship_type: references + role: business_unit + nature: informational + lookup_method: primary_key +``` + +--- + +## 9. Bundled Declaration Expansion + +When a consumer includes resource configuration as bundled fields (e.g., storage within a VM request), the Request Payload Processor expands these into first-class entities and relationship records. + +### 9.1 Expansion Process + +``` +Consumer submits bundled VM request with storage fields + โ”‚ + โ–ผ +Request Payload Processor + โ”‚ Reads expansion rules from Resource Type Specification + โ”‚ For each expandable field: + โ”‚ 1. Creates a Resource/Service Entity stub (PENDING state) + โ”‚ with its own UUID, Tenant membership, Resource Type + โ”‚ 2. Creates a Relationship record on both the parent stub + โ”‚ and the child stub + โ”‚ 3. Applies lifecycle policy from: + โ”‚ consumer declaration โ†’ provider default โ†’ Resource Type default + โ”‚ โ†’ DCM System Policy override + โ”‚ 4. Adds the child entity stub to the relationship graph + โ–ผ +Policy Engine validates: + โ”‚ Binding type is permitted by Resource Type Specification + โ”‚ Consumer has override_matrix permission to declare binding type + โ”‚ Lifecycle policy is not overridden by a DCM System Policy + โ–ผ +Service Provider receives: + โ”‚ Parent entity request payload + โ”‚ Child entity stub UUIDs embedded in parent payload + โ”‚ Provisions resources natively + โ”‚ Returns realized payloads for all entities in DCM unified format + โ–ผ +DCM updates: + โ”‚ Parent entity: PENDING โ†’ REALIZED + โ”‚ Child entities: PENDING โ†’ REALIZED + โ”‚ All relationship records: status โ†’ active + โ”‚ Full provenance recorded on all entities and relationships +``` + +### 9.2 Expansion Rules in Resource Type Specification + +The expansion rule declares which fields expand into entities and how: + +```yaml +field_definition: + field_name: storage + type: object + expansion: + expand_to_entity: true + entity_resource_type_uuid: + entity_resource_type_name: Storage.Block + default_binding_type: owned + binding_types_permitted: [owned, referenced] + default_lifecycle_policy: + on_related_destroy: destroy + on_related_suspend: suspend + consumer_can_override_lifecycle: true + consumer_can_override_binding_type: true +``` + +--- + +## 10. The Entity Relationship Graph + +All relationships across all entities form a traversable **Entity Relationship Graph** โ€” the complete map of how all entities in DCM relate to each other. + +### 10.1 Graph Properties + +- Every node is a Resource/Service Entity (internal or external reference) +- Every edge is a Relationship with a UUID +- The graph is bidirectional โ€” traversable from any node in any direction +- Every node exists exactly once โ€” shared entities appear once with multiple relationship edges +- Circular relationships are invalid and must be rejected + +### 10.2 Graph and the Four States + +The relationship graph exists across all four states: + +| State | Graph Role | +|-------|-----------| +| Intent State | Graph declared at request time โ€” nodes are intent stubs | +| Requested State | Graph fully assembled โ€” nodes are PENDING entity stubs with UUIDs | +| Realized State | Graph populated โ€” nodes are REALIZED entities with full provenance | +| Discovered State | Graph used for comparison โ€” discovered entities matched against realized graph | + +### 10.3 Graph Applications + +| Application | How the Graph is Used | +|-------------|----------------------| +| **Rehydration** | Full graph traversal from a root entity โ€” all related entities identified and realized in dependency order | +| **Cost Rollup** | Graph traversal accumulates costs across all related constituent entities | +| **Drift Detection** | Discovered State graph compared against Realized State graph โ€” structural and data differences identified | +| **Decommission** | Graph traversal determines decommission order โ€” lifecycle policies applied at each edge | +| **Placement** | Pre-realization graph used to understand full resource footprint for placement decisions | +| **Impact Analysis** | Graph traversal from any node identifies all entities affected by a change | + +--- + +## 11. Relationship Integrity + +### 11.1 DCM System Policies for Relationships + +| Policy | Rule | +|--------|------| +| `REL-001` | Every relationship must have a UUID | +| `REL-002` | Every relationship must be recorded on both participating entities | +| `REL-003` | Circular relationships are invalid and must be rejected | +| `REL-004` | A constituent relationship must declare a lifecycle policy | +| `REL-005` | External relationships must reference a registered Information Provider | +| `REL-006` | Relationship types must be from the standard vocabulary | +| `REL-007` | Consumer-declared binding types must be permitted by the Resource Type Specification | + +### 11.2 Relationship Versioning and Deprecation + +Relationships follow the universal versioning and deprecation model. A relationship version changes when its lifecycle policy, nature, or role changes. Terminated relationships are retained in provenance permanently. + +--- + +## 12. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Policy model | โ“ Unresolved | +| 2 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Operational complexity | โ“ Unresolved | +| 3 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Multi-tenancy | โ“ Unresolved | +| 4 | Should there be a maximum relationship graph depth to prevent runaway complexity? | Operational governance | โ“ Unresolved | +| 5 | How are shared entities represented in the relationship graph โ€” an entity required by multiple parents? | Graph model | โ“ Unresolved | + +--- + +## 13. Related Concepts + +- **Entity Relationship Graph** โ€” the complete traversable graph of all entity relationships in DCM +- **Information Provider** โ€” provider type for external data entities referenced in relationships +- **Bundled Declaration Expansion** โ€” processor mechanism for expanding bundled fields into entities and relationships +- **Lifecycle Policy** โ€” declares what happens to an entity when its related entity changes state +- **Service Dependencies** โ€” document covering rehydration ordering and failure handling on the relationship graph +- **Resource Type Specification** โ€” declares possible relationships for a resource type +- **External Entity Reference** โ€” stable pointer to data owned by an external system + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/four-states.md b/content/docs/data-model/four-states.md new file mode 100644 index 0000000..60db14d --- /dev/null +++ b/content/docs/data-model/four-states.md @@ -0,0 +1,442 @@ +--- +title: Four States +type: docs +weight: 2 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) + +--- + +## 1. Purpose + +The four states are the foundational model for how DCM tracks the complete lifecycle of any resource or service. Every entity in DCM exists in one or more of these states simultaneously. The states are not sequential stages โ€” they are parallel, independently maintained records that together provide a complete, auditable picture of what was requested, what was approved, what was built, and what actually exists. + +The four states answer four distinct questions: + +| State | Question Answered | Store Type | +|-------|------------------|------------| +| **Intent State** | What did the consumer ask for? | GitOps Store | +| **Requested State** | What was approved and dispatched to the provider? | GitOps Store | +| **Realized State** | What did the provider actually build? | Event Stream Store | +| **Discovered State** | What does DCM observe actually existing right now? | Event Stream Store (ephemeral) | + +--- + +## 2. State Definitions + +### 2.1 Intent State + +The **Intent State** is the immutable record of a consumer's original declaration. It is captured at the moment a request is submitted โ€” before any layer assembly, before any policy evaluation, before any provider selection. + +**Characteristics:** +- Immutable once created โ€” the consumer's original intent is never modified +- Stored in a GitOps store โ€” branched, reviewed, merged +- The CI/CD pipeline operates on the Intent State โ€” policy pre-validation, cost estimation, sovereignty check, approval workflow +- Versioned via Git history โ€” every revision of an intent is traceable +- Supports human review and debate via the PR mechanism +- The entity UUID is assigned at Intent State creation โ€” it follows the entity through all subsequent states + +**When created:** Every request submission, every rehydration operation, every drift remediation authorization + +**Content:** The consumer's raw declaration in DCM Unified Data Model format โ€” what they want, not what will be built + +### 2.2 Requested State + +The **Requested State** is the fully assembled, policy-processed, provider-ready payload. It is produced by the Request Payload Processor from the Intent State โ€” after layer assembly, after all policy evaluation, after provider selection. + +**Characteristics:** +- Immutable once created โ€” a new Requested State is created for each request cycle +- Stored in a GitOps store โ€” committed, versioned, triggering CD pipeline +- The CD pipeline dispatches from the Requested State to the provider +- Contains the complete assembled payload with full field-level provenance +- Contains the results of all policy evaluations โ€” which policies ran, what they did, what they locked +- Contains provider selection โ€” which provider will realize this request +- Is the authoritative record of what DCM instructed a provider to build + +**When created:** After Intent State approval (merge), after successful policy processing + +**Content:** The complete assembled payload in DCM Unified Data Model format, with full provenance chain, policy evaluation results, provider selection, and override control metadata + +### 2.3 Realized State + +The **Realized State** is the provider-confirmed record of what was actually built. It is produced by the provider after successful realization โ€” the denaturalized result of the provider's execution, translated back to DCM Unified Data Model format. + +**Characteristics:** +- Append-only event stream โ€” each state change is a new event, never an overwrite +- Stored in an Event Stream Store โ€” high-frequency writes, entity-keyed streams +- The entity UUID is the stream key โ€” all realized state events for an entity share the same stream +- Contains provider-specific details not in the Requested State โ€” assigned IPs, generated passwords, actual storage sizes, provider-internal IDs +- Is the authoritative record of what actually exists from DCM's perspective +- Drift is detected by comparing Realized State against Discovered State + +**When created:** After provider confirms realization, updated on every provider lifecycle event + +**Content:** The realized entity in DCM Unified Data Model format, with provider-added fields, full field-level provenance including provider attribution + +### 2.4 Discovered State + +The **Discovered State** is what DCM observes actually existing through active discovery โ€” polling providers, querying Kubernetes APIs, interrogating infrastructure. It is the ground truth of what physically exists, independent of what DCM thinks exists. + +**Characteristics:** +- Append-only snapshot stream โ€” each discovery cycle produces a new snapshot +- Stored in an Event Stream Store (ephemeral) โ€” recent history retained, older snapshots archived or discarded +- High-frequency and machine-generated โ€” not appropriate for human review +- Used exclusively for drift detection โ€” comparing against Realized State +- May contain resources DCM did not provision โ€” brownfield resources discovered for ingestion + +**When created:** On every discovery cycle, on demand for specific entities + +**Content:** Raw discovered resource state in DCM Unified Data Model format, with discovery metadata (timestamp, discovery method, provider interrogated) + +--- + +## 3. The Entity UUID โ€” Universal Linking Key + +Every entity has a single UUID assigned at Intent State creation. This UUID is the universal key linking the entity across all four states and all stores: + +``` +Intent Store: file path includes entity_uuid, content declares entity_uuid +Requested Store: file path includes entity_uuid, content declares entity_uuid +Realized Store: event stream keyed by entity_uuid +Discovered Store: snapshot stream keyed by entity_uuid (matched via provider labels) +Audit Store: all provenance events indexed by entity_uuid +Search Index: entity_uuid โ†’ git_path mapping for Git stores +``` + +Given an entity UUID, DCM can reconstruct the complete history of that entity across its entire lifecycle โ€” from the consumer's original intent through every state transition to the current discovered state. + +--- + +## 4. Physical Representation โ€” Storage Provider Model + +DCM describes store **contracts**, not implementations. Each store is a Storage Provider โ€” a formal DCM provider type with registration, health check, and trust obligations. Implementors choose the technology that satisfies the contract. + +See [Storage Providers](11-storage-providers.md) for the complete contract specifications. + +### 4.1 GitOps Stores (Intent and Requested) + +**Contract characteristics:** +- Branch-per-request โ€” each request is a branch in the store +- Pull Request semantics โ€” review, comment, approve, merge +- Immutable history โ€” commits are permanent records +- CI/CD hook support โ€” commits trigger pipeline execution +- Indexed for query โ€” a Search Index projection enables field-based queries at scale +- Entity UUID โ†’ file path mapping maintained in the Search Index + +**Typical implementations:** GitHub, GitLab, Gitea, Forgejo (with Elasticsearch/OpenSearch as the Search Index) + +**Repository structure:** Deferred pending Q54 resolution (provider selection in Requested State affects directory structure). Will be documented in `04-examples.md`. + +### 4.2 Event Stream Stores (Realized and Discovered) + +**Contract characteristics:** +- Append-only โ€” events are never overwritten or deleted +- Entity-keyed streams โ€” each entity has its own event stream identified by entity UUID +- Queryable by entity UUID โ€” O(1) lookup of an entity's event stream +- Replayable โ€” the stream can be replayed from any point to reconstruct state at any timestamp +- Distributed and redundant โ€” data is replicated across nodes with configurable consistency guarantees +- High throughput โ€” designed for machine-generated, high-frequency writes + +**Typical implementations:** Kafka with log compaction, EventStoreDB, Apache Pulsar + +### 4.3 Search Index (Git Store Projection) + +**Contract characteristics:** +- Derived from Git stores โ€” rebuilt from Git history on demand +- Explicitly non-authoritative โ€” Git always wins if index and Git disagree +- Queryable by indexed fields: entity_uuid, tenant_uuid, resource_type, lifecycle_state, timestamp, cost_center, business_unit, provider_uuid +- Lightweight โ€” stores indexed fields only, not full payloads +- Fast โ€” designed for millisecond query response at millions of records + +**Typical implementations:** Elasticsearch, OpenSearch, Meilisearch + +--- + +## 5. Rehydration + +Rehydration is the process of using a previously stored state record as the starting point for a new request. It is not a shortcut around governance โ€” **all relevant governance policies always apply regardless of rehydration source.** Rehydration is a new request that happens to start from a known prior state. + +### 5.1 Three Rehydration Sources + +**From Intent State:** +- The consumer's original declaration is replayed +- Full layer assembly runs โ€” current layers applied +- All governance policies run โ€” current policies applied +- Provider selection runs fresh +- Most likely to produce a different result than the original โ€” policies and layers may have changed +- Use cases: upgrade resource to current standards, apply new sovereignty constraints, environment refresh + +**From Requested State:** +- The previously assembled, policy-processed payload is loaded +- Layer assembly is skipped โ€” layers were already applied +- All governance policies run โ€” current policies applied +- Provider selection: configurable via flag (see Section 5.3) +- Use cases: reproduce a resource as closely as possible to the approved specification + +**From Realized State:** +- The provider-confirmed realized payload is loaded +- Provider-specific fields are stripped โ€” DCM unified format only +- Layer assembly is skipped +- All governance policies run โ€” current policies applied +- Provider selection: configurable via flag +- Use cases: exact reproduction for disaster recovery, environment cloning, replacing a failed resource + +### 5.2 The Common Governance Pipeline + +Regardless of rehydration source, all requests flow through the same governance pipeline: + +``` +Rehydration source selected and loaded + โ”‚ + โ”‚ Source payload becomes the basis for a new Intent State record + โ”‚ New entity UUID assigned (or existing UUID preserved โ€” policy decision) + โ”‚ Rehydration provenance recorded: source_store, source_record_uuid, + โ”‚ rehydration_reason, requested_by_uuid, rehydration_timestamp + โ–ผ +If source = Intent: + โ”‚ Full layer assembly runs (Steps 1-7) + โ”‚ Current layers applied + โ–ผ +If source = Requested or Realized: + โ”‚ Layer assembly skipped + โ”‚ Payload loaded as pre-assembled + โ”‚ If source = Realized: provider-specific fields stripped + โ–ผ +Placement evaluation + โ”‚ See Section 5.3 โ€” configurable + โ–ผ +Policy Engine โ€” ALL governance policies applied + โ”‚ Authorization policies: does this actor have permission to rehydrate? + โ”‚ Transformation policies: current enrichment applied + โ”‚ Validation policies: current constraints checked + โ”‚ GateKeeper policies: current field locks applied + โ”‚ Gatekeeping policies: is this resource type still permitted? + โ”‚ + โ”‚ Governance is NEVER skippable โ€” not for any rehydration source, + โ”‚ not for any actor, not for any urgency claim + โ–ผ +New Requested State produced and stored + โ”‚ New record โ€” never overwrites the source record + โ”‚ Source record remains immutable + โ”‚ Provenance chain links to source record + โ–ผ +Provider dispatch + โ”‚ Dispatched to selected provider + โ–ผ +New Realized State events produced + โ”‚ New event stream or continuation of existing stream + โ”‚ Provenance links to rehydration Requested State +``` + +### 5.3 Placement Flag โ€” Provider-Portable Rehydration + +When rehydrating from Requested State or Realized State, provider selection is configurable via an explicit flag in the rehydration request: + +```yaml +rehydration_request: + uuid: + source_store: + source_record_uuid: + + placement: + re_evaluate: false + # false (default): honor provider selection from source record + # Use when: original provider is available and appropriate + # Result: resource reproduced on same provider + # + # true: strip provider selection, run placement policies fresh + # Use when: original provider unavailable, decommissioned, + # at capacity, or no longer sovereign-compliant + # Result: placement policies select provider from current landscape + # Named concept: Provider-Portable Rehydration + + placement_constraints: + # Optional โ€” additional constraints for re-evaluation + # Only applicable when re_evaluate: true + exclude_provider_uuids: [, ...] + require_region: + require_sovereignty_capability: + + governance: + apply_all_policies: true + # Always true โ€” governance is never skippable + # Included explicitly for auditability โ€” the rehydration record + # must declare that governance was applied + + policy_version: current + # current (default): apply today's policies + # pinned: apply policies as of a specific timestamp + # Use when: exact historical reproduction required + # (audit evidence, regulatory examination, environment reconstruction) + # Requires elevated authorization โ€” bypasses current GateKeeper policies + # Only SRE and Admin actors may use pinned policy version + + pinned_timestamp: + # Required when policy_version: pinned + + rehydration_reason: + requested_by_uuid: +``` + +### 5.4 The Four Rehydration Modes + +Two independent axes โ€” placement and policy version โ€” produce four distinct rehydration configurations: + +| Mode | re_evaluate | policy_version | Use Case | +|------|-------------|----------------|----------| +| **Faithful** | false | current | Same provider, current governance | +| **Provider-Portable** | true | current | New provider, current governance | +| **Historical Exact** | false | pinned | Same provider, historical governance (audit evidence) | +| **Historical Portable** | true | pinned | New provider, historical governance | + +Historical modes require elevated authorization. All modes run governance โ€” the difference is whether governance uses current or pinned policies. + +### 5.5 Partial Resolution of Q54 โ€” Provider Selection + +The placement flag model clarifies the Q54 question (selected_provider as policy output vs placement component). The emerging answer: + +**Policies set placement constraints โ€” the placement component selects the provider.** + +A GateKeeper policy may output: "must be in region EU-WEST, must support sovereignty capability PCI-DSS." The placement component reads these constraints and selects the specific provider within those constraints. The policy does not name the provider. The placement component names the provider. + +This is consistent with the portability model โ€” a policy that names a specific provider would be portability-breaking. Policies set constraints. Placement honors constraints and selects. + +--- + +## 6. Drift Detection + +Drift is the difference between what DCM believes exists (Realized State) and what actually exists (Discovered State). + +### 6.1 Drift Detection Flow + +``` +Discovery cycle completes + โ”‚ Provider interrogated โ†’ Discovered State snapshot written + โ–ผ +Drift Detection component + โ”‚ Loads latest Discovered State for entity UUID + โ”‚ Loads latest Realized State events for entity UUID + โ”‚ Field-by-field comparison + โ–ผ +No drift detected + โ”‚ Discovery timestamp updated + โ”‚ No action + โ–ผ +Drift detected + โ”‚ Drift record created with: + โ”‚ - entity_uuid + โ”‚ - drifted_fields: [{field_path, realized_value, discovered_value}] + โ”‚ - discovery_timestamp + โ”‚ - drift_severity: + โ–ผ +Policy Engine evaluates drift + โ”‚ Drift response policy determines action: + โ”‚ REVERT: submit a rehydration request from Realized State to restore + โ”‚ UPDATE_DEFINITION: promote discovered state to new Realized State + โ”‚ ALERT: notify personas, no automatic action + โ”‚ ESCALATE: trigger human review workflow + โ”‚ + โ”‚ Response determined by drift severity, resource type, + โ”‚ resource ownership, and organizational policy + โ–ผ +Audit Store records drift event with full provenance +``` + +### 6.2 Unsanctioned Changes + +A specific category of drift โ€” a change made directly to a resource without a corresponding DCM request. Detected by: +- Kubernetes: CR spec change without DCM request annotation +- VMware/OpenStack: resource modification not traceable to a DCM Requested State record +- General: any Discovered State field value that differs from Realized State without a Requested State record explaining the change + +Unsanctioned changes are always reported to the Policy Engine as `UNSANCTIONED_CHANGE` events. Policy determines the response. + +--- + +## 7. CI/CD Integration + +The GitOps stores are the natural integration point for CI/CD pipelines. DCM does not prescribe a specific CI/CD tool โ€” the GitOps store contract requires hook support, and the CI/CD tool is a deployment choice. + +### 7.1 CI Pipeline (Intent State) + +Triggered on: branch creation or update (new or revised intent) + +``` +CI pipeline executes: + 1. Policy pre-validation (dry run โ€” no state changes) + โ†’ Reports: which policies would apply, what they would do + 2. Cost estimation + โ†’ Reports: estimated cost for lifecycle of this resource + 3. Dependency graph validation + โ†’ Reports: all required dependent resources, any conflicts + 4. Sovereignty constraint check + โ†’ Reports: which sovereignty constraints apply, any violations + 5. Authorization check + โ†’ Reports: does this actor have permission to request this resource type? + 6. Auto-approve evaluation + โ†’ Reports: can this be merged automatically, or does it require human review? + +All results posted as PR comments on the Intent State branch +Consumer and approvers can review and debate before merge +``` + +### 7.2 CD Pipeline (Requested State) + +Triggered on: Intent State merge (PR merged to main) + +``` +CD pipeline executes: + 1. Request Payload Processor assembles full payload + 2. Full policy evaluation (binding โ€” not dry run) + 3. Provider selection (or re-evaluation if placement flag set) + 4. Requested State committed to Git store + 5. Provider dispatch via API Gateway + 6. Status monitoring โ€” poll or receive callbacks until terminal state + 7. Status written back to PR or status file + 8. Consumer notification +``` + +### 7.3 The Third Rail โ€” Direct API Ingress + +Not all requests come through the GitOps PR workflow. Some requests come through direct API submission โ€” automated systems, CI/CD pipelines, Terraform providers, programmatic consumers. These bypass the human review workflow but not governance. + +Direct API ingress: +- Creates an Intent State record (the submitted payload becomes the intent) +- Runs the same CI validation pipeline but non-interactively +- If auto-approve policy permits: proceeds directly to assembly and dispatch +- If human review required: creates a PR for review before proceeding +- Same governance pipeline regardless of ingress path + +The three ingress paths โ€” PR workflow, direct API, and programmatic (Terraform/Ansible) โ€” all converge on the same governance pipeline. The ingress path affects the review workflow; it never affects governance. + +--- + +## 8. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Git repository structure for Intent and Requested stores โ€” deferred pending Q54 resolution | Store design | โ“ Unresolved | +| 2 | Should the entity UUID be preserved or regenerated on rehydration? | Entity identity | โ“ Unresolved | +| 3 | For pinned policy version rehydration โ€” what is the minimum authorization level required? | Security | โ“ Unresolved | +| 4 | How are concurrent rehydration requests for the same entity handled โ€” serialized or rejected? | Concurrency | โ“ Unresolved | +| 5 | Should the Discovered Store retain full history or only a configurable window? | Retention | โ“ Unresolved | +| 6 | How does the Search Index handle Git store unavailability โ€” serve stale results or fail? | Reliability | โ“ Unresolved | + +--- + +## 9. Related Concepts + +- **Storage Provider** โ€” the formal provider type for all DCM stores +- **Entity UUID** โ€” the universal linking key across all four states +- **Rehydration** โ€” using a prior state record as the starting point for a new request +- **Provider-Portable Rehydration** โ€” rehydration with provider selection re-evaluated +- **Drift Detection** โ€” comparing Realized State against Discovered State +- **Unsanctioned Change** โ€” a resource modification not traceable to a DCM request +- **CI/CD Integration** โ€” GitOps stores as the natural CI/CD integration point +- **Search Index** โ€” queryable projection of GitOps stores, explicitly non-authoritative + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/information-providers.md b/content/docs/data-model/information-providers.md new file mode 100644 index 0000000..c263acd --- /dev/null +++ b/content/docs/data-model/information-providers.md @@ -0,0 +1,434 @@ +--- +title: Information Providers +type: docs +weight: 9 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) + +--- + +## 1. Purpose + +An **Information Provider** is a registered DCM provider that serves as the authoritative source for a specific category of data that DCM needs to reference but does not own. It exposes external data to DCM through a standard interface, enabling DCM to look up, verify, and relate external records without caching or owning them. + +Information Providers are a first-class provider type in DCM alongside Service Providers and Meta Providers. They follow the same registration, health check, trust, and contract model as Service Providers โ€” adapted where applicable to the lookup-only nature of information retrieval. + +--- + +## 2. Why Information Providers Exist + +DCM manages the lifecycle of resources it provisions. But resources exist in a broader organizational context โ€” they are owned by business units, attributed to cost centers, associated with product owners, governed by regulatory scopes. This contextual data lives in authoritative external systems (HR systems, finance systems, CMDBs, ITSM tools) that DCM does not and should not own. + +Without a formal model for referencing external data, organizations face two bad choices: +- **Copy the data into DCM** โ€” creating duplication, staleness, and an ownership conflict with the authoritative system +- **Ignore the data** โ€” losing business context, cost attribution, and compliance traceability + +Information Providers solve this by giving DCM a standard, stable, governed interface to external data without requiring ownership transfer. + +--- + +## 3. The Three Provider Types + +DCM recognizes three provider types. All follow the same base contract model: + +| Provider Type | Purpose | Data Direction | DCM Owns Result? | +|--------------|---------|---------------|-----------------| +| **Service Provider** | Executes work, realizes resources | DCM โ†’ Provider โ†’ DCM | Yes โ€” DCM owns the realized entity | +| **Information Provider** | Serves authoritative external data | DCM โ†’ Provider (lookup only) | No โ€” external system is authoritative | +| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Child Providers โ†’ DCM | Yes โ€” DCM owns the composite result | + +--- + +## 4. Information Provider Contract + +Information Providers follow the same provider contract model as Service Providers where applicable. The contract dimensions are: + +### 4.1 Registration Contract +Same model as Service Providers. Information Providers register with DCM declaring their endpoint, the information types they implement, their lookup capabilities, and their extended schema. + +### 4.2 Health Check Contract +Same model as Service Providers. Information Providers expose a `/health` endpoint. DCM polls it on the same configurable interval. `Ready`/`NotReady` state machine applies. An `NotReady` Information Provider is excluded from lookups โ€” relationships referencing it are flagged for on-demand verification fallback. + +### 4.3 Trust Contract +Same model as Service Providers. Information Providers must be registered, validated, and certified before DCM will accept their data. The chain of trust applies to data returned by Information Providers โ€” provenance records the provider UUID for every field sourced from an Information Provider. + +### 4.4 Capacity Contract +Adapted for lookup capacity rather than resource provisioning capacity. Information Providers declare and report their query capacity โ€” requests per second, rate limits, availability windows. + +```yaml +capacity_registration: + provider_uuid: + registration_timestamp: + capacity_by_information_type: + - information_type_uuid: + queries_per_second: 1000 + rate_limit_window: 60s + availability: 99.9% +``` + +### 4.5 Lifecycle Event Contract +Same model as Service Providers. Information Providers have a contractual obligation to notify DCM when records they have provided references for change status. DCM receives the notification and updates the external entity reference record accordingly. + +**Reportable event types for Information Providers:** + +| Event Type | Description | DCM Response | +|------------|-------------|--------------| +| `RECORD_DEACTIVATED` | A referenced record has been deactivated | Update reference status, Policy Engine evaluation | +| `RECORD_MERGED` | Two records merged โ€” UUID may change | Update external_uuid in reference record | +| `RECORD_SPLIT` | One record split into multiple | Policy Engine evaluation โ€” which new record applies? | +| `UUID_CHANGED` | Record UUID changed in external system | Update external_uuid, re-verify all references | +| `DATA_UPDATED` | Standard field values changed | Update last_verified, notify relationships | +| `PROVIDER_DEGRADED` | Provider is degraded but operational | DCM flags affected references for on-demand verification | + +### 4.6 Naturalization/Denaturalization Contract +Information Providers translate their native data format (HR system JSON, finance system XML, LDAP records, REST APIs) into the DCM unified data model format. The translation is the provider's responsibility โ€” DCM always receives data in DCM format. + +--- + +## 5. Standard vs Extended Data + +### 5.1 Standard Data (DCM-defined) + +Fields that are part of the DCM-specified schema for an information type. DCM core uses these fields for lookups, relationship matching, policy evaluation, and display. They are portable across all implementations of that information type. + +DCM only relies on standard data for operational decisions. Extended data is carried in the payload but is not used for DCM core operations. + +### 5.2 Extended Data (organization-defined) + +Additional fields organizations add to enrich the standard schema for their specific needs. Declared in the provider's extended schema registration. DCM carries extended data in the payload for downstream consumers โ€” policy engines, cost analysis tools, reporting โ€” that know how to use them. + +```yaml +# Standard + Extended data example โ€” Business.BusinessUnit +business_unit_record: + # Standard fields โ€” DCM defined, used for lookups + uuid: "bu-uuid-001" + name: "Payments Platform" + code: "BU-PAY" + parent_uuid: "bu-uuid-root" + organization_uuid: "org-uuid-001" + status: active + + # Extended fields โ€” organization defined + extensions: + profit_center_code: "PC-4421" + regulatory_jurisdiction: "EU" + trading_desk_id: "TD-007" + risk_tier: 1 + internal_charge_code: "IC-PAY-001" +``` + +--- + +## 6. Lookup Key Model + +DCM looks up external records using a stable primary key โ€” always the external UUID where available โ€” with a fallback chain for systems that don't support UUID-based lookup. + +### 6.1 External Entity Reference Structure + +```yaml +external_entity_reference: + uuid: + # DCM UUID is what gets stored in relationship declarations + # If the external system changes its UUID, only this record changes + # All relationships pointing to dcm-uuid remain valid + + external_uuid: + information_provider_uuid: + information_type_uuid: + information_type_name: Business.BusinessUnit + + lookup_method: + primary_key: external_uuid + # Always attempted first + fallback_keys: + - field: code + value: "BU-PAY" + - field: name + value: "Payments Platform" + # Fallback keys tried in order if primary_key lookup fails + + # Non-authoritative display cache โ€” for UI convenience only + display_name: "Payments Platform" + display_name_authoritative: false + + verification: + last_verified: + last_verified_method: + verification_status: + next_scheduled_verification: + + status: + state: + + provenance: + +``` + +### 6.2 Why DCM UUID Wraps External UUID + +The DCM-generated UUID is the stable internal anchor. This means: +- All relationship declarations inside DCM reference the DCM UUID +- If the external system changes its UUID (migration, system upgrade), only the `external_entity_reference` record needs updating +- All relationships pointing to the DCM UUID remain valid without modification +- The provenance chain tracks the change via the `UUID_CHANGED` lifecycle event + +--- + +## 7. Three-Mode Verification Model + +DCM uses a trust-but-verify approach to external entity references. The external system is trusted as authoritative for the data โ€” DCM does not validate content. But DCM verifies that references remain valid โ€” the UUID still exists and the record is still active. + +### 7.1 Mode 1 โ€” Scheduled Verification (DCM-initiated) + +DCM calls the Information Provider's `/verify/{uuid}` endpoint on a configurable schedule for all registered external entity references. Default frequency: configurable โ€” suggested minimum twice daily. Updates `last_verified` and `verification_status`. + +### 7.2 Mode 2 โ€” Provider Push (Information Provider obligation) + +The Information Provider notifies DCM when a referenced record changes status. This is a contractual obligation โ€” same model as Service Provider lifecycle events. DCM receives the notification, updates the external entity reference, and the Policy Engine evaluates the appropriate response. + +### 7.3 Mode 3 โ€” On-Demand Verification (fallback) + +When a relationship involving an external entity reference is accessed during request processing, policy evaluation, or drift detection, DCM can verify the reference in real time before relying on it. Used when: +- `last_verified` is beyond the acceptable staleness window +- The operation requires high confidence +- Scheduled verification returned `stale` or `unverifiable` + +### 7.4 Verification Fallback Chain + +``` +External entity reference accessed + โ”‚ + โ–ผ +Is verification_status: verified AND last_verified within window? + โ”‚ Yes โ†’ proceed with reference + โ”‚ No โ†“ + โ–ผ +Mode 3 โ€” on-demand verify via Information Provider /verify/{uuid} + โ”‚ Success โ†’ update last_verified, verification_status: verified, proceed + โ”‚ Failure โ†“ + โ–ผ +Policy Engine evaluates: + Options (configurable per information type and organizational policy): + block_request โ€” reject request until reference is verified + warn_and_proceed โ€” proceed with warning recorded in provenance + use_display_only โ€” use display_name only, no operational reliance + escalate โ€” notify appropriate personas for human resolution +``` + +--- + +## 8. Information Type Registry + +Information types live in the same DCM Resource Type Registry as Resource Types, distinguished by category prefix. Same versioning, same deprecation model, same governance. + +### 8.1 Standard Information Type Categories + +| Category | Description | Examples | +|----------|-------------|---------| +| `Business.*` | Business organizational data | BusinessUnit, CostCenter, ProductOwner | +| `Identity.*` | Identity and access data | Person, ServiceAccount, Group | +| `Compliance.*` | Regulatory and compliance data | RegulatoryScope, AuditFramework | +| `Operations.*` | Operational reference data | Runbook, SLA, SupportContract | + +### 8.2 DCM Default Information Types + +```yaml +# Business.BusinessUnit +information_type: + uuid: + name: Business.BusinessUnit + category: Business + version: 1.0.0 + standard_fields: + - name: uuid + type: string + required: true + lookup_supported: true + - name: name + type: string + required: true + lookup_supported: true + - name: code + type: string + required: false + lookup_supported: true + - name: parent_uuid + type: string + required: false + lookup_supported: false + - name: organization_uuid + type: string + required: true + lookup_supported: false + - name: status + type: enum + values: [active, inactive] + required: true + lookup_supported: false + extended_fields_permitted: true + status: active + +# Business.CostCenter +information_type: + uuid: + name: Business.CostCenter + standard_fields: + - name: uuid + lookup_supported: true + - name: name + lookup_supported: true + - name: code + lookup_supported: true + - name: owner_uuid + lookup_supported: false + - name: budget_period + lookup_supported: false + - name: status + lookup_supported: false + +# Identity.Person +information_type: + uuid: + name: Identity.Person + standard_fields: + - name: uuid + lookup_supported: true + - name: name + lookup_supported: true + - name: email + lookup_supported: true + - name: employee_id + lookup_supported: true + - name: department_uuid + lookup_supported: false + - name: status + lookup_supported: false +``` + +### 8.3 Custom Information Types + +Organizations register custom information types following the same model: + +```yaml +custom_information_type: + uuid: + name: + # Must use a non-reserved category prefix or register a new one + category: + version: + registered_by_tenant_uuid: + standard_fields: + + extended_fields_permitted: + status: +``` + +--- + +## 9. Information Provider Registration + +```yaml +information_provider_registration: + uuid: + name: + display_name: + version: + + implements: + - information_type_uuid: + information_type_name: Business.BusinessUnit + information_type_version: + lookup_methods_supported: [primary_key, code] + extended_fields_supported: true + extended_schema: + + + endpoint: + + capacity: + queries_per_second: + rate_limit_window: + update_frequency: + + sovereignty_constraints: + + + trust_declaration: + + + health_check: + endpoint: /health + poll_interval_seconds: + + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + + provenance: + +``` + +--- + +## 10. Mandatory Information Provider API Endpoints + +All Information Providers must implement these endpoints as part of their provider contract: + +| Method | Endpoint | Description | +|--------|----------|-------------| +| `GET` | `/health` | Provider health check โ€” same as Service Provider | +| `GET` | `/lookup/{uuid}` | Returns standard + extended data for a record by external UUID | +| `GET` | `/verify/{uuid}` | Lightweight โ€” confirms UUID exists and is active | +| `POST` | `/search` | Finds records matching standard field criteria (fallback lookup) | +| `POST` | `/notify` | DCM calls this to acknowledge receipt of provider push events | + +--- + +## 11. Internally Owned Business Data + +When an organization decides to manage business context data in DCM rather than reference an external system, they define it as a DCM Resource Type in the `Business.*` or custom category. Internally owned business data follows the **standard resource entity model** exactly: + +- Has a UUID +- Has a Resource Type (`Business.BusinessUnit`, `Business.CostCenter`, etc.) +- Has provenance +- Has versioning +- Has relationships to other entities +- Follows the universal lifecycle (active โ†’ deprecated โ†’ retired) +- Can be grouped under Tenants and Resource Groups + +The relationship model is identical whether the related entity is internal or external โ€” the `related_entity_type` field (`internal` vs `external`) is the only difference from the consuming entity's perspective. + +This means an organization can start with an external Information Provider reference and migrate to internally owned business data later โ€” relationships remain structurally the same, only the `related_entity_type` changes. + +--- + +## 12. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are conflicting provider push events handled โ€” two Information Providers claim authority for the same record? | Data integrity | โ“ Unresolved | +| 2 | Should Information Providers support write-back โ€” DCM updating external records via the provider? | Scope expansion | โ“ Unresolved | +| 3 | How is the extended schema versioned โ€” if a provider adds or removes extended fields, how are existing references affected? | Versioning | โ“ Unresolved | +| 4 | Should DCM maintain a registry of well-known Information Providers (HR systems, finance systems) to simplify onboarding? | Adoption | โ“ Unresolved | +| 5 | How does the verification model interact with air-gapped environments where Information Providers may be unreachable? | Sovereignty | โ“ Unresolved | + +--- + +## 13. Related Concepts + +- **External Entity Reference** โ€” the stable pointer record DCM uses to reference external data +- **Entity Relationships** โ€” the universal relationship model that uses Information Provider references +- **Service Provider** โ€” counterpart provider type for resource provisioning +- **Resource Type Registry** โ€” the unified registry containing both Resource Types and Information Types +- **Trust Contract** โ€” the provider trust model shared across all provider types +- **Naturalization/Denaturalization** โ€” translation between external native format and DCM unified format + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/layering-and-versioning.md b/content/docs/data-model/layering-and-versioning.md new file mode 100644 index 0000000..37da55e --- /dev/null +++ b/content/docs/data-model/layering-and-versioning.md @@ -0,0 +1,1212 @@ +--- +title: Data Layers and Assembly +type: docs +weight: 3 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) + +--- + +## 1. Purpose + +Data Layers are the mechanism by which DCM assembles a complete, contextually correct request payload from a set of composable, reusable data definitions. Rather than requiring consumers to specify every field of every resource they request, layers allow standards, organizational context, service-specific configuration, and consumer intent to be declared independently and merged into a unified payload at request time. + +Layers are the answer to the question: **how does a single consumer request become a complete, policy-validated, provider-ready payload?** + +The layering model enables: +- **Reuse** โ€” a base configuration defined once is inherited by thousands of resources +- **Standardization** โ€” organizational standards are encoded in layers, not in every individual request +- **Separation of concerns** โ€” infrastructure teams own core and service layers; consumers own request layers; policy owners own policy layers +- **Scale** โ€” 36 layer definitions can govern 40,000 VMs without duplication +- **Auditability** โ€” every field in the merged payload knows which layer set it and why + +--- + +## 2. What is a Layer? + +A Layer is a **declarative, immutable, versioned unit of data** that contributes some or all of its fields to a merged payload. Layers do not execute โ€” they declare. The assembly process is what merges them. + +Every layer: +- Has a **UUID** that uniquely identifies it +- Has a **version** following the universal Major.Minor.Revision scheme +- Is **immutable once published** โ€” changes produce a new version +- Carries a **reference to its parent entity** (UUID and version) +- Has an **origination timestamp** +- Can be **deprecated** following the universal deprecation model +- Contributes **provenance metadata** for every field it sets โ€” any field set by a layer records that layer's UUID as its source + +Layers are stored in Git following GitOps practices. They are the configuration source of truth โ€” not the assembled payload. + +--- + +## 3. Layer Types + +DCM defines six layer types. Each has a distinct purpose, scope, ownership model, and position in the assembly precedence chain. + +### 3.1 Base Layer + +**Purpose:** The foundation entity for a resource. Defines the minimum required fields and their default values for a given resource context. Everything starts with a Base Layer. + +**Scope:** Can be type-agnostic (a universal base) or type-scoped (a base specific to a Resource Type). A Base Layer that is type-scoped must declare its Resource Type. + +**Ownership:** DCM platform or platform implementor. + +**Characteristics:** +- Every layer chain must begin with a Base Layer +- Base Layers contain only universal fields โ€” no provider-specific data +- A Base Layer for a typed resource must conform to the Resource Type Specification's universal field requirements +- Multiple Base Layers can exist for the same context โ€” the applicable one is selected based on the request context + +**Examples:** +- CIS Benchmark base configuration +- Baseline OS configuration +- DMZ network base configuration + +--- + +### 3.2 Core Layers + +**Purpose:** Provide data that is applicable across any resource type. Core Layers carry organizational, infrastructure, and contextual data that is not specific to any one service. + +**Scope:** Type-agnostic by default. Core Layers apply to all resource types unless explicitly scoped. This is the primary distinction from Service Layers. + +**Ownership:** DCM platform, infrastructure teams, or platform implementors. + +**Characteristics:** +- Applied to every request regardless of resource type +- Cannot contain service-specific or provider-specific data +- Carry location, organizational, and infrastructure context +- Stored in the Core Layer Store +- Cached in the Service Layer Cache at deployment time + +**Examples:** +- Data Center layer (DC1, DC2) +- Zone layer (Zone 1, Zone 2) +- Rack layer +- Geographic region layer +- Environment layer (production, staging, development) + +--- + +### 3.3 Intermediate / Customization Layers + +**Purpose:** Provide organizational or contextual overrides and customizations that sit between the base standards and the service-specific configuration. These layers encode the organizational hierarchy and deployment context. + +**Scope:** Can be type-agnostic or type-scoped. Scope is declared per layer. + +**Ownership:** Organizational teams, domain owners, platform implementors. + +**Characteristics:** +- Stack between Core Layers and Service Layers in the precedence chain +- Encode organizational structure (business unit, enclave, logical unit) +- Allow organizational customization without modifying base standards +- The Git repo hierarchy typically mirrors the intermediate layer hierarchy + +**Examples:** +- Ship layer (in Navy context: specific vessel configuration) +- Enclave layer (isolated network segment configuration) +- Business unit layer +- DMZ customization layer +- Production web tier layer + +--- + +### 3.4 Service Layers + +**Purpose:** Provide service-specific data required to build a complete request payload for a specific Resource Type. Service Layers are the bridge between general organizational context and provider-ready configuration. + +**Scope:** **Must be type-scoped.** A Service Layer without a declared Resource Type scope is invalid. The scope inheritance behavior is configurable per Service Layer declaration. + +**Type Scope Declaration:** +```yaml +type_scope: + resource_type_uuid: + resource_type_fully_qualified_name: + scope_inheritance: + # exact: applies only to the declared Resource Type + # descendants: applies to the declared Resource Type and all child types via inheritance +``` + +**Ownership:** Service Providers or service domain teams. Stored in Service Layer SCM (source control management). Registered with DCM as part of Service Provider registration. + +**Characteristics:** +- Only applied when the request resource type matches the layer's declared type scope +- Carry service-specific configuration, defaults, and constraints +- Must not contain provider-specific data unless marked as portability-breaking +- Cached in the Service Layer Cache at Service Provider registration time + +**Examples:** +- VM sizing layer (small, medium, large configurations for `Compute.VirtualMachine`) +- Web server configuration layer for `Compute.VirtualMachine` +- Network port configuration layer for `Network.Port` +- CL Web Service Data Layer for `Compute.VirtualMachine` (exact scope) +- General compute placement layer for `Compute.VirtualMachine` and descendants + +--- + +### 3.5 Request Layer + +**Purpose:** Carries the consumer's declared intent. The Request Layer is what the consumer provides โ€” the fields they explicitly specify for their resource request. + +**Scope:** Scoped to the Resource Type the consumer is requesting. + +**Ownership:** Consumer (via Web UI or Consumer API). + +**Characteristics:** +- Created at the time the consumer submits a request +- Contains only what the consumer explicitly declares โ€” it does not need to be complete +- The gap between what the consumer declares and what the provider needs is filled by the lower layers in the chain +- Has higher precedence than all data layers below it โ€” consumer-declared values override layer defaults +- Is the direct source of the **Intent State** โ€” the Request Layer as submitted by the consumer is stored in the Intent Store before any processing occurs +- After assembly and policy processing, the enriched payload becomes the **Requested State** + +**Examples:** +- Consumer requests a VM with `cpu_count: 8`, `ram_gb: 32`, `os: RHEL9`, `environment: production` +- Consumer requests a firewall rule with source/target network and port + +--- + +### 3.6 Policy Layers + +**Purpose:** Policy Layers are not data layers in the traditional sense โ€” they do not add fields to the merge chain. Instead, they operate on the assembled payload after the data layers have been merged. They are the governance layer of the assembly process. + +**Scope:** Scoped by policy type and domain. Core Policies apply to all requests. Service Policies apply to specific Resource Types. Organizational and domain policies apply to specific organizational scopes. + +**Ownership:** Policy creators, security teams, compliance teams, organizational domain owners. + +**Policy Layer Types and Their Behavior:** + +| Policy Type | Behavior | Precedence Effect | +|-------------|----------|-------------------| +| **Validation** | Checks data against rules. Does not modify data. Returns pass/fail. If fail, request is rejected. | No precedence โ€” pass/fail only | +| **Transformation** | Enriches or modifies data in the payload. Adds missing fields, applies standards, fills gaps. | Adds to or modifies the assembled payload โ€” recorded in provenance | +| **GateKeeper** | Highest authority. Can override any field regardless of what was declared in lower layers or the Request Layer. Can halt execution entirely. Used for sovereignty constraints, security mandates, and hard compliance rules. | Overrides everything โ€” including consumer input | + +**Characteristics:** +- Policies operate only on the policy definition, core data, and the data in the request payload +- Policy outcomes are deterministic โ€” same input always produces same output for a given policy version +- All policy modifications are recorded in field-level provenance with policy UUID, operation type, and reason +- Policies are versioned using the universal versioning scheme +- Policies are maintained via GitOps practices + +--- + +## 4. Layer Identity โ€” Domain, Handle, and Priority + +Every layer has a formal identity model with three components that together make it uniquely identifiable, locatable, and orderable within DCM. + +### 4.1 Layer Domain + +The **Layer Domain** is the organizational and architectural home of a layer. It declares ownership, storage location, and authorization scope โ€” who can create and modify this layer, and which other layers it can override. + +| Domain | Meaning | Authorization | Can Override | +|--------|---------|--------------|-------------| +| `system` | DCM built-in layers, shipped with DCM | DCM maintainers only | Nothing above system | +| `platform` | Platform team layers, apply across all tenants | Platform team | tenant, service, provider | +| `tenant` | Tenant-specific layers | Tenant Admin | service, provider within tenant | +| `service` | Service Provider contributed layers | Service Provider owner | provider | +| `provider` | Provider Catalog Item layers | Provider owner | Nothing above provider | + +A lower-domain layer cannot override a higher-domain layer. A `tenant` layer cannot override a `platform` layer. This is enforced at ingestion โ€” the conflict detection pipeline checks domain authority before allowing a merge. + +### 4.2 Layer Handle + +The **Layer Handle** is the human-readable, stable identifier for a layer within DCM. It complements the UUID (machine-meaningful) with a reference that humans can use in conversation, documentation, policy declarations, and audit records. + +**Format:** `{domain}/{layer_type}/{name}` + +**Examples:** +``` +platform/core/cis-benchmark-linux +platform/core/security-cpu-limits +tenant/service/payments-vm-standards +system/base/universal-defaults +service/service/kubevirt-vm-defaults +provider/service/cloudnativepg-database-config +``` + +**Rules:** +- Unique within DCM โ€” enforced at ingestion +- Stable across versions โ€” the handle does not change when a new version is published +- URL-safe characters only โ€” lowercase, hyphens, forward slashes +- The Git file path mirrors the handle structure exactly + +**Git path from handle:** +``` +{layer_store_root}/{domain}/{layer_type}/{name}/v{Major}.{Minor}.{Revision}.yaml + +# Example: +dcm-layers/platform/core/security-cpu-limits/v1.2.0.yaml +dcm-layers/tenant/{tenant-uuid}/service/payments-vm-standards/v1.0.0.yaml +``` + +### 4.3 Priority Schema + +The **Priority Schema** is the deterministic ordering mechanism for resolving conflicts between layers of the same type and scope. It uses a hierarchical dotted-notation system that supports infinite differentiation โ€” new priority values can always be inserted between any two existing values. + +**Format:** `{integer}.{integer}.{integer}...` โ€” unlimited depth + +**Comparison:** Left-to-right, segment by segment. Lower numeric value = higher priority. + +``` +100.10 beats 200.10 (100 < 200 at segment 1) +200.10 beats 200.20 (10 < 20 at segment 2) +200.10.5 beats 200.10 (longer path with matching prefix โ€” 5 at segment 3) +200.10.5 beats 200.10.10 (5 < 10 at segment 3) +``` + +**Infinite insertability:** Between `200.10` and `200.20` insert `200.15`. Between `200.10.5` and `200.10.10` insert `200.10.7`. There is no practical limit โ€” depth can always be extended. + +**Priority Label:** Semantic context for the numeric value โ€” human-readable, does not affect ordering. + +**Reference Priority Taxonomy (advisory โ€” not enforced by DCM):** + +DCM ships a reference taxonomy as documentation and starter templates. Organizations adopt, adapt, or ignore it โ€” DCM resolves conflicts purely by numeric comparison. The taxonomy is a suggested convention, not a system constraint. + +| Suggested Range | Category | Rationale | +|-----------------|----------|-----------| +| `100.*` | Compliance | Regulatory mandates | +| `200.*` | Security | Security standards | +| `300.*` | Sovereignty | Data residency constraints | +| `400.*` | Operations | SRE and operational standards | +| `500.*` | Platform | Platform-level defaults | +| `600.*` | Service | Service-specific configuration | +| `700.*` | Organization | Organizational defaults | +| `800.*` | Site | Location-specific overrides | +| `900.*` | Custom | Implementor-defined | + +Lower number = higher priority. An organization that prioritizes sovereignty above security would simply swap their `200.*` and `300.*` ranges. + +**Priority in a layer definition:** + +```yaml +priority: + value: "200.30.10" + label: "security.container.cpu_limits" + category: security + rationale: > + CPU limit enforcement for container workloads per + CISO mandate SEC-2024-047. Overrides platform defaults + to ensure no container can exceed approved limits. +``` + +--- + +## 4b. Artifact Metadata Standard + +Every DCM artifact โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions, and all other defined or stored objects โ€” carries a standard **Artifact Metadata** block. This is a structural requirement, not optional. + +The artifact metadata block answers: **who created this, when, who owns it, what changed, and how do we contact them?** + +### 4b.1 Universal Artifact Metadata Structure + +```yaml +artifact_metadata: + + # Identity + uuid: + handle: + + # Versioning + version: + status: + + # Status detail โ€” populated per status + status_detail: + # When status: proposed + proposed_at: + proposed_by: + uuid: + display_name: + email: + shadow_execution: + enabled: + started_at: + evaluation_count: + validation_dashboard_url: + + # When status: deprecated + deprecated_at: + deprecated_by: + uuid: + display_name: + replacement_uuid: + replacement_handle: + deprecation_reason: + migration_guidance: + sunset_date: + + # When status: retired + retired_at: + retired_by: + uuid: + display_name: + + # Origination + created_by: + uuid: + display_name: + email: + notification_endpoint: + created_at: + created_via: + # pr: submitted via GitOps PR workflow โ€” full review history available + # api: submitted via direct API + # migration: imported from external system โ€” provenance depth may be limited + # system: created by DCM itself (entity stubs, system artifacts) + + # Ownership โ€” may differ from creator + owned_by: + uuid: + display_name: + email: + notification_endpoint: + # Note: created_by is the audit record (who physically submitted it) + # owned_by is the accountability record (who is responsible and gets notified) + + # Modification history โ€” append-only + modifications: + - sequence: 1 + modified_by: + uuid: + display_name: + email: + modified_at: + modification_type: + version_before: + version_after: + change_summary: + pr_reference: + reason: +``` + +### 4b.2 The Five Artifact Statuses + +| Status | Meaning | Executes? | Output Applied? | Output Captured? | Merges to Active? | +|--------|---------|-----------|----------------|-----------------|------------------| +| `developing` | In active development. Development mode / dev pipeline only. | Dev mode only | No | Dev logs only | No โ€” must transition to proposed first | +| `proposed` | Development complete. Submitted for validation. Shadow mode for policies. | Yes (shadow) | No | Yes โ€” validation report | Yes โ€” after review approval | +| `active` | Live and governing. Applied to all relevant requests. | Yes | Yes | Yes โ€” audit/provenance | N/A | +| `deprecated` | Being phased out. Replacement available. Works but warns. | Yes | Yes | Yes โ€” with deprecation warning | N/A | +| `retired` | End of life. Cannot be used. | No | No | No | No | + +**Status transition rules:** +``` +developing โ†’ proposed (author submits for review) +developing โ†’ retired (author abandons without proposing) +proposed โ†’ active (reviewers approve โ€” via PR merge or API approval) +proposed โ†’ developing (returned for rework) +active โ†’ deprecated (replacement available โ€” sunset date declared) +deprecated โ†’ retired (sunset date reached or manual retirement) +retired โ†’ (terminal โ€” no transitions out) +``` + +### 4b.3 Proposed Status โ€” Shadow Execution for Policies + +When a policy artifact is in `proposed` status, it runs in **shadow mode** against real request traffic: + +- Executes alongside active policies on every relevant request +- Output is captured in a `proposed_evaluation_record` โ€” what it would have done +- Output is **never applied** to the actual request +- Shadow output feeds the Validation Dashboard for reviewer analysis +- Policy authors can see aggregate impact before activation + +```yaml +# Shadow output record โ€” captured per real request evaluated +proposed_evaluation_record: + policy_uuid: + policy_version: + request_uuid: + tenant_uuid: + evaluated_at: + would_have_applied: + shadow_output: + would_have_rejected: + rejection_reason: + would_have_patched: + - field: + current_value: + would_have_set: + reason: + would_have_locked: + - field: + lock_type: + reason: + would_have_selected_provider: + impact_assessment: + category: + # none: policy would not have applied to this request + # low: minor enrichment only + # medium: significant field modifications + # high: would have rejected or locked critical fields + # critical: would have rejected or overridden consumer intent +``` + +### 4b.4 Contact Info โ€” Two Modes + +Contact information supports both IdP-backed and standalone deployments: + +**Mode 1 โ€” Identity Provider backed:** +The `uuid` field contains the DCM external entity reference UUID linking to an Identity.Person or Identity.Team in a registered Information Provider. The `display_name` is cached non-authoritatively for UI display. DCM can resolve the full identity record via the Information Provider on demand. + +**Mode 2 โ€” Standalone (no Identity Provider):** +The `uuid` field is absent. `display_name`, `email`, and `notification_endpoint` are the primary identity fields. DCM accepts and records these directly without external verification. This mode supports bootstrapping, air-gapped deployments, and organizations that have not yet registered an Identity Information Provider. + +Both modes are fully supported. An organization can start in standalone mode and migrate to IdP-backed mode by adding `uuid` fields to existing artifact metadata โ€” no other changes required. + +### 4b.5 Notifications from Artifact Metadata + +The `owned_by.notification_endpoint` is the target for all proactive DCM notifications about an artifact: + +| Event | Who Is Notified | +|-------|----------------| +| Layer conflict detected at ingestion | Owner of new layer AND owner of conflicting existing layer | +| Layer deprecated | Owners of all artifacts that reference the deprecated layer | +| Provider deregistered | Owners of all catalog items backed by that provider | +| Policy violation | Owner of the entity that violated the policy | +| Drift detected | Owner of the entity that drifted | +| Proposed policy shadow shows high/critical impact | Policy owner and designated reviewers | +| Artifact approaching sunset date | Artifact owner | + +--- + +## 4c. Conflict Detection at Ingestion + +Conflict detection runs at layer ingestion time โ€” not at request assembly time. This ensures all layers in DCM are conflict-free before they are ever used. + +### 4c.1 Ingestion CI Pipeline + +When a layer is committed to the Layer Store (Git branch created or updated): + +``` +Layer committed to Git branch + โ”‚ + โ–ผ +CI Pipeline fires automatically + โ”‚ + โ”œโ”€โ”€ 1. Schema validation + โ”‚ Is the layer well-formed per the layer schema? + โ”‚ Does it carry required artifact metadata? + โ”‚ Is the version correctly incremented? + โ”‚ + โ”œโ”€โ”€ 2. Handle validation + โ”‚ Is the handle unique in DCM? + โ”‚ Does the handle match the Git path? + โ”‚ Does the domain match the submitting actor's authorization? + โ”‚ + โ”œโ”€โ”€ 3. Scope validation + โ”‚ If type-scoped: do declared resource types exist in the registry? + โ”‚ Is the layer type consistent with the domain? + โ”‚ + โ”œโ”€โ”€ 4. Priority validation + โ”‚ Is the priority value in valid dotted-notation format? + โ”‚ Does the priority category match the domain advisory range? + โ”‚ (Warning only if category/domain mismatch โ€” not a block) + โ”‚ + โ”œโ”€โ”€ 5. Conflict detection + โ”‚ For each field in this layer: + โ”‚ Find all active layers of the same type and overlapping scope + โ”‚ Check if any declare the same field + โ”‚ If conflict found: + โ”‚ โ†’ Does the new layer declare a higher priority? โ†’ Allowed, documented + โ”‚ โ†’ Does the existing layer declare a higher priority? โ†’ Allowed, documented + โ”‚ โ†’ Neither declares priority? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ โ†’ Both declare equal priority? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ โ†’ Domain authority violation? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ + โ”‚ Conflict notification: + โ”‚ Posted as PR comment with: conflicting layer UUID, handle, owner + โ”‚ Both layer owners notified via notification_endpoint + โ”‚ + โ”œโ”€โ”€ 6. Deprecation reference validation + โ”‚ If status: deprecated โ€” does replacement UUID exist? + โ”‚ + โ””โ”€โ”€ 7. Result + All checks pass โ†’ PR approved for merge + Any check fails โ†’ PR blocked, detailed error comment posted +``` + +### 4c.2 Conflict Resolution Rules + +| Situation | Resolution | Action | +|-----------|-----------|--------| +| New layer and existing layer conflict, no priority on either | CONFLICT ERROR | PR blocked. Both owners notified. One must declare priority or remove the conflicting field. | +| New layer has higher priority (lower value) than existing | Allowed โ€” new layer wins | Documented in provenance. Warning posted if domain authority is unusual. | +| Existing layer has higher priority | Allowed โ€” existing layer wins | New layer is a lower-priority alternative. Documented. | +| Both layers have equal priority | CONFLICT ERROR | PR blocked. Priority must be differentiated. | +| New layer from lower domain overrides higher domain | CONFLICT ERROR | Domain authority violation. Platform cannot be overridden by service layer. | +| Priority category suggests domain mismatch | WARNING | PR comment posted, not blocked. Merge allowed but reviewers are notified. | + +### 4c.3 Pre-Validation of All Layers + +Because conflict detection runs at ingestion, all layers resident in DCM are pre-validated: + +- No two active layers of the same type and scope conflict without explicit priority resolution +- The assembly process never encounters an ambiguous merge โ€” all conflicts are resolved at definition time +- If a conflict is discovered after the fact (e.g., a new layer is activated that conflicts with an existing one that was already active when the new layer was ingested), the newer layer's ingestion pipeline should have caught this. A background validation job runs periodically to detect any edge cases. + +--- + +## 4d. Complete Layer Definition Structure + +Combining all elements โ€” identity, artifact metadata, scope, priority, and fields: + +```yaml +# Complete layer definition +layer: + # === ARTIFACT METADATA (universal โ€” required on all artifacts) === + artifact_metadata: + uuid: "layer-uuid-001" + handle: "platform/core/security-cpu-limits" + version: "1.2.0" + status: active + created_by: + uuid: "actor-uuid-001" # Optional โ€” present if IdP registered + display_name: "Jane Smith" + email: "jane.smith@example.com" + notification_endpoint: "https://notify.example.com/webhooks/jane" + created_at: "2026-01-15T10:30:00Z" + created_via: pr + owned_by: + uuid: "team-uuid-security" # Optional โ€” present if IdP registered + display_name: "Platform Security Team" + email: "platform-security@example.com" + notification_endpoint: "https://notify.example.com/webhooks/platform-security" + modifications: + - sequence: 1 + modified_by: + display_name: "Jane Smith" + email: "jane.smith@example.com" + modified_at: "2026-01-15T10:30:00Z" + modification_type: create + version_before: null + version_after: "1.0.0" + change_summary: "Initial creation โ€” CPU limits per CISO mandate SEC-2024-047" + pr_reference: "https://github.com/org/dcm-layers/pull/42" + reason: "CISO mandate SEC-2024-047 requires CPU limits on all containers" + - sequence: 2 + modified_by: + display_name: "Bob Jones" + email: "bob.jones@example.com" + modified_at: "2026-02-20T14:00:00Z" + modification_type: update + version_before: "1.0.0" + version_after: "1.2.0" + change_summary: "Increased CPU limit from 4 to 8 per updated mandate" + pr_reference: "https://github.com/org/dcm-layers/pull/67" + reason: "Updated CISO mandate SEC-2024-047-rev2 allows 8 CPU" + + # === LAYER IDENTITY === + domain: platform + layer_type: core + + scope: + resource_types: + - Compute.Container + - Compute.Pod + # Empty list = type-agnostic (applies to all resource types) + + priority: + value: "200.30.10" + label: "security.container.cpu_limits" + category: security + rationale: > + CPU limit enforcement for container workloads per + CISO mandate SEC-2024-047. Overrides platform defaults. + + # === LAYER CHAIN === + parent_chain: + - uuid: "base-layer-uuid-001" + handle: "system/base/universal-defaults" + version: "1.0.0" + layer_type: base + + # === FIELDS === + fields: + cpu_limit: + value: 8 + metadata: + basis_for_value: "CISO mandate SEC-2024-047-rev2" + baseline_value: 4 + override: constrained + constraint_schema: + minimum: 1 + maximum: 8 +``` + +--- + +--- + +## 5. Precedence and Merge Rules + +When layers are merged to produce the assembled payload, fields from higher-precedence layers override fields from lower-precedence layers. The precedence order from lowest to highest is: + +``` +1. Base Layer (lowest precedence โ€” foundation defaults) +2. Core Layers (organizational and infrastructure context) +3. Intermediate/Customization (organizational hierarchy overrides) +4. Service Layers (service-specific configuration) +5. Request Layer (consumer intent โ€” overrides all data layers) +6. Transformation Policies (enrichment โ€” adds or modifies fields) +7. Validation Policies (pass/fail โ€” no field modification) +8. GateKeeper Policies (highest authority โ€” overrides everything) +``` + +### 5.1 Override Behavior + +- A higher-precedence layer that declares a field **overrides** the value from all lower-precedence layers +- A higher-precedence layer that does **not** declare a field leaves the lower-precedence value intact +- Fields not declared at any layer level are absent from the payload โ€” providers must declare all required fields as being covered by at least one layer in the chain +- GateKeeper policies can override **any** field including consumer-declared Request Layer values โ€” this is the mechanism for enforcing sovereignty constraints, security mandates, and hard compliance rules + +### 5.2 Additive vs. Override Fields + +Some fields are **scalar** (a single value โ€” one layer wins) and some are **additive** (a list or set โ€” layers contribute to a collection). The field type in the Resource Type Specification declares which behavior applies: + +```yaml +field_name: + type: + merge_behavior: + # override: higher precedence layer's value replaces lower precedence value + # additive: all layers contribute their values to a merged collection +``` + +### 5.3 Conflict Resolution + +When two layers at the same precedence level declare conflicting values for the same field: +- The conflict is recorded and surfaced as a validation error +- The request is not processed until the conflict is resolved +- Conflict resolution is never silent โ€” it is always recorded in provenance + +--- + +## 5a. Field Override Control + +Field override control is the mechanism by which DCM governs **who can change what, under what conditions**, across the layer precedence chain. It was present in the original data model rules as "override preference" metadata on fields โ€” this section formalizes that concept as a graduated model that is **simple by default and powerful when needed**. + +**Design Principle:** A field with no override declaration is fully overridable by anyone. Restrictions are always opt-in. The model has three levels โ€” you use only the level you need. Levels 1 and 2 cover the vast majority of real-world cases. Level 3 exists for fields that genuinely require nuanced, actor-specific governance. + +--- + +### 5a.1 Two Categories of Override Rule + +**Category 1 โ€” Structural Rules (Request Payload Processor โ€” non-overridable)** + +Enforced by the Request Payload Processor as DCM System behavior. Not configurable. Always applied: + +- A layer entity is immutable once versioned โ€” no override can modify a published version +- A child layer cannot remove a field declared in a parent layer โ€” it can only override the value +- The layer precedence order is fixed โ€” Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request โ†’ Policy +- Circular layer references are rejected unconditionally +- A Service Layer without a declared type scope is rejected unconditionally + +**Category 2 โ€” Business Rules (Policy Engine โ€” configurable)** + +Enforced by the Policy Engine using the Validation/Transformation/GateKeeper mechanism. Override control metadata is set exclusively by the Policy Engine and carried in the payload as part of field-level provenance. Data layers and the Request Payload Processor never set override control. + +--- + +### 5a.2 Where Override Control is Declared + +Override control can be declared at two static levels and applied dynamically at runtime: + +**Level A โ€” Resource Type Specification (portable, sets the ceiling)** +Declares the default override behavior for a field across all implementations of that Resource Type. These defaults travel with the type definition and apply to all providers and catalog items that implement the type. This sets the maximum permissiveness ceiling โ€” lower levels can only restrict further. + +**Level B โ€” Catalog Item (offering-specific, can only restrict)** +Declares additional restrictions for a specific curated offering beyond the Resource Type defaults. A "PCI Production VM" catalog item can lock `encryption_standard` to a single value even if the VM Resource Type allows a broader enum. Cannot expand beyond what the Resource Type permits. + +**Level C โ€” Policy Engine (runtime, within static bounds)** +Applies override control at request processing time based on current organizational policies. Can only restrict within the bounds established by the Catalog Item (or Resource Type if no Catalog Item restriction exists). Higher-authority policy levels (Global) can grant expansion to trusted actors within their authority scope. + +**Inheritance Rule:** Override control can only be made more restrictive as it flows down the declaration hierarchy โ€” Resource Type โ†’ Catalog Item โ†’ Runtime Policy. The sole exception is explicit trusted grants made by higher-authority actors (see Section 5a.6). + +--- + +### 5a.3 Level 1 โ€” No Declaration (Default) + +No override control declaration on a field means it is fully overridable by any actor. This is the default for all fields. Zero configuration required. + +```yaml +# Level 1 โ€” fully overridable, no declaration needed +cpu_count: + value: 4 +``` + +This covers the majority of fields in most implementations. + +--- + +### 5a.4 Level 2 โ€” Simple Declaration + +A single `override` property covers the most common governance needs without requiring a full matrix. Sufficient for most governed fields. + +```yaml +# Level 2a โ€” nobody can change this +sovereignty_zone: + value: us-east + override: immutable + +# Level 2b โ€” anyone can change but only within these values +encryption_standard: + value: AES-256 + override: constrained + constraint_schema: + enum: [AES-256, AES-128] + +# Level 2c โ€” explicit allow (same as default, but self-documenting) +display_name: + value: my-vm + override: allow +``` + +| Value | Meaning | Enforcement | +|-------|---------|-------------| +| `allow` | Default. Any actor may override. | Structural rules | +| `constrained` | Any actor may override within `constraint_schema` | Policy Engine โ€” Validation | +| `immutable` | No actor may override at any level | Policy Engine โ€” GateKeeper | + +--- + +### 5a.5 Level 3 โ€” Matrix Declaration + +Full actor-level control for fields that require nuanced governance. Used only when Level 2 is insufficient. + +```yaml +billing_tag: + value: engineering + override_matrix: + default: allow + # Default permission for any actor not explicitly listed + # Options: allow | constrained | deny + + inheritance: restrict_only + # Catalog Items and lower-level declarations can only restrict + # Higher-authority actors can grant expansion via trusted_grants + + actors: + - actor: policy.global + permission: allow + can_expand: true + # Global policies can always override and can grant expansion + # to lower actors via trusted_grants + + - actor: policy.tenant + permission: allow + can_expand: true + # Tenant policies can override and grant within global ceiling + + - actor: policy.user + permission: deny + can_expand: false + # User policies cannot override and cannot grant to others + + - actor: consumer_request + permission: constrained + constraint_schema: + pattern: "^[a-z0-9-]+$" + can_expand: false + # Consumers can override within pattern, cannot grant expansion + + - actor: process_resource + permission: deny + can_expand: false + # Automation denied by default โ€” grant via trusted_grants + + - actor: provider + permission: deny + can_expand: false + # Providers cannot modify this field + + - actor: sre_override + permission: allow + can_expand: false + # SREs have operational authority but cannot grant to others + + - actor: admin_override + permission: allow + can_expand: true + # Admins can override and grant within their scope level + + trusted_grants: + # Explicit expansion grants from higher-authority actors + # Used when an actor needs more permission than their default + - granted_to_uuid: + actor_type: process_resource + permission: allow + granted_by_policy_uuid: + reason: Patching automation trusted to update billing_tag + expires: + + constraint_schema: + pattern: "^[a-z0-9-]+$" + # Applied to all actors with permission: constrained +``` + +--- + +### 5a.6 Actor Registry + +The actor list is extensible. DCM ships with built-in actors. Organizations register custom actors following the same model. Custom actors default to `deny` until explicitly granted permissions. + +**Built-in actors:** + +| Actor | Default Scope | Can Expand | Notes | +|-------|--------------|------------|-------| +| `policy.global` | All tenants | โœ… | Highest authority โ€” can grant to any actor | +| `policy.tenant` | Single tenant | โœ… | Within global ceiling | +| `policy.user` | Single user | โŒ | Can only restrict | +| `consumer_request` | Request submitter | โŒ | Can only restrict | +| `process_resource` | Automation execution | โŒ by default | Requires trusted grant | +| `provider` | Service Provider | โŒ | Can only restrict | +| `sre_override` | SRE team | โŒ | Operational authority, cannot grant | +| `admin_override` | DCM Admin | โœ… | Within their scope level | + +**Custom actor registration:** + +```yaml +custom_actor: + uuid: + name: + description: + registered_by_tenant_uuid: + default_permission: deny + # Custom actors always default to deny until explicitly granted + can_expand: false + # Custom actors cannot expand by default โ€” requires explicit grant + version: + status: + provenance: + +``` + +Custom actors follow the universal versioning and deprecation model. A custom actor registered at Tenant scope cannot be granted Global-level authority. + +--- + +### 5a.7 Expansion Rules + +Actor expansion follows a strict hierarchy: + +- **`policy.global`** and **`admin_override`** at global scope โ€” can grant expansion to any actor for any field, including fields declared `immutable` at lower levels +- **`policy.tenant`** and **`admin_override`** at tenant scope โ€” can grant expansion within their tenant, cannot expand beyond what Global permits +- **`policy.user`**, **`consumer_request`**, **`provider`** โ€” can never grant expansion regardless of what they receive +- **`sre_override`** โ€” can never grant expansion but can be granted expansion by Tenant or Global +- **`process_resource`** โ€” denied by default, can be granted expansion by Tenant or Global via `trusted_grants` +- **Custom actors** โ€” denied by default, can be granted expansion by the level that registered them or higher + +**Trusted grants expire** โ€” if an `expires` timestamp is set, the grant is automatically revoked at that time. Expired grants are retained in provenance for audit purposes but are no longer applied. + +--- + +### 5a.8 Override Control in the Assembly Process + +Override control is applied during Step 5 (Policy Processing) of the assembly process: + +``` +Layer Merge complete (Steps 1-4) + โ”‚ Fields have values โ€” all fields default to Level 1 (allow) + โ”‚ Static override declarations from Resource Type and Catalog Item are loaded + โ–ผ +Transformation Policies + โ”‚ May set override: constrained or override_matrix on fields + โ”‚ May set baseline_value and basis_for_value metadata + โ”‚ Records policy UUID, level, and reason in field provenance + โ–ผ +Validation Policies + โ”‚ Verify existing override declarations are not violated + โ”‚ Verify actor permissions against current override_matrix + โ”‚ Pass/fail โ€” no modification to override control + โ–ผ +GateKeeper Policies + โ”‚ May set override: immutable on fields + โ”‚ May override field values before locking + โ”‚ May issue trusted_grants to specific actors + โ”‚ Records policy UUID, level, lock type, and reason in provenance + โ–ผ +Requested State + โ”‚ All governed fields carry full override control metadata + โ”‚ Provenance chain complete โ€” every lock and grant is traceable + โ–ผ +``` + +--- + +### 5a.9 Override Control and Rehydration + +During rehydration, the Intent State is replayed through the **current** Policy Engine. Override control declared in current policies is applied fresh. A field that was `allow` in the original request may be `immutable` if a new GateKeeper policy was added since. This is by design โ€” rehydration applies current governance standards, not historical ones. + +The original consumer intent is preserved unchanged in the Intent Store. The new realized state reflects current governance. Both are auditable and traceable. + +--- + +### 5a.10 Override Control Metadata โ€” Full Structure + +The complete field metadata structure carrying override control in the payload: + +```yaml +field_name: + value: + metadata: + # Simple declaration (Level 2) โ€” set by Policy Engine at runtime + override: + # OR matrix declaration (Level 3) โ€” set by Policy Engine at runtime + override_matrix: + + + # Always present regardless of level + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + constraint_schema: + + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_uuid: + operation_type: + actor: + timestamp: + reason: +``` + +--- + +The Request Payload Processor assembles the final payload by executing the following steps in order. Each step is recorded in the payload's provenance chain. + +### Step 1 โ€” Intent Capture +The consumer's Request Layer is received and stored as the **Intent State** in the Intent Store. No modification occurs at this step. The Intent State is the immutable record of what the consumer asked for. + +### Step 2 โ€” Layer Resolution +The Request Payload Processor determines which layers apply to this request: +- Identifies the Resource Type from the Request Layer +- Retrieves the applicable Base Layer for the request context +- Retrieves all applicable Core Layers (type-agnostic โ€” all apply) +- Retrieves applicable Intermediate/Customization Layers based on organizational context +- Retrieves applicable Service Layers whose declared type scope matches the request Resource Type +- Orders all retrieved layers according to the precedence chain + +### Step 3 โ€” Layer Merge +Layers are merged in precedence order (lowest to highest). For each field: +- The value from the highest-precedence layer that declares it is used +- The source layer UUID and layer type are recorded in the field's provenance metadata +- Additive fields accumulate values from all layers that declare them + +### Step 4 โ€” Request Layer Application +The consumer's Request Layer is applied last in the data layer merge. Consumer-declared values override all data layer values. Each override is recorded in provenance. + +### Step 5 โ€” Policy Processing +Policies are applied to the merged payload in order. Each policy step may read and set `override_control` metadata on fields โ€” see Section 5a for full detail. +1. **Transformation Policies** โ€” enrich and modify the payload. May set `override_preference: constrained` on fields, declaring that future overrides must satisfy a constraint schema. Each transformation records the policy UUID, operation type, reason, and any override control declarations in the affected fields' provenance. +2. **Validation Policies** โ€” check the payload against rules and verify that existing `override_preference` declarations have not been violated. Failures reject the request with a detailed reason. No field modification occurs. +3. **GateKeeper Policies** โ€” apply hard overrides and blocks. May set `override_preference: immutable` on fields, permanently locking them against further modification. GateKeeper overrides record the policy UUID, the overridden value, the new value, the lock type, and the reason in provenance. + +### Step 6 โ€” Requested State Storage +The fully assembled, policy-processed payload is stored as the **Requested State** in the Request Store. This is the complete, provider-ready payload with full provenance chain intact. + +### Step 7 โ€” Provider Dispatch +The Requested State payload is dispatched to the appropriate Service Provider via the API Gateway. + +--- + +## 7. Layer Assembly Diagram + +``` +Consumer Request + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REQUEST LAYER โ”‚ โ† Consumer declared intent โ†’ stored as INTENT STATE +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ LAYER RESOLUTION โ”‚ +โ”‚ โ”‚ +โ”‚ Base Layer (lowest precedence) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Core Layers (type-agnostic context) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Intermediate Layers (organizational context) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Service Layers (type-scoped service config) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Request Layer (consumer intent โ€” highest โ”‚ +โ”‚ data layer precedence) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Merged payload with full provenance + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ POLICY PROCESSING โ”‚ +โ”‚ โ”‚ +โ”‚ Transformation Policies (enrich / modify) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ Validation Policies (pass / fail check) โ”‚ +โ”‚ โ†“ โ”‚ +โ”‚ GateKeeper Policies (override / block) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Complete, validated, policy-processed payload + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REQUESTED STATE โ”‚ โ† Stored in Request Store +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ–ผ + Service Provider +``` + +--- + +## 8. Layer Scope and Type Enforcement + +### 8.1 Core Layer Scope Enforcement +Core Layers are type-agnostic by default. They are applied to every request regardless of Resource Type. A Core Layer that contains service-specific or provider-specific data is invalid and must be rejected. + +### 8.2 Service Layer Scope Enforcement +Service Layers must declare a Resource Type scope. The Request Payload Processor enforces this during Layer Resolution: +- A Service Layer whose declared Resource Type does not match the request Resource Type is excluded from the merge +- A Service Layer with `scope_inheritance: exact` is only included if the request Resource Type exactly matches the declared type +- A Service Layer with `scope_inheritance: descendants` is included if the request Resource Type is the declared type or any descendant type in the inheritance hierarchy +- A Service Layer with no declared type scope is invalid and must be rejected + +### 8.3 Unanticipated Data Interaction Prevention +The type scoping rules for Service Layers are the primary mechanism for preventing unanticipated data interactions โ€” one of the core data model objectives. Because Service Layers can only contribute to requests of their declared type, data from one service domain cannot inadvertently affect requests in another service domain. + +--- + +## 9. Layer Versioning + +All layers follow the universal DCM versioning scheme: **Major.Minor.Revision** + +| Component | Trigger | +|-----------|---------| +| **Major** | Breaking changes โ€” removing fields, changing field types, changing a field from optional to required | +| **Minor** | Additive changes โ€” adding new optional fields, adding new contextual data | +| **Revision** | Data/configuration changes โ€” updating field values, updating descriptions, updating metadata | + +**Immutability:** Once a layer version is published it cannot be modified. Any change produces a new version. Previous versions remain accessible and can be referenced by existing realized entities. + +**Parent Chain Versioning:** A layer's parent chain references specific versions of parent layers. Updating a parent layer does not automatically update child layers โ€” child layers must be explicitly updated to reference the new parent version, producing a new version of the child layer. + +--- + +## 10. Artifact Lifecycle โ€” The Five Statuses + +All DCM artifacts โ€” layers, policies, resource types, catalog items, and all other defined objects โ€” follow a five-status lifecycle. The statuses are defined in Section 4b.2. For layers specifically: + +| Status | Layer Behavior | +|--------|---------------| +| `developing` | Layer is in active development. Only usable in development mode pipelines. Not loaded by the assembly process in production. | +| `proposed` | Layer has been submitted for review (PR open). Not yet active. For policy layers: shadow execution runs. For data layers: layer is visible in the registry but not applied. Cannot merge to active until PR is approved. | +| `active` | Layer is current and applied in assembly. Can be included in new layer chains. | +| `deprecated` | Layer is being phased out. Existing chains using it continue to function. New chains should use the replacement. Deprecation warning recorded in assembly provenance. Must include replacement UUID, reason, migration guidance, and sunset date. | +| `retired` | Layer cannot be included in new layer chains. Existing realized entities that reference it retain the reference for audit purposes but cannot be used for new requests. | + +**Status transition rules for layers:** +``` +developing โ†’ proposed (author submits PR) +developing โ†’ retired (author abandons) +proposed โ†’ active (PR merged โ€” approval complete) +proposed โ†’ developing (PR returned for rework) +active โ†’ deprecated (replacement available โ€” sunset declared) +deprecated โ†’ retired (sunset date reached or manual retirement) +``` + +--- + +## 11. Scale Example โ€” 40,000 Linux VMs + +This example illustrates the power of the layering model at scale. 40,000 distinct VM configurations are governed by 36 layer definitions: + +``` +Base Entity (3 variants) +โ”œโ”€โ”€ CIS Benchmark +โ”œโ”€โ”€ Baseline +โ””โ”€โ”€ DMZ / Payments + + โ””โ”€โ”€ Layer Entity โ€” OS Family (3 variants per base = 9 total) + โ”œโ”€โ”€ Common Linux Config / RHEL + โ”œโ”€โ”€ Common Linux Config / CoreOS + โ””โ”€โ”€ Common Linux Config / OEL + + โ””โ”€โ”€ Layer Entity โ€” OS Version (4 variants per OS layer = 36 total) + โ”œโ”€โ”€ RHEL 6 + โ”œโ”€โ”€ RHEL 7 + โ”œโ”€โ”€ RHEL 8 + โ””โ”€โ”€ RHEL 9 + + โ””โ”€โ”€ Realized Entity โ€” one per VM (40,000 total) + Each realized entity carries FK references to its + full layer chain (Base UUID + Layer UUIDs) + and is stored in the CMDB +``` + +**Result:** 3 ร— 3 ร— 4 = **36 layer definitions** govern **40,000 VM configurations**. Each VM's realized entity is a lightweight reference to its layer chain โ€” not a copy of all the configuration data. + +This also means: +- Updating the CIS Benchmark base layer creates one new layer version that cascades to all 40,000 VMs at their next realization +- Drift detection compares each VM's discovered state against its realized entity's layer chain +- Any VM can be reproduced exactly by replaying its layer chain through the assembly process + +--- + +## 12. Relationship to the Four States + +| Layer | State Relationship | +|-------|-------------------| +| Request Layer (as submitted) | Directly captured as **Intent State** โ€” stored in Intent Store before any processing | +| Assembled payload (post-merge, pre-policy) | Intermediate โ€” not a named state, internal to assembly process | +| Assembled payload (post-policy) | Becomes **Requested State** โ€” stored in Request Store | +| Provider execution result | Becomes **Realized State** โ€” stored in Realized Store | +| Discovery interrogation result | Becomes **Discovered State** โ€” stored in Discovered Store | + +The layer chain of a Realized Entity is always traceable โ€” given a Realized State record, the complete layer chain that produced it can be reconstructed, providing full audit capability back to the original Base Layer. + +--- + +## 13. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are conflicting Service Layers at the same precedence level resolved? | Assembly determinism | โœ… Resolved โ€” priority schema + conflict detection at ingestion | +| 2 | Should Core Layers be ordered within their precedence level? | Merge determinism | โœ… Resolved โ€” priority schema provides deterministic ordering | +| 3 | Can a consumer explicitly exclude a layer from their request? | Consumer control vs. standardization | โ“ Unresolved | +| 4 | How are Service Layers registered and versioned relative to Service Provider registration? | Provider contract | โ“ Unresolved | +| 5 | Should assembly support conditional layer inclusion? | Assembly flexibility | โ“ Unresolved | +| 6 | How does the layer chain interact with service dependencies? | Dependency model | โ“ Unresolved | +| 7 | Should `override_preference` be declarable in layer definitions as a hint to the Policy Engine? | Override control | โ“ Unresolved | +| 8 | When `override_preference: immutable` is set by a Global policy, can a higher-priority Global policy still override it? | Override control precedence | โ“ Unresolved | +| 9 | Should the `constraint_schema` on a constrained field be visible to consumers in the Service Catalog UI? | Consumer experience | โ“ Unresolved | +| 10 | Should the background validation job for detecting post-ingestion conflicts run on a schedule or be event-triggered? | Operational | โ“ Unresolved | +| 11 | What is the minimum validation review period for a proposed policy before it can be activated? | Policy governance | โ“ Unresolved | + +--- + +## 14. Related Concepts + +- **Request Payload Processor** โ€” the control plane component that executes the assembly process; enforces structural layer rules +- **Policy Engine** โ€” executes Policy Layers (Validation, Transformation, GateKeeper) during the assembly process; the sole authority for setting field override control +- **Field Override Control** โ€” the mechanism governing who can change what field, under what conditions, at what policy level +- **Override Preference** โ€” per-field metadata declaring `allow`, `constrained`, or `immutable` โ€” the formalization of the original data model "override preference" subtag +- **Service Layer Cache** โ€” caches Service Layer data at Service Provider registration time for efficient retrieval during assembly +- **Core Layer Store** โ€” stores all Core Layer definitions +- **Intent State** โ€” the Request Layer as submitted, before assembly +- **Requested State** โ€” the fully assembled, policy-processed payload +- **Field-Level Provenance** โ€” every field in the assembled payload records which layer set it and which policy modified it +- **Resource Type Hierarchy** โ€” defines the type scope that Service Layers must declare and that the assembly process enforces +- **GitOps** โ€” all layers are stored in Git, versioned and immutable + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/resource-grouping.md b/content/docs/data-model/resource-grouping.md new file mode 100644 index 0000000..d4a1389 --- /dev/null +++ b/content/docs/data-model/resource-grouping.md @@ -0,0 +1,292 @@ +--- +title: Resource Grouping +type: docs +weight: 7 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) + +--- + +## 1. Purpose + +This document defines how Resource/Service Entities are organized into groups within DCM. Grouping provides the ownership, organizational context, cost attribution, policy scope, and rehydration targeting that makes DCM operationally meaningful at scale. + +Two concepts are defined here: +1. **DCM Tenant** โ€” the mandatory, first-class ownership boundary for all Resource/Service Entities +2. **Resource Groups** โ€” flexible, composable grouping entities that provide additional organizational context + +--- + +## 2. DCM Tenant + +### 2.1 Definition + +A **DCM Tenant** is the primary ownership and isolation boundary for Resource/Service Entities in DCM. Every Resource/Service Entity โ€” including Process Resources โ€” must belong to exactly one DCM Tenant at any point in time. + +Tenant membership is the answer to the question: **who owns this resource?** + +### 2.2 Tenant as a DCM System Policy + +Mandatory Tenant membership is a **non-overridable DCM System Policy**: + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `TEN-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation โ€” no Tenant = request rejected | +| `TEN-002` | Tenant membership cannot be empty โ€” a Tenant must exist before resources can be created in it | Enforced at request processing | +| `TEN-003` | A Resource/Service Entity cannot exist without a Tenant | Enforced at all lifecycle states | + +### 2.3 What Tenant Provides + +The Tenant boundary enables the following DCM capabilities for all resources it owns: + +| Capability | Description | +|------------|-------------| +| **Ownership** | Unambiguous answer to "who owns this resource" โ€” always answerable, always auditable | +| **Isolation** | Resources in one Tenant are isolated from resources in another โ€” hard tenancy enforcement | +| **Cost Attribution** | All resource costs roll up to the owning Tenant | +| **Policy Scope** | Tenant-level policies apply to all resources in the Tenant | +| **Drift Detection Scope** | Drift detection can be scoped to a Tenant | +| **Rehydration Scope** | A full Tenant can be targeted for rehydration | +| **Audit Scope** | All activity within a Tenant is auditable as a unit | +| **Sovereignty Boundary** | Sovereignty constraints can be applied at the Tenant level | + +### 2.4 Tenant Entity Definition + +```yaml +dcm_tenant: + uuid: + name: + description: + version: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + ownership: + owner_uuid: + owner_type: + created_timestamp: + membership_policy: + exclusive: true + # A resource belongs to exactly one Tenant + # This is non-overridable + sovereignty_constraints: + + policies: + + provenance: + +``` + +### 2.5 Tenant and Resource Consumption + +A resource belongs to exactly one Tenant โ€” its **owner**. However, a resource can be **consumed** by multiple Tenants via the DCM Service Catalog. Ownership and consumption are distinct: + +- **Ownership** (Tenant membership) โ€” who is responsible for the lifecycle, cost, and compliance of this resource +- **Consumption** โ€” who uses or depends on this resource as a service + +Cross-tenant consumption is tracked through service requests and cost attribution โ€” not through Tenant membership. A shared DNS service owned by a Platform Tenant can be consumed by any number of application Tenants. The DNS Entity belongs to the Platform Tenant. Consumption is tracked via service requests from each consuming Tenant. + +--- + +## 3. Resource Groups + +### 3.1 Definition + +A **Resource Group** is a flexible, composable grouping entity that provides organizational context, operational scope, and policy targeting beyond what Tenant membership provides. + +Resource Groups function like **structured tags** โ€” a resource accumulates group memberships that describe its context from multiple dimensions simultaneously. A VM could simultaneously belong to: +- `Deployment: WebApp-v2` (what deployment it is part of) +- `BusinessUnit: Payments` (which business unit owns the workload) +- `RegulatoryScope: PCI-DSS` (which compliance regime applies) +- `CostCenter: CC-4421` (where costs are attributed) + +Each group membership is a different dimension of context โ€” not a hierarchy within a single dimension. + +### 3.2 Resource Group Classes + +DCM defines two classes of Resource Group, both implementing the same **Resource Group Interface**: + +**Class 1 โ€” DCM Default Resource Group** +Built into DCM. The standard mechanism for grouping resources. No implementor customization required to use it. + +**Class 2 โ€” Custom Resource Group** +Implementor-defined grouping entities. Tied to internal business structures โ€” business units, product lines, regulatory scopes, cost centers, etc. Full parity with DCM Default Resource Groups in terms of DCM capabilities. + +Both classes implement the same interface. The DCM Default Resource Group is simply DCM's own implementation of the Resource Group Interface. Custom groups are implementor-defined implementations of the same interface. + +### 3.3 The Resource Group Interface + +Every Resource Group โ€” both DCM default and custom โ€” must implement this interface: + +```yaml +resource_group: + uuid: + name: + description: + group_class: + group_type: + version: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + nesting: + supported: + # If true, this group can contain other groups as members + max_depth: + membership: + members: + - member_uuid: + member_type: + joined_timestamp: + joined_by_uuid: + membership_policy: + exclusive: + # If true, a resource can only belong to one group of this type at a time + # If false, a resource can belong to multiple groups of this type + max_memberships: + # Maximum number of groups of this type a resource can belong to + allowed_entity_types: + + policies: + + provenance: + +``` + +### 3.4 Multi-Group Membership + +A Resource/Service Entity can belong to multiple Resource Groups across all classes. This multi-dimensional membership is what gives groups their tag-like flexibility. + +**Membership constraints are configurable per group definition:** +- A group can declare `exclusive: true` โ€” meaning a resource can only belong to one group of that type at a time +- Example: A `RegulatoryScope` group might declare `exclusive: true` โ€” a resource cannot be in both EU-GDPR and US-FISMA regulatory scopes simultaneously +- Example: A `Deployment` group might declare `exclusive: false` โ€” a resource could participate in multiple deployments + +**Policy-governed membership:** +Organizational policies can further restrict multi-group membership. For example, a sovereignty policy could declare that resources in a PCI-DSS scope cannot be in the same group as resources in a non-PCI scope. + +### 3.5 Nesting + +Resource Groups that declare `nesting: true` can contain other Resource Groups as members in addition to individual Resource/Service Entities. + +**Example nesting structure:** +``` +Tenant: Payments Platform + โ”‚ + โ””โ”€โ”€ Resource Group: Deployment โ€” WebApp-v2 (nesting: true) + โ”œโ”€โ”€ Resource Group: Service โ€” Frontend (nesting: true) + โ”‚ โ”œโ”€โ”€ Entity: Web Server VM 1 + โ”‚ โ”œโ”€โ”€ Entity: Web Server VM 2 + โ”‚ โ””โ”€โ”€ Entity: Load Balancer + โ””โ”€โ”€ Resource Group: Service โ€” Backend (nesting: true) + โ”œโ”€โ”€ Entity: App Server VM 1 + โ”œโ”€โ”€ Entity: App Server VM 2 + โ””โ”€โ”€ Entity: Database +``` + +**Nesting rules:** +- Circular nesting is invalid โ€” a group cannot contain itself directly or transitively +- Nesting depth is declared per group โ€” `max_depth: unlimited` allows arbitrary depth +- A child group inherits policy scope from parent groups โ€” policies applied to a parent group propagate to all child groups and their members +- Cost rollup propagates up the nesting hierarchy + +--- + +## 4. DCM System Policies for Resource Grouping + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `GRP-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation | +| `GRP-002` | A Resource/Service Entity cannot be removed from its Tenant without being transferred to another Tenant | Enforced at all lifecycle states | +| `GRP-003` | Circular nesting in Resource Groups is invalid | Enforced at group membership modification | +| `GRP-004` | Custom Resource Groups must implement the full Resource Group Interface | Enforced at group registration | +| `GRP-005` | Exclusive membership groups must reject membership requests that violate exclusivity | Enforced at group membership addition | + +--- + +## 5. Grouping and DCM Capabilities + +Resource Groups enable the following DCM capabilities at the group scope: + +| Capability | Tenant | Resource Group | +|------------|--------|---------------| +| Cost Attribution | โœ… Primary | โœ… Rollup within group | +| Policy Scope | โœ… | โœ… | +| Drift Detection Scope | โœ… | โœ… | +| Rehydration Scope | โœ… Full Tenant | โœ… Group and dependencies | +| Audit Scope | โœ… | โœ… | +| Placement Constraints | โœ… | โœ… | +| Sovereignty Boundary | โœ… | โœ… | + +--- + +## 6. Process Resources and Grouping + +Process Resource Entities follow the same grouping rules as Infrastructure Resource Entities: + +- Must belong to exactly one DCM Tenant โ€” non-overridable +- Can optionally belong to Resource Groups +- Typically grouped under the same Deployment or Service group as the resources they operate on +- Tenant membership ensures cost attribution for execution resources +- Group membership enables operational scoping โ€” "show me all automation jobs that ran against this Deployment" + +--- + +## 7. Custom Resource Group Registration + +Implementors register custom Resource Group types as part of their DCM implementation. Custom group types must declare their full interface implementation: + +```yaml +custom_group_type_registration: + uuid: + type_name: + version: + description: + implementing_organization_uuid: + interface_version: + default_membership_policy: + exclusive: + max_memberships: + nesting_supported: + allowed_entity_types: + status: + state: +``` + +--- + +## 8. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should there be a DCM-maintained registry of well-known custom group types to encourage standardization? | Interoperability | โ“ Unresolved | +| 2 | How does group membership interact with sovereignty โ€” can a group span sovereignty boundaries? | Sovereignty model | โ“ Unresolved | +| 3 | When a Tenant is decommissioned, what happens to its resources and group memberships? | Lifecycle management | โ“ Unresolved | +| 4 | Should Resource Groups support time-bounded membership โ€” a resource belongs to a group for a defined period? | Operational flexibility | โ“ Unresolved | +| 5 | How are group-level policies inherited by nested child groups โ€” is inheritance opt-in or opt-out? | Policy model | โ“ Unresolved | + +--- + +## 9. Related Concepts + +- **DCM Tenant** โ€” primary ownership boundary, mandatory for all entities +- **Resource/Service Entity** โ€” the thing being grouped +- **Policy Engine** โ€” enforces grouping system policies and evaluates group-level organizational policies +- **Cost Analysis** โ€” rolls up costs through group hierarchies +- **Drift Detection** โ€” can be scoped to a group +- **Rehydration** โ€” can target a group as the unit of reconstruction +- **Field-Level Provenance** โ€” group membership changes are recorded in entity provenance + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/resource-service-entities.md b/content/docs/data-model/resource-service-entities.md new file mode 100644 index 0000000..a7f0c34 --- /dev/null +++ b/content/docs/data-model/resource-service-entities.md @@ -0,0 +1,472 @@ +--- +title: Resource and Service Entities +type: docs +weight: 5 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) + +--- + +## 1. Purpose + +This document defines the two fundamental transactional concepts in DCM โ€” the **Resource/Service Request** and the **Resource/Service Entity** โ€” and establishes the ownership models, lifecycle principles, and provider relationship rules that govern them. + +Understanding the distinction between a Request and an Entity, and understanding DCM's role as the authoritative owner of all resource data regardless of operational ownership, is essential to understanding how DCM achieves its core goals of auditability, lifecycle management, and sovereignty. + +--- + +## 2. Core Terminology + +### 2.1 Resource/Service Request + +A **Resource/Service Request** is what a consumer submits to DCM โ€” the declared intent to consume a resource or service. It is the consumer side of the transaction. + +- Created when a consumer submits a request via the Web UI or Consumer API +- Captured as the **Intent State** before any processing +- Processed into the **Requested State** after assembly and policy validation +- Is the initiating event that causes a Resource/Service Entity to be created + +A Request is not a thing โ€” it is an **instruction**. It describes what the consumer wants. The provider acts on the Requested State to produce an Entity. + +### 2.2 Resource/Service Entity + +A **Resource/Service Entity** is the "thing" produced by a provider as a result of fulfilling a Resource/Service Request. It is the provider side of the transaction โ€” the allocation made real. + +- Created when a provider fulfills a Requested State payload +- Returned to DCM in unified data model format via Denaturalization +- Captured as the **Realized State** in the Realized Store +- Assigned to a **DCM Tenant** โ€” the ownership boundary +- Has a UUID, full provenance chain, and complete lifecycle from creation to decommission +- Is the unit of consumption, cost attribution, drift detection, and audit in DCM + +A Resource/Service Entity IS a thing โ€” it exists, it has state, it has an owner, and DCM manages its lifecycle. + +### 2.3 The Critical Distinction + +``` +Consumer submits โ†’ Resource/Service REQUEST โ†’ Intent/Requested State +Provider fulfills โ†’ Resource/Service ENTITY โ†’ Realized State +DCM manages lifecycle โ†’ ENTITY persists โ†’ Drift/Audit/Cost/Rehydration +``` + +--- + +## 3. DCM as Authoritative Owner of All Resource Data + +This is the most fundamental principle governing Resource/Service Entities: + +**DCM is ALWAYS the system of record for Resource/Service Entity data. DCM is ALWAYS authoritative for the resource definition. DCM ALWAYS owns the lifecycle. This applies regardless of the operational ownership model.** + +The operational ownership model (described in Section 4) determines who has authority to operate on a Resource/Service Entity. It does not affect DCM's data ownership. Specifically: + +- DCM owns the **data definition** of every Resource/Service Entity โ€” what it is, what it should be, what it was +- DCM owns the **lifecycle** โ€” from Requested through Realized to Decommissioned +- DCM is **authoritative** โ€” if a provider reports a change DCM was not aware of, DCM acts on it according to policy +- DCM acts as the **Tenant advocate** โ€” it protects the Tenant's interests in all provider interactions +- Providers are **custodians** of the underlying infrastructure โ€” they are not the system of record + +**When a provider reports an unsanctioned change:** + +If a provider reports a state change that was not initiated by a DCM request, the Policy Engine evaluates the change and determines the appropriate response: + +| Response | Description | +|----------|-------------| +| `ALERT` | Notify appropriate personas โ€” Tenant owner, SRE, Auditor | +| `REVERT` | Instruct provider to revert to DCM-declared realized state | +| `UPDATE_DEFINITION` | Accept the change and update the realized state definition | +| `INVESTIGATE` | Flag for human review before action | +| `DECOMMISSION` | Initiate decommission if the change represents unrecoverable deviation | +| `ESCALATE` | Escalate to higher policy tier for decision | + +The response is determined by Policy Engine evaluation against: +- The Resource/Service definition +- Service/Resource dependencies +- Consumer preferences +- Organizational and Tenant policies +- Sovereignty requirements + +--- + +## 4. Ownership Models + +DCM supports four ownership models for Resource/Service Entities. Every Provider Catalog Item must declare which ownership model(s) it supports. The ownership model is recorded in the Resource/Service Entity's provenance at creation time. + +### 4.1 Allocation Model + +The provider retains internal ownership of the underlying infrastructure. The consumer owns the Resource/Service Entity (the allocation) in their DCM Tenant. The provider can reclaim the underlying resource when the entity is decommissioned. + +**Characteristics:** +- Provider retains asset ownership +- Consumer owns the allocation โ€” the Entity in their DCM Tenant +- Provider has reclaim rights on decommission +- Underlying infrastructure may be shared or subdivided +- DCM manages the Entity lifecycle; provider manages the underlying resource + +**Examples:** Virtual Machine, Container, Network Port, IP Address, Firewall Rule, Database Instance + +--- + +### 4.2 Whole Allocation Model + +The entire physical or logical resource is allocated as a single indivisible unit to one consumer's DCM Tenant. The provider retains internal ownership but the consumer has exclusive use of the whole resource. The resource cannot be subdivided or shared during the allocation period. + +**Characteristics:** +- Provider retains asset ownership +- Consumer has exclusive, indivisible use +- The resource is not shared or subdivided +- Provider has reclaim rights on decommission +- DCM manages the Entity lifecycle + +**Examples:** Dedicated Bare Metal server (provider-owned), Dedicated Network appliance, Whole storage array allocation + +--- + +### 4.3 Full Transfer Model + +The provider transfers complete ownership of the underlying resource to the consumer's DCM Tenant. The Resource/Service Entity IS the resource โ€” there is no separation between the allocation and the underlying infrastructure from DCM's perspective. The consumer controls the full lifecycle including decommissioning. The provider has no reclaim rights after transfer. + +**Characteristics:** +- Ownership of the underlying resource transfers to consumer's DCM Tenant +- The Entity IS the resource โ€” no allocation/infrastructure separation +- Consumer controls full lifecycle including decommission +- Provider has no reclaim rights post-transfer +- Transfer is recorded in provenance โ€” permanent audit record +- DCM remains authoritative for data and lifecycle regardless of transfer + +**Examples:** Transferred Bare Metal server, Licensed software asset, Dedicated hardware appliance transferred to consumer + +--- + +### 4.4 Hybrid Transfer Model + +Ownership can transfer multiple times across the lifecycle of the Resource/Service Entity. The current owner is always exactly one DCM Tenant, but ownership can be formally reassigned through a DCM-governed ownership transfer process. Every transfer is tracked, auditable, and policy-governed. + +**Characteristics:** +- Ownership is held by exactly one DCM Tenant at any point in time +- Ownership can be transferred to another DCM Tenant through a formal DCM process +- Every transfer is recorded in the Entity's provenance chain โ€” complete ownership history +- Transfer requires Policy Engine validation and authorization +- The receiving Tenant must accept the transfer โ€” it cannot be forced +- DCM remains authoritative for data and lifecycle through all transfers + +**Transfer Provenance Record:** +```yaml +ownership_transfer: + sequence: + from_tenant_uuid: + to_tenant_uuid: + transfer_timestamp: + authorized_by: + transfer_reason: + policy_uuid: +``` + +**Examples:** Bare Metal server reallocated between tenants, Hardware asset transferred between business units, Licensed resource reassigned + +--- + +### 4.5 Ownership Model Declaration + +Every Provider Catalog Item must declare the ownership model(s) it supports: + +```yaml +catalog_item: + uuid: + ownership_models_supported: + - allocation + - whole_allocation + - full_transfer + - hybrid_transfer + default_ownership_model: + transfer_policy_required: + # If true, a policy must be referenced in any transfer request +``` + +--- + +## 5. Resource/Service Entity Lifecycle + +Every Resource/Service Entity progresses through a defined lifecycle. The lifecycle states are: + +``` +REQUESTED โ†’ PENDING โ†’ PROVISIONING โ†’ REALIZED โ†’ OPERATIONAL + โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ โ–ผ + DEGRADED MAINTENANCE SUSPENDED + โ”‚ โ”‚ โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ–ผ + DECOMMISSIONING + โ”‚ + โ–ผ + DECOMMISSIONED +``` + +| State | Description | +|-------|-------------| +| `REQUESTED` | Request submitted, Intent State captured | +| `PENDING` | Requested State assembled, awaiting provider dispatch | +| `PROVISIONING` | Provider is fulfilling the request | +| `REALIZED` | Provider has fulfilled the request, Entity exists, Realized State captured | +| `OPERATIONAL` | Entity is in active use | +| `DEGRADED` | Entity is functioning but below expected operational characteristics | +| `MAINTENANCE` | Entity is undergoing planned maintenance | +| `SUSPENDED` | Entity is temporarily suspended โ€” not operational but not decommissioned | +| `DECOMMISSIONING` | Decommission process initiated | +| `DECOMMISSIONED` | Entity no longer exists. Record retained permanently for audit. | + +**Terminal states:** `DECOMMISSIONED` is the only terminal state. Once decommissioned, the Entity record is immutable and retained permanently. + +--- + +## 6. Process Resource Entities + +A **Process Resource Entity** is a distinct class of Resource/Service Entity representing ephemeral execution resources โ€” automation jobs, playbooks, pipelines, workflows, and similar process-oriented resources. + +### 6.1 Characteristics + +- **Ephemeral lifecycle** โ€” exists for the duration of execution, then terminates +- **No ongoing realized state to manage** โ€” lifecycle ends at COMPLETED or FAILED +- **Execution record retained permanently** โ€” the record of what the process did is immutable and permanent +- **Must belong to a DCM Tenant** โ€” even ephemeral resources must be owned +- **Must be in the provenance chain** of any Resource/Service Entity they affect + +### 6.2 Process Resource Lifecycle + +``` +REQUESTED โ†’ INITIATED โ†’ EXECUTING โ†’ COMPLETED + โ†’ FAILED + โ†’ CANCELLED +``` + +| State | Description | +|-------|-------------| +| `REQUESTED` | Process request submitted | +| `INITIATED` | Provider has begun execution | +| `EXECUTING` | Process is actively running | +| `COMPLETED` | Process completed successfully โ€” terminal | +| `FAILED` | Process failed โ€” terminal | +| `CANCELLED` | Process cancelled before completion โ€” terminal | + +All terminal states are permanent. The execution record is immutable after reaching a terminal state. + +### 6.3 Process Resource Entity Data Model + +```yaml +process_resource_entity: + uuid: + entity_class: process + process_type: + tenant_uuid: + version: + lifecycle_state: + input_payload: + + output_payload: + + affected_entities: + - entity_uuid: + effect_type: + effect_description: + execution_record: + initiated_timestamp: + completed_timestamp: + executing_provider_uuid: + authorized_by_policy_uuid: + provenance: + +``` + +### 6.4 Provenance Obligation for Process Resources + +If a Process Resource modifies the state of a Resource/Service Entity, that Entity's realized state provenance MUST reference the Process Resource Entity UUID as the source of the modification. This ensures that every change to an Infrastructure Entity can be traced back to the Process that caused it. + +--- + +## 7. Provider Internal Lifecycle Model + +Providers have their own internal infrastructure that underpins the Resource/Service Entities they create. While that internal infrastructure is opaque to consumers, DCM needs visibility into it for placement, cost analysis, and operational governance. + +### 7.1 Provider Capacity Model + +DCM supports three capacity information modes. Mode 3 is mandatory for all providers. Modes 1 and 2 are configurable per provider registration. + +**Mode 1 โ€” Dynamic Query (on-demand)** +DCM queries the provider for current capacity as part of request processing. Used when real-time accuracy is critical or when the provider cannot maintain a registration schedule. + +```yaml +capacity_query_response: + provider_uuid: + resource_type_uuid: + location_uuid: + query_timestamp: + available_capacity: + reserved_capacity: + committed_capacity: + sovereignty_capabilities: +``` + +**Mode 2 โ€” Provider Registration (scheduled, preferred)** +Provider registers capacity data with DCM on a configurable schedule. DCM maintains an internal capacity rating per provider, per Resource Type, per location. Default minimum update frequency: twice daily. Update frequency is configurable per provider registration. + +```yaml +capacity_registration: + provider_uuid: + registration_timestamp: + next_scheduled_registration: + capacity_by_resource_type: + - resource_type_uuid: + location_uuid: + available_capacity: + reserved_capacity: + committed_capacity: + sovereignty_capabilities: +``` + +**Mode 3 โ€” Provider Denial (reactive, mandatory)** +The provider validates it can fulfill a request before executing. If it cannot, it denies the request with reason `INSUFFICIENT_RESOURCES`. DCM receives the denial and can retry with an alternative provider. The denial triggers an immediate update to DCM's internal capacity rating for that provider. + +```yaml +provider_denial: + provider_uuid: + request_uuid: + denial_reason: INSUFFICIENT_RESOURCES + denial_timestamp: + resource_type_uuid: + location_uuid: + estimated_available_at: +``` + +### 7.2 Provider Lifecycle Events + +Any provider event that affects Resource/Service Entity availability or operational characteristics MUST be reported to DCM immediately. Providers have a contractual obligation to report these events โ€” this is non-negotiable. + +**Reportable Event Types:** + +| Event Type | Description | DCM Response | +|------------|-------------|--------------| +| `CAPACITY_CHANGE` | Available capacity increased or decreased | Update internal capacity rating | +| `DEGRADATION` | Underlying resource is degraded | Policy Engine evaluation โ†’ ALERT/REVERT/ESCALATE | +| `MAINTENANCE_SCHEDULED` | Planned maintenance window declared | Policy Engine evaluation โ†’ notify, migrate if needed | +| `MAINTENANCE_STARTED` | Maintenance has begun | Update Entity state to MAINTENANCE | +| `MAINTENANCE_COMPLETED` | Maintenance completed | Restore Entity state, trigger drift detection | +| `UNSANCTIONED_CHANGE` | Change occurred that was not initiated by DCM | Policy Engine evaluation โ†’ REVERT/UPDATE/ALERT | +| `ENTITY_HEALTH_CHANGE` | Entity health status changed | Policy Engine evaluation | +| `PROVIDER_DEGRADATION` | Provider itself is degraded | Policy Engine evaluation โ†’ reroute new requests | +| `DECOMMISSION_NOTICE` | Provider is decommissioning underlying resource | Policy Engine evaluation โ†’ migrate or decommission Entity | + +**Event Payload Format:** +All provider lifecycle events must be reported in DCM unified data model format: + +```yaml +provider_lifecycle_event: + event_uuid: + event_type: + provider_uuid: + affected_entity_uuids: + - + event_timestamp: + event_details: + + severity: + requires_immediate_action: +``` + +**Maximum Reporting Latency:** +Providers must report lifecycle events within the timeframe declared in their provider registration. For CRITICAL severity events, immediate reporting is required. The reporting latency SLA is part of the Provider SLA/Operational Contract. + +### 7.3 DCM Capacity Rating + +DCM maintains an internal capacity rating per provider, per Resource Type, per location. This rating is used by the Policy Engine for placement decisions. + +```yaml +dcm_capacity_rating: + provider_uuid: + resource_type_uuid: + location_uuid: + last_updated: + update_source: + available_capacity: + capacity_confidence: + # high: updated within last scheduled window + # medium: updated within 2x scheduled window + # low: stale โ€” beyond 2x scheduled window + next_scheduled_update: +``` + +--- + +## 8. Entity Relationships + +Every Resource/Service Entity carries a `relationships` section declaring its relationships to other entities โ€” internal DCM entities, external data entities, and business context entities. The relationship model is universal โ€” the same structure is used for all relationship types. + +See [Entity Relationships](09-entity-relationships.md) for the complete relationship model. + +```yaml +resource_service_entity: + uuid: + # ... other entity fields ... + relationships: + - relationship_uuid: + this_entity_uuid: + this_role: + related_entity_uuid: + related_entity_type: + relationship_type: + nature: + lifecycle_policy: + on_related_destroy: + on_related_suspend: + on_related_modify: + status: + provenance: + +``` + +--- + +## 9. DCM System Policies for Resource/Service Entities + +The following are **non-overridable DCM System Policies** that apply to all Resource/Service Entities: + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `RSE-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation โ€” no Tenant = request rejected | +| `RSE-002` | Every Resource/Service Entity must have a UUID | Enforced at Entity creation | +| `RSE-003` | Every Resource/Service Entity must have a complete provenance chain | Enforced at every state transition | +| `RSE-004` | Realized State payloads must be complete โ€” not a status code | Enforced at provider response receipt | +| `RSE-005` | Decommissioned Entity records are immutable and permanent | Enforced at decommission โ€” records cannot be deleted | +| `RSE-006` | Provider lifecycle events must be recorded in Entity provenance | Enforced at event receipt | +| `RSE-007` | Ownership transfers must be authorized by policy | Enforced at transfer initiation | +| `RSE-008` | Process Resource Entities must reference all affected Entity UUIDs | Enforced at process completion | + +--- + +## 10. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | For Hybrid Transfer โ€” what is the maximum number of ownership transfers allowed, or is it unlimited? | Operational complexity | โ“ Unresolved | +| 2 | For Whole Allocation of bare metal โ€” how is the indivisibility enforced at the provider level? | Provider contract | โ“ Unresolved | +| 3 | Should capacity confidence ratings trigger automatic actions (e.g., LOW confidence triggers a Mode 1 query)? | Capacity model | โ“ Unresolved | +| 4 | For Process Resources โ€” should there be a maximum execution time after which DCM escalates? | Operational governance | โ“ Unresolved | +| 5 | How does the SUSPENDED state interact with cost analysis โ€” is a suspended Entity still billable? | Cost model | โ“ Unresolved | + +--- + +## 11. Related Concepts + +- **DCM Tenant** โ€” the mandatory ownership boundary for all Resource/Service Entities +- **Four States** โ€” Intent, Requested, Realized, Discovered โ€” the state lifecycle of a Resource/Service Request and Entity +- **Field-Level Provenance** โ€” every state transition and ownership transfer is recorded in Entity provenance +- **Policy Engine** โ€” evaluates provider events and unsanctioned changes, determines response actions +- **Service Dependencies** โ€” Resource/Service Entities declare dependencies on other Entities +- **Resource Grouping** โ€” Entities belong to a Tenant and optionally to additional Resource Groups +- **Provider Contract** โ€” governs provider obligations including capacity reporting and event notification + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/resource-type-hierarchy.md b/content/docs/data-model/resource-type-hierarchy.md new file mode 100644 index 0000000..b51a2ad --- /dev/null +++ b/content/docs/data-model/resource-type-hierarchy.md @@ -0,0 +1,443 @@ +--- +title: Resource Type Hierarchy +type: docs +weight: 4 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) + +--- + +## 1. Purpose + +The DCM Resource Type Hierarchy is the structural model that defines how services and resources are categorized, specified, and exposed through the DCM Service Catalog. It is the mechanism by which DCM achieves **resource portability** โ€” the ability to express what a consumer needs independently of which specific provider delivers it. + +The hierarchy serves four goals: + +1. **Portability** โ€” consumer intent can be fulfilled by any provider that satisfies the resource type contract, without the consumer needing to know which provider that is +2. **Standardization** โ€” a common vocabulary and data contract for all resource types encourages interoperability across providers, implementors, and the broader DCM community +3. **Extensibility** โ€” the model can be extended at every level without breaking existing definitions +4. **Transparency** โ€” any deviation from full portability is explicitly declared, versioned, and surfaced to consumers + +--- + +## 2. The DCM Resource Type Registry + +DCM maintains an official **Resource Type Registry** โ€” the authoritative source of standard resource type definitions. The registry is the foundation of portability across the DCM ecosystem. + +### 2.1 Registry Principles + +- The registry is **open** โ€” third parties, implementors, and the community can propose new resource type definitions +- Registry entries are **versioned and immutable** once published โ€” changes produce new versions +- Registry definitions are **vendor-neutral by hard requirement** โ€” no vendor-specific data is permitted in a DCM-specified resource type unless that vendor is the exclusive provider of that technology stack +- The registry itself is subject to the same **deprecation model** as all other DCM definitions +- All registry entries follow the **universal versioning scheme** (Major.Minor.Revision) + +### 2.2 Default Resource Type Categories + +DCM ships with a default set of Resource Type Categories. Implementors may define additional categories following the specification. The registry contains both **Resource Types** (for provisioned resources) and **Information Types** (for external data references) โ€” distinguished by category prefix. + +**Resource Type Categories:** + +| Category | Description | +|----------|-------------| +| `Compute` | Processing resources โ€” virtual machines, containers, bare metal | +| `Network` | Networking resources โ€” IP addresses, VLANs, firewall rules, load balancers | +| `Storage` | Storage resources โ€” block, object, file, databases | +| `Platform` | Platform services โ€” Kubernetes clusters, application platforms | +| `Security` | Security resources โ€” certificates, secrets, HSMs, identity | +| `Observability` | Monitoring and logging resources | +| `Data` | Data services โ€” streams, queues, pipelines | + +**Information Type Categories:** + +| Category | Description | +|----------|-------------| +| `Business` | Business organizational data โ€” BusinessUnit, CostCenter, ProductOwner | +| `Identity` | Identity and access data โ€” Person, ServiceAccount, Group | +| `Compliance` | Regulatory and compliance data โ€” RegulatoryScope, AuditFramework | +| `Operations` | Operational reference data โ€” Runbook, SLA, SupportContract | + +All categories follow the same versioning, deprecation, and registry governance model. The `implements_type` field on provider registrations distinguishes whether a provider is a Service Provider (`service`) or an Information Provider (`information`). + +### 2.3 Registry Entry Structure + +Every entry in the Resource Type Registry carries the following metadata: + +```yaml +registry_entry: + uuid: + name: + fully_qualified_name: + version: + parent_uuid: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + replacement_version: + deprecation_reason: + migration_guidance: + portability: + classification: + portability_breaking: + portability_notes: + ownership: + owner: + owner_uuid: + origination_date: + description: + specification_ref: +``` + +--- + +## 3. Resource Type Hierarchy Levels + +The hierarchy has four levels, from most abstract to most concrete. Each level builds on the one above it. + +### Level 1 โ€” Resource Type Category + +The broadest classification. Defines the domain of a resource without any specificity about what the resource is. + +- DCM ships with default categories (see Section 2.2) +- Implementors may define additional categories +- Categories have no data fields โ€” they are organizational containers +- Categories are versioned and can be deprecated + +**Example:** `Compute`, `Network`, `Storage` + +--- + +### Level 2 โ€” Resource Type + +Defines an abstract resource within a category. A Resource Type represents a class of resource that multiple providers can implement. Resource Types are the primary unit of portability in DCM. + +- DCM maintains default Resource Types in the registry +- Community and implementors can define and register new Resource Types +- Resource Types must be **vendor-neutral** โ€” no provider-specific data +- Resource Types declare their **base field specification** (universal fields only) +- Resource Types are versioned and can be deprecated + +**Example:** `Compute.VirtualMachine`, `Network.IPAddress`, `Network.FirewallRule` + +--- + +### Level 3 โ€” Resource Type Specification + +The data contract for a Resource Type. Defines all fields โ€” universal, conditional, and any declared extension points โ€” along with their types, constraints, and portability classifications. + +- Every field in a specification carries a **portability classification** (see Section 4) +- Specifications define which fields are required vs. optional +- Specifications define validation constraints for each field +- Specifications declare **extension points** where providers may add fields +- Specifications are versioned independently of their Resource Type +- Specifications can be deprecated + +**Example:** `Compute.VirtualMachine` specification defines: `cpu_count` (universal, required), `ram_gb` (universal, required), `storage_gb` (universal, required), `os_image` (universal, required), `high_availability` (conditional, optional) + +--- + +### Level 4 โ€” Provider Catalog Item + +A specific provider's concrete implementation of a Resource Type Specification. This is where provider-specific detail lives and where the abstract becomes actionable. + +- Provider Catalog Items are registered against a specific Resource Type Specification version +- They must implement **all universal fields** of the parent specification +- They may implement **conditional fields** (declared in their registration) +- They may add **provider-specific extension fields** (must be marked portability-breaking) +- They are versioned and can be deprecated +- They declare their **sovereignty capabilities** (see Section 6) +- They declare their **supported lifecycle operations** (see Section 7) + +**Example:** `Nutanix.VM.Small` implements `Compute.VirtualMachine` with `cpu_count: 4`, `ram_gb: 16`, `storage_gb: 60` + +--- + +## 4. Portability Classification + +Every field in every Resource Type Specification carries a portability classification. This classification is part of the field's metadata and is immutable once published for a given version. + +### 4.1 Classification Levels + +| Classification | Description | Portability Impact | +|---|---|---| +| `universal` | Part of the DCM standard spec. All providers implementing this type must support it. | Fully portable across all implementing providers | +| `conditional` | Supported by multiple providers but not all. Providers declare support in their registration. | Portable across providers that declare support | +| `provider-specific` | Specific to one provider or technology stack. Using this field locks the request to that provider. | Portability-breaking โ€” must be explicitly marked | +| `exclusive` | Only one provider supports this technology stack. Portability is not applicable by definition. | Not applicable โ€” acknowledged and declared | + +### 4.2 Hard Portability Requirements + +The following are non-negotiable requirements for any DCM-specified Resource Type: + +1. All **universal** fields MUST be supported by ALL providers implementing that Resource Type +2. **Provider-specific** fields MUST be explicitly marked as portability-breaking in the field metadata +3. Consumers MUST be warned when their request contains portability-breaking fields +4. The only exception to vendor-neutrality is the **exclusive** classification โ€” where one provider is the sole implementor of a technology stack, explicitly acknowledged and declared in the registry +5. Any Resource Type in the DCM registry that contains provider-specific fields as universal fields is invalid and must be rejected + +### 4.3 Portability Field Metadata + +Every field in a Resource Type Specification carries the following portability metadata: + +```yaml +field_name: + type: + required: + description: + portability: + classification: + portability_breaking: + portability_notes: + supported_by: + constraints: + - + default_value: + provenance: + +``` + +--- + +## 5. Inheritance Model + +Resource Types support inheritance, enabling specialization without duplication. A child type inherits all fields from its parent and may add new fields. + +### 5.1 Inheritance Rules + +1. A child type inherits **all fields** from its parent type โ€” no field can be removed or redefined +2. A child type may **add new fields** beyond its parent's specification +3. A child type's portability classification can only be **equal to or more restrictive** than its parent โ€” a child of a `universal` type may be `conditional`, but not vice versa +4. Each level of the hierarchy is **independently versioned** +5. Each level maintains a **reference to its parent UUID and version** +6. Deprecating a parent type **does not automatically deprecate child types** โ€” each must be independently deprecated with appropriate migration guidance + +### 5.2 Inheritance Example + +``` +Compute # Category + โ””โ”€โ”€ VirtualMachine # Base Resource Type + โ”œโ”€โ”€ VirtualMachine.GPU # Inherits VirtualMachine + โ”‚ โ”œโ”€โ”€ gpu_count (conditional) + โ”‚ โ”œโ”€โ”€ gpu_memory_gb (conditional) + โ”‚ โ””โ”€โ”€ VirtualMachine.GPU.HighMemory # Inherits VirtualMachine.GPU + โ”‚ โ””โ”€โ”€ extended_memory_gb (conditional) + โ””โ”€โ”€ VirtualMachine.HighAvailability # Inherits VirtualMachine + โ”œโ”€โ”€ ha_mode (conditional) + โ””โ”€โ”€ failover_policy (conditional) +``` + +### 5.3 Inheritance Metadata + +Every Resource Type that inherits from a parent carries the following inheritance metadata: + +```yaml +inheritance: + parent_uuid: + parent_version: + parent_fully_qualified_name: + inherited_fields: + added_fields: +``` + +--- + +## 6. Provider Registration and Catalog Item Declaration + +For a provider to participate in the DCM ecosystem and have its catalog items available for request resolution, it must register against the Resource Type Hierarchy. + +### 6.1 Provider Registration Declaration + +A provider's registration is a machine-readable declaration that DCM consumes to understand what the provider offers and how to route requests to it: + +```yaml +provider_registration: + uuid: + name: + version: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + catalog_items: + - + sovereignty_capabilities: + + supported_lifecycle_operations: + + trust_declaration: + +``` + +### 6.2 Catalog Item Declaration + +Each catalog item a provider offers is declared against a specific Resource Type Specification version: + +```yaml +catalog_item: + uuid: + name: + version: + implements: + resource_type_uuid: + resource_type_version: + resource_type_fully_qualified_name: + status: + state: + deprecation_date: + sunset_date: + replacement_uuid: + deprecation_reason: + migration_guidance: + universal_fields: + + conditional_fields_supported: + + provider_specific_extensions: + + + portability_warning: +``` + +--- + +## 7. Request Resolution โ€” Specificity Narrowing + +Provider selection in DCM is never explicit. The consumer declares intent using Resource Types and field values. The appropriate provider catalog item is selected by the DCM Policy Engine through progressive specificity narrowing. + +### 7.1 Resolution Steps + +``` +Step 1: Resource Type declared + โ†’ matches all providers implementing that Resource Type + +Step 2: Universal fields specified + โ†’ still matches all providers (all must support universal fields) + +Step 3: Conditional fields specified + โ†’ narrows to providers that declare support for those fields + +Step 4: Provider-specific fields used + โ†’ narrows to single provider + โ†’ portability warning issued and recorded in request provenance + โ†’ enforcement mode applied (block|warn|allow) per organizational policy + +Step 5: Placement and sovereignty constraints applied + โ†’ Policy Engine applies placement policies + โ†’ Provider sovereignty capabilities matched against request requirements + โ†’ Final provider catalog item selected + +Step 6: Provider catalog item UUID recorded in request payload provenance +``` + +### 7.2 Portability Warning Enforcement + +When a request contains portability-breaking fields, the Policy Engine applies the configured enforcement mode. This is organizational policy โ€” configurable at the organization, domain, or service level: + +| Enforcement Mode | Behavior | +|---|---| +| `block` | Request is rejected. Consumer must remove portability-breaking fields or explicitly acknowledge the lock-in. | +| `warn` | Request proceeds. Portability warning is recorded in request provenance and surfaced to the consumer. | +| `allow` | Request proceeds silently. Portability-breaking fields are still recorded in provenance but no warning is surfaced. | + +The enforcement mode is itself a versioned, auditable policy โ€” subject to the same provenance tracking as all other data in DCM. + +--- + +## 8. Deprecation Model + +Every definition at every level of the Resource Type Hierarchy can be deprecated. Deprecation is a first-class concept in DCM โ€” not an afterthought. + +### 8.1 Deprecation Lifecycle + +``` +active โ†’ deprecated โ†’ retired +``` + +| State | Meaning | System Behavior | +|---|---|---| +| `active` | Definition is current and fully supported | Normal operation | +| `deprecated` | Definition is being phased out. Replacement is available. | Deprecation warning surfaced to consumers. Requests still processed. Warning recorded in provenance. | +| `retired` | Definition is no longer honored. | Requests using retired definitions are rejected by the Policy Engine. | + +### 8.2 Deprecation Cascade Rules + +- Deprecating a **Resource Type** does not automatically deprecate its child types or provider catalog items โ€” each must be independently deprecated +- Deprecating a **Provider Catalog Item** does not affect other catalog items implementing the same Resource Type +- Retiring a **Resource Type Specification version** causes all catalog items registered against that version to require re-registration against a current version +- **Sunset dates** must provide sufficient migration runway โ€” minimum notice periods may be defined by organizational policy + +### 8.3 Migration Guidance Requirement + +Any definition marked `deprecated` MUST include: +- A reference to the replacement definition (UUID and version) +- A human-readable deprecation reason +- Human-readable migration guidance explaining how to transition +- A sunset date giving consumers time to migrate + +--- + +## 9. Versioning + +All definitions in the Resource Type Hierarchy follow the universal DCM versioning scheme. + +### 9.1 Version Scheme + +`Major.Minor.Revision` + +| Component | Trigger | +|---|---| +| **Major** | Breaking changes to the contract โ€” removing fields, changing field types, changing required/optional status of universal fields | +| **Minor** | Additive changes, backward compatible โ€” adding new optional fields, adding new conditional fields, adding new extension points | +| **Revision** | Data or configuration changes with no contract impact โ€” updating descriptions, updating constraints that don't break existing data, updating metadata | + +### 9.2 Version Constraints in Requests + +Consumers and dependencies may declare version constraints in their requests: + +```yaml +resource_type: + uuid: + version_constraint: + version: +``` + +### 9.3 Version Immutability + +Once a version is published it is immutable. Any change โ€” even a documentation correction โ€” produces a new version. This applies to all definitions at all levels of the hierarchy. + +--- + +## 10. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | What is the governance model for proposing and approving new Resource Types to the DCM registry? | Community adoption, quality control | โ“ Unresolved | +| 2 | Should the registry support a formal review/approval workflow before a Resource Type becomes `active`? | Registry integrity, community trust | โ“ Unresolved | +| 3 | What is the minimum sunset period for deprecated definitions? | Migration planning, operational stability | โ“ Unresolved | +| 4 | Should version constraints in requests be strictly enforced or advisory? | Operational flexibility vs. predictability | โ“ Unresolved | +| 5 | How are conflicts resolved when multiple providers satisfy all narrowing criteria equally? | Request resolution determinism | โ“ Unresolved | +| 6 | Should the registry be distributed or centralized? How does this interact with sovereignty requirements? | Registry availability, sovereignty | โ“ Unresolved | + +--- + +## 11. Related Concepts + +- **Portability** โ€” the ability to fulfill a resource intent using any provider that satisfies the resource type contract +- **Naturalization** โ€” provider's responsibility to transform DCM unified data into provider-specific format +- **Denaturalization** โ€” provider's responsibility to transform provider-specific results back into DCM unified format +- **Sovereign Execution Posture** โ€” sovereignty capabilities declared in provider registration inform placement decisions +- **Policy Engine** โ€” applies portability enforcement, placement policies, and request resolution logic +- **Field-Level Provenance** โ€” every field modification during request resolution is recorded with source UUID and operation type +- **Universal Versioning** โ€” Major.Minor.Revision applies to all definitions at all levels of the hierarchy +- **Deprecation** โ€” universal model for phasing out definitions at any level with migration guidance + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/service-dependencies.md b/content/docs/data-model/service-dependencies.md new file mode 100644 index 0000000..0c92bc4 --- /dev/null +++ b/content/docs/data-model/service-dependencies.md @@ -0,0 +1,371 @@ +--- +title: Service Dependencies +type: docs +weight: 6 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Entity Relationships](09-entity-relationships.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) + +> **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](09-entity-relationships.md). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document. + +--- + +## 1. Purpose + +This document defines how service dependencies are declared, resolved, and managed within DCM. Dependencies are a core data model concern โ€” not an orchestration concern. The data structures defined here enable DCM to know the complete resource footprint of any request before execution begins, which is essential for cost analysis, placement decisions, rehydration, and audit. + +--- + +## 2. Why Dependencies Must Be Declared in Advance + +Dependencies must be declared in the data model โ€” not discovered at runtime by providers. This is a hard requirement driven by four core DCM goals: + +**Auditability** โ€” the complete dependency graph must be known before execution. Every resource that will be created as part of fulfilling a request must be visible in the request's provenance chain from the start. + +**Cost Analysis** โ€” accurate cost estimation and cost-based placement require knowing the full resource footprint before provisioning. Hidden dependencies produce hidden costs that only become visible after the fact. + +**Placement** โ€” the Policy Engine cannot make optimal placement decisions without knowing all resources that will be created. A Web Server request that implicitly spawns a VM, IP address, and firewall rule has placement requirements that span multiple resource types. + +**Idempotency and Consistency** โ€” if dependencies are declared in the service definition, the same request always produces the same dependency graph. Provider-driven dependency discovery at runtime breaks idempotency โ€” different provider implementations could produce different dependency graphs for the same logical request. + +--- + +## 3. Hybrid Dependency Declaration Model + +DCM uses a hybrid model for dependency declaration that operates at two levels: + +### 3.1 Type-Level Dependencies (Resource Type Specification) + +Dependencies declared at the Resource Type Specification level are **portable and provider-agnostic**. They define what kinds of resources are needed โ€” not which specific provider supplies them. + +- Declared in the Resource Type Specification +- Apply to all Provider Catalog Items implementing that Resource Type +- Use Resource Type UUIDs โ€” not provider-specific references +- Required for all implementations of the Resource Type +- Portable โ€” the dependency can be fulfilled by any provider implementing the required Resource Type + +**Example:** +```yaml +resource_type: Compute.VirtualMachine +type_level_dependencies: + - dependency_uuid: + required_resource_type_uuid: + required_resource_type_name: Network.IPAddress + dependency_type: hard + cardinality: one_to_one + description: Every VM requires exactly one IP address + - dependency_uuid: + required_resource_type_uuid: + required_resource_type_name: Network.FirewallRule + dependency_type: hard + cardinality: one_to_many + description: Every VM requires at least one firewall rule +``` + +### 3.2 Provider-Specific Dependencies (Provider Catalog Item) + +Dependencies declared at the Provider Catalog Item level are **provider-specific additions** beyond the type-level dependencies. They must be marked as portability-breaking. + +- Declared in the Provider Catalog Item registration +- Apply only to requests fulfilled by that specific provider +- Must be marked `portability_breaking: true` +- Visible to the Policy Engine for governance decisions +- Surfaced to consumers as portability warnings + +**Example:** +```yaml +catalog_item: Nutanix.VM.Small +provider_specific_dependencies: + - dependency_uuid: + required_resource_type_uuid: + required_resource_type_name: Nutanix.StorageContainer + dependency_type: hard + portability_breaking: true + description: Nutanix VMs require a Nutanix Storage Container + portability_warning: This dependency locks this request to Nutanix providers +``` + +--- + +## 4. Dependency Types + +Every declared dependency must specify its type: + +| Type | Description | Behavior | +|------|-------------|----------| +| `hard` | Must be realized before or alongside the dependent resource | Failure of dependency fails the dependent resource | +| `soft` | Preferred but not blocking | Failure of dependency is recorded but does not block the dependent resource | +| `conditional` | Required only if specific conditions in the request payload are met | Evaluated by Policy Engine against request data | + +--- + +## 5. Dependency Cardinality + +Every declared dependency must specify its cardinality: + +| Cardinality | Description | Example | +|-------------|-------------|---------| +| `one_to_one` | Exactly one dependency resource required | One VM needs exactly one primary IP | +| `one_to_many` | One or more dependency resources required | One VM needs one or more firewall rules | +| `one_to_optional` | Zero or one dependency resource | One VM may optionally have a secondary IP | +| `one_to_range` | A specific numeric range required | One load balancer needs 2-6 backend VMs | + +--- + +## 6. Dependency Graph + +When a request is processed, the Request Payload Processor constructs a **Dependency Graph** โ€” a complete map of all resources that must be created to fulfill the request, including all transitive dependencies. + +### 6.1 Dependency Graph Structure + +```yaml +dependency_graph: + graph_uuid: + root_request_uuid: + tenant_uuid: + created_timestamp: + nodes: + - node_uuid: + resource_type_uuid: + resource_type_name: + request_uuid: + entity_uuid: + lifecycle_state: + dependencies: + - dependency_uuid: + dependent_node_uuid: + dependency_type: + status: + edges: + - from_node_uuid: + to_node_uuid: + dependency_uuid: + dependency_type: +``` + +### 6.2 Transitive Dependencies + +DCM resolves transitive dependencies โ€” the full chain of dependencies, not just direct ones. + +**Example โ€” Web Server request:** +``` +Web Server (requested) + โ”œโ”€โ”€ VM (hard dependency of Web Server) + โ”‚ โ”œโ”€โ”€ IP Address (hard dependency of VM) + โ”‚ โ”‚ โ””โ”€โ”€ Network (hard dependency of IP Address) + โ”‚ โ””โ”€โ”€ Firewall Rule (hard dependency of VM) + โ”‚ โ””โ”€โ”€ IP Address (reference โ€” already in graph) + โ””โ”€โ”€ DNS Record (soft dependency of Web Server) + โ””โ”€โ”€ IP Address (reference โ€” already in graph) +``` + +The dependency graph contains each resource exactly once โ€” circular references and duplicate nodes are detected and resolved. A resource that appears as a dependency of multiple nodes is represented as a single node with multiple incoming edges. + +### 6.3 Dependency Graph and the Four States + +The dependency graph is part of the request's data from the moment it is constructed: + +- **Intent State** โ€” consumer's request, no dependency graph yet +- **Requested State** โ€” dependency graph constructed and attached, all nodes in PENDING state +- **Realized State** โ€” nodes updated to REALIZED as providers fulfill each dependency +- **Discovered State** โ€” dependency graph used to scope discovery โ€” discover all nodes in the graph + +--- + +## 7. Dependency Payload Passing + +When a dependency resource is realized, its realized payload must be passed to the dependent resource's provider. This is how a provider knows the details of the resources it depends on โ€” IP addresses, network configurations, security group IDs, etc. + +### 7.1 The Payload Passing Mechanism + +``` +Dependency Resource realized + โ”‚ + โ–ผ +Realized State payload captured in Realized Store + โ”‚ + โ–ผ +Dependency node in graph updated: entity_uuid recorded, status โ†’ SATISFIED + โ”‚ + โ–ผ +Dependent resource's Requested State payload enriched with dependency data + โ”‚ Recorded in field-level provenance โ€” source_type: dependency_payload + โ”‚ source_uuid: + โ–ผ +Enriched payload dispatched to dependent resource's provider +``` + +### 7.2 Dependency Data in Request Payloads + +When a dependency is satisfied, the dependent resource's Requested State payload is enriched with the dependency entity's UUID and relevant realized data: + +```yaml +# Original request payload for VM +vm_request: + cpu_count: 8 + ram_gb: 32 + os: RHEL9 + +# After IP Address dependency is realized +vm_request: + cpu_count: 8 + ram_gb: 32 + os: RHEL9 + dependencies: + ip_address: + entity_uuid: + ip_address: 192.168.1.45 + network_uuid: + subnet: 192.168.1.0/24 + provenance: + source_type: dependency_payload + source_uuid: + timestamp: +``` + +--- + +## 8. Dependency Resolution Order + +The dependency graph determines resolution order. Resources with no unsatisfied hard dependencies can be dispatched immediately. Resources with unsatisfied hard dependencies wait until their dependencies are satisfied. + +### 8.1 Resolution Rules + +- A resource node can only be dispatched when all its `hard` dependencies are in SATISFIED state +- `soft` dependencies do not block dispatch โ€” they are attempted but failure does not block +- `conditional` dependencies are evaluated by the Policy Engine before the graph is constructed โ€” if conditions are not met, the conditional dependency node is not added to the graph +- Independent branches of the dependency graph can be resolved in parallel โ€” the Orchestration component determines parallelism +- Circular dependencies are invalid โ€” the Policy Engine rejects any dependency graph with circular references + +### 8.2 Failure Handling + +Dependency failure handling is **configurable per request or per policy**: + +| Failure Mode | Behavior | +|-------------|---------| +| `fail_all` | Any hard dependency failure fails the entire request. All partially realized nodes are decommissioned. | +| `fail_dependent` | A hard dependency failure fails only the dependent resource and its dependents. Independent branches continue. | +| `retry` | Failed dependencies are retried with the same or alternative provider before failing. Retry count and provider selection policy are configurable. | +| `partial_complete` | Request is marked partially complete. Failed nodes are flagged for retry or manual intervention. | + +The failure mode is declared in the request payload or in an applicable organizational policy. + +--- + +## 9. Rehydration and the Dependency Graph + +The dependency graph is the primary mechanism enabling **DC Rehydration** โ€” the ability to reconstruct any resource and its dependencies from scratch. + +### 9.1 Rehydration Process + +Rehydration uses the **Intent State** of the original request โ€” not the Realized State โ€” to reconstruct the dependency graph. This ensures that rehydration applies current policies and standards rather than replaying an old realized state. + +``` +Rehydration initiated for a Tenant / Group / Entity + โ”‚ + โ–ผ +Intent State(s) retrieved from Intent Store + โ”‚ + โ–ผ +Dependency graphs reconstructed from Intent States + โ”‚ + โ–ผ +Graphs processed through current Policy Engine + โ”‚ Current policies applied โ€” may differ from original request + โ”‚ Current placement policies applied + โ”‚ Current sovereignty constraints applied + โ–ผ +New Requested State payloads generated + โ”‚ + โ–ผ +Resources realized in dependency order + โ”‚ + โ–ผ +New Realized States recorded +``` + +### 9.2 Intent Portability in Rehydration + +Because rehydration uses Intent State rather than Realized State: +- Resources can be rehydrated to a different provider โ€” as long as the provider supports the required Resource Types +- Current organizational standards and policies are applied โ€” ensuring rehydrated resources meet current compliance requirements +- Provider-specific dependencies (portability-breaking) may prevent rehydration to a different provider โ€” this is surfaced as a portability warning during rehydration planning + +### 9.3 Rehydration Scope + +Rehydration can be scoped to: +- A single Resource/Service Entity and its full dependency graph +- A Resource Group โ€” all entities in the group and their dependency graphs +- A Tenant โ€” all entities owned by the Tenant +- A full Data Center โ€” all entities across all Tenants in a location + +The dependency graph ensures that rehydration is always complete โ€” no orphaned resources, no missing dependencies. + +--- + +## 10. Dependency Declaration in Service Catalog Items + +Service Catalog Items must declare their dependencies as part of their definition. A catalog item with undeclared dependencies is invalid and will be rejected by the Policy Engine. + +```yaml +catalog_item: + uuid: + name: Web Server Service + resource_type_uuid: + type_level_dependencies: + - dependency_uuid: + required_resource_type_uuid: + dependency_type: hard + cardinality: one_to_one + provider_specific_dependencies: [] + conditional_dependencies: + - dependency_uuid: + required_resource_type_uuid: + dependency_type: conditional + condition: + field: high_availability + operator: equals + value: true + description: Load balancer required when high_availability is true +``` + +--- + +## 11. DCM System Policies for Dependencies + +| Policy | Rule | Enforcement | +|--------|------|-------------| +| `DEP-001` | All dependencies must be declared before a catalog item is active | Enforced at catalog item registration | +| `DEP-002` | Circular dependencies are invalid | Enforced at dependency graph construction | +| `DEP-003` | Provider-specific dependencies must be marked portability-breaking | Enforced at provider catalog item registration | +| `DEP-004` | Dependency payloads must be passed to dependent providers in DCM unified format | Enforced at dependency satisfaction | +| `DEP-005` | Every node in a dependency graph must have a UUID | Enforced at graph construction | + +--- + +## 12. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How are dependency graphs versioned โ€” does a new version of a catalog item invalidate existing dependency graphs? | Versioning model | โ“ Unresolved | +| 2 | Should the dependency graph be stored as a separate entity or embedded in the request payload? | Data model structure | โ“ Unresolved | +| 3 | How are cross-tenant dependencies handled โ€” where one tenant's resource depends on another tenant's resource? | Multi-tenancy | โ“ Unresolved | +| 4 | Should there be a maximum dependency graph depth to prevent runaway transitive dependencies? | Operational complexity | โ“ Unresolved | +| 5 | How does the dependency graph interact with the Meta Provider model โ€” where one provider orchestrates others? | Provider model | โ“ Unresolved | + +--- + +## 13. Related Concepts + +- **Resource Type Specification** โ€” declares type-level dependencies for a Resource Type +- **Provider Catalog Item** โ€” declares provider-specific additional dependencies +- **Request Payload Processor** โ€” constructs the dependency graph during assembly +- **Policy Engine** โ€” evaluates conditional dependencies, enforces dependency policies, governs failure handling +- **Intent Portability** โ€” rehydration uses Intent State to allow replay with different providers +- **Field-Level Provenance** โ€” dependency payload data is recorded with source Entity UUID +- **Resource Grouping** โ€” rehydration can be scoped to groups and tenants + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/storage-providers.md b/content/docs/data-model/storage-providers.md new file mode 100644 index 0000000..762d08a --- /dev/null +++ b/content/docs/data-model/storage-providers.md @@ -0,0 +1,299 @@ +--- +title: Storage Providers +type: docs +weight: 10 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents:** [Four States](02-four-states.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Information Providers](10-information-providers.md) + +--- + +## 1. Purpose + +A **Storage Provider** is the fourth formal DCM provider type. It is the interface through which DCM persists, retrieves, and streams all state data. DCM defines the contract โ€” the characteristics, capabilities, and obligations each store must satisfy. The implementation technology is a deployment choice made by implementors. + +This is consistent with DCM's governing framework philosophy: DCM does not prescribe technology. It defines what is required and what is guaranteed. An organization using GitHub and Kafka satisfies the same contracts as one using Gitea and EventStoreDB. + +--- + +## 2. The Four Provider Types + +| Provider Type | Purpose | Data Direction | DCM Owns Result? | +|--------------|---------|---------------|-----------------| +| **Service Provider** | Realizes resources | DCM โ†’ Provider โ†’ DCM | Yes | +| **Information Provider** | Serves external authoritative data | DCM โ†’ Provider (lookup) | No | +| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Children โ†’ DCM | Yes | +| **Storage Provider** | Persists and streams DCM state | DCM โ†” Provider | Yes โ€” DCM is authoritative | + +--- + +## 3. Storage Provider Contract โ€” Base Requirements + +All Storage Providers share these base contract requirements regardless of store type: + +### 3.1 Registration +Same model as Service and Information Providers. Storage Providers register with DCM declaring their endpoint, store type, capabilities, and sovereignty characteristics. + +```yaml +storage_provider_registration: + uuid: + name: + display_name: + store_type: + version: + endpoint: + capabilities: + sovereignty_constraints: + trust_declaration: + status: +``` + +### 3.2 Health Check +Same model as all providers. `GET /health` endpoint, DCM polls on configurable interval. + +### 3.3 Trust +Same model as all providers. DCM validates Storage Provider identity before writing or reading state data. A compromised Storage Provider is treated as a sovereignty incident. + +### 3.4 Provenance Emission Obligation +Every Storage Provider that holds state data has a contractual obligation to emit provenance events to the Audit component when state is written or modified. This is not optional โ€” it is part of the Storage Provider contract. + +```yaml +# Provenance emission event โ€” sent to Audit component on every write +provenance_emission: + store_type: + operation: + entity_uuid: + record_uuid: + actor_uuid: + timestamp: + payload_hash: + store_reference: +``` + +### 3.5 Consistency Guarantee Declaration +Each Storage Provider must declare its consistency model in registration. DCM components read this declaration and adapt their behavior accordingly. + +```yaml +consistency_declaration: + consistency_model: + replication_factor: + durability_guarantee: + max_data_loss_window: +``` + +--- + +## 4. GitOps Store Contract + +Used for: Intent State, Requested State, Layer Store, Policy Store + +### 4.1 Required Capabilities + +```yaml +gitops_capabilities: + branching: true # Branch-per-request support + pull_request: true # PR creation, review, merge + immutable_history: true # Commits are permanent + ci_cd_hooks: true # Webhook triggers on push/merge + search_index_integration: true # Search Index companion required + access_control: true # Per-branch, per-path access control + signed_commits: optional # Recommended for audit integrity +``` + +### 4.2 Required API Operations + +| Operation | Description | Used By | +|-----------|-------------|---------| +| `create_branch` | Create a new branch from main | Intent State creation | +| `commit_file` | Commit a file to a branch | Intent and Requested State write | +| `create_pr` | Open a Pull Request for review | Intent State review workflow | +| `merge_pr` | Merge an approved PR to main | Intent State approval | +| `get_file` | Retrieve a file by path or commit | State retrieval | +| `get_history` | Retrieve commit history for a path | Audit and rehydration | +| `trigger_ci` | Trigger CI pipeline on branch | Policy pre-validation | +| `trigger_cd` | Trigger CD pipeline on merge | Requested State assembly and dispatch | +| `post_comment` | Post a comment on a PR | CI pipeline result reporting | + +### 4.3 File Structure Convention + +``` +{store_root}/ + tenants/ + {tenant_uuid}/ + {entity_uuid}/ + intent.yaml # Intent State record + # OR + requested-state.yaml # Requested State record +``` + +### 4.4 Search Index Companion + +Every GitOps store deployment requires a companion Search Index. The Search Index is a separate Storage Provider that maintains a queryable projection of the GitOps store. See Section 6. + +--- + +## 5. Event Stream Store Contract + +Used for: Realized State, Discovered State + +### 5.1 Required Capabilities + +```yaml +event_stream_capabilities: + append_only: true # Events are never modified or deleted + entity_keyed_streams: true # Each entity has its own event stream + stream_replay: true # Streams can be replayed from any offset + entity_uuid_lookup: true # O(1) lookup of stream by entity UUID + at_least_once_delivery: true # Events are never silently lost + configurable_retention: true # Retention period configurable per stream type + distributed_replication: true # Data replicated across nodes + high_throughput_write: true # Optimized for machine-generated writes +``` + +### 5.2 Stream Naming Convention + +``` +dcm.realized.{entity_uuid} # Realized State stream per entity +dcm.discovered.{entity_uuid} # Discovered State stream per entity +dcm.audit.{tenant_uuid} # Audit event stream per tenant +dcm.system # DCM system-level events +``` + +### 5.3 Required API Operations + +| Operation | Description | Used By | +|-----------|-------------|---------| +| `append_event` | Append an event to an entity stream | Provider callbacks, discovery | +| `read_stream` | Read events from an entity stream from offset | State retrieval, drift detection | +| `read_latest` | Read the most recent event in a stream | Current state queries | +| `replay_stream` | Replay all events from beginning | Audit, historical reconstruction | +| `list_streams` | List streams matching a pattern | Tenant-level queries | +| `get_stream_metadata` | Get stream statistics and metadata | Health monitoring | + +### 5.4 Event Envelope + +Every event written to the Event Stream Store uses this envelope: + +```yaml +event_envelope: + event_uuid: + stream_id: + entity_uuid: + tenant_uuid: + event_type: + sequence_number: + timestamp: + schema_version: + payload_hash: + payload: + provenance: + written_by_uuid: + triggered_by_request_uuid: + triggered_by_actor_uuid: +``` + +### 5.5 Retention Model + +| Stream Type | Default Retention | Rationale | +|-------------|------------------|-----------| +| Realized State | Permanent | Complete audit trail required | +| Discovered State | Configurable window | Operational use only โ€” older snapshots archived | +| Audit | Regulatory period (configurable โ€” minimum 7 years for FSI) | Compliance requirement | + +--- + +## 6. Search Index Contract + +Used for: Queryable projection of GitOps stores + +### 6.1 Role and Authority + +The Search Index is explicitly **non-authoritative**. If the Search Index and the GitOps store disagree on any record, the GitOps store wins unconditionally. The Search Index is a performance layer โ€” it is never the source of truth. + +The Search Index can be rebuilt from scratch from Git history at any time. This replaceability is a contract requirement โ€” implementors must support full index rebuild from the GitOps store. + +### 6.2 Required Indexed Fields + +At minimum the Search Index must index these fields from Intent and Requested State records: + +```yaml +indexed_fields: + - entity_uuid # Universal linking key + - tenant_uuid # Tenant ownership + - resource_type_name # e.g., Compute.VirtualMachine + - resource_type_uuid # Registry UUID + - lifecycle_state # Current state + - provider_uuid # Selected provider + - created_timestamp # When the record was created + - updated_timestamp # When the record was last updated + - cost_center # Business context (if declared) + - business_unit_uuid # Business context (if declared) + - git_path # Path in GitOps store โ€” used to retrieve full record + - git_commit_hash # Specific commit โ€” used for point-in-time retrieval +``` + +### 6.3 Required Query Operations + +| Operation | Example | Used By | +|-----------|---------|---------| +| `find_by_entity_uuid` | Find all records for entity xyz | Rehydration, audit | +| `find_by_tenant` | All entities for Tenant A | Tenant management | +| `find_by_resource_type` | All VMs across all tenants | Catalog reporting | +| `find_by_lifecycle_state` | All PENDING entities | Operational monitoring | +| `find_by_field` | All entities with cost_center=BU-PAY | FinOps reporting | +| `full_text_search` | Search across all indexed text fields | Discovery, debugging | +| `count_by_field` | Count entities grouped by resource_type | Analytics | + +--- + +## 7. DCM-Internal Caches + +DCM may maintain internal performance caches between components and stores. These are not Storage Providers โ€” they are internal implementation details that do not require external registration or trust. + +### 7.1 Cache Characteristics + +- **Non-authoritative** โ€” explicitly marked. Cache hits are not treated as ground truth. +- **Cache-aside pattern** โ€” DCM checks cache first; on miss, reads from authoritative store and populates cache +- **Invalidation on write** โ€” any write to an authoritative store invalidates the corresponding cache entry +- **Bounded staleness** โ€” maximum staleness window configured per cache; entries older than the window are treated as misses +- **Rebuildable** โ€” any cache can be cleared and rebuilt from its authoritative store + +### 7.2 Candidate Cache Locations + +| Cache | Authoritative Source | Purpose | +|-------|---------------------|---------| +| Layer Cache | Layer Store (Git) | Avoid repeated Git reads for frequently used layers | +| Policy Cache | Policy Store (Git) | OPA policy bundles cached in Policy Engine memory | +| Catalog Cache | Catalog Store (Git) | Service catalog items cached for presentation | +| Provider Registry Cache | Provider Registry | Registered provider list cached for routing | +| Search Index | GitOps stores | Queryable projection (also functions as a cache) | + +--- + +## 8. Storage Provider vs Service Provider โ€” Key Differences + +| Dimension | Service Provider | Storage Provider | +|-----------|-----------------|-----------------| +| **Purpose** | Realizes resources | Persists DCM state | +| **Data direction** | DCM sends, provider executes | DCM reads and writes | +| **Naturalization** | Required โ€” DCM format โ†’ native | Not required โ€” DCM format throughout | +| **Denaturalization** | Required โ€” native โ†’ DCM format | Not required | +| **Provenance emission** | Required (realized state) | Required (all writes) | +| **Capacity model** | Resource capacity | Storage capacity and throughput | +| **Health model** | Is provider healthy? | Is store reachable and consistent? | + +--- + +## 9. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should Storage Providers support multi-region replication as a declared capability? | Sovereignty | โ“ Unresolved | +| 2 | How are Storage Provider failures handled โ€” failover, queuing, or rejection? | Reliability | โ“ Unresolved | +| 3 | Should the Search Index be a separate registered Storage Provider or bundled with the GitOps store? | Architecture | โ“ Unresolved | +| 4 | How does the Storage Provider model interact with air-gapped environments? | Sovereignty | โ“ Unresolved | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/specifications/_index.md b/content/docs/specifications/_index.md new file mode 100644 index 0000000..09a3b15 --- /dev/null +++ b/content/docs/specifications/_index.md @@ -0,0 +1,19 @@ +--- +title: Specifications +type: docs +weight: 2 +sidebar: + open: true +--- + +# DCM Specifications + +Formal technical specifications for DCM components, integration interfaces, and +community strategy. + +## Documents + +- **[DCM Operator Interface Specification](operator-interface-spec/)** โ€” The formal contract defining how Kubernetes operators integrate with DCM as Service Providers. Includes conformance levels, registration API, field mapping specification, and conformance testing. +- **[Kubernetes Compatibility](kubernetes-compatibility/)** โ€” Formal mapping between Kubernetes concepts and DCM concepts. Defines DCM as a superset of Kubernetes. +- **[DCM Operator SDK API](operator-sdk-api/)** โ€” Go library API design for the DCM Operator SDK โ€” enables Level 1 conformance in one day. +- **[CNCF Strategy](cncf-strategy/)** โ€” CNCF submission strategy and community engagement plan. diff --git a/content/docs/specifications/cncf-strategy.md b/content/docs/specifications/cncf-strategy.md new file mode 100644 index 0000000..f90aeb7 --- /dev/null +++ b/content/docs/specifications/cncf-strategy.md @@ -0,0 +1,227 @@ +--- +title: CNCF Strategy +type: docs +weight: 4 +--- + +**Version:** 0.1.0-draft +**Status:** Draft +**Document Type:** Strategic Planning +**Maintainers:** Red Hat FlightPath Team +**Last Updated:** 2026-03 + +--- + +## 1. Strategic Intent + +DCM's goal is to become the community standard for enterprise data center and private cloud management โ€” a neutral, open standard that the industry adopts the way it adopted CSI, CNI, and CRI. This requires DCM to exist in a community-trusted home, not as a vendor product. + +The CNCF (Cloud Native Computing Foundation) is the appropriate home. It provides the neutral governance model, the community infrastructure, and the ecosystem relationships needed to drive broad adoption. The FSI consortium already engaging with DCM (Barclays, BofA, JPMC, Morgan Stanley, and others) provides the multi-organization sponsorship and real production use case evidence needed for a credible CNCF proposal. + +--- + +## 2. CNCF Landscape and Positioning + +### 2.1 Where DCM Fits + +The CNCF landscape has strong coverage of Kubernetes runtime concerns โ€” container runtimes, networking, storage, service mesh, observability. It has weaker coverage of the management plane โ€” the layer above Kubernetes that governs what gets provisioned, owned, and decommissioned across multiple clusters and infrastructure types. + +DCM fills this gap. It is not competing with existing CNCF projects โ€” it extends and governs them. + +**Related CNCF projects and how DCM relates:** + +| CNCF Project | Relationship to DCM | +|-------------|---------------------| +| **Kubernetes** | DCM is a superset โ€” extends Kubernetes upward to the management plane | +| **Crossplane** | Complementary โ€” Crossplane provisions cloud resources via Kubernetes CRDs; DCM governs what Crossplane provisions and adds the management plane | +| **Cluster API (CAPI)** | DCM can manage Kubernetes clusters via CAPI as a Service Provider | +| **Argo CD / Flux** | Complementary โ€” DCM governs provisioning requests; GitOps manages deployment | +| **OpenCost** | DCM's cost analysis is a superset โ€” OpenCost data can feed DCM cost attribution | +| **Kessel** | Shares inventory and relationship goals โ€” potential collaboration or alignment | +| **OPA/Gatekeeper** | DCM's Policy Engine uses OPA internally; Gatekeeper is the cluster-level enforcement | + +### 2.2 The Gap DCM Fills + +No current CNCF project addresses all of: +- Multi-cluster, multi-infrastructure lifecycle management from a single control plane +- First-class multi-tenancy with Tenant ownership model +- Policy governance with field-level override control across the full request lifecycle +- Data sovereignty and compliance evidence for regulated industries +- Service catalog with self-service consumer experience +- Cost attribution across heterogeneous infrastructure + +This is the gap DCM fills. The positioning is not "another Kubernetes tool" โ€” it is "the management plane that governs your entire data center, of which Kubernetes is one component." + +--- + +## 3. CNCF Submission Path + +### 3.1 CNCF Maturity Levels + +CNCF accepts projects at three maturity levels: + +| Level | Requirements | DCM Target Timeline | +|-------|-------------|---------------------| +| **Sandbox** | Alignment with CNCF mission, basic governance, active development | Target for initial submission | +| **Incubating** | Production users, healthy contributor base, defined governance, security audit | 12-18 months post-Sandbox | +| **Graduated** | Broad adoption, stable API, long-term maintainer commitment | 24-36 months post-Sandbox | + +### 3.2 Sandbox Submission Requirements + +For CNCF Sandbox acceptance, DCM needs: + +**Technical requirements:** +- Clear alignment with CNCF's cloud native mission +- Open source license (Apache 2.0 โ€” already in place) +- Publicly accessible source code (GitHub โ€” already in place) +- Documented roadmap +- Basic security practices (vulnerability disclosure process, etc.) + +**Governance requirements:** +- Defined governance model (maintainers, decision process) +- Code of conduct +- Multi-organization contributor base (this is the key requirement โ€” Red Hat alone is insufficient) + +**Community requirements:** +- Evidence of community interest beyond the founding organization +- At least one non-founding organization actively contributing + +**DCM's strong position:** +The FSI consortium provides exactly the multi-organization evidence CNCF requires. Having Barclays, BofA, JPMC, and Morgan Stanley as active contributors or committed users is an unusually strong foundation for a Sandbox proposal. Most projects submit to Sandbox without any production users โ€” DCM can submit with evidence of production interest from systemically important financial institutions. + +### 3.3 Recommended Submission Path + +**Step 1 โ€” CNCF TAG (Technical Advisory Group) engagement** +Before formal submission, engage with CNCF TAG App Delivery and TAG Runtime. These groups review cloud native tooling proposals and can provide informal feedback before the formal Due Diligence process. Presenting DCM at a TAG meeting builds awareness and surfaces concerns early. + +**Step 2 โ€” Prepare the Due Diligence document** +The CNCF Due Diligence document is a detailed technical and governance questionnaire. Key sections: project description, statement on alignment with CNCF mission, comparison to similar projects, security practices, roadmap, adopters. The FSI consortium adopters section will be a significant differentiator. + +**Step 3 โ€” TOC sponsor identification** +CNCF Technical Oversight Committee (TOC) members sponsor project proposals. Red Hat's relationships in the Kubernetes community make identifying a TOC sponsor feasible. Target TOC members with expertise in multi-cluster management or enterprise Kubernetes. + +**Step 4 โ€” Sandbox vote** +TOC votes on Sandbox acceptance. With a strong Due Diligence document, FSI adopter evidence, and a TOC sponsor, acceptance probability is high. + +--- + +## 4. Community Engagement Strategy + +### 4.1 Operator Ecosystem โ€” The Primary Leverage Point + +The DCM Operator Interface Specification is the primary community artifact for driving ecosystem adoption. The strategy is to make conformance attractive enough that operator maintainers want to implement it. + +**Priority operator communities for engagement:** + +| Operator | Community | Why Priority | Engagement Approach | +|----------|-----------|-------------|---------------------| +| **KubeVirt** | Red Hat/Community | Active DCM development already | Direct contribution โ€” DCM team contributes Level 2 support | +| **CloudNativePG** | CNPG Community | High FSI adoption โ€” databases in regulated environments | Present DCM at CNPG community calls, contribute SDK example | +| **Strimzi (Kafka)** | Red Hat/Community | Messaging infrastructure โ€” DCM Message Bus use case | Direct contribution via Red Hat maintainership | +| **Cert-Manager** | Jetstack/Venafi | Security resources โ€” every DCM deployment needs certificates | SDK contribution, present at KubeCon | +| **ACM** | Red Hat | Cluster management โ€” natural DCM complement | Direct โ€” internal Red Hat alignment | +| **Rook (Ceph)** | CNCF | Storage operator โ€” core DCM service provider use case | CNCF relationship โ€” present at SIG Storage | + +### 4.2 KubeCon Strategy + +KubeCon is the primary conference for Kubernetes ecosystem influence. DCM needs a presence at KubeCon North America and Europe: + +**KubeCon NA (target โ€” next edition):** +- Submit a talk: "DCM โ€” Managing the Management Plane: Kubernetes as a Component of Enterprise Infrastructure" +- Submit a contribfest session: hands-on DCM Operator SDK implementation workshop +- Engage Kubernetes SIG Cluster Lifecycle about CAPI integration + +**KubeCon EU (following year):** +- Present CNCF Sandbox submission (if accepted by then) +- Case study talk with FSI consortium member (Barclays/BofA presenting their DCM deployment) +- Operator Interface Specification BOF (Birds of a Feather) session + +### 4.3 The Developer Value Proposition โ€” What We Need to Communicate + +The community message must be concrete and compelling, not abstract. Avoid "unified management plane" as the opener โ€” lead with what operators get: + +**For operator developers:** +> "Add DCM support to your operator and your users get self-service catalog, multi-tenancy, cost attribution, and cross-cluster management โ€” for free. It takes one day using our SDK." + +**For platform engineering teams:** +> "Manage your entire data center from one control plane. VMs, databases, Kubernetes clusters, networking โ€” all with the same declarative model, the same policy engine, and the same audit trail." + +**For FSI/regulated industry teams:** +> "Every provisioning request produces a complete audit chain โ€” who asked for what, what policies applied, what was approved, what was built. Sovereignty constraints enforced at the management plane, not bolted on afterward." + +### 4.4 Contributor Onboarding + +A project cannot become a standard without contributors beyond the founding organization. The contributor onboarding strategy: + +**Good first issues:** +Maintain a curated list of well-scoped, well-documented issues labeled `good-first-issue`. These should be achievable in a few hours without deep DCM knowledge โ€” documentation improvements, test coverage, example implementations, SDK feature additions. + +**Operator SDK examples:** +Each operator SDK example is a potential contributor touchpoint. An operator maintainer who wants to add DCM support to their operator is a natural contributor. The example for their specific CRD framework (kubebuilder, operator-sdk, raw controller-runtime) lowers the barrier. + +**RFC process:** +Establish a lightweight RFC (Request for Comments) process for significant changes to the DCM Operator Interface Specification. This gives external contributors a formal path to influence the specification direction โ€” which is essential for community trust. + +**Monthly community calls:** +Regular community calls (video, recorded, published) signal active project health and give contributors a forum to discuss ideas. Target: bi-weekly during active development, monthly once stable. + +--- + +## 5. Standards Positioning + +Beyond CNCF, DCM should engage with relevant standards bodies where appropriate: + +### 5.1 DMTF (Distributed Management Task Force) +DMTF maintains the TOSCA (Topology and Orchestration Specification for Cloud Applications) and other cloud management standards. DCM's data model has some conceptual overlap with TOSCA. Rather than competing, DCM should position as a Kubernetes-native, GitOps-native evolution of the same problem TOSCA addressed โ€” bringing the conversation into the cloud native era. + +### 5.2 FinOS Foundation +FinOS is the open source community for financial services. The FSI consortium involvement makes FinOS a natural secondary community for DCM. Presenting DCM at FinOS events reaches exactly the regulated industry audience that benefits most from DCM's sovereignty and compliance capabilities. + +### 5.3 OpenInfra Foundation +The OpenInfra Foundation hosts OpenStack, Kata Containers, and StarlingX โ€” all relevant to DCM's target environments (private cloud, edge, regulated infrastructure). DCM should present at OpenInfra Summit to the platform engineering teams who manage these environments. + +--- + +## 6. What Needs to Exist Before CNCF Submission + +The following artifacts must be ready before a CNCF Sandbox submission is credible: + +| Artifact | Status | Owner | Target | +|----------|--------|-------|--------| +| DCM Operator Interface Specification v1.0 | ๐Ÿ”„ Draft | DCM Project | Ready | +| DCM Operator SDK v0.1.0 (Level 1 + Level 2) | ๐Ÿ“‹ Not started | DCM Project | 3-6 months | +| KubeVirt reference implementation (Level 2) | ๐Ÿ”„ In progress | DCM/KubeVirt teams | 3-6 months | +| Conformance test suite (Level 1 + Level 2) | ๐Ÿ“‹ Not started | DCM Project | 3-6 months | +| CNCF Due Diligence document | ๐Ÿ“‹ Not started | DCM Project | 6 months | +| Governance model document | ๐Ÿ“‹ Not started | Red Hat/Consortium | 3 months | +| Security vulnerability disclosure process | ๐Ÿ“‹ Not started | Red Hat Security | 1 month | +| FSI consortium adopter statements | ๐Ÿ“‹ Not started | Consortium members | 3 months | +| Second non-Red Hat maintainer | ๐Ÿ“‹ Not started | Community | 6 months | + +--- + +## 7. Risk Considerations + +| Risk | Likelihood | Impact | Mitigation | +|------|-----------|--------|------------| +| CNCF TAG sees overlap with Crossplane | Medium | Medium | Prepare clear differentiation โ€” DCM governs, Crossplane provisions; they are complementary | +| Operator communities resist specification adoption | Medium | High | Lead with SDK ease, reference implementations, concrete value; don't mandate, make it attractive | +| Red Hat perceived as controlling the standard | Medium | High | Establish CNCF governance early, actively recruit non-Red Hat maintainers, FSI consortium co-ownership | +| Specification fragmentation โ€” forks or competing standards | Low | High | CNCF neutral governance prevents this; be the first mover in this space | +| Key contributor departure | Low | Medium | CNCF governance ensures project continuity beyond any single contributor | + +--- + +## 8. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the CNCF submission be for DCM as a whole or for the DCM Operator Interface Specification as a standalone standard? | Scope of submission | โ“ Unresolved | +| 2 | Which FSI consortium members are willing to be named as public adopters in the CNCF submission? | Submission strength | โ“ Unresolved | +| 3 | Is there a TOC member with relevant expertise who could sponsor the DCM proposal? | Submission path | โ“ Unresolved | +| 4 | Should DCM engage with the Kubernetes SIG structure before or after CNCF Sandbox submission? | Community positioning | โ“ Unresolved | +| 5 | What is the timeline for the KubeVirt reference implementation reaching Level 2 conformance? | Readiness milestone | โ“ Unresolved | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/specifications/kubernetes-compatibility.md b/content/docs/specifications/kubernetes-compatibility.md new file mode 100644 index 0000000..ed9ce12 --- /dev/null +++ b/content/docs/specifications/kubernetes-compatibility.md @@ -0,0 +1,347 @@ +--- +title: Kubernetes Compatibility +type: docs +weight: 2 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) + +--- + +## 1. Purpose + +DCM is designed as a **superset of Kubernetes** โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries that Kubernetes alone cannot address. + +This document serves three purposes: + +1. **Defines the formal mapping** between Kubernetes concepts and DCM concepts โ€” enabling implementors to understand how the two models relate and where DCM extends beyond Kubernetes +2. **Establishes DCM Resource Types** for standard Kubernetes resources โ€” so that Kubernetes-managed resources participate in the DCM registry alongside non-Kubernetes resources +3. **Documents the boundary** between what Kubernetes governs and what DCM governs โ€” making clear that DCM extends Kubernetes rather than replacing it + +--- + +## 2. The Superset Relationship + +### 2.1 What Kubernetes Provides + +Kubernetes is a container orchestration platform that provides: +- Declarative desired-state management within a single cluster +- A controller/operator pattern for extending resource management +- Namespace-based isolation within a cluster +- RBAC for access control within a cluster +- A rich ecosystem of operators for managing complex stateful resources + +### 2.2 What DCM Adds + +DCM extends Kubernetes upward by providing: + +| Capability | Kubernetes | DCM | +|------------|-----------|-----| +| Scope | Single cluster | Multi-cluster, multi-infrastructure | +| Tenancy | Namespace isolation | First-class Tenant model with ownership | +| Policy | RBAC + admission webhooks | Full Policy Engine with Validation/Transformation/GateKeeper | +| Data lineage | Not provided | Field-level provenance on all data | +| Cost attribution | Not provided | Full lifecycle cost analysis | +| Drift detection | Basic โ€” controller reconciles | Full four-state model with Intent/Requested/Realized/Discovered | +| Service catalog | Not provided | Full self-service catalog with RBAC-governed presentation | +| Sovereignty | Not provided | Sovereignty declarations, placement constraints, compliance evidence | +| Information context | Labels/annotations | First-class Information Provider relationships | +| Non-Kubernetes resources | Not provided | VMware, bare metal, OpenStack, etc. all managed through same model | + +### 2.3 What DCM Does Not Replace + +DCM does not replace Kubernetes at the runtime level. Kubernetes continues to: +- Schedule and run containers +- Manage Pod lifecycle within a cluster +- Enforce network policies within a cluster +- Provide the Kubernetes API for cluster-native tooling +- Run operators that manage complex stateful resources + +DCM manages the management plane โ€” the lifecycle of what gets requested, provisioned, owned, governed, and decommissioned. Kubernetes manages the execution plane โ€” the runtime behavior of what is running. + +--- + +## 3. Core Concept Mappings + +### 3.1 Resource Model + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Custom Resource Definition (CRD) | Resource Type Specification | CRD schema โ†’ DCM Resource Type fields | DCM Resource Type is the portable, provider-agnostic equivalent. CRD is the Kubernetes-specific implementation schema. | +| Custom Resource (CR) | Requested State payload โ†’ Realized State entity | CR is the naturalized form of the DCM payload | The operator translates DCM Requested State into a CR (Naturalization) and translates CR status back to DCM Realized State (Denaturalization). | +| Built-in resource (Pod, Service, PV) | DCM Resource Type in Compute.*, Network.*, Storage.* | Kubernetes built-ins are valid DCM Resource Types | See Section 5 for standard Kubernetes resource type mappings. | +| Kubernetes object | Resource/Service Entity | Every Kubernetes object managed by DCM has a corresponding DCM entity with UUID and provenance | | + +### 3.2 Control Loop + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Operator reconciliation loop | Realization + Drift Detection combined | Reconciliation IS the realization process โ€” the operator drives actual state toward desired state | DCM's Drift Detection compares Discovered State against Realized State. The operator's reconciliation loop is the mechanism that corrects drift. | +| Desired state (CR spec) | Requested State | CR spec is the naturalized form of the DCM Requested State | DCM stores the Requested State in DCM format. The operator translates it to CR spec format. | +| Actual state (CR status) | Realized State | CR status is the Kubernetes-native form of the DCM Realized State | The operator must denaturalize CR status back to DCM Realized State format and report it to DCM. | +| Watch/Inform pattern | DCM Discovered State polling | Kubernetes watch events are the mechanism for keeping DCM Discovered State current | | + +### 3.3 Isolation and Multi-tenancy + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Namespace | DCM Tenant boundary | One namespace per DCM Tenant (per_tenant strategy) | Kubernetes namespace provides the physical isolation enforcement. DCM Tenant provides the ownership and governance model. A single DCM Tenant maps to exactly one namespace per cluster. | +| Namespace | DCM Resource Group | In shared namespace strategies, Resource Group labels replace namespace isolation | When multiple Tenants share a namespace, DCM Resource Group labels provide logical separation. | +| Kubernetes RBAC | DCM IDM/IAM + Policy Engine | Kubernetes RBAC is the runtime enforcement mechanism. DCM Policy Engine governs who can request what via the service catalog. | DCM policies determine what a user can request. Kubernetes RBAC determines what a running workload can do. These are complementary, not duplicative. | +| ServiceAccount | DCM Identity.ServiceAccount Information Type | Kubernetes ServiceAccounts that DCM provisions or references are modeled as DCM Information Type entities | | + +### 3.4 Relationships and Dependencies + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| ownerReference | Entity Relationship (`contains`/`contained_by`) | Kubernetes ownerReferences are a subset of DCM entity relationships โ€” ownership only | DCM relationships are richer โ€” supporting `requires`, `depends_on`, `references`, `peer`, `manages` in addition to ownership. During Denaturalization, ownerReferences are translated to DCM `contains` relationships. | +| Finalizers | Lifecycle policy (`retain`, `detach`) | Kubernetes finalizers implement DCM lifecycle policies at the Kubernetes level | When DCM declares `on_parent_destroy: retain` for a storage entity, the operator implements this using Kubernetes finalizers to prevent deletion until DCM confirms the lifecycle policy has been applied. | +| Label selectors | Resource Group membership | Kubernetes label selectors used for DCM Resource Group filtering | DCM mandatory labels (`dcm-tenant-id`, `dcm-entity-id`) are used as label selectors for Resource Group queries. | + +### 3.5 Data Model + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Labels | DCM entity metadata + relationships | DCM-mandatory labels (`dcm-managed`, `dcm-tenant-id`, `dcm-entity-id`, etc.) carry core DCM identity data. Custom labels may map to DCM Information Type relationships. | | +| Annotations | DCM field-level provenance + metadata | Annotations used by DCM to carry request correlation data during the request lifecycle | `dcm-request-id` annotation on a CR identifies the DCM request that created or last modified it โ€” enabling unsanctioned change detection. | +| Resource version | Entity version (Revision component) | Kubernetes resource versions map to DCM entity Revision increments | Major and Minor versions are managed by DCM based on breaking/non-breaking changes. Kubernetes resource version increments map to DCM Revision increments. | +| Generation | Requested State version | CR generation increments correspond to new DCM Requested State records | Each new generation of a CR corresponds to a new intent/request cycle in DCM. | + +### 3.6 Lifecycle + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Pod phases (Pending, Running, Succeeded, Failed, Unknown) | DCM lifecycle states | Pod phases map to DCM lifecycle states via condition_mappings declaration | | +| CRD conditions | DCM lifecycle states and events | Standard conditions (Ready, Degraded, Progressing) map to DCM states and events via the field mapping specification | | +| Kubernetes events | DCM lifecycle events | Kubernetes watch events trigger DCM lifecycle event reports | The operator translates Kubernetes events into DCM lifecycle event types (ENTITY_HEALTH_CHANGE, DEGRADATION, UNSANCTIONED_CHANGE, etc.) | +| Cluster deletion | DCM decommission workflow | Cluster deletion triggers DCM's full decommission lifecycle โ€” lifecycle policies applied to all related entities | | + +--- + +## 4. Where DCM Extends Beyond Kubernetes + +These are concepts that exist in DCM but have no Kubernetes equivalent. They are the capabilities DCM adds that justify the superset positioning. + +### 4.1 Intent State + +Kubernetes has no concept of a consumer's original intent separate from the desired state. Once you apply a manifest, Kubernetes only knows the current desired state โ€” not what the consumer originally asked for or why. + +DCM's Intent State is the immutable record of what the consumer asked for, stored before any policy processing or layer enrichment. This enables: +- Rehydration โ€” replaying the original intent through current policies to produce a new request +- Intent portability โ€” the same intent applied to a different provider +- Audit โ€” answering "what did the consumer originally ask for?" independently of what was realized + +### 4.2 Field-Level Provenance + +Kubernetes has no concept of where a field value came from or why it was set. A field in a CR spec is a field โ€” there is no lineage. + +DCM's field-level provenance carries the full lineage of every field value through the entire lifecycle โ€” which layer set it, which policy modified it, which provider realized it, and why each change was made. This enables complete audit trails and sovereignty evidence. + +### 4.3 Data Layers and Assembly + +Kubernetes has no equivalent to DCM's layering model. A Kubernetes manifest is a flat declaration โ€” there is no concept of organizational standards, site-specific configuration, and service-specific configuration being separate layers that compose into a final manifest. + +DCM's layering model enables 36 layer definitions to govern 40,000 VMs without duplication โ€” impossible in the Kubernetes model. + +### 4.4 Policy Engine + +Kubernetes admission webhooks provide some policy capability (validation, mutation) but are cluster-scoped, apply at admission time only, and have no concept of hierarchy (Global โ†’ Tenant โ†’ User policy levels) or field-level override control. + +DCM's Policy Engine operates at the management plane level, applies across all clusters and providers, enforces a three-level hierarchy with field-level override control (allow/constrained/immutable), and carries policy decisions as provenance metadata in the payload. + +### 4.5 Cost Analysis + +Kubernetes has no native cost attribution model. Tools like Kubecost exist but are add-ons with no integration into the request lifecycle. + +DCM's cost analysis is built into the lifecycle model โ€” cost attribution is tracked from request time through realization, operation, and decommission for every entity. + +### 4.6 Information Providers + +Kubernetes has no concept of structured relationships to external organizational data (Business Units, Cost Centers, Product Owners). Labels and annotations are unstructured key-value pairs with no type safety, no external system integration, and no verification model. + +DCM's Information Provider model gives every entity structured, verified, versioned relationships to external organizational data with a stable external key model. + +### 4.7 Cross-Cluster Lifecycle + +Kubernetes manages resources within a single cluster. Multi-cluster management requires additional tools (ACM, Argo CD, Fleet) that are not part of the core Kubernetes model. + +DCM manages the lifecycle of resources across multiple clusters as a first-class capability โ€” the same Resource Type can be instantiated on any cluster that has a conformant Service Provider registered. + +--- + +## 5. Standard Kubernetes Resource Type Mappings + +These are the DCM Resource Type registry entries for standard Kubernetes resource types. Operators implementing these types should use these registry UUIDs and field definitions. + +### 5.1 Compute + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Compute.Pod` | Pod | Lowest-level compute unit | +| `Compute.Container` | Container (within a Pod) | Sub-entity of Pod โ€” expanded via bundled declaration | +| `Compute.Deployment` | Deployment | Managed set of Pods | +| `Compute.StatefulSet` | StatefulSet | Stateful managed set of Pods | +| `Compute.Job` | Job | One-time execution workload | +| `Compute.CronJob` | CronJob | Scheduled execution workload | + +### 5.2 Network + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Network.Service` | Service | In-cluster service discovery and load balancing | +| `Network.Ingress` | Ingress | External HTTP/HTTPS routing | +| `Network.NetworkPolicy` | NetworkPolicy | In-cluster network isolation | + +### 5.3 Storage + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Storage.PersistentVolume` | PersistentVolume | Cluster-level storage resource | +| `Storage.PersistentVolumeClaim` | PersistentVolumeClaim | Consumer's storage declaration โ€” expanded into Storage.PersistentVolume relationship | +| `Storage.StorageClass` | StorageClass | Storage type definition โ€” maps to DCM Provider Catalog Item | +| `Storage.ConfigMap` | ConfigMap | Configuration data storage | +| `Storage.Secret` | Secret | Sensitive data storage | + +### 5.4 Platform + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Platform.KubernetesCluster` | Kubernetes Cluster (via CAPI or managed service) | The cluster itself is a DCM-managed resource | +| `Platform.Namespace` | Namespace | Maps to DCM Tenant boundary in per_tenant strategy | +| `Platform.CustomResourceDefinition` | CRD | CRD registration maps to DCM Resource Type registration | + +### 5.5 Identity + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Security.ServiceAccount` | ServiceAccount | Kubernetes identity for workloads | +| `Security.Role` | Role / ClusterRole | Kubernetes RBAC role | +| `Security.RoleBinding` | RoleBinding / ClusterRoleBinding | Kubernetes RBAC binding | + +--- + +## 6. The Kubernetes Information Provider + +Kubernetes clusters function as both Service Providers (for provisioning resources) and Information Providers (for querying existing state). As an Information Provider, a Kubernetes cluster exposes its current resource state to DCM for: + +- **Brownfield ingestion** โ€” discovering existing resources and bringing them under DCM lifecycle management +- **Discovered State** โ€” DCM's Discovered State for Kubernetes resources comes from querying the Kubernetes API +- **Drift detection** โ€” comparing DCM Realized State against what Kubernetes actually has + +### 6.1 Kubernetes as Information Provider Registration + +```yaml +information_provider_registration: + name: kubernetes-cluster-01 + implements: + - information_type: Platform.KubernetesCluster + - information_type: Compute.Pod + - information_type: Storage.PersistentVolume + # ... all resource types the cluster contains + endpoint: + kubernetes_credentials: + auth_method: + discovery_capabilities: + label_selector: "dcm-managed=true" + # Only returns DCM-managed resources by default + full_discovery: true + # Can also return all resources for brownfield ingestion +``` + +### 6.2 Discovered State from Kubernetes + +DCM queries the Kubernetes API using the Kubernetes Information Provider to populate Discovered State: + +``` +DCM Drift Detection + โ”‚ + โ–ผ +Kubernetes Information Provider + โ”‚ GET /apis/{group}/{version}/namespaces/{ns}/{kind} + โ”‚ Filter: label dcm-entity-id = {entity_uuid} + โ–ผ +Discovered State payload (DCM format) + โ”‚ Kubernetes object denaturalized to DCM format + โ–ผ +Compare against Realized State + โ”‚ Field-by-field comparison + โ–ผ +UNSANCTIONED_CHANGE if differences found + โ”‚ Reported to Policy Engine for response determination +``` + +--- + +## 7. Kubernetes-Native Patterns and DCM Equivalents + +### 7.1 GitOps + +Kubernetes GitOps (Argo CD, Flux) manages Kubernetes manifests in Git and synchronizes them to clusters. DCM's data model is also Git-based โ€” all layers, Resource Type definitions, and policy definitions are stored in Git. + +The relationship: DCM manages the **request lifecycle** (what gets asked for, approved, and provisioned). GitOps manages the **deployment lifecycle** (what gets deployed to a cluster from a Git repository). These are complementary: + +- DCM governs the provisioning request โ€” "is this consumer allowed to provision this resource?" +- GitOps deploys application code to the provisioned resource +- DCM and GitOps together form a complete lifecycle: DCM provisions the cluster, GitOps deploys applications to it + +### 7.2 Helm + +Helm charts are packages of Kubernetes manifests that can be parameterized. In DCM terms, a Helm chart is a form of Catalog Item โ€” a curated, parameterized offering of a set of Kubernetes resources. + +DCM does not replace Helm โ€” it can use Helm as a delivery mechanism inside a Service Provider. The Service Provider receives the DCM Requested State, translates it to Helm values, and uses Helm to deploy the resources. The operator pattern is preferred for Day 2 management (Helm has limited reconciliation), but Helm remains valid for initial provisioning. + +### 7.3 Cluster API (CAPI) + +CAPI is the Kubernetes sub-project for managing Kubernetes clusters themselves using the Kubernetes API and operator pattern. CAPI clusters are a natural fit for DCM's `Platform.KubernetesCluster` Resource Type โ€” a CAPI-based operator would be the Service Provider for provisioning new Kubernetes clusters as DCM-managed resources. + +This is particularly significant: DCM managing the lifecycle of Kubernetes clusters through CAPI means DCM can provision the very infrastructure that operators run on. The superset relationship becomes concrete โ€” DCM provisions the cluster, the cluster runs the operators, the operators provision the resources that DCM manages. + +--- + +## 8. Migration Path โ€” Kubernetes-Native to DCM-Managed + +Organizations running Kubernetes today can adopt DCM incrementally: + +### Phase 1 โ€” Observation (no operator changes) +Deploy DCM with the Kubernetes Information Provider. DCM observes existing resources via the Kubernetes API and builds a Discovered State inventory. No changes to existing operators or workloads. + +### Phase 2 โ€” Brownfield Ingestion (no operator changes) +DCM promotes Discovered State records to Realized State โ€” assuming lifecycle management of existing resources. Resources get DCM UUIDs, Tenant assignments, and provenance records. Existing resources are now DCM-managed without any operator changes. + +### Phase 3 โ€” Level 1 Conformance (minimal operator changes) +Operators implement Level 1 of this specification via the DCM Operator SDK. New resources are provisioned through DCM's service catalog. Existing resources managed via brownfield ingestion continue as-is. + +### Phase 4 โ€” Level 2 Conformance (moderate operator changes) +Operators implement Level 2 โ€” full field mappings, capacity reporting, lifecycle events. DCM gains placement intelligence, drift detection, and cross-cluster management capabilities. + +### Phase 5 โ€” Level 3 Conformance (complete integration) +Operators implement Level 3 โ€” sovereignty declarations, provenance, discovery endpoint. Full DCM capabilities available. + +--- + +## 9. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How does the Namespace-to-Tenant mapping work when a cluster has existing namespaces that predate DCM adoption? | Brownfield migration | โ“ Unresolved | +| 2 | Should `Platform.KubernetesCluster` be the boundary for a DCM deployment, or can DCM manage resources across clusters without treating the cluster as a DCM entity? | Architecture scope | โ“ Unresolved | +| 3 | How does DCM interact with Kubernetes admission webhooks โ€” do they duplicate Policy Engine functions or complement them? | Policy model | โ“ Unresolved | +| 4 | Should the Kubernetes Information Provider be a built-in DCM component or a separately deployed provider? | Deployment architecture | โ“ Unresolved | +| 5 | How does the DCM superset model interact with managed Kubernetes services (EKS, GKE, AKS) where cluster management is outside the user's control? | Cloud provider integration | โ“ Unresolved | + +--- + +## 10. Related Concepts + +- **DCM Operator Interface Specification** โ€” the technical contract for operators integrating with DCM +- **DCM Operator SDK** โ€” Go library implementing this specification for operator developers +- **Entity Relationships** โ€” DCM's universal relationship model, of which Kubernetes ownerReferences are a subset +- **Resource Type Hierarchy** โ€” the DCM registry where Kubernetes Resource Types are registered +- **Information Providers** โ€” the DCM model for the Kubernetes API as a discoverable information source +- **Four States** โ€” DCM's Intent/Requested/Realized/Discovered model, which extends Kubernetes' desired/actual model + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/specifications/operator-interface-spec.md b/content/docs/specifications/operator-interface-spec.md new file mode 100644 index 0000000..a2a99ae --- /dev/null +++ b/content/docs/specifications/operator-interface-spec.md @@ -0,0 +1,830 @@ +--- +title: DCM Operator Interface Specification +type: docs +weight: 1 +--- + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Not yet ratified +**Document Type:** Technical Specification +**Maintainers:** Red Hat FlightPath Team +**GitHub:** https://github.com/dcm-project +**Last Updated:** 2026-03 + +--- + +## Abstract + +This specification defines the interface by which Kubernetes operators integrate with the DCM (Data Center Management) control plane as first-class Service Providers. An operator that conforms to this specification becomes a DCM Service Provider, enabling its managed resources to participate in DCM's unified lifecycle management, multi-tenancy, policy governance, cost analysis, drift detection, and service catalog. + +DCM is designed as a superset of Kubernetes โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries. This specification is the technical contract that enables that extension without requiring operators to abandon their existing Kubernetes-native design. + +--- + +## 1. Introduction + +### 1.1 Motivation + +Kubernetes operators are the most mature pattern for managing complex, stateful resources declaratively on Kubernetes. However, operators operate within a single cluster and lack the cross-cluster lifecycle management, multi-tenancy, cost attribution, sovereignty governance, and policy enforcement that enterprise organizations require at scale. + +DCM provides these capabilities at the management plane level โ€” above individual clusters. By conforming to this specification, an operator's managed resources become: + +- **Multi-tenant** โ€” DCM Tenant ownership and isolation applied automatically +- **Cost-attributed** โ€” resource costs tracked and attributed across the full lifecycle +- **Policy-governed** โ€” organizational policies applied at request time via DCM's Policy Engine +- **Cross-cluster** โ€” the same resource type managed across multiple clusters through DCM +- **Self-service** โ€” automatically available in the DCM Service Catalog for consumer request +- **Sovereignty-compliant** โ€” placement and operational constraints enforced by DCM's GateKeeper policies +- **Audit-complete** โ€” full provenance chain from intent through realization + +### 1.2 Scope + +This specification defines: +- The HTTP API an operator must expose to participate in DCM +- The data format for all API payloads (DCM Unified Data Model) +- The registration, health, capacity, status, and lifecycle event contracts +- The field mapping specification for translating between DCM format and CRD format +- Conformance levels and what each level unlocks in DCM + +This specification does not define: +- How operators implement their internal reconciliation logic +- Which specific Kubernetes distributions operators must support +- The internal architecture of the DCM control plane +- Provider-specific business logic or domain knowledge + +### 1.3 Relationship to the DCM Service Provider Contract + +This specification is a Kubernetes-specific instantiation of the DCM Service Provider Contract. All general Service Provider Contract requirements apply. This specification adds Kubernetes-specific requirements and guidance. Where this specification and the general Service Provider Contract conflict, this specification takes precedence for Kubernetes operator implementations. + +### 1.4 Terminology + +- **Operator** โ€” a Kubernetes controller that manages custom resources via a Custom Resource Definition (CRD) +- **DCM Control Plane** โ€” the DCM management system that routes requests and manages lifecycle +- **Adapter** โ€” a component that sits between DCM and an operator, implementing this specification on the operator's behalf (used when the operator cannot be modified directly) +- **Native implementation** โ€” an operator that implements this specification directly, without an adapter +- **CR** โ€” Custom Resource โ€” an instance of a CRD managed by the operator +- **CRD** โ€” Custom Resource Definition โ€” the Kubernetes schema definition for a CR +- **Reconciliation loop** โ€” the operator's control loop that drives actual state toward desired state + +--- + +## 2. Conformance Levels + +This specification defines three conformance levels. Higher levels unlock additional DCM capabilities. An operator may implement any level โ€” DCM accepts operators at all levels, with capabilities gated by the declared conformance level. + +**Design principle:** Level 1 must be achievable in a single day of work for an existing operator. Level 3 is the target for operators that want full DCM integration. The SDK (see Section 9) handles all protocol concerns โ€” operator developers only implement business logic. + +### 2.1 Level 1 โ€” Basic + +**What it requires:** +- Operator registration with DCM on startup +- Health check endpoint (`GET /health`) +- Basic status reporting to DCM when resource state changes + +**What it unlocks:** +- Operator resources appear in the DCM Service Catalog +- Basic lifecycle state tracking (PROVISIONING, OPERATIONAL, FAILED, DECOMMISSIONED) +- Health monitoring via DCM Observability +- Basic cost tracking (resource exists/does not exist) + +**Estimated implementation effort:** 1 day using the DCM Operator SDK + +### 2.2 Level 2 โ€” Standard + +**What it requires:** All Level 1 requirements, plus: +- Capacity reporting to DCM (scheduled registration) +- Full lifecycle event reporting (DEGRADED, MAINTENANCE, UNSANCTIONED_CHANGE, etc.) +- Complete realized state payloads in DCM Unified Data Model format +- Field mapping declaration (CRD fields mapped to DCM Resource Type fields) + +**What it unlocks:** All Level 1 capabilities, plus: +- Intelligent placement โ€” DCM can route requests based on real capacity data +- Drift detection โ€” DCM compares discovered state against realized state +- Full cost attribution โ€” granular resource cost tracking throughout lifecycle +- Cross-cluster management โ€” DCM can route the same resource type to multiple clusters +- Dependency graph participation โ€” operator resources participate in DCM entity relationships + +**Estimated implementation effort:** 2-3 days using the DCM Operator SDK + +### 2.3 Level 3 โ€” Full + +**What it requires:** All Level 2 requirements, plus: +- Sovereignty capability declaration +- Field-level provenance in realized state payloads +- Override control metadata support +- Discovery endpoint (`POST /discover`) โ€” operator can discover existing resources for brownfield ingestion +- Decommission confirmation callback + +**What it unlocks:** All Level 2 capabilities, plus: +- Sovereignty enforcement โ€” DCM can enforce placement and operational constraints per regulatory requirements +- Full audit chain โ€” complete provenance from intent through realization +- Brownfield ingestion โ€” existing resources can be imported into DCM lifecycle management +- Override control enforcement โ€” policy-set field locks honored in operator requests + +**Estimated implementation effort:** 3-5 days using the DCM Operator SDK + +--- + +## 3. Registration API + +### 3.1 Overview + +Operators register with DCM on startup. Registration informs DCM of the operator's endpoint, the resource types it manages, its capabilities, and its conformance level. Registration is idempotent โ€” re-registering with the same name updates the existing registration rather than creating a duplicate. + +### 3.2 Registration Endpoint + +**DCM endpoint:** `POST /api/v1/providers` + +**Timing:** Called by the operator (or adapter) during startup, after the HTTP server is ready. Retried with exponential backoff on failure. Registration failure does not block operator startup โ€” the operator functions normally for Kubernetes consumers even if DCM registration fails. + +### 3.3 Registration Payload + +```yaml +# Registration request payload +provider_registration: + name: + display_name: + conformance_level: <1|2|3> + endpoint: + version: + + service_types: + - service_type: + service_type_uuid: + crd_reference: + group: + version: + kind: + operations_supported: [CREATE, READ, UPDATE, DELETE, DISCOVER] + # DISCOVER only required for Level 3 + field_mapping_ref: + + kubernetes: + cluster_id: + cluster_endpoint: + namespace_strategy: + # per_tenant: one namespace per DCM Tenant + # shared: all DCM resources in one namespace, isolated by labels + # per_resource: one namespace per resource instance + + metadata: + region: + zone: + cluster_type: + cluster_version: + + # Level 2+ required + capacity: + update_mode: + update_frequency_seconds: + + # Level 3 required + sovereignty_capabilities: + data_residency_regions: [] + operational_sovereignty: + hard_tenancy_supported: + air_gapped_capable: + compliance_frameworks: [] +``` + +### 3.4 Registration Response + +```yaml +# Success response +provider_registration_response: + provider_id: + name: + status: + conformance_level_accepted: <1|2|3> + capabilities_enabled: + - service_catalog + - health_monitoring + - cost_tracking + # Level 2+ + - placement + - drift_detection + - cross_cluster_management + # Level 3 + - sovereignty_enforcement + - brownfield_ingestion + - full_audit_chain +``` + +--- + +## 4. Health Check API + +### 4.1 Overview + +DCM polls the operator's health endpoint every 10 seconds (configurable). A healthy operator is eligible to receive new resource requests. An unhealthy operator is excluded from placement decisions. + +### 4.2 Health Endpoint + +**Endpoint:** `GET /health` +**Authentication:** Unauthenticated (or internally secured โ€” operator choice) +**Expected response:** HTTP 200 OK for healthy, any non-200 for unhealthy + +```yaml +# Health response body (optional but recommended) +health_response: + status: pass # pass | warn | fail + version: + uptime_seconds: + kubernetes_connectivity: + dcm_registration_status: + details: + +``` + +### 4.3 State Machine + +- **Ready** โ€” HTTP 200 received. Operator eligible for new requests. +- **NotReady** โ€” Non-200 or timeout received 3 consecutive times (configurable threshold). Operator excluded from placement. Existing resources not affected. +- **Recovery** โ€” Single HTTP 200 transitions NotReady back to Ready immediately. + +--- + +## 5. Capacity Reporting API + +*Required for Level 2 conformance.* + +### 5.1 Overview + +DCM maintains an internal capacity rating per operator, per service type, per location. Operators report capacity on a configurable schedule. DCM uses capacity data for intelligent placement decisions. + +### 5.2 Capacity Registration + +**DCM endpoint:** `POST /api/v1/providers/{provider_id}/capacity` + +```yaml +capacity_report: + provider_id: + report_timestamp: + next_report_at: + capacity_by_service_type: + - service_type_uuid: + available_units: + reserved_units: + committed_units: + unit_definition: + kubernetes_resources: + available_cpu: + available_memory: + available_storage: + node_count: +``` + +### 5.3 Capacity Denial + +When DCM dispatches a request the operator cannot fulfill, the operator **must** reject it with `INSUFFICIENT_RESOURCES`. DCM receives the denial and retries with an alternative provider. + +```yaml +# Denial response to a resource creation request +denial_response: + request_id: + denial_reason: INSUFFICIENT_RESOURCES + denial_timestamp: + service_type_uuid: + estimated_available_at: + details: +``` + +DCM updates its internal capacity rating for this operator immediately upon receiving a denial. + +--- + +## 6. Resource Lifecycle API + +### 6.1 Overview + +DCM dispatches resource lifecycle operations to the operator via standard REST endpoints. The operator translates these into Kubernetes CR operations (Naturalization) and reports results back to DCM in DCM Unified Data Model format (Denaturalization). + +### 6.2 Standard Endpoints + +| Method | Endpoint | Description | Required Level | +|--------|----------|-------------|---------------| +| `POST` | `/api/v1/{service_type}` | Create a new resource | Level 1 | +| `GET` | `/api/v1/{service_type}` | List all resources | Level 1 | +| `GET` | `/api/v1/{service_type}/{resource_id}` | Get a specific resource | Level 1 | +| `PUT` | `/api/v1/{service_type}/{resource_id}` | Update a resource | Level 2 | +| `DELETE` | `/api/v1/{service_type}/{resource_id}` | Delete a resource | Level 1 | +| `POST` | `/api/v1/{service_type}/discover` | Discover existing resources | Level 3 | + +### 6.3 Create Request + +DCM sends the Requested State payload to the operator. The operator naturalizes it to a Kubernetes CR and submits it. The operator responds immediately with a PROVISIONING status โ€” not waiting for reconciliation to complete. + +```yaml +# Create request from DCM โ€” Requested State payload in DCM format +create_request: + request_id: + tenant_uuid: + resource_type_uuid: + resource_type_name: Storage.Database + spec: + + relationships: + + metadata: + override_control: + +``` + +```yaml +# Create response โ€” immediate acknowledgment +create_response: + resource_id: + dcm_request_id: + lifecycle_state: PROVISIONING + kubernetes_reference: + namespace: + name: + uid: +``` + +### 6.4 Realized State Payload + +When the operator's reconciliation loop completes provisioning, it pushes the realized state to DCM. This is the critical Denaturalization step โ€” translating Kubernetes-native status into DCM Unified Data Model format. + +**DCM endpoint:** `PUT /api/v1/instances/{resource_id}/status` + +```yaml +# Realized state payload โ€” DCM Unified Data Model format +realized_state: + resource_id: + dcm_entity_uuid: + lifecycle_state: + realized_timestamp: + + spec: + + + + # Level 3 โ€” provenance for each field + field_provenance: + : + source_type: provider + source_uuid: + timestamp: + + kubernetes_reference: + namespace: + name: + uid: + resource_version: + + relationships: + +``` + +### 6.5 Delete and Decommission + +When DCM requests deletion, the operator deletes the CR and confirms decommission via the realized state endpoint with `lifecycle_state: DECOMMISSIONED`. + +For **Level 3**, the operator must wait for DCM confirmation before deleting โ€” this allows DCM to apply lifecycle policies (retain, detach) before the operator acts. + +```yaml +# Decommission confirmation callback (Level 3) +# DCM calls this before the operator deletes +decommission_confirmation: + resource_id: + lifecycle_policies_applied: + - entity_uuid: + policy_applied: retain + # storage was retained, not deleted with the parent + - entity_uuid: + policy_applied: destroy + proceed_with_deletion: +``` + +--- + +## 7. Field Mapping Specification + +*Required for Level 2 conformance.* + +### 7.1 Overview + +The field mapping declaration tells DCM how to translate between DCM Unified Data Model fields and the operator's CRD fields. This mapping enables DCM to: +- Generate CRs from DCM Requested State payloads (Naturalization) +- Extract DCM Realized State from CR status (Denaturalization) +- Understand which DCM fields correspond to which CRD fields for drift detection + +### 7.2 Field Mapping Declaration Format + +```yaml +field_mapping: + service_type: Storage.Database + service_type_uuid: + crd_reference: + group: postgresql.cnpg.io + version: v1 + kind: Cluster + + # DCM Requested State โ†’ Kubernetes CR (Naturalization) + dcm_to_cr: + - dcm_path: resources.cpu + cr_path: spec.instances[0].resources.requests.cpu + transform: + required: true + + - dcm_path: resources.memory + cr_path: spec.instances[0].resources.requests.memory + transform: gigabytes_to_kubernetes_memory + required: true + + - dcm_path: engine + cr_path: spec.imageName + transform: engine_version_to_image + # engine: postgresql, version: 15 โ†’ imageName: ghcr.io/cloudnative-pg/postgresql:15 + required: true + + - dcm_path: metadata.name + cr_path: metadata.name + required: true + + - dcm_path: tenant_uuid + cr_path: metadata.labels.dcm-tenant-id + required: true + + - dcm_path: dcm_entity_uuid + cr_path: metadata.labels.dcm-entity-id + required: true + # All DCM-managed CRs must be labeled with their DCM entity UUID + # This enables discovery and drift detection + + # Kubernetes CR status โ†’ DCM Realized State (Denaturalization) + cr_status_to_dcm: + - cr_path: status.phase + dcm_path: lifecycle_state + transform: cr_phase_to_dcm_state + # Mapping defined in condition_mappings below + + - cr_path: status.readyInstances + dcm_path: realized_data.ready_instances + transform: none + + - cr_path: status.instancesStatus[0].ip + dcm_path: realized_data.connection.host + transform: none + + - cr_path: status.certificates.serverCASecret + dcm_path: realized_data.tls.ca_secret_ref + transform: none + + # Kubernetes conditions โ†’ DCM lifecycle states + condition_mappings: + - kubernetes_condition: "Ready=True" + dcm_lifecycle_state: OPERATIONAL + + - kubernetes_condition: "Ready=False,Progressing=True" + dcm_lifecycle_state: PROVISIONING + + - kubernetes_condition: "Ready=False,Progressing=False" + dcm_lifecycle_state: FAILED + + - kubernetes_condition: "Degraded=True" + dcm_lifecycle_state: DEGRADED + + # Kubernetes events โ†’ DCM lifecycle events + lifecycle_event_mappings: + - kubernetes_event: condition_change + condition: "Ready=False" + dcm_event: ENTITY_HEALTH_CHANGE + severity: WARNING + + - kubernetes_event: condition_change + condition: "Degraded=True" + dcm_event: DEGRADATION + severity: CRITICAL + + - kubernetes_event: spec_change_without_dcm_request + dcm_event: UNSANCTIONED_CHANGE + severity: WARNING + # Detected when CR spec changes without a corresponding DCM request ID + # Indicates drift โ€” someone modified the CR directly in Kubernetes + + # Namespace strategy implementation + namespace_strategy: + type: per_tenant + namespace_name_pattern: "dcm-{tenant_uuid_short}" + # {tenant_uuid_short} = first 8 chars of tenant UUID + labels_required: + dcm-managed: "true" + dcm-tenant-id: "{tenant_uuid}" + dcm-entity-id: "{entity_uuid}" +``` + +### 7.3 Mandatory CR Labels + +All CRs created by a DCM-conformant operator must carry these labels. These labels enable DCM's discovery and drift detection capabilities: + +| Label | Value | Purpose | +|-------|-------|---------| +| `dcm-managed` | `"true"` | Identifies this CR as DCM-managed | +| `dcm-tenant-id` | DCM Tenant UUID | Tenant ownership | +| `dcm-entity-id` | DCM Entity UUID | Links CR to DCM entity record | +| `dcm-provider-id` | DCM Provider UUID | Which provider created this | +| `dcm-request-id` | DCM Request UUID | Which request created this | + +Any CR change that does not have a corresponding DCM request ID in its update metadata is flagged as an UNSANCTIONED_CHANGE and reported to DCM. + +--- + +## 8. Lifecycle Event API + +*Required for Level 2 conformance.* + +### 8.1 Overview + +Operators must notify DCM of any event that affects the operational status of a managed resource. DCM acts as the Tenant advocate โ€” it receives events, evaluates them through the Policy Engine, and determines the appropriate response. + +### 8.2 Event Endpoint + +**DCM endpoint:** `POST /api/v1/instances/{resource_id}/events` + +### 8.3 Standard Event Types + +| Event Type | Trigger | Severity | Required Level | +|------------|---------|----------|---------------| +| `ENTITY_HEALTH_CHANGE` | CR condition changes | INFO/WARNING | Level 2 | +| `DEGRADATION` | Resource is degraded but operational | WARNING | Level 2 | +| `MAINTENANCE_SCHEDULED` | Planned maintenance window | INFO | Level 2 | +| `MAINTENANCE_STARTED` | Maintenance has begun | INFO | Level 2 | +| `MAINTENANCE_COMPLETED` | Maintenance completed | INFO | Level 2 | +| `UNSANCTIONED_CHANGE` | CR modified without DCM request | WARNING | Level 2 | +| `CAPACITY_CHANGE` | Available capacity changed significantly | INFO | Level 2 | +| `DECOMMISSION_NOTICE` | Operator is shutting down | CRITICAL | Level 2 | +| `PROVIDER_DEGRADATION` | Operator itself is degraded | CRITICAL | Level 2 | + +```yaml +# Event payload +lifecycle_event: + event_uuid: + event_type: UNSANCTIONED_CHANGE + provider_id: + resource_id: + dcm_entity_uuid: + event_timestamp: + severity: WARNING + requires_immediate_action: true + + details: + changed_fields: + - field_path: spec.instances[0].resources.requests.cpu + previous_value: "2000m" + current_value: "4000m" + changed_by: + changed_at: + + kubernetes_reference: + namespace: + name: + resource_version: +``` + +--- + +## 9. DCM Operator SDK + +### 9.1 Overview + +The DCM Operator SDK is an open source Go library that handles all DCM protocol concerns for operator developers. Using the SDK, an operator developer only needs to: + +1. Import the SDK +2. Configure field mappings (declarative YAML) +3. Add SDK hooks at key points in the reconciliation loop + +The SDK handles registration, health check endpoint exposure, capacity reporting, status translation, lifecycle event emission, provenance generation, and label management. + +### 9.2 SDK Initialization + +```go +import dcmsdk "github.com/dcm-project/operator-sdk" + +func main() { + // Load field mapping configuration + mappings, err := dcmsdk.LoadFieldMappings("dcm-mappings.yaml") + + // Initialize DCM SDK + dcm, err := dcmsdk.New(dcmsdk.Config{ + ProviderName: "cloudnativepg-provider", + DisplayName: "CloudNativePG Service Provider", + ConformanceLevel: dcmsdk.Level2, + DCMEndpoint: os.Getenv("DCM_ENDPOINT"), + OperatorEndpoint: os.Getenv("OPERATOR_ENDPOINT"), + FieldMappings: mappings, + CapacityReporter: &PostgresCapacityReporter{}, + }) + + // Start HTTP server with DCM endpoints automatically registered + dcm.StartServer(":8080") + + // Register with DCM on startup + dcm.Register(context.Background()) + + // Start operator manager + mgr.Start(ctrl.SetupSignalHandler()) +} +``` + +### 9.3 Reconciliation Loop Integration + +```go +func (r *ClusterReconciler) Reconcile( + ctx context.Context, + req ctrl.Request, +) (ctrl.Result, error) { + + cluster := &cnpgv1.Cluster{} + if err := r.Get(ctx, req.NamespacedName, cluster); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + + // Check if this CR is DCM-managed + if !r.DCM.IsManagedResource(cluster) { + return ctrl.Result{}, nil + // Not a DCM resource โ€” normal operator behavior + } + + // Detect unsanctioned changes + if r.DCM.IsUnsanctionedChange(cluster) { + r.DCM.ReportEvent(ctx, cluster, dcmsdk.UnsanctionedChange{ + ChangedFields: r.DCM.DetectChangedFields(cluster), + }) + } + + // ... existing reconciliation logic ... + + // Report current state to DCM + realizedState, err := r.DCM.TranslateStatus(cluster) + if err != nil { + return ctrl.Result{}, err + } + r.DCM.ReportStatus(ctx, cluster, realizedState) + + return ctrl.Result{}, nil +} +``` + +### 9.4 SDK Responsibilities + +The SDK automatically handles: +- Self-registration on startup with retry and exponential backoff +- Health check HTTP endpoint (`GET /health`) +- Capacity reporting on configurable schedule +- CR label injection on creation (`dcm-managed`, `dcm-tenant-id`, etc.) +- Unsanctioned change detection (spec change without DCM request ID) +- Status translation using field mapping configuration +- Lifecycle event formatting and delivery to DCM +- Provenance metadata generation for realized state payloads (Level 3) + +--- + +## 10. Kubernetes-to-DCM Concept Mappings + +Understanding how Kubernetes concepts map to DCM concepts is essential for implementing this specification correctly. + +| Kubernetes Concept | DCM Concept | Notes | +|-------------------|-------------|-------| +| Custom Resource Definition (CRD) | Resource Type Specification | CRD schema maps to DCM Resource Type fields | +| Custom Resource (CR) | Requested State โ†’ Realized State | CR is the naturalized form of the DCM payload | +| Operator reconciliation loop | Realization + Drift Detection | Reconciliation IS the realization process | +| CR status subresource | Realized State payload | Status must be denaturalized to DCM format | +| Kubernetes Namespace | DCM Tenant boundary | One namespace per Tenant (per_tenant strategy) | +| ownerReference | Entity Relationship | ownerReferences map to `contains`/`contained_by` relationships | +| Labels/Annotations | DCM Entity metadata | DCM-specific labels declared as mandatory | +| Finalizers | Lifecycle policy enforcement | Finalizers implement `retain` lifecycle policies | +| Kubernetes conditions | DCM lifecycle states | Mapped via condition_mappings declaration | +| Watch events | DCM lifecycle events | Kubernetes watch โ†’ DCM event translation | +| Kubernetes RBAC | DCM IDM/IAM + Policy Engine | Kubernetes RBAC is the runtime enforcement; DCM Policy Engine governs the request | +| Kubernetes cluster | DCM Resource Type: Platform.KubernetesCluster | The cluster itself is a DCM-managed resource | + +--- + +## 11. Conformance Testing + +### 11.1 Overview + +The DCM project provides a conformance test suite that validates an operator's implementation against this specification. Operators that pass the conformance test suite at their declared level can claim DCM conformance. + +### 11.2 Test Suite Structure + +``` +dcm-operator-conformance/ +โ”œโ”€โ”€ level1/ +โ”‚ โ”œโ”€โ”€ registration_test.go +โ”‚ โ”œโ”€โ”€ health_check_test.go +โ”‚ โ””โ”€โ”€ basic_status_test.go +โ”œโ”€โ”€ level2/ +โ”‚ โ”œโ”€โ”€ capacity_test.go +โ”‚ โ”œโ”€โ”€ lifecycle_events_test.go +โ”‚ โ”œโ”€โ”€ realized_state_test.go +โ”‚ โ””โ”€โ”€ field_mapping_test.go +โ””โ”€โ”€ level3/ + โ”œโ”€โ”€ sovereignty_test.go + โ”œโ”€โ”€ provenance_test.go + โ”œโ”€โ”€ discovery_test.go + โ””โ”€โ”€ decommission_confirmation_test.go +``` + +### 11.3 Running the Conformance Tests + +```bash +# Run Level 1 conformance tests against a running operator +dcm-conformance test \ + --level 1 \ + --operator-endpoint https://my-operator:8080 \ + --dcm-endpoint https://dcm-control-plane:8080 \ + --service-type Storage.Database + +# Run all levels +dcm-conformance test --level 3 --operator-endpoint ... +``` + +### 11.4 Conformance Certification + +Operators that pass the conformance test suite may: +- Use the "DCM Compatible โ€” Level N" badge in their documentation +- Be listed in the DCM Operator Registry +- Receive inclusion in the DCM default Service Catalog for participating organizations + +--- + +## 12. Security Considerations + +### 12.1 Authentication + +DCM authenticates outbound requests to operators using the trust model established during registration. Operators must validate that incoming requests originate from the DCM control plane. The specific authentication mechanism is declared in the provider registration: + +```yaml +trust_declaration: + auth_method: + auth_config: +``` + +### 12.2 Namespace Isolation + +When using the `per_tenant` namespace strategy, operators must enforce that resources in one namespace cannot access resources in another namespace. This is the physical enforcement of DCM's hard tenancy model at the Kubernetes level. + +### 12.3 Unsanctioned Change Detection + +Operators must monitor for changes to DCM-managed CRs that did not originate from a DCM request. Any such change is an UNSANCTIONED_CHANGE event and must be reported to DCM immediately. DCM's Policy Engine determines the appropriate response (REVERT, UPDATE_DEFINITION, ALERT, etc.). + +--- + +## 13. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the specification be submitted to CNCF as a sandbox project or proposed as a Kubernetes SIG? | Community adoption strategy | โ“ Unresolved | +| 2 | Should conformance certification be self-certified (test suite passes) or require DCM project review? | Community trust | โ“ Unresolved | +| 3 | How should the specification handle operators that manage cluster-scoped (non-namespaced) resources? | Namespace strategy | โ“ Unresolved | +| 4 | Should the SDK support non-Go operator frameworks (Java Operator SDK, Python kopf)? | Ecosystem breadth | โ“ Unresolved | +| 5 | How does the specification interact with Kubernetes Cluster API โ€” can CAPI clusters be DCM-managed resources? | Scope | โ“ Unresolved | +| 6 | Should there be a Level 0 โ€” a pure label-based passive mode requiring no operator changes? | Adoption friction | โ“ Unresolved | + +--- + +## Appendix A โ€” Example Implementation Checklist + +### Level 1 Checklist +- [ ] Operator registers with DCM on startup via `POST /api/v1/providers` +- [ ] Registration retried with exponential backoff on failure +- [ ] `GET /health` endpoint returns HTTP 200 when healthy +- [ ] `GET /health` returns non-200 when operator cannot fulfill requests +- [ ] Status reported to DCM when resource transitions to OPERATIONAL, FAILED, or DECOMMISSIONED +- [ ] All DCM-managed CRs labeled with mandatory DCM labels +- [ ] Create response returns PROVISIONING state immediately + +### Level 2 Checklist +- [ ] All Level 1 items complete +- [ ] Capacity reported to DCM on configurable schedule +- [ ] Capacity denial returns `INSUFFICIENT_RESOURCES` with proper payload +- [ ] Full realized state payload in DCM Unified Data Model format +- [ ] Field mapping declaration complete and validated +- [ ] All standard lifecycle event types implemented +- [ ] Unsanctioned change detection active +- [ ] CR condition changes translated to DCM lifecycle events + +### Level 3 Checklist +- [ ] All Level 2 items complete +- [ ] Sovereignty capabilities declared in registration +- [ ] Field-level provenance included in realized state payloads +- [ ] `POST /discover` endpoint implemented +- [ ] Decommission confirmation callback handled +- [ ] Override control metadata honored in CR creation + +--- + +## Appendix B โ€” Relationship to Other Specifications + +- **DCM Data Model** โ€” defines the Unified Data Model format used in all API payloads +- **DCM Service Provider Contract** โ€” the general provider contract this specification extends +- **DCM Resource Type Registry** โ€” where DCM Resource Types are registered; operators must reference registry UUIDs +- **AEP (API Enhancement Proposals)** โ€” the DCM API follows AEP standards for REST API design +- **OpenAPI 3.1.0** โ€” all API schemas are defined in OpenAPI 3.1.0 + +--- + +*This specification is maintained by the DCM Project. For questions, contributions, or conformance certification see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/specifications/operator-sdk-api.md b/content/docs/specifications/operator-sdk-api.md new file mode 100644 index 0000000..4c9ec29 --- /dev/null +++ b/content/docs/specifications/operator-sdk-api.md @@ -0,0 +1,638 @@ +--- +title: DCM Operator SDK API +type: docs +weight: 3 +--- + +**Version:** 0.1.0-draft +**Status:** Design โ€” Not yet implemented +**Document Type:** Technical Design +**Language:** Go +**Repository:** https://github.com/dcm-project/operator-sdk +**Related Documents:** [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Kubernetes Compatibility](11-kubernetes-compatibility.md) + +--- + +## 1. Purpose + +This document defines the public API of the DCM Operator SDK โ€” the Go library that enables Kubernetes operators to implement the DCM Operator Interface Specification with minimal code changes. The SDK handles all DCM protocol concerns so that operator developers only need to implement business logic โ€” field mappings and reconciliation hooks. + +**Design principle:** The SDK must be adoptable in a single day. If implementing Level 1 takes more than a day, the API is too complex. + +--- + +## 2. Package Structure + +``` +github.com/dcm-project/operator-sdk/ +โ”œโ”€โ”€ pkg/ +โ”‚ โ”œโ”€โ”€ client/ # DCM control plane client +โ”‚ โ”œโ”€โ”€ config/ # SDK configuration +โ”‚ โ”œโ”€โ”€ mapping/ # Field mapping engine +โ”‚ โ”œโ”€โ”€ reconciler/ # Reconciliation loop helpers +โ”‚ โ”œโ”€โ”€ registration/ # Provider registration +โ”‚ โ”œโ”€โ”€ server/ # HTTP server with DCM endpoints +โ”‚ โ”œโ”€โ”€ status/ # Status translation and reporting +โ”‚ โ”œโ”€โ”€ events/ # Lifecycle event types and emission +โ”‚ โ”œโ”€โ”€ discovery/ # Brownfield discovery helpers (Level 3) +โ”‚ โ””โ”€โ”€ provenance/ # Provenance metadata generation (Level 3) +โ”œโ”€โ”€ api/ +โ”‚ โ””โ”€โ”€ v1/ # DCM API type definitions +โ””โ”€โ”€ examples/ + โ”œโ”€โ”€ level1/ # Minimal Level 1 implementation example + โ”œโ”€โ”€ level2/ # Full Level 2 implementation example + โ””โ”€โ”€ level3/ # Complete Level 3 implementation example +``` + +--- + +## 3. Core Types + +### 3.1 Config + +```go +// Config is the primary SDK configuration structure. +// All fields have sensible defaults โ€” only DCMEndpoint, +// OperatorEndpoint, and ProviderName are required. +type Config struct { + // Required + ProviderName string + DCMEndpoint string + OperatorEndpoint string + + // Required โ€” at least one ServiceType must be declared + ServiceTypes []ServiceTypeConfig + + // Optional โ€” defaults to Level1 if not specified + ConformanceLevel ConformanceLevel + + // Optional โ€” defaults to "unknown" if not specified + DisplayName string + Version string + + // Optional โ€” field mappings loaded from file if not inline + FieldMappings []FieldMapping + FieldMappingFiles []string + + // Level 2+ โ€” capacity reporter + // If nil and ConformanceLevel >= Level2, SDK returns error on init + CapacityReporter CapacityReporter + + // Level 3 โ€” sovereignty and provenance + SovereigntyCapabilities *SovereigntyCapabilities + + // Optional โ€” HTTP server configuration + ServerConfig ServerConfig + + // Optional โ€” registration retry configuration + RegistrationConfig RegistrationConfig + + // Optional โ€” health check configuration + HealthConfig HealthConfig + + // Optional โ€” logger (defaults to zap logger) + Logger logr.Logger +} + +// ConformanceLevel declares the operator's DCM conformance level +type ConformanceLevel int + +const ( + Level1 ConformanceLevel = 1 + Level2 ConformanceLevel = 2 + Level3 ConformanceLevel = 3 +) + +// ServiceTypeConfig declares a DCM Resource Type this operator implements +type ServiceTypeConfig struct { + // DCM Resource Type name โ€” e.g., "Storage.Database" + ServiceTypeName string + // DCM Resource Type UUID from the registry + ServiceTypeUUID string + // Kubernetes CRD this service type maps to + CRDReference CRDReference + // Operations this operator supports for this type + OperationsSupported []Operation +} + +type CRDReference struct { + Group string + Version string + Kind string +} + +type Operation string + +const ( + OperationCreate Operation = "CREATE" + OperationRead Operation = "READ" + OperationUpdate Operation = "UPDATE" + OperationDelete Operation = "DELETE" + OperationDiscover Operation = "DISCOVER" // Level 3 only +) +``` + +### 3.2 Client โ€” DCM Control Plane Interface + +```go +// Client is the interface for communicating with the DCM control plane. +// The SDK creates and manages this internally โ€” operator developers +// use it only through the higher-level SDK methods. +type Client interface { + // Register sends the provider registration to DCM. + // Returns the DCM-assigned provider UUID on success. + Register(ctx context.Context, reg ProviderRegistration) (string, error) + + // ReportStatus sends a realized state payload to DCM. + ReportStatus(ctx context.Context, resourceID string, status RealizedState) error + + // ReportEvent sends a lifecycle event to DCM. + ReportEvent(ctx context.Context, resourceID string, event LifecycleEvent) error + + // ReportCapacity sends a capacity update to DCM. + // Required for Level 2+. + ReportCapacity(ctx context.Context, capacity CapacityReport) error + + // ConfirmDecommission acknowledges a decommission request from DCM. + // Required for Level 3. + ConfirmDecommission(ctx context.Context, resourceID string, confirmation DecommissionConfirmation) error +} +``` + +### 3.3 SDK โ€” Primary Interface + +```go +// SDK is the primary interface for the DCM Operator SDK. +// Operator developers interact with DCM through this interface. +type SDK interface { + // --- Lifecycle --- + + // Register sends the provider registration to DCM. + // Called during operator startup. Retries with exponential backoff. + // Does not block โ€” runs in background goroutine. + Register(ctx context.Context) + + // Shutdown gracefully deregisters the operator from DCM and + // stops background goroutines. + Shutdown(ctx context.Context) error + + // --- HTTP Server --- + + // StartServer starts the HTTP server with all DCM-required endpoints. + // Blocks until context is cancelled. + StartServer(ctx context.Context, addr string) error + + // Handler returns an http.Handler for use with an existing HTTP server. + // Alternative to StartServer when the operator already has an HTTP server. + Handler() http.Handler + + // --- Reconciliation Helpers --- + + // IsManagedResource returns true if the Kubernetes object + // carries DCM management labels. + IsManagedResource(obj client.Object) bool + + // IsUnsanctionedChange returns true if the object's spec has changed + // without a corresponding DCM request annotation. + // Used in reconciliation loops to detect drift. + IsUnsanctionedChange(obj client.Object) bool + + // DetectChangedFields returns the list of fields that changed + // relative to the last known DCM request state. + DetectChangedFields(obj client.Object) []FieldChange + + // InjectLabels adds DCM-required labels to a Kubernetes object + // before creation. Called before submitting a CR to Kubernetes. + InjectLabels(obj client.Object, req CreateRequest) client.Object + + // AnnotateRequest adds the DCM request ID annotation to a + // Kubernetes object. Used to mark changes as DCM-sanctioned. + AnnotateRequest(obj client.Object, requestID string) client.Object + + // --- Status Translation --- + + // TranslateStatus translates a Kubernetes object's status + // to a DCM RealizedState using the configured field mappings. + TranslateStatus(obj client.Object) (RealizedState, error) + + // ReportStatus translates and reports status to DCM in one call. + // Convenience wrapper for TranslateStatus + Client.ReportStatus. + ReportStatus(ctx context.Context, obj client.Object) error + + // --- Event Emission --- + + // ReportEvent sends a lifecycle event to DCM. + ReportEvent(ctx context.Context, obj client.Object, event LifecycleEventType, details EventDetails) error + + // ReportUnsanctionedChange is a convenience method for reporting + // an unsanctioned change event with the detected changed fields. + ReportUnsanctionedChange(ctx context.Context, obj client.Object, changes []FieldChange) error + + // ReportDegradation reports a DEGRADATION event to DCM. + ReportDegradation(ctx context.Context, obj client.Object, reason string) error + + // ReportHealthChange reports an ENTITY_HEALTH_CHANGE event. + ReportHealthChange(ctx context.Context, obj client.Object, healthy bool, reason string) error + + // --- Capacity --- + + // StartCapacityReporting starts the background capacity reporting + // goroutine. Required for Level 2+. Called automatically by StartServer. + StartCapacityReporting(ctx context.Context) + + // --- Discovery (Level 3) --- + + // BuildDiscoveryResponse queries Kubernetes for existing resources + // and returns them in DCM Realized State format. + // Used to implement the POST /discover endpoint. + BuildDiscoveryResponse(ctx context.Context, k8sClient client.Client, opts DiscoveryOptions) ([]RealizedState, error) +} +``` + +--- + +## 4. Field Mapping API + +```go +// FieldMapping declares how a DCM Resource Type maps to a Kubernetes CRD. +// Can be loaded from a YAML file or declared inline in Go. +type FieldMapping struct { + ServiceTypeName string + ServiceTypeUUID string + CRDReference CRDReference + + // DCM Requested State โ†’ Kubernetes CR spec (Naturalization) + DCMToCR []FieldMap + + // Kubernetes CR status โ†’ DCM Realized State (Denaturalization) + CRStatusToDCM []FieldMap + + // Kubernetes conditions โ†’ DCM lifecycle states + ConditionMappings []ConditionMapping + + // Kubernetes events โ†’ DCM lifecycle event types + LifecycleEventMappings []LifecycleEventMapping + + // Namespace strategy for this resource type + NamespaceStrategy NamespaceStrategy +} + +// FieldMap declares a single field translation +type FieldMap struct { + // Source field path โ€” dot-notation, supports array indexing + // e.g., "resources.cpu" or "nodes.controlPlane[0].cpu" + SourcePath string + + // Destination field path + DestPath string + + // Transform function name โ€” registered in the transform registry + // "none" for direct copy, or a named transform + Transform string + + // Required โ€” if true and source field is absent, returns error + Required bool + + // Default โ€” used when source field is absent and Required is false + Default interface{} +} + +// ConditionMapping maps a Kubernetes condition to a DCM lifecycle state +type ConditionMapping struct { + // Kubernetes condition expression โ€” e.g., "Ready=True" + // Supports AND: "Ready=False,Progressing=True" + KubernetesCondition string + + // DCM lifecycle state + DCMLifecycleState LifecycleState +} + +// LifecycleEventMapping maps a Kubernetes event to a DCM event type +type LifecycleEventMapping struct { + // "condition_change" | "spec_change_without_dcm_request" | "deletion" + KubernetesEvent string + + // Condition that triggers this mapping (for condition_change events) + Condition string + + // DCM event type + DCMEventType LifecycleEventType + + // Severity + Severity EventSeverity +} + +// Transform registry โ€” operator developers register custom transforms +type TransformRegistry interface { + // Register adds a named transform function + Register(name string, fn TransformFunc) error + + // Get retrieves a transform function by name + Get(name string) (TransformFunc, error) +} + +// TransformFunc transforms a value from source to destination format +type TransformFunc func(value interface{}) (interface{}, error) +``` + +--- + +## 5. Status and State Types + +```go +// LifecycleState represents the DCM lifecycle state of a resource +type LifecycleState string + +const ( + LifecycleStateProvisioning LifecycleState = "PROVISIONING" + LifecycleStateOperational LifecycleState = "OPERATIONAL" + LifecycleStateDegraded LifecycleState = "DEGRADED" + LifecycleStateSuspended LifecycleState = "SUSPENDED" + LifecycleStateFailed LifecycleState = "FAILED" + LifecycleStateDecommissioned LifecycleState = "DECOMMISSIONED" +) + +// RealizedState is the DCM Unified Data Model representation of +// a resource's realized state. This is what the operator sends +// to DCM after successful provisioning or status change. +type RealizedState struct { + // DCM resource ID (returned by DCM in the create request) + ResourceID string + + // DCM entity UUID + DCMEntityUUID string + + // Current lifecycle state + LifecycleState LifecycleState + + // Timestamp of this realization + RealizedTimestamp time.Time + + // All realized fields in DCM Unified Data Model format + Spec map[string]interface{} + + // Level 3 โ€” field-level provenance + FieldProvenance map[string]FieldProvenance + + // Kubernetes reference for correlation + KubernetesReference KubernetesReference + + // Relationships created during realization + Relationships []RelationshipRecord +} + +// KubernetesReference carries Kubernetes-specific identity for correlation +type KubernetesReference struct { + Namespace string + Name string + UID types.UID + ResourceVersion string + Generation int64 +} + +// FieldProvenance carries lineage for a single field (Level 3) +type FieldProvenance struct { + SourceType string // "provider" + SourceUUID string // operator provider UUID + Timestamp time.Time + Reason string +} +``` + +--- + +## 6. Event Types + +```go +// LifecycleEventType represents a DCM lifecycle event type +type LifecycleEventType string + +const ( + EventEntityHealthChange LifecycleEventType = "ENTITY_HEALTH_CHANGE" + EventDegradation LifecycleEventType = "DEGRADATION" + EventMaintenanceScheduled LifecycleEventType = "MAINTENANCE_SCHEDULED" + EventMaintenanceStarted LifecycleEventType = "MAINTENANCE_STARTED" + EventMaintenanceCompleted LifecycleEventType = "MAINTENANCE_COMPLETED" + EventUnsanctionedChange LifecycleEventType = "UNSANCTIONED_CHANGE" + EventCapacityChange LifecycleEventType = "CAPACITY_CHANGE" + EventDecommissionNotice LifecycleEventType = "DECOMMISSION_NOTICE" + EventProviderDegradation LifecycleEventType = "PROVIDER_DEGRADATION" +) + +// EventSeverity represents the severity of a lifecycle event +type EventSeverity string + +const ( + SeverityInfo EventSeverity = "INFO" + SeverityWarning EventSeverity = "WARNING" + SeverityCritical EventSeverity = "CRITICAL" +) + +// LifecycleEvent is the payload sent to DCM for a lifecycle event +type LifecycleEvent struct { + EventUUID string + EventType LifecycleEventType + ProviderID string + ResourceID string + DCMEntityUUID string + EventTimestamp time.Time + Severity EventSeverity + RequiresImmediateAction bool + Details EventDetails + KubernetesReference KubernetesReference +} + +// EventDetails carries event-specific detail data +type EventDetails struct { + // For UNSANCTIONED_CHANGE events + ChangedFields []FieldChange + + // For DEGRADATION events + DegradationReason string + AffectedComponents []string + + // For MAINTENANCE events + MaintenanceWindow *MaintenanceWindow + MaintenanceReason string + + // For CAPACITY_CHANGE events + PreviousCapacity *CapacityReport + CurrentCapacity *CapacityReport + + // Human-readable message for any event type + Message string +} + +// FieldChange describes a single field change in an unsanctioned change event +type FieldChange struct { + FieldPath string + PreviousValue interface{} + CurrentValue interface{} + ChangedBy string // Kubernetes user or service account + ChangedAt time.Time +} +``` + +--- + +## 7. Capacity Types + +```go +// CapacityReporter is the interface operator developers implement +// to report capacity data to DCM. The SDK calls this on schedule. +type CapacityReporter interface { + // GetCapacity returns the current capacity for all service types. + // Called by the SDK on the configured reporting schedule. + GetCapacity(ctx context.Context) (CapacityReport, error) +} + +// CapacityReport contains capacity data for all service types +type CapacityReport struct { + ProviderID string + ReportTimestamp time.Time + NextReportAt time.Time + CapacityByServiceType []ServiceTypeCapacity +} + +// ServiceTypeCapacity contains capacity for a single service type +type ServiceTypeCapacity struct { + ServiceTypeUUID string + AvailableUnits int + ReservedUnits int + CommittedUnits int + UnitDefinition string + KubernetesResources KubernetesResourceCapacity +} + +// KubernetesResourceCapacity contains raw Kubernetes resource availability +type KubernetesResourceCapacity struct { + AvailableCPUMillicores int64 + AvailableMemoryBytes int64 + AvailableStorageBytes int64 + NodeCount int +} +``` + +--- + +## 8. Constructor and Initialization + +```go +// New creates and initializes a new DCM SDK instance. +// Returns an error if the configuration is invalid or +// if required components for the declared conformance level +// are missing. +func New(config Config) (SDK, error) + +// NewWithClient creates a new SDK instance with a pre-configured +// DCM client. Used primarily for testing. +func NewWithClient(config Config, client Client) (SDK, error) + +// LoadFieldMappings loads field mapping declarations from YAML files. +// Accepts one or more file paths or glob patterns. +func LoadFieldMappings(paths ...string) ([]FieldMapping, error) + +// MustNew creates a new SDK instance and panics if initialization fails. +// Convenience function for use in main() where error handling via +// panic is acceptable. +func MustNew(config Config) SDK +``` + +--- + +## 9. Minimal Level 1 Example + +```go +package main + +import ( + "context" + "os" + + dcmsdk "github.com/dcm-project/operator-sdk" + ctrl "sigs.k8s.io/controller-runtime" +) + +func main() { + // Minimal Level 1 configuration + dcm, err := dcmsdk.New(dcmsdk.Config{ + ProviderName: "my-operator", + DisplayName: "My Operator DCM Provider", + DCMEndpoint: os.Getenv("DCM_ENDPOINT"), + OperatorEndpoint: os.Getenv("OPERATOR_ENDPOINT"), + ConformanceLevel: dcmsdk.Level1, + ServiceTypes: []dcmsdk.ServiceTypeConfig{ + { + ServiceTypeName: "Storage.Database", + ServiceTypeUUID: "dcm-registry-uuid-for-storage-database", + CRDReference: dcmsdk.CRDReference{ + Group: "postgresql.cnpg.io", + Version: "v1", + Kind: "Cluster", + }, + OperationsSupported: []dcmsdk.Operation{ + dcmsdk.OperationCreate, + dcmsdk.OperationRead, + dcmsdk.OperationDelete, + }, + }, + }, + FieldMappingFiles: []string{"dcm-mappings.yaml"}, + }) + if err != nil { + panic(err) + } + + ctx := ctrl.SetupSignalHandler() + + // Register with DCM in background โ€” does not block startup + dcm.Register(ctx) + + // Start HTTP server with health + DCM endpoints + go dcm.StartServer(ctx, ":8080") + + // Start operator manager (existing code unchanged) + mgr, _ := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{}) + mgr.Start(ctx) +} + +// In reconciliation loop โ€” minimal Level 1 additions +func (r *ClusterReconciler) Reconcile( + ctx context.Context, + req ctrl.Request, +) (ctrl.Result, error) { + + cluster := &cnpgv1.Cluster{} + if err := r.Get(ctx, req.NamespacedName, cluster); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + + // Only process DCM-managed resources + if !r.DCM.IsManagedResource(cluster) { + return ctrl.Result{}, nil + } + + // Existing reconciliation logic here... + + // Report status to DCM (SDK handles translation via field mappings) + r.DCM.ReportStatus(ctx, cluster) + + return ctrl.Result{}, nil +} +``` + +--- + +## 10. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the SDK support non-Go operator frameworks via a language-agnostic REST adapter? | Ecosystem breadth | โ“ Unresolved | +| 2 | How should the SDK handle DCM endpoint unavailability โ€” queue events locally or drop? | Reliability | โ“ Unresolved | +| 3 | Should field mappings support dynamic resolution โ€” a transform that queries external data? | Flexibility | โ“ Unresolved | +| 4 | Should the SDK provide a testing framework for unit testing operator-DCM integration? | Developer experience | โ“ Unresolved | +| 5 | Should the SDK expose metrics (Prometheus) for DCM registration status, event delivery success, etc.? | Observability | โ“ Unresolved | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md new file mode 100644 index 0000000..7e3a3dc --- /dev/null +++ b/content/docs/taxonomy.md @@ -0,0 +1,279 @@ +--- +title: "DCM Taxonomy" +type: docs +weight: 5 +--- + +> **Purpose:** This document defines the authoritative vocabulary for all DCM architecture, documentation, and implementation work. It ensures precision and clarity by giving every term a distinct, contextual definition and identifying ambiguous terms to avoid. +> +> When contributing to DCM โ€” code, documentation, Jira tickets, design discussions โ€” use these terms consistently. Vocabulary proposals are submitted via PR following standard registry governance. + +--- + +## Part 1 โ€” Core Vocabulary + +### Foundational Architecture Terms + +**Artifact** +Any versioned, GitOps-managed object in DCM โ€” layers, policies, resource type specifications, provider registrations, group definitions, entity declarations. Every artifact carries universal artifact metadata (uuid, handle, version, status, owned_by). Artifacts are immutable once published; changes produce a new version. + +**Assembly Process** +The nine-step process by which DCM transforms a consumer's Intent State into a provider-ready Requested State. Steps: Intent Capture โ†’ Layer Resolution โ†’ Layer Merge โ†’ Request Layer Application โ†’ Pre-Placement Policies โ†’ Placement Engine Loop โ†’ Post-Placement Policies โ†’ Requested State Storage โ†’ Provider Dispatch. The assembly process is the core operational function of the DCM Control Plane. + +**Control Plane** +The central nervous system of DCM. Maintains the Unified API and Data Model, enforces multi-tenancy, executes the assembly process, manages artifact lifecycle, and coordinates Service Providers. Does not directly manage physical infrastructure โ€” manages the data that represents and governs infrastructure. + +**DCM (Data Center Management)** +A framework designed to provide a "hyperscaler-like cloud experience" for on-premises infrastructure. Centralizes and automates the management, observation, and lifecycle of IT/IS services within an enterprise data center. Technology-agnostic โ€” defines roles, responsibilities, interactions, and expected capabilities rather than prescribing specific tools. + +**DCM Instance** +A running deployment of the DCM Control Plane and its associated stores. Manages a defined set of resources, providers, and tenants. Multiple DCM instances may be federated. See: Hub DCM, Regional DCM, Sovereign DCM. + +**Deployment Posture** +How a DCM instance's infrastructure behaves โ€” redundancy, enforcement strictness, audit retention, tenancy model, cross-tenant defaults. Expressed as a Deployment Posture Group (posture-minimal through posture-sovereign). One of the two dimensions of a Profile. + +**Domain (Policy and Layer)** +The organizational and architectural home of a Policy or Layer. Determines authority scope and override precedence. Ordered highest to lowest: system > platform > tenant > service > provider > request. Lower-domain artifacts cannot override higher-domain artifacts. + +**Drift** +The difference between what DCM believes exists (Realized State) and what actually exists (Discovered State). May be authorized (change made through DCM) or unsanctioned (change made outside DCM). The Drift Detection component continuously compares these states and triggers remediation per policy. + +**Entity** +A specific, realized instance of a resource type โ€” a particular VM, a specific VLAN, a named DNS record. Entities have UUIDs preserved across their entire lifecycle including provider migrations. Entities pass through Intent, Requested, Realized, and Discovered states. + +**Four States** +The four representations of a resource in DCM: **Intent State** (consumer's original declaration, stored in Git), **Requested State** (fully assembled, policy-processed payload, stored in Git), **Realized State** (what the provider actually provisioned, stored in Event Stream), **Discovered State** (what active interrogation finds currently exists, stored in Discovered Store). + +**Handle** +The human-readable, stable identifier for an artifact within DCM. Complements the UUID (machine-meaningful). Format: `{domain}/{concern-or-type}/{name}`. Stable across versions โ€” the handle does not change when a new version is published. + +**Intent State** +The consumer's original resource declaration as submitted, stored verbatim in the Intent Store (GitOps) before any assembly processing. Used as the source for rehydration. + +**Layer** +A declarative, immutable, versioned unit of data that contributes field values to an assembled request payload. Layers are passive โ€” they declare values but do not execute logic. Layers answer "what values should these fields have?" See also: Policy. + +**Layer Chain** +The ordered set of layers applied during the assembly process for a specific request. Immutable once assembled. The layer chain is the deduplication key in the deduplicated provenance model. + +**Lifecycle Constraint Enforcer** +The DCM control plane component that enforces time-based constraints โ€” TTLs, valid_from/valid_until on memberships, max_execution_time on Process Resources. + +**Placement Engine** +The DCM control plane component that selects the Service Provider for a request based on constraints, capacity, sovereignty, and policy. Operates within a reserve-query loop during Step 6 of the assembly process. Uses a deterministic seven-step tie-breaking hierarchy. + +**Policy** +An executable rule that evaluates the assembled payload and takes action. Policies answer "given this data, is it valid? what should change? should this proceed?" Three types: GateKeeper (approve or reject), Validation (verify correctness), Transformation (modify or enrich). *Policies are logic; Layers are data โ€” they serve different purposes and must not be confused.* + +**Profile** +A named composition of a Deployment Posture Group and one or more Compliance Domain Groups. Defines the full governance posture for a DCM deployment. DCM ships six core profiles (minimal through sovereign) and eight extended profiles (hipaa-prod, fedramp-moderate, dod-il4, etc.). + +**Provenance** +The full audit trail of where a field value came from and what changed it. Recorded per-field in the assembled payload. Three configurable models: full_inline (all provenance on entity record), deduplicated (content-addressed, layer chain as dedup key), tiered archive (hot/warm/cold tiers). + +**Rehydration** +Replaying an entity's Intent State through the assembly process โ€” potentially to a different provider, in a different context. The entity's UUID is always preserved. Provider-side identifiers change and are recorded in rehydration_history. + +**Requested State** +The fully assembled, policy-processed payload stored in the Requested Store (GitOps). Authoritative record of what was actually requested and how assembly enriched it. Includes: assembled payload, assembly provenance, placement decisions, and dependency resolution. + +**Realized State** +The actual provisioned state of a resource as reported by the Service Provider. Stored in the Realized Store (Event Stream). Includes provider-side identifiers and actual provisioned field values. + +**Resource Type Specification** +The formal definition of a resource type โ€” its fields, constraints, dependencies, lifecycle rules, editable fields, and allowed service providers. Versioned artifacts stored in the Resource Type Registry. The contract between consumers and the assembly process. + +**Sovereignty Zone** +A declared geographic, legal, or organizational boundary within which data must remain. Enforced at the placement engine level. Cannot be overridden by consumer requests. Mandatory check before DCM-to-DCM tunnel establishment. + +**Tenant** +The primary ownership and authorization boundary for resources in DCM. All resources are owned by exactly one Tenant. Provides multi-tenancy isolation. See also: tenant_boundary (group class). + +**Unified Data Model** +The standardized data format used throughout DCM for all resource declarations, provider payloads, and state records. Service Providers implement Naturalization (DCM โ†’ native) and Denaturalization (native โ†’ DCM) for translation. + +--- + +### Provider Types + +**Service Provider** +A DCM component that realizes resources โ€” provisions, configures, and manages physical or virtual infrastructure on behalf of DCM consumers. Responsibilities: naturalization, realization, denaturalization, capacity reporting (reserve_query), and sovereignty declaration maintenance. *This is the DCM taxonomy term for what general software architecture calls a "producer."* + +**Information Provider** +A DCM component that supplies authoritative external data to enrich entity records. Examples: CMDB, IPAM, HR system, asset management. DCM computes confidence scores for all values they supply. *Not to be confused with Service Providers โ€” Information Providers supply data; they do not provision infrastructure.* + +**Meta Provider** +A Service Provider that composes multiple sub-providers to deliver a higher-order service. Declares composition_visibility: opaque, transparent, or selective. + +**Policy Provider** +A DCM component that supplies external policy logic. Four modes: Mode 1 (read-only query), Mode 2 (stateless evaluation), Mode 3 (execute external code), Mode 4 (black-box query with sovereignty checks). Trust elevation requires formal approval workflow. + +**Storage Provider** +A DCM component that persists DCM state. Sub-types: GitOps Store, Event Stream Store, Search Index, Audit Store, Validation Store, Discovered Store. + +**Message Bus Provider** +A DCM component bridging internal and external event streams. Used for curated observability events, webhook delivery, and federation message passing. + +**Credential Provider** +A DCM component that resolves secrets from external stores. Credentials never stored in Git or audit records. + +**Auth Provider** +A DCM component that authenticates identities and resolves permissions. Supports OIDC, LDAP/AD, FreeIPA, GitHub/GitLab OAuth, mTLS, static API key, local users. + +**DCM Provider** +A DCM component that wraps another DCM instance's API for cross-DCM resource sharing. Always mTLS. Sovereignty checks mandatory. Local DCM policies govern all resources from DCM Provider tunnels. + +--- + +### Federation Topology + +**Hub DCM** +The central/global DCM instance. Authoritative registry origin, governance authority, and federation routing hub. Applies placement engine logic to route requests to Regional or Sovereign DCMs. *Replaces "Shore" (defense IT terminology).* + +**Regional DCM** +A distributed regional DCM instance managing resources within its region. Caches layers and catalog items from Hub DCM. Treated as a DCM Provider instance by the Hub DCM's placement engine. *Replaces "Ship" (defense IT terminology).* + +**Sovereign DCM** +An air-gapped or compliance-isolated DCM instance. No live external connectivity. Updates via signed bundles during connectivity windows. Required for classified, sovereign, or highly regulated deployments. *Replaces "Enclave" (defense IT terminology).* + +**Federation Depth** +Number of hops from deepest DCM instance to Hub DCM. Profile-governed maximum: 5 (minimal/dev), 3 (standard/prod), 2 (fsi/sovereign). + +**Federation Trust Score** +0-100 score on a DCM-to-DCM tunnel. Computed from: identity verification, sovereignty compatibility, certifications currency, audit integrity, uptime, compliance. Used in: cross_dcm_confidence = source_confidence ร— (tunnel_trust_score / 100). + +--- + +### Data Model Terms + +**Compliance Domain Group** +A Policy Group governing which regulatory frameworks apply to DCM-managed resources. One of the two dimensions of a Profile. 16 built-in groups: FSI, PCI-DSS, HIPAA, FedRAMP Moderate, FedRAMP High, DoD IL2-IL6, Government, GDPR, ISO 27001, NIST 800-53, SOC2, NERC-CIP, Sovereign. + +**Confidence Descriptor** +The primary data model for Information Provider field value confidence. Four stored fields: authority_level, corroboration, source_trust, last_updated_at. Confidence score (0-100) and band (very_high through very_low) are derived at query time โ€” never stored as primary data. + +**Core Layers** +Data layers applicable across any resource type โ€” organizational, infrastructure, and contextual data not specific to any one service. + +**Denaturalization** +Converting provider-native result data back into the DCM Unified Data Model. Inverse of Naturalization. + +**Naturalization** +Converting a DCM Unified Data Model payload into provider-native format. Inverse of Denaturalization. + +**native_passthrough** +A sanctioned field for provider-specific data genuinely untranslatable to the Unified Data Model. Always audit-logged. Opaque mode blocked in fsi/sovereign profiles. + +**Policy Group** +A cohesive collection of related policies addressing a single concern. Managed as DCMGroup with group_class: policy_collection. + +**Service Layer** +A data layer contributed by a Service Provider containing service-specific configuration defaults. Independently versioned from its Service Provider. + +--- + +### Operational Terms + +**Brownfield** +Existing infrastructure provisioned outside of DCM, brought under management via the Ingestion Model. + +**DCM Group (DCMGroup)** +The universal grouping construct. All grouping uses DCMGroup with a declared group_class. Eight classes: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, provider_grouping, composite, federation. + +**Discovered State** +Result of active infrastructure interrogation. Ephemeral operational data in the Discovered Store. Used by Drift Detection to compare against Realized State. NOT the source of truth. Never stored in the Audit Store. + +**Editable Field** +A field on a realized entity modifiable via targeted delta update without reprovisioning. Independent of override_preference (which governs assembly time). + +**Fulfillment** +The complete process from consumer submission through Service Provider realization. A resource is Fulfilled when its Realized State matches its Requested State. + +**Implementation Posture** +A Policy Group concern_type for implementation complexity vs capability trade-offs. Governs provenance model selection, auth simplicity, deployment complexity. + +**Ingestion Model** +The three-step process (INGEST โ†’ ENRICH โ†’ PROMOTE) for bringing brownfield resources under DCM management. + +**Rehydration Lease** +An exclusive time-bounded lock per entity during rehydration. Prevents concurrent rehydrations. Priority ordering: security/compliance emergency > manual admin > automated sovereignty migration > provider decommission > manual consumer. + +**Reserve Query** +Placement engine mechanism asking candidate Service Providers "can you fulfill this resource request right now?" Providers respond with capacity availability and sovereignty compatibility. + +**Shadow Mode** +The operational state of a proposed Policy or Policy Provider. Evaluates real requests; captures outputs in Validation Store; does not apply outputs to requests. + +**Step-Up MFA** +Additional MFA challenge at sensitive operations within an already-authenticated session. Protects against session hijacking for high-stakes operations. + +**Targeted Delta** +Update mechanism for editable fields. Applies only changed fields to Realized State. Does not re-run the layer assembly chain. + +**Validation Store** +Storage Provider sub-type for shadow evaluation records. Separate from Audit Store โ€” queryable, modifiable, P90D default retention. + +--- + +## Part 2 โ€” Anti-Vocabulary + +Terms to avoid and what to use instead. + +| Avoid | Reason | Use Instead | +|-------|--------|-------------| +| **Data Center** | A building โ€” architecturally irrelevant | **Region**, **Zone**, **Availability Zone** | +| **Realize / Realization** | Means too many things | **Provision** (a VM), **Install** (software), **Fulfill** (a request) | +| **Tangible / Intangible** | Weasel words โ€” no architectural meaning | Specific terms: physical resource, virtual resource, logical construct | +| **Widgets** | Vague โ€” what things exactly? | The specific resource type: VirtualMachine, VLAN, DNSRecord | +| **Producer** | Generic term not in DCM vocabulary | **Service Provider** โ€” carries the full DCM contract model | +| **Shore / Ship / Enclave** | Defense IT terminology โ€” not universally understood | **Hub DCM**, **Regional DCM**, **Sovereign DCM** | +| **User** (generic) | Means different things at different layers | **Developer** / **Application Owner** (Application domain); **Platform Engineer** / **SRE** (platform) | +| **Service** (unqualified) | Overloaded โ€” means different things at each layer | **Catalog Item** (Application), **Resource Type** (Control Plane), **Service Provider** (provider) | +| **Config / Configuration** | Could mean Layer, Resource Type Spec, Policy, or settings | Specify: **Layer**, **Resource Type Specification**, **Policy**, **Platform configuration** | +| **Manage** | Means everything and nothing | **Provision**, **configure**, **monitor**, **decommission**, **migrate**, **govern** | +| **Enrich** (unqualified) | Could mean layer assembly, policy transformation, or information push | **Layer assembly**, **Policy transformation**, **Information Provider enrichment** | + +--- + +## Part 3 โ€” Roles and Personas + +| Role | Domain | Meaning | +|------|--------|---------| +| **Developer / Application Owner** | Application | Human consumer of cloud services via DCM | +| **Platform Engineer** | Control Plane | Operates and maintains the DCM platform | +| **Infrastructure Operations** | Data Center / Resource | Manages physical infrastructure and service providers | +| **Policy Owner** | Governance | Authors, reviews, and activates DCM policies | +| **Risk and Compliance Manager** | Governance | Reviews audit records, compliance reports, and policy effectiveness | +| **Data Protection Officer** | Governance | Responsible for GDPR, HIPAA, and data protection compliance | +| **Platform Admin** | Platform | Highest-privilege DCM operator | +| **SRE** | Platform | Manages DCM operational health and incident response | +| **Tenant Admin** | Tenant | Manages resources and users within a Tenant boundary | +| **Service Provider Team** | Provider | Builds and maintains Service Provider integrations | + +--- + +## Part 4 โ€” Capability Domain Prefixes + +Capability IDs used in the DCM Capabilities Matrix for Jira and implementation tracking. + +| Prefix | Domain | +|--------|--------| +| IAM | Identity and Access Management | +| CAT | Service Catalog | +| REQ | Request Lifecycle Management | +| PRV | Provider Contract and Realization | +| LCM | Resource Lifecycle Management | +| DRF | Drift Detection and Remediation | +| POL | Policy Management | +| LAY | Data Layer Management | +| INF | Information and Data Integration | +| ING | Ingestion and Brownfield Management | +| AUD | Audit and Compliance | +| OBS | Observability and Operations | +| STO | Storage and State Management | +| FED | DCM Federation and Multi-Instance | +| GOV | Platform Governance and Administration | + +--- + +*Document maintained by the DCM Project. Vocabulary proposals submitted via PR. For questions or contributions see [GitHub](https://github.com/dcm-project).* From b4605ce966ee6cfeaabbf1245ac40f0233df1a85 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Fri, 27 Mar 2026 13:15:03 -0500 Subject: [PATCH 21/49] Cleaned up storage for request pipeline stores. Added notification provider. Added capabilities static content. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 300 ++++++- .../data-model/context-and-purpose.md | 12 +- .../data-model/entity-relationships.md | 67 +- .../architecture/data-model/entity-types.md | 341 ++++++++ .../docs/architecture/data-model/examples.md | 480 ++++++++++ .../architecture/data-model/four-states.md | 107 ++- .../data-model/notification-model.md | 593 +++++++++++++ .../ownership-sharing-allocation.md | 302 +++++++ .../data-model/resource-service-entities.md | 155 +++- .../data-model/storage-providers.md | 104 ++- .../data-model/webhooks-messaging.md | 17 +- .../architecture/specifications/_index.md | 2 + .../specifications/consumer-api-spec.md | 778 +++++++++++++++++ .../specifications/operator-interface-spec.md | 129 +++ .../capabilities/DCM-Capabilities-Matrix.csv | 96 ++ static/capabilities/map.html | 824 ++++++++++++++++++ 16 files changed, 4269 insertions(+), 38 deletions(-) create mode 100644 content/docs/architecture/data-model/entity-types.md create mode 100644 content/docs/architecture/data-model/examples.md create mode 100644 content/docs/architecture/data-model/notification-model.md create mode 100644 content/docs/architecture/data-model/ownership-sharing-allocation.md create mode 100644 content/docs/architecture/specifications/consumer-api-spec.md create mode 100644 static/capabilities/DCM-Capabilities-Matrix.csv create mode 100644 static/capabilities/map.html diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 8738823..d5e2495 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -417,6 +417,19 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | Model | Description | Example | |-------|-------------|---------| | **Allocation** | Provider retains internal ownership. Consumer owns the Entity (the allocation). Provider has reclaim rights on decommission. | VM, Container, IP Address | +| **Notification Provider** | Ninth DCM provider type; translates unified notification envelope to delivery channel; handles delivery, retry, dead letter, and delivery confirmation callbacks | +| **Notification Router** | DCM control plane component that resolves notification audiences and routes envelopes to Notification Providers | +| **audience resolution** | Deriving notification recipients by traversing the entity relationship graph from the changed entity at event time | +| **notification_uuid** | Idempotency key on notification envelopes; Notification Providers use this to deduplicate on retry | +| **audience_role** | owner / stakeholder / approver / observer โ€” why this actor is in the notification audience | +| **stakeholder_reason** | Notification envelope field explaining which relationship caused the actor to be in the stakeholder audience | +| **Tier 1 / Tier 2 / Tier 3 notifications** | Mandatory system (non-suppressable) / Tenant defaults / Actor subscriptions โ€” three subscription tiers that compose | +| **NOT-001 through NOT-008** | Notification model system policies | +| **write-once snapshot store** | Realized Store implementation model: each record is a complete immutable entity state snapshot; no event replay; direct point-in-time lookup; supersession chain links snapshots | +| **corresponding_requested_state_uuid** | Mandatory non-nullable field on every Realized State snapshot; traces every Realized State change to an authorized request | +| **Provider Update Notification** | Formal API for providers to report authorized state changes; DCM evaluates via Policy Engine; approved โ†’ new Requested State + Realized State; rejected โ†’ drift event | +| **notification_uuid** | Idempotency key on Provider Update Notifications; safe to resend on provider crash | +| **pre-authorized update** | Category of provider update pre-approved by GateKeeper policy; processed automatically without per-change human review | | **Whole Allocation** | Entire resource allocated as indivisible unit. Provider retains ownership. Consumer has exclusive use. Not subdivided or shared. | Dedicated Bare Metal (provider-owned) | | **Full Transfer** | Provider transfers complete ownership to consumer's DCM Tenant. Consumer controls full lifecycle including decommission. | Transferred Bare Metal, Licensed asset | | **Hybrid Transfer** | Ownership can transfer multiple times. Current owner is always exactly one DCM Tenant. Every transfer is tracked and auditable. | Bare Metal reallocated between tenants | @@ -2443,7 +2456,31 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | Former Term | Replacement | Meaning | |-------------|-------------|---------| -| Shore | **federation routing** | Hub DCM applies placement engine logic at the DCM instance level; Regional DCMs are DCM Provider instances; sovereignty is a hard pre-filter; same tie-breaking hierarchy as provider selection | +| Shore | **Notification Provider** | Ninth DCM provider type; translates unified notification envelope to delivery channel; handles delivery, retry, dead letter, and delivery confirmation callbacks | +| **Notification Router** | DCM control plane component that resolves notification audiences and routes envelopes to Notification Providers | +| **audience resolution** | Deriving notification recipients by traversing the entity relationship graph from the changed entity at event time | +| **notification_uuid** | Idempotency key on notification envelopes; Notification Providers use this to deduplicate on retry | +| **audience_role** | owner / stakeholder / approver / observer โ€” why this actor is in the notification audience | +| **stakeholder_reason** | Notification envelope field explaining which relationship caused the actor to be in the stakeholder audience | +| **Tier 1 / Tier 2 / Tier 3 notifications** | Mandatory system (non-suppressable) / Tenant defaults / Actor subscriptions โ€” three subscription tiers that compose | +| **NOT-001 through NOT-008** | Notification model system policies | +| **write-once snapshot store** | Realized Store implementation model: each record is a complete immutable entity state snapshot; no event replay; direct point-in-time lookup; supersession chain links snapshots | +| **corresponding_requested_state_uuid** | Mandatory non-nullable field on every Realized State snapshot; traces every Realized State change to an authorized request | +| **Provider Update Notification** | Formal API for providers to report authorized state changes; DCM evaluates via Policy Engine; approved โ†’ new Requested State + Realized State; rejected โ†’ drift event | +| **notification_uuid** | Idempotency key on Provider Update Notifications; safe to resend on provider crash | +| **pre-authorized update** | Category of provider update pre-approved by GateKeeper policy; processed automatically without per-change human review | +| **Whole Allocation** | Ownership pattern: consumer owns the entire resource entity outright in their Tenant; no pool involved | +| **Allocation** | Ownership pattern: pool yields independently-owned sub-resources; consumer owns their allocation; AllocationRecord relationship links to pool | +| **Shareable** | Ownership pattern: one resource, multiple stakeholders; consumers hold stakes (relationships) only; no consumer owns any portion | +| **AllocationRecord** | Cross-tenant relationship from an allocation entity back to its source pool entity | +| **stake_strength** | Relationship property on shareable resource attachments: required (blocks decommission) / preferred / optional | +| **PENDING_REVIEW** | Formal Infrastructure Resource Entity lifecycle state for conflicts requiring human resolution (sovereignty, cross-tenant auth revocation, ownership transfer conflicts) | +| **Consumer API** | DCM REST API for consumers: catalog browsing, request submission, resource management, audit trail access | +| **Consumer Request Status** | Lifecycle: ACKNOWLEDGED โ†’ ASSEMBLING โ†’ AWAITING_APPROVAL โ†’ APPROVED โ†’ DISPATCHED โ†’ PROVISIONING โ†’ COMPLETED/FAILED/CANCELLED | +| **01-entity-types.md** | Entity type taxonomy: Infrastructure Resource, Composite Resource, Process Resource; sub-types and invariants | +| **04-examples.md** | Worked examples: VM end-to-end, IP allocation, VLAN sharing, brownfield ingestion, drift remediation; Git repo structure | +| **04b-ownership-sharing-allocation.md** | Authoritative ownership model: whole allocation, allocation, shareable; policies OWN-001 through OWN-008 | +| **federation routing** | Hub DCM applies placement engine logic at the DCM instance level; Regional DCMs are DCM Provider instances; sovereignty is a hard pre-filter; same tie-breaking hierarchy as provider selection | | **independent_with_overlap** | Certificate rotation model: old cert valid P30D after new cert issued; allows peers to update trust stores without coordinated downtime | | **alert_and_hold** | Federated drift detection response when peer DCM is unavailable: do not assume drift; hold state; escalate to platform admin after PT24H | | **AUDIT_STORE_UNAVAILABLE** | Gap record inserted in Audit Store hash chain after recovery from Audit Store failure; timestamps the exact outage window; makes gap explicit and auditable | @@ -2775,7 +2812,234 @@ IAM, CAT, REQ, PRV, LCM, DRF, POL, LAY, INF, ING, AUD, OBS, STO, FED, GOV โ€” se --- -## SECTION 46 โ€” PERSONAS +## SECTION 46 โ€” GROUP 1: MISSING DOCUMENTS + +### 46.1 01-entity-types.md โ€” Entity Types Taxonomy + +Three primary entity types in DCM: + +**Infrastructure Resource Entity** โ€” persistent, full lifecycle (REQUESTED โ†’ PENDING โ†’ PROVISIONING โ†’ REALIZED โ†’ OPERATIONAL โ†’ SUSPENDED โ†’ DECOMMISSIONED). Owned by exactly one Tenant. Drift detection active. TTL management. `PENDING_REVIEW` is a valid state for sovereignty/tenancy conflicts during rehydration โ€” not an error state; requires human resolution. + +**Composite Resource Entity** โ€” Meta Provider composition of multiple Infrastructure Resource Entities. Owns its UUID; constituents own theirs. `lifecycle_state` reflects aggregate health โ€” OPERATIONAL only when all required constituents OPERATIONAL. Two-level drift detection (composite + constituent). Staged decommission (composite first, then constituents in reverse dependency order). `composition_visibility: opaque|transparent|selective`. + +**Process Resource Entity** โ€” ephemeral execution (automation jobs, playbooks, pipelines). Short lifecycle (REQUESTED โ†’ INITIATED โ†’ EXECUTING โ†’ terminal). No SUSPENDED state. No PENDING_REVIEW. `max_execution_time` mandatory โ€” no default. Must record `affected_entity_uuids` if any infrastructure modifications made. + +Entity sub-types: Shared Resource Entity (`ownership_model: shareable`), Allocatable Pool Resource (pool entity), Allocation entity (`ownership_model: allocation`). + +Entity identity invariants: UUID never changes (including rehydration); single Tenant ownership always; provider entity ID is separate from DCM UUID; audit records preserved per retention policy. + +### 46.2 04b-ownership-sharing-allocation.md โ€” Ownership, Sharing, and Allocation + +**The three ownership patterns โ€” use these terms precisely:** + +**Whole Allocation** โ€” consumer receives entire resource entity; owns it outright in their Tenant; no pool involved; full lifecycle control; decommission is straightforward. + +**Allocation** โ€” pool resource (owned by platform Tenant) yields new independently-owned sub-resources. Consumer owns their allocation outright. AllocationRecord relationship links allocation โ†’ pool. Decommissioning the allocation releases it back to the pool. Pool entity unaffected. Example: IPAddressPool โ†’ IPAddress entities. + +**Shareable** โ€” single resource owned by one Tenant; multiple consumers hold stakes (relationships) but own nothing. No new entity created per consumer. Decommission deferred while required stakes active. Consumer holds an `attached_to` or `depends_on` relationship with declared `stake_strength: required|preferred|optional`. Example: VLAN shared by multiple VMs. + +**Critical distinction:** Shareable = one resource, multiple stake-holders. Allocation = one pool, multiple independently-owned sub-resources. Never confuse these. + +Hybrid case: an allocation from a shareable pool. Consumer owns their /28 subnet (allocation). The parent /16 is shareable (NetworkOps owns it, multiple /28s have stakes in it). + +OWN-001 through OWN-008 policies govern these patterns. + +### 46.3 04-examples.md โ€” Worked Examples and Git Repository Structure + +**Git repository structure (resolves Q54 deferred item):** +- Intent Store: `intent-store/{tenant-uuid}/{resource-type-category}/{resource-type}/{entity-uuid}/intent.yaml` +- Requested Store: `requested-store/{tenant-uuid}/{resource-type-category}/{resource-type}/{entity-uuid}/` with: `requested.yaml`, `assembly-provenance.yaml`, `placement.yaml`, `dependencies.yaml` +- Provider selection is in `placement.yaml` โ€” not encoded in directory structure (Q54 resolved) + +**Five worked examples:** +1. VM provision end-to-end (layer assembly, policy evaluation, placement, all four states) +2. IP Address allocation (pool โ†’ consumer-owned allocation entity, AllocationRecord relationship) +3. VLAN attachment (shareable ownership โ€” stake relationship, decommission deferral) +4. Brownfield ingestion (INGEST โ†’ ENRICH โ†’ PROMOTE with CMDB Information Provider) +5. Drift detection and remediation (unsanctioned memory change, severity, ESCALATE, UPDATE_DEFINITION resolution) + +### 46.4 consumer-api-spec.md โ€” Consumer API Specification + +Consumer API base URL: `/api/v1/`. Three ingress surfaces: REST API (this spec), Web UI, Git PR. + +**Authentication:** Bearer token from `/api/v1/auth/token`. Tenant context via `X-DCM-Tenant` header โ€” always required when actor has multiple Tenants. Step-up MFA via `X-DCM-StepUp-Token` for sensitive operations. + +**Service Catalog:** `GET /api/v1/catalog` (list, filtered by RBAC), `GET /api/v1/catalog/{uuid}` (describe with full schema, constraints, cost estimate), `GET /api/v1/catalog/search`. + +**Request submission:** `POST /api/v1/requests` โ†’ 202 with entity_uuid and status_url. Consumer request status lifecycle: ACKNOWLEDGED โ†’ ASSEMBLING โ†’ AWAITING_APPROVAL โ†’ APPROVED โ†’ DISPATCHED โ†’ PROVISIONING โ†’ COMPLETED|FAILED|CANCELLED. `DELETE /api/v1/requests/{uuid}` for cancellation (only before PROVISIONING). + +**Resource management:** `GET /api/v1/resources` (list), `GET /api/v1/resources/{uuid}` (describe with confidence scores, drift status, editable flags), `PATCH /api/v1/resources/{uuid}` (targeted delta for editable fields), `POST /suspend`, `DELETE` (decommission with deferred response if stakes active), `POST /rehydrate`. + +**Audit:** `GET /api/v1/resources/{uuid}/audit` with chain_integrity field, `GET /api/v1/audit/correlation/{id}` for cross-state timeline. + +Three conformance levels: Level 1 (read-only), Level 2 (standard), Level 3 (full including rehydration and audit). + +### 46.5 Context-and-Purpose Fix + +Section 5 subsections were incorrectly numbered 3.1/3.2/3.3 โ€” corrected to 5.1/5.2/5.3. Q6 garbled row in open questions table โ€” corrected. + +### 46.6 Q54 Resolution + +Git repository structure is independent of provider selection. Provider selection is stored in `placement.yaml` within the entity's directory. The deferred note in the four states doc has been updated to reference the worked examples document for the complete layout. + +--- + +## SECTION 47 โ€” STORE ARCHITECTURE: INTENT, REQUESTED, REALIZED, DISCOVERED + +### 47.1 The Four Stores โ€” Corrected Model + +| Store | Type | Implementation | Why | +|-------|------|---------------|-----| +| Intent | GitOps (required) | GitHub/GitLab/Gitea | PR workflow is first-class feature, not implementation detail | +| Requested | Write-once Storage Provider | GitOps (reference); PostgreSQL (production scale) | Machine-generated; no PR benefit; Git degrades at scale | +| Realized | Write-once Snapshot Store | PostgreSQL; CockroachDB | Snapshot-based (not event stream); request-traceable only | +| Discovered | Ephemeral Snapshot Stream | Kafka; EventStoreDB | High-frequency; never a rehydration source; ephemeral | + +**Intent Store must be GitOps** โ€” the PR workflow, branch-per-request, and human review are architectural features. + +**Requested Store should NOT be GitOps at production scale** โ€” Git throughput degrades under high-frequency machine writes; PR mechanics add latency with no benefit for machine-generated content. Write-once document store with hash-chain integrity satisfies the contract. + +**Realized Store is NOT an event stream** โ€” it is a write-once snapshot store. Each record is a complete entity state, not a field-level event. This makes rehydration a direct lookup (not a replay) and makes point-in-time queries trivial. + +**Discovered Store remains an event stream** โ€” high-frequency, machine-generated, ephemeral; never a rehydration source. + +### 47.2 The Fundamental Realized Store Constraint + +> **Realized State only changes when an authorized request produces a corresponding Requested State record. No exceptions.** + +Three write sources โ€” all require `corresponding_requested_state_uuid` (non-nullable): +1. `initial_realization` โ€” provider confirms first provisioning +2. `consumer_update` โ€” consumer targeted delta approved and confirmed +3. `provider_update` โ€” DCM approves a Provider Update Notification + +**What does NOT write to Realized Store:** +- Drift detection (reads only) +- Discovery cycles (writes to Discovered Store only) +- Unsanctioned provider changes (become drift events) +- Direct admin writes (bypassing the request pipeline is forbidden) + +**Drift is always unsanctioned** โ€” there are no "legitimate drift events." Every authorized change goes through a request and produces a Requested State record. If Discovered State differs from Realized State without a corresponding Requested State record, it is drift. + +### 47.3 Provider Update Notification + +Formal mechanism for providers to report authorized state changes (auto-scaling, auto-healing, maintenance). Not drift โ€” the provider is asserting the change was authorized. + +**DCM processing pipeline:** +``` +Provider submits POST /api/v1/provider/entities/{uuid}/update-notification + โ†’ Authentication (provider mTLS) + โ†’ Policy Engine evaluates (pre-authorized? requires consumer approval?) + โ†’ APPROVED: create provider_update Requested State โ†’ write Realized State snapshot + โ†’ REQUIRES_APPROVAL: entity enters PENDING_REVIEW; consumer notified + โ†’ REJECTED: Realized State unchanged; discrepancy becomes drift +``` + +**Pre-authorization:** Providers declare update capabilities at registration. Organizations pre-authorize categories of updates via GateKeeper policy (e.g., auto-scale within 2ร— bounds). Pre-authorized updates are processed automatically. + +**Consumer approval API:** `GET /api/v1/resources/{uuid}/provider-notifications` and `POST /approve` or `/reject`. On approval โ†’ new Requested State + Realized State. On rejection โ†’ drift event. + +**Idempotency:** `notification_uuid` is the idempotency key. Safe to resend on provider crash. + +**Level 2 conformance requirement** in the Operator Interface Specification for providers implementing auto-scaling, auto-healing, or provider-side maintenance. + +### 47.4 Realized State Snapshot Structure + +```yaml +realized_state_snapshot: + realized_state_uuid: + entity_uuid: + realized_at: + source_type: initial_realization | consumer_update | provider_update + corresponding_requested_state_uuid: # mandatory, not nullable + supersedes_realized_state_uuid: + superseded_by_realized_state_uuid: + fields: { # complete entity state } +``` + +### 47.5 Rehydration from Realized State + +Rehydration picks a specific snapshot โ€” direct lookup by `realized_state_uuid` or by timestamp. Not a replay. Not a projection. A complete entity state that was explicitly authorized through DCM's governance pipeline. The supersession chain enables historical rehydration ("rehydrate as of March 15"). + +### 47.6 New Policies + +- `STO-007`: Realized Store is write-once snapshot; every write requires non-nullable `corresponding_requested_state_uuid`; enforcement at store API level +- `STO-008`: Intent Store requires GitOps; Requested Store requires write-once semantics (GitOps reference impl; write-once document stores supported at scale) +- `RSE-010`: Realized State only changes via authorized request; drift detection never writes to Realized Store +- `RSE-011`: Provider Update Notifications evaluated by Policy Engine before any Realized State change +- `RSE-012`: Categories of provider updates may be pre-authorized via GateKeeper policy +- `RSE-013`: Provider updates requiring consumer approval place entity in PENDING_REVIEW + +--- + +## SECTION 48 โ€” NOTIFICATION MODEL + +### 48.1 Core Principle: Relationship Graph Determines Audience + +The audience for every notification is derived from the **entity relationship graph at event time** โ€” not from a manually maintained subscriber list. When VLAN-100 is decommissioned, every VM attached to it gets notified automatically through their relationship edges. No subscription management required. + +### 48.2 Notification Provider โ€” Ninth Provider Type + +| # | Type | +|---|------| +| 1-8 | (existing providers) | +| **9** | **Notification Provider** โ€” translates DCM unified envelope to delivery channel (Slack, PagerDuty, email, ServiceNow, webhook, SMS); handles delivery, retry, dead letter; reports delivery status back to DCM | + +Notification Providers register with DCM declaring supported channels, sovereignty, and delivery guarantees. Organizations configure which channel to use per subscription. + +### 48.3 Three Subscription Tiers + +**Tier 1 โ€” Mandatory system notifications (non-suppressable):** Security events, sovereignty violations, audit chain breaks. Always delivered to Security Team + Platform Admin. Cannot be filtered. + +**Tier 2 โ€” Tenant defaults:** Tenant admin configures baseline for all resources in Tenant โ€” which event categories fire, which channels, urgency routing. + +**Tier 3 โ€” Actor subscriptions:** Individual actors subscribe to specific events on specific resources or resource types. + +Tiers compose: Tier 1 always fires; Tier 2 applies to all Tenant resources; Tier 3 adds specifics. Actor subscriptions can add channels but cannot suppress Tier 1. + +### 48.4 Audience Resolution Algorithm (6 steps) + +1. Direct owner of changed entity (role: owner) +2. Traverse relationship graph โ€” for each relationship: check event relevance, check min stake_strength, resolve related entity's owner (role: stakeholder) +3. Approval requirements โ€” add approvers (role: approver) +4. Mandatory system audiences (Security Team, Platform Admin for security events) +5. Actor subscription overrides (can add; cannot remove mandatory) +6. Deduplicate; same actor via multiple paths โ†’ one notification with all roles listed + +### 48.5 Event Taxonomy (closed vocabulary โ€” 7 categories) + +1. **Request lifecycle:** acknowledged, requires_approval, approved, dispatched, completed, failed, cancelled, gatekeeper_rejected +2. **Resource lifecycle:** realized, state_changed, ttl_warning, ttl_expired, suspended, resumed, decommissioning, decommissioned, decommission_deferred, ownership_transferred, pending_review +3. **Drift and discovery:** drift.detected, drift.severity_escalated, drift.resolved, drift.escalated, unsanctioned_change.detected +4. **Provider update:** submitted, requires_approval, approved, rejected, auto_approved +5. **Dependency and relationship:** dependency.state_changed, stakeholder.resource_decommissioning, allocation.pool_capacity_low, cross_tenant_auth.revoked +6. **Governance:** policy.activated, policy_provider.trust_elevated, profile.changed, catalog_item.deprecated +7. **Security/system (mandatory):** audit.chain_integrity_alert, sovereignty.violation, federation.tunnel_degraded, security.unsanctioned_provider_write + +### 48.6 Notification Envelope (unified โ€” all channels) + +Key fields: notification_uuid (idempotency), correlation_id (links to audit record), event_type, urgency (critical/high/medium/low), entity info, audience role + stakeholder_reason (WHY this actor is in audience), context (previous/new state, changed_fields), action (type/url/deadline for approvals), deep links. + +### 48.7 Provider Update + Notifications Integration + +`provider_update.requires_approval` fires โ†’ consumer receives notification with `action.type: approve`, `action.deadline` (default PT24H). Approved โ†’ `provider_update.approved` + `entity.state_changed` to stakeholders. Rejected โ†’ `provider_update.rejected` + drift event. + +### 48.8 Webhooks Are Now a Notification Channel + +Outbound webhooks (doc 18) are superseded by the Notification Model. Webhooks are one channel type within a Notification Provider. Existing webhook registrations are auto-converted to actor-level subscriptions with a webhook-type Notification Provider โ€” no migration needed. + +### 48.9 Delivery Pipeline + +Event โ†’ Audit record โ†’ Notification Router resolves audience โ†’ Subscription resolution โ†’ Envelope generation per actor โ†’ Route to Notification Provider(s) โ†’ Provider delivers โ†’ Delivery confirmation โ†’ NOTIFICATION_DISPATCHED audit record. + +### 48.10 Policies + +NOT-001 through NOT-008. Key: audience derived from relationship graph (NOT-001); mandatory notifications never suppressable (NOT-002); cross-tenant notifications sovereignty-checked (NOT-003); every dispatch is audited (NOT-004); Notification Provider must be registered for external delivery (NOT-007); event taxonomy is closed vocabulary (NOT-008). + +REL-022 through REL-024: traversal depth declared in Resource Type Spec; default depth 1; sovereignty respected; same actor via multiple paths โ†’ one notification with all roles. + +--- + +## SECTION 49 โ€” PERSONAS | Persona | Primary Concern | |---------|----------------| @@ -2792,7 +3056,7 @@ IAM, CAT, REQ, PRV, LCM, DRF, POL, LAY, INF, ING, AUD, OBS, STO, FED, GOV โ€” se --- -## SECTION 47 โ€” TERMINOLOGY GLOSSARY +## SECTION 50 โ€” TERMINOLOGY GLOSSARY | Term | Definition | |------|-----------| @@ -2855,6 +3119,30 @@ IAM, CAT, REQ, PRV, LCM, DRF, POL, LAY, INF, ING, AUD, OBS, STO, FED, GOV โ€” se | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail | | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class | | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation | +| **Notification Provider** | Ninth DCM provider type; translates unified notification envelope to delivery channel; handles delivery, retry, dead letter, and delivery confirmation callbacks | +| **Notification Router** | DCM control plane component that resolves notification audiences and routes envelopes to Notification Providers | +| **audience resolution** | Deriving notification recipients by traversing the entity relationship graph from the changed entity at event time | +| **notification_uuid** | Idempotency key on notification envelopes; Notification Providers use this to deduplicate on retry | +| **audience_role** | owner / stakeholder / approver / observer โ€” why this actor is in the notification audience | +| **stakeholder_reason** | Notification envelope field explaining which relationship caused the actor to be in the stakeholder audience | +| **Tier 1 / Tier 2 / Tier 3 notifications** | Mandatory system (non-suppressable) / Tenant defaults / Actor subscriptions โ€” three subscription tiers that compose | +| **NOT-001 through NOT-008** | Notification model system policies | +| **write-once snapshot store** | Realized Store implementation model: each record is a complete immutable entity state snapshot; no event replay; direct point-in-time lookup; supersession chain links snapshots | +| **corresponding_requested_state_uuid** | Mandatory non-nullable field on every Realized State snapshot; traces every Realized State change to an authorized request | +| **Provider Update Notification** | Formal API for providers to report authorized state changes; DCM evaluates via Policy Engine; approved โ†’ new Requested State + Realized State; rejected โ†’ drift event | +| **notification_uuid** | Idempotency key on Provider Update Notifications; safe to resend on provider crash | +| **pre-authorized update** | Category of provider update pre-approved by GateKeeper policy; processed automatically without per-change human review | +| **Whole Allocation** | Ownership pattern: consumer owns the entire resource entity outright in their Tenant; no pool involved | +| **Allocation** | Ownership pattern: pool yields independently-owned sub-resources; consumer owns their allocation; AllocationRecord relationship links to pool | +| **Shareable** | Ownership pattern: one resource, multiple stakeholders; consumers hold stakes (relationships) only; no consumer owns any portion | +| **AllocationRecord** | Cross-tenant relationship from an allocation entity back to its source pool entity | +| **stake_strength** | Relationship property on shareable resource attachments: required (blocks decommission) / preferred / optional | +| **PENDING_REVIEW** | Formal Infrastructure Resource Entity lifecycle state for conflicts requiring human resolution (sovereignty, cross-tenant auth revocation, ownership transfer conflicts) | +| **Consumer API** | DCM REST API for consumers: catalog browsing, request submission, resource management, audit trail access | +| **Consumer Request Status** | Lifecycle: ACKNOWLEDGED โ†’ ASSEMBLING โ†’ AWAITING_APPROVAL โ†’ APPROVED โ†’ DISPATCHED โ†’ PROVISIONING โ†’ COMPLETED/FAILED/CANCELLED | +| **01-entity-types.md** | Entity type taxonomy: Infrastructure Resource, Composite Resource, Process Resource; sub-types and invariants | +| **04-examples.md** | Worked examples: VM end-to-end, IP allocation, VLAN sharing, brownfield ingestion, drift remediation; Git repo structure | +| **04b-ownership-sharing-allocation.md** | Authoritative ownership model: whole allocation, allocation, shareable; policies OWN-001 through OWN-008 | | **federation routing** | Hub DCM applies placement engine logic at the DCM instance level; Regional DCMs are DCM Provider instances; sovereignty is a hard pre-filter; same tie-breaking hierarchy as provider selection | | **independent_with_overlap** | Certificate rotation model: old cert valid P30D after new cert issued; allows peers to update trust stores without coordinated downtime | | **alert_and_hold** | Federated drift detection response when peer DCM is unavailable: do not assume drift; hold state; escalate to platform admin after PT24H | @@ -3019,7 +3307,7 @@ IAM, CAT, REQ, PRV, LCM, DRF, POL, LAY, INF, ING, AUD, OBS, STO, FED, GOV โ€” se --- -## SECTION 48 โ€” OPEN QUESTIONS +## SECTION 51 โ€” OPEN QUESTIONS These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -3116,7 +3404,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 49 โ€” DOCUMENTATION STRUCTURE +## SECTION 52 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -3164,7 +3452,7 @@ content/ --- -## SECTION 50 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 53 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md index 50c7ceb..ced7f58 100644 --- a/content/docs/architecture/data-model/context-and-purpose.md +++ b/content/docs/architecture/data-model/context-and-purpose.md @@ -1,7 +1,7 @@ --- title: "Context and Purpose" type: docs -weight: 1 +weight: 0 --- > **โš ๏ธ Active Development Notice** @@ -12,7 +12,7 @@ weight: 1 **Document Status:** ๐Ÿ”„ In Progress -**Related Documents:** [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) +**Related Documents:** [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) --- @@ -179,7 +179,7 @@ The Audit capability in DCM reads provenance data that is intrinsic to every dat The DCM Data Model is governed by three foundational constraints that apply universally and without exception: -### 3.1 Declarative +### 5.1 Declarative Data in DCM describes **what something is or should be**, not how to achieve it. Every entity in the data model is a complete, self-describing statement of state. The procedures required to achieve that state are the concern of the Service Provider, not the data model. @@ -188,7 +188,7 @@ This means: - A policy declares its conditions and outcomes, not its execution logic - A layer declares its overrides, not the merge algorithm used to apply them -### 3.2 Idempotent in Operation +### 5.2 Idempotent in Operation Applying the same data to the same system multiple times must always produce the same result. No operation on DCM data should have different outcomes based on how many times it has been applied. @@ -198,7 +198,7 @@ This is critical for: - **Retry scenarios** โ€” failed operations can be safely retried without risk of inconsistent state - **Audit and compliance** โ€” the same data, applied by anyone at any time, produces the same verifiable outcome -### 3.3 Immutable if Versioned +### 5.3 Immutable if Versioned Once a version of any entity is published, it cannot be modified. If a change is required, a new version must be created. The previous version remains intact and accessible. @@ -354,7 +354,7 @@ The following questions remain unresolved and require decisions before the data | 3 | Which cache is authoritative when caches diverge? | Conflict resolution | โœ… Resolved โ€” GitOps stores always authoritative; caches are projections; divergence triggers rebuild from authoritative store (CACHE-003) | | 4 | What mechanism maintains consistency across distributed caches? | Data integrity | โœ… Resolved โ€” hash-based heartbeat divergence detection + push invalidation; PT2H staleness alert; signed bundles for Sovereign DCM (CACHE-004) | | 5 | Should the data model allow embedded target-technology-specific data bundles? | Portability | โœ… Resolved โ€” native_passthrough field sanctioned; always audit-logged; opaque mode blocked in fsi/sovereign (DATA-001) | -| 6 | How are the four states represented physically? | Physical model | โœ… Resolved โ€” Intent/Requested in Git; Realized in Event Stream; Discovered in Discovered Store (STO-005) |res?) | Implementation architecture | โ“ Unresolved | +| 6 | How are the four states represented physically? | Physical model | โœ… Resolved โ€” Intent/Requested in Git; Realized in Event Stream; Discovered in Discovered Store (STO-005) | | 7 | What is the performance impact of field-level provenance at scale? What optimization strategies are acceptable? | Scalability, storage cost | โœ… Resolved โ€” three configurable provenance models: full_inline, deduplicated (Model B recommended), tiered; profile-appropriate Policy Groups; see docs 03 and 06 (OPS-001) | | 8 | Should provenance metadata be stored inline with field data or in a linked provenance document? | Data model structure, query performance | โœ… Resolved โ€” three-level structure: implicit chain ref, inline delta, linked history document; all reconstructable from stored facts; see doc 03 (OPS-002) | diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md index ee7ce8c..34aa500 100644 --- a/content/docs/architecture/data-model/entity-relationships.md +++ b/content/docs/architecture/data-model/entity-relationships.md @@ -1,7 +1,7 @@ --- title: "Entity Relationships" type: docs -weight: 8 +weight: 9 --- > **โš ๏ธ Active Development Notice** @@ -845,7 +845,7 @@ The relationship graph exists across all four states: | `REL-012` | A Tenant with `hard_tenancy.cross_tenant_relationships: deny_all` may not participate in any cross-tenant relationship in any direction | | `REL-013` | `โŒ Invalid` relationship type ร— nature combinations (per the matrix in Section 6a) must be rejected by the Policy Engine at request time | | `REL-014` | An allocated resource claim requires a matching `available` allocation record on the parent entity | -| `REL-015` | A destructive lifecycle action on a shared resource entity (`shareable: true`) is deferred until `active_relationship_count` reaches `minimum_relationship_count` | +| `REL-015` | A destructive lifecycle action on a shared resource entity (`ownership_model: shareable` (see [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md))) is deferred until `active_relationship_count` reaches `minimum_relationship_count` | | `REL-016` | Informational relationships do not contribute to `active_relationship_count` on shared resource entities | | `REL-017` | A Resource Type Specification with `shareability.allowed: false` must reject any attempt to create more than one active constituent or operational relationship to an instance of that type | | `REL-018` | When a lifecycle event produces multiple action recommendations on a shared resource, the most conservative action wins per the hierarchy: `retain > notify > suspend > detach > cascade > destroy` (save_overrides_destroy) | @@ -920,6 +920,69 @@ Relationships follow the universal versioning and deprecation model. A relations | 4 | Should there be a maximum relationship graph depth to prevent runaway complexity? | Operational governance | โœ… Resolved โ€” profile-governed max depth: 15 standard/prod, 10 fsi/sovereign; circular detection always enforced; depth = traversal distance; see doc 09 Section 12 (REL-021) | | 5 | How are shared entities represented in the relationship graph โ€” an entity required by multiple parents? | Graph model | โœ… Resolved โ€” sharing_model declaration; active_relationship_count; save_overrides_destroy hierarchy (REL-018); lifecycle_conflict_record; REL-015 through REL-019 | + +--- + +## 14. Notification Traversal Rules + +The entity relationship graph is the source of truth for notification audiences. This section defines how relationships govern notification traversal for the Notification Model (doc 23). + +### 14.1 Relationship Properties Relevant to Notifications + +Every relationship carries two properties that the Notification Router uses for audience resolution: + +```yaml +relationship: + type: attached_to + stake_strength: + notification_relevance: + # Declared in the Resource Type Spec for this relationship type + # Can be overridden per relationship instance + notifiable_events: [entity.decommissioning, entity.state_changed, entity.ttl_expired] + traversal_depth: 1 # how many hops from this relationship + audience_role: stakeholder # role assigned to notified party +``` + +### 14.2 Stake Strength and Notification Threshold + +Different event types use different minimum stake strengths for notification: + +| Event Category | Minimum Stake Strength | Rationale | +|---------------|----------------------|-----------| +| `entity.decommissioning` | optional | All stakeholders should know | +| `entity.decommissioned` | optional | All stakeholders should know | +| `entity.state_changed` (to FAILED/DEGRADED) | required | Only required stakeholders are affected | +| `entity.state_changed` (to OPERATIONAL) | preferred | Recovery notification broader | +| `entity.ttl_expired` | required | Only required stakeholders need to act | +| `drift.detected` | โ€” (owner only) | Drift is the owner's concern | +| `dependency.state_changed` | required | Only affects required dependents | + +The minimum stake strength threshold per event type is declared in the resource type specification and can be overridden by a platform-domain policy. + +### 14.3 Notification Traversal and Graph Depth + +Notification traversal respects the same depth limits as other graph operations (REL-021: max depth 15 standard/prod, 10 fsi/sovereign). However, notification traversal depth is typically much shallower โ€” most event types only traverse depth 1 (direct relationships). + +``` +VLAN-100 decommissioning (depth 1 traversal): + Direct relationships: + โ”œโ”€โ”€ VM-A (attached_to, required) โ†’ AppTeam notified as stakeholder + โ”œโ”€โ”€ VM-B (attached_to, required) โ†’ DevTeam notified as stakeholder + โ””โ”€โ”€ VM-C (attached_to, optional) โ†’ OpsTeam notified as observer + No depth-2 traversal โ€” VM-A's dependencies are not notified about VLAN changes +``` + +Security events (sovereignty violation, audit chain break) use depth 0 (system audiences only โ€” no relationship traversal needed). + +### 14.4 Notification Traversal Policies + +| Policy | Rule | +|--------|------| +| `REL-022` | Notification traversal follows relationship edges from the changed entity. Traversal depth per event type is declared in the Resource Type Specification. Default traversal depth is 1. | +| `REL-023` | Notification traversal respects sovereignty boundaries. Cross-tenant notifications carry only content authorized for the receiving Tenant. | +| `REL-024` | The same actor reached via multiple relationship paths receives a single notification with all applicable audience_roles listed. | + + --- ## 13. Related Concepts diff --git a/content/docs/architecture/data-model/entity-types.md b/content/docs/architecture/data-model/entity-types.md new file mode 100644 index 0000000..0930238 --- /dev/null +++ b/content/docs/architecture/data-model/entity-types.md @@ -0,0 +1,341 @@ +--- +title: "Entity Types" +type: docs +weight: 1 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) + +--- + +## 1. Purpose + +This document defines the complete taxonomy of entity types in DCM. Every resource, service, group, and process managed by DCM is an entity โ€” and every entity belongs to one of the types defined here. The entity type determines the lifecycle state machine, the ownership model, the decommission behavior, and which data model fields are applicable. + +Understanding entity types is prerequisite to understanding: +- How lifecycle states are assigned and transition +- How ownership and allocation interact +- How drift detection operates at different levels +- How decommission cascades through dependent entities + +--- + +## 2. The Three Primary Entity Types + +DCM defines three primary entity types. Every entity is exactly one of these. + +### 2.1 Infrastructure Resource Entity + +An **Infrastructure Resource Entity** is a realized physical or virtual infrastructure resource that persists across time and has a full operational lifecycle. + +**Characteristics:** +- Persists after provisioning โ€” it continues to exist and consume resources until explicitly decommissioned +- Owned by exactly one Tenant at any point in time +- Has a full bidirectional lifecycle including OPERATIONAL and SUSPENDED states +- Subject to drift detection โ€” its Realized State is continuously compared against Discovered State +- Subject to TTL management โ€” may declare an expiry after which decommission is triggered +- May have relationships to other entities โ€” dependencies, attachments, allocations, business data +- Carries field-level provenance across its full lifecycle + +**Lifecycle State Machine:** + +``` + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ REQUESTED โ”‚ + โ”‚ (Intent State assembled, โ”‚ + โ”‚ Requested State committed) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Provider dispatch + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ PENDING โ”‚ + โ”‚ (Awaiting provider capacity โ”‚ + โ”‚ or dependency resolution) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Provider begins work + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ PROVISIONING โ”‚ + โ”‚ (Provider actively realizing) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Provider confirms realization + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ REALIZED โ”‚ + โ”‚ (Provider-confirmed, DCM has โ”‚ + โ”‚ full Realized State record) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Passes health checks + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ—„โ”€โ”€ Primary operational state + โ”‚ OPERATIONAL โ”‚ Drift detection active + โ”‚ (Active, healthy, in use) โ”‚ Cost analysis active + โ””โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ Policy evaluation active + โ”‚ โ”‚ + Suspend โ”‚ โ”‚ Decommission request + request โ–ผ โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ SUSPENDED โ”‚ โ”‚ DECOMMISSIONING โ”‚ + โ”‚ (Paused, not in โ”‚ โ”‚ (Provider removing, โ”‚ + โ”‚ active use, โ”‚ โ”‚ dependencies โ”‚ + โ”‚ may be billed โ”‚ โ”‚ being released) โ”‚ + โ”‚ at reduced rate)โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ + โ”‚ Resume โ”‚ Provider confirms removal + โ”‚ or decommission โ–ผ + โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บ DECOMMISSIONED โ”‚ โ—„โ”€โ”€ Terminal state + โ”‚ (Removed from infra, โ”‚ + โ”‚ audit records โ”‚ + โ”‚ preserved) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + +Any state except DECOMMISSIONED: + PROVISIONING_FAILED โ†’ rolls back to REQUESTED or terminal FAILED + PENDING_REVIEW โ†’ sovereignty/tenancy conflict during rehydration (see Section 2.1.2) +``` + +**Applicable to:** VirtualMachine, VLAN, IPAddress, StorageVolume, Container, LoadBalancer, DNSRecord, FirewallRule, NetworkPort, Subnet, and all other persistent infrastructure resource types. + +#### 2.1.1 Infrastructure Resource Entity Data Model + +```yaml +infrastructure_resource_entity: + # Universal artifact metadata + uuid: # stable across full lifecycle including rehydration + handle: # human-readable stable identifier + resource_type: # e.g., Compute.VirtualMachine + resource_type_spec_version: + lifecycle_state: + created_at: + updated_at: + + # Ownership + owned_by_tenant_uuid: # exactly one Tenant; mandatory + created_by_actor_uuid: + + # Ownership model โ€” see doc 04b + ownership_model: + # whole_allocation: consumer owns this entity outright + # allocation: this entity is an allocation carved from a pool (consumer owns it) + # shareable: consumer has a stake; ownership remains with pool owner + + # If this is an allocation from a pool resource + allocated_from_pool_uuid: # UUID of the pool entity; null if not an allocation + allocation_ref_uuid: # UUID of the AllocationRecord relationship + + # If this is a shareable stake + shared_resource_uuid: # UUID of the shared resource; null if not a stake + + # Provider details (populated after REALIZED) + provider_uuid: + provider_entity_id: # provider's own identifier (e.g., "vm-12345") + provider_entity_id_history: [...] # history of provider IDs (rehydration changes these) + + # Lifecycle constraints + ttl: + ttl_expires_at: + on_expiry: + billing_state: + + # Rehydration + rehydration_constraints: + min_auth_level: + allow_delegated_rehydration: + rehydration_history: [...] + + # Drift tracking + last_discovered_at: + drift_status: + last_drift_severity: + + # Relationships (see doc 09) + relationships: [...] + + # Field-level provenance on all data fields (see doc 00, Section 4) + # [all resource-type-specific fields carry provenance metadata] +``` + +#### 2.1.2 PENDING_REVIEW State + +`PENDING_REVIEW` is a formal lifecycle state for Infrastructure Resource Entities (not Process Resources). An entity enters `PENDING_REVIEW` when an automated operation detects a conflict that requires human resolution before the operation can proceed: + +| Trigger | Description | +|---------|-------------| +| Rehydration sovereignty conflict | Rehydration discovers the target provider no longer satisfies the entity's sovereignty constraints | +| Cross-tenant authorization revoked | An authorization enabling a cross-tenant resource reference was revoked while the resource is still allocated | +| Ownership transfer conflict | An ownership transfer request conflicts with active relationships that prevent transfer | + +An entity in `PENDING_REVIEW`: +- Is not actively drifting from its Realized State (the underlying resource is unchanged) +- Has an active `pending_review_record` on the entity with trigger, timestamp, and resolution options +- Generates notifications to the entity owner, Tenant admin, and platform admin +- Remains in `PENDING_REVIEW` until a resolution action is taken (re_authorize, release, escalate, or manual override) +- Is never automatically resolved โ€” all resolutions require explicit human or policy authorization + +### 2.2 Composite Resource Entity + +A **Composite Resource Entity** is produced by a Meta Provider that orchestrates multiple constituent Infrastructure Resource Entities to deliver a higher-order service. The composite is a first-class entity โ€” it has its own UUID, Tenant ownership, and lifecycle. Its constituents each retain their own entity identity. + +**Characteristics:** +- Represents the logical aggregate, not a physical resource +- Owned by exactly one Tenant (the Tenant that requested the composite service) +- Constituents may be owned by the same Tenant or may be allocations/stakes in pool resources owned by another Tenant +- Drift detection operates at two levels: the composite level (is the composite healthy as a whole?) and the constituent level (is each underlying resource still in its expected state?) +- Decommission is staged: composite decommissioned first, then constituents in reverse dependency order + +**Lifecycle state machine:** Same as Infrastructure Resource Entity. The composite's `lifecycle_state` reflects the aggregate health of all constituents โ€” a composite is OPERATIONAL only when all required constituents are OPERATIONAL. + +**Constituent relationship:** Each constituent is recorded as a `constituent_of` relationship from the constituent to the composite. The composite holds `has_constituent` relationships to each constituent. The composite UUID is the correlation key across all constituent audit records. + +```yaml +composite_resource_entity: + uuid: + resource_type: # e.g., ApplicationStack.WebApp + lifecycle_state: + owned_by_tenant_uuid: + composition_visibility: + # opaque: consumers see composite only; constituents hidden + # transparent: consumers see composite and all constituents + # selective: policy declares which constituents are visible + + constituents: + - constituent_entity_uuid: + role: + required_for_composite_operational: + # If a required constituent fails, the composite enters DEGRADED + constituent_lifecycle_state: + composite_health: +``` + +### 2.3 Process Resource Entity + +A **Process Resource Entity** represents an ephemeral execution โ€” an automation job, playbook, pipeline, workflow, or script execution. It does not persist after completion. Its lifecycle is terminal-focused: every Process Resource Entity ends in either COMPLETED, FAILED, or CANCELLED. + +**Characteristics:** +- Does not persist after reaching a terminal state โ€” no ongoing Realized State to manage +- Must declare `max_execution_time` โ€” mandatory, not optional +- If max_execution_time is exceeded, the process enters FAILED state and DCM generates a `PROCESS_TIMEOUT` event +- If the process modifies any Infrastructure Resource Entity, it must record the modified entity UUIDs in its provenance +- Owned by the Tenant that initiated the execution +- Subject to audit โ€” every process execution produces a full audit trail + +**Lifecycle state machine:** + +``` +REQUESTED โ†’ INITIATED โ†’ EXECUTING โ†’ COMPLETED (terminal) + โ†’ FAILED (terminal) + โ†’ CANCELLED (terminal โ€” requires explicit cancel request) +``` + +No SUSPENDED state. No PENDING_REVIEW state. Process Resources are ephemeral โ€” they do not enter states that require ongoing management. + +```yaml +process_resource_entity: + uuid: + resource_type: # e.g., Automation.AnsiblePlaybook + lifecycle_state: + owned_by_tenant_uuid: + created_by_actor_uuid: + + max_execution_time: # mandatory + started_at: + completed_at: + execution_timeout_at: # computed: started_at + max_execution_time + + # Entities this process modified (mandatory if any modifications made) + affected_entity_uuids: [, ...] + + # Execution details + provider_uuid: # which automation provider executed this + provider_job_id: # provider's own job identifier + exit_status: + execution_log_ref: # reference to log store entry + + # Provenance on all execution parameters carries field-level lineage +``` + +--- + +## 3. Sub-Types and Specializations + +### 3.1 Shared Resource Entity (Infrastructure Resource sub-type) + +A **Shared Resource Entity** is an Infrastructure Resource Entity where multiple consumers hold stakes โ€” references, attachments, or dependencies โ€” without any consumer owning an allocation of the resource. The resource has a single owner (typically a platform or network operations Tenant). Consumers reference it through relationships. + +See [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) for the complete model. + +**Examples:** VLAN, NetworkSegment, SharedStorageCluster, DNS Zone, NTP Server, Certificate Authority. + +**Key property:** `ownership_model: shareable` + +Decommission is deferred while any active stakeholder relationships exist. The `minimum_relationship_count` on the resource type spec declares the safe minimum โ€” typically 0 (can be decommissioned when all stakes are released) but may be higher for infrastructure that must always have at least one consumer. + +### 3.2 Allocatable Pool Resource (Infrastructure Resource sub-type) + +An **Allocatable Pool Resource** is an Infrastructure Resource Entity that serves as a pool from which consumers receive owned allocations. The pool itself is owned by a platform Tenant. Each allocation request produces a new, independently owned Infrastructure Resource Entity carved from the pool. + +See [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) for the complete model. + +**Examples:** IPAddressPool (allocates IPAddress entities), SubnetPool (allocates Subnet entities), VLANPool (allocates VLAN entities), StoragePool (allocates StorageVolume entities). + +**Key property:** `ownership_model: whole_allocation` on the pool entity; allocation products have `ownership_model: allocation`. + +The pool tracks available capacity. Allocation requests go through the placement engine like any other resource request. The produced allocation entity is owned by the requesting Tenant. + +--- + +## 4. Entity Identity Invariants + +These invariants apply to all entity types without exception: + +| Invariant | Rule | +|-----------|------| +| UUID stability | An entity's UUID never changes across its full lifecycle, including rehydration and provider migration | +| Single Tenant ownership | Every Infrastructure Resource Entity and Process Resource Entity is owned by exactly one Tenant at all times | +| Composite constituent ownership | A Composite Resource Entity's constituents are owned individually โ€” the composite UUID does not override constituent Tenant ownership | +| Immutable Realized State | Realized State events are append-only; a new event is created for every state change | +| Audit trail preservation | Audit records for an entity are never destroyed while any related entity is active; preservation policy governs post-terminal retention | +| Provider ID separation | The entity UUID is the DCM stable identity; the provider entity ID is the provider's own reference. These are separate and the provider ID may change on rehydration | + +--- + +## 5. Entity Type to Resource Type Mapping + +Not all resource types produce the same entity type. The entity type is declared in the Resource Type Specification: + +```yaml +resource_type_spec: + fqn: Compute.VirtualMachine + entity_type: infrastructure_resource # infrastructure_resource | composite_resource | process_resource + ownership_model: whole_allocation # whole_allocation | allocation | shareable + allocatable_from_pool_type: null # if allocation: the pool resource type this comes from + pool_resource_type: null # if pool: declare this is a pool resource + shareable: false # if shareable: true +``` + +--- + +## 6. Related Policies + +| Policy | Rule | +|--------|------| +| `ENT-001` | Every Infrastructure Resource Entity must be owned by exactly one Tenant at all times | +| `ENT-002` | Process Resource Entities must declare max_execution_time โ€” this field has no default and is not optional | +| `ENT-003` | Process Resource Entities must record all affected entity UUIDs if any infrastructure modifications are made during execution | +| `ENT-004` | Composite Resource Entity lifecycle_state reflects aggregate constituent health โ€” OPERATIONAL only when all required constituents are OPERATIONAL | +| `ENT-005` | PENDING_REVIEW is a valid Infrastructure Resource Entity state requiring human resolution โ€” it is never an error state and never automatically resolved | +| `ENT-006` | The entity UUID is immutable across the full entity lifecycle including rehydration, provider migration, and ownership transfer | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/examples.md b/content/docs/architecture/data-model/examples.md new file mode 100644 index 0000000..1dc7670 --- /dev/null +++ b/content/docs/architecture/data-model/examples.md @@ -0,0 +1,480 @@ +--- +title: "Worked Examples" +type: docs +weight: 4 +--- + +> **โš ๏ธ Active Development Notice** +> +> Examples in this document are illustrative of the intended architecture. YAML structures, field names, and sequences represent the design intent and will be refined as implementation proceeds. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Reference Examples +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) + +--- + +## 1. Purpose + +This document provides end-to-end worked examples that make the DCM data model concrete. Each example traces the complete lifecycle of a resource through DCM โ€” from consumer intent through the four states, showing exactly what data exists at each stage. + +These examples also resolve outstanding implementation details deferred from other documents, specifically the Git repository structure for the Intent and Requested stores. + +--- + +## 2. Git Repository Structure + +This resolves the deferred Q54 item from the Four States document (Section 4.1). + +The Intent and Requested stores use a handle-based directory structure within Git. Tenant isolation is enforced at the directory level. Provider selection (the Q54 concern) is recorded in the assembled payload, not in the directory structure โ€” so the directory structure is independent of which provider was selected. + +### 2.1 Intent Store Layout + +``` +intent-store/ +โ”œโ”€โ”€ {tenant-uuid}/ +โ”‚ โ”œโ”€โ”€ {resource-type-category}/ +โ”‚ โ”‚ โ”œโ”€โ”€ {resource-type}/ +โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ {entity-uuid}/ +โ”‚ โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ intent.yaml โ† consumer's raw declaration +โ”‚ โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ .metadata.yaml โ† intent metadata (created_by, timestamp, ingress surface) +โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ {entity-uuid-2}/ +โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ intent.yaml +โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ .metadata.yaml +โ”‚ โ”‚ โ””โ”€โ”€ ... +โ”‚ โ””โ”€โ”€ ... +โ””โ”€โ”€ ... + +# Example: +intent-store/ +โ””โ”€โ”€ a1b2c3d4-tenant-uuid/ + โ””โ”€โ”€ Compute/ + โ””โ”€โ”€ VirtualMachine/ + โ””โ”€โ”€ f5e6d7c8-entity-uuid/ + โ”œโ”€โ”€ intent.yaml + โ””โ”€โ”€ .metadata.yaml +``` + +**Branch naming:** `intent/{tenant-uuid}/{entity-uuid}` for new requests. `intent/{tenant-uuid}/{entity-uuid}/v{n}` for revisions. + +**Merge to main:** Triggers the CD pipeline โ€” Request Payload Processor begins assembly. + +### 2.2 Requested Store Layout + +``` +requested-store/ +โ””โ”€โ”€ {tenant-uuid}/ + โ””โ”€โ”€ {resource-type-category}/ + โ””โ”€โ”€ {resource-type}/ + โ””โ”€โ”€ {entity-uuid}/ + โ”œโ”€โ”€ requested.yaml โ† fully assembled payload + โ”œโ”€โ”€ assembly-provenance.yaml โ† complete layer chain and policy evaluation record + โ”œโ”€โ”€ placement.yaml โ† provider selection and placement constraints + โ””โ”€โ”€ dependencies.yaml โ† resolved dependency graph + +# Example: +requested-store/ +โ””โ”€โ”€ a1b2c3d4-tenant-uuid/ + โ””โ”€โ”€ Compute/ + โ””โ”€โ”€ VirtualMachine/ + โ””โ”€โ”€ f5e6d7c8-entity-uuid/ + โ”œโ”€โ”€ requested.yaml + โ”œโ”€โ”€ assembly-provenance.yaml + โ”œโ”€โ”€ placement.yaml + โ””โ”€โ”€ dependencies.yaml +``` + +### 2.3 Layer and Policy Store Layout + +``` +layers/ +โ”œโ”€โ”€ system/ +โ”‚ โ”œโ”€โ”€ core/ +โ”‚ โ”‚ โ”œโ”€โ”€ datacenter-layer.yaml +โ”‚ โ”‚ โ””โ”€โ”€ environment-layer.yaml +โ”‚ โ””โ”€โ”€ compliance/ +โ”‚ โ””โ”€โ”€ pci-dss-layer.yaml +โ”œโ”€โ”€ {tenant-uuid}/ +โ”‚ โ””โ”€โ”€ org/ +โ”‚ โ””โ”€โ”€ payments-team-layer.yaml +โ””โ”€โ”€ providers/ + โ””โ”€โ”€ {provider-uuid}/ + โ””โ”€โ”€ vm-defaults-layer.yaml + +policies/ +โ”œโ”€โ”€ system/ +โ”‚ โ”œโ”€โ”€ gatekeeper/ +โ”‚ โ”‚ โ””โ”€โ”€ vm-size-limits.yaml +โ”‚ โ””โ”€โ”€ transformation/ +โ”‚ โ””โ”€โ”€ inject-monitoring.yaml +โ””โ”€โ”€ {tenant-uuid}/ + โ””โ”€โ”€ gatekeeper/ + โ””โ”€โ”€ approved-os-images.yaml +``` + +--- + +## 3. Example 1 โ€” VM Provision End-to-End + +A developer on the AppTeam Tenant requests a standard Linux VM. This example traces the complete lifecycle through all four states. + +### 3.1 Consumer Submits Intent (Intent State) + +The developer submits the following intent via the Consumer API: + +```yaml +# intent-store/a1b2c3d4-tenant/Compute/VirtualMachine/f5e6d7c8-entity/intent.yaml + +apiVersion: dcm.io/v1 +kind: ResourceIntent +metadata: + entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0 + resource_type: Compute.VirtualMachine + tenant_uuid: a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6 # AppTeam Tenant + submitted_by: b2c3d4e5-actor-uuid + submitted_at: 2026-03-15T09:00:00Z + ingress_surface: consumer_api + +spec: + # Consumer declares what they need โ€” not how to provision it + cpu_count: 4 + memory_gb: 8 + storage_gb: 100 + os_family: rhel + environment: production + name: "payments-api-server-01" + # No provider specified โ€” consumer does not choose the provider +``` + +**CI pipeline runs immediately:** +- Policy pre-validation: no GateKeeper violations detected (4 CPU is within AppTeam's quota) +- Cost estimation: ~$0.32/hour based on current provider rates +- Dependency check: no dependencies declared โ€” clean +- Sovereignty check: AppTeam's Tenant has `data_residency: EU-WEST` โ€” placement must honor this +- Authorization check: actor b2c3d4e5 has `request:compute:vm` permission in AppTeam Tenant +- Auto-approve evaluation: meets all auto-approve criteria โ†’ PR auto-merged + +### 3.2 Assembly Produces Requested State + +After intent merge, the Request Payload Processor runs the nine-step assembly: + +**Step 3 โ€” Layer Resolution and Merge:** + +```yaml +# Layer chain assembled (in precedence order, highest to lowest): +# 1. system/core/datacenter-layer.yaml (system domain) +# 2. system/core/environment-layer.yaml (system domain) +# 3. system/compliance/eu-west-layer.yaml (system domain) +# 4. org/appteam-defaults-layer.yaml (tenant domain) +# 5. providers/openstack/vm-defaults-layer.yaml (provider domain โ€” pre-selected by policy) +# 6. Consumer intent (request domain) + +# Resulting merged fields before policy evaluation: +cpu_count: + value: 4 # from consumer intent + provenance.origin.source_type: consumer + provenance.origin.source_uuid: f5e6d7c8-entity + +memory_gb: + value: 8 # from consumer intent + provenance.origin.source_type: consumer + +storage_gb: + value: 100 # from consumer intent + +data_center: + value: "EU-WEST-DC1" # from datacenter layer + provenance.origin.source_type: base_layer + provenance.origin.source_uuid: dc-layer-uuid + +environment: + value: production # from consumer intent (overrides layer default "dev") + provenance.modifications: + - sequence: 1 + previous_value: dev # layer default + modified_value: production # consumer override + source_type: consumer + +monitoring_agent: + value: "datadog-agent:7.42" # injected by org layer โ€” consumer did not declare this + provenance.origin.source_type: intermediate_layer + provenance.origin.source_uuid: appteam-defaults-layer-uuid + +backup_policy: + value: "daily-30d-eu-west" # injected by compliance layer + provenance.origin.source_type: intermediate_layer + provenance.origin.source_uuid: eu-west-compliance-layer-uuid +``` + +**Step 5-7 โ€” Policy Evaluation:** + +```yaml +# GateKeeper policy: vm-size-limits evaluates +# Result: APPROVED (4 CPU within AppTeam's 16 CPU limit) + +# Transformation policy: inject-monitoring evaluates +# Result: monitoring_endpoint field injected +monitoring_endpoint: + value: "https://metrics.internal.eu-west.example.com" + provenance.modifications: + - sequence: 1 + previous_value: null + modified_value: "https://metrics.internal.eu-west.example.com" + source_type: policy + source_uuid: inject-monitoring-policy-uuid + operation_type: enrichment + reason: "Standard monitoring endpoint for EU-WEST production resources" + +# GateKeeper policy: approved-os-images evaluates (AppTeam's tenant policy) +# Result: APPROVED (rhel is in AppTeam's approved images list) +``` + +**Step 6 โ€” Placement Engine selects provider:** +- Sovereignty pre-filter: eligible providers must satisfy `data_residency: EU-WEST` +- Reserve query to 3 eligible OpenStack instances +- EU-WEST-Prod-1 responds: capacity available, confidence 94 +- EU-WEST-Prod-2 responds: capacity available, confidence 87 +- EU-WEST-Prod-3: insufficient capacity +- Tie-breaking: EU-WEST-Prod-1 selected (highest confidence score) + +**Requested State committed:** + +```yaml +# requested-store/a1b2c3d4-tenant/Compute/VirtualMachine/f5e6d7c8-entity/requested.yaml + +apiVersion: dcm.io/v1 +kind: RequestedState +metadata: + entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0 + resource_type: Compute.VirtualMachine + tenant_uuid: a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6 + assembled_at: 2026-03-15T09:00:47Z + intent_state_ref: f5e6d7c8-intent-ref-uuid + +spec: + cpu_count: { value: 4, provenance: {...} } + memory_gb: { value: 8, provenance: {...} } + storage_gb: { value: 100, provenance: {...} } + os_family: { value: rhel, provenance: {...} } + environment: { value: production, provenance: {...} } + name: { value: "payments-api-server-01", provenance: {...} } + data_center: { value: "EU-WEST-DC1", provenance: {...} } + monitoring_agent: { value: "datadog-agent:7.42", provenance: {...} } + backup_policy: { value: "daily-30d-eu-west", provenance: {...} } + monitoring_endpoint: { value: "https://metrics...", provenance: {...} } + +placement: + selected_provider_uuid: eu-west-prod-1-provider-uuid + placement_reason: "highest confidence score among eligible providers" + sovereignty_satisfied: true + reserve_query_response_ref: +``` + +### 3.3 Provider Realizes the Resource (Realized State) + +OpenStack EU-WEST-Prod-1 receives the payload, naturalizes it to OpenStack format, provisions the VM, and returns the denaturalized result: + +```yaml +# Event written to Realized Store event stream (entity_uuid key) + +event_type: REALIZED +entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0 +realized_at: 2026-03-15T09:03:12Z +provider_uuid: eu-west-prod-1-provider-uuid + +# DCM unified fields +cpu_count: { value: 4, provenance: { ...plus provider attribution } } +memory_gb: { value: 8, provenance: {...} } +storage_gb: { value: 100, provenance: {...} } + +# Provider-added fields (not in Requested State โ€” added by provider after realization) +provider_entity_id: "vm-0a1b2c3d" # OpenStack's internal VM ID +assigned_ip_address: "10.1.45.23" # IP assigned by provider at realization +hypervisor_host: "compute-node-07.eu-west" # where the VM was physically placed +actual_storage_gb: 102 # actual allocated (rounded up) +console_url: "https://console.eu-west.example.com/vm/0a1b2c3d" +``` + +### 3.4 Discovery Cycle (Discovered State) + +24 hours after realization, the discovery cycle runs: + +```yaml +# Snapshot written to Discovered Store + +snapshot_type: DISCOVERED +entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0 +discovered_at: 2026-03-16T09:00:00Z +discovery_method: openstack_api_query +provider_uuid: eu-west-prod-1-provider-uuid + +cpu_count: 4 # matches Realized State โ€” no drift +memory_gb: 8 # matches +storage_gb: 102 # matches (actual_storage_gb from provider) +provider_entity_id: "vm-0a1b2c3d" +status: ACTIVE +``` + +Drift Detection runs field-by-field comparison: all fields match Realized State. No drift event generated. + +--- + +## 4. Example 2 โ€” IP Address Allocation + +An allocation request showing the `allocation` ownership model (pool โ†’ owned allocation). + +```yaml +# Consumer submits intent for an IP address +# intent-store/a1b2c3d4-tenant/Network/IPAddress/ip-entity-uuid/intent.yaml + +spec: + requested_from: network # request from the network pool + address_family: IPv4 + purpose: vm_interface + attachment_ref: f5e6d7c8-entity-uuid # the VM this IP will be assigned to + +# Assembly runs โ€” placement engine finds eligible IPAddressPool +# Pool: NetworkOps/Network/IPAddressPool/10.1.0.0-16 (owned by NetworkOps Tenant) +# Available capacity: 65420 addresses + +# Provider carves allocation: +# New entity created: IPAddress 10.1.45.23/32 +# Owned by: AppTeam Tenant (a1b2c3d4) +# AllocationRecord relationship created: +# IPAddress 10.1.45.23/32 --[allocated_from]--> IPAddressPool 10.1.0.0/16 + +# Realized State event for the new IPAddress entity: +entity_uuid: ip-entity-uuid +resource_type: Network.IPAddress +ownership_model: allocation +owned_by_tenant_uuid: a1b2c3d4-appteam-uuid # AppTeam owns this +allocated_from_pool_uuid: pool-entity-uuid # NetworkOps owns the pool +address: "10.1.45.23" +prefix_length: 32 +address_family: IPv4 +``` + +When AppTeam decommissions their VM, the IP address entity can also be decommissioned. The pool's available capacity increases by 1. NetworkOps Tenant is unaffected. + +--- + +## 5. Example 3 โ€” VLAN Attachment (Shareable) + +A VM attaches to an existing VLAN โ€” the `shareable` ownership model (stake, not ownership). + +```yaml +# VLAN-100 exists โ€” owned by NetworkOps Tenant +# entity_uuid: vlan-100-entity-uuid +# ownership_model: shareable + +# Consumer (AppTeam) requests VM attachment to VLAN-100 +# No new VLAN entity is created โ€” a stake relationship is established: + +relationship: + type: attached_to + source_entity_uuid: f5e6d7c8-vm-entity-uuid # AppTeam's VM + target_entity_uuid: vlan-100-entity-uuid # NetworkOps's VLAN + source_tenant_uuid: a1b2c3d4-appteam-uuid + target_tenant_uuid: netops-tenant-uuid + stake: + is_active: true + stake_strength: required # VM cannot function without VLAN + staked_at: 2026-03-15T09:03:12Z + +# If NetworkOps tries to decommission VLAN-100: +# active required stakes: 3 (VM-A, VM-B, VM-C all have required stakes) +# Result: DECOMMISSION_DEFERRED +# NetworkOps notified: "VLAN-100 has 3 required stakeholders. Decommission deferred." +# Each stakeholder (AppTeam, DevTeam, OpsTeam) notified: +# "NetworkOps has requested decommission of VLAN-100. Please migrate your workloads." +``` + +--- + +## 6. Example 4 โ€” Brownfield Ingestion + +A VM discovered by the provider that DCM did not provision is brought under DCM lifecycle management. + +```yaml +# Step 1: INGEST โ€” discovery finds unknown VM +discovered_entity: + provider_entity_id: "vm-legacy-0001" + resource_type: Compute.VirtualMachine + lifecycle_state: OPERATIONAL # it's running + discovered_at: 2026-03-15T06:00:00Z + discovery_confidence: low # no DCM provenance + transitional_tenant: __transitional__ # held in transitional Tenant during ingestion + +# Step 2: ENRICH โ€” CMDB Information Provider enriches the entity +# CMDB lookup by IP address finds the business owner record: +enrichment: + owner_business_unit: "Payments Platform" + cost_center: "PAYM-4421" + product_owner: "Jane Smith" + compliance_scope: PCI-DSS + confidence_descriptor: + authority_level: primary # CMDB is primary authority for ownership data + corroboration: single_source # only CMDB has this data + source_trust: verified + +# Step 3: PROMOTE โ€” operator assigns to AppTeam Tenant, creates entity record +promotion: + target_tenant_uuid: a1b2c3d4-appteam-uuid + created_via: ingestion + intent_state_created: true # Intent State created from discovered configuration + provenance_basis: discovered # provenance chain starts from discovery + promoted_by: operator-actor-uuid + promoted_at: 2026-03-15T11:30:00Z +``` + +After promotion, the entity is a full DCM-managed entity. Drift detection is active. The operator can now request updates (targeted delta) or decommission through DCM. + +--- + +## 7. Example 5 โ€” Drift Detection and Remediation + +Six hours after the VM from Example 1 was realized, discovery finds a discrepancy: + +```yaml +# Discovery finds: +cpu_count: 4 # matches +memory_gb: 16 # DRIFT โ€” realized says 8, discovered says 16 + +# Drift record created: +drift_record: + entity_uuid: f5e6d7c8-entity-uuid + detected_at: 2026-03-15T15:00:00Z + drifted_fields: + - field_path: memory_gb + realized_value: 8 + discovered_value: 16 + drift_severity: significant # memory doubling is significant + unsanctioned: true # no DCM Requested State explains this change +``` + +**Policy Engine evaluates the drift record:** + +```yaml +# Drift response policy for Compute.VirtualMachine at significant severity: +# action: ESCALATE for unsanctioned changes + +escalation: + entity_uuid: f5e6d7c8-entity-uuid + notified: + - actor: b2c3d4e5-consumer-actor # the entity owner + - actor: appteam-admin-actor # AppTeam admin + - actor: sre-oncall-actor # SRE on-call + escalation_reason: "Unsanctioned memory change: 8Gi โ†’ 16Gi" + resolution_options: + - REVERT: "Submit rehydration from Realized State to restore 8Gi memory" + - UPDATE_DEFINITION: "Promote discovered state โ€” update entity definition to 16Gi" + - ACCEPT: "Accept the change; add to next review cycle" +``` + +The consumer reviews and chooses UPDATE_DEFINITION โ€” the memory was legitimately increased by the infrastructure team for a critical workload. They submit an UPDATE_DEFINITION resolution, which creates a new Requested State reflecting 16Gi memory and updates the Realized State record. Future drift detection will compare against 16Gi. + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md index f3d0276..a9cc256 100644 --- a/content/docs/architecture/data-model/four-states.md +++ b/content/docs/architecture/data-model/four-states.md @@ -24,10 +24,10 @@ The four states answer four distinct questions: | State | Question Answered | Store Type | |-------|------------------|------------| -| **Intent State** | What did the consumer ask for? | GitOps Store | -| **Requested State** | What was approved and dispatched to the provider? | GitOps Store | -| **Realized State** | What did the provider actually build? | Event Stream Store | -| **Discovered State** | What does DCM observe actually existing right now? | Event Stream Store (ephemeral) | +| **Intent State** | What did the consumer ask for? | GitOps Store (required) | +| **Requested State** | What was approved and dispatched to the provider? | Write-once Storage Provider (GitOps reference impl) | +| **Realized State** | What did the provider actually build? | Write-once Snapshot Store (authorized changes only) | +| **Discovered State** | What does DCM observe actually existing right now? | Ephemeral Snapshot Store | --- @@ -71,16 +71,30 @@ The **Requested State** is the fully assembled, policy-processed, provider-ready The **Realized State** is the provider-confirmed record of what was actually built. It is produced by the provider after successful realization โ€” the denaturalized result of the provider's execution, translated back to DCM Unified Data Model format. **Characteristics:** -- Append-only event stream โ€” each state change is a new event, never an overwrite -- Stored in an Event Stream Store โ€” high-frequency writes, entity-keyed streams -- The entity UUID is the stream key โ€” all realized state events for an entity share the same stream +- Write-once complete snapshots โ€” each Realized State record is a full entity state, never modified after writing +- Every Realized State record is traceable to exactly one Requested State record โ€” no exceptions +- Stored in a Write-once Snapshot Store keyed by entity UUID - Contains provider-specific details not in the Requested State โ€” assigned IPs, generated passwords, actual storage sizes, provider-internal IDs - Is the authoritative record of what actually exists from DCM's perspective -- Drift is detected by comparing Realized State against Discovered State +- Drift is detected by comparing the most recent Realized State snapshot against Discovered State +- Carries a supersession chain โ€” each snapshot knows which snapshot it superseded and which superseded it -**When created:** After provider confirms realization, updated on every provider lifecycle event +**Three write sources (all require a corresponding Requested State record):** -**Content:** The realized entity in DCM Unified Data Model format, with provider-added fields, full field-level provenance including provider attribution +| Source | Requested State record type | Example | +|--------|---------------------------|---------| +| Initial realization | `initial_realization` | Consumer provisions a new VM | +| Consumer update request | `consumer_update` | Consumer patches an editable field | +| Provider update notification | `provider_update` | Provider reports an authorized state change (auto-healing, maintenance) | + +**What does NOT write to the Realized Store:** +- Drift detection โ€” drift only compares, never writes +- Discovery cycles โ€” discovery writes to Discovered Store only +- Unsanctioned provider changes โ€” these are drift events until DCM evaluates and explicitly approves them + +**When created:** When a provider confirms realization of any authorized request (initial, consumer update, or approved provider update notification) + +**Content:** Complete entity state snapshot in DCM Unified Data Model format, with provider-added fields, full field-level provenance including provider attribution, and supersession chain references ### 2.4 Discovered State @@ -132,19 +146,76 @@ See [Storage Providers](11-storage-providers.md) for the complete contract speci - Indexed for query โ€” a Search Index projection enables field-based queries at scale - Entity UUID โ†’ file path mapping maintained in the Search Index -**Typical implementations:** GitHub, GitLab, Gitea, Forgejo (with Elasticsearch/OpenSearch as the Search Index) +**Intent Store โ€” GitOps implementation required:** +The PR workflow, branch-per-request semantics, and human review flow are first-class features of the Intent Store โ€” not implementation details. GitOps is the only supported implementation for the Intent Store. + +**Requested Store โ€” write-once Storage Provider (GitOps is the reference implementation):** +The Requested Store requires write-once semantics and hash-chain integrity but does not require GitOps PR mechanics โ€” the Requested State is machine-generated output, not consumer input. The GitOps implementation is the reference implementation and suitable for small-to-medium deployments. For production scale (thousands of requests per day), a purpose-built write-once document store is explicitly supported. + +The distinction matters for the following reasons: +- Git performance degrades at scale (large repo size, high-frequency machine writes) +- PR semantics (branch creation, merge, CI hooks) add latency and overhead with no workflow benefit for machine-generated content +- Sensitive assembled payload data may warrant stricter field-level access control than Git provides +- A write-once document store with hash-chain integrity satisfies all Requested Store contracts without Git's operational constraints -**Repository structure:** Deferred pending Q54 resolution (provider selection in Requested State affects directory structure). Will be documented in `04-examples.md`. +**Typical implementations (Intent Store):** GitHub, GitLab, Gitea, Forgejo -### 4.2 Event Stream Stores (Realized and Discovered) +**Typical implementations (Requested Store):** GitHub/GitLab/Gitea (reference implementation); PostgreSQL with write-once enforcement + Merkle hash chain (production scale); CockroachDB (geo-distributed) + +**Repository structure:** Resolved. See [Worked Examples](04-examples.md) Section 2 for the complete Git directory layout. Provider selection is recorded in the assembled payload (placement.yaml), not in the directory structure โ€” directories are independent of provider selection (Q54 resolved). + +### 4.2 Write-once Snapshot Store (Realized State) + +The Realized Store uses a **write-once snapshot model** โ€” each Realized State record is a complete entity state snapshot, not a field-level event. This model aligns with the constraint that Realized State only changes via authorized requests. **Contract characteristics:** -- Append-only โ€” events are never overwritten or deleted -- Entity-keyed streams โ€” each entity has its own event stream identified by entity UUID -- Queryable by entity UUID โ€” O(1) lookup of an entity's event stream -- Replayable โ€” the stream can be replayed from any point to reconstruct state at any timestamp -- Distributed and redundant โ€” data is replicated across nodes with configurable consistency guarantees +- Write-once โ€” each snapshot record is immutable after creation; updates create new snapshot records +- Complete snapshots โ€” each record captures the full entity state, not a delta from previous state +- Entity-UUID-keyed โ€” O(1) lookup of all snapshots for a given entity +- Supersession chain โ€” each record references the snapshot it superseded and the Requested State record that authorized the change +- Queryable by timestamp โ€” point-in-time state reconstruction is a direct lookup, not a replay +- Traceable โ€” every record has a non-nullable `corresponding_requested_state_uuid` field + +**Realized State snapshot record structure:** + +```yaml +realized_state_snapshot: + realized_state_uuid: # this snapshot's identity + entity_uuid: # entity this belongs to + realized_at: + + # Always traceable to a request โ€” mandatory, not nullable + source_type: + corresponding_requested_state_uuid: + + # Supersession chain + supersedes_realized_state_uuid: # null for first realization + superseded_by_realized_state_uuid: # null for current record + + # Complete entity state at this point โ€” all fields, all provenance + fields: + # [full entity state in DCM Unified Data Model format] + + # Provider-added fields + provider_entity_id: + provider_reported_at: +``` + +**Why snapshots instead of events:** +Rehydration from Realized State requires a complete entity state โ€” not a replay of field-level events. A snapshot model makes rehydration a direct lookup rather than an event replay. Point-in-time queries ("what was the Realized State on March 15?") are also direct lookups. The Realized Store does not need the high-frequency write throughput of an event stream โ€” it is written only when an authorized change completes. + +**Typical implementations:** PostgreSQL with write-once constraints; CockroachDB; etcd (small deployments) + +### 4.3 Ephemeral Snapshot Store (Discovered State) + +The Discovered Store retains its event stream model โ€” discovery is high-frequency, machine-generated, and ephemeral. It is never a rehydration source by definition (discovered state was never authorized through DCM). + +**Contract characteristics:** +- Append-only snapshot stream โ€” each discovery cycle produces a new snapshot +- Entity-UUID-keyed streams โ€” each entity has its own snapshot stream +- Replayable โ€” for trending and historical discovery analysis - High throughput โ€” designed for machine-generated, high-frequency writes +- Ephemeral โ€” retention policy governs how long snapshots are kept (see RHY-008) **Typical implementations:** Kafka with log compaction, EventStoreDB, Apache Pulsar diff --git a/content/docs/architecture/data-model/notification-model.md b/content/docs/architecture/data-model/notification-model.md new file mode 100644 index 0000000..4a034e8 --- /dev/null +++ b/content/docs/architecture/data-model/notification-model.md @@ -0,0 +1,593 @@ +--- +title: "Notification Model" +type: docs +weight: 23 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Webhooks, Messaging, and External Integration](18-webhooks-messaging.md) | [Entity Relationships](09-entity-relationships.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Auth Providers](19-auth-providers.md) | [Universal Audit](16-universal-audit.md) + +--- + +## 1. Purpose + +The DCM Notification Model defines a **unified, configurable notification pipeline** that routes event notifications to all parties with a stake in a changed resource โ€” not just the original requestor. The audience for any notification is derived from the **entity relationship graph**, not from who submitted the original request. + +This document defines: +- The Notification Provider โ€” the ninth DCM provider type +- The event taxonomy โ€” a closed vocabulary of notification-worthy events +- The audience resolution model โ€” how the relationship graph determines who gets notified +- The subscription model โ€” how actors declare their notification preferences +- The notification payload structure โ€” the unified envelope all Notification Providers receive +- The delivery pipeline โ€” from event trigger through audience resolution through provider delivery + +This model supersedes the standalone outbound webhook model in doc 18. Outbound webhooks are now one delivery channel of the Notification Provider, not a parallel mechanism. + +--- + +## 2. Design Principles + +**Relationship graph determines audience.** When a resource changes, DCM traverses the entity relationship graph to find all stakeholders. A VLAN decommission notifies every VM attached to that VLAN, regardless of which Tenant owns each VM. The graph is the source of truth for notification scope. + +**Delivery mechanism is configurable, not prescribed.** DCM generates and routes notifications. How they are delivered โ€” email, Slack, PagerDuty, ServiceNow, webhook, SMS โ€” is the concern of a Notification Provider. Organizations register the Notification Provider(s) that fit their operations. + +**Three notification tiers.** Some notifications are mandatory and non-suppressable (security, sovereignty violations, audit chain breaks). Some are Tenant-default (all resource lifecycle events in a Tenant). Some are actor-subscription (specific events on specific resources). All three compose without conflict. + +**Audience role shapes the notification.** The same event produces different notifications for an owner ("your resource changed") versus a stakeholder ("a resource you depend on changed") versus an approver ("your approval is required"). The audience role is part of the notification envelope. + +**Delivery is audited.** Every notification dispatch is an audit record. Delivery failures are tracked and escalated per policy. + +--- + +## 3. The Notification Provider + +The Notification Provider is the ninth formal DCM provider type. It handles the translation from DCM's unified notification envelope to the delivery channel's native format, and it handles delivery, retry, and delivery confirmation. + +### 3.1 Provider Types Table Update + +| # | Type | Purpose | +|---|------|---------| +| 1 | Service Provider | Realizes resources | +| 2 | Information Provider | Serves authoritative external data | +| 3 | Meta Provider | Composes multiple providers | +| 4 | Storage Provider | Persists DCM state | +| 5 | Message Bus Provider | Event streaming and messaging | +| 6 | Policy Provider | External policy logic | +| 7 | Credential Provider | Resolves secrets | +| 8 | Auth Provider | Authenticates identities | +| **9** | **Notification Provider** | **Delivers notifications via configured channels** | + +### 3.2 Notification Provider Registration + +```yaml +notification_provider_registration: + artifact_metadata: + uuid: + handle: "org/notifications/slack-provider" + version: "1.0.0" + status: active + owned_by: { display_name: "Platform Engineering" } + + provider_type: notification + display_name: "Slack Notification Provider" + description: "Delivers DCM notifications to configured Slack channels" + + # Delivery channels this provider supports + delivery_channels: + - channel_type: slack + config_schema_ref: # JSON Schema for channel config + supports_threading: true + supports_urgency_routing: true # different channels per urgency level + - channel_type: webhook + config_schema_ref: + + # Sovereignty declaration โ€” same model as all providers + sovereignty_declaration: + data_residency_guarantee: EU + operating_jurisdictions: [DE, FR, NL] + + # Delivery guarantees this provider offers + delivery_guarantees: + at_least_once: true + idempotency_key: notification_uuid + max_delivery_latency: PT30S # for critical urgency + retry_policy: + max_attempts: 7 + backoff: exponential + initial_interval: PT5S + max_interval: PT1H + on_exhaustion: dead_letter + + # Health check endpoint (on the provider) + health_endpoint: https://notif-provider.corp.example.com/health + + # Callback endpoint (DCM calls this to submit notifications) + delivery_endpoint: https://notif-provider.corp.example.com/deliver +``` + +### 3.3 Multiple Notification Providers + +Organizations may register multiple Notification Providers โ€” one for Slack, one for PagerDuty, one for ServiceNow tickets. Notification subscriptions declare which provider to use for delivery. The Notification Router in DCM routes each notification to the correct provider based on the subscription's `notification_provider_uuid`. + +--- + +## 4. The Event Taxonomy + +The notification event taxonomy is a **closed vocabulary** โ€” a finite, versioned set of event types that DCM can generate notifications for. Events are grouped by category. Subscriptions reference these event types by name. + +### 4.1 Request Lifecycle Events + +| Event Type | Trigger | Default Audience | +|-----------|---------|-----------------| +| `request.acknowledged` | Request received, Intent State created | Owner | +| `request.requires_approval` | Policy requires human review before dispatch | Owner, Approvers | +| `request.approved` | Intent State PR merged; proceeding to assembly | Owner | +| `request.dispatched` | Requested State committed; dispatched to provider | Owner | +| `request.completed` | Provider confirmed realization; Realized State written | Owner | +| `request.failed` | Request failed at any stage | Owner | +| `request.cancelled` | Consumer cancelled; request terminated | Owner | +| `request.gatekeeper_rejected` | GateKeeper policy rejected the request | Owner, Policy Owner | + +### 4.2 Resource Lifecycle Events + +| Event Type | Trigger | Default Audience | +|-----------|---------|-----------------| +| `entity.realized` | Entity first realized by provider | Owner, Stakeholders (depth 1) | +| `entity.state_changed` | Entity lifecycle state transition | Owner, Stakeholders (required) | +| `entity.ttl_warning` | TTL expires within declared warning window | Owner | +| `entity.ttl_expired` | TTL reached; expiry action triggered | Owner, Stakeholders (required) | +| `entity.suspended` | Entity entered SUSPENDED state | Owner, Stakeholders (required) | +| `entity.resumed` | Entity exited SUSPENDED state | Owner, Stakeholders (required) | +| `entity.decommissioning` | Decommission initiated | Owner, Stakeholders (all) | +| `entity.decommissioned` | Entity fully decommissioned | Owner, Stakeholders (all) | +| `entity.decommission_deferred` | Decommission blocked by active stakes | Owner, Stakeholders (required) | +| `entity.ownership_transferred` | Ownership moved to a different Tenant | Previous Owner, New Owner | +| `entity.pending_review` | Entity entered PENDING_REVIEW state | Owner, Platform Admin | + +### 4.3 Drift and Discovery Events + +| Event Type | Trigger | Default Audience | +|-----------|---------|-----------------| +| `drift.detected` | Discovered State differs from Realized State | Owner | +| `drift.severity_escalated` | Drift severity increased | Owner, Platform Admin | +| `drift.resolved` | Drift resolved (REVERT or UPDATE_DEFINITION) | Owner | +| `drift.escalated` | Drift escalated to human review | Owner, Platform Admin, SRE | +| `unsanctioned_change.detected` | Change detected with no corresponding Requested State record | Owner, Security Team, Platform Admin | + +### 4.4 Provider Update Events + +| Event Type | Trigger | Default Audience | +|-----------|---------|-----------------| +| `provider_update.submitted` | Provider submitted an update notification | Owner | +| `provider_update.requires_approval` | Provider update requires consumer approval | Owner (approval required) | +| `provider_update.approved` | Provider update approved; Realized State updated | Owner | +| `provider_update.rejected` | Provider update rejected; becomes drift | Owner, Provider Team | +| `provider_update.auto_approved` | Provider update auto-approved by pre-authorization policy | Owner (informational) | + +### 4.5 Dependency and Relationship Events + +| Event Type | Trigger | Default Audience | +|-----------|---------|-----------------| +| `dependency.state_changed` | A required dependency's state changed | Owner of dependent entity | +| `stakeholder.resource_decommissioning` | A shared resource the actor stakes is decommissioning | All stakeholders | +| `allocation.pool_capacity_low` | Allocation pool capacity below threshold | Pool Owner, Platform Admin | +| `allocation.released` | Allocation decommissioned; capacity returned to pool | Pool Owner | +| `cross_tenant_auth.expiring` | Cross-tenant authorization expiring | Both Tenant Admins | +| `cross_tenant_auth.revoked` | Cross-tenant authorization revoked while allocation active | Both Tenant Admins, Affected Resource Owners | + +### 4.6 Governance Events + +| Event Type | Trigger | Default Audience | +|-----------|---------|-----------------| +| `policy.activated` | Policy moved to active status | Platform Admin, Policy Owner | +| `policy.deactivated` | Policy deactivated | Platform Admin, Policy Owner | +| `policy_provider.trust_elevated` | Policy Provider mode level elevated | Platform Admin, Security Team | +| `profile.changed` | Active deployment profile changed | Platform Admin, All Tenant Admins | +| `catalog_item.deprecated` | Catalog item deprecated | All consumers with active resources of that type | + +### 4.7 Security and System Events (Mandatory โ€” Non-Suppressable) + +| Event Type | Trigger | Audience | +|-----------|---------|---------| +| `audit.chain_integrity_alert` | Hash chain verification failure detected | Security Team, Platform Admin | +| `sovereignty.violation` | Resource in violation of sovereignty constraints | Platform Admin, Security Team, Resource Owner | +| `sovereignty.migration_required` | Provider sovereignty change requires entity migration | Platform Admin, Resource Owner | +| `federation.tunnel_degraded` | DCM-to-DCM federation tunnel health degraded | Platform Admin, SRE | +| `auth.provider_failover` | Auth Provider failed over to secondary | Platform Admin | +| `rehydration.blocked` | Concurrent rehydration attempt rejected | Requesting Actor, Platform Admin | +| `security.unsanctioned_provider_write` | Attempted write to Realized Store without Requested State ref | Security Team, Platform Admin | + +--- + +## 5. Audience Resolution โ€” The Relationship Graph Model + +### 5.1 The Fundamental Rule + +**The audience for a notification is every entity with a stake in the changed resource, resolved by traversing the relationship graph from the changed entity.** + +The notification system does not maintain a separate subscriber list per entity. It derives the audience at event time by traversing the relationship graph. This means the audience is always current โ€” adding a new VM attachment to a VLAN automatically includes that VM's owner in future VLAN notifications, without any subscription update required. + +### 5.2 Audience Resolution Algorithm + +``` +Event fires on entity E (e.g., VLAN-100 decommissioning) + โ”‚ + โ–ผ Step 1: Resolve direct owner + โ”‚ entity.owned_by_tenant_uuid โ†’ Tenant Admin and resource owner actors + โ”‚ Audience role: owner + โ”‚ + โ–ผ Step 2: Traverse relationship graph + โ”‚ For each relationship on entity E: + โ”‚ Check: is this relationship type notification-relevant for this event type? + โ”‚ Check: does the relationship's stake_strength meet the minimum for this event? + โ”‚ If yes: resolve the related entity's owner โ†’ add to audience + โ”‚ Audience role: stakeholder + โ”‚ + โ–ผ Step 3: Check for approval requirements + โ”‚ Does this event require approval from a specific actor? + โ”‚ If yes: add approver to audience + โ”‚ Audience role: approver + โ”‚ + โ–ผ Step 4: Apply mandatory system audiences + โ”‚ Security events: always include Security Team and Platform Admin + โ”‚ Governance events: always include Policy Owner and Platform Admin + โ”‚ (These cannot be filtered out by subscription preferences) + โ”‚ + โ–ผ Step 5: Apply actor subscription overrides + โ”‚ Actors with explicit subscriptions to this event type โ†’ include/exclude per subscription + โ”‚ (Subscriptions can add additional audience; they cannot remove mandatory audiences) + โ”‚ + โ–ผ Step 6: Deduplicate and resolve contact details + โ”‚ Same actor via multiple paths โ†’ one notification with all audience_roles listed + โ”‚ Resolve each actor to their configured notification channels + โ”‚ + โ–ผ Step 7: Route to Notification Provider(s) + One notification per actor per configured channel + Notification envelope includes audience_role +``` + +### 5.3 Relationship Notification Relevance + +The Resource Type Specification declares which relationship types are notification-relevant and for which events: + +```yaml +resource_type_spec: + fqn: Network.VLAN + notification_rules: + - event_type: entity.decommissioning + notify_relationships: + - relationship_type: attached_to # source direction (VMs attached to this VLAN) + min_stake_strength: required # only required stakes get notified + traversal_depth: 1 # direct relationships only + audience_role: stakeholder + - relationship_type: attached_to + min_stake_strength: optional # optional stakes get informational notice + traversal_depth: 1 + audience_role: observer + + - event_type: entity.state_changed + notify_relationships: + - relationship_type: attached_to + min_stake_strength: required + traversal_depth: 1 + audience_role: stakeholder +``` + +**Traversal depth:** `1` means direct relationships only. `2` means relationships of related entities. In most cases `1` is correct โ€” deeper traversal is reserved for critical security events that affect the entire graph. + +### 5.4 Cross-Tenant Notification + +Notification traversal follows relationship graphs across Tenant boundaries. If a VM in AppTeam Tenant has a `required` stake in a VLAN owned by NetworkOps Tenant, and the VLAN is decommissioned, AppTeam receives a stakeholder notification โ€” even though the VLAN belongs to a different Tenant. + +Cross-tenant notifications are governed by the same sovereignty rules as cross-tenant data access: +- Notification content is limited to what the receiving Tenant is authorized to know +- The notification identifies the changed resource but does not expose the owning Tenant's configuration details +- Sovereignty checks apply to notification delivery (a notification about an EU-sovereign resource cannot be delivered to a US-based endpoint) + +```yaml +notification_sovereignty_check: + # Before delivering cross-tenant notification: + check: + - receiver_tenant_sovereignty_compatible: true + - notification_content_authorized_for_receiver: true + - delivery_endpoint_jurisdiction_compatible: true + on_failure: redact_and_deliver # or: suppress_with_audit | block_with_alert +``` + +--- + +## 6. Notification Subscriptions + +### 6.1 Three Subscription Tiers + +**Tier 1 โ€” Mandatory System Notifications (non-suppressable):** +Security events, sovereignty violations, audit chain breaks. Always delivered to the declared system audiences (Security Team, Platform Admin) regardless of any subscription configuration. No actor or policy can suppress these. + +**Tier 2 โ€” Tenant Default Notifications:** +Configured by Tenant admins for all resources in their Tenant. Establishes the baseline notification behavior โ€” which events trigger notifications, which channels to use, and which urgency mapping to apply. + +```yaml +tenant_notification_defaults: + tenant_uuid: + notification_provider_uuid: + + default_channel_config: + channel_type: slack + workspace: "corp" + urgency_routing: + critical: "#platform-incidents" + high: "#platform-alerts" + medium: "#platform-notifications" + low: "#platform-digest" # batched hourly + + # Which event categories are enabled by default for all resources in this Tenant + enabled_event_categories: + request_lifecycle: [request.completed, request.failed, request.gatekeeper_rejected] + resource_lifecycle: [entity.state_changed, entity.ttl_warning, entity.decommissioning] + drift: [drift.detected, unsanctioned_change.detected] + provider_update: [provider_update.requires_approval, provider_update.rejected] + dependency: [stakeholder.resource_decommissioning, cross_tenant_auth.revoked] + + # Urgency defaults per event type + urgency_overrides: + unsanctioned_change.detected: critical + drift.detected: high + entity.ttl_warning: medium + request.completed: low +``` + +**Tier 3 โ€” Actor-Level Subscriptions:** +Individual actors subscribe to specific events on specific resources or resource types. Most useful for service accounts (CI/CD pipelines, monitoring tools) that need targeted event feeds. + +```yaml +actor_notification_subscription: + subscription_uuid: + actor_uuid: + notification_provider_uuid: + + channel_config: + channel_type: pagerduty + service_id: "payments-api-on-call" + escalation_policy_id: "payments-prod" + + subscriptions: + # Subscribe to all drift events on VMs in AppTeam Tenant + - scope: + tenant_uuid: + resource_type: Compute.VirtualMachine + events: [drift.detected, unsanctioned_change.detected] + urgency_override: high + + # Subscribe to decommission of a specific VLAN I depend on + - scope: + entity_uuid: + events: [entity.decommissioning, entity.decommissioned] + urgency_override: critical +``` + +### 6.2 Subscription Composition Rules + +When multiple subscription tiers match for the same actor and event: +- Mandatory system notifications always fire (cannot be suppressed) +- Tenant defaults fire unless the actor's subscription explicitly opts out for that event type +- Actor subscriptions can add additional channels or override urgency โ€” they do not suppress Tenant defaults unless the subscription explicitly declares `suppress_tenant_default: true` +- Deduplication: if the same notification would be delivered to the same actor via two channels from two subscription matches, deliver once per channel (not once per subscription match) + +--- + +## 7. Notification Payload โ€” The Unified Envelope + +Every notification delivered to a Notification Provider uses this unified envelope. The Notification Provider translates it to the delivery channel's native format. + +```yaml +notification: + # Identity + notification_uuid: # idempotency key + correlation_id: # links to the audit record for the triggering event + generated_at: + + # The event + event_type: entity.decommissioning # from the closed taxonomy + event_uuid: # the triggering event's UUID + urgency: + + # The subject entity + entity: + uuid: + handle: + resource_type: Network.VLAN + display_name: "VLAN-100 (EU-WEST Production)" + tenant_uuid: + tenant_display_name: "NetworkOps" + + # Audience context + audience: + actor_uuid: + actor_display_name: "Jane Smith" + audience_role: + # stakeholder: explains WHY this actor is in the audience + stakeholder_reason: + via_entity_uuid: # "because your VM-A is attached to this VLAN" + via_relationship_type: attached_to + via_entity_display_name: "VM-A (payments-api-server-01)" + + # What changed + context: + previous_state: OPERATIONAL + new_state: DECOMMISSIONING + change_summary: "VLAN-100 decommission initiated by NetworkOps team" + changed_fields: [] + changed_by: + actor_uuid: + actor_display_name: "Bob Jones (NetworkOps)" + effective_at: + + # Action required (if any) + requires_action: false + action: + type: null # approve | acknowledge | migrate | release_stake + description: null + action_url: null + deadline: null + + # Deep links + links: + entity_url: "https://dcm.corp.example.com/resources/" + event_url: "https://dcm.corp.example.com/audit/" + related_entities: + - uuid: + display_name: "VM-A (payments-api-server-01)" + url: "https://dcm.corp.example.com/resources/" +``` + +### 7.1 Urgency Mapping + +| Urgency | Meaning | Typical delivery target | +|---------|---------|------------------------| +| `critical` | Immediate action required; outage or security risk imminent | On-call pager, incident channel | +| `high` | Action required; significant impact if not addressed | Alert channel, SRE queue | +| `medium` | Action recommended; non-urgent but should not be ignored | Notification channel, daily digest | +| `low` | Informational; no action required | Digest, async channel | + +Default urgency per event type is declared in the event taxonomy. Tenant defaults and actor subscriptions may override upward or downward. + +--- + +## 8. The Delivery Pipeline + +``` +Event fires (e.g., VLAN-100 enters DECOMMISSIONING state) + โ”‚ + โ–ผ Stage 1: Audit record written (Stage 1 Commit Log โ€” synchronous) + โ”‚ ENTITY_STATE_CHANGED audit record committed + โ”‚ event_uuid assigned + โ”‚ + โ–ผ Stage 2: Notification Router evaluates + โ”‚ Load entity relationship graph for VLAN-100 + โ”‚ Run audience resolution algorithm (Section 5.2) + โ”‚ Result: [AppTeam, DevTeam, OpsTeam] as stakeholders; [NetworkOps] as owner + โ”‚ + โ–ผ Stage 3: Subscription resolution + โ”‚ For each audience member: + โ”‚ Resolve Tier 1 mandatory notifications + โ”‚ Apply Tier 2 Tenant defaults + โ”‚ Apply Tier 3 actor subscriptions + โ”‚ Determine: which Notification Provider(s); which channel config; urgency + โ”‚ + โ–ผ Stage 4: Notification envelope generation + โ”‚ One envelope per audience member per delivery + โ”‚ Audience role set correctly (owner / stakeholder / approver / observer) + โ”‚ Stakeholder reason populated for non-owners + โ”‚ + โ–ผ Stage 5: Route to Notification Provider(s) + โ”‚ POST to provider delivery endpoint with notification envelope + โ”‚ Provider translates to delivery channel (Slack, PagerDuty, email, etc.) + โ”‚ Provider returns delivery_uuid and status + โ”‚ + โ–ผ Stage 6: Delivery confirmation + โ”‚ Provider reports: delivered | failed | queued + โ”‚ Delivery record written to Notification Delivery Store + โ”‚ NOTIFICATION_DISPATCHED audit record written (async) + โ”‚ + โ–ผ Stage 7: Failure handling + On provider delivery failure: + Retry per provider's declared retry policy + On exhaustion: dead_letter to platform admin + On critical urgency exhaustion: escalate immediately + NOTIFICATION_DELIVERY_FAILED audit record written +``` + +### 8.1 Notification Delivery Store + +A lightweight store (not the Audit Store) tracking delivery status per notification: + +```yaml +notification_delivery_record: + delivery_uuid: + notification_uuid: + actor_uuid: + notification_provider_uuid: + channel_type: slack + status: + dispatched_at: + delivered_at: + failure_reason: + retry_count: 2 +``` + +--- + +## 9. Provider Update Notification Integration + +Provider Update Notifications (doc 06, Section 7a) integrate with the notification model at two points: + +**When provider submits update notification:** +- `provider_update.submitted` fires โ†’ Owner notified (informational) + +**When provider update requires consumer approval:** +- `provider_update.requires_approval` fires โ†’ Owner notified (action required) +- `action.type: approve` +- `action.action_url` points to `/api/v1/resources/{uuid}/provider-notifications/{uuid}/approve` +- `action.deadline` set per policy (default PT24H โ€” if no response, escalate) + +**On resolution:** +- Approved โ†’ `provider_update.approved` fires โ†’ Owner notified; Stakeholders notified of state change via `entity.state_changed` +- Rejected โ†’ `provider_update.rejected` fires โ†’ Owner notified; becomes drift event โ†’ `drift.detected` fires + +--- + +## 10. Relationship to Webhooks and Message Bus + +### 10.1 Webhooks as a Notification Channel + +Outbound webhooks (doc 18) are now **one delivery channel type within the Notification Provider model** rather than a parallel mechanism. A Notification Provider with `channel_type: webhook` delivers notifications to configured HTTP endpoints using the unified notification envelope. + +The webhook registration model (doc 18, Section 3.2) is superseded for new implementations by actor-level subscriptions (Section 6.1, Tier 3) with a webhook-type Notification Provider. Existing webhook registrations remain supported via a compatibility layer. + +### 10.2 Message Bus as Notification Infrastructure + +The Message Bus Provider (doc 18, Section 5) is the **internal transport** for the notification pipeline. The Notification Router publishes notification events to the Message Bus. Notification Providers subscribe to their assigned topics. This decouples event generation from delivery and enables high-throughput notification processing. + +``` +DCM Event โ†’ Notification Router โ†’ Message Bus โ†’ Notification Provider subscription +``` + +The Message Bus is infrastructure โ€” not a notification channel. Consumers do not subscribe to the Message Bus directly for notifications; they use the subscription model (Section 6.1). + +--- + +## 11. System Policies + +| Policy | Rule | +|--------|------| +| `NOT-001` | The audience for every notification is derived from the entity relationship graph at event time. DCM does not maintain static subscriber lists per entity. | +| `NOT-002` | Mandatory system notifications (Tier 1: security, sovereignty, audit chain) are never suppressable by any subscription configuration or policy. | +| `NOT-003` | Cross-tenant notifications carry only information the receiving Tenant is authorized to see. Sovereignty checks apply to notification delivery endpoints. | +| `NOT-004` | Every notification dispatch is an audit record. Delivery failures are tracked. Critical urgency delivery exhaustion triggers immediate escalation to Platform Admin. | +| `NOT-005` | Provider Update Notifications that require consumer approval carry `action.type: approve` and `action.deadline`. If the deadline passes without resolution, the notification escalates per policy. | +| `NOT-006` | Notification traversal depth is bounded. Resource Type Specifications declare the maximum traversal depth for each event type. Default: depth 1 (direct relationships only). | +| `NOT-007` | A Notification Provider must be registered and active before notifications can be delivered. DCM does not have a built-in delivery channel โ€” at minimum a webhook-type Notification Provider must be configured for external delivery. | +| `NOT-008` | The notification event taxonomy is a closed vocabulary. Custom event types are not supported. New event types require a DCM registry proposal following standard governance. | + +--- + +## 12. Related Concepts + +- **Notification Provider** โ€” the ninth DCM provider type; handles translation and delivery +- **Notification Router** โ€” DCM control plane component that resolves audiences and routes to providers +- **Audience Resolution** โ€” deriving notification recipients from the entity relationship graph +- **Notification Subscription** โ€” actor or Tenant declaration of notification preferences +- **Notification Delivery Store** โ€” lightweight store tracking delivery status +- **Provider Update Notification** โ€” formal provider mechanism for reporting authorized state changes (see doc 06, Section 7a) +- **Outbound Webhook** โ€” one delivery channel type within the Notification Provider model + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/ownership-sharing-allocation.md b/content/docs/architecture/data-model/ownership-sharing-allocation.md new file mode 100644 index 0000000..145adc4 --- /dev/null +++ b/content/docs/architecture/data-model/ownership-sharing-allocation.md @@ -0,0 +1,302 @@ +--- +title: "Ownership, Sharing, and Allocation" +type: docs +weight: 5 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Entity Types](01-entity-types.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md) + +--- + +## 1. Purpose + +This document defines the complete ownership model for DCM entities โ€” specifically the three ownership patterns that govern how resources are owned, shared, and allocated across Tenants. It establishes precise vocabulary and clear boundaries between concepts that are frequently conflated: + +- **Ownership** โ€” who is accountable for a resource's lifecycle and costs +- **Shareable** โ€” multiple entities have a stake in a single resource that they do not own +- **Allocatable** โ€” a pool resource yields independently owned sub-resources to consumers + +Getting this model right is foundational. It governs decommission safety (can this resource be removed?), cost attribution (who pays for this?), cross-tenant visibility (who can see this?), drift accountability (whose responsibility is remediation?), and placement decisions (which providers serve allocation requests?). + +--- + +## 2. The Three Ownership Patterns + +### 2.1 Whole Allocation (Consumer Owns the Entity) + +**What it means:** The consumer receives the entire resource entity. It belongs exclusively to them. They own it outright โ€” it is in their Tenant, they control its lifecycle, they bear its costs. + +**Structural model:** +``` +Platform Tenant owns: the infrastructure, compute capacity, network fabric +Consumer Tenant owns: VirtualMachine-A (a distinct entity) +Consumer Tenant owns: VirtualMachine-B (another distinct entity) +``` +There is no relationship between VirtualMachine-A and the infrastructure Tenant โ€” the consumer simply used DCM's Service Provider to provision a resource. Once provisioned, the entity belongs to the consumer's Tenant entirely. The platform Tenant has no visibility into the consumer's entity unless an explicit Information Provider or cross-tenant relationship is established. + +**Examples:** VirtualMachine, Container, StorageVolume, NetworkInterface, DNSRecord. + +**Resource Type Spec declaration:** +```yaml +resource_type_spec: + fqn: Compute.VirtualMachine + ownership_model: whole_allocation + # Each consumer request produces an entity owned entirely by the requesting Tenant +``` + +**Decommission behavior:** Straightforward. The owning Tenant decommissions the entity. The provider releases the underlying physical resources. No other Tenant is affected. + +--- + +### 2.2 Allocation (Consumer Owns a Carved Portion) + +**What it means:** A *pool* resource is owned by a platform or provider Tenant (the pool owner). When a consumer requests a resource of this type, they receive an *allocation* โ€” a new, distinct entity carved from the pool. The consumer **owns** their allocation. The pool owner retains ownership of the pool. + +**The key distinction from Shareable:** The consumer's allocation is an independent entity with its own UUID, its own Tenant membership, its own lifecycle. It is not a reference to the pool โ€” it is a new thing that came from the pool. + +**Structural model:** +``` +NetworkOps Tenant owns: IPAddressPool 10.0.0.0/16 (pool entity) + โ”‚ + โ”œโ”€โ”€ AppTeam Tenant owns: IPAddress 10.0.1.45/32 โ† allocation entity (new UUID, AppTeam's Tenant) + โ”œโ”€โ”€ DevTeam Tenant owns: IPAddress 10.0.1.46/32 โ† allocation entity (new UUID, DevTeam's Tenant) + โ””โ”€โ”€ OpsTeam Tenant owns: IPAddress 10.0.1.47/32 โ† allocation entity (new UUID, OpsTeam's Tenant) +``` + +The IPAddress entities are not pointers to the pool โ€” they are real entities owned by their Tenants. If AppTeam decommissions 10.0.1.45/32, it is released back to the pool. The pool capacity increases. No other Tenant's allocation is affected. + +**Examples:** IPAddress (from IPAddressPool), Subnet (from SubnetPool), VLAN ID (from VLANIDPool), StorageVolume (from StoragePool), PublicCertificate (from CertificateAuthorityPool). + +**Resource Type Spec declarations (both pool and allocation):** +```yaml +# The pool resource type +resource_type_spec: + fqn: Network.IPAddressPool + ownership_model: whole_allocation # the pool entity is owned outright by the platform Tenant + is_pool: true + allocation_produces_type: Network.IPAddress + capacity_tracking: true # DCM tracks used/available capacity + +# The allocation resource type +resource_type_spec: + fqn: Network.IPAddress + ownership_model: allocation # each instance is an allocation from a pool + allocated_from_pool_type: Network.IPAddressPool + # When a consumer requests Network.IPAddress, DCM: + # 1. Runs placement to find an eligible IPAddressPool + # 2. The pool provider carves out a specific IP + # 3. DCM creates a new IPAddress entity owned by the requesting Tenant + # 4. Records an AllocationRecord relationship between the entity and the pool +``` + +**AllocationRecord relationship:** +Every allocation entity carries an `allocated_from` relationship to its source pool: + +```yaml +relationship: + relationship_uuid: + type: allocated_from + source_entity_uuid: # the allocation (e.g., IPAddress 10.0.1.45/32) + target_entity_uuid: # the pool (e.g., IPAddressPool 10.0.0.0/16) + source_tenant_uuid: # AppTeam Tenant + target_tenant_uuid: # NetworkOps Tenant โ€” cross-tenant relationship + allocation_ref: + allocation_size: "1/32" # what was carved from the pool + allocated_at: + allocated_by_actor_uuid: + allocation_metadata: + pool_capacity_before: 65534 + pool_capacity_after: 65533 +``` + +**Decommission behavior:** When the consumer decommissions their allocation entity, DCM dispatches a decommission payload to the provider. The provider releases the specific allocated resource back to the pool. The pool's available capacity increases. The AllocationRecord relationship is terminated. The allocation entity enters DECOMMISSIONED state. The pool entity is unaffected. + +**Cross-tenant visibility:** The allocation entity is in the consumer's Tenant. The consumer cannot see the pool entity unless an explicit cross-tenant relationship or Information Provider is configured. The pool owner can see allocation counts and capacity via the Cost Analysis component and the provider's capacity reporting API โ€” they cannot see the consumer's entity data. + +--- + +### 2.3 Shareable (Consumer Has a Stake, Not Ownership) + +**What it means:** A single resource entity is owned by one Tenant (the resource owner) and multiple consumers attach to, depend on, or reference it. Consumers have a *stake* โ€” a relationship that affects the resource's lifecycle โ€” but they do not own any portion of it. The resource's lifecycle is governed entirely by its owner. + +**The key distinction from Allocation:** No new entity is created for the consumer. The consumer receives a relationship to the existing resource, not a new sub-entity. The consumer does not own anything โ€” they hold a stake. + +**Structural model:** +``` +NetworkOps Tenant owns: VLAN-100 (single entity โ€” there is only one VLAN-100) + โ”‚ + โ”œโ”€โ”€ AppTeam has stake: VM-A attached to VLAN-100 (relationship, not ownership) + โ”œโ”€โ”€ DevTeam has stake: VM-B attached to VLAN-100 (relationship, not ownership) + โ””โ”€โ”€ OpsTeam has stake: VM-C attached to VLAN-100 (relationship, not ownership) +``` + +VLAN-100 belongs to NetworkOps. AppTeam, DevTeam, and OpsTeam each have a VM attached to it. If DevTeam decommissions VM-B, VLAN-100 is unaffected โ€” it still exists and serves VM-A and VM-C. If NetworkOps wants to decommission VLAN-100, they cannot do so while VMs are attached. Decommission is deferred until all stakes are released. + +**Examples:** VLAN (network fabric shared by many VMs), NetworkSegment, SharedFileSystem, DNS Zone, NTP Server, Certificate Authority (as a service), Transit Gateway. + +**Resource Type Spec declaration:** +```yaml +resource_type_spec: + fqn: Network.VLAN + ownership_model: shareable + # A single VLAN entity exists; consumers attach to it via relationships + # Consumers do not receive their own VLAN entity + decommission_policy: + defer_while_active_stakes: true + minimum_stake_count: 0 # can decommission when all stakes released + # Some resources may require minimum_stake_count: 1 + # e.g., a DNS Zone that should never be empty +``` + +**Stake relationship:** +```yaml +relationship: + relationship_uuid: + type: attached_to # or: depends_on, references, uses + source_entity_uuid: # VM-A (consumer's entity) + target_entity_uuid: # VLAN-100 (shared resource) + source_tenant_uuid: # AppTeam Tenant + target_tenant_uuid: # NetworkOps Tenant + stake: + is_active: true + staked_at: + staked_by_actor_uuid: + stake_strength: + # required: VM cannot function without VLAN attachment (blocks VLAN decommission) + # preferred: VM prefers attachment but can function without it + # optional: informational stake only +``` + +**Decommission behavior:** VLAN-100 cannot be decommissioned while any `stake_strength: required` stakes exist. The decommission attempt is deferred โ€” not rejected โ€” and a `DECOMMISSION_DEFERRED` event is generated. DCM notifies the resource owner of all active required stakes. The owner can request that stakeholders release their stakes (by decommissioning their VMs or migrating to a different VLAN) before decommission proceeds. + +**Cross-tenant visibility:** The consumer's VM can reference VLAN-100 (read access for attachment purposes). The consumer cannot modify VLAN-100 or see its owner's configuration details unless an explicit cross-tenant authorization grants that. The resource owner (NetworkOps) can see all active stakes on their resource โ€” this is how they know which consumers are affected by a planned decommission. + +--- + +## 3. Hybrid Case โ€” Allocation from a Shareable Pool + +Some resources combine both patterns. A Subnet Pool is an allocatable pool. Each allocation (a specific /28) is owned by the consumer. But the /28 has a stake relationship to the parent /16 (which is shareable โ€” owned by the network team, referenced by all subnets). + +``` +NetworkOps Tenant owns: SupernetPool 10.0.0.0/8 (allocatable pool) + โ”‚ + โ”œโ”€โ”€ NetworkOps Tenant owns: 10.0.0.0/16 (allocation from /8 โ€” NetworkOps-owned) + โ”‚ NetworkOps Tenant owns: 10.0.0.0/24 (allocation from /16 โ€” NetworkOps-owned) + โ”‚ + โ””โ”€โ”€ NetworkOps Tenant owns: 10.1.0.0/16 (allocation from /8 โ€” NetworkOps-owned, shared) + โ”œโ”€โ”€ AppTeam Tenant owns: 10.1.0.0/24 โ† consumer allocation (owned by AppTeam) + โ”‚ โ””โ”€โ”€ stake: attached to 10.1.0.0/16 (shareable โ€” NetworkOps) + โ””โ”€โ”€ DevTeam Tenant owns: 10.1.1.0/24 โ† consumer allocation (owned by DevTeam) + โ””โ”€โ”€ stake: attached to 10.1.0.0/16 (shareable โ€” NetworkOps) +``` + +The consumer-facing 10.1.0.0/24 is an allocation โ€” AppTeam owns it. The parent 10.1.0.0/16 is shareable โ€” NetworkOps owns it, AppTeam and DevTeam both have stakes. The 10.1.0.0/24 has both an `allocated_from` relationship (to the /24 pool that produced it) and an `attached_to` relationship (stake in the parent /16). + +--- + +## 4. Ownership Model Summary + +| Pattern | Consumer Gets | Consumer Owns | New Entity Created | Lifecycle Governed By | Cost Attribution | +|---------|--------------|--------------|-------------------|----------------------|-----------------| +| **Whole Allocation** | The entire resource | Yes, outright | Yes (same type) | Consumer Tenant | Consumer Tenant | +| **Allocation** | A carved sub-resource | Yes, the allocation | Yes (sub-type) | Consumer Tenant | Consumer Tenant | +| **Shareable** | A stake/relationship | No โ€” stake only | No new entity | Resource Owner Tenant | Resource Owner Tenant (shared cost attribution possible via policy) | + +--- + +## 5. Placement Engine Interaction + +The placement engine handles all three ownership models, but the selection criteria differ: + +**Whole Allocation:** Standard placement. The placement engine selects a provider with available capacity. The provider provisions the resource and returns it owned by the requesting Tenant. + +**Allocation:** The placement engine selects an eligible pool resource owned by a platform Tenant with sufficient available capacity. The pool provider carves an allocation and returns it. DCM creates the new allocation entity in the requesting Tenant. + +```yaml +# Placement engine for allocation requests: +# Step 1: Find providers that offer Network.IPAddressPool +# Step 2: Filter by sovereignty constraints +# Step 3: Filter by available capacity (pool.available_count > 0) +# Step 4: Apply tie-breaking hierarchy +# Step 5: Dispatch allocation request to selected pool provider +# Step 6: Provider returns the specific carved allocation +# Step 7: DCM creates IPAddress entity in requesting Tenant +# Step 8: AllocationRecord relationship created +``` + +**Shareable:** The placement engine finds the shareable resource instance that satisfies the consumer's attachment constraints. No new entity is provisioned โ€” the provider registers the stake relationship. If no eligible shareable instance exists, the request fails with a clear error (unlike Allocation where failure means insufficient pool capacity). + +--- + +## 6. Decommission Safety Model + +The three patterns have different decommission safety behaviors: + +**Whole Allocation decommission:** +- Owner Tenant initiates decommission +- Policy checks for required relationships (do other entities depend on this?) +- If required dependencies exist โ†’ `DECOMMISSION_DEFERRED` until dependencies release +- If no required dependencies โ†’ dispatch decommission to provider โ†’ DECOMMISSIONED + +**Allocation decommission:** +- Consumer Tenant initiates decommission of their allocation entity +- DCM dispatches decommission to pool provider +- Pool provider releases the resource back to pool +- AllocationRecord relationship terminated +- Pool `available_count` increases +- Allocation entity โ†’ DECOMMISSIONED +- Pool entity is unaffected + +**Shareable decommission:** +- Resource owner Tenant initiates decommission of the shared resource +- DCM checks `active_stake_count` for `stake_strength: required` stakes +- If required stakes > 0 โ†’ `DECOMMISSION_DEFERRED` + - Notifications to all required stakeholders + - Owner retries decommission after stakeholders release +- If required stakes == 0 โ†’ dispatch decommission to provider โ†’ DECOMMISSIONED + - Any remaining `optional` stakes are automatically terminated + +--- + +## 7. Cost Attribution Model + +**Whole Allocation and Allocation:** Cost is attributed entirely to the owning Tenant. Standard Cost Analysis billing. The entity's `billing_state` governs the rate (billable/non_billable/reduced_rate). + +**Shareable:** The shared resource's cost is attributed to its owner Tenant by default. Organizations that want to distribute shared resource costs to stakeholders configure a cost attribution policy: + +```yaml +cost_attribution_policy: + resource_type: Network.VLAN + model: + # owner_bears_all: NetworkOps Tenant pays for VLAN regardless of how many VMs attach + # equal_split: cost divided equally among active stakeholders + # proportional_by_usage: cost allocated by traffic volume or similar metric + # chargeback: each stakeholder is invoiced for their declared portion +``` + +--- + +## 8. System Policies + +| Policy | Rule | +|--------|------| +| `OWN-001` | `whole_allocation` resources are owned entirely by the requesting Tenant from the moment of realization. The providing platform Tenant has no ownership claim. | +| `OWN-002` | `allocation` resources are owned entirely by the requesting Tenant. The pool owner retains ownership of the pool entity only. AllocationRecord relationships are the only cross-Tenant link. | +| `OWN-003` | `shareable` resources are owned by a single Tenant. Consumers hold stakes (relationships) only. No consumer owns any portion of a shareable resource. | +| `OWN-004` | Decommission of a shareable resource is deferred while any `required` strength stakes are active. Optional stakes are terminated automatically on shareable resource decommission. | +| `OWN-005` | Allocation entity decommission releases the allocation back to the source pool. The pool entity is never decommissioned by an allocation decommission. | +| `OWN-006` | Cost attribution for shareable resources defaults to the resource owner Tenant. A cost attribution policy may redistribute costs to stakeholders. | +| `OWN-007` | The ownership model for a resource type is declared in the Resource Type Specification and cannot be changed at the entity level. Ownership model is a type-level invariant. | +| `OWN-008` | Cross-tenant AllocationRecord and stake relationships require that the requesting Tenant has either an active cross-tenant authorization or the resource type is declared as publicly allocatable/stakeable in its Resource Type Spec. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md index d406190..113ecdf 100644 --- a/content/docs/architecture/data-model/resource-service-entities.md +++ b/content/docs/architecture/data-model/resource-service-entities.md @@ -1,7 +1,7 @@ --- title: "Resource and Service Entities" type: docs -weight: 5 +weight: 6 --- > **โš ๏ธ Active Development Notice** @@ -12,7 +12,7 @@ weight: 5 **Document Status:** ๐Ÿ”„ In Progress -**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) --- @@ -406,6 +406,157 @@ dcm_capacity_rating: --- + +--- + +## 7a. Provider Update Notification Model + +### 7a.1 The Fundamental Constraint โ€” Realized State Only Changes via a Request + +DCM enforces a single foundational rule for the Realized Store: + +> **Realized State only changes when an authorized request produces a corresponding Requested State record. No exceptions.** + +This constraint unifies all state change pathways and eliminates ambiguity: + +- **Drift is always unsanctioned** โ€” if Discovered State differs from Realized State and there is no corresponding Requested State record explaining the difference, it is drift. There is no such thing as "legitimate drift." +- **Discovery does not update Realized State** โ€” discovery writes only to the Discovered Store. It never updates the Realized Store, even if discovery shows an authorized change (the authorization produces its own Requested State and Realized State records). +- **Providers cannot write directly to Realized State** โ€” providers report changes via the Provider Update Notification API. DCM evaluates the notification and creates a Requested State record if approved. Only then does a new Realized State record get written. + +### 7a.2 Provider Update Notification + +A **Provider Update Notification** is a formal mechanism by which a Service Provider reports an authorized state change to DCM. This is distinct from a lifecycle event (which reports provider health) and distinct from an unsanctioned change (which triggers drift). A Provider Update Notification is the provider saying: "I made an authorized change to this entity โ€” please record it as the new Realized State." + +**When is a Provider Update Notification appropriate:** + +| Scenario | Correct mechanism | Why | +|----------|------------------|-----| +| Provider auto-heals a failed disk | Provider Update Notification | Authorized maintenance action; new disk is the correct state | +| Provider scales resources per pre-authorized auto-scale policy | Provider Update Notification | DCM pre-authorized the scaling policy; each scaling event is an authorized change | +| Provider performs planned maintenance that changes an IP assignment | Provider Update Notification | Planned, coordinated change | +| Unauthorized human modifies VM configuration at provider console | Drift event | No DCM authorization; treated as unsanctioned change | +| Provider silently changes configuration without notifying DCM | Drift event (detected by discovery) | Unreported change is unsanctioned until evaluated | + +### 7a.3 Provider Update Notification API + +Service Providers submit update notifications via a dedicated endpoint on the DCM API Gateway: + +``` +POST /api/v1/provider/entities/{entity_uuid}/update-notification +Authorization: + +Request body: +{ + "provider_uuid": "", + "notification_uuid": "", # idempotency key + "notification_type": "", + "changed_fields": { + "memory_gb": { + "previous_value": 8, + "new_value": 16, + "change_reason": "Auto-scale policy: payments-api-scale-up triggered at 85% memory utilization", + "authorizing_policy_ref": "" + } + }, + "effective_at": "", + "provider_evidence_ref": "" +} +``` + +### 7a.4 DCM Processing of Provider Update Notifications + +``` +Provider submits update notification + โ”‚ + โ–ผ Authentication and authorization check + โ”‚ Verify: provider UUID is registered and active + โ”‚ Verify: provider has authority over this entity + โ”‚ + โ–ผ Policy Engine evaluates notification + โ”‚ Evaluate: is this type of change pre-authorized for this entity/provider? + โ”‚ Evaluate: does the change violate any GateKeeper constraints? + โ”‚ Evaluate: does this change require consumer notification or approval? + โ”‚ + โ”œโ”€โ”€ REJECTED + โ”‚ The change is not authorized. + โ”‚ DCM does NOT update Realized State. + โ”‚ The discrepancy between provider state and DCM Realized State becomes drift. + โ”‚ Provider receives rejection response with reason. + โ”‚ UNSANCTIONED_CHANGE event logged. + โ”‚ + โ”œโ”€โ”€ REQUIRES_CONSUMER_APPROVAL + โ”‚ The change is plausible but requires consumer sign-off. + โ”‚ Notification queued. Consumer notified. + โ”‚ Entity enters PENDING_REVIEW state. + โ”‚ Provider receives "pending_approval" response. + โ”‚ On consumer approval โ†’ proceeds to APPROVED path. + โ”‚ On consumer rejection โ†’ treated as REJECTED. + โ”‚ + โ””โ”€โ”€ APPROVED + DCM creates a Requested State record: + source_type: provider_update + actor: provider (service account) + authorizing_policy_uuid: + changed_fields: [as reported by provider] + + DCM writes new Realized State snapshot: + source_type: provider_update + corresponding_requested_state_uuid: + supersedes_realized_state_uuid: + + Audit record written: PROVIDER_UPDATE_APPLIED + Provider receives "accepted" response. +``` + +### 7a.5 Pre-Authorization of Provider Updates + +Organizations can pre-authorize categories of provider updates through policy, eliminating the need for per-change human approval: + +```yaml +policy: + type: gatekeeper + handle: "tenant/payments/allow-auto-scale" + rules: + - condition: + notification_type: auto_scale + provider_uuid: + entity.resource_type: Compute.VirtualMachine + changed_fields: [memory_gb, cpu_count] + change_within_bounds: + memory_gb: { max_increase_factor: 2 } + cpu_count: { max_increase_factor: 2 } + action: approve + audit_note: "Auto-scale approved per payments team scaling policy" +``` + +This pre-authorization pattern allows providers to implement auto-scaling, auto-healing, and maintenance operations without requiring per-change manual approval, while keeping DCM's Realized Store accurate and traceable. + +### 7a.6 Updated Provider Lifecycle Events Table + +The following table supersedes the table in Section 7.2 with clearer DCM response categorization: + +| Event Type | Mechanism | DCM Response | Realized Store Updated? | +|------------|-----------|-------------|------------------------| +| `CAPACITY_CHANGE` | Lifecycle event | Update internal capacity rating | No | +| `DEGRADATION` | Lifecycle event | Policy Engine โ†’ ALERT/ESCALATE | No | +| `MAINTENANCE_SCHEDULED` | Lifecycle event | Notify, plan migration if needed | No | +| `MAINTENANCE_CHANGE` | **Provider Update Notification** | Evaluate โ†’ Requested State if approved | Yes (if approved) | +| `AUTO_SCALE` | **Provider Update Notification** | Evaluate per pre-auth policy โ†’ Requested State if approved | Yes (if approved) | +| `AUTO_HEAL` | **Provider Update Notification** | Evaluate per pre-auth policy โ†’ Requested State if approved | Yes (if approved) | +| `UNSANCTIONED_CHANGE` | Lifecycle event (no notification) | Drift event โ†’ Policy Engine โ†’ REVERT/ALERT/ESCALATE | No (drift, not update) | +| `ENTITY_HEALTH_CHANGE` | Lifecycle event | Policy Engine evaluation | No | +| `DECOMMISSION_NOTICE` | Lifecycle event | Policy Engine โ†’ migrate or decommission | No | + +### 7a.7 System Policies + +| Policy | Rule | +|--------|------| +| `RSE-010` | Realized State only changes via an authorized request that produces a corresponding Requested State record. Drift detection, discovery cycles, and lifecycle events do not write to the Realized Store. | +| `RSE-011` | Provider Update Notifications are evaluated by the Policy Engine before any Realized State change. Rejected notifications do not update Realized State โ€” the discrepancy becomes drift. | +| `RSE-012` | Categories of provider updates may be pre-authorized via GateKeeper policy. Pre-authorized updates are processed automatically without per-change human approval. | +| `RSE-013` | Provider Update Notifications that require consumer approval place the entity in PENDING_REVIEW state. The provider receives a "pending_approval" response and the change is queued until resolution. | + + ## 8. Entity Relationships Every Resource/Service Entity carries a `relationships` section declaring its relationships to other entities โ€” internal DCM entities, external data entities, and business context entities. The relationship model is universal โ€” the same structure is used for all relationship types. diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md index 6143b73..6114bbf 100644 --- a/content/docs/architecture/data-model/storage-providers.md +++ b/content/docs/architecture/data-model/storage-providers.md @@ -47,7 +47,7 @@ storage_provider_registration: uuid: name: display_name: - store_type: + store_type: version: endpoint: capabilities: @@ -210,6 +210,103 @@ event_envelope: --- + +--- + +## 5a. Write-once Snapshot Store Contract (Realized Store) + +The Realized Store is a **write-once snapshot store** โ€” distinct from the Event Stream Store used for Discovered State. It holds complete entity state snapshots where every record is traceable to an authorized DCM request. + +### 5a.1 Store Characteristics + +The Realized Store occupies a middle ground between the GitOps store (structured, human-navigable, PR-mediated) and the Event Stream store (high-frequency, field-level events, ephemeral). The Realized Store is: + +- **Write-once** โ€” records are immutable after creation; a new record is created for each authorized state change +- **Snapshot-based** โ€” each record is a complete entity state, not a field-level delta +- **Request-traceable** โ€” every record has a non-nullable `corresponding_requested_state_uuid` +- **Moderate frequency** โ€” written only on authorized changes (initial realization, consumer updates, approved provider updates), not on every event +- **Long-lived** โ€” records persist for the full entity lifetime plus audit retention period + +### 5a.2 Write Authorization Model + +The Realized Store has a strictly controlled set of write sources. No component may write to the Realized Store without a corresponding Requested State record: + +```yaml +realized_store_write_authorization: + allowed_sources: + - source_type: initial_realization + trigger: provider_confirms_realization + required: corresponding_requested_state_uuid (type: initial_realization) + + - source_type: consumer_update + trigger: provider_confirms_targeted_delta + required: corresponding_requested_state_uuid (type: consumer_update) + + - source_type: provider_update + trigger: dcm_approves_provider_update_notification + required: corresponding_requested_state_uuid (type: provider_update) + + explicitly_forbidden: + - drift_detection # drift only reads, never writes + - discovery_cycles # discovery writes to Discovered Store only + - unsanctioned_changes # unsanctioned changes remain drift events + - direct_admin_writes # no bypassing the request pipeline +``` + +**Enforcement:** The write authorization model is enforced at the Realized Store API level โ€” not by convention. A write without a valid `corresponding_requested_state_uuid` is rejected with `401 Unauthorized`. + +### 5a.3 Required Capabilities + +```yaml +write_once_snapshot_store: + write_once_enforcement: true # writes without corresponding_requested_state_uuid rejected + entity_uuid_keyed: true # O(1) lookup by entity UUID + snapshot_retention: per_entity # all snapshots for an entity retained per retention policy + point_in_time_query: true # query state as of any timestamp + supersession_chain: true # each record knows predecessor and successor + hash_chain_integrity: true # each record hashes previous โ€” tamper evident + concurrent_write_handling: optimistic_lock # retry on conflict, no silent overwrite +``` + +### 5a.4 Required API Operations + +| Operation | Description | +|-----------|-------------| +| `write_snapshot(entity_uuid, payload, requested_state_uuid)` | Write new snapshot; validate non-null requested_state_uuid; return snapshot UUID | +| `get_current(entity_uuid)` | Return the most recent snapshot for entity | +| `get_at_timestamp(entity_uuid, timestamp)` | Return snapshot valid at given timestamp | +| `get_by_uuid(realized_state_uuid)` | Return specific snapshot | +| `list_history(entity_uuid)` | Return supersession chain for entity | +| `verify_chain(entity_uuid)` | Verify hash chain integrity for entity's snapshots | + +### 5a.5 Retention Policy + +Realized State snapshots are retained for the full entity lifecycle. After an entity is DECOMMISSIONED, snapshots are retained per the audit retention policy โ€” the same policy that governs Audit Store records. The entity's final Realized State snapshot is preserved even after all preceding snapshots are archived. + +### 5a.6 Relationship to the Audit Store + +The Realized Store and Audit Store are complementary โ€” not redundant: + +| Aspect | Realized Store | Audit Store | +|--------|---------------|-------------| +| Content | Complete entity state snapshots | Atomic action records (who did what when) | +| Query pattern | "What was the state of X at time T?" | "Who changed X and why?" | +| Write frequency | Per authorized change | Per every DCM action | +| Hash chain | Per entity | Per instance | +| Rehydration source | Yes | No | + +The Audit Store records the action. The Realized Store records the result. Both are required for complete auditability. + +### 5a.7 Typical Implementations + +| Scale | Implementation | Notes | +|-------|---------------|-------| +| Minimal / dev | SQLite or PostgreSQL single-instance | Simple; acceptable for evaluation | +| Standard | PostgreSQL with write-once constraints | Reliable; familiar operational model | +| Production | CockroachDB or PostgreSQL HA | Geo-distributed; strong consistency | +| FSI / Sovereign | PostgreSQL with HSM-backed encryption | Encryption at rest required | + + ## 6. Search Index Contract Used for: Queryable projection of GitOps stores @@ -420,7 +517,8 @@ storage_failure_policy: **By store type:** - **Commit Log:** Quorum unavailable โ†’ operation aborted. Minority failure โ†’ continues via Raft reelection. - **GitOps Stores:** Unavailable โ†’ writes queue locally; reads serve from cache. Queue exhausted โ†’ explicit rejection. -- **Event Stream:** Producer queues locally. Consumer resumes from last committed offset on recovery. No data loss. +- **Event Stream (Discovered Store):** Producer queues locally. Consumer resumes from last committed offset on recovery. No data loss. +- **Write-once Snapshot Store (Realized Store):** Writes queue locally with the same WAL pattern as the Audit Store. Write retry on recovery. Rejected writes (missing requested_state_uuid) are never retried โ€” they are logged as security events. - **Audit Store:** Two-stage model โ€” Commit Log accumulates `pending_forward` entries. Operations not blocked. - **Search Index:** Non-authoritative. Unavailable โ†’ degraded response + reference to authoritative store. Recovery โ†’ full index rebuild. @@ -673,6 +771,8 @@ Full audit trail: sovereignty_violation_record links to migration request | `STO-001` | Storage Providers must declare replication capabilities. Active Profile determines minimum replication requirements. Providers not meeting Profile minimum cannot be activated for that Profile's stores. | | `STO-002` | Storage Provider failure behavior is declared per store type and governed by the active Profile. GitOps unavailability queues writes locally โ€” does not silently drop. Commit Log quorum loss aborts the triggering operation. Audit Store unavailability accumulates entries in the Commit Log. Search Index unavailability degrades query responses without impacting write operations. | | `STO-003` | The Search Index is a separate Storage Provider sub-type โ€” non-authoritative and rebuildable. API queries may specify `freshness: authoritative` to bypass the index. | +| `STO-007` | The Realized Store is a write-once snapshot store. Every write requires a non-nullable corresponding_requested_state_uuid. Drift detection, discovery cycles, and unsanctioned provider changes do not write to the Realized Store. Enforcement is at the store API level, not by convention. | +| `STO-008` | The Intent Store requires a GitOps implementation. The Requested Store requires write-once semantics; GitOps is the reference implementation; write-once document stores are supported for production scale. | | `STO-004` | The Audit Store is a specialized Storage Provider sub-type โ€” append-only, hash chain integrity, reference-based retention, compliance-grade queries. The Event Stream is the delivery channel only. | | `STO-005` | GitOps stores use a handle-based directory structure. The main branch is authoritative. Minimal and dev profiles may use a monorepo; standard and above should use separate repositories per store type. | diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md index 2855a08..176be3c 100644 --- a/content/docs/architecture/data-model/webhooks-messaging.md +++ b/content/docs/architecture/data-model/webhooks-messaging.md @@ -1,7 +1,7 @@ --- title: "Webhooks, Messaging, and External Integration" type: docs -weight: 17 +weight: 18 --- > **โš ๏ธ Active Development Notice** @@ -10,7 +10,8 @@ weight: 17 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** ๐Ÿ”„ In Progress +**Related Documents (updated):** [Notification Model](23-notification-model.md) | [Entity Relationships](09-entity-relationships.md) **Related Documents:** [Universal Audit Model](16-universal-audit.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Authentication and Authorization](19-auth-providers.md) | [Policy Organization](14-policy-profiles.md) --- @@ -162,6 +163,16 @@ policy: "If ingress.surface == message_bus_inbound AND message_bus_provider.juri ## 3. Outbound Webhooks +> **โš ๏ธ Architecture Update โ€” Notification Model Supersedes Outbound Webhooks** +> +> The outbound webhook model described in Section 3 has been superseded by the **Unified Notification Model** (see [doc 23: Notification Model](23-notification-model.md)). Outbound webhooks are now one delivery channel type within the Notification Provider model rather than a parallel mechanism. +> +> **For new implementations:** Use the Notification Provider subscription model (doc 23, Section 6) with a webhook-type Notification Provider. +> +> **For existing webhook registrations:** The registration model below remains supported via a compatibility layer. Existing registrations are automatically treated as actor-level subscriptions with a webhook-type Notification Provider. No migration required for current deployments. +> +> The key improvement in the new model: audience is derived from the **entity relationship graph**, not from a manually maintained subscriber list. A webhook subscription for VLAN drift events will now automatically include all VMs attached to that VLAN as audience context. + ### 3.1 Concept DCM pushes event notifications to registered external HTTP endpoints. Outbound webhooks are **optional and policy-governed** โ€” the active Profile and Policy Groups determine which events require external notification. @@ -246,6 +257,8 @@ webhook_registration: ### 3.3 Event Taxonomy +> **See [doc 23: Notification Model](23-notification-model.md) Section 4 for the authoritative event taxonomy.** The table below is the legacy taxonomy for existing webhook registrations. + The event taxonomy maps onto the Universal Audit action vocabulary. All are versioned registry entries: | Category | Events | diff --git a/content/docs/architecture/specifications/_index.md b/content/docs/architecture/specifications/_index.md index 79c3bf7..76f9d0e 100644 --- a/content/docs/architecture/specifications/_index.md +++ b/content/docs/architecture/specifications/_index.md @@ -17,3 +17,5 @@ superset strategy, and community engagement. {{< card link="operator-sdk-api" title="Operator SDK API" icon="code" subtitle="Go library API design for the DCM Operator SDK โ€” Level 1 conformance in one day." >}} {{< card link="cncf-strategy" title="CNCF Strategy" icon="globe-alt" subtitle="CNCF submission strategy, community engagement plan, and operator ecosystem approach." >}} {{< /cards >}} + +{{< card link="consumer-api-spec" title="Consumer API Specification" icon="code" subtitle="REST API specification for DCM consumers โ€” service catalog, request submission, resource management, and audit access." >}} diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md new file mode 100644 index 0000000..fb0c8fb --- /dev/null +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -0,0 +1,778 @@ +--- +title: "DCM Consumer API Specification" +type: docs +weight: 0 +--- + +> **โš ๏ธ Work in Progress** +> +> **This specification is a work in progress and is less mature than the core DCM data model documentation.** API endpoint paths, request/response structures, and authentication flows represent design intent and will be refined as implementation proceeds. +> +> **Do not build against this specification yet.** It is published to share design direction and invite feedback. +> +> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). + +**Version:** 0.1.0-draft +**Status:** Design โ€” Not yet implemented +**Document Type:** Technical Specification +**Related Documents:** [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Four States](../data-model/02-four-states.md) | [Auth Providers](../data-model/19-auth-providers.md) | [Webhooks and Messaging](../data-model/18-webhooks-messaging.md) + +--- + +## Abstract + +This specification defines the interface by which consumers interact with the DCM Control Plane. It is the counterpart to the [Operator Interface Specification](dcm-operator-interface-spec.md), which covers what Service Providers implement. This specification covers what consumers call. + +The Consumer API is the boundary between the Application domain and the Control Plane. It exposes DCM's service catalog, request submission, resource management, and audit capabilities in a unified interface. All operations are authenticated, authorized against the actor's Tenant scope, and fully audited. + +--- + +## 1. Introduction + +### 1.1 Scope + +This specification covers: +- Authentication and session management +- Service Catalog browsing and discovery +- Resource request submission (all ingress paths) +- Resource lifecycle management (updates, suspension, decommission, rehydration) +- Request and resource status +- Audit trail access + +It does not cover: +- Platform administration operations (covered by future Admin API spec) +- Service Provider registration and management (covered by Operator Interface Spec) +- Webhook and Message Bus subscription management (covered by doc 18) + +### 1.2 Ingress Surfaces + +The Consumer API is accessible via three ingress surfaces. All three are authenticated. All three run the same governance pipeline. The ingress surface affects the review workflow, never governance. + +| Surface | Protocol | Review Model | Use Case | +|---------|----------|-------------|----------| +| **REST API** | HTTPS REST | Synchronous acknowledgment; async realization | Programmatic consumers, automation, Terraform providers | +| **Web UI** | Browser | Interactive PR-like review flow | Human consumers, Service Catalog browsing | +| **Git PR Ingress** | Git + webhook | Full GitOps PR workflow | GitOps-native teams, infrastructure-as-code workflows | + +This specification primarily documents the REST API surface. The Git PR ingress YAML structure is documented in [Worked Examples](../data-model/04-examples.md), Section 2. + +### 1.3 Base URL and Versioning + +``` +https://{dcm-instance}/api/v1/ +``` + +All Consumer API endpoints are versioned. Breaking changes increment the version. Non-breaking additions do not. + +### 1.4 Content Type + +All requests and responses use `application/json`. The DCM Unified Data Model is expressed as JSON throughout the Consumer API. + +--- + +## 2. Authentication + +### 2.1 Token Acquisition + +Consumers obtain a session token from the Auth Provider. The token acquisition method depends on the configured Auth Provider: + +``` +POST /api/v1/auth/token + +# OIDC flow (most common): +{ + "grant_type": "authorization_code", + "code": "", + "redirect_uri": "" +} + +# API key flow (service accounts): +{ + "grant_type": "api_key", + "api_key": "" +} + +# Response: +{ + "token": "", + "token_type": "Bearer", + "expires_at": "", + "actor_uuid": "", + "mfa_verified": true, + "scopes": ["read:catalog", "request:compute", "manage:owned"] +} +``` + +### 2.2 Request Authentication + +All requests carry the session token as a Bearer token: + +``` +Authorization: Bearer +``` + +### 2.3 Tenant Context + +Actors may have access to multiple Tenants. The Tenant context for a request is declared in the header: + +``` +X-DCM-Tenant: +``` + +If omitted and the actor has access to exactly one Tenant, that Tenant is used. If the actor has access to multiple Tenants and no Tenant header is provided, the request is rejected with `400 Bad Request` โ€” Tenant ambiguity is never resolved silently. + +### 2.4 Step-Up MFA + +Some operations require step-up MFA regardless of session MFA status. When a step-up challenge is required, the API returns `403 Forbidden` with a challenge token: + +```json +{ + "error": "step_up_required", + "challenge_token": "", + "challenge_expires_at": "", + "allowed_methods": ["totp", "push_notification"] +} +``` + +The consumer completes the MFA challenge and retries the request with the completed challenge token: + +``` +X-DCM-StepUp-Token: +``` + +--- + +## 3. Service Catalog + +### 3.1 List Catalog Items + +Returns catalog items available to the authenticated actor in their Tenant, filtered by RBAC. + +``` +GET /api/v1/catalog + +Query parameters: + category= e.g., Compute, Network, Storage + search= full-text search across name and description + tag= filter by tag (repeatable) + page= pagination (default: 1) + page_size= results per page (default: 25, max: 100) + +Response 200: +{ + "catalog_items": [ + { + "catalog_item_uuid": "", + "resource_type": "Compute.VirtualMachine", + "provider_uuid": "", + "display_name": "Standard Linux VM", + "description": "General-purpose virtual machine with standard OS images", + "tier": 1, + "portability_class": "portable", + "estimated_cost": { + "unit": "per-hour", + "amount": 0.32, + "currency": "USD", + "cost_confidence": "high" + }, + "tags": ["compute", "linux", "general-purpose"], + "deprecated": false + } + ], + "total": 47, + "page": 1, + "page_size": 25 +} +``` + +### 3.2 Describe Catalog Item + +Returns the full schema for a catalog item โ€” all fields, constraints, editability declarations, dependencies, and cost estimate. + +``` +GET /api/v1/catalog/{catalog_item_uuid} + +Response 200: +{ + "catalog_item_uuid": "", + "resource_type": "Compute.VirtualMachine", + "resource_type_spec_version": "2.1.0", + "display_name": "Standard Linux VM", + + "schema": { + "fields": [ + { + "field_name": "cpu_count", + "type": "integer", + "required": true, + "editable_post_realization": false, + "constraint": { + "visibility": "full", # full | summary | hidden + "type": "range", + "min": 1, + "max": 32, + "allowed_values": [1, 2, 4, 8, 16, 32], + "reason": "CPU counts must be powers of 2 for NUMA alignment" + } + }, + { + "field_name": "memory_gb", + "type": "integer", + "required": true, + "editable_post_realization": false, + "constraint": { + "visibility": "full", + "type": "range", + "min": 2, + "max": 256 + } + }, + { + "field_name": "monitoring_agent", + "type": "string", + "required": false, + "editable_post_realization": false, + "constraint": { + "visibility": "hidden", # injected by policy โ€” consumer cannot set + "override": "immutable" + } + } + ] + }, + + "dependencies": [ + { + "resource_type": "Network.IPAddress", + "relationship": "requires", + "fulfillment": "automatic", # DCM auto-allocates; consumer does not need to request separately + "count": 1 + } + ], + + "estimated_cost": { + "breakdown": [ + { "component": "compute", "unit": "per-hour", "amount": 0.28, "currency": "USD" }, + { "component": "ip-allocation", "unit": "per-hour", "amount": 0.04, "currency": "USD" } + ], + "total_per_hour": 0.32, + "currency": "USD" + }, + + "sovereignty": { + "available_in_regions": ["EU-WEST", "EU-NORTH"], + "data_residency_guarantee": "EU" + } +} +``` + +### 3.3 Catalog Search + +``` +GET /api/v1/catalog/search?q= + +Returns catalog items matching the query across name, description, resource type, and tags. +Same response shape as List Catalog Items. +``` + +--- + +## 4. Request Submission + +### 4.1 Submit Resource Request + +Submits a new resource request. Returns immediately with an acknowledgment โ€” realization is asynchronous. + +``` +POST /api/v1/requests + +Request body: +{ + "catalog_item_uuid": "", + "fields": { + "cpu_count": 4, + "memory_gb": 8, + "storage_gb": 100, + "os_family": "rhel", + "name": "payments-api-server-01" + }, + "options": { + "auto_approve": true, # request auto-approval if policy permits + "notify_on_completion": true, + "notification_endpoint": "https://my-system.example.com/dcm/webhook" + } +} + +Response 202 Accepted: +{ + "request_uuid": "", + "entity_uuid": "", # the UUID the entity will have when realized + "status": "ACKNOWLEDGED", + "intent_state_ref": "", + "estimated_completion": "", + "status_url": "/api/v1/requests/{request_uuid}/status", + "dry_run_result": null # null if auto-approve; populated if review required +} + +Response 200 OK (if policy requires pre-validation report before submission): +{ + "dry_run": true, + "validation_result": { + "policies_evaluated": [...], + "gatekeeper_decisions": [{ "policy": "vm-size-limits", "result": "approved" }], + "estimated_cost": {...}, + "sovereignty_check": { "satisfied": true, "constraints": ["data_residency: EU"] }, + "would_auto_approve": true + } +} +``` + +### 4.2 Request Status + +``` +GET /api/v1/requests/{request_uuid}/status + +Response 200: +{ + "request_uuid": "", + "entity_uuid": "", + "status": "PROVISIONING", + "status_history": [ + { "status": "ACKNOWLEDGED", "at": "2026-03-15T09:00:00Z" }, + { "status": "ASSEMBLING", "at": "2026-03-15T09:00:02Z" }, + { "status": "DISPATCHED", "at": "2026-03-15T09:00:47Z" }, + { "status": "PROVISIONING", "at": "2026-03-15T09:01:05Z" } + ], + "current_step": "Provider is provisioning the resource", + "estimated_completion": "2026-03-15T09:05:00Z" +} + +# Terminal status response: +{ + "request_uuid": "", + "entity_uuid": "", + "status": "COMPLETED", + "completed_at": "2026-03-15T09:03:12Z", + "resource_url": "/api/v1/resources/{entity_uuid}" +} + +# Failed request: +{ + "request_uuid": "", + "entity_uuid": "", + "status": "FAILED", + "failed_at": "2026-03-15T09:02:45Z", + "failure_reason": "Provider capacity exhausted โ€” all eligible providers at capacity", + "retry_eligible": true, + "retry_after": "PT15M" +} +``` + +### 4.3 Consumer Request Status Lifecycle + +``` +ACKNOWLEDGED โ†’ request received; intent created +ASSEMBLING โ†’ Request Payload Processor running layer assembly +AWAITING_APPROVAL โ†’ policy requires human review before dispatch (PR open) +APPROVED โ†’ PR merged; dispatching to provider +DISPATCHED โ†’ provider received payload; awaiting confirmation +PROVISIONING โ†’ provider executing +COMPLETED โ†’ provider confirmed realization; Realized State written +FAILED โ†’ terminal; failure_reason and retry_eligible populated +CANCELLED โ†’ consumer-initiated cancellation before PROVISIONING +CANCELLING โ†’ cancellation in progress (provider notified) +``` + +### 4.4 Cancel Request + +Cancellation is only available before the PROVISIONING state. Once a provider is executing, cancellation moves to CANCELLING and depends on provider support. + +``` +DELETE /api/v1/requests/{request_uuid} + +Response 202 Accepted: +{ + "request_uuid": "", + "status": "CANCELLING", + "message": "Cancellation requested. Provider will be notified if dispatch has occurred." +} + +Response 409 Conflict (if cancellation not possible): +{ + "error": "cancellation_not_available", + "reason": "Resource is in PROVISIONING state. Cancellation requires provider support.", + "provider_supports_cancellation": false +} +``` + +--- + +## 5. Resource Management + +### 5.1 List Owned Resources + +``` +GET /api/v1/resources + +Query parameters: + resource_type= + lifecycle_state= + drift_status= + tag= + page= + page_size= + +Response 200: +{ + "resources": [ + { + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "display_name": "payments-api-server-01", + "lifecycle_state": "OPERATIONAL", + "drift_status": "clean", + "owned_by_tenant_uuid": "", + "created_at": "", + "provider_uuid": "", + "estimated_cost_per_hour": 0.32 + } + ], + "total": 12 +} +``` + +### 5.2 Describe Resource + +``` +GET /api/v1/resources/{entity_uuid} + +Response 200: +{ + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "lifecycle_state": "OPERATIONAL", + "drift_status": "clean", + "last_discovered_at": "", + + "fields": { + "cpu_count": { + "value": 4, + "confidence": { "band": "very_high", "score": 98 }, + "editable": false + }, + "memory_gb": { + "value": 8, + "confidence": { "band": "very_high", "score": 98 }, + "editable": false + } + }, + + "relationships": [ + { + "type": "attached_to", + "related_entity_uuid": "", + "related_entity_type": "Network.VLAN", + "stake_strength": "required" + } + ], + + "cost": { + "current_billing_state": "billable", + "estimated_cost_per_hour": 0.32, + "currency": "USD" + }, + + "rehydration_constraints": { + "min_auth_level": "oidc_mfa" + }, + + "pending_provider_notifications": [ + { + "notification_uuid": "", + "notification_type": "auto_scale", + "submitted_at": "", + "status": "pending_approval", + "changed_fields": ["memory_gb"], + "approval_url": "/api/v1/resources/{entity_uuid}/provider-notifications/{notification_uuid}/approve" + } + ] +} +``` + +### 5.3 Update Editable Fields (Targeted Delta) + +Updates one or more editable fields on a realized resource. Does not re-run layer assembly โ€” only the declared changes are dispatched to the provider. + +``` +PATCH /api/v1/resources/{entity_uuid} + +Request body: +{ + "updates": { + "name": "payments-api-server-01-renamed" + }, + "reason": "Renamed to align with new naming convention" +} + +Response 202 Accepted: +{ + "update_request_uuid": "", + "entity_uuid": "", + "status": "DISPATCHED", + "fields_updated": ["name"], + "status_url": "/api/v1/requests/{update_request_uuid}/status" +} + +Response 422 Unprocessable (if field is not editable): +{ + "error": "field_not_editable", + "field": "cpu_count", + "reason": "cpu_count is not declared as editable post-realization for this resource type" +} +``` + +### 5.4 Suspend Resource + +``` +POST /api/v1/resources/{entity_uuid}/suspend + +Request body: +{ + "reason": "Taking offline for maintenance window", + "auto_resume_at": "2026-03-16T06:00:00Z" # optional +} + +Response 202 Accepted: +{ + "entity_uuid": "", + "status": "SUSPENDING", + "auto_resume_at": "2026-03-16T06:00:00Z" +} +``` + +### 5.5 Decommission Resource + +``` +DELETE /api/v1/resources/{entity_uuid} + +Request body: +{ + "reason": "Project completed โ€” resource no longer needed", + "force": false # true to force even if non-required stakes/relationships exist + # cannot force decommission if required stakes exist +} + +Response 202 Accepted: +{ + "entity_uuid": "", + "status": "DECOMMISSIONING" +} + +Response 409 Conflict (required stakes or dependencies active): +{ + "error": "decommission_deferred", + "reason": "Resource has active required stake relationships", + "active_required_stakes": [ + { + "stakeholder_entity_uuid": "", + "stakeholder_resource_type": "Compute.VirtualMachine", + "stake_strength": "required" + } + ], + "resolution": "Release all required stakes before decommissioning, or request stakeholders to migrate" +} +``` + +### 5.6 Trigger Rehydration + +``` +POST /api/v1/resources/{entity_uuid}/rehydrate + +Request body: +{ + "source": "realized", # intent | requested | realized + "placement": { + "re_evaluate": false + }, + "governance": { + "policy_version": "current" + }, + "reason": "Provider migration โ€” EU-WEST-Prod-1 being decommissioned" +} + +Response 202 Accepted: +{ + "rehydration_request_uuid": "", + "entity_uuid": "", + "status": "ACKNOWLEDGED", + "lease_uuid": "", + "status_url": "/api/v1/requests/{rehydration_request_uuid}/status" +} + +Response 409 Conflict (rehydration lease already held): +{ + "error": "rehydration_lease_held", + "lease_held_since": "", + "lease_expires_at": "", + "retry_after": "PT2H" +} + +Response 403 Forbidden (step-up MFA required): +{ + "error": "step_up_required", + "reason": "Entity min_auth_level requires hardware_token_mfa for rehydration" +} +``` + +--- + + +### 5.7 Provider Update Notification Approval + +When a provider submits an update notification that requires consumer approval, the consumer receives a notification and the resource enters `PENDING_REVIEW` state. The consumer approves or rejects via this endpoint. + +``` +GET /api/v1/resources/{entity_uuid}/provider-notifications + +Response 200: +{ + "notifications": [ + { + "notification_uuid": "", + "notification_type": "auto_scale", + "provider_uuid": "", + "submitted_at": "", + "status": "pending_approval", + "change_summary": "Provider reports memory_gb increased from 8 to 16", + "change_reason": "Auto-scale policy triggered at 85% memory utilization", + "changed_fields": { + "memory_gb": { "previous_value": 8, "new_value": 16 } + } + } + ] +} + +POST /api/v1/resources/{entity_uuid}/provider-notifications/{notification_uuid}/approve +{ + "decision": "approve | reject", + "reason": "" +} + +Response 202 Accepted: +{ + "notification_uuid": "", + "decision": "approve", + "processed_at": "", + "realized_state_uuid": "" +} +``` + +**On approval:** A new Requested State record is created (`source_type: provider_update`, actor: consumer approver). A new Realized State snapshot is written. The entity exits PENDING_REVIEW. + +**On rejection:** The notification is rejected. The discrepancy between provider state and DCM Realized State becomes a drift event. The entity exits PENDING_REVIEW with an active drift record. + + +## 6. Audit Trail + +### 6.1 Query Audit Records for a Resource + +``` +GET /api/v1/resources/{entity_uuid}/audit + +Query parameters: + from= start of time range + to= end of time range + action= filter by action type + actor_type= + page= + page_size= + +Response 200: +{ + "audit_records": [ + { + "record_uuid": "", + "timestamp": "", + "action": "PROVISION", + "actor": { + "uuid": "", + "type": "human", + "display_name": "Jane Smith" + }, + "summary": "VirtualMachine provisioned via EU-WEST-Prod-1", + "correlation_id": "" + } + ], + "total": 47, + "chain_integrity": "verified" # verified | unverifiable | compromised +} +``` + +### 6.2 Follow Correlation ID + +For cross-state correlation โ€” following a request from Intent through all states: + +``` +GET /api/v1/audit/correlation/{correlation_id} + +Response 200: +{ + "correlation_id": "", + "entity_uuid": "", + "timeline": [ + { "state": "intent", "record_uuid": "", "timestamp": "..." }, + { "state": "requested", "record_uuid": "", "timestamp": "..." }, + { "state": "realized", "record_uuid": "", "timestamp": "..." } + ], + "cross_dcm_refs": [] # cross-DCM records if federation involved +} +``` + +--- + +## 7. Error Model + +All error responses follow a consistent structure: + +```json +{ + "error": "", + "message": "", + "request_id": "", + "timestamp": "", + "details": {} # error-specific additional context +} +``` + +**Standard error codes:** + +| HTTP Status | Error Code | Meaning | +|-------------|-----------|---------| +| 400 | `invalid_request` | Malformed request or missing required fields | +| 400 | `tenant_ambiguous` | Actor has multiple Tenants; X-DCM-Tenant header required | +| 401 | `authentication_required` | No token or expired token | +| 403 | `authorization_denied` | Token valid but insufficient permissions | +| 403 | `step_up_required` | Operation requires step-up MFA challenge | +| 404 | `not_found` | Entity, catalog item, or request not found | +| 409 | `decommission_deferred` | Decommission blocked by active stakes or dependencies | +| 409 | `rehydration_lease_held` | Entity already being rehydrated | +| 409 | `field_not_editable` | Targeted delta attempted on non-editable field | +| 422 | `policy_rejected` | GateKeeper policy rejected the request | +| 422 | `constraint_violated` | Field value violates declared constraint | +| 429 | `rate_limit_exceeded` | Actor has exceeded request rate limit | +| 503 | `assembly_unavailable` | Request Payload Processor temporarily unavailable | + +--- + +## 8. Conformance Levels + +The Consumer API defines three conformance levels, mirroring the Operator Interface Specification model: + +**Level 1 โ€” Read-Only:** Catalog browsing and resource status queries only. No request submission or resource management. Suitable for reporting and dashboard integrations. + +**Level 2 โ€” Standard:** Full request submission, status tracking, and basic resource management (update editable fields, decommission). Required for all self-service portal implementations. + +**Level 3 โ€” Full:** All Level 2 operations plus rehydration, audit trail access, and correlation queries. Required for ITSM integrations and compliance tooling. + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md index 2c5bc80..9ffadad 100644 --- a/content/docs/architecture/specifications/operator-interface-spec.md +++ b/content/docs/architecture/specifications/operator-interface-spec.md @@ -110,6 +110,9 @@ This specification defines three conformance levels. Higher levels unlock additi ### 2.2 Level 2 โ€” Standard +Level 2 conformance is required for providers that support auto-scaling, auto-healing, or provider-side maintenance operations. Level 2 includes all Level 1 requirements plus the Provider Update Notification API (Section 7a). + + **What it requires:** All Level 1 requirements, plus: - Capacity reporting to DCM (scheduled registration) - Full lifecycle event reporting (DEGRADED, MAINTENANCE, UNSANCTIONED_CHANGE, etc.) @@ -418,6 +421,132 @@ decommission_confirmation: --- + +--- + +## 7a. Provider Update Notification API + +This section defines the Provider Update Notification endpoint โ€” the formal mechanism by which Service Providers report authorized state changes to DCM. This is a **Level 2** conformance requirement for providers that support auto-scaling, auto-healing, or provider-side maintenance operations. + +### 7a.1 Overview + +The Provider Update Notification API enables providers to report authorized state changes so DCM can update its Realized State with a traceable Requested State record. This is distinct from drift โ€” a provider submitting an update notification is asserting that the change was authorized (by a pre-existing policy or operational agreement). DCM evaluates the assertion and decides whether to accept or reject it. + +**Key principle:** Providers never write directly to DCM's Realized State. They submit a notification; DCM processes it through its governance pipeline; DCM writes the Realized State if approved. + +### 7a.2 Conformance Requirements + +| Conformance Level | Requirement | +|------------------|-------------| +| Level 1 โ€” Basic | Not required. Providers at Level 1 report all state changes as lifecycle events; DCM handles them as drift. | +| Level 2 โ€” Standard | Required for providers that implement auto-scaling, auto-healing, or provider-side maintenance. | +| Level 3 โ€” Full | Required. All authorized provider-side state changes must use this API. | + +### 7a.3 Endpoint + +``` +POST /api/v1/provider/entities/{entity_uuid}/update-notification +Host: {dcm-instance} +Authorization: mTLS (provider certificate) +Content-Type: application/json +``` + +**Note:** This endpoint is on the DCM API Gateway, not on the provider. Providers call DCM; DCM does not poll providers for updates. + +### 7a.4 Request Payload + +```json +{ + "provider_uuid": "", + "notification_uuid": "", + "notification_type": "authorized_change | maintenance_change | auto_scale | auto_heal", + "changed_fields": { + "": { + "previous_value": "", + "new_value": "", + "change_reason": "", + "authorizing_policy_ref": "" + } + }, + "effective_at": "", + "provider_evidence_ref": "" +} +``` + +**`notification_uuid`** is an idempotency key. If DCM receives the same `notification_uuid` twice, it acknowledges the second request without reprocessing. + +**`authorizing_policy_ref`** is the UUID of the DCM policy that pre-authorized this type of change. If null, DCM will evaluate whether a policy covers this change. If no policy covers it, the notification is rejected. + +### 7a.5 Response Codes + +| Response | Meaning | +|----------|---------| +| `202 Accepted` | Notification accepted. DCM is processing. Use `notification_status_url` to poll. | +| `200 OK` (with `status: approved`) | Notification accepted and Realized State updated. | +| `200 OK` (with `status: pending_approval`) | Notification queued pending consumer approval. Entity in PENDING_REVIEW. | +| `200 OK` (with `status: rejected`) | Notification rejected. Realized State not updated. Discrepancy is now drift. | +| `409 Conflict` | A notification for this entity is already being processed. Retry after the `retry_after` interval. | +| `422 Unprocessable` | Notification payload malformed or entity UUID not found in this provider's scope. | + +```json +{ + "notification_uuid": "", + "status": "approved | pending_approval | rejected", + "realized_state_uuid": "", + "rejection_reason": "", + "retry_after": "", + "notification_status_url": "/api/v1/provider/notifications/{notification_uuid}" +} +``` + +### 7a.6 Notification Status Polling + +``` +GET /api/v1/provider/notifications/{notification_uuid} + +Response: +{ + "notification_uuid": "", + "status": "processing | approved | pending_approval | rejected", + "entity_uuid": "", + "realized_state_uuid": "", + "consumer_approval_required": true | false, + "consumer_notified_at": "", + "resolved_at": "" +} +``` + +### 7a.7 Idempotency + +Provider Update Notifications are idempotent by `notification_uuid`. If DCM crashes between receiving a notification and writing the Realized State, the provider can safely resend the same notification. DCM will not create duplicate Realized State records. + +### 7a.8 Pre-Authorization Declarations + +Providers may declare categories of updates they routinely make โ€” enabling organizations to pre-authorize them in policy rather than reviewing each one: + +```json +{ + "provider_uuid": "", + "update_capabilities": [ + { + "notification_type": "auto_scale", + "affected_fields": ["cpu_count", "memory_gb"], + "max_change_magnitude": "2x", + "typical_trigger": "Resource utilization threshold" + }, + { + "notification_type": "auto_heal", + "affected_fields": ["storage_device_id", "network_interface_id"], + "max_change_magnitude": "replacement", + "typical_trigger": "Hardware failure" + } + ] +} +``` + +This declaration is part of provider registration (Section 3.3) and is surfaced in the Service Catalog to help consumers understand what provider-side changes they can expect. + + ## 7. Field Mapping Specification *Required for Level 2 conformance.* diff --git a/static/capabilities/DCM-Capabilities-Matrix.csv b/static/capabilities/DCM-Capabilities-Matrix.csv new file mode 100644 index 0000000..e81e1f2 --- /dev/null +++ b/static/capabilities/DCM-Capabilities-Matrix.csv @@ -0,0 +1,96 @@ +Capability ID,Domain,Capability,Consumer Perspective,Service Provider Perspective,Platform/Admin Perspective,Depends On +IAM-001,Identity and Access Management,Actor Authentication,Authenticate to DCM via configured IdP,โ€”,Register and configure Auth Providers; manage local user store,โ€” +IAM-002,Identity and Access Management,Session Token Management,Receive and use session tokens; token refresh,โ€”,"Configure session TTL, failover chain",IAM-001 +IAM-003,Identity and Access Management,Role-Based Access Control,Receive role-appropriate service catalog and API responses,โ€”,Declare role mappings; assign roles to actors,IAM-001 +IAM-004,Identity and Access Management,Group Membership Resolution,Group memberships automatically applied from IdP,โ€”,Map IdP groups to DCM groups; declare group-role relationships,"IAM-001, IAM-003" +IAM-005,Identity and Access Management,Multi-Factor Authentication,Satisfy per-session and step-up MFA challenges,โ€”,Configure MFA methods; declare step-up operations,IAM-001 +IAM-006,Identity and Access Management,SCIM Automated Provisioning,Actor created/updated/deprovisioned from IdP automatically,โ€”,Configure SCIM endpoint and attribute mappings,IAM-001 +IAM-007,Identity and Access Management,Tenant Scope Enforcement,Access restricted to authorized Tenants,โ€”,Declare Tenant membership; configure cross-tenant policies,"IAM-003, IAM-004" +CAT-001,Service Catalog,Service Catalog Presentation,Browse available services filtered by RBAC,Declare catalog items for offered resource types,Activate catalog items; configure catalog visibility policies,"IAM-003, IAM-007" +CAT-002,Service Catalog,Service Schema Discovery,"View field schemas, constraints, and edit constraints for a catalog item",Declare field schemas in Resource Type Spec,Configure constraint visibility level per profile,CAT-001 +CAT-003,Service Catalog,Catalog Item Search and Filter,"Search catalog by keyword, resource type, tag",โ€”,Configure Search Index for catalog,CAT-001 +CAT-004,Service Catalog,Catalog Item Versioning,Request a specific version of a catalog item,Publish new catalog item versions following semver,Manage version lifecycle; enforce deprecation timelines,CAT-001 +CAT-005,Service Catalog,Cost Estimation,Receive estimated cost before submitting a request,Declare cost metadata on provider registration,Configure Cost Analysis component,CAT-001 +CAT-006,Service Catalog,Dependency Visualization,See required dependencies for a catalog item before requesting,Declare dependency graph in Resource Type Spec,โ€”,CAT-001 +CAT-007,Service Catalog,Catalog Item Deprecation,Receive deprecation warnings on deprecated catalog items,Declare successor types in deprecation notice,Manage deprecation lifecycle; notify consumers,CAT-004 +REQ-001,Request Lifecycle Management,Submit Service Request,"Submit a resource request via UI, API, or Git PR",โ€”,Configure request ingress surfaces,"IAM-007, CAT-001" +REQ-002,Request Lifecycle Management,Intent State Capture,Request stored as versioned GitOps artifact before processing,โ€”,Configure Intent Store; manage Git repository structure,REQ-001 +REQ-003,Request Lifecycle Management,Layer Assembly,Request enriched with organizational defaults and context layers,Contribute Service Layers for resource types,Manage Core Layers; configure Layer Cache,REQ-002 +REQ-004,Request Lifecycle Management,Policy Evaluation,"Request validated, transformed, and gated by applicable policies",Contribute provider-specific policies,Manage Policy Engine; configure Policy Groups and Profiles,REQ-003 +REQ-005,Request Lifecycle Management,Placement Engine Execution,Resource placed with the best available provider instance,Implement capacity reserve_query response,Configure placement constraints; manage provider priorities,REQ-004 +REQ-006,Request Lifecycle Management,Requested State Persistence,Assembled payload stored as authoritative GitOps record,โ€”,Configure Requested Store; manage storage redundancy,REQ-005 +REQ-007,Request Lifecycle Management,Provider Dispatch,Request payload delivered to selected provider,Implement Services API to receive DCM payloads,Configure API Gateway and egress,REQ-006 +REQ-008,Request Lifecycle Management,Request Status Tracking,Monitor request status from submitted through realized,Report realization status back to DCM,Configure observability for request tracking,REQ-007 +REQ-009,Request Lifecycle Management,Request Cancellation,Cancel a pending request before realization,Handle cancellation payloads,Configure cancellation policies,REQ-002 +REQ-010,Request Lifecycle Management,Git PR Request Ingress,Submit requests via Git Pull Request with policy dry-run feedback,โ€”,Configure Git Request Watcher; manage repository structure,"REQ-001, IAM-001" +PRV-001,Provider Contract and Realization,Provider Registration,โ€”,"Register provider with DCM: declare type, capabilities, sovereignty, cost metadata",Configure Provider Registry; validate sovereignty declarations,IAM-001 +PRV-002,Provider Contract and Realization,Naturalization,โ€”,Convert DCM unified payload to provider-native format,โ€”,"PRV-001, REQ-007" +PRV-003,Provider Contract and Realization,Realization,โ€”,Execute required actions to provision/configure/change resource,โ€”,PRV-002 +PRV-004,Provider Contract and Realization,Denaturalization,โ€”,Convert provider-native result back to DCM unified format,โ€”,PRV-003 +PRV-005,Provider Contract and Realization,Realized State Reporting,โ€”,Report realized payload and status to DCM API Gateway,Configure Realized State Store; manage Event Stream,PRV-004 +PRV-006,Provider Contract and Realization,Capacity Reporting,โ€”,Respond to reserve_query with current capacity and availability,Configure placement engine; manage capacity confidence,PRV-001 +PRV-007,Provider Contract and Realization,Provider Health Reporting,โ€”,Expose health check endpoint; report availability,Monitor provider health; configure trust score updates,PRV-001 +PRV-008,Provider Contract and Realization,Sovereignty Declaration Maintenance,โ€”,Notify DCM when sovereignty data changes within declared SLA,Monitor sovereignty changes; trigger re-evaluation,PRV-001 +PRV-009,Provider Contract and Realization,Meta Provider Orchestration,โ€”,Compose sub-providers to deliver higher-order services; manage composition visibility,Configure composite provider federation eligibility,"PRV-001, PRV-003" +LCM-001,Resource Lifecycle Management,Resource State Transitions,"Trigger lifecycle actions: suspend, resume, decommission",Handle state transition payloads,Configure lifecycle policies; manage state machine,REQ-008 +LCM-002,Resource Lifecycle Management,Post-Realization Field Updates,Update editable fields on realized resources (targeted delta),Handle delta update payloads; apply partial changes,Configure editable field declarations; manage edit policies,PRV-005 +LCM-003,Resource Lifecycle Management,Resource TTL Management,Declare and extend resource TTLs; receive expiry notifications,Handle TTL-triggered decommission payloads,Configure Lifecycle Constraint Enforcer; manage expiry policies,LCM-001 +LCM-004,Resource Lifecycle Management,Ownership Transfer,Transfer resource ownership to a different Tenant,โ€”,Authorize and execute ownership transfers; record transfer history,"IAM-007, LCM-001" +LCM-005,Resource Lifecycle Management,Rehydration,Replay a resource's intent state to a new provider or context,Receive and execute rehydration payloads,Manage rehydration leases; configure auth level requirements,"REQ-002, PRV-003" +LCM-006,Resource Lifecycle Management,Billing State Management,โ€”,โ€”,Configure billing state policies; integrate with Cost Analysis,LCM-001 +LCM-007,Resource Lifecycle Management,Resource Decommission,Decommission resources individually or as part of group decommission,Handle decommission payloads; release resources,Manage decommission workflows; coordinate dependency teardown,LCM-001 +DRF-001,Drift Detection and Remediation,Active Discovery,โ€”,Expose discovery endpoint; respond to interrogation queries,Configure discovery schedules; manage Discovered Store,PRV-005 +DRF-002,Drift Detection and Remediation,Drift Comparison,Receive drift notifications for owned resources,โ€”,Configure drift detection policies; manage comparison logic,"DRF-001, PRV-005" +DRF-003,Drift Detection and Remediation,Drift Notification,Receive actionable drift alerts with field-level detail,โ€”,Configure drift notification channels and escalation policies,DRF-002 +DRF-004,Drift Detection and Remediation,Drift Remediation,Approve or reject automatic drift remediation,Execute remediation payloads,Configure remediation policies (revert/update/alert/escalate),"DRF-002, LCM-002" +DRF-005,Drift Detection and Remediation,Unsanctioned Change Detection,Receive alerts on unauthorized resource modifications,Report all external state changes to DCM,Configure unsanctioned change policies,DRF-001 +POL-001,Policy Management,Policy Authoring,โ€”,Contribute provider-specific policy rules,Author and manage policies in GitOps store,IAM-003 +POL-002,Policy Management,Policy Validation and Shadow Mode,View shadow evaluation results on own requests,โ€”,Configure shadow mode; review shadow results in Validation Store,POL-001 +POL-003,Policy Management,Policy Activation and Review,โ€”,โ€”,Manage policy review periods; authorize policy activation,"POL-001, POL-002" +POL-004,Policy Management,Policy Group Management,โ€”,โ€”,Compose Policy Groups; manage profile assignments,POL-003 +POL-005,Policy Management,Profile Management,โ€”,โ€”,Configure deployment profiles; manage compliance domain groups,POL-004 +POL-006,Policy Management,Policy Provider Registration,โ€”,Register Policy Providers; maintain provider in declared mode,Configure Policy Provider trust levels; manage trust elevation workflow,"PRV-001, POL-001" +POL-007,Policy Management,Policy Override and Constraint Visibility,View constraint details for service catalog fields,Declare constraint schemas on Resource Type Specs,Configure constraint visibility levels per profile,"CAT-002, POL-003" +LAY-001,Data Layer Management,Core Layer Authoring,โ€”,โ€”,Author and manage Core and Organizational Layers in GitOps,IAM-003 +LAY-002,Data Layer Management,Service Layer Contribution,โ€”,Contribute Service Layers for offered resource types,Manage layer compatibility declarations,"PRV-001, LAY-001" +LAY-003,Data Layer Management,Layer Cache Management,โ€”,โ€”,Manage Layer Cache synchronization; handle cache invalidation,"LAY-001, LAY-002" +LAY-004,Data Layer Management,Layer Exclusion,Declare layer exclusions on specific requests,โ€”,Configure which layers may be excluded; manage non-excludable declarations,REQ-003 +LAY-005,Data Layer Management,Layer Versioning and Lifecycle,โ€”,โ€”,Manage layer versions; handle deprecation; enforce immutability,LAY-001 +INF-001,Information and Data Integration,Information Provider Registration,โ€”,Register Information Provider; declare authority scope and schema,Configure Information Provider Registry; manage authority layers,IAM-001 +INF-002,Information and Data Integration,Information Provider Push,โ€”,Push field value updates to DCM; respond to conflict notifications,Configure ingestion pipeline; manage conflict resolution policies,INF-001 +INF-003,Information and Data Integration,Information Provider Pull / Discovery,โ€”,Expose data query endpoint for DCM pull operations,Configure pull schedules; manage cache TTLs,INF-001 +INF-004,Information and Data Integration,Write-Back,โ€”,Implement write-back endpoint to receive DCM-initiated updates,Configure write-back triggers via policy,"INF-001, INF-002" +INF-005,Information and Data Integration,Confidence Score Visibility,View confidence bands on entity field values; query confidence aggregation API,โ€”,Configure confidence scoring formula; manage trust score thresholds,INF-001 +INF-006,Information and Data Integration,Conflict Resolution Management,โ€”,โ€”,Review and resolve contested field values; manage conflict escalation,INF-002 +ING-001,Ingestion and Brownfield Management,Resource Discovery and Ingestion,โ€”,Expose discovery endpoints for brownfield resources,Configure ingestion pipeline; manage transitional Tenant,DRF-001 +ING-002,Ingestion and Brownfield Management,Ingested Entity Review,โ€”,โ€”,Review ingested entities; resolve conflicts; promote to active Tenants,ING-001 +ING-003,Ingestion and Brownfield Management,Bulk Promotion,โ€”,โ€”,Execute bulk entity promotions with preview and rollback,ING-002 +ING-004,Ingestion and Brownfield Management,Catalog Item Association,โ€”,โ€”,Associate ingested entities with Resource Type Specs; create catalog items,"ING-002, CAT-001" +AUD-001,Audit and Compliance,Audit Trail Access,Query audit records for own resources,โ€”,Configure Audit Store; manage retention policies,IAM-003 +AUD-002,Audit and Compliance,Compliance Reporting,โ€”,โ€”,Generate compliance reports; manage report schedules,AUD-001 +AUD-003,Audit and Compliance,Hash Chain Verification,โ€”,โ€”,Run scheduled and on-demand hash chain verification; manage integrity incidents,AUD-001 +AUD-004,Audit and Compliance,Cross-DCM Audit Correlation,โ€”,โ€”,Correlate audit records across DCM instances via correlation_id; authorize cross-DCM pulls,"AUD-001, DCM-001" +AUD-005,Audit and Compliance,Audit Record Retention Management,โ€”,โ€”,Configure reference-based retention; manage post-lifecycle retention,AUD-001 +OBS-001,Observability and Operations,Operational Dashboard,View health and status of own resources,โ€”,Configure and manage observability dashboard,โ€” +OBS-002,Observability and Operations,Metrics and Telemetry Export,โ€”,Expose resource-level metrics to DCM,Configure observability export; integrate enterprise observability platform,โ€” +OBS-003,Observability and Operations,Curated Event Stream Subscription,Subscribe to observability event types via Message Bus,โ€”,Configure event stream publication policies; manage subscriber roles,OBS-002 +OBS-004,Observability and Operations,Alert and Notification Management,Receive resource and policy alerts via declared channels,โ€”,Configure alert routing; manage notification channels and escalation,OBS-001 +OBS-005,Observability and Operations,Cost Analysis and Attribution,View cost estimates and actuals for owned resources,Provide cost metadata; report utilization,Configure Cost Analysis component; manage cost attribution policies,PRV-006 +STO-001,Storage and State Management,GitOps Store Management,โ€”,โ€”,Configure and manage Intent and Requested Stores; manage Git repository structure,โ€” +STO-002,Storage and State Management,Realized State Store Management,โ€”,โ€”,Configure Event Stream and Realized Store; manage retention,PRV-005 +STO-003,Storage and State Management,Discovered State Store Management,โ€”,โ€”,Configure Discovered Store; manage retention policies per profile,DRF-001 +STO-004,Storage and State Management,Search Index Management,Use entity and catalog search,โ€”,Configure Search Index; manage rebuild on failure,STO-001 +STO-005,Storage and State Management,Backup and Recovery,โ€”,โ€”,Configure backup schedules; test recovery procedures,"STO-001, STO-002" +STO-006,Storage and State Management,Provenance Model Configuration,โ€”,โ€”,Select and configure provenance model (full_inline/deduplicated/tiered); manage tier transitions,STO-001 +FED-001,DCM Federation and Multi-Instance,DCM Provider Registration,Submit requests that are routed to peer DCMs,Register as DCM Provider in peer instances,Configure DCM Provider registrations; manage federation trust,"PRV-001, IAM-001" +FED-002,DCM Federation and Multi-Instance,Federation Routing,Requests automatically routed to appropriate Regional/Sovereign DCM,Respond to reserve queries from Hub DCM,Configure federation placement policies; manage sovereignty pre-filters,"FED-001, REQ-005" +FED-003,DCM Federation and Multi-Instance,Federation Trust Management,โ€”,โ€”,Manage mTLS certificates; monitor federation trust scores; handle cert rotation,FED-001 +FED-004,DCM Federation and Multi-Instance,Cross-DCM Drift Detection,Receive drift alerts for federated resources,Publish Discovered State events to federation Message Bus,Configure federated drift detection; manage alert-and-hold policies,"FED-001, DRF-002" +FED-005,DCM Federation and Multi-Instance,DCM Export and Import,โ€”,โ€”,Export and import DCM state packages; verify import trust scores,"STO-001, STO-002" +GOV-001,Platform Governance and Administration,Tenant Management,โ€”,โ€”,"Create, configure, and decommission Tenants; manage compliance overlays",IAM-007 +GOV-002,Platform Governance and Administration,Group Management,โ€”,โ€”,Create and manage DCM Groups; configure sovereignty rules; manage time-bounded memberships,IAM-003 +GOV-003,Platform Governance and Administration,Registry Management,โ€”,Register and maintain Resource Type Specifications in organization registry,Manage registry sync; configure registry policies; manage Tier 3 types,PRV-001 +GOV-004,Platform Governance and Administration,Resource Type Lifecycle,โ€”,Manage deprecation notices; declare successor types; maintain migration guidance,Enforce deprecation timelines; manage sunset periods,GOV-003 +GOV-005,Platform Governance and Administration,Platform Configuration Management,โ€”,โ€”,Manage platform-wide layers; configure profiles; manage deployment manifest,"LAY-001, POL-005" +GOV-006,Platform Governance and Administration,Bootstrap and Self-Hosting,โ€”,โ€”,Manage DCM self-deployment; verify bootstrap manifest; handle repave scenarios,STO-001 +GOV-007,Platform Governance and Administration,Sovereign Deployment Management,โ€”,โ€”,Manage air-gapped DCM instances; configure signed bundle import; manage offline registry,"FED-001, STO-001" diff --git a/static/capabilities/map.html b/static/capabilities/map.html new file mode 100644 index 0000000..7dc4c62 --- /dev/null +++ b/static/capabilities/map.html @@ -0,0 +1,824 @@ + + + + + +DCM Capabilities Matrix + + + +
+ +
+
DCM Project ยท Red Hat FlightPath
+

Foundational Capabilities Matrix

+

100 core operational capabilities required for DCM to perform lifecycle management. Each capability is mapped across consumer, service provider, and platform/admin perspectives.

+
+
95
Capabilities
+
15
Domains
+
21
MVP Critical Path
+
3
Perspectives
+
+
+ + +
+ + +
+
+ โŒ• + +
+ +
+ + +
+
+ +
Showing 95 of 95 capabilities
+ + +
+
+ + + + + + + + + + + + +
IDCapabilityConsumerService ProviderPlatform / AdminDepends On
+
+
+ + + + + + +
+ + + + From df0f7f815575ff291c888064aaa85c280ed3596e Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Fri, 27 Mar 2026 15:42:00 -0500 Subject: [PATCH 22/49] Added timeout / cancelleation procedures. Add recovery policy. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 139 ++- .../architecture/data-model/four-states.md | 16 + .../data-model/operational-models.md | 854 ++++++++++++++++++ .../data-model/policy-profiles.md | 53 +- .../data-model/resource-service-entities.md | 2 +- .../data-model/service-dependencies.md | 56 +- .../specifications/consumer-api-spec.md | 80 +- .../specifications/operator-interface-spec.md | 61 ++ 8 files changed, 1242 insertions(+), 19 deletions(-) create mode 100644 content/docs/architecture/data-model/operational-models.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index d5e2495..965d4c5 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -417,6 +417,19 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | Model | Description | Example | |-------|-------------|---------| | **Allocation** | Provider retains internal ownership. Consumer owns the Entity (the allocation). Provider has reclaim rights on decommission. | VM, Container, IP Address | +| **Recovery Policy** | Formal DCM policy type mapping trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions; same authoring model as GateKeeper/Validation/Transformation | +| **recovery_posture** | Fifth Policy Group concern_type governing failure and ambiguity response; binds a recovery profile group to the deployment | +| **DRIFT_RECONCILE** | Recovery action: schedule discovery; let drift detection resolve actual state | +| **DISCARD_AND_REQUEUE** | Recovery action: best-effort cleanup; new request cycle created immediately | +| **NOTIFY_AND_WAIT** | Recovery action: notify human; wait for explicit decision up to declared deadline | +| **TIMEOUT_PENDING** | Infrastructure Resource Entity state: dispatch timeout fired; recovery policy evaluating | +| **LATE_REALIZATION_PENDING** | Entity state: provider responded after timeout; NOTIFY_AND_WAIT recovery decision pending | +| **INDETERMINATE_REALIZATION** | Entity state: state ambiguous; drift detection resolving | +| **COMPENSATION_FAILED** | Entity state: compound service rollback itself failed; orphan detection active | +| **orphan_candidate** | Resource discovered at provider with no corresponding Realized State record; surfaced to platform admin for human resolution | +| **Discovery Scheduler** | DCM control plane component maintaining priority queue of discovery requests; dispatches to provider discovery endpoints | +| **recovery-automated-reconciliation** | Built-in recovery profile: trust drift detection; accept late responses; appropriate for dev/standard | +| **recovery-notify-and-wait** | Built-in recovery profile: notify human; never act automatically; appropriate for prod/fsi/sovereign | | **Notification Provider** | Ninth DCM provider type; translates unified notification envelope to delivery channel; handles delivery, retry, dead letter, and delivery confirmation callbacks | | **Notification Router** | DCM control plane component that resolves notification audiences and routes envelopes to Notification Providers | | **audience resolution** | Deriving notification recipients by traversing the entity relationship graph from the changed entity at event time | @@ -2456,7 +2469,20 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | Former Term | Replacement | Meaning | |-------------|-------------|---------| -| Shore | **Notification Provider** | Ninth DCM provider type; translates unified notification envelope to delivery channel; handles delivery, retry, dead letter, and delivery confirmation callbacks | +| Shore | **Recovery Policy** | Formal DCM policy type mapping trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions; same authoring model as GateKeeper/Validation/Transformation | +| **recovery_posture** | Fifth Policy Group concern_type governing failure and ambiguity response; binds a recovery profile group to the deployment | +| **DRIFT_RECONCILE** | Recovery action: schedule discovery; let drift detection resolve actual state | +| **DISCARD_AND_REQUEUE** | Recovery action: best-effort cleanup; new request cycle created immediately | +| **NOTIFY_AND_WAIT** | Recovery action: notify human; wait for explicit decision up to declared deadline | +| **TIMEOUT_PENDING** | Infrastructure Resource Entity state: dispatch timeout fired; recovery policy evaluating | +| **LATE_REALIZATION_PENDING** | Entity state: provider responded after timeout; NOTIFY_AND_WAIT recovery decision pending | +| **INDETERMINATE_REALIZATION** | Entity state: state ambiguous; drift detection resolving | +| **COMPENSATION_FAILED** | Entity state: compound service rollback itself failed; orphan detection active | +| **orphan_candidate** | Resource discovered at provider with no corresponding Realized State record; surfaced to platform admin for human resolution | +| **Discovery Scheduler** | DCM control plane component maintaining priority queue of discovery requests; dispatches to provider discovery endpoints | +| **recovery-automated-reconciliation** | Built-in recovery profile: trust drift detection; accept late responses; appropriate for dev/standard | +| **recovery-notify-and-wait** | Built-in recovery profile: notify human; never act automatically; appropriate for prod/fsi/sovereign | +| **Notification Provider** | Ninth DCM provider type; translates unified notification envelope to delivery channel; handles delivery, retry, dead letter, and delivery confirmation callbacks | | **Notification Router** | DCM control plane component that resolves notification audiences and routes envelopes to Notification Providers | | **audience resolution** | Deriving notification recipients by traversing the entity relationship graph from the changed entity at event time | | **notification_uuid** | Idempotency key on notification envelopes; Notification Providers use this to deduplicate on retry | @@ -3039,7 +3065,95 @@ REL-022 through REL-024: traversal depth declared in Resource Type Spec; default --- -## SECTION 49 โ€” PERSONAS +## SECTION 49 โ€” GROUP 2: OPERATIONAL MODELS + +### 49.1 Three Timeout Scopes + +All three independently configurable and audited: +- **Assembly timeout** โ€” max time for Request Payload Processor nine-step assembly (standard: PT3M; prod: PT2M) +- **Dispatch timeout** โ€” max time waiting for provider realization after dispatch (standard/prod: PT30M-PT1H; resource-type overrides for legitimately long types) +- **Reserve-query timeout** โ€” max time for a single provider to respond to reserve query (prod: PT5S); on timeout: skip that candidate, continue placement loop + +### 49.2 Cancellation โ€” Three Scenarios + +1. **Before dispatch:** Clean cancel; no provider interaction; entity โ†’ CANCELLED +2. **After dispatch, provider not started:** DCM sends cancellation; provider confirms; entity โ†’ CANCELLED +3. **During PROVISIONING:** Provider capability-dependent: + - Supports cancellation: send cancel; provider attempts rollback; outcome โ†’ Recovery Policy + - No cancellation support: CANCEL_PENDING; wait for completion; LATE_RESPONSE_RECEIVED fires + +Cancellation is always best-effort โ€” never guaranteed. Provider declares `supports_cancellation` and `partial_rollback_possible` at registration. + +### 49.3 Discovery Scheduling โ€” Three Trigger Types + +1. **Scheduled (cron):** Each Resource Type Spec declares discovery interval; profile overrides; profile_min=PT4H minimal, PT5M fsi/sovereign +2. **Event-triggered:** After entity.realized (PT30S delay), drift.resolved (PT60S), provider.degraded (immediate), TIMEOUT_PENDING (PT5M orphan detection), COMPENSATION_FAILED (immediate) +3. **On-demand:** `POST /api/v1/admin/discovery/trigger` by platform admin; also used by CI/CD pre-validation and brownfield ingestion + +Discovery Scheduler component maintains priority queue (Critical โ†’ High โ†’ Standard โ†’ Background). Queue depth bounded per profile. + +### 49.4 Recovery Policy Model โ€” The Unified Framework + +Recovery Policies are a formal DCM policy type (alongside GateKeeper, Validation, Transformation). Same authoring, GitOps store, shadow mode, activation workflow, and audit trail. + +**Trigger vocabulary (closed):** ASSEMBLY_TIMEOUT, DISPATCH_TIMEOUT, RESERVE_QUERY_ALL_EXHAUSTED, LATE_RESPONSE_RECEIVED, CANCELLATION_SENT, CANCELLATION_CONFIRMED, CANCELLATION_FAILED, PARTIAL_REALIZATION, COMPENSATION_IN_PROGRESS, COMPENSATION_FAILED + +**Action vocabulary (closed):** DRIFT_RECONCILE, DISCARD_AND_REQUEUE, DISCARD_NO_REQUEUE, ACCEPT_LATE_REALIZATION, COMPENSATE_AND_FAIL, NOTIFY_AND_WAIT (with deadline + on_deadline_exceeded), ESCALATE, RETRY (with backoff + max_attempts + on_exhaustion) + +### 49.5 Four Built-in Recovery Profile Groups + +| Group | Posture | Profile Default | +|-------|---------|----------------| +| `recovery-automated-reconciliation` | Trust drift detection to converge | minimal/dev/standard | +| `recovery-discard-and-requeue` | Clean up and restart on ambiguity | (opt-in) | +| `recovery-notify-and-wait` | Notify human; never act automatically | prod/fsi/sovereign | +| `recovery-aggressive-retry` | Retry everything before giving up | (opt-in) | + +Binding hierarchy: resource-type override > Tenant override > profile default > system default (automated-reconciliation). + +`recovery_posture` is the fifth Policy Group concern_type (alongside security, compliance, operational, implementation posture). + +### 49.6 Late Response Pipeline + +Provider responds after DCM timeout: +1. Late Response Handler activates (entity in TIMEOUT_PENDING state) +2. Cancel the pending cancellation if not yet sent +3. Write realized payload to Realized Store +4. Entity โ†’ LATE_REALIZATION_PENDING (if NOTIFY_AND_WAIT) or action per policy (if DRIFT_RECONCILE or DISCARD_AND_REQUEUE) + +NOTIFY_AND_WAIT consumer interface: `GET /api/v1/resources/{uuid}/recovery-decisions` and `POST` with chosen action. Platform admin can resolve any entity's pending decision via Admin API. + +### 49.7 Compound Service Compensation + +Declared per component in service definition: +- `required_for_delivery: atomic` โ€” failure triggers full compensation rollback +- `required_for_delivery: partial` โ€” failure โ†’ DEGRADED (not FAILED); no compensation triggered +- `compensation_on_failure: decommission_immediately | release_allocation | skip | notify` +- `compensation_order: ` โ€” reverse order = first-decommissioned; lowest compensation_order runs last in reverse + +Partial delivery policy: `min_required_components` declares minimum for DEGRADED delivery; `auto_retry_optional_components` retries failed optional components. + +### 49.8 Five New Lifecycle States + +| State | Entry | Recovery Trigger | +|-------|-------|-----------------| +| TIMEOUT_PENDING | Dispatch timeout fired | DISPATCH_TIMEOUT | +| LATE_REALIZATION_PENDING | Late response received + NOTIFY_AND_WAIT | LATE_RESPONSE_RECEIVED | +| INDETERMINATE_REALIZATION | DRIFT_RECONCILE action taken | โ€” | +| COMPENSATION_IN_PROGRESS | Compound rollback underway | โ€” | +| COMPENSATION_FAILED | Rollback itself failed | COMPENSATION_FAILED | + +### 49.9 Orphan Detection Pipeline + +Triggers: timeout with cancellation sent, cancellation failed, compensation failed, DISCARD_NO_REQUEUE. Queries provider for resources matching Requested State characteristics in the provisioning time window, excluding known Realized State UUIDs. Creates ORPHAN_CANDIDATE records; notifies platform admin (urgency: high); human resolves (manual decommission, adopt into DCM, or mark false positive). + +### 49.10 Policies + +OPS-010 through OPS-019. Key: cancellation always best-effort (OPS-011); recovery policies are formal DCM policy type (OPS-014); four built-in recovery profiles (OPS-015); binding hierarchy resource-type > Tenant > profile (OPS-016); compensation in reverse dependency order (OPS-017); orphan detection on any uncertain cleanup (OPS-018); NOTIFY_AND_WAIT deadline always has on_deadline_exceeded action (OPS-019). + +--- + +## SECTION 50 โ€” PERSONAS | Persona | Primary Concern | |---------|----------------| @@ -3056,7 +3170,7 @@ REL-022 through REL-024: traversal depth declared in Resource Type Spec; default --- -## SECTION 50 โ€” TERMINOLOGY GLOSSARY +## SECTION 51 โ€” TERMINOLOGY GLOSSARY | Term | Definition | |------|-----------| @@ -3119,6 +3233,19 @@ REL-022 through REL-024: traversal depth declared in Resource Type Spec; default | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail | | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class | | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation | +| **Recovery Policy** | Formal DCM policy type mapping trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions; same authoring model as GateKeeper/Validation/Transformation | +| **recovery_posture** | Fifth Policy Group concern_type governing failure and ambiguity response; binds a recovery profile group to the deployment | +| **DRIFT_RECONCILE** | Recovery action: schedule discovery; let drift detection resolve actual state | +| **DISCARD_AND_REQUEUE** | Recovery action: best-effort cleanup; new request cycle created immediately | +| **NOTIFY_AND_WAIT** | Recovery action: notify human; wait for explicit decision up to declared deadline | +| **TIMEOUT_PENDING** | Infrastructure Resource Entity state: dispatch timeout fired; recovery policy evaluating | +| **LATE_REALIZATION_PENDING** | Entity state: provider responded after timeout; NOTIFY_AND_WAIT recovery decision pending | +| **INDETERMINATE_REALIZATION** | Entity state: state ambiguous; drift detection resolving | +| **COMPENSATION_FAILED** | Entity state: compound service rollback itself failed; orphan detection active | +| **orphan_candidate** | Resource discovered at provider with no corresponding Realized State record; surfaced to platform admin for human resolution | +| **Discovery Scheduler** | DCM control plane component maintaining priority queue of discovery requests; dispatches to provider discovery endpoints | +| **recovery-automated-reconciliation** | Built-in recovery profile: trust drift detection; accept late responses; appropriate for dev/standard | +| **recovery-notify-and-wait** | Built-in recovery profile: notify human; never act automatically; appropriate for prod/fsi/sovereign | | **Notification Provider** | Ninth DCM provider type; translates unified notification envelope to delivery channel; handles delivery, retry, dead letter, and delivery confirmation callbacks | | **Notification Router** | DCM control plane component that resolves notification audiences and routes envelopes to Notification Providers | | **audience resolution** | Deriving notification recipients by traversing the entity relationship graph from the changed entity at event time | @@ -3307,7 +3434,7 @@ REL-022 through REL-024: traversal depth declared in Resource Type Spec; default --- -## SECTION 51 โ€” OPEN QUESTIONS +## SECTION 52 โ€” OPEN QUESTIONS These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -3404,7 +3531,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 52 โ€” DOCUMENTATION STRUCTURE +## SECTION 53 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -3452,7 +3579,7 @@ content/ --- -## SECTION 53 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 54 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md index a9cc256..28ffafc 100644 --- a/content/docs/architecture/data-model/four-states.md +++ b/content/docs/architecture/data-model/four-states.md @@ -111,6 +111,22 @@ The **Discovered State** is what DCM observes actually existing through active d **Content:** Raw discovered resource state in DCM Unified Data Model format, with discovery metadata (timestamp, discovery method, provider interrogated) + +### 2.5 Recovery States + +Five additional states apply to Infrastructure Resource Entities when the normal provisioning lifecycle encounters timeouts, cancellation failures, or partial realization. These states are governed by Recovery Policies (see [Operational Models](24-operational-models.md) Section 5). + +| State | Meaning | Entry Trigger | +|-------|---------|--------------| +| `TIMEOUT_PENDING` | Dispatch timeout fired; cancellation sent to provider | `DISPATCH_TIMEOUT` recovery trigger | +| `LATE_REALIZATION_PENDING` | Provider responded after timeout; NOTIFY_AND_WAIT active | `LATE_RESPONSE_RECEIVED` recovery trigger | +| `INDETERMINATE_REALIZATION` | State ambiguous; drift detection resolving | `DRIFT_RECONCILE` recovery action | +| `COMPENSATION_IN_PROGRESS` | Compound service rollback underway | `PARTIAL_REALIZATION` trigger | +| `COMPENSATION_FAILED` | Rollback itself failed; orphaned resources possible | Compensation step failure | + +See [Operational Models](24-operational-models.md) for the complete recovery state machine and Recovery Policy model. + + --- ## 3. The Entity UUID โ€” Universal Linking Key diff --git a/content/docs/architecture/data-model/operational-models.md b/content/docs/architecture/data-model/operational-models.md new file mode 100644 index 0000000..ad87424 --- /dev/null +++ b/content/docs/architecture/data-model/operational-models.md @@ -0,0 +1,854 @@ +--- +title: "Operational Models" +type: docs +weight: 24 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Policy Profiles](14-policy-profiles.md) | [Notification Model](23-notification-model.md) + +--- + +## 1. Purpose + +This document defines the operational models that govern DCM behavior at the edges of the normal provisioning lifecycle โ€” when things go wrong, take too long, or produce ambiguous outcomes. Four operational models are defined: + +1. **Timeout Model** โ€” assembly, dispatch, and reserve-query timeouts +2. **Cancellation Propagation Model** โ€” consumer-initiated cancellation at any lifecycle stage +3. **Discovery Scheduling Model** โ€” what triggers discovery cycles and how they are managed +4. **Recovery Policy Model** โ€” the unified, policy-governed response to all failure and ambiguity scenarios + +The Recovery Policy Model is the foundational concept. Timeouts, cancellation outcomes, partial realization, and compensation failures all produce trigger conditions that Recovery Policies handle. Organizations declare their recovery posture via profile-bound Policy Groups โ€” not via ad-hoc per-incident decisions. + +--- + +## 2. Timeout Model + +### 2.1 Three Timeout Scopes + +There are three distinct timeout concerns in the DCM pipeline. Each is independently configurable and independently audited. + +```yaml +timeout_declarations: + assembly_timeout: + description: "Maximum time for the Request Payload Processor to complete nine-step assembly" + profile_defaults: + minimal: PT5M + dev: PT5M + standard: PT3M + prod: PT2M + fsi: PT2M + sovereign: PT2M + on_timeout: trigger ASSEMBLY_TIMEOUT recovery policy + includes: layer_resolution, policy_evaluation, placement_engine_loop + # Assembly timeout fires if the total assembly pipeline exceeds this duration + # Individual sub-steps also have per-step timeouts (see below) + + dispatch_timeout: + description: "Maximum time to wait for provider realization after dispatch" + profile_defaults: + minimal: PT2H + dev: PT1H + standard: PT1H + prod: PT30M + fsi: PT30M + sovereign: PT30M + resource_type_overrides: + # Some resource types legitimately take longer to provision + Compute.BareMetalServer: PT4H + Storage.LargeVolume: PT2H + on_timeout: trigger DISPATCH_TIMEOUT recovery policy + + reserve_query_timeout: + description: "Maximum time for a single provider to respond to a reserve query" + profile_defaults: + minimal: PT30S + dev: PT30S + standard: PT10S + prod: PT5S + fsi: PT5S + sovereign: PT10S + on_timeout: skip this provider; continue placement loop with remaining candidates + # Reserve query timeout does not trigger RESERVE_QUERY_TIMEOUT recovery policy + # unless ALL candidates have timed out or been exhausted +``` + +### 2.2 Per-Step Assembly Sub-Timeouts + +The nine-step assembly has per-step sub-timeouts. These are not independently configurable โ€” they are proportional fractions of the assembly_timeout: + +| Step | Fraction of assembly_timeout | +|------|----------------------------| +| Layer Resolution | 20% | +| Layer Merge | 10% | +| Policy Evaluation (each policy) | 15% total, 5% per Mode 1/2, 30s per Mode 3, PT2M per Mode 4 | +| Placement Engine Loop | 40% | +| Requested State Persistence | 10% | + +A Mode 4 Policy Provider that takes longer than PT2M per query causes an ASSEMBLY_TIMEOUT regardless of the overall assembly_timeout remaining. This prevents a single slow Policy Provider from consuming the entire assembly budget. + +### 2.3 Timeout Audit Records + +Every timeout produces an audit record: + +```yaml +audit_record: + action: ASSEMBLY_TIMEOUT | DISPATCH_TIMEOUT | RESERVE_QUERY_TIMEOUT + actor: + type: system + system_actor: + component: request_payload_processor | provider_dispatch | placement_engine + trigger: timeout + entity_uuid: + details: + timeout_duration: + actual_elapsed: + step_at_timeout: + recovery_policy_triggered: +``` + +--- + +## 3. Cancellation Propagation Model + +### 3.1 Three Cancellation Scenarios + +Cancellation behavior depends on the entity's lifecycle state at the time the consumer submits a cancellation request. + +**Scenario 1 โ€” Cancel before dispatch (ACKNOWLEDGED โ†’ ASSEMBLING โ†’ AWAITING_APPROVAL):** + +``` +Consumer submits DELETE /api/v1/requests/{uuid} + โ”‚ + โ–ผ Entity state: pre-DISPATCHED + โ”‚ Assembly halted immediately + โ”‚ No provider interaction required + โ”‚ Intent State record marked CANCELLED + โ”‚ Entity enters CANCELLED state (terminal) + โ”‚ Audit: REQUEST_CANCELLED + โ”‚ Recovery Policy: not triggered (clean cancel) + โ”‚ + โ””โ”€โ”€ Response: 200 OK { "status": "CANCELLED" } +``` + +**Scenario 2 โ€” Cancel after dispatch, provider not yet started (DISPATCHED):** + +``` +Consumer submits DELETE /api/v1/requests/{uuid} + โ”‚ + โ–ผ Entity state: DISPATCHED (provider received payload but has not started) + โ”‚ DCM sends cancellation payload to provider cancel endpoint + โ”‚ Provider acknowledges: "not started, cancellation clean" + โ”‚ Entity enters CANCELLED state (terminal) + โ”‚ Recovery Policy: not triggered (clean cancel) + โ”‚ + โ””โ”€โ”€ Response: 202 Accepted { "status": "CANCELLING" } + โ†’ status polling shows CANCELLED when provider confirms +``` + +**Scenario 3 โ€” Cancel while provider is executing (PROVISIONING):** + +``` +Consumer submits DELETE /api/v1/requests/{uuid} + โ”‚ + โ–ผ Entity state: PROVISIONING + โ”‚ DCM checks provider.supports_cancellation + โ”‚ + โ”œโ”€โ”€ Provider supports cancellation: + โ”‚ DCM sends cancellation payload + โ”‚ Provider attempts rollback + โ”‚ โ”œโ”€โ”€ Rollback clean: entity โ†’ CANCELLED (terminal) + โ”‚ โ”œโ”€โ”€ Rollback partial: trigger CANCELLATION_FAILED recovery policy + โ”‚ โ””โ”€โ”€ No response: trigger CANCELLATION_FAILED recovery policy + โ”‚ + โ””โ”€โ”€ Provider does not support cancellation: + Entity enters CANCEL_PENDING state + DCM waits for provider to complete + On provider REALIZED response: + Recovery Policy LATE_RESPONSE_RECEIVED fires + (configured action: typically DISCARD_AND_REQUEUE for cancellation context) + On provider FAILED response: + Entity โ†’ FAILED (terminal) โ€” no compensation needed +``` + +### 3.2 Provider Cancellation Capability Declaration + +Providers declare cancellation support in their registration: + +```yaml +provider_cancellation_capabilities: + supports_cancellation: true + cancellation_supported_during: [DISPATCHED, PROVISIONING] + # DISPATCHED: can cancel before work begins + # PROVISIONING: can cancel and roll back mid-execution + cancellation_endpoint: POST /api/v1/provider/entities/{entity_uuid}/cancel + cancellation_response_time: PT30S # SLA for cancellation response + partial_rollback_possible: true + # true: cancellation may leave partial resources โ†’ CANCELLATION_FAILED path + # false: cancellation is all-or-nothing (rare) +``` + +### 3.3 Cancellation Payload + +```json +{ + "cancellation_uuid": "", + "entity_uuid": "", + "requested_state_uuid": "", + "reason": "consumer_requested | timeout | policy_triggered", + "requested_at": "", + "best_effort": true +} +``` + +`best_effort: true` is always set โ€” DCM never guarantees cancellation success. The provider makes a best-effort attempt; outcomes flow through the Recovery Policy model. + +--- + +## 4. Discovery Scheduling Model + +### 4.1 The Discovery Scheduler Component + +The **Discovery Scheduler** is a DCM control plane component responsible for triggering discovery cycles. It maintains a priority queue of pending discovery requests and dispatches them to the appropriate Service Provider's discovery endpoint. + +The Discovery Scheduler is distinct from drift detection. The Discovery Scheduler triggers discovery and writes Discovered State. Drift Detection reads Discovered State and compares it to Realized State. These are separate, independent components. + +### 4.2 Three Discovery Trigger Types + +**Trigger Type 1 โ€” Scheduled (cron-based):** + +Discovery schedules are declared in the Resource Type Specification and in provider registrations. The Discovery Scheduler runs these on the declared cadence. + +```yaml +resource_type_spec: + fqn: Compute.VirtualMachine + discovery_schedule: + default_interval: PT15M # discover VMs every 15 minutes + # Override by profile: + profile_overrides: + minimal: PT4H # less frequent in home lab + fsi: PT5M # more frequent in regulated environments + sovereign: PT5M + + # Per-provider discovery endpoint + discovery_endpoint_path: /api/v1/provider/discover + discovery_method: api_query # api_query | passive_event | hybrid +``` + +```yaml +provider_registration: + discovery_capabilities: + supports_discovery: true + discovery_endpoint: POST /api/v1/provider/entities/discover + max_entities_per_discovery_batch: 1000 + discovery_latency_p95: PT10S # how long discovery typically takes + supports_incremental_discovery: true + # incremental: only entities changed since last_discovery_timestamp + # full: all entities every time +``` + +**Trigger Type 2 โ€” Event-triggered:** + +Specific DCM events automatically schedule an out-of-cycle discovery pass: + +```yaml +event_triggered_discovery: + triggers: + - event: entity.realized + discovery_delay: PT30S # allow provider to stabilize + scope: this_entity + reason: "Confirm realization matches Requested State" + + - event: drift.resolved + discovery_delay: PT60S + scope: this_entity + reason: "Confirm remediation took effect" + + - event: provider_update.approved + discovery_delay: PT30S + scope: this_entity + reason: "Confirm provider update is reflected in infrastructure" + + - event: provider.degraded + discovery_delay: PT0S # immediate + scope: all_entities_on_provider + reason: "Assess impact of provider degradation" + + - event: TIMEOUT_PENDING # recovery trigger + discovery_delay: PT5M + scope: this_entity + reason: "Orphan detection after timeout" + + - event: COMPENSATION_FAILED + discovery_delay: PT0S # immediate + scope: this_entity_and_dependents + reason: "Find orphaned resources after compensation failure" +``` + +**Trigger Type 3 โ€” On-demand:** + +Platform admins and SREs can trigger discovery manually: + +``` +POST /api/v1/admin/discovery/trigger + +{ + "scope": "entity | resource_type | provider | tenant", + "entity_uuid": "", # if scope: entity + "resource_type": "", # if scope: resource_type + "provider_uuid": "", # if scope: provider + "tenant_uuid": "", # if scope: tenant + "reason": "incident investigation", + "priority": "high" +} +``` + +On-demand discovery is also used by: +- The CI/CD pipeline pre-validation step (confirm current state before assembly) +- The brownfield ingestion pipeline (initial discovery of existing infrastructure) +- The orphan detection pipeline (targeted search for potentially-orphaned resources) + +### 4.3 Discovery Queue Management + +The Discovery Scheduler manages a priority queue. Priority order: + +1. **Critical** โ€” COMPENSATION_FAILED orphan detection, sovereignty violation assessment +2. **High** โ€” on-demand from platform admin, event-triggered (provider.degraded) +3. **Standard** โ€” event-triggered (entity.realized, drift.resolved) +4. **Background** โ€” scheduled discovery passes + +Queue depth is bounded per profile. When the queue is full, new Background-priority items are dropped (with a log entry). Standard and above are never dropped โ€” they wait. + +### 4.4 Discovery Audit + +Every discovery cycle produces an audit record: + +```yaml +audit_record: + action: DISCOVERY_CYCLE_COMPLETED | DISCOVERY_CYCLE_FAILED + actor: + type: system + system_actor: + component: discovery_scheduler + trigger: scheduled | event_triggered | on_demand + trigger_event_uuid: + entity_uuid: # null for batch discovery + details: + entities_discovered: 47 + new_entities_found: 2 # brownfield candidates + duration: PT8S +``` + +--- + +## 5. Recovery Policy Model + +### 5.1 Recovery Policy as a Policy Type + +Recovery Policies are a formal DCM policy type alongside GateKeeper, Validation, and Transformation. They use the same authoring model, the same GitOps store, the same shadow mode validation, the same activation workflow, and the same audit trail. + +```yaml +recovery_policy: + artifact_metadata: + uuid: + handle: "system/recovery/discard-on-timeout" + version: "1.0.0" + status: active + owned_by: { display_name: "DCM Core Team" } + + policy_type: recovery + trigger: DISPATCH_TIMEOUT # the trigger condition this policy handles + action: DISCARD_AND_REQUEUE # the action to take + + # Optional additional conditions + conditions: + - field: entity.resource_type + operator: in + value: [Compute.VirtualMachine, Container.Pod] + - field: entity.owned_by_tenant.profile + operator: equals + value: prod + + # Action parameters (depend on action type) + action_parameters: + requeue_delay: PT0S # immediate requeue + notify_before_action: true + notification_urgency: high + + # Deadline for NOTIFY_AND_WAIT actions + deadline: null # not applicable for DISCARD_AND_REQUEUE + on_deadline_exceeded: null +``` + +### 5.2 Trigger Vocabulary (Closed) + +| Trigger | Description | +|---------|-------------| +| `ASSEMBLY_TIMEOUT` | Assembly pipeline exceeded configured timeout | +| `DISPATCH_TIMEOUT` | Provider did not respond within dispatch_timeout | +| `RESERVE_QUERY_ALL_EXHAUSTED` | All placement candidates timed out or rejected | +| `LATE_RESPONSE_RECEIVED` | Provider responded after DCM declared timeout | +| `CANCELLATION_SENT` | DCM sent cancellation to provider | +| `CANCELLATION_CONFIRMED` | Provider confirmed clean cancellation | +| `CANCELLATION_FAILED` | Provider could not cancel; partial state possible | +| `PARTIAL_REALIZATION` | Compound service partially realized | +| `COMPENSATION_IN_PROGRESS` | Rollback of partial components underway | +| `COMPENSATION_FAILED` | Rollback itself failed; orphaned resources possible | + +### 5.3 Action Vocabulary (Closed) + +| Action | Description | +|--------|-------------| +| `DRIFT_RECONCILE` | Schedule discovery pass; let drift detection resolve actual state via configured drift response policy | +| `DISCARD_AND_REQUEUE` | Best-effort cleanup sent to provider; new request cycle created immediately from Intent State | +| `DISCARD_NO_REQUEUE` | Best-effort cleanup sent to provider; entity FAILED; no automatic requeue | +| `ACCEPT_LATE_REALIZATION` | Accept late provider response; write Realized State; entity proceeds to OPERATIONAL | +| `COMPENSATE_AND_FAIL` | Execute compensation rollback for compound service; entity FAILED when complete | +| `NOTIFY_AND_WAIT` | Fire notification to configured audience; wait for human decision up to deadline | +| `ESCALATE` | Notify platform admin immediately; no automatic action | +| `RETRY` | Retry the failed operation with configured backoff | + +### 5.4 The Four Built-in Recovery Profile Groups + +#### recovery-automated-reconciliation + +"Let the system converge on correct state โ€” trust drift detection and policy." + +Appropriate for: standard and dev environments where operational continuity takes priority over strict consistency. + +```yaml +recovery_policy_group: + handle: "system/group/recovery-automated-reconciliation" + concern_type: recovery_posture + policies: + - trigger: ASSEMBLY_TIMEOUT + action: RETRY + max_attempts: 3 + backoff: exponential + initial_interval: PT30S + on_exhaustion: ESCALATE + + - trigger: DISPATCH_TIMEOUT + action: DRIFT_RECONCILE + # Discovery finds what actually exists; drift response policy handles it + + - trigger: LATE_RESPONSE_RECEIVED + action: ACCEPT_LATE_REALIZATION + # Provider did the work; accept it + + - trigger: CANCELLATION_FAILED + action: DRIFT_RECONCILE + # Cannot confirm cleanup; discovery finds orphans + + - trigger: PARTIAL_REALIZATION + action: DRIFT_RECONCILE + # Discover what's there; drift policy handles component gaps + + - trigger: COMPENSATION_FAILED + action: ESCALATE + # Human needed when cleanup itself fails +``` + +#### recovery-discard-and-requeue + +"On any ambiguity, clean up and start fresh โ€” prioritize consistency over continuity." + +Appropriate for: environments where reproducibility is paramount, resources are cheap to reprovision, untracked resources are a compliance concern. + +```yaml +recovery_policy_group: + handle: "system/group/recovery-discard-and-requeue" + concern_type: recovery_posture + policies: + - trigger: ASSEMBLY_TIMEOUT + action: RETRY + max_attempts: 2 + on_exhaustion: DISCARD_NO_REQUEUE + + - trigger: DISPATCH_TIMEOUT + action: DISCARD_AND_REQUEUE + # Best-effort cleanup; new request cycle immediately + + - trigger: LATE_RESPONSE_RECEIVED + action: DISCARD_AND_REQUEUE + # Provider completed after DCM moved on; discard that work + # (requeue already happened on timeout; this prevents duplicate resources) + + - trigger: CANCELLATION_FAILED + action: DISCARD_NO_REQUEUE + # Cannot clean up; FAILED; human reviews orphans before requeue + + - trigger: PARTIAL_REALIZATION + action: COMPENSATE_AND_FAIL + # Roll back everything; start fresh + + - trigger: COMPENSATION_FAILED + action: ESCALATE + # Cannot even roll back; human needed +``` + +#### recovery-notify-and-wait + +"Never act automatically โ€” always notify a human and wait for explicit authorization." + +Appropriate for: FSI and sovereign environments where automated resource creation or deletion has regulatory implications, where change control processes must be honored. + +```yaml +recovery_policy_group: + handle: "system/group/recovery-notify-and-wait" + concern_type: recovery_posture + policies: + - trigger: ASSEMBLY_TIMEOUT + action: NOTIFY_AND_WAIT + deadline: PT2H + notification_urgency: high + on_deadline_exceeded: ESCALATE + + - trigger: DISPATCH_TIMEOUT + action: NOTIFY_AND_WAIT + deadline: PT4H + notification_urgency: high + on_deadline_exceeded: ESCALATE + + - trigger: LATE_RESPONSE_RECEIVED + action: NOTIFY_AND_WAIT + deadline: PT4H + notification_urgency: medium + on_deadline_exceeded: DISCARD_NO_REQUEUE + + - trigger: CANCELLATION_FAILED + action: NOTIFY_AND_WAIT + deadline: PT8H + notification_urgency: high + on_deadline_exceeded: ESCALATE + + - trigger: PARTIAL_REALIZATION + action: NOTIFY_AND_WAIT + deadline: PT8H + notification_urgency: high + on_deadline_exceeded: COMPENSATE_AND_FAIL + + - trigger: COMPENSATION_FAILED + action: ESCALATE + # Always escalate compensation failures โ€” no deadline +``` + +#### recovery-aggressive-retry + +"Retry everything before giving up โ€” maximize first-time success rate." + +Appropriate for: environments with transient provider issues, where retries are cheap and manual intervention capacity is limited. + +```yaml +recovery_policy_group: + handle: "system/group/recovery-aggressive-retry" + concern_type: recovery_posture + policies: + - trigger: ASSEMBLY_TIMEOUT + action: RETRY + max_attempts: 5 + backoff: exponential + initial_interval: PT15S + max_interval: PT5M + on_exhaustion: NOTIFY_AND_WAIT + deadline: PT2H + + - trigger: DISPATCH_TIMEOUT + action: RETRY + max_attempts: 3 + backoff: linear + interval: PT5M + on_exhaustion: DRIFT_RECONCILE + + - trigger: RESERVE_QUERY_ALL_EXHAUSTED + action: RETRY + max_attempts: 3 + backoff: exponential + initial_interval: PT1M + on_exhaustion: ESCALATE + + - trigger: PARTIAL_REALIZATION + action: RETRY + retry_scope: failed_components_only # preserve succeeded components + max_attempts: 3 + interval: PT15M + on_exhaustion: COMPENSATE_AND_FAIL + + - trigger: CANCELLATION_FAILED + action: DRIFT_RECONCILE + + - trigger: COMPENSATION_FAILED + action: ESCALATE +``` + +### 5.5 Profile Binding + +Recovery profile groups bind to deployment profiles as defaults, with override at Tenant and resource-type levels: + +```yaml +profile_recovery_defaults: + minimal: recovery-automated-reconciliation + dev: recovery-automated-reconciliation + standard: recovery-automated-reconciliation + prod: recovery-notify-and-wait + fsi: recovery-notify-and-wait + sovereign: recovery-notify-and-wait + +# Tenant-level override +tenant_config: + tenant_uuid: + recovery_profile_override: recovery-discard-and-requeue + +# Resource-type-level override (most specific; wins over Tenant and profile) +resource_type_recovery_override: + resource_type: Compute.VirtualMachine + recovery_profile: recovery-aggressive-retry + # VMs use aggressive retry; other types use Tenant/profile default +``` + +### 5.6 NOTIFY_AND_WAIT Consumer Interface + +When a recovery policy fires `NOTIFY_AND_WAIT`, a notification is sent to the entity owner with a time-bounded decision interface: + +``` +GET /api/v1/resources/{entity_uuid}/recovery-decisions + +Response: +{ + "recovery_decision_uuid": "", + "trigger": "DISPATCH_TIMEOUT", + "entity_uuid": "", + "deadline": "", + "available_actions": [ + { + "action": "DRIFT_RECONCILE", + "description": "Let discovery determine actual state and reconcile automatically" + }, + { + "action": "DISCARD_AND_REQUEUE", + "description": "Best-effort cleanup, then requeue as a new request" + }, + { + "action": "DISCARD_NO_REQUEUE", + "description": "Best-effort cleanup only; no automatic requeue" + } + ] +} + +POST /api/v1/resources/{entity_uuid}/recovery-decisions/{recovery_decision_uuid} +{ + "action": "DISCARD_AND_REQUEUE", + "reason": "Provider was known to be degraded at time of timeout" +} +``` + +Platform admins may also use the Admin API to resolve pending recovery decisions for any entity regardless of Tenant. + +### 5.7 Recovery Policy Evaluation Precedence + +The Policy Engine evaluates recovery policies in domain precedence order, same as all other policies: + +``` +1. Resource-type-level override (most specific) +2. Tenant-level override +3. Active profile's recovery posture group +4. System default (automated-reconciliation) + +First matching policy for the trigger condition wins. +Multiple recovery policies for the same trigger at the same domain level +โ†’ policy conflict; CONFLICT_ERROR at ingestion; platform admin notified. +``` + +--- + +## 6. Compound Service Compensation Model + +### 6.1 Compensation Declaration in Service Dependencies + +Each component in a compound service declares its compensation behavior: + +```yaml +compound_service_spec: + service_type: ApplicationStack.WebApp + components: + - id: vm + resource_type: Compute.VirtualMachine + required_for_delivery: atomic # must succeed; failure triggers compensation + compensation_on_failure: decommission_immediately + compensation_order: 3 # decommissioned last (highest number = last) + + - id: ip + resource_type: Network.IPAddress + required_for_delivery: atomic + compensation_on_failure: release_allocation + compensation_order: 1 # decommissioned first + depends_on: [] + + - id: dns + resource_type: DNS.Record + required_for_delivery: partial # failure โ†’ DEGRADED, not FAILED + compensation_on_failure: skip # DNS failure doesn't trigger VM decommission + depends_on: [vm, ip] + + - id: loadbalancer + resource_type: Network.LoadBalancer + required_for_delivery: partial + compensation_on_failure: skip + depends_on: [vm, ip] + + partial_delivery_policy: + min_required_components: [vm, ip] # compound DEGRADED if only these succeed + degraded_is_acceptable: true # DEGRADED entity is delivered; not FAILED + auto_retry_optional_components: + enabled: true + max_attempts: 3 + interval: PT15M + on_exhaustion: notify_owner +``` + +### 6.2 Compensation Execution Order + +Compensation always runs in reverse dependency order โ€” last-provisioned is first-decommissioned: + +``` +Successful so far: vm โœ“, ip โœ“ +Failed: dns โœ— (atomic) +Compensation triggered: + Step 1: decommission vm (compensation_order: 3 โ†’ runs first in reverse) + Step 2: release ip allocation (compensation_order: 1 โ†’ runs second in reverse) + Compound entity โ†’ FAILED (terminal for this request cycle) +``` + +### 6.3 Compensation Failure + +If a compensation step fails (the VM decommission itself fails): + +``` +Compensation of vm FAILED + Entity enters COMPENSATION_FAILED state + COMPENSATION_FAILED recovery policy fires: + default: ESCALATE to platform admin + Orphan detection triggered immediately: + Scoped to provider + entity characteristics + Finds the VM that couldn't be decommissioned + Creates ORPHAN_CANDIDATE record + Platform admin reviews: + Manually decommission at provider + OR adopt into DCM lifecycle as a new entity +``` + +--- + +## 7. Orphan Detection Pipeline + +When cleanup cannot be guaranteed, DCM runs an orphan detection pass to find resources that may have been provisioned but have no corresponding Realized State record. + +### 7.1 Orphan Detection Triggers + +- Dispatch timeout with cancellation sent +- Cancellation failed +- Compensation failed +- DISCARD_NO_REQUEUE action taken +- Manual platform admin trigger + +### 7.2 Orphan Detection Query + +```yaml +orphan_detection_query: + provider_uuid: + time_window: + from: + to: + match_criteria: + resource_type: + characteristics: # key fields from the Requested State + name_pattern: + size_class: + tags: + exclude: + known_realized_state_uuids: [, ...] # entities DCM knows about +``` + +### 7.3 Orphan Candidate Lifecycle + +```yaml +orphan_candidate: + orphan_candidate_uuid: + suspected_request_uuid: # the request that may have created this + provider_entity_id: # what the provider calls it + provider_uuid: + discovered_at: + characteristics: { ... } + status: + resolution: + action: + resolved_by: + resolved_at: +``` + +Orphan candidates are surfaced in the Platform Admin dashboard and generate a NOTIFICATION (audience: Platform Admin) with urgency: high. + +--- + +## 8. New Lifecycle States + +Five new states are added to the Infrastructure Resource Entity lifecycle: + +| State | Meaning | Recovery Policy Trigger | +|-------|---------|------------------------| +| `TIMEOUT_PENDING` | Dispatch timeout fired; cancellation sent; awaiting outcome | `DISPATCH_TIMEOUT` | +| `LATE_REALIZATION_PENDING` | Provider responded after timeout; NOTIFY_AND_WAIT active | `LATE_RESPONSE_RECEIVED` | +| `INDETERMINATE_REALIZATION` | State is ambiguous; drift detection resolving | โ€” (drift detection runs) | +| `COMPENSATION_IN_PROGRESS` | Compound service rollback underway | โ€” | +| `COMPENSATION_FAILED` | Rollback itself failed; orphaned resources possible | `COMPENSATION_FAILED` | + +Updated state machine (additions to doc 02 and doc 06): + +``` +Normal flow: +REQUESTED โ†’ PENDING โ†’ PROVISIONING โ†’ REALIZED โ†’ OPERATIONAL โ†’ DECOMMISSIONED + +Recovery states: +PROVISIONING โ†’ [timeout] โ†’ TIMEOUT_PENDING + TIMEOUT_PENDING โ†’ [late response + NOTIFY_AND_WAIT] โ†’ LATE_REALIZATION_PENDING + TIMEOUT_PENDING โ†’ [DRIFT_RECONCILE] โ†’ INDETERMINATE_REALIZATION + TIMEOUT_PENDING โ†’ [DISCARD_AND_REQUEUE] โ†’ FAILED + new REQUESTED (new cycle) + +PROVISIONING โ†’ [partial failure] โ†’ COMPENSATION_IN_PROGRESS + COMPENSATION_IN_PROGRESS โ†’ [all compensated] โ†’ FAILED + COMPENSATION_IN_PROGRESS โ†’ [compensation fails] โ†’ COMPENSATION_FAILED + +LATE_REALIZATION_PENDING โ†’ [human accepts / ACCEPT_LATE] โ†’ REALIZED โ†’ OPERATIONAL +LATE_REALIZATION_PENDING โ†’ [human discards / DISCARD] โ†’ FAILED + +INDETERMINATE_REALIZATION โ†’ [drift reconciles] โ†’ REALIZED or FAILED +COMPENSATION_FAILED โ†’ [human resolves] โ†’ FAILED (after manual cleanup) +``` + +--- + +## 9. System Policies + +| Policy | Rule | +|--------|------| +| `OPS-010` | Assembly timeout, dispatch timeout, and reserve-query timeout are independently configurable. All are profile-governed with resource-type overrides permitted for types with legitimately long provisioning times. | +| `OPS-011` | Cancellation is always best-effort. DCM never guarantees cancellation success. All cancellation outcomes flow through the Recovery Policy model. | +| `OPS-012` | Provider cancellation capability is declared at registration. Providers that do not support cancellation use the CANCEL_PENDING โ†’ LATE_RESPONSE_RECEIVED path when a cancel is requested during PROVISIONING. | +| `OPS-013` | Discovery is triggered by three independent mechanisms: scheduled (cron), event-triggered, and on-demand. All three write to the Discovered Store independently. | +| `OPS-014` | Recovery Policies are a formal DCM policy type. They use the same authoring, activation, shadow mode, and audit model as GateKeeper, Validation, and Transformation policies. | +| `OPS-015` | Four built-in recovery profile groups are provided: recovery-automated-reconciliation, recovery-discard-and-requeue, recovery-notify-and-wait, recovery-aggressive-retry. | +| `OPS-016` | Recovery profile defaults are bound to deployment profiles. Organizations may override at Tenant or resource-type level. Resource-type override wins over Tenant override wins over profile default. | +| `OPS-017` | Compound service compensation runs in reverse dependency order. Compensation failure triggers COMPENSATION_FAILED state and immediate orphan detection. | +| `OPS-018` | Orphan detection triggers on any path where cleanup cannot be guaranteed. Orphan candidates are surfaced to platform admin with urgency: high. | +| `OPS-019` | NOTIFY_AND_WAIT recovery actions carry a deadline. If the deadline passes without human resolution, the configured on_deadline_exceeded action fires automatically. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index a906013..0c549bc 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -1,7 +1,7 @@ --- title: "Policy Organization: Groups, Profiles, and Providers" type: docs -weight: 13 +weight: 14 --- > **โš ๏ธ Active Development Notice** @@ -1495,6 +1495,57 @@ policy_provider_airgap: | `PROF-009` | Policy Provider delivery in air-gapped deployments uses signed bundles identical to the registry bundle model. Mode 4 providers in sovereign profiles may only call endpoints within the sovereignty boundary. | + +--- + +## 9. Recovery Posture Policy Groups + +### 9.1 recovery_posture as a Concern Type + +`recovery_posture` is a Policy Group concern_type that governs how DCM responds to provisioning failures, timeouts, and ambiguous states. It is the fifth concern type alongside security, compliance, operational, and implementation posture. + +Recovery posture groups contain Recovery Policies โ€” a formal DCM policy type that maps trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions (DRIFT_RECONCILE, DISCARD_AND_REQUEUE, NOTIFY_AND_WAIT, etc.). + +See [Operational Models](24-operational-models.md) Section 5 for the complete Recovery Policy model, trigger vocabulary, and action vocabulary. + +### 9.2 Four Built-in Recovery Posture Groups + +| Group Handle | Posture | Appropriate For | +|-------------|---------|----------------| +| `system/group/recovery-automated-reconciliation` | Let drift detection converge on correct state | Dev, standard environments | +| `system/group/recovery-discard-and-requeue` | Clean up and restart on any ambiguity | Consistency-critical environments | +| `system/group/recovery-notify-and-wait` | Always notify human; never act automatically | FSI, sovereign, regulated environments | +| `system/group/recovery-aggressive-retry` | Retry everything before giving up | High-transient-failure environments | + +### 9.3 Profile Binding Defaults + +| Profile | Default Recovery Posture | +|---------|------------------------| +| `minimal` | recovery-automated-reconciliation | +| `dev` | recovery-automated-reconciliation | +| `standard` | recovery-automated-reconciliation | +| `prod` | recovery-notify-and-wait | +| `fsi` | recovery-notify-and-wait | +| `sovereign` | recovery-notify-and-wait | + +### 9.4 Override Hierarchy + +Organizations override recovery posture at Tenant or resource-type level without changing the deployment profile: + +```yaml +# Tenant override โ€” all resources in this Tenant use discard-and-requeue +tenant_config: + recovery_profile_override: recovery-discard-and-requeue + +# Resource-type override โ€” VMs get aggressive retry regardless of Tenant/profile +resource_type_recovery_override: + resource_type: Compute.VirtualMachine + recovery_profile: recovery-aggressive-retry +``` + +Resource-type override wins over Tenant override wins over profile default. + + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md index 113ecdf..f83d51f 100644 --- a/content/docs/architecture/data-model/resource-service-entities.md +++ b/content/docs/architecture/data-model/resource-service-entities.md @@ -12,7 +12,7 @@ weight: 6 **Document Status:** ๐Ÿ”„ In Progress -**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Operational Models](24-operational-models.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) --- diff --git a/content/docs/architecture/data-model/service-dependencies.md b/content/docs/architecture/data-model/service-dependencies.md index 5e72930..b9e454c 100644 --- a/content/docs/architecture/data-model/service-dependencies.md +++ b/content/docs/architecture/data-model/service-dependencies.md @@ -1,7 +1,7 @@ --- title: "Service Dependencies" type: docs -weight: 6 +weight: 7 --- > **โš ๏ธ Active Development Notice** @@ -484,6 +484,60 @@ meta_provider_registration: - **Field-Level Provenance** โ€” dependency payload data is recorded with source Entity UUID - **Resource Grouping** โ€” rehydration can be scoped to groups and tenants + +--- + +## 8. Compound Service Compensation Declaration + +### 8.1 Overview + +Compound services (delivered by Meta Providers) must declare compensation behavior for each component. This declaration is part of the service definition โ€” not discovered at runtime. See [Operational Models](24-operational-models.md) Section 6 for the full compensation execution model. + +### 8.2 Compensation Fields on Service Components + +```yaml +service_component: + id: vm + resource_type: Compute.VirtualMachine + required_for_delivery: + # atomic: must succeed; failure triggers full compensation rollback + # partial: failure โ†’ DEGRADED state; compound service delivered partially + + compensation_on_failure: + # decommission_immediately: decommission this component as part of rollback + # release_allocation: release allocation back to pool (for allocatable resources) + # skip: do not compensate; used for partial delivery components + # notify: notify owner; human decides compensation + + compensation_order: + # Lower numbers compensate first; higher numbers compensate last + # Reverse dependency order is the default if not declared + + depends_on: [] +``` + +### 8.3 Partial Delivery Policy + +```yaml +partial_delivery_policy: + min_required_components: [vm, ip] # compound DEGRADED if only these succeed + degraded_is_acceptable: true + auto_retry_optional_components: + enabled: true + max_attempts: 3 + interval: PT15M + on_exhaustion: notify_owner +``` + +### 8.4 System Policies โ€” Compensation + +| Policy | Rule | +|--------|------| +| `DEP-010` | Compensation executes in reverse dependency order (highest compensation_order first). | +| `DEP-011` | Compensation failure triggers COMPENSATION_FAILED state and immediate orphan detection. | +| `DEP-012` | Components with required_for_delivery: partial are not compensation-triggering. Their failure produces a DEGRADED compound entity. | + + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index fb0c8fb..ed9e76f 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -370,16 +370,22 @@ Response 200: ### 4.3 Consumer Request Status Lifecycle ``` -ACKNOWLEDGED โ†’ request received; intent created -ASSEMBLING โ†’ Request Payload Processor running layer assembly -AWAITING_APPROVAL โ†’ policy requires human review before dispatch (PR open) -APPROVED โ†’ PR merged; dispatching to provider -DISPATCHED โ†’ provider received payload; awaiting confirmation -PROVISIONING โ†’ provider executing -COMPLETED โ†’ provider confirmed realization; Realized State written -FAILED โ†’ terminal; failure_reason and retry_eligible populated -CANCELLED โ†’ consumer-initiated cancellation before PROVISIONING -CANCELLING โ†’ cancellation in progress (provider notified) +ACKNOWLEDGED โ†’ request received; intent created +ASSEMBLING โ†’ Request Payload Processor running layer assembly +AWAITING_APPROVAL โ†’ policy requires human review before dispatch +APPROVED โ†’ proceeding to assembly and dispatch +DISPATCHED โ†’ provider received payload; awaiting confirmation +PROVISIONING โ†’ provider executing +COMPLETED โ†’ provider confirmed realization; Realized State written +FAILED โ†’ terminal; failure_reason and retry_eligible populated +CANCELLED โ†’ consumer-initiated cancellation; clean terminal +CANCELLING โ†’ cancellation in progress; provider notified +TIMEOUT_PENDING โ†’ dispatch timeout fired; recovery policy evaluating +LATE_REALIZATION_PENDING โ†’ provider responded after timeout; recovery decision pending +INDETERMINATE_REALIZATION โ†’ state ambiguous; drift detection resolving +COMPENSATION_IN_PROGRESS โ†’ compound service rollback underway +COMPENSATION_FAILED โ†’ rollback failed; platform admin notified; orphan detection active +PENDING_REVIEW โ†’ conflict detected requiring human resolution ``` ### 4.4 Cancel Request @@ -671,6 +677,60 @@ Response 202 Accepted: **On rejection:** The notification is rejected. The discrepancy between provider state and DCM Realized State becomes a drift event. The entity exits PENDING_REVIEW with an active drift record. + +### 5.8 Recovery Decisions + +When a recovery policy fires `NOTIFY_AND_WAIT`, the entity owner can query and respond to the pending decision. + +``` +GET /api/v1/resources/{entity_uuid}/recovery-decisions + +Response 200: +{ + "recovery_decision_uuid": "", + "trigger": "DISPATCH_TIMEOUT", + "entity_uuid": "", + "entity_state": "TIMEOUT_PENDING", + "deadline": "", + "deadline_action": "ESCALATE", + "context": { + "timeout_fired_at": "", + "cancellation_sent": true, + "cancellation_status": "unknown" + }, + "available_actions": [ + { + "action": "DRIFT_RECONCILE", + "description": "Let discovery determine actual state and reconcile automatically" + }, + { + "action": "DISCARD_AND_REQUEUE", + "description": "Best-effort cleanup; new request cycle created immediately" + }, + { + "action": "DISCARD_NO_REQUEUE", + "description": "Best-effort cleanup only; no automatic requeue" + } + ] +} + +POST /api/v1/resources/{entity_uuid}/recovery-decisions/{recovery_decision_uuid} +{ + "action": "DISCARD_AND_REQUEUE", + "reason": "Provider was known degraded; clean restart preferred" +} + +Response 202 Accepted: +{ + "recovery_decision_uuid": "", + "action_taken": "DISCARD_AND_REQUEUE", + "new_request_uuid": "" # the new request cycle UUID +} +``` + +**Note:** Recovery decisions are only available when the active recovery profile includes `NOTIFY_AND_WAIT`. With other profiles (automated-reconciliation, discard-and-requeue) the system acts automatically and no decision endpoint is exposed. + + ## 6. Audit Trail ### 6.1 Query Audit Records for a Resource diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md index 9ffadad..4339816 100644 --- a/content/docs/architecture/specifications/operator-interface-spec.md +++ b/content/docs/architecture/specifications/operator-interface-spec.md @@ -547,6 +547,67 @@ Providers may declare categories of updates they routinely make โ€” enabling org This declaration is part of provider registration (Section 3.3) and is surfaced in the Service Catalog to help consumers understand what provider-side changes they can expect. + +--- + +## 7b. Cancellation API + +This section defines the cancellation endpoint that Service Providers implement for Level 2+ conformance. Providers that declare `supports_cancellation: true` in their registration must implement this endpoint. + +### 7b.1 Cancellation Endpoint + +``` +POST /cancel (on the provider, called by DCM) +Authorization: DCM mTLS certificate + +Body: +{ + "cancellation_uuid": "", + "entity_uuid": "", + "requested_state_uuid": "", + "reason": "consumer_requested | timeout | policy_triggered", + "requested_at": "", + "best_effort": true +} +``` + +### 7b.2 Response + +| Code | Meaning | +|------|---------| +| `200 OK` (status: cancelled) | Cancellation clean; no resources provisioned | +| `200 OK` (status: partial_rollback) | Cancellation attempted; some resources may remain | +| `200 OK` (status: too_late) | Provider completed before cancellation arrived; late response forthcoming | +| `409 Conflict` | Already cancelled or already completed | + +```json +{ + "cancellation_uuid": "", + "status": "cancelled | partial_rollback | too_late", + "resources_remaining": [], + "late_response_expected": false, + "notes": "" +} +``` + +### 7b.3 Late Response After Cancellation + +If the provider returns `status: too_late`, it must still send the completed realization response via the standard realized-state callback. DCM's Late Response Pipeline handles this โ€” the provider does not need to do anything different. The `LATE_RESPONSE_RECEIVED` Recovery Policy fires on the DCM side. + +### 7b.4 Capability Declaration + +```json +{ + "cancellation_capabilities": { + "supports_cancellation": true, + "cancellation_supported_during": ["DISPATCHED", "PROVISIONING"], + "partial_rollback_possible": true, + "cancellation_response_time_seconds": 30 + } +} +``` + + ## 7. Field Mapping Specification *Required for Level 2 conformance.* From b010fbd0df75d03885db853ff126ab0b58c7afdd Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Fri, 27 Mar 2026 17:02:39 -0500 Subject: [PATCH 23/49] =?UTF-8?q?docs:=20architecture=20review=20pass=20?= =?UTF-8?q?=E2=80=94=20operational=20models,=20store=20model,=20notificati?= =?UTF-8?q?on=20pipeline,=20and=20final=20gap=20closure?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 161 ++++++++- .../data-model/control-plane-components.md | 266 ++++++++++++++ .../architecture/data-model/dcm-federation.md | 2 +- .../data-model/deployment-redundancy.md | 188 +++++++++- .../architecture/data-model/four-states.md | 117 +++++++ .../data-model/policy-profiles.md | 28 ++ .../data-model/resource-grouping.md | 84 ++++- .../data-model/resource-type-hierarchy.md | 46 ++- .../data-model/universal-groups.md | 86 ++++- .../data-model/webhooks-messaging.md | 79 +++++ .../specifications/flow-gui-spec.md | 174 ++++++++++ .../specifications/opa-integration-spec.md | 324 ++++++++++++++++++ content/docs/taxonomy.md | 11 + 13 files changed, 1554 insertions(+), 12 deletions(-) create mode 100644 content/docs/architecture/data-model/control-plane-components.md create mode 100644 content/docs/architecture/specifications/flow-gui-spec.md create mode 100644 content/docs/architecture/specifications/opa-integration-spec.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 965d4c5..11d728d 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -417,6 +417,27 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | Model | Description | Example | |-------|-------------|---------| | **Allocation** | Provider retains internal ownership. Consumer owns the Entity (the allocation). Provider has reclaim rights on decommission. | VM, Container, IP Address | +| **Cost Analysis Information Provider** | Specialized Information Provider supplying cost estimates, placement cost signals, cost actuals, and budget alerts; DCM provides input data; provider performs calculations | +| **Orchestrator** | DCM control plane component sequencing multi-step workflows; conducts request lifecycle pipeline; executes named workflow artifacts | +| **Workflow (DCM)** | First-class versioned GitOps artifact defining named sequence of operations; included in Profiles; triggered manually/scheduled/event/policy | +| **Provider Catalog Item** | What a specific Service Provider offers consumers: specific resource allocation or process with cost, availability, SLAs; linked to Resource Type Specification version | +| **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | +| **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | +| **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Request Orchestrator** | DCM control plane event bus; routes lifecycle events to Policy Engine; coordinates pipeline via event-condition-action; does not contain hardcoded pipeline logic | +| **Cost Analysis Component** | Internal DCM control plane component; three functions: pre-request estimation, placement input, ongoing attribution; not a billing system; not a provider type | +| **Module** | DCM capability extension adding new functions; distinct from Profile (which configures behavior) | +| **orchestration_flow** | Policy Group concern_type for static sequential flows; ordered: true; both static and dynamic flows compose through the same Policy Engine | +| **payload_type** | Closed vocabulary of event types the Request Orchestrator publishes; policies pattern-match on payload type + state | +| **OPA integration** | Reference implementation for Mode 3 Policy Providers; DCM payload as OPA input document; built-in Rego functions provided by DCM | +| **Flow GUI** | Visual policy composer and orchestration manager; execution graph view, policy canvas, shadow mode dashboard, flow simulation | +| **__platform__** | Immutable system Tenant owning DCM control plane resources; created at bootstrap before Policy Engine comes online | +| **__transitional__** | Immutable system Tenant holding brownfield entities during INGEST phase | +| **bootstrap manifest** | Signed manifest declaring initial system Tenants, bootstrap admin, and initial profile; hash-verified at every DCM startup | +| **cross_tenant_authorization** | DCMGroup with this group_class formally grants one Tenant access to another's resources; has lifecycle (duration, renewal, revocation); revocation places active allocations in PENDING_REVIEW | +| **drift_criticality** | Field-level property in Resource Type Spec (low/medium/high/critical); combined with change magnitude to produce drift severity | +| **Ingress API** | Infrastructure-layer entry point for all inbound DCM requests; sets ingress block; routes to Consumer/Provider/Admin API surfaces | +| **Provider Catalog Item** | Provider-specific instantiation of a Resource Type Specification; what consumers actually request; distinct from the Resource Type Specification itself | | **Recovery Policy** | Formal DCM policy type mapping trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions; same authoring model as GateKeeper/Validation/Transformation | | **recovery_posture** | Fifth Policy Group concern_type governing failure and ambiguity response; binds a recovery profile group to the deployment | | **DRIFT_RECONCILE** | Recovery action: schedule discovery; let drift detection resolve actual state | @@ -2469,7 +2490,28 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | Former Term | Replacement | Meaning | |-------------|-------------|---------| -| Shore | **Recovery Policy** | Formal DCM policy type mapping trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions; same authoring model as GateKeeper/Validation/Transformation | +| Shore | **Cost Analysis Information Provider** | Specialized Information Provider supplying cost estimates, placement cost signals, cost actuals, and budget alerts; DCM provides input data; provider performs calculations | +| **Orchestrator** | DCM control plane component sequencing multi-step workflows; conducts request lifecycle pipeline; executes named workflow artifacts | +| **Workflow (DCM)** | First-class versioned GitOps artifact defining named sequence of operations; included in Profiles; triggered manually/scheduled/event/policy | +| **Provider Catalog Item** | What a specific Service Provider offers consumers: specific resource allocation or process with cost, availability, SLAs; linked to Resource Type Specification version | +| **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | +| **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | +| **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Request Orchestrator** | DCM control plane event bus; routes lifecycle events to Policy Engine; coordinates pipeline via event-condition-action; does not contain hardcoded pipeline logic | +| **Cost Analysis Component** | Internal DCM control plane component; three functions: pre-request estimation, placement input, ongoing attribution; not a billing system; not a provider type | +| **Module** | DCM capability extension adding new functions; distinct from Profile (which configures behavior) | +| **orchestration_flow** | Policy Group concern_type for static sequential flows; ordered: true; both static and dynamic flows compose through the same Policy Engine | +| **payload_type** | Closed vocabulary of event types the Request Orchestrator publishes; policies pattern-match on payload type + state | +| **OPA integration** | Reference implementation for Mode 3 Policy Providers; DCM payload as OPA input document; built-in Rego functions provided by DCM | +| **Flow GUI** | Visual policy composer and orchestration manager; execution graph view, policy canvas, shadow mode dashboard, flow simulation | +| **__platform__** | Immutable system Tenant owning DCM control plane resources; created at bootstrap before Policy Engine comes online | +| **__transitional__** | Immutable system Tenant holding brownfield entities during INGEST phase | +| **bootstrap manifest** | Signed manifest declaring initial system Tenants, bootstrap admin, and initial profile; hash-verified at every DCM startup | +| **cross_tenant_authorization** | DCMGroup with this group_class formally grants one Tenant access to another's resources; has lifecycle (duration, renewal, revocation); revocation places active allocations in PENDING_REVIEW | +| **drift_criticality** | Field-level property in Resource Type Spec (low/medium/high/critical); combined with change magnitude to produce drift severity | +| **Ingress API** | Infrastructure-layer entry point for all inbound DCM requests; sets ingress block; routes to Consumer/Provider/Admin API surfaces | +| **Provider Catalog Item** | Provider-specific instantiation of a Resource Type Specification; what consumers actually request; distinct from the Resource Type Specification itself | +| **Recovery Policy** | Formal DCM policy type mapping trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions; same authoring model as GateKeeper/Validation/Transformation | | **recovery_posture** | Fifth Policy Group concern_type governing failure and ambiguity response; binds a recovery profile group to the deployment | | **DRIFT_RECONCILE** | Recovery action: schedule discovery; let drift detection resolve actual state | | **DISCARD_AND_REQUEUE** | Recovery action: best-effort cleanup; new request cycle created immediately | @@ -3153,7 +3195,93 @@ OPS-010 through OPS-019. Key: cancellation always best-effort (OPS-011); recover --- -## SECTION 50 โ€” PERSONAS +## SECTION 50 โ€” GROUPS 3, 4, AND 5: FINAL ARCHITECTURE GAPS + +### 50.1 Cost Analysis โ€” Information Provider Model (Group 3) + +Cost Analysis is an **Information Provider** โ€” not a built-in DCM component. DCM does not calculate costs; it provides input data and consumes cost signals. Integration target: Red Hat Cost Management or any external cost management platform. + +**DCM provides to Cost Analysis:** entity lifecycle events (realized/suspended/decommissioned with billing_state), provider catalog item declared costs, provider capacity utilization, request payload previews for pre-request estimates. + +**Cost Analysis provides to DCM:** pre-request cost estimates (pulled by service catalog and CI pipeline), placement cost signals (pulled during placement Step 4), cost actuals (pushed after billing period), budget alerts (pushed when thresholds approached). + +**Fallback chain:** Cost Analysis provider โ†’ static declared cost (provider registration) โ†’ resource type default estimate โ†’ no estimate. Staleness thresholds govern fallback (PT24H standard; PT1H sovereign). + +CMP-001, CMP-002. + +### 50.2 The Orchestrator โ€” Workflow Engine (Group 3) + +The **Orchestrator** sequences and executes multi-step DCM operations. Primary use case: the request lifecycle pipeline. General use case: any named workflow artifact. + +**Workflows are first-class DCM artifacts** โ€” versioned, GitOps-managed, same lifecycle as all other artifacts. Workflows can be triggered: manually, scheduled (cron), event-triggered, or by policy output. Profiles include workflow bindings โ€” activating `fsi` profile automatically activates compliance, drift remediation, sovereignty verification, and audit verification workflows. + +**Step types:** discovery_trigger, policy_evaluation, policy_evaluation_batch, notification_trigger, provider_dispatch, wait_for_event, wait_for_condition, report_generation, entity_state_transition, sub_workflow, parallel, human_approval, cost_analysis_query, data_transform. + +**Request lifecycle pipeline is a built-in system workflow** โ€” cannot be deactivated; can be extended via Policy Groups. + +**Workflow execution records** have UUID, state machine, step results, and full audit trail. CMP-003, CMP-004, CMP-005. + +### 50.3 Ingress API vs Consumer API (Group 5 fix) + +The **Ingress API** is the network infrastructure layer (API Gateway) โ€” TLS termination, auth validation, rate limiting, ingress block population, routing. It routes to three logical API surfaces on distinct path prefixes: +- `/api/v1/` โ†’ **Consumer API** (catalog, requests, resource management, audit) +- `/api/v1/provider/` โ†’ **Provider API** (callbacks, update notifications, cancellation) +- `/api/v1/admin/` โ†’ **Admin API** (discovery triggers, orphan review, tenant management) + +The Ingress API is not a separate service โ€” it is the API Gateway component. CMP-007. + +### 50.4 Consumer Rate Limiting and Quota Model (Group 5 fix) + +**Request rate quotas** โ€” enforced at Ingress API level per actor; returns 429 with Retry-After. Configured in platform-domain layer. + +**Resource quotas** โ€” enforced by GateKeeper policies at Step 5 (pre-placement). No hardcoded mechanism โ€” quotas are declared policies. Quota exceeded โ†’ QUOTA_EXCEEDED GateKeeper rejection. Quota increase requests submitted via `Process.QuotaIncreaseRequest` catalog item โ†’ Orchestrator routes to platform admin for approval โ†’ GateKeeper policy updated. + +CMP-006. + +### 50.5 Drift Severity โ€” Three-Tier Classification (Group 4 fix) + +**Tier 1 โ€” Field criticality** (declared in Resource Type Spec): `drift_criticality: minor|significant|critical` per field. + +**Tier 2 โ€” Magnitude thresholds** (system layer, overridable at platform/tenant): >50% change on significant field upgrades to critical; 10+ changed items upgrades minor to significant. + +**Tier 3 โ€” Provider and consumer injection:** Providers suggest severity in update notifications (raise only). Consumers override sensitivity on specific entities (raise or lower โ€” entity owner controls their resource's sensitivity). + +**Resolution:** highest severity from all three tiers wins. + +### 50.6 Cross-Tenant Authorization Lifecycle (Group 4 fix) + +`cross_tenant_authorization` is a DCMGroup with `group_class: cross_tenant_authorization`. Created by: granting Tenant admin (standard), Platform Admin (emergency), or pre-authorization policy (automated). Has declared duration or perpetual. On revocation: all active allocations/stakes under that authorization enter PENDING_REVIEW; notifications to both Tenant admins and affected resource owners; PT72H default resolution deadline; on_deadline_exceeded recovery policy fires. CTX-001 through CTX-004. + +### 50.7 Bootstrap Tenant Creation Sequence (Group 4 fix) + +Three foundation Tenants created during bootstrap (declared in bootstrap manifest, cannot be decommissioned): +- `__platform__` โ€” owns DCM's own control plane resources +- `__transitional__` โ€” holds brownfield entities during ingestion +- `__system__` โ€” owns system-level artifacts + +Bootstrap sequence: verify manifest โ†’ initialize storage โ†’ create foundation Tenants โ†’ create initial Platform Admin actor โ†’ activate system layers/policies/recovery profiles โ†’ register built-in providers โ†’ ready. RED-016. + +### 50.8 Catalog Item vs Resource Type Clarification (Group 5 fix) + +**Resource Type** โ€” classification category; vendor-neutral; declares field schema expectations; groups catalog items for portability. + +**Resource Type Specification** โ€” versioned formal definition in registry; providers implement against this. + +**Provider Catalog Item** โ€” what a specific provider offers to consumers: specific options, cost, availability, SLAs, linked to a Resource Type Specification version. Can be a resource allocation OR a process (automation job, playbook, pipeline). *Consumers request by Resource Type; DCM resolves to a catalog item.* + +Anti-vocabulary: never say "catalog item" when you mean "resource type specification." Never say "resource type" when you mean a specific offering. + +### 50.9 BBQ-001 and Federation Routing Reconciliation (Group 5 fix) + +These operate at different scopes โ€” complementary not conflicting: +- **DCM-010 sovereignty pre-filter (Hub level):** Which Regional DCMs are eligible for this request? +- **BBQ-001 check (Regional DCM level):** Is this Mode 4 Policy Provider endpoint within my sovereignty boundary? + +Hub selects Regional DCM using DCM-010. Regional DCM applies BBQ-001 for its own Mode 4 queries. Hub sovereignty pre-filter does NOT bypass Regional DCM's BBQ-001 check. + +--- + +## SECTION 51 โ€” PERSONAS | Persona | Primary Concern | |---------|----------------| @@ -3170,7 +3298,7 @@ OPS-010 through OPS-019. Key: cancellation always best-effort (OPS-011); recover --- -## SECTION 51 โ€” TERMINOLOGY GLOSSARY +## SECTION 52 โ€” TERMINOLOGY GLOSSARY | Term | Definition | |------|-----------| @@ -3233,6 +3361,27 @@ OPS-010 through OPS-019. Key: cancellation always best-effort (OPS-011); recover | **Raft** | Consensus protocol used by Commit Log (etcd) for quorum writes; guarantees durability even if minority of replicas fail | | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class | | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation | +| **Cost Analysis Information Provider** | Specialized Information Provider supplying cost estimates, placement cost signals, cost actuals, and budget alerts; DCM provides input data; provider performs calculations | +| **Orchestrator** | DCM control plane component sequencing multi-step workflows; conducts request lifecycle pipeline; executes named workflow artifacts | +| **Workflow (DCM)** | First-class versioned GitOps artifact defining named sequence of operations; included in Profiles; triggered manually/scheduled/event/policy | +| **Provider Catalog Item** | What a specific Service Provider offers consumers: specific resource allocation or process with cost, availability, SLAs; linked to Resource Type Specification version | +| **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | +| **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | +| **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Request Orchestrator** | DCM control plane event bus; routes lifecycle events to Policy Engine; coordinates pipeline via event-condition-action; does not contain hardcoded pipeline logic | +| **Cost Analysis Component** | Internal DCM control plane component; three functions: pre-request estimation, placement input, ongoing attribution; not a billing system; not a provider type | +| **Module** | DCM capability extension adding new functions; distinct from Profile (which configures behavior) | +| **orchestration_flow** | Policy Group concern_type for static sequential flows; ordered: true; both static and dynamic flows compose through the same Policy Engine | +| **payload_type** | Closed vocabulary of event types the Request Orchestrator publishes; policies pattern-match on payload type + state | +| **OPA integration** | Reference implementation for Mode 3 Policy Providers; DCM payload as OPA input document; built-in Rego functions provided by DCM | +| **Flow GUI** | Visual policy composer and orchestration manager; execution graph view, policy canvas, shadow mode dashboard, flow simulation | +| **__platform__** | Immutable system Tenant owning DCM control plane resources; created at bootstrap before Policy Engine comes online | +| **__transitional__** | Immutable system Tenant holding brownfield entities during INGEST phase | +| **bootstrap manifest** | Signed manifest declaring initial system Tenants, bootstrap admin, and initial profile; hash-verified at every DCM startup | +| **cross_tenant_authorization** | DCMGroup with this group_class formally grants one Tenant access to another's resources; has lifecycle (duration, renewal, revocation); revocation places active allocations in PENDING_REVIEW | +| **drift_criticality** | Field-level property in Resource Type Spec (low/medium/high/critical); combined with change magnitude to produce drift severity | +| **Ingress API** | Infrastructure-layer entry point for all inbound DCM requests; sets ingress block; routes to Consumer/Provider/Admin API surfaces | +| **Provider Catalog Item** | Provider-specific instantiation of a Resource Type Specification; what consumers actually request; distinct from the Resource Type Specification itself | | **Recovery Policy** | Formal DCM policy type mapping trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions; same authoring model as GateKeeper/Validation/Transformation | | **recovery_posture** | Fifth Policy Group concern_type governing failure and ambiguity response; binds a recovery profile group to the deployment | | **DRIFT_RECONCILE** | Recovery action: schedule discovery; let drift detection resolve actual state | @@ -3434,7 +3583,7 @@ OPS-010 through OPS-019. Key: cancellation always best-effort (OPS-011); recover --- -## SECTION 52 โ€” OPEN QUESTIONS +## SECTION 53 โ€” OPEN QUESTIONS These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -3531,7 +3680,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 53 โ€” DOCUMENTATION STRUCTURE +## SECTION 54 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -3579,7 +3728,7 @@ content/ --- -## SECTION 54 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 55 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md new file mode 100644 index 0000000..c066713 --- /dev/null +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -0,0 +1,266 @@ +--- +title: "Control Plane Components" +type: docs +weight: 25 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Operational Models](24-operational-models.md) | [Policy Profiles](14-policy-profiles.md) + +--- + +## 1. Purpose + +This document formally defines the DCM control plane components that are referenced throughout the data model documents but not previously specified in detail. Two components are defined here: + +1. **The Request Orchestrator** โ€” the event bus and coordinator of the request lifecycle pipeline +2. **The Cost Analysis Component** โ€” the internal DCM component that provides cost signals for placement, catalog, and attribution + +--- + +## 2. The Request Orchestrator + +### 2.1 Role + +The Request Orchestrator is the **event bus and pipeline coordinator** for all DCM request lifecycle operations. It does not perform any pipeline work itself โ€” it listens for events, evaluates which components need to act on them, and routes work to the appropriate components. + +The Request Orchestrator embodies DCM's **data-driven, policy-triggered orchestration model**: the pipeline is not a fixed procedural sequence. It is a cascade of event-condition-action responses, where policies define what happens when specific payload states are observed. + +### 2.2 Data-Driven Orchestration Principle + +**Policies ARE the orchestration.** The Request Orchestrator does not contain hardcoded pipeline logic. It publishes events to the Policy Engine; policies match on payload type and state; policy actions produce new payload states; those new states trigger further policy evaluations. + +This means: +- Adding a new pipeline step = writing a new policy (no code change) +- Removing a step = deactivating a policy +- Changing when a step fires = changing a policy condition +- A static workflow (e.g., always require human approval for prod VMs) = a policy that always matches for those conditions +- A dynamic workflow (e.g., route to different approval processes based on cost) = a policy with conditional logic + +Static and dynamic flows compose naturally โ€” a static policy defines a guaranteed step; a dynamic policy defines a conditional step. Both are expressed as policies, evaluated by the same engine, producing deterministic outcomes. + +**Determinism guarantee:** Dynamic execution remains deterministic because: +- The payload type vocabulary is a closed set +- Policy evaluation order within a domain level is deterministic (domain precedence) +- The payload mutation model is immutable (each policy produces a new payload version) +- The same input state always produces the same output state + +### 2.3 The Payload Type Vocabulary + +Every event in DCM carries a payload with a declared type. Policies pattern-match on these types. The payload type vocabulary is the foundational contract of the orchestration model. + +```yaml +payload_types: + # Request lifecycle + request.initiated: # consumer submitted a request + request.intent_captured: # Intent State written + request.layers_assembled: # layer assembly complete + request.policies_evaluated: # all active policies evaluated + request.placement_complete: # provider selected + request.dispatched: # sent to provider + request.realized: # provider confirmed realization + request.failed: # terminal failure + request.cancelled: # cancelled + + # Provider update + provider_update.received: # provider submitted update notification + provider_update.evaluated: # policy evaluation complete + provider_update.accepted: # accepted; Realized State updating + provider_update.rejected: # rejected; becomes drift + + # Drift and discovery + discovery.cycle_complete: + drift.detected: + drift.resolved: + + # Recovery + recovery.timeout_fired: + recovery.late_response: + recovery.compensation_triggered: + + # Governance + policy.activated: + layer.updated: + profile.changed: +``` + +### 2.4 Event Routing Model + +``` +Event published: { type: "request.initiated", payload: {...}, entity_uuid: X } + โ”‚ + โ–ผ Request Orchestrator receives event + โ”‚ Routes to Policy Engine: "evaluate all policies matching request.initiated" + โ”‚ + โ–ผ Policy Engine evaluates in domain precedence order + โ”‚ Matching policies fire; payload mutations accumulated + โ”‚ New payload state produced: { type: "request.layers_assembled", ... } + โ”‚ + โ–ผ Request Orchestrator receives new event + โ”‚ Routes to Policy Engine for next evaluation cycle + โ”‚ (parallel if no data dependencies between active policies) + โ”‚ + โ–ผ Continues until terminal state (request.realized or request.failed) +``` + +**Parallel execution:** Policies that have no data dependencies on each other evaluate concurrently. The Request Orchestrator tracks dependency declarations between policies and executes in parallel where safe. + +### 2.5 Static Flow Support + +Organizations that require guaranteed sequential flows express them as ordered policy sets: + +```yaml +static_flow_policy_group: + handle: "org/flows/prod-vm-approval-flow" + concern_type: orchestration_flow + ordered: true # policies execute in declared sequence, not parallel + policies: + - step: 1 + handle: "org/policies/cost-check" + condition: "request.initiated AND resource_type=Compute.VirtualMachine AND tenant.profile=prod" + on_fail: halt + - step: 2 + handle: "org/policies/manager-approval" + condition: "request.cost_estimated > 500" + on_fail: halt + - step: 3 + handle: "org/policies/security-review" + condition: "always" + on_fail: halt +``` + +A static flow is a Policy Group with `concern_type: orchestration_flow` and `ordered: true`. The Request Orchestrator respects the declared order. Static flows integrate with dynamic policies โ€” a dynamic policy can fire alongside the static flow steps. + +### 2.6 Request Orchestrator Responsibilities + +| Responsibility | Description | +|----------------|-------------| +| Event routing | Receive all request lifecycle events; route to appropriate components | +| Pipeline coordination | Sequence component interactions per data dependencies | +| Timeout monitoring | Track dispatch_timeout and assembly_timeout; fire recovery triggers | +| Dependency resolution | For compound services, sequence component provisioning per dependency graph | +| Status tracking | Maintain current status of all in-flight requests; respond to status queries | +| Recovery coordination | On timeout/failure, invoke Recovery Policy evaluation | + +--- + +## 3. The Cost Analysis Component + +### 3.1 Role + +The Cost Analysis Component is an **internal DCM control plane component** that provides cost signals to other components. It is not a billing system and not a provider type. It does not manage financial transactions, produce invoices, or serve as the authoritative financial record. It provides cost *signals* that DCM uses for placement decisions, pre-request estimation, and ongoing attribution. + +The authoritative billing record lives in the organization's financial system. A billing system can register as an Information Provider to push authoritative cost data back into DCM for attribution records. + +### 3.2 Three Cost Functions + +**Function 1 โ€” Pre-request cost estimation:** +Given a catalog item and assembled field values, compute the estimated lifecycle cost. Used by: +- Service Catalog describe endpoint (consumer sees cost before requesting) +- CI pipeline pre-validation (cost estimate in PR comment) +- Placement engine tie-breaker step 4 (cheapest eligible provider) + +**Function 2 โ€” Placement cost input:** +During Step 6 placement, provide current cost data per eligible provider for the requested resource type. If Cost Analysis data is unavailable, the placement engine falls back to static declared costs per REG-011. + +**Function 3 โ€” Ongoing cost attribution:** +For realized entities, track ongoing consumption and attribute costs to the owning Tenant. Consumed by OBS-005 (consumer cost view) and the resource describe endpoint (`estimated_cost_per_hour` field). + +### 3.3 Cost Data Sources + +The Cost Analysis Component ingests cost data from two sources, following the REG-011 hybrid model: + +```yaml +cost_data_sources: + static: + source: provider_registration # declared at provider registration time + update_frequency: manual # updated when rates change + fields: [capex_per_unit, opex_per_unit_per_hour, currency] + + dynamic: + source: external_cost_api # external billing API or cloud pricing API + registered_as: information_provider + query_interval: PT1H + fallback: static # use static if dynamic unavailable + fallback_max_age: PT24H +``` + +### 3.4 Cost Estimation Model + +```yaml +cost_estimation_request: + catalog_item_uuid: + assembled_fields: + cpu_count: 4 + memory_gb: 8 + storage_gb: 100 + tenant_uuid: + requested_duration: P30D # optional; lifecycle estimate + +cost_estimation_response: + estimated_cost: + per_hour: 0.32 + per_month: 230.40 + lifecycle_estimate: 691.20 # if requested_duration provided + currency: USD + confidence: high # high: current Cost Analysis data + # medium: data > PT1H old + # low: static fallback + breakdown: + - component: compute + per_hour: 0.28 + - component: ip_allocation + per_hour: 0.04 + cost_data_timestamp: +``` + +### 3.5 Cost Attribution for Realized Entities + +```yaml +entity_cost_attribution: + entity_uuid: + tenant_uuid: + billing_state: billable # billable | non_billable | reduced_rate + current_rate: + per_hour: 0.32 + currency: USD + rate_effective_since: + monthly_accrual: 230.40 + cost_data_source: cost_analysis # cost_analysis | static | unknown +``` + +### 3.6 Integration with Placement Engine + +The placement engine queries Cost Analysis at step 4 of the tie-breaking hierarchy: + +``` +Step 4 โ€” Cost Analysis (if available and determinable): + Query Cost Analysis for each eligible provider + Cost Analysis returns: estimated cost per unit per provider + Placement engine prefers lowest cost among equally-ranked candidates + If Cost Analysis unavailable: skip step 4; proceed to step 5 + # Cost Analysis unavailability never blocks placement +``` + +--- + +## 4. Related Policies + +| Policy | Rule | +|--------|------| +| `CTL-001` | The Request Orchestrator is the single event bus for all request lifecycle events. No component communicates directly with another component outside of events published to the Request Orchestrator. | +| `CTL-002` | Policies ARE the orchestration. The Request Orchestrator does not contain hardcoded pipeline logic. Pipeline behavior is modified by adding, removing, or changing policies โ€” not by changing the orchestrator. | +| `CTL-003` | Dynamic and static flows compose naturally. Static flows are Policy Groups with concern_type: orchestration_flow and ordered: true. Both types are evaluated by the same Policy Engine. | +| `CTL-004` | Cost Analysis is not a billing system. It provides cost signals for placement and attribution. The authoritative billing record lives in the organization's financial system, which may register as an Information Provider. | +| `CTL-005` | Cost Analysis unavailability never blocks placement. The placement engine falls back to static declared costs per REG-011 and skips the Cost Analysis tie-breaking step. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/dcm-federation.md b/content/docs/architecture/data-model/dcm-federation.md index 6cdff51..56be7b9 100644 --- a/content/docs/architecture/data-model/dcm-federation.md +++ b/content/docs/architecture/data-model/dcm-federation.md @@ -1,7 +1,7 @@ --- title: "DCM Federation and Cross-Instance Coordination" type: docs -weight: 21 +weight: 22 --- > **โš ๏ธ Active Development Notice** diff --git a/content/docs/architecture/data-model/deployment-redundancy.md b/content/docs/architecture/data-model/deployment-redundancy.md index 47e5c1f..03006af 100644 --- a/content/docs/architecture/data-model/deployment-redundancy.md +++ b/content/docs/architecture/data-model/deployment-redundancy.md @@ -1,7 +1,7 @@ --- -title: "Deployment and Redundancy" +title: "Deployment and Redundancy Model" type: docs -weight: 16 +weight: 17 --- > **โš ๏ธ Active Development Notice** @@ -588,6 +588,190 @@ DCM's own deployment is a DCM-managed resource subject to the same drift detecti | `RED-015` | DCM's own deployment is a DCM-managed resource subject to the same drift detection as any other resource. Bootstrap manifest hash verification (RED-011) provides independent verification. Audit Store hash chain breaks are detectable externally. | + +--- + +## 9. Bootstrap Tenant Creation Sequence + +### 9.1 The Bootstrap Problem + +DCM requires every entity to belong to exactly one Tenant. But during initial deployment, no Tenants exist. The bootstrap sequence resolves this by creating the foundational Tenants as part of DCM startup, declared in the bootstrap manifest. + +### 9.2 The Three Foundation Tenants + +The bootstrap manifest declares three system Tenants that are created before any consumer can submit requests: + +```yaml +bootstrap_tenants: + - handle: "__platform__" + display_name: "DCM Platform" + purpose: "Owns DCM's own control plane resources (components, stores, providers)" + automatically_created: true + cannot_be_decommissioned: true + + - handle: "__transitional__" + display_name: "Transitional" + purpose: "Holds brownfield entities during ingestion before promotion to a real Tenant" + automatically_created: true + cannot_be_decommissioned: true + + - handle: "__system__" + display_name: "System" + purpose: "Owns system-level artifacts (system layers, system policies, system workflows)" + automatically_created: true + cannot_be_decommissioned: true +``` + +### 9.3 Bootstrap Startup Sequence + +``` +DCM starts + โ”‚ + โ–ผ Step 1: Verify bootstrap manifest hash and signature + โ”‚ + โ–ผ Step 2: Initialize storage providers + โ”‚ GitOps stores initialized + โ”‚ Audit Store initialized + โ”‚ Commit Log initialized + โ”‚ + โ–ผ Step 3: Create foundation Tenants (if not already existing) + โ”‚ __platform__, __transitional__, __system__ + โ”‚ + โ–ผ Step 4: Create initial Platform Admin actor + โ”‚ Declared in bootstrap manifest + โ”‚ Assigned to __platform__ Tenant + โ”‚ Given platform_admin role + โ”‚ + โ–ผ Step 5: Activate system domain layers and policies + โ”‚ System layers loaded from GitOps store + โ”‚ System policies activated + โ”‚ Built-in recovery profiles activated + โ”‚ + โ–ผ Step 6: Register built-in providers + โ”‚ Built-in Auth Provider + โ”‚ Search Index Storage Provider + โ”‚ Audit Store Storage Provider + โ”‚ (All owned by __platform__ Tenant) + โ”‚ + โ–ผ Step 7: DCM ready + Consumer API, Provider API, Admin API accepting requests + Platform Admin can now create organization Tenants + Organization Tenants can request resources +``` + +### 9.4 System Policy + +| Policy | Rule | +|--------|------| +| `RED-016` | The three foundation Tenants (__platform__, __transitional__, __system__) are created during bootstrap and cannot be decommissioned. All DCM control plane resources are owned by __platform__. All brownfield ingested entities enter __transitional__ before promotion. | + + + +--- + +## 9. Bootstrap Sequence and Initial Tenant Creation + +### 9.1 The Bootstrap Problem + +The DCM data model requires every entity to be owned by a Tenant. But Tenants are themselves DCM entities. The bootstrap sequence defines how the initial Tenants and platform admin actor are created before DCM can accept external requests. + +### 9.2 Bootstrap Manifest + +The bootstrap manifest (RED-011) declares the initial state required for DCM to start. It includes: + +```yaml +bootstrap_manifest: + version: "1.0.0" + signed_by: + + # Initial system Tenants (created before any external requests) + system_tenants: + - uuid: + handle: "__platform__" + display_name: "DCM Platform" + description: "System Tenant owning DCM's own control plane resources" + immutable: true # cannot be decommissioned or modified by regular operators + + - uuid: + handle: "__transitional__" + display_name: "Transitional" + description: "Holding Tenant for brownfield ingestion (INGEST phase)" + immutable: true + + # Initial platform admin actor + bootstrap_admin: + uuid: + username: "dcm-bootstrap-admin" + auth_provider: builtin + roles: [platform_admin] + credential_ref: + # This credential is rotated on first login + + # Active profile for initial deployment + initial_profile: + deployment_posture: minimal # or as declared; can be changed post-bootstrap + compliance_domains: [] + + # Bootstrap admin's initial Tenant + initial_tenant: + uuid: + handle: "org-default" + display_name: "Default Organization Tenant" + owned_by: bootstrap_admin +``` + +### 9.3 Bootstrap Sequence + +``` +DCM starts โ†’ bootstrap manifest hash verified (RED-011) + โ”‚ + โ–ผ System Tenants created (before Policy Engine active): + โ”‚ __platform__ Tenant โ€” owns DCM control plane resources + โ”‚ __transitional__ Tenant โ€” brownfield ingestion holding + โ”‚ These are created by the bootstrap process itself, not through the request pipeline + โ”‚ + โ–ผ Bootstrap admin actor created + โ”‚ Auth Provider initialized with bootstrap credential + โ”‚ Platform Admin role assigned + โ”‚ + โ–ผ Initial profile activated + โ”‚ Deployment posture policies loaded + โ”‚ Compliance domain policies loaded (if declared) + โ”‚ + โ–ผ Policy Engine comes online + โ”‚ All subsequent operations go through the standard request pipeline + โ”‚ + โ–ผ Bootstrap admin creates the initial organization Tenant (optional) + โ”‚ First real request through the pipeline + โ”‚ Creates the initial production Tenant for organizational resources + โ”‚ + โ–ผ Bootstrap admin credential rotation notification sent + โ”‚ Bootstrap credential must be rotated on first login + โ”‚ After rotation, bootstrap_admin becomes a standard platform admin actor + โ”‚ + โ–ผ DCM accepts external requests +``` + +### 9.4 System Tenants + +The `__platform__` and `__transitional__` Tenants are created by the bootstrap process and are immutable: + +| System Tenant | Purpose | Who can modify | +|--------------|---------|---------------| +| `__platform__` | Owns DCM's own control plane resources | Platform Admin (restricted operations only) | +| `__transitional__` | Brownfield ingestion holding area | Ingestion pipeline only | + +These Tenants are exempt from the normal Tenant decommission workflow โ€” they cannot be decommissioned while DCM is operational. + +### 9.5 System Policy + +| Policy | Rule | +|--------|------| +| `BOOT-001` | The __platform__ and __transitional__ system Tenants are created by the bootstrap process before the Policy Engine comes online. They are immutable and cannot be decommissioned while DCM is running. | +| `BOOT-002` | The bootstrap admin credential must be rotated on first login. The bootstrap manifest declares the initial credential reference only; the credential itself is managed by the Credential Provider. | +| `BOOT-003` | After bootstrap, all Tenant creation and modification goes through the standard request pipeline. The bootstrap process is a one-time operation. | + + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md index 28ffafc..4f47ad0 100644 --- a/content/docs/architecture/data-model/four-states.md +++ b/content/docs/architecture/data-model/four-states.md @@ -491,6 +491,123 @@ Policy Engine evaluates drift Audit Store records drift event with full provenance ``` + +### 6.3 Drift Severity Classification + +Drift severity is determined by combining three independent tiers. The final severity is the highest tier that applies. + +**Tier 1 โ€” Field criticality (declared in Resource Type Specification):** + +```yaml +resource_type_spec: + fields: + display_name: + drift_criticality: minor # non-functional change + cpu_count: + drift_criticality: significant + memory_gb: + drift_criticality: significant + security_group_ids: + drift_criticality: critical # security-relevant change + firewall_rules: + drift_criticality: critical +``` + +**Tier 2 โ€” Profile/layer magnitude thresholds:** + +```yaml +# system/drift/severity-thresholds layer (overridable at platform/tenant domain) +drift_severity_thresholds: + significant_field_magnitude_upgrade: + percentage_change_threshold: 50 # >50% change upgrades significant โ†’ critical + minor_field_magnitude_upgrade: + item_count_threshold: 10 # 10+ changed items upgrades minor โ†’ significant +``` + +**Tier 3 โ€” Provider and consumer injection:** + +Providers may suggest severity in update notifications (raise only): +```yaml +provider_drift_hint: + field: memory_gb + suggested_severity: critical + reason: "Memory decrease on running workload risks OOM" +``` + +Consumers may override sensitivity on specific entities (raise or lower): +```yaml +entity: + drift_sensitivity_overrides: + - field: cpu_count + override_criticality: critical + reason: "Production payments workload โ€” any CPU change is critical" +``` + +**Resolution rule:** The Drift Detection component takes the highest severity from all three tiers. Provider injection can raise but not lower the Tier 1/2 result. Consumer injection can raise or lower (entity owner controls their own resource's sensitivity). Profile governs whether consumer lowering is permitted. + + + +### 6.3 Drift Severity Classification + +Drift severity is determined by two independent dimensions declared in the Resource Type Specification โ€” field criticality and change magnitude. The combination produces a deterministic severity classification for any drift event. + +#### Field Criticality (declared per field in Resource Type Spec) + +```yaml +resource_type_spec: + fqn: Compute.VirtualMachine + fields: + display_name: + drift_criticality: low # cosmetic; never affects function + cpu_count: + drift_criticality: medium # affects performance; not security + memory_gb: + drift_criticality: medium + security_group_ids: + drift_criticality: critical # security boundary field; always critical + os_image: + drift_criticality: critical # security posture; always critical + storage_gb: + drift_criticality: medium + network_interface_ids: + drift_criticality: high # connectivity; significant operational impact +``` + +**Criticality levels:** `low | medium | high | critical` + +#### Change Magnitude (profile-governed thresholds) + +```yaml +drift_magnitude_thresholds: + profile_defaults: + standard: + minor: change_pct < 10% + significant: change_pct 10-50% + critical: change_pct > 50% OR value_disappeared OR type_changed + prod: + minor: change_pct < 5% + significant: change_pct 5-25% + critical: change_pct > 25% OR value_disappeared OR type_changed +``` + +#### Severity Matrix + +| Field Criticality | Change Magnitude | Drift Severity | +|------------------|-----------------|----------------| +| low | any | minor | +| medium | minor | minor | +| medium | significant | significant | +| medium | critical | significant | +| high | minor | significant | +| high | significant | significant | +| high | critical | critical | +| critical | any | critical | + +**Unsanctioned changes** (no corresponding Requested State record) are always elevated one severity level above what the matrix produces. A `significant` unsanctioned change becomes `critical`. + +**Multi-field drift:** when multiple fields drift simultaneously, the overall severity is the highest severity among all drifted fields. + + ### 6.2 Unsanctioned Changes A specific category of drift โ€” a change made directly to a resource without a corresponding DCM request. Detected by: diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 0c549bc..dd0b8e6 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -47,6 +47,34 @@ Policy Providers โ€” external authoritative policy sources ## 2. Policy Groups ## 1a. Two-Dimensional Profile Model + +### 1a.0 Profile โ€” One Posture, Multiple Compliance Domains + +**A DCM deployment runs exactly one Deployment Posture and zero or more Compliance Domain Groups simultaneously.** This is the complete profile model. No new concept is needed. + +``` +Active DCM Governance = one Deployment Posture + [zero or more Compliance Domains] + +Examples: + prod + hipaa โ† healthcare production + prod + hipaa + gdpr โ† EU healthcare production + sovereign + fedramp-high + dod-il5 โ† classified federal + standard โ† general enterprise, no compliance overlay + dev + hipaa โ† healthcare development (hipaa policies active, + but operational cost reduced by dev posture) +``` + +**Deployment Postures are mutually exclusive** โ€” you cannot be both `prod` and `dev`. One posture governs the operational characteristics of the entire DCM deployment. + +**Compliance domains are additive** โ€” HIPAA + FSI is valid and common. Each compliance domain group adds its own set of policies and constraints on top of the posture. They do not conflict with each other at the domain level (they govern different aspects of data handling); they may produce policy conflicts at the field level, which are resolved through the standard policy conflict resolution process. + +**The dev posture and compliance domains** โ€” applying `dev` posture to a HIPAA-scoped deployment does not remove HIPAA obligations. It relaxes the *operational cost* of meeting them: less redundancy, shorter retention windows where permitted, advisory enforcement where HIPAA allows flexibility. The HIPAA compliance domain group remains active and its mandatory controls remain enforced. + +**Modules vs Profiles** โ€” DCM uses both concepts with distinct meanings: +- A **Profile** is a governance configuration: it declares how DCM behaves and what operational and compliance requirements apply. +- A **Module** is a capability extension: it adds new functions to DCM (e.g., a HIPAA record validator, a custom resource type). Modules are not profiles and do not configure DCM behavior โ€” they extend what DCM can do. + + ### 1a.1 The Gap in the Original Model The original six profiles (minimal โ†’ sovereign) are organized around **deployment posture** โ€” how strict, how redundant, how governed. But organizations need compliance governance that is orthogonal to posture. A healthcare organization needs HIPAA controls regardless of whether they deploy at `standard` or `sovereign` posture. A payment processor needs PCI-DSS regardless of their redundancy profile. diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md index 32a4895..00631e5 100644 --- a/content/docs/architecture/data-model/resource-grouping.md +++ b/content/docs/architecture/data-model/resource-grouping.md @@ -1,7 +1,7 @@ --- title: "Resource Grouping and Tenancy" type: docs -weight: 7 +weight: 8 --- > **โš ๏ธ Active Development Notice** @@ -296,6 +296,88 @@ custom_group_type_registration: - **Rehydration** โ€” can target a group as the unit of reconstruction - **Field-Level Provenance** โ€” group membership changes are recorded in entity provenance + +--- + +## 10. Cross-Tenant Authorization Lifecycle + +Cross-tenant authorizations are the formal mechanism by which one Tenant grants another Tenant access to a shared resource or allocation. They are DCMGroup instances with `group_class: cross_tenant_authorization`. + +### 10.1 Creation + +Cross-tenant authorizations are created by: + +| Actor | Mechanism | When | +|-------|-----------|------| +| Granting Tenant Admin | Explicit manual grant via Admin API | Normal cross-tenant sharing setup | +| Platform Admin | Emergency authorization | Operational incident; requires dual approval in fsi/sovereign | +| Policy (pre-authorization) | GateKeeper policy auto-creates authorization | Pre-approved sharing patterns | + +```yaml +cross_tenant_authorization: + artifact_metadata: + uuid: + handle: "xta/networkops/appteam/vlan-100" + version: "1.0.0" + status: active + group_class: cross_tenant_authorization + granting_tenant_uuid: + consuming_tenant_uuid: + authorized_resource_types: [Network.VLAN, Network.IPAddressPool] + authorized_entity_uuids: [] # null = all resources of declared types + duration: P1Y # null = perpetual until revoked + expires_at: + created_by: + created_at: + purpose: "AppTeam VMs require VLAN-100 attachment for production network access" +``` + +### 10.2 Duration and Renewal + +- **Fixed duration** (`duration: P1Y`): expires automatically. Notification sent P30D before expiry. Consuming Tenant must request renewal. If not renewed, enters EXPIRING state, then EXPIRED. +- **Perpetual** (`duration: null`): active until explicitly revoked. No automatic expiry. +- **Renewal**: consuming Tenant submits a new authorization request. Granting Tenant approves. New authorization created; old one superseded. + +### 10.3 Revocation + +``` +Granting Tenant admin revokes authorization + โ”‚ + โ–ผ Authorization status โ†’ REVOKED + โ”‚ + โ–ผ Identify active allocations and stakes under this authorization + โ”‚ All cross-tenant allocations/stakes enter PENDING_REVIEW + โ”‚ + โ–ผ Notifications sent to: + โ”‚ Consuming Tenant Admin (action required: migrate or release) + โ”‚ Affected resource owners in consuming Tenant + โ”‚ Platform Admin (informational) + โ”‚ + โ–ผ Resolution deadline: P30D (configurable; P7D for fsi/sovereign) + โ”‚ + โ”œโ”€โ”€ Consuming Tenant releases stakes/allocations โ†’ authorization closes cleanly + โ””โ”€โ”€ Deadline exceeded โ†’ Platform Admin escalation + Policy may declare automatic release on deadline exceeded +``` + +### 10.4 What Happens to Active Allocations on Revocation + +Resources already allocated under a now-revoked authorization are NOT immediately decommissioned โ€” this would break production workloads. Instead: +- The allocation/stake relationship enters PENDING_REVIEW +- The authorization revocation is recorded as the `pending_review_trigger` +- The consuming Tenant has a grace period to migrate or release +- Automatic decommission on deadline exceeded is a policy declaration, not a default + +### 10.5 System Policies + +| Policy | Rule | +|--------|------| +| `XTA-001` | Cross-tenant authorizations require explicit creation by the granting Tenant admin, a platform admin, or a pre-authorization policy. They are never implicitly created. | +| `XTA-002` | Fixed-duration authorizations generate a P30D expiry warning notification. Non-renewal results in EXPIRING then EXPIRED states. | +| `XTA-003` | Authorization revocation places active allocations and stakes in PENDING_REVIEW with a policy-governed grace period. Resources are not automatically decommissioned on revocation. | +| `XTA-004` | Automatic decommission of resources on authorization expiry or revocation requires explicit policy declaration. It is not the default behavior. | + + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-type-hierarchy.md b/content/docs/architecture/data-model/resource-type-hierarchy.md index c30bacb..5532bd5 100644 --- a/content/docs/architecture/data-model/resource-type-hierarchy.md +++ b/content/docs/architecture/data-model/resource-type-hierarchy.md @@ -1,7 +1,7 @@ --- title: "Resource Type Hierarchy and Service Catalog" type: docs -weight: 4 +weight: 5 --- > **โš ๏ธ Active Development Notice** @@ -29,10 +29,54 @@ The hierarchy serves four goals: --- + +### 1a. Precise Vocabulary โ€” Resource Type vs Catalog Item + +These terms are frequently conflated. The distinction is architectural: + +**Resource Type** โ€” the classification category. Groups catalog items for portability and discovery. Vendor-neutral by requirement. Defines the field schema that any provider offering this type must support. Examples: `Compute.VirtualMachine`, `Network.IPAddress`, `Process.AnsiblePlaybook`. + +**Resource Type Specification** โ€” the versioned, formal definition of a Resource Type: field schema, constraints, lifecycle rules, portability classification, and allowed relationship types. Stored in the Resource Type Registry. Providers implement against a specific version. Example: `Compute.VirtualMachine v2.1.0`. + +**Provider Catalog Item** โ€” what a specific Service Provider is offering to consumers. The provider's declaration: "I can fulfill `Compute.VirtualMachine v2.1.0` with these specific options, at this cost, with these availability characteristics, in this region." A catalog item is always linked to a specific Resource Type Specification version. Catalog items can represent resource allocations (a VM, a subnet) or processes (an automation job, a playbook execution, a pipeline run) โ€” anything a provider offers for consumption. + +**The key relationship:** Consumers request by Resource Type (or Resource Type Specification version). DCM resolves to a Provider Catalog Item through the specificity narrowing algorithm. The catalog item is what actually gets provisioned. The resource type is the portable, vendor-neutral expression of intent. + +**Anti-vocabulary update:** Never say "catalog item" when you mean "resource type specification." Never say "resource type" when you mean a specific provider offering โ€” use "catalog item" or "provider catalog item." + + ## 2. The DCM Resource Type Registry DCM maintains an official **Resource Type Registry** โ€” the authoritative source of standard resource type definitions. The registry is the foundation of portability across the DCM ecosystem. + +### 2.1a Catalog Item vs Resource Type Specification โ€” Critical Distinction + +These two terms are frequently conflated throughout the documentation. They are distinct concepts at different levels of the hierarchy: + +**Resource Type Specification (Registry entry):** +- Vendor-neutral definition of a resource type's fields, constraints, lifecycle rules, and portability classification +- Lives in the Resource Type Registry (Tier 1, 2, or 3) +- Examples: `Compute.VirtualMachine v2.1.0`, `Network.VLAN v1.0.0` +- Defines what the resource TYPE is, not what any specific provider offers + +**Provider Catalog Item (Service Catalog entry):** +- A specific provider's offering implementing a Resource Type Specification +- Includes provider-specific pricing, availability, SLAs, and performance characteristics +- What consumers actually request via the Service Catalog +- Examples: "EU-WEST-Prod-1's 4-CPU VM offering", "NetworkOps's VLAN service" +- Tied to a specific provider; multiple providers can offer catalog items for the same Resource Type Spec + +**When to use each term:** +- "The consumer requests a catalog item" โœ“ โ€” they request a provider's specific offering +- "The resource type specification defines the field schema" โœ“ โ€” the spec defines structure +- "The catalog item schema" โœ— โ€” should be "the resource type specification schema" +- "The consumer browses resource types" โœ“ โ€” they browse the type hierarchy +- "The consumer selects a catalog item" โœ“ โ€” they select a specific provider offering + +**In the anti-vocabulary:** "Catalog Item" should not be used when "Resource Type Specification" is meant, and vice versa. The hierarchy is: Resource Type Category โ†’ Resource Type โ†’ Resource Type Specification โ†’ Provider Catalog Item. + + ### 2.1 Registry Principles - The registry is **open** โ€” third parties, implementors, and the community can propose new resource type definitions diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md index 284d02f..a5de9fb 100644 --- a/content/docs/architecture/data-model/universal-groups.md +++ b/content/docs/architecture/data-model/universal-groups.md @@ -1,7 +1,7 @@ --- title: "Universal Group Model" type: docs -weight: 14 +weight: 15 --- > **โš ๏ธ Active Development Notice** @@ -583,6 +583,90 @@ dcm_group: - **Universal Audit Model** (doc 16) โ€” all group changes produce audit records - **Ingestion Model** (doc 13) โ€” migration of existing constructs to universal groups + +--- + +## 13. Cross-Tenant Authorization Lifecycle + +### 13.1 What Cross-Tenant Authorizations Are + +A `cross_tenant_authorization` is a DCMGroup with `group_class: cross_tenant_authorization`. It is the formal mechanism by which one Tenant grants another Tenant permission to reference, allocate from, or stake a resource that belongs to the granting Tenant. + +Without a cross-tenant authorization, entities in different Tenants cannot form relationships. The authorization is the bridge that enables cross-Tenant resource sharing while maintaining isolation. + +### 13.2 Authorization Lifecycle + +```yaml +cross_tenant_authorization: + artifact_metadata: + uuid: + handle: "org/cross-tenant-auth/networkops-to-appteam-vlan100" + version: "1.0.0" + status: active + + granting_tenant_uuid: + receiving_tenant_uuid: + authorized_resources: + - resource_uuid: + permitted_operations: [stake, read] + - resource_type: Network.IPAddress + source_pool_uuid: + permitted_operations: [allocate] + + # Duration + valid_from: + valid_until: # null = perpetual until revoked + auto_renew: false + + # Who created this + granted_by_actor_uuid: + granted_at: +``` + +### 13.3 Who Creates Cross-Tenant Authorizations + +| Creator | Scenario | Authorization type | +|---------|---------|-------------------| +| Granting Tenant Admin | Standard: NetworkOps authorizes AppTeam to use VLAN-100 | explicit | +| Platform Admin | Emergency or platform-managed shared infrastructure | platform_managed | +| Pre-authorization policy | Policy automatically authorizes based on conditions | policy_auto | + +### 13.4 Revocation and Its Consequences + +When a cross-tenant authorization is revoked: + +``` +Authorization revoked (by granting Tenant admin, platform admin, or expiry) + โ”‚ + โ–ผ All active allocations and stakes under this authorization are identified + โ”‚ + โ–ผ For each active allocation / stake: + โ”‚ Entity enters PENDING_REVIEW state + โ”‚ pending_review_record created: + โ”‚ trigger: cross_tenant_auth.revoked + โ”‚ resolution_options: [re_authorize, release, migrate, escalate] + โ”‚ + โ–ผ Notifications sent: + โ”‚ Granting Tenant Admin + โ”‚ Receiving Tenant Admin + โ”‚ Each affected resource owner + โ”‚ Platform Admin (if platform_managed authorization) + โ”‚ + โ–ผ Resolution deadline: PT72H (configurable per profile) + โ”‚ + โ””โ”€โ”€ On deadline: on_deadline_exceeded recovery policy fires +``` + +### 13.5 System Policies โ€” Cross-Tenant Authorization + +| Policy | Rule | +|--------|------| +| `CTX-001` | Cross-tenant relationships require an active cross-tenant authorization or a resource type declared publicly_stakeable / publicly_allocatable in its Resource Type Spec. | +| `CTX-002` | Cross-tenant authorization revocation places all active dependent entities in PENDING_REVIEW. Revocation does not immediately release allocations. | +| `CTX-003` | Cross-tenant authorization expiry is treated identically to explicit revocation. | +| `CTX-004` | Platform Admin may create cross-tenant authorizations on behalf of any Tenant. All platform-managed authorizations carry a platform_managed flag and are visible in the platform admin audit log. | + + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md index 176be3c..7559ce3 100644 --- a/content/docs/architecture/data-model/webhooks-messaging.md +++ b/content/docs/architecture/data-model/webhooks-messaging.md @@ -148,6 +148,85 @@ policy: "If ingress.surface == message_bus_inbound AND ingress.actor.type != web policy: "If ingress.surface == message_bus_inbound AND message_bus_provider.jurisdiction != tenant.sovereignty_zone THEN gatekeep" ``` + +### 2.5 Ingress API vs Consumer API โ€” Relationship Clarification + +These two terms refer to different architectural layers: + +**Ingress API (infrastructure layer):** +The network-level entry point for all inbound requests to the DCM control plane. It handles: +- TLS termination +- Authentication token validation +- Setting the immutable `ingress` block on every request (surface, actor, timestamp, mfa_verified) +- Rate limiting at the network level +- Routing to the appropriate internal component (Consumer API handlers, Provider API handlers, Admin API handlers) + +The Ingress API is infrastructure โ€” it is not directly defined in any consumer-facing specification. + +**Consumer API (application layer):** +The logical REST API surface that consumers interact with, as defined in the [Consumer API Specification](../specifications/consumer-api-spec.md). The Consumer API is *served through* the Ingress API. When a consumer calls `POST /api/v1/requests`, that request enters through the Ingress API (which sets the ingress block) and is then handled by the Consumer API component. + +**Other APIs served through the Ingress API:** +- **Provider API** โ€” the callback and notification endpoints that Service Providers call (`/api/v1/provider/...`) +- **Admin API** โ€” platform administration operations (`/api/v1/admin/...`) +- **Webhook Inbound** โ€” external systems calling DCM (`/api/v1/webhooks/...`) + +**The Ingress API is one, the Consumer API is one of several logical surfaces routed through it.** + +--- + +### 2.6 Consumer Rate Limiting and Quota Model + +Consumer-side rate limiting and resource quotas are enforced by GateKeeper policies โ€” not hardcoded limits. This keeps quota enforcement consistent with DCM's policy-driven model. + +**Request rate limiting (per actor):** + +```yaml +rate_limit_policy: + type: gatekeeper + handle: "system/quotas/api-rate-limit" + trigger: request.initiated + conditions: + - field: ingress.actor_uuid + rate_window: PT1M + max_requests: 60 # configurable per Tenant policy + action: reject + rejection_code: 429 + rejection_message: "Rate limit exceeded. Retry after PT1M." +``` + +**Resource quotas (per Tenant per resource type):** + +```yaml +quota_policy: + type: gatekeeper + handle: "tenant/payments/vm-quota" + trigger: request.initiated + conditions: + - field: request.resource_type + equals: Compute.VirtualMachine + - field: tenant.active_entity_count + resource_type: Compute.VirtualMachine + operator: gte + value: 100 # max 100 concurrent VMs for this Tenant + action: reject + rejection_message: "VM quota exceeded (100). Request a quota increase via the Admin API." +``` + +**Quota increase process:** Tenants request quota increases through the standard request process. A quota change request produces a Requested State record, goes through policy evaluation, and requires Platform Admin approval for significant increases. + +**Profile-governed defaults:** + +| Profile | Default API rate limit | Default resource quota | +|---------|----------------------|----------------------| +| minimal | 10 req/min | Unlimited | +| dev | 60 req/min | Unlimited | +| standard | 60 req/min | Policy-declared | +| prod | 120 req/min | Policy-declared | +| fsi | 60 req/min | Strict policy-declared | +| sovereign | 30 req/min | Strict policy-declared | + + ### 2.4 System Policies | Policy | Rule | diff --git a/content/docs/architecture/specifications/flow-gui-spec.md b/content/docs/architecture/specifications/flow-gui-spec.md new file mode 100644 index 0000000..eb509f7 --- /dev/null +++ b/content/docs/architecture/specifications/flow-gui-spec.md @@ -0,0 +1,174 @@ +--- +title: "DCM Flow GUI Specification" +type: docs +weight: 6 +--- + +> **โš ๏ธ Work in Progress** +> +> This specification defines the DCM Flow GUI โ€” the visual interface for managing policies, orchestration flows, and the request lifecycle pipeline. Published to share design direction and invite feedback. + +**Version:** 0.1.0-draft +**Status:** Design โ€” Not yet implemented +**Document Type:** Technical Specification +**Related Documents:** [Control Plane Components](../data-model/25-control-plane-components.md) | [OPA Integration Specification](dcm-opa-integration-spec.md) | [Policy Profiles](../data-model/14-policy-profiles.md) + +--- + +## Abstract + +The DCM Flow GUI is the visual interface for platform engineers and integrators to compose, test, and manage DCM's data-driven orchestration flows. Because policies ARE the orchestration in DCM, the Flow GUI is fundamentally a **visual policy composer** โ€” it makes the active policy graph visible and editable without requiring direct YAML or Rego authoring. + +--- + +## 1. Core Views + +### 1.1 Execution Graph View + +The primary view shows the live execution graph: which policies are currently active, which payload types they match, and the sequence in which they fire for a given request type. + +``` +[request.initiated] โ”€โ”€โ†’ [IntentCapturePolicy] โ”€โ”€โ†’ [request.intent_captured] + โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ โ–ผ + [LayerAssembly] [CostCheck] [AuthzCheck] + (system domain) (tenant domain)(system domain) + โ”‚ + โ–ผ + [request.layers_assembled] + โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ โ–ผ + [GateKeeper1] [Transform1] [Validate1] + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ–ผ + [request.policies_evaluated] +``` + +**Interactive features:** +- Click any node to see the policy definition, trigger conditions, and current status +- Hover to see firing frequency (how often this policy fires per hour) +- Colour-coded by domain (system=blue, platform=green, tenant=yellow, provider=orange) +- Filter by payload type, domain, policy type, or resource type + +### 1.2 Policy Canvas (Static Flow Builder) + +For organizations that want to define fixed sequential workflows, the Policy Canvas provides a drag-and-drop interface: + +- Drag policy types from the palette onto the canvas +- Connect them with dependency arrows +- Set conditions on each step (fires when X AND Y) +- Set failure behavior (halt, skip, escalate) +- Export as a Policy Group with `concern_type: orchestration_flow` and `ordered: true` + +The canvas produces valid DCM YAML that can be committed to the GitOps policy store. + +### 1.3 Payload Type Browser + +Shows the complete payload type vocabulary. For each type: +- Which policies currently match it +- Sample payload structure +- Which downstream types it can produce +- Historical volume (how many events of this type per day) + +### 1.4 Shadow Mode Dashboard + +Shows active shadow policies and their evaluation results: +- Policy name and handle +- Shadow vs active comparison: "This policy would have rejected 3 requests in the last 24h" +- One-click promotion to active (if within review period) +- Side-by-side diff of shadow output vs actual outcome + +--- + +## 2. Policy Authoring Interface + +The Flow GUI includes a policy authoring interface for creating and editing policies without leaving the browser: + +### 2.1 Visual Condition Builder + +For simple conditions (field comparisons, role checks, quota checks), a visual condition builder generates valid Rego without requiring Rego knowledge: + +``` +Trigger: [request.initiated โ–ผ] + +Conditions: + [resource_type โ–ผ] [equals โ–ผ] [Compute.VirtualMachine] [+ AND] + [actor.roles โ–ผ] [does not contain โ–ผ] [platform_admin] [+ AND] + [payload.fields.cpu_count.value โ–ผ] [greater than โ–ผ] [32] + +Action: [Reject โ–ผ] +Rejection message: "CPU count exceeds maximum for this resource type" +``` + +### 2.2 Rego Editor + +For complex policies requiring full Rego expressiveness, the GUI includes an embedded Rego editor with: +- DCM input schema autocomplete +- DCM built-in function reference +- Real-time syntax validation +- Test case runner (against the test harness) + +### 2.3 Test Case Management + +Each policy can have associated test cases managed in the GUI: +- Create test cases from recent real requests ("save this request as a test case") +- Run test suite before committing a policy change +- View shadow mode results as test case comparisons + +--- + +## 3. Flow Simulation + +Platform engineers can simulate a request through the active policy graph without actually submitting it: + +``` +Simulate: resource_type=Compute.VirtualMachine, tenant=payments, cpu_count=64 + โ”‚ + โ–ผ Execution trace: + IntentCapturePolicy: PASS + VmSizeLimits (GateKeeper): REJECT โ€” cpu_count 64 exceeds maximum 32 + โ† Request would be rejected at this step +``` + +Simulation mode is read-only โ€” it uses the current active policies and a synthetic payload. No audit records are written. + +--- + +## 4. Profile and Module Management + +### 4.1 Active Profile View + +Shows the current active governance composition: +- Active deployment posture (with description of what it enforces) +- Active compliance domains (with summary of key requirements each adds) +- Active recovery posture profile +- Policy count per active profile group + +### 4.2 Profile Activation + +Change the deployment posture or add/remove compliance domains through the GUI. Produces a profile change request (through the standard request pipeline with appropriate approvals). + +--- + +## 5. Notification Flow View + +An extension of the Execution Graph View specific to the Notification Model: +- Shows active notification subscriptions per event type +- Visualizes the relationship graph traversal for a specific entity +- Shows which Notification Providers are active and their delivery health + +--- + +## 6. Integration with OPA + +The Flow GUI connects to the OPA integration for: +- Live policy evaluation display (showing OPA decisions in real time) +- Policy testing via the OPA test harness +- Shadow mode result display from OPA shadow evaluations +- Bundle upload and validation + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/opa-integration-spec.md b/content/docs/architecture/specifications/opa-integration-spec.md new file mode 100644 index 0000000..5a2dc74 --- /dev/null +++ b/content/docs/architecture/specifications/opa-integration-spec.md @@ -0,0 +1,324 @@ +--- +title: "DCM OPA Integration Specification" +type: docs +weight: 5 +--- + +> **โš ๏ธ Work in Progress** +> +> This specification defines the OPA integration contract for DCM Policy Providers. It is published to share design direction and invite feedback. Do not build production integrations against this specification until it reaches draft status. + +**Version:** 0.1.0-draft +**Status:** Design โ€” Not yet implemented +**Document Type:** Technical Specification +**Related Documents:** [Policy Profiles](../data-model/14-policy-profiles.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) + +--- + +## Abstract + +This specification defines how Open Policy Agent (OPA) integrates with the DCM Policy Engine as the reference implementation for Mode 3 Policy Providers. It defines the DCM payload schema as an OPA input document, the expected decision schema as OPA output, the built-in functions DCM provides to Rego policies, and the test harness contract for validating policies before activation. + +OPA is not required to implement DCM โ€” any Mode 3 Policy Provider can implement DCM's policy contract. However, OPA with Rego is the recommended reference implementation, and this specification enables implementors and integrators to build standards-compliant DCM policy engines. + +--- + +## 1. Introduction + +### 1.1 The Policy Engine Contract + +DCM's Policy Engine evaluates policies at multiple points in the request lifecycle. The engine receives a payload, evaluates all active matching policies, and accumulates mutations. The OPA integration maps this contract to Rego evaluation. + +DCM policy types: +- **GateKeeper** โ€” approve or reject; output is a decision (allow/deny + reason) +- **Validation** โ€” verify correctness; output is a validation result (pass/fail + details) +- **Transformation** โ€” enrich or modify; output is a set of field mutations +- **Recovery** โ€” respond to failure/ambiguity; output is a recovery action +- **Orchestration Flow** โ€” coordinate pipeline steps; output is a flow directive + +All five types share the same OPA input schema. The output schema differs per type. + +### 1.2 Mode 3 Policy Provider + +A Mode 3 Policy Provider executes OPA Rego bundles. DCM dispatches the policy input document to the OPA instance and receives the decision document. The OPA instance may be: +- Embedded within DCM (the reference implementation) +- A sidecar OPA instance (co-located with DCM) +- A remote OPA instance (requires network call; latency considerations apply) + +--- + +## 2. Input Schema โ€” DCM Payload as OPA Document + +Every OPA policy evaluation receives the following input document: + +```rego +# input document structure +input := { + # The current payload being evaluated + "payload": { + "type": "request.initiated", # payload type from the vocabulary + "entity_uuid": "...", + "resource_type": "Compute.VirtualMachine", + "version": "2.1.0", + "fields": { + "cpu_count": { + "value": 4, + "provenance": { "origin": {...}, "modifications": [...] } + } + # ... all assembled fields with provenance + } + }, + + # The requesting actor context + "actor": { + "uuid": "...", + "type": "human", # human | service_account | system + "tenant_uuid": "...", + "roles": ["developer"], + "groups": ["payments-team", "eu-west-users"], + "mfa_verified": true, + "auth_level": "oidc_mfa" + }, + + # The active deployment governance + "deployment": { + "posture": "prod", + "compliance_domains": ["hipaa", "gdpr"], + "recovery_posture": "notify-and-wait", + "profile_uuid": "..." + }, + + # Entity context (null for new requests) + "entity": { + "uuid": "...", + "lifecycle_state": "OPERATIONAL", + "ownership_model": "whole_allocation", + "owned_by_tenant_uuid": "...", + "relationship_count": 3, + "drift_status": "clean" + }, + + # Provider context (null before placement) + "provider": { + "uuid": "...", + "sovereignty_declaration": {...}, + "trust_score": 94, + "capacity_confidence": "high" + }, + + # DCM built-in data (resolved by DCM before OPA evaluation) + "dcm": { + "tenant": { + "uuid": "...", + "display_name": "Payments Platform", + "active_entity_count": { "Compute.VirtualMachine": 47 }, + "compliance_overlays": ["hipaa"] + }, + "cost_estimate": { + "per_hour": 0.32, + "confidence": "high" + } + } +} +``` + +--- + +## 3. Output Schema โ€” OPA Decision Documents + +### 3.1 GateKeeper Output + +```rego +package dcm.gatekeeper.vm_size_limits + +import future.keywords + +# Main decision +allow if { + input.payload.fields.cpu_count.value <= max_cpu +} + +deny contains reason if { + input.payload.fields.cpu_count.value > max_cpu + reason := sprintf("cpu_count %d exceeds maximum %d for tenant %s", + [input.payload.fields.cpu_count.value, max_cpu, input.actor.tenant_uuid]) +} + +# DCM reads the deny set; empty = allow +max_cpu := 32 +``` + +DCM output contract: +```json +{ + "allow": true, + "deny": [], + "warnings": [], + "policy_uuid": "...", + "evaluated_at": "..." +} +``` + +### 3.2 Transformation Output + +```rego +package dcm.transformation.inject_monitoring + +mutations contains mutation if { + input.payload.type == "request.layers_assembled" + not input.payload.fields.monitoring_endpoint + mutation := { + "field": "monitoring_endpoint", + "value": concat(".", ["https://metrics.internal", input.deployment.posture, "example.com"]), + "source_type": "policy", + "operation_type": "enrichment", + "reason": "Standard monitoring endpoint injection" + } +} +``` + +DCM output contract: +```json +{ + "mutations": [ + { + "field": "monitoring_endpoint", + "value": "https://metrics.internal.prod.example.com", + "source_type": "policy", + "operation_type": "enrichment", + "reason": "Standard monitoring endpoint injection" + } + ], + "policy_uuid": "..." +} +``` + +### 3.3 Recovery Policy Output + +```rego +package dcm.recovery.discard_on_timeout + +action := "DISCARD_AND_REQUEUE" if { + input.payload.type == "recovery.timeout_fired" + input.entity.lifecycle_state == "TIMEOUT_PENDING" +} +``` + +DCM output contract: +```json +{ + "action": "DISCARD_AND_REQUEUE", + "action_parameters": { "requeue_delay": "PT0S" }, + "policy_uuid": "..." +} +``` + +--- + +## 4. DCM Built-in Functions for Rego + +DCM provides built-in functions callable from Rego policies: + +```rego +# Entity relationship graph queries +dcm.entity.relationships(entity_uuid) + # Returns: array of relationship records for the entity + +dcm.entity.has_relationship(entity_uuid, relationship_type) + # Returns: bool + +dcm.entity.stakeholder_count(entity_uuid, min_stake_strength) + # Returns: int + +# Information Provider data +dcm.entity.field_confidence(entity_uuid, field_path) + # Returns: { band, score, authority_level } + +# Sovereignty checks +dcm.sovereignty.compatible(entity_uuid, provider_uuid) + # Returns: bool + +dcm.sovereignty.violates(entity_uuid, data_residency_requirement) + # Returns: bool + +# Cost queries +dcm.cost.estimate(catalog_item_uuid, fields) + # Returns: { per_hour, currency, confidence } + +# Tenant quota queries +dcm.tenant.active_count(tenant_uuid, resource_type) + # Returns: int + +dcm.tenant.has_authorization(granting_tenant_uuid, consuming_tenant_uuid, resource_type) + # Returns: bool +``` + +--- + +## 5. Policy Bundle Structure + +OPA policies for DCM are packaged as bundles: + +``` +dcm-policy-bundle/ +โ”œโ”€โ”€ .manifest +โ”‚ { +โ”‚ "roots": ["dcm"], +โ”‚ "metadata": { +โ”‚ "dcm_policy_type": "gatekeeper", +โ”‚ "resource_types": ["Compute.VirtualMachine"], +โ”‚ "domain": "tenant", +โ”‚ "handle": "org/policies/vm-size-limits", +โ”‚ "version": "1.0.0" +โ”‚ } +โ”‚ } +โ”œโ”€โ”€ dcm/ +โ”‚ โ””โ”€โ”€ gatekeeper/ +โ”‚ โ””โ”€โ”€ vm_size_limits/ +โ”‚ โ””โ”€โ”€ policy.rego +โ””โ”€โ”€ tests/ + โ””โ”€โ”€ vm_size_limits_test.rego +``` + +--- + +## 6. Test Harness + +DCM provides a test harness that policy authors use to validate policies against sample payloads before activation: + +``` +POST /api/v1/admin/policies/test + +{ + "policy_bundle": "", + "test_cases": [ + { + "description": "VM within CPU limit should be allowed", + "input": { + "payload": { "type": "request.initiated", "fields": { "cpu_count": { "value": 4 } } }, + "actor": { "roles": ["developer"] }, + "deployment": { "posture": "prod" } + }, + "expected_output": { "allow": true, "deny": [] } + } + ] +} +``` + +The test harness is also used during shadow mode โ€” DCM runs the policy against real traffic and compares actual output to expected output before the policy activates. + +--- + +## 7. Policy Shadow Mode with OPA + +When a policy is in `proposed` status, DCM evaluates it in shadow mode: + +1. Policy bundle loaded into a shadow OPA instance +2. Every real request payload is evaluated by both active policies AND shadow policies +3. Shadow outputs recorded in the Validation Store (not applied to requests) +4. Policy authors review shadow results via the Admin API or Flow GUI +5. On approval (no adverse results): policy status โ†’ `active` + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index 7e3a3dc..88cb09f 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -174,6 +174,15 @@ A data layer contributed by a Service Provider containing service-specific confi ### Operational Terms +**Cost Analysis Information Provider** +The specialized Information Provider that supplies cost estimation, placement cost signals, cost actuals, and budget alerts to DCM. DCM does not perform cost calculations โ€” it provides input data and consumes signals from this provider. Integrates with external cost management platforms (e.g., Red Hat Cost Management). Falls back to static declared costs from provider registration when unavailable. + +**Orchestrator** +The DCM control plane component that sequences and executes ordered workflows. Conducts the request lifecycle pipeline and executes named workflow artifacts. Relies heavily on policies (workflow steps invoke Policy Engine) and data (workflow steps read/write entity state). Workflows are first-class DCM artifacts included in Profiles. + +**Workflow (DCM)** +A versioned, GitOps-managed DCM artifact defining a named sequence of operations. Can be triggered manually, on a schedule, by events, or by policy output. Included in Profiles to automatically activate functionality when the profile is applied. The request lifecycle pipeline is itself a system-domain built-in workflow. + **Brownfield** Existing infrastructure provisioned outside of DCM, brought under management via the Ingestion Model. @@ -228,6 +237,8 @@ Terms to avoid and what to use instead. | **Producer** | Generic term not in DCM vocabulary | **Service Provider** โ€” carries the full DCM contract model | | **Shore / Ship / Enclave** | Defense IT terminology โ€” not universally understood | **Hub DCM**, **Regional DCM**, **Sovereign DCM** | | **User** (generic) | Means different things at different layers | **Developer** / **Application Owner** (Application domain); **Platform Engineer** / **SRE** (platform) | +| **"catalog item"** used to mean "resource type specification" | These are distinct: resource type specification = vendor-neutral contract in registry; catalog item = provider's specific offering implementing that contract | **Provider Catalog Item** or **Resource Type Specification** โ€” be specific | +| **"resource type"** used to mean a specific provider offering | Resource type is the classification; the offering is the catalog item | **Provider Catalog Item** | | **Service** (unqualified) | Overloaded โ€” means different things at each layer | **Catalog Item** (Application), **Resource Type** (Control Plane), **Service Provider** (provider) | | **Config / Configuration** | Could mean Layer, Resource Type Spec, Policy, or settings | Specify: **Layer**, **Resource Type Specification**, **Policy**, **Platform configuration** | | **Manage** | Means everything and nothing | **Provision**, **configure**, **monitor**, **decommission**, **migrate**, **govern** | From fce908249c073b5f56992a45b3ba56d97eb44d20 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Fri, 27 Mar 2026 20:09:52 -0500 Subject: [PATCH 24/49] docs: add accreditation, data authorization matrix, and zero trust model Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 119 +++- .../accreditation-and-authorization-matrix.md | 599 ++++++++++++++++++ .../data-model/policy-profiles.md | 36 ++ .../data-model/resource-service-entities.md | 33 + .../specifications/consumer-api-spec.md | 19 +- 5 files changed, 800 insertions(+), 6 deletions(-) create mode 100644 content/docs/architecture/data-model/accreditation-and-authorization-matrix.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 11d728d..177986a 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -424,6 +424,17 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **data_classification** | First-class field metadata: public/internal/confidential/restricted/phi/pci/sovereign/classified; phi/sovereign/classified are immutable once set | +| **Accreditation** | Formal versioned attestation that a component satisfies a compliance framework; issued by an Accreditor; carries validity period; lifecycle: developingโ†’proposedโ†’activeโ†’expired/revoked | +| **Accreditor** | Entity that issues accreditations: government body, regulatory body, QSA, certification body, or internal audit team | +| **Accreditation Gap** | Missing, expired, or revoked accreditation required for an active interaction; always high/critical severity; Recovery Policy governs response | +| **Data/Capability Authorization Matrix** | Policy Group artifact (concern_type: data_authorization_boundary) declaring what data fields and capabilities are permitted across interaction boundaries given data classification and accreditation level | +| **zero_trust_posture** | Sixth Policy Group concern type; four levels: none/boundary/full/hardware_attested; profile defaults: minimal=none, dev/standard=boundary, prod/fsi=full, sovereign=hardware_attested | +| **Five-check boundary model** | Identity โ†’ Authorization โ†’ Accreditation โ†’ Matrix โ†’ Sovereignty; all five checks at every DCM interaction boundary; all produce audit records | +| **Federation tunnel** | Mutually authenticated, encrypted, scoped DCM-to-DCM channel; zero trust model; establishes secure transport only, not implicit trust; per-message signing; scoped non-transferable credentials | +| **hard_constraint** | Data/Capability Matrix declaration that cannot be overridden by any policy; sovereign/classified data never crossing federation boundaries is a hard_constraint | +| **STRIP_FIELD** | Matrix enforcement action: remove non-permitted field from payload and proceed; if stripped field is required โ†’ escalates to DENY_REQUEST | +| **DENY_REQUEST** | Matrix enforcement action: block entire interaction; entity enters PENDING_REVIEW; notification dispatched | | **Request Orchestrator** | DCM control plane event bus; routes lifecycle events to Policy Engine; coordinates pipeline via event-condition-action; does not contain hardcoded pipeline logic | | **Cost Analysis Component** | Internal DCM control plane component; three functions: pre-request estimation, placement input, ongoing attribution; not a billing system; not a provider type | | **Module** | DCM capability extension adding new functions; distinct from Profile (which configures behavior) | @@ -2497,6 +2508,17 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **data_classification** | First-class field metadata: public/internal/confidential/restricted/phi/pci/sovereign/classified; phi/sovereign/classified are immutable once set | +| **Accreditation** | Formal versioned attestation that a component satisfies a compliance framework; issued by an Accreditor; carries validity period; lifecycle: developingโ†’proposedโ†’activeโ†’expired/revoked | +| **Accreditor** | Entity that issues accreditations: government body, regulatory body, QSA, certification body, or internal audit team | +| **Accreditation Gap** | Missing, expired, or revoked accreditation required for an active interaction; always high/critical severity; Recovery Policy governs response | +| **Data/Capability Authorization Matrix** | Policy Group artifact (concern_type: data_authorization_boundary) declaring what data fields and capabilities are permitted across interaction boundaries given data classification and accreditation level | +| **zero_trust_posture** | Sixth Policy Group concern type; four levels: none/boundary/full/hardware_attested; profile defaults: minimal=none, dev/standard=boundary, prod/fsi=full, sovereign=hardware_attested | +| **Five-check boundary model** | Identity โ†’ Authorization โ†’ Accreditation โ†’ Matrix โ†’ Sovereignty; all five checks at every DCM interaction boundary; all produce audit records | +| **Federation tunnel** | Mutually authenticated, encrypted, scoped DCM-to-DCM channel; zero trust model; establishes secure transport only, not implicit trust; per-message signing; scoped non-transferable credentials | +| **hard_constraint** | Data/Capability Matrix declaration that cannot be overridden by any policy; sovereign/classified data never crossing federation boundaries is a hard_constraint | +| **STRIP_FIELD** | Matrix enforcement action: remove non-permitted field from payload and proceed; if stripped field is required โ†’ escalates to DENY_REQUEST | +| **DENY_REQUEST** | Matrix enforcement action: block entire interaction; entity enters PENDING_REVIEW; notification dispatched | | **Request Orchestrator** | DCM control plane event bus; routes lifecycle events to Policy Engine; coordinates pipeline via event-condition-action; does not contain hardcoded pipeline logic | | **Cost Analysis Component** | Internal DCM control plane component; three functions: pre-request estimation, placement input, ongoing attribution; not a billing system; not a provider type | | **Module** | DCM capability extension adding new functions; distinct from Profile (which configures behavior) | @@ -3281,7 +3303,83 @@ Hub selects Regional DCM using DCM-010. Regional DCM applies BBQ-001 for its own --- -## SECTION 51 โ€” PERSONAS +## SECTION 51 โ€” ACCREDITATION, DATA AUTHORIZATION MATRIX, AND ZERO TRUST + +### 51.1 Three Interconnected Models + +Three models compose to govern trust and data handling across all DCM boundaries: +1. **Accreditation** โ€” is this component certified to handle this data type? +2. **Data/Capability Authorization Matrix** โ€” given certification, what data/capabilities are permitted across this boundary? +3. **Zero Trust** โ€” is this specific call, right now, from who it claims to be, permitted to do what it's attempting? + +All three checks run at every interaction boundary. All five boundary checks (identity โ†’ authorization โ†’ accreditation โ†’ matrix โ†’ sovereignty) produce audit records regardless of outcome. + +### 51.2 Data Classification โ€” First-Class Field Metadata + +Seven classification levels: `public | internal | confidential | restricted | phi | pci | sovereign | classified` + +Carried as `data_classification` on every field in every DCM payload. Declared in: Resource Type Specification (default per field), Data Layer (domain-wide override), explicit field instance (highest precedence). `phi`, `sovereign`, `classified` are **immutable once set** โ€” no layer or policy may downgrade them (ACC-003). Default for unclassified fields: `internal`. + +### 51.3 Accreditation Model + +First-class versioned artifacts. Seven types (ascending trust): `self_declared`, `first_party`, `third_party`, `qsa_assessment`, `baa`, `regulatory_certification`, `sovereign_authorization`. Lifecycle: developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired. Renewal warning P90D before expiry. On expiry/revocation: **Accreditation Gap** record created; Recovery Policy evaluates response; affected entities potentially blocked. + +Accreditations cover: `data_classifications`, `capabilities`, `geographic_scope`. DCM deployments themselves carry accreditations (enabling federated trust verification). Providers declare accreditations via `POST /api/v1/provider/accreditations` โ†’ proposed โ†’ platform admin activates. + +### 51.4 Data/Capability Authorization Matrix + +Policy Group artifact with `concern_type: data_authorization_boundary`. Activated as part of compliance domain group (HIPAA domain โ†’ HIPAA boundary matrix). Three sections: + +**Outbound data permissions:** `data_classification ร— required_accreditation_type โ†’ ALLOW | STRIP_FIELD | DENY_REQUEST | WARN_AND_ALLOW`. PHI requires BAA โ€” no BAA โ†’ DENY_REQUEST. Restricted requires third_party โ€” no third_party โ†’ STRIP_FIELD. + +**Capability permissions:** STORE_AT_REST on PHI requires BAA. REPLICATE_CROSS_REGION on PHI requires BAA + replication target also has BAA. EXPORT_TO_EXTERNAL_SYSTEM on PHI/restricted/sovereign requires regulatory_cert. + +**Inbound data permissions:** What provider may return; which partition stores it; consumer visibility requirements. + +**Federation boundary matrix:** `sovereign` and `classified` data = `hard_constraint: true` โ†’ NEVER crosses any federation boundary regardless of accreditation. This cannot be overridden by any policy. + +**Enforcement pipeline:** Classification inventory โ†’ Accreditation resolution โ†’ Matrix evaluation per field โ†’ ALLOW/STRIP/DENY/WARN โ†’ Audit record. + +### 51.5 Zero Trust Interaction Model + +**Network position grants zero trust.** Five checks at every boundary: +1. Identity verification (mTLS mutual; certificate pinning; hardware attestation for sovereign) +2. Authorization verification (explicit permission; scoped credential; not revoked) +3. Accreditation check (target holds required cert; current; in-scope) +4. Data/Capability Matrix check (fields and capabilities permitted) +5. Sovereignty check (BBQ-001; endpoint within boundary) + +All five produce audit records on pass AND fail. + +**Credentials:** Scoped (minimum necessary operation), short-lived (PT15M for fsi/sovereign; PT30M prod; PT1H standard), non-transferable. Bound to specific entity + provider + operation type. + +### 51.6 Zero Trust Posture โ€” Sixth Policy Group Concern Type + +Four levels: `none` (minimal) โ†’ `boundary` (dev/standard; external boundaries only) โ†’ `full` (prod/fsi; everywhere including internal) โ†’ `hardware_attested` (sovereign; TPM/HSM required). + +Profile defaults: minimal=none, dev/standard=boundary, prod/fsi=full, sovereign=hardware_attested. + +### 51.7 Federation Tunnel Zero Trust + +Federation tunnels = secure transport, not implicit trust. Structure: mTLS with certificate pinning + per-message signing (ed25519) + replay protection (nonce + PT5M window). Federation credentials scoped to specific operation + specific tunnel + specific resource types. Non-transferable. + +Hub-spoke: Hub presents its own credential to Regional DCMs. Regional DCM credentials are never relayed. Each DCM instance verifies the Hub's accreditation before accepting federation messages. + +Data boundary: sovereign/classified NEVER crosses federation tunnel (hard_constraint). fsi: max classification = restricted within same jurisdiction. sovereign: internal only, same instance. + +### 51.8 Policies + +ZT-001 through ZT-005 (zero trust) + ACC-001 through ACC-006 (accreditation). Key: +- ZT-001: network position = zero trust +- ZT-003: sovereign/classified never crosses any boundary (hard constraint) +- ZT-004: federation tunnel = secure transport, not trust +- ACC-003: phi/sovereign/classified classification is immutable +- ACC-004: matrix enforced at every outbound boundary before dispatch +- ACC-006: zero_trust_posture is the sixth Policy Group concern type + +--- + +## SECTION 52 โ€” PERSONAS | Persona | Primary Concern | |---------|----------------| @@ -3298,7 +3396,7 @@ Hub selects Regional DCM using DCM-010. Regional DCM applies BBQ-001 for its own --- -## SECTION 52 โ€” TERMINOLOGY GLOSSARY +## SECTION 53 โ€” TERMINOLOGY GLOSSARY | Term | Definition | |------|-----------| @@ -3368,6 +3466,17 @@ Hub selects Regional DCM using DCM-010. Regional DCM applies BBQ-001 for its own | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **data_classification** | First-class field metadata: public/internal/confidential/restricted/phi/pci/sovereign/classified; phi/sovereign/classified are immutable once set | +| **Accreditation** | Formal versioned attestation that a component satisfies a compliance framework; issued by an Accreditor; carries validity period; lifecycle: developingโ†’proposedโ†’activeโ†’expired/revoked | +| **Accreditor** | Entity that issues accreditations: government body, regulatory body, QSA, certification body, or internal audit team | +| **Accreditation Gap** | Missing, expired, or revoked accreditation required for an active interaction; always high/critical severity; Recovery Policy governs response | +| **Data/Capability Authorization Matrix** | Policy Group artifact (concern_type: data_authorization_boundary) declaring what data fields and capabilities are permitted across interaction boundaries given data classification and accreditation level | +| **zero_trust_posture** | Sixth Policy Group concern type; four levels: none/boundary/full/hardware_attested; profile defaults: minimal=none, dev/standard=boundary, prod/fsi=full, sovereign=hardware_attested | +| **Five-check boundary model** | Identity โ†’ Authorization โ†’ Accreditation โ†’ Matrix โ†’ Sovereignty; all five checks at every DCM interaction boundary; all produce audit records | +| **Federation tunnel** | Mutually authenticated, encrypted, scoped DCM-to-DCM channel; zero trust model; establishes secure transport only, not implicit trust; per-message signing; scoped non-transferable credentials | +| **hard_constraint** | Data/Capability Matrix declaration that cannot be overridden by any policy; sovereign/classified data never crossing federation boundaries is a hard_constraint | +| **STRIP_FIELD** | Matrix enforcement action: remove non-permitted field from payload and proceed; if stripped field is required โ†’ escalates to DENY_REQUEST | +| **DENY_REQUEST** | Matrix enforcement action: block entire interaction; entity enters PENDING_REVIEW; notification dispatched | | **Request Orchestrator** | DCM control plane event bus; routes lifecycle events to Policy Engine; coordinates pipeline via event-condition-action; does not contain hardcoded pipeline logic | | **Cost Analysis Component** | Internal DCM control plane component; three functions: pre-request estimation, placement input, ongoing attribution; not a billing system; not a provider type | | **Module** | DCM capability extension adding new functions; distinct from Profile (which configures behavior) | @@ -3583,7 +3692,7 @@ Hub selects Regional DCM using DCM-010. Regional DCM applies BBQ-001 for its own --- -## SECTION 53 โ€” OPEN QUESTIONS +## SECTION 54 โ€” OPEN QUESTIONS These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -3680,7 +3789,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 54 โ€” DOCUMENTATION STRUCTURE +## SECTION 55 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -3728,7 +3837,7 @@ content/ --- -## SECTION 55 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 56 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: diff --git a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md new file mode 100644 index 0000000..33f109a --- /dev/null +++ b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md @@ -0,0 +1,599 @@ +--- +title: "Accreditation, Data Authorization Matrix, and Zero Trust" +type: docs +weight: 26 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Policy Profiles](14-policy-profiles.md) | [Resource/Service Entities](06-resource-service-entities.md) | [DCM Federation](22-dcm-federation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Operational Models](24-operational-models.md) + +--- + +## 1. Purpose + +This document defines three interconnected models that together govern how DCM handles trust, data handling obligations, and compliance verification across all interaction boundaries: + +1. **Accreditation Model** โ€” how DCM records, verifies, and enforces third-party compliance certifications for providers, policy engines, and DCM deployments themselves +2. **Data/Capability Authorization Matrix** โ€” what data and capabilities are permitted across any DCM boundary given a component's accreditation level and the data's classification +3. **Zero Trust Interaction Model** โ€” the authentication, authorization, and verification requirements for every interaction in DCM, regardless of network position + +These three models compose: Zero Trust verifies identity and authorization on every call. Accreditation verifies compliance certification status. The Authorization Matrix declares what is permitted given that certification status. Together they ensure that no interaction in DCM is implicitly trusted โ€” every boundary crossing is verified against all three models. + +--- + +## 2. Data Classification + +Data classification is a **first-class field-level metadata property** in the DCM data model. Every field in every payload carries a `data_classification` value. This classification is the primary axis of the authorization matrix and is the key input to sovereignty and compliance enforcement. + +### 2.1 Classification Levels + +| Level | Description | Examples | +|-------|-------------|---------| +| `public` | No restrictions; freely shareable | Resource display names, catalog item descriptions | +| `internal` | Organization-internal; not for external disclosure | Configuration details, operational metadata | +| `confidential` | Sensitive business data; restricted access | Cost data, business unit assignments | +| `restricted` | Highly sensitive; regulated or contractually protected | Security group IDs, network topology details | +| `phi` | Protected Health Information under HIPAA/HITECH | Patient IDs, diagnosis codes, treatment plans | +| `pci` | Payment Card Industry data under PCI-DSS | Cardholder data, authentication data | +| `sovereign` | Nationally classified or sovereignty-restricted data | Data subject to national security law | +| `classified` | Government-classified information | Classified defense or intelligence data | + +### 2.2 Classification as Field Metadata + +Every field in a DCM payload carries data classification as part of its field metadata: + +```yaml +field_definition: + field_name: patient_record_id + value: "PAT-00421" + data_classification: phi + classification_basis: "Contains patient identifier โ€” HIPAA 45 CFR 164.514" + metadata: + override: immutable # classification cannot be changed by policy + locked_by: system/compliance/hipaa-field-classifier +``` + +**Classification is declared in three places:** +- **Resource Type Specification** โ€” default classification per field for all instances of that type +- **Data Layer** โ€” classification applied across a domain (e.g., an org layer that marks all cost_center fields as `confidential`) +- **Field-level override** โ€” explicit classification on a specific field instance (highest precedence, immutable once set for `phi`, `sovereign`, `classified`) + +### 2.3 Classification Immutability + +Fields classified as `phi`, `sovereign`, or `classified` cannot be downgraded by any layer or policy โ€” their classification is immutable once set. A GateKeeper policy attempting to downgrade a PHI field is rejected with a classification violation audit record. + +--- + +## 3. Accreditation Model + +### 3.1 What Accreditation Is + +An **Accreditation** is a formal, versioned, time-bounded attestation that a DCM component โ€” a Service Provider, a Policy Provider, a Storage Provider, a Notification Provider, or a DCM deployment itself โ€” satisfies the requirements of a specific compliance framework. Accreditations are issued by an **Accreditor** and registered with DCM as first-class artifacts. + +Accreditation answers: **"Is this component certified to handle this type of data?"** + +### 3.2 Accreditation Types and Trust Levels + +| Type | Issued By | Trust Level | Examples | +|------|-----------|-------------|---------| +| `self_declared` | Component itself | Lowest | Dev/homelab; provider asserts own compliance | +| `first_party` | DCM organization's own audit team | Low-Medium | Internal compliance review | +| `third_party` | Independent certifying body | High | ISO 27001, SOC 2 Type II | +| `qsa_assessment` | Qualified Security Assessor | High | PCI-DSS QSA report | +| `baa` | Legal BAA with covered entity | High | HIPAA Business Associate Agreement | +| `regulatory_certification` | Government regulatory body | Highest | FedRAMP P-ATO, DoD Provisional Authorization | +| `sovereign_authorization` | National sovereignty authority | Highest | National cloud authorization | + +### 3.3 Accreditation Record Structure + +```yaml +accreditation: + # Standard artifact metadata + artifact_metadata: + uuid: + handle: "accreditations/providers/eu-west-prod-1/fedramp-high" + version: "1.0.0" + status: active + owned_by: { display_name: "Compliance Team" } + + subject_uuid: # what is being accredited + subject_type: service_provider | policy_provider | storage_provider | + notification_provider | dcm_deployment + + accreditation_type: + framework: fedramp_high | fedramp_moderate | hipaa | pci_dss_v4 | + iso_27001 | soc2_type2 | dod_il4 | dod_il5 | dod_il6 | + sovereign | classified | + + accreditor: + uuid: + name: "DISA" | "HHS OIG" | "PCI SSC" | "BSI" | + type: government | regulatory_body | qsa | certification_body | internal | self + contact_url: + + # Validity + issued_at: + valid_until: # null = perpetual until revoked + renewal_warning_before: P90D + last_verified_at: # when DCM last confirmed still active + + # What the accreditation covers + scope: + data_classifications: [phi, restricted] # which classifications this covers + capabilities: [data_at_rest, data_in_transit, access_control, audit_logging] + geographic_scope: [US, EU-WEST] + exclusions: [] + + # Evidence + certificate_ref: + audit_report_ref: + external_registry_id: "FR2024-0042" # e.g., FedRAMP Marketplace ID + + # Status + status: active | suspended | revoked | expired | pending_renewal + revocation_reason: + revoked_at: +``` + +### 3.4 Accreditation Lifecycle + +``` +Accreditation submitted (via API or GitOps PR) + โ”‚ + โ–ผ DCM validates structure and accreditor registration + โ”‚ + โ–ผ status: proposed + โ”‚ Shadow mode: compliance policies use this accreditation in shadow evaluation + โ”‚ Platform admin reviews certificate_ref and audit_report_ref + โ”‚ + โ–ผ Platform admin approves โ†’ status: active + โ”‚ Accreditation now enforced in compliance checks + โ”‚ All affected providers/deployments re-evaluated against new accreditation + โ”‚ + โ–ผ Expiry monitoring: + โ”‚ At valid_until - renewal_warning_before: + โ”‚ notification.accreditation_expiring โ†’ Compliance Team, Platform Admin + โ”‚ At valid_until: + โ”‚ status โ†’ expired + โ”‚ Providers relying on this accreditation flagged: ACCREDITATION_GAP + โ”‚ + โ–ผ Revocation: + Accreditor or Platform Admin revokes + status โ†’ revoked + All active provider interactions using this accreditation suspended + notification.accreditation_revoked โ†’ Platform Admin (urgency: critical) +``` + +### 3.5 Accreditation Gap + +When a required accreditation is missing, expired, or revoked, DCM enters an **Accreditation Gap** state for the affected provider: + +```yaml +accreditation_gap_record: + uuid: + provider_uuid: + required_framework: hipaa + required_for: [phi data fields in active requests] + gap_type: missing | expired | revoked | suspended + detected_at: + severity: critical # accreditation gaps are always high or critical + affected_entity_uuids: [, ...] # entities currently hosted at this provider + policy_response: + # Default: NOTIFY_AND_WAIT for fsi/sovereign; ESCALATE for standard/prod +``` + +### 3.6 DCM Deployment Accreditation + +DCM deployments themselves can carry accreditations โ€” a FedRAMP-authorized DCM deployment, for example. This enables cross-organization trust: a consuming organization's DCM can verify the providing organization's DCM deployment holds the required accreditation before federating with it. + +```yaml +deployment_accreditation: + subject_type: dcm_deployment + subject_uuid: + framework: fedramp_high + # The DCM deployment itself is accredited, not just the providers it manages +``` + +--- + +## 4. Data/Capability Authorization Matrix + +### 4.1 Purpose + +The Data/Capability Authorization Matrix declares what data fields and provider capabilities are permitted across any DCM interaction boundary given the data's classification and the receiving component's accreditation level. It is the enforcement model that sits between compliance domain policies and the actual provider interaction. + +### 4.2 Matrix as a Policy Artifact + +The authorization matrix is a **Policy Group artifact** with `concern_type: data_authorization_boundary`. It is activated as part of the compliance domain group โ€” enabling the HIPAA compliance domain automatically activates the HIPAA boundary matrix. Organizations extend or restrict matrices via their own policy groups at the Tenant level. + +```yaml +data_authorization_matrix: + artifact_metadata: + uuid: + handle: "system/matrix/hipaa-provider-boundary" + version: "1.0.0" + status: active + + concern_type: data_authorization_boundary + applicable_compliance_domains: [hipaa] + + # OUTBOUND: what DCM may send to a provider + outbound_data_permissions: + - data_classification: phi + required_accreditation_type: baa + required_accreditation_framework: hipaa + on_missing_accreditation: DENY_REQUEST + # DENY_REQUEST: block the entire request (PHI is required; cannot strip) + # STRIP_FIELD: remove field and proceed (for optional PHI fields) + # WARN_AND_ALLOW: allow but audit (dev profile only) + + - data_classification: restricted + required_accreditation_type: third_party + on_missing_accreditation: STRIP_FIELD + + - data_classification: internal + required_accreditation_type: self_declared + on_missing_accreditation: WARN_AND_ALLOW # always has self_declared minimum + + - data_classification: [public, internal] + required_accreditation_type: self_declared + on_missing_accreditation: ALLOW + + # CAPABILITY: what operations the provider may perform on classified data + capability_permissions: + - capability: STORE_AT_REST + data_classification: phi + required_accreditation_type: baa + required_scope: [data_at_rest] + on_missing_accreditation: DENY_CAPABILITY + + - capability: REPLICATE_CROSS_REGION + data_classification: phi + required_accreditation_type: baa + additional_requirement: replication_target_has_baa + on_missing_accreditation: DENY_CAPABILITY + + - capability: EXPORT_TO_EXTERNAL_SYSTEM + data_classification: [phi, restricted, sovereign] + required_accreditation_type: regulatory_certification + on_missing_accreditation: DENY_CAPABILITY + + - capability: PROVIDER_UPDATE_NOTIFICATION + data_classification: phi + required_accreditation_type: baa + # Provider may only notify DCM of changes to PHI-containing resources + # if it holds a valid BAA + on_missing_accreditation: DENY_CAPABILITY + + # INBOUND: what the provider may return to DCM + inbound_data_permissions: + - data_classification: phi + provider_must_strip_before_return: false + # DCM receives PHI in Realized State but access-controls it + consumer_visibility_requires_accreditation: baa + stored_in_partition: realized_store_phi + # PHI partition has additional encryption and access control +``` + +### 4.3 Federation Boundary Matrix + +A dedicated matrix governs what crosses DCM-to-DCM federation boundaries: + +```yaml +federation_boundary_matrix: + artifact_metadata: + handle: "system/matrix/federation-boundary" + concern_type: data_authorization_boundary + applicable_to: federation_tunnel + + outbound_data_permissions: + - data_classification: sovereign + on_missing_accreditation: DENY_REQUEST + # Sovereign data NEVER crosses a federation boundary + # This is a hard system constraint, not a configurable policy + hard_constraint: true + + - data_classification: classified + on_missing_accreditation: DENY_REQUEST + hard_constraint: true + + - data_classification: phi + required_accreditation_type: baa + on_missing_accreditation: DENY_REQUEST + + - data_classification: restricted + required_accreditation_type: third_party + additional_requirement: remote_dcm_holds_equivalent_accreditation + on_missing_accreditation: STRIP_FIELD + + - data_classification: [public, internal] + required_accreditation_type: self_declared + on_missing_accreditation: ALLOW +``` + +### 4.4 Matrix Enforcement Pipeline + +The authorization matrix check is a distinct pipeline step executed at every interaction boundary: + +``` +Outbound interaction assembled (DCM โ†’ Provider OR DCM โ†’ DCM) + โ”‚ + โ–ผ Data Classification Inventory: + โ”‚ For every field in the payload: + โ”‚ Resolve data_classification (field metadata โ†’ layer โ†’ resource type spec default) + โ”‚ Record classification โ†’ field mapping + โ”‚ + โ–ผ Accreditation Resolution: + โ”‚ Load active accreditations for the target component + โ”‚ For each required classification level in the payload: + โ”‚ Does the target hold an active, in-scope accreditation? + โ”‚ Is the accreditation within its valid_until date? + โ”‚ + โ–ผ Matrix Evaluation (per field): + โ”‚ Look up data_classification ร— accreditation_level in active matrix + โ”‚ Determine: ALLOW | STRIP_FIELD | DENY_REQUEST | DENY_CAPABILITY | WARN_AND_ALLOW + โ”‚ + โ”œโ”€โ”€ All ALLOW โ†’ proceed + โ”‚ + โ”œโ”€โ”€ STRIP_FIELD โ†’ remove field from payload; write FIELD_STRIPPED audit record + โ”‚ If stripped field is required for service โ†’ escalate to DENY_REQUEST + โ”‚ + โ”œโ”€โ”€ DENY_REQUEST โ†’ block interaction; entity enters PENDING_REVIEW + โ”‚ notification.accreditation_gap dispatched to owner + platform admin + โ”‚ + โ””โ”€โ”€ WARN_AND_ALLOW โ†’ proceed but write ACCREDITATION_ADVISORY audit record + (dev profile only; blocked in standard+) +``` + +--- + +## 5. Zero Trust Interaction Model + +### 5.1 Principle + +**Network position grants zero trust.** A component inside the DCM control plane has no more implicit trust than one outside it. Every interaction โ€” internal or external, synchronous or asynchronous โ€” is authenticated, authorized, and verified as if the caller were an untrusted external party. + +Zero trust in DCM is not a network topology โ€” it is a **per-interaction verification discipline** applied at every call, every event, every tunnel message. + +### 5.2 The Five-Check Boundary Model + +Every DCM interaction boundary applies five checks in sequence. All five must pass: + +``` +Interaction attempt + โ”‚ + โ–ผ Check 1: Identity Verification + โ”‚ mTLS certificate verification (mutual โ€” both sides present certificates) + โ”‚ Certificate chain validation against registered trust anchor + โ”‚ Certificate not in revocation list + โ”‚ Hardware attestation (fsi/sovereign profiles with hardware_attested posture) + โ”‚ โ†’ FAIL: connection refused; IDENTITY_VERIFICATION_FAILED audit record + โ”‚ + โ–ผ Check 2: Authorization Verification + โ”‚ Does this identity have explicit permission for this operation type? + โ”‚ Is the presented credential scoped to this operation? + โ”‚ Has this credential been revoked or expired? + โ”‚ Does the scope match the minimum necessary for this call? + โ”‚ โ†’ FAIL: 403 Forbidden; AUTHORIZATION_DENIED audit record + โ”‚ + โ–ผ Check 3: Accreditation Check + โ”‚ Does the target hold the required accreditation for the data classifications present? + โ”‚ Is the accreditation current and not suspended? + โ”‚ โ†’ FAIL: ACCREDITATION_GAP; recovery policy evaluates response + โ”‚ + โ–ผ Check 4: Data/Capability Matrix Check + โ”‚ Is each field permitted to cross this boundary? + โ”‚ Is each capability permitted for this data classification? + โ”‚ โ†’ FAIL: FIELD_STRIPPED or DENY_REQUEST per matrix declaration + โ”‚ + โ–ผ Check 5: Sovereignty Check + โ”‚ Is the target endpoint within the sovereignty boundary? + โ”‚ Does the interaction violate any sovereignty constraints? + โ”‚ BBQ-001 evaluation for Mode 4 endpoints + โ”‚ โ†’ FAIL: SOVEREIGNTY_VIOLATION; platform admin notified + โ”‚ + โ–ผ All checks pass โ†’ interaction proceeds + โ”‚ + โ””โ”€โ”€ Audit record written regardless of outcome: + INTERACTION_AUTHORIZED or INTERACTION_DENIED_{CHECK} + All five check results recorded + Credential UUID, interaction UUID for correlation +``` + +### 5.3 Credential Model โ€” Scoped, Short-Lived, Non-Transferable + +Zero trust requires that credentials are scoped to the minimum necessary operation and expire quickly: + +```yaml +dcm_interaction_credential: + credential_uuid: + issued_to: + issued_at: + valid_until: # short-lived; typically PT15M to PT1H + operation_scope: + operation_type: dispatch | discovery | cancel | query | notify + entity_uuid: # scoped to specific entity + provider_uuid: # scoped to specific provider + non_transferable: true # cannot be delegated or relayed + bound_to_ip: # optional IP binding for fsi/sovereign +``` + +**Credential lifetimes by profile:** + +| Profile | Max credential lifetime | Renewal model | +|---------|------------------------|---------------| +| minimal | PT8H | Manual or long-lived | +| dev | PT4H | Automatic refresh | +| standard | PT1H | Automatic refresh | +| prod | PT30M | Automatic refresh | +| fsi | PT15M | Automatic refresh; dual approval for elevation | +| sovereign | PT15M + hardware attestation | Hardware-bound; HSM-required | + +### 5.4 Zero Trust Posture as a Policy Group Concern Type + +`zero_trust_posture` is the sixth Policy Group concern type. Four posture levels: + +| Posture | Description | Profile Default | +|---------|-------------|----------------| +| `none` | No zero trust enforcement; perimeter model acceptable | minimal | +| `boundary` | Zero trust at external boundaries (consumerโ†’DCM, DCMโ†’provider); internal components trust service mesh | dev, standard | +| `full` | Zero trust everywhere including internal component communication; every call authenticated and authorized | prod, fsi | +| `hardware_attested` | Full zero trust plus hardware attestation (TPM/HSM); component identity backed by hardware | sovereign | + +```yaml +zero_trust_policy_group: + handle: "system/group/zt-full" + concern_type: zero_trust_posture + posture: full + policies: + - all_component_communication: mtls_required + - credential_lifetime: PT30M + - revocation_check: every_call # not just at credential issuance + - session_continuation: re_verify_PT15M # re-verify identity during long operations + - failed_verification_response: terminate_and_alert +``` + +--- + +## 6. Federation Zero Trust โ€” The Tunnel Model + +### 6.1 Federation Tunnel as a Zero Trust Boundary + +A federation tunnel between DCM instances is a **mutually authenticated, encrypted, scoped channel** where both sides verify each other on every interaction. It is not a VPN โ€” it does not establish perimeter trust. Every message crossing the tunnel is authenticated, authorized, and subject to the five-check model. + +**"Zero trust to any outside DCM/provider"** is implemented by: the remote DCM instance has no implicit access to local resources. Every cross-instance operation requires a scoped federation credential. The tunnel establishes secure transport โ€” it does not establish trust. + +### 6.2 Federation Tunnel Structure + +```yaml +federation_tunnel: + uuid: + local_dcm_uuid: + remote_dcm_uuid: + tunnel_type: peer | parent_child | hub_spoke + trust_model: zero_trust # always; non-negotiable + + # Mutual authentication + authentication: + protocol: mtls + local_certificate_ref: + remote_certificate_pin: # pinned; not just chain-valid + trust_anchor: # common or cross-signed CA + certificate_rotation_interval: P90D + revocation_check: ocsp_stapling # real-time revocation check + + # Per-message signing + message_integrity: + signing_algorithm: ed25519 + local_signing_key_ref: + remote_verification_key_ref: + replay_protection: true # nonce + timestamp window PT5M + + # What the remote DCM may request from this DCM (inbound) + inbound_authorization: + - operation: catalog_query + permitted_resource_types: [Compute.VirtualMachine, Network.VLAN] + requires_cross_tenant_authorization: true + - operation: allocation_request + permitted_resource_types: [Network.IPAddress] + max_allocations_per_request: 10 + requires_cross_tenant_authorization: true + + # What this DCM may request from the remote (outbound) + outbound_authorization: + - operation: placement_query + permitted_resource_types: [Compute.VirtualMachine] + - operation: realized_state_query + permitted_entity_uuids: [] # scoped to specific entities + + # Data classification boundary (hard constraints) + data_boundary: + max_outbound_classification: restricted # never send sovereign/classified + max_inbound_classification: restricted + # sovereign profile: max_*_classification: internal + # classified profile: no federation permitted + + # Sovereignty scope + sovereignty_scope: + local_jurisdiction: EU + remote_jurisdiction: EU + cross_jurisdiction_permitted: false # fsi/sovereign: always false +``` + +### 6.3 Federation Credential Scoping + +Federation credentials are scoped to the specific operations declared in the tunnel authorization. A federation credential issued for `catalog_query` cannot be used for `allocation_request`: + +```yaml +federation_credential: + credential_uuid: + issued_by_dcm_uuid: + issued_to_dcm_uuid: + valid_until: # PT15M for fsi/sovereign + operation_scope: catalog_query + scoped_resource_types: [Compute.VirtualMachine] + non_transferable: true + tunnel_uuid: # bound to specific tunnel +``` + +### 6.4 Zero Trust in Hub-Spoke Federation + +In hub-spoke federation, the Hub DCM coordinates Regional DCMs. Zero trust means: +- The Hub DCM does not have root-level access to Regional DCMs โ€” it has explicitly scoped federation credentials +- A Regional DCM cannot impersonate the Hub DCM to another Regional DCM +- Cross-Regional-DCM operations route through the Hub with the Hub's authorization, not the originating Regional DCM's authorization +- The Hub DCM's accreditation is visible to Regional DCMs โ€” Regional DCMs can verify the Hub before accepting federation messages + +``` +RegionalDCM-A โ†’ HubDCM: authenticated; scoped to allocation_request +HubDCM โ†’ RegionalDCM-B: authenticated; scoped to realization_request + Hub presents its own credential to RegionalDCM-B + Not RegionalDCM-A's credential +RegionalDCM-B verifies: Hub certificate; Hub accreditation; data classification boundary +``` + +--- + +## 7. Profile-Governed Zero Trust Enforcement + +Zero trust enforcement levels are bound to deployment profiles. The profile determines which zero trust posture group is active: + +| Profile | Zero Trust Posture | Data Boundary | Federation | +|---------|-------------------|---------------|-----------| +| `minimal` | none | public/internal only | Not recommended | +| `dev` | boundary | up to confidential | Permitted with warnings | +| `standard` | boundary | up to restricted (with third-party accreditation) | Permitted | +| `prod` | full | up to restricted | Permitted with accreditation | +| `fsi` | full | up to restricted (with regulatory cert) | Restricted to same jurisdiction | +| `sovereign` | hardware_attested | sovereign stays sovereign (no crossing) | Zero crossing of sovereign data | + +The `sovereign` profile enforces the hardest constraint: **sovereign-classified data never crosses any boundary** โ€” not to providers, not to federation tunnels, not to Notification Providers with external endpoints. The enforcement is at the Data/Capability Matrix level as a `hard_constraint: true` rule that cannot be overridden by any policy. + +--- + +## 8. System Policies + +| Policy | Rule | +|--------|------| +| `ZT-001` | Network position grants zero trust. Every interaction is subject to the five-check model regardless of the caller's network location. | +| `ZT-002` | All DCM interaction credentials are scoped, short-lived, and non-transferable. Credential lifetime is profile-governed. | +| `ZT-003` | Data classified as `sovereign` or `classified` never crosses any DCM interaction boundary (provider dispatch, federation tunnel, notification delivery). This is a hard constraint enforced by the Data/Capability Matrix, not a configurable policy. | +| `ZT-004` | Federation tunnels use mutual TLS with certificate pinning and per-message signing. A tunnel establishes secure transport, not implicit trust. | +| `ZT-005` | Every interaction boundary check produces an audit record regardless of outcome. A denied interaction is audited as rigorously as a permitted one. | +| `ACC-001` | Accreditations are first-class DCM artifacts. They follow the standard lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired) and are subject to GitOps governance. | +| `ACC-002` | Accreditation gaps (missing, expired, or revoked accreditations required for active interactions) are always high or critical severity. The Recovery Policy governs the response. | +| `ACC-003` | PHI, sovereign, and classified field classifications are immutable once set. No policy may downgrade these classifications. | +| `ACC-004` | The Data/Capability Authorization Matrix is enforced at every outbound interaction boundary before dispatch. Fields failing the matrix check are stripped (STRIP_FIELD) or the request is blocked (DENY_REQUEST) per the matrix declaration. | +| `ACC-005` | DCM deployments themselves carry accreditations. A federation peer DCM can verify the remote DCM deployment's accreditation before accepting federation messages. | +| `ACC-006` | `zero_trust_posture` is the sixth Policy Group concern type. Profile defaults are: minimal=none, dev/standard=boundary, prod/fsi=full, sovereign=hardware_attested. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index dd0b8e6..417b013 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -1574,6 +1574,42 @@ resource_type_recovery_override: Resource-type override wins over Tenant override wins over profile default. + +--- + +## 10. Zero Trust Posture Policy Groups + +### 10.1 zero_trust_posture as a Concern Type + +`zero_trust_posture` is the sixth Policy Group concern type. It governs authentication requirements, credential lifetime, revocation check frequency, and hardware attestation requirements for all DCM interactions. + +See [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) Section 5 for the complete zero trust model. + +### 10.2 Four Zero Trust Posture Levels + +| Posture | Boundary | Internal | Hardware | Profile Default | +|---------|---------|----------|----------|----------------| +| `none` | Perimeter model | Trusted | Not required | minimal | +| `boundary` | Zero trust at external boundaries | Service mesh | Not required | dev, standard | +| `full` | Zero trust everywhere | Per-call auth | Not required | prod, fsi | +| `hardware_attested` | Zero trust everywhere | Per-call auth | Required (TPM/HSM) | sovereign | + +### 10.3 Credential Lifetime Defaults + +| Profile | Max credential lifetime | +|---------|------------------------| +| minimal | PT8H | +| dev | PT4H | +| standard | PT1H | +| prod | PT30M | +| fsi | PT15M | +| sovereign | PT15M + hardware attestation | + +### 10.4 Hard Data Boundary Constraints + +The sovereign profile enforces a hard constraint via the Data/Capability Authorization Matrix: **data classified as `sovereign` or `classified` never crosses any interaction boundary**. This constraint is declared with `hard_constraint: true` in the federation boundary matrix and cannot be overridden by any policy, profile, or operator action. It is enforced at the matrix level, not the policy level. + + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md index f83d51f..de638d1 100644 --- a/content/docs/architecture/data-model/resource-service-entities.md +++ b/content/docs/architecture/data-model/resource-service-entities.md @@ -557,6 +557,39 @@ The following table supersedes the table in Section 7.2 with clearer DCM respons | `RSE-013` | Provider Update Notifications that require consumer approval place the entity in PENDING_REVIEW state. The provider receives a "pending_approval" response and the change is queued until resolution. | + +### 7c. Provider Accreditation Registration + +Every Service Provider must declare its accreditation status during registration. Accreditation declarations are references to accreditation records registered in DCM's accreditation registry (see [doc 26](26-accreditation-and-authorization-matrix.md)). + +```yaml +provider_registration: + # ... existing fields ... + accreditations: + - accreditation_uuid: # reference to registered accreditation record + framework: fedramp_high + status: active + valid_until: "2026-12-31" + + - accreditation_uuid: + framework: hipaa + accreditation_type: baa + status: active + + # Self-declared compliance (lowest trust; used when no formal accreditation exists) + self_declared_compliance: + frameworks: [iso_27001] + last_self_review: "2026-01-15" + evidence_ref: + + # Maximum data classification this provider is permitted to handle + # DCM computes this from active accreditations; self_declared_max is the fallback + self_declared_max_data_classification: confidential +``` + +Providers without any accreditation records are treated as `self_declared` level and are subject to the most restrictive authorization matrix rules. They may only receive data classified as `public` or `internal`. + + ## 8. Entity Relationships Every Resource/Service Entity carries a `relationships` section declaring its relationships to other entities โ€” internal DCM entities, external data entities, and business context entities. The relationship model is universal โ€” the same structure is used for all relationship types. diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index ed9e76f..2219a76 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -261,7 +261,18 @@ Response 200: "sovereignty": { "available_in_regions": ["EU-WEST", "EU-NORTH"], "data_residency_guarantee": "EU" - } + }, + "accreditations": [ + { + "framework": "hipaa", + "accreditation_type": "baa", + "status": "active", + "valid_until": "", + "max_data_classification": "phi" + } + ], + "zero_trust_posture": "full", + "max_data_classification_accepted": "phi" } ``` @@ -491,6 +502,12 @@ Response 200: "min_auth_level": "oidc_mfa" }, + "data_classification_summary": { + "fields_with_phi": 0, + "fields_with_restricted": 2, + "highest_classification": "restricted" + }, + "pending_provider_notifications": [ { "notification_uuid": "", From bc1181a265ecbe18a5388d5644dbc465ec98eaa5 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Fri, 27 Mar 2026 21:17:14 -0500 Subject: [PATCH 25/49] Additional details for Placement engine details, Lifecycle Constraint Enforcer, search index, admin api spec. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 21 + .../data-model/control-plane-components.md | 372 ++++++++++++++ .../specifications/admin-api-spec.md | 483 ++++++++++++++++++ 3 files changed, 876 insertions(+) create mode 100644 content/docs/architecture/specifications/admin-api-spec.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 177986a..5021711 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -424,6 +424,13 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | +| **reserve_query** | Parallel capacity queries to all eligible provider candidates; PT5M capacity hold; non-responders and insufficient-capacity providers excluded | +| **consistent hash** | Final placement tie-breaker: SHA-256(request_uuid+resource_type+sorted_candidates); deterministic; never round-robin | +| **Lifecycle Constraint Enforcer** | Monitors TTL/expiry/max_execution_time; fires expiry actions through standard pipeline; grace period before action; Process Resources: immediate FAILED on breach; LCE-001โ€“005 | +| **Search Index** | Non-authoritative queryable projection of GitOps stores; indexes key fields; returns git_path for full payload; max staleness PT5M; always rebuildable; SIX-001โ€“004 | +| **Admin API** | Platform admin REST interface: Tenant lifecycle, provider review, accreditation approval, discovery trigger, orphan resolution, recovery decisions, quota management, Search Index rebuild, bootstrap operations | +| **PENDING_EXPIRY_ACTION** | Entity state when expiry action fails to execute; Lifecycle Constraint Enforcer retries per Recovery Policy; Platform Admin notified urgency: high | | **data_classification** | First-class field metadata: public/internal/confidential/restricted/phi/pci/sovereign/classified; phi/sovereign/classified are immutable once set | | **Accreditation** | Formal versioned attestation that a component satisfies a compliance framework; issued by an Accreditor; carries validity period; lifecycle: developingโ†’proposedโ†’activeโ†’expired/revoked | | **Accreditor** | Entity that issues accreditations: government body, regulatory body, QSA, certification body, or internal audit team | @@ -2508,6 +2515,13 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | +| **reserve_query** | Parallel capacity queries to all eligible provider candidates; PT5M capacity hold; non-responders and insufficient-capacity providers excluded | +| **consistent hash** | Final placement tie-breaker: SHA-256(request_uuid+resource_type+sorted_candidates); deterministic; never round-robin | +| **Lifecycle Constraint Enforcer** | Monitors TTL/expiry/max_execution_time; fires expiry actions through standard pipeline; grace period before action; Process Resources: immediate FAILED on breach; LCE-001โ€“005 | +| **Search Index** | Non-authoritative queryable projection of GitOps stores; indexes key fields; returns git_path for full payload; max staleness PT5M; always rebuildable; SIX-001โ€“004 | +| **Admin API** | Platform admin REST interface: Tenant lifecycle, provider review, accreditation approval, discovery trigger, orphan resolution, recovery decisions, quota management, Search Index rebuild, bootstrap operations | +| **PENDING_EXPIRY_ACTION** | Entity state when expiry action fails to execute; Lifecycle Constraint Enforcer retries per Recovery Policy; Platform Admin notified urgency: high | | **data_classification** | First-class field metadata: public/internal/confidential/restricted/phi/pci/sovereign/classified; phi/sovereign/classified are immutable once set | | **Accreditation** | Formal versioned attestation that a component satisfies a compliance framework; issued by an Accreditor; carries validity period; lifecycle: developingโ†’proposedโ†’activeโ†’expired/revoked | | **Accreditor** | Entity that issues accreditations: government body, regulatory body, QSA, certification body, or internal audit team | @@ -3466,6 +3480,13 @@ ZT-001 through ZT-005 (zero trust) + ACC-001 through ACC-006 (accreditation). Ke | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | +| **reserve_query** | Parallel capacity queries to all eligible provider candidates; PT5M capacity hold; non-responders and insufficient-capacity providers excluded | +| **consistent hash** | Final placement tie-breaker: SHA-256(request_uuid+resource_type+sorted_candidates); deterministic; never round-robin | +| **Lifecycle Constraint Enforcer** | Monitors TTL/expiry/max_execution_time; fires expiry actions through standard pipeline; grace period before action; Process Resources: immediate FAILED on breach; LCE-001โ€“005 | +| **Search Index** | Non-authoritative queryable projection of GitOps stores; indexes key fields; returns git_path for full payload; max staleness PT5M; always rebuildable; SIX-001โ€“004 | +| **Admin API** | Platform admin REST interface: Tenant lifecycle, provider review, accreditation approval, discovery trigger, orphan resolution, recovery decisions, quota management, Search Index rebuild, bootstrap operations | +| **PENDING_EXPIRY_ACTION** | Entity state when expiry action fails to execute; Lifecycle Constraint Enforcer retries per Recovery Policy; Platform Admin notified urgency: high | | **data_classification** | First-class field metadata: public/internal/confidential/restricted/phi/pci/sovereign/classified; phi/sovereign/classified are immutable once set | | **Accreditation** | Formal versioned attestation that a component satisfies a compliance framework; issued by an Accreditor; carries validity period; lifecycle: developingโ†’proposedโ†’activeโ†’expired/revoked | | **Accreditor** | Entity that issues accreditations: government body, regulatory body, QSA, certification body, or internal audit team | diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index c066713..3c4bc22 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -264,3 +264,375 @@ Step 4 โ€” Cost Analysis (if available and determinable): --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + +--- + +## 4. The Placement Engine + +### 4.1 Role + +The Placement Engine selects the specific Service Provider that will fulfill a resource request. It runs as step 6 of the Request Payload Processor assembly pipeline and is invoked by the Request Orchestrator after all policies have been evaluated and the assembled payload is ready for dispatch. + +The Placement Engine does not make business decisions โ€” those are made by policies (which inject constraints and preferences). The Placement Engine applies those constraints to find eligible providers, then deterministically resolves ties using a declared hierarchy. + +### 4.2 Input and Output + +**Input:** +- Assembled payload with full field-level provenance +- Sovereignty constraints (from compliance domain profile and any policy-injected constraints) +- Accreditation requirements (from Data/Capability Authorization Matrix) +- Preference scores or preferred provider UUIDs (if injected by Transformation policy) +- Tenant affinity declarations + +**Output:** +- Selected provider UUID +- Placement reason (why this provider was selected) +- Sovereignty satisfaction record (which constraints were checked and passed) +- Reserve confirmation (provider confirmed it has capacity for this specific request) + +This output is written to `placement.yaml` in the Requested Store directory. + +### 4.3 Placement Algorithm โ€” Six Steps + +``` +Step 1: Sovereignty Pre-Filter + Eliminate any provider whose sovereignty_declaration does not satisfy + the request's sovereignty constraints. + โ†’ Providers that fail this step are never contacted. + โ†’ If zero providers remain: RESERVE_QUERY_ALL_EXHAUSTED recovery trigger fires. + +Step 2: Accreditation Filter + Eliminate any provider that does not hold the required accreditations + for the data classifications present in the assembled payload. + โ†’ Checked against the active Data/Capability Authorization Matrix. + โ†’ Providers with accreditation gaps are excluded. + +Step 3: Capability Filter + Eliminate any provider that does not declare support for the + requested resource type and all required capabilities. + โ†’ Based on provider registration catalog item declarations. + +Step 4: Reserve Query + Send a reserve query to each remaining candidate provider in parallel. + โ†’ Providers have reserve_query_timeout to respond (profile-governed: PT5โ€“30S). + โ†’ Providers that do not respond within timeout: excluded from this placement cycle. + โ†’ Providers that respond with INSUFFICIENT_CAPACITY: excluded; DCM updates + internal capacity rating for that provider. + โ†’ Providers that confirm capacity: advance to tie-breaking. + +Step 5: Tie-Breaking (deterministic hierarchy) + Applied when multiple providers confirmed capacity in Step 4: + + Priority 1: Policy preference + A Transformation policy injected a preference_score or preferred_provider_uuid. + Highest preference_score wins. preferred_provider_uuid is absolute โ€” skips Steps 2-6. + + Priority 2: Provider declared priority + Providers declare a numeric priority at registration (default: 50). + Higher value = preferred when all else equal. + + Priority 3: Tenant affinity + Tenant's Policy Group declares preferred providers for specific resource types. + Affinity preference is a soft preference โ€” does not override accreditation or sovereignty. + + Priority 4: Cost Analysis + Cost Analysis component provides current cost per unit per candidate provider. + Prefer lower total cost (CapEx + OpEx + licensing). + Skip if: Cost Analysis unavailable, data stale > PT1H, or cost difference < 5%. + + Priority 5: Least loaded + Prefer provider with lower current capacity utilization from reserve_query response. + Skip if: utilization difference < 10%, or utilization data not returned. + + Priority 6: Consistent hash (final tiebreaker โ€” always resolves) + SHA-256(request_uuid + resource_type + sorted_candidate_uuids) + Deterministic โ€” same request always resolves to the same provider in a stable cluster. + Never round-robin. + +Step 6: Reserve Confirmation + Notify the selected provider that its reservation is confirmed. + Other providers that responded to the reserve query receive a reservation release. + โ†’ Prevents capacity holds from accumulating across providers for the same request. +``` + +### 4.4 Reserve Query Protocol + +```yaml +reserve_query: + query_uuid: # idempotency key + entity_uuid: + resource_type: Compute.VirtualMachine + resource_type_spec_version: "2.1.0" + requested_fields: + cpu_count: 4 + memory_gb: 8 + storage_gb: 100 + sovereignty_requirements: + data_residency: EU + reservation_hold_ttl: PT5M # provider holds capacity for this duration + # released when: confirmed, rejected, or TTL expires +``` + +```yaml +reserve_query_response: + query_uuid: + provider_uuid: + status: confirmed | insufficient_capacity | capability_not_supported + capacity_held_until: # if confirmed + utilization_pct: 42 # current load; used for Step 5 tiebreaking + cost_per_hour: 0.32 # if Cost Analysis integration enabled + currency: USD +``` + +### 4.5 Placement Configuration + +```yaml +placement_engine_config: + reserve_query_timeout: PT10S # profile-governed default + parallel_reserve_queries: true # always true; all candidates queried simultaneously + max_candidates_per_placement: 10 # cap on parallel reserve queries + cost_freshness_max: PT1H + cost_difference_threshold: 0.05 # 5% โ€” skip cost step if within this band + utilization_difference_threshold: 0.10 # 10% โ€” skip utilization step if within this band + reservation_hold_ttl: PT5M +``` + +### 4.6 Placement Failure and Recovery + +When placement cannot find an eligible provider: + +| Failure Reason | Recovery Trigger | +|---------------|-----------------| +| All providers fail sovereignty filter | `RESERVE_QUERY_ALL_EXHAUSTED` | +| All providers fail accreditation filter | `RESERVE_QUERY_ALL_EXHAUSTED` | +| All providers respond INSUFFICIENT_CAPACITY | `RESERVE_QUERY_ALL_EXHAUSTED` | +| All reserve queries time out | `RESERVE_QUERY_ALL_EXHAUSTED` | + +The `RESERVE_QUERY_ALL_EXHAUSTED` trigger fires the active Recovery Policy. Default action per profile: standard/prod โ†’ `NOTIFY_AND_WAIT`; dev โ†’ `RETRY` with exponential backoff. + +### 4.7 Placement System Policies + +| Policy | Rule | +|--------|------| +| `PLC-001` | Sovereignty pre-filter runs before any provider is contacted. Providers that fail sovereignty constraints never receive reserve queries. | +| `PLC-002` | Accreditation filter runs before reserve queries. Providers without required accreditations for the payload's data classifications are excluded. | +| `PLC-003` | Reserve queries are sent in parallel to all eligible candidates. Sequential querying is not permitted โ€” it introduces latency and prevents fair capacity comparison. | +| `PLC-004` | The consistent hash tiebreaker is always the final tiebreaker. It ensures deterministic provider selection for identical inputs without round-robin non-determinism. | +| `PLC-005` | A confirmed reservation hold must be released when not used โ€” either by confirmation dispatch or by explicit release on timeout. Capacity holds must not accumulate silently. | +| `PLC-006` | The Placement Engine never selects a provider based solely on network position or co-location. Every selection is based on declared constraints, policies, and the tie-breaking hierarchy. | + +--- + +## 5. The Lifecycle Constraint Enforcer + +### 5.1 Role + +The Lifecycle Constraint Enforcer is a DCM control plane component that monitors all realized entities against their declared lifecycle constraints and fires expiry actions when constraints are reached. It is the authoritative enforcer of TTL, expiry date, and maximum execution time declarations. + +Lifecycle constraint enforcement is a DCM concern โ€” not a provider concern. The provider does not need to know about or implement any TTL logic. + +### 5.2 What It Monitors + +The Lifecycle Constraint Enforcer monitors three categories of constraint: + +**Category 1 โ€” Entity TTL:** +Duration-based: entity expires T duration after a reference point (realization, creation, last modification). + +**Category 2 โ€” Entity Expiry Date:** +Calendar-based: entity expires at an absolute timestamp. + +**Category 3 โ€” Process Resource Maximum Execution Time:** +Process Resources must declare `max_execution_time`. The Enforcer monitors all executing Process Resources and fires `on_max_exceeded` when the limit is reached. + +### 5.3 Monitoring Loop + +``` +Lifecycle Constraint Enforcer runs continuously: + + Every cycle (interval: PT1M for standard/prod; PT5M for minimal/dev): + + Query Realized Store for entities with: + lifecycle_state IN [OPERATIONAL, SUSPENDED, EXECUTING] + AND lifecycle_constraints declared + AND NOT already in terminal state + + For each entity: + Compute time_remaining = constraint_expiry - now() + + If time_remaining <= warn_before_expiry: + If warn_not_yet_sent: + Emit: entity.ttl_warning notification + Record: WARNING_EMITTED in entity provenance + + If time_remaining <= 0: + Execute on_expiry action (see Section 5.4) +``` + +### 5.4 Expiry Action Execution + +When a lifecycle constraint fires, the Enforcer executes the declared `on_expiry` action: + +| Action | Behavior | +|--------|---------| +| `decommission` | Submit a decommission request through the standard pipeline โ€” produces Requested State, dispatches to provider, full audit trail | +| `suspend` | Submit a suspend request through the standard pipeline | +| `notify` | Fire `entity.ttl_expired` notification to entity owner; no automated action | +| `review` | Entity enters PENDING_EXPIRY_ACTION state; Platform Admin and owner notified | +| `escalate` | Immediately escalate to Platform Admin; entity enters PENDING_EXPIRY_ACTION state | + +**Grace period:** Expiry actions are not immediate. The Enforcer respects the declared `grace_period` (default PT1H) โ€” the action fires `grace_period` after the constraint expires, giving human operators a window to intervene. + +**Action failure:** If the expiry action fails to execute (provider unreachable, dependency conflict), the entity enters `PENDING_EXPIRY_ACTION` state (LTC-005). The Enforcer retries per the active Recovery Policy. Platform Admin is notified with urgency: high. + +### 5.5 Expiry Audit Records + +Every expiry-related event produces an audit record: + +```yaml +audit_record: + action: EXPIRY_WARNING | EXPIRY_ACTION_FIRED | EXPIRY_ACTION_FAILED | + PENDING_EXPIRY_ACTION_ENTERED + actor: + type: system + system_actor: + component: lifecycle_constraint_enforcer + trigger: ttl_reached | expires_at_reached | max_execution_time_reached + entity_uuid: + details: + constraint_type: ttl | expires_at | max_execution_time + constraint_value: + action_taken: decommission | suspend | notify | review | escalate + grace_period_remaining: +``` + +### 5.6 Process Resource Enforcement + +Process Resources require `max_execution_time` (mandatory). The Enforcer monitors all EXECUTING Process Resources: + +``` +Process Resource enters EXECUTING state + โ”‚ + โ–ผ Enforcer records: execution_started_at; computes execution_timeout_at + โ”‚ + โ–ผ On every monitoring cycle: + โ”‚ If now() >= execution_timeout_at: + โ”‚ Emit: PROCESS_TIMEOUT event + โ”‚ Entity state โ†’ FAILED + โ”‚ Recovery Policy: COMPENSATION_FAILED trigger if resources were modified + โ”‚ Notification: entity owner + Platform Admin (urgency: high) +``` + +### 5.7 Lifecycle Constraint Enforcer Policies + +| Policy | Rule | +|--------|------| +| `LCE-001` | The Lifecycle Constraint Enforcer runs as a continuous monitor. It does not rely on provider callbacks or event triggers for expiry detection โ€” it polls based on declared constraints. | +| `LCE-002` | Expiry actions are submitted through the standard DCM request pipeline. Decommission-on-expiry produces a Requested State record with `actor: system/lifecycle-constraint-enforcer`. | +| `LCE-003` | The Enforcer respects the declared grace_period before firing expiry actions. Grace period gives human operators a window to intervene before automated action. | +| `LCE-004` | Process Resource max_execution_time enforcement fires immediately on breach โ€” no grace period. Hung processes are failed immediately to prevent resource leaks. | +| `LCE-005` | Expiry action failures enter PENDING_EXPIRY_ACTION state. The Enforcer retries per the active Recovery Policy. Indefinite retry without escalation is not permitted. | + +--- + +## 6. The Search Index + +### 6.1 Role + +The Search Index is a **non-authoritative, queryable projection** of the GitOps stores (Intent Store and Requested Store). It enables millisecond-latency queries against stored entities without traversing Git history, while the GitOps stores remain the authoritative source of truth. + +The Search Index is a Storage Provider sub-type. It has its own registration, health check, and sovereignty declaration. It is never the source of truth โ€” if the Search Index and the GitOps store disagree, the GitOps store wins unconditionally. + +### 6.2 What It Indexes + +The Search Index maintains a projection of key fields from Intent State and Requested State records, enabling queries without retrieving full payloads from Git: + +```yaml +search_index_record: + entity_uuid: + entity_handle: + resource_type: Compute.VirtualMachine + resource_type_category: Compute + tenant_uuid: + lifecycle_state: OPERATIONAL + drift_status: clean + provider_uuid: + deployment_posture: prod + compliance_domains: [hipaa] + data_classifications: [restricted] # highest classification in entity + created_at: + updated_at: + cost_per_hour: 0.32 + currency: USD + git_path: intent-store/tenant-uuid/Compute/VirtualMachine/entity-uuid/intent.yaml + # git_path is the pointer back to the authoritative record + tags: { environment: production, team: payments } +``` + +### 6.3 Required Query Operations + +| Operation | Description | +|-----------|-------------| +| `find_by_uuid(entity_uuid)` | Return index record for a single entity | +| `find_by_tenant(tenant_uuid, filters)` | Return all entities for a Tenant with optional field filters | +| `find_by_resource_type(fqn, filters)` | Return all entities of a resource type | +| `find_by_provider(provider_uuid, filters)` | Return all entities hosted at a provider | +| `find_by_lifecycle_state(state, tenant_uuid)` | Return entities in a given lifecycle state | +| `find_by_drift_status(status, tenant_uuid)` | Return drifted or clean entities | +| `find_by_data_classification(classification)` | Return entities containing data of a given classification | +| `full_text_search(query, tenant_uuid)` | Full-text search across handle, display_name, tags | + +All queries return the `git_path` โ€” consumers fetch the full payload from Git if needed. + +### 6.4 Consistency Model + +The Search Index is **eventually consistent** with the GitOps stores. There is a defined maximum staleness: + +```yaml +search_index_consistency: + max_staleness: PT5M # index must be within 5 minutes of GitOps store + profile_overrides: + prod: PT2M + fsi: PT1M + sovereign: PT1M + on_staleness_exceeded: + action: degrade_with_warning # serve results with staleness warning + alert: platform_admin # alert on staleness exceeding 2ร— max + rebuild_on_recovery: true # full index rebuild from Git history on failure + rebuild_max_duration: PT4H # must complete within 4 hours for standard+ +``` + +### 6.5 Unavailability Behavior + +If the Search Index is unavailable: +- DCM degrades search operations gracefully: returns a `503 Service Degraded` response with a reference to the authoritative Git store +- Writes are not affected โ€” GitOps stores are written directly; the index is updated asynchronously +- On recovery: the Search Index rebuilds from Git history +- No data is lost if the index is lost โ€” it is always reconstructable from Git + +### 6.6 Search Index Policies + +| Policy | Rule | +|--------|------| +| `SIX-001` | The Search Index is non-authoritative. GitOps stores win on any disagreement. Consumers must be prepared to receive a git_path and fetch from the authoritative store. | +| `SIX-002` | The Search Index must be rebuildable from Git history at any time. Implementations that cannot perform a full index rebuild are non-conformant. | +| `SIX-003` | Search Index staleness beyond the profile-governed maximum triggers a platform admin alert. Staleness is surfaced in query responses โ€” consumers are never served stale data silently. | +| `SIX-004` | Search Index unavailability degrades queries without impacting writes. Write operations proceed directly to the authoritative GitOps stores regardless of Search Index availability. | + +--- + +## 7. Related Policies โ€” Full Component Set + +| Policy | Rule | +|--------|------| +| `CTL-001` | The Request Orchestrator is the single event bus for all request lifecycle events. No component communicates directly with another component outside of events published to the Request Orchestrator. | +| `CTL-002` | Policies ARE the orchestration. The Request Orchestrator does not contain hardcoded pipeline logic. | +| `CTL-003` | Dynamic and static flows compose naturally. Static flows are Policy Groups with concern_type: orchestration_flow and ordered: true. | +| `CTL-004` | Cost Analysis is not a billing system. It provides cost signals for placement and attribution. | +| `CTL-005` | Cost Analysis unavailability never blocks placement. | +| `PLC-001` through `PLC-006` | Placement Engine policies (see Section 4.7) | +| `LCE-001` through `LCE-005` | Lifecycle Constraint Enforcer policies (see Section 5.7) | +| `SIX-001` through `SIX-004` | Search Index policies (see Section 6.6) | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/admin-api-spec.md b/content/docs/architecture/specifications/admin-api-spec.md new file mode 100644 index 0000000..4ce221a --- /dev/null +++ b/content/docs/architecture/specifications/admin-api-spec.md @@ -0,0 +1,483 @@ +--- +title: "DCM Admin API Specification" +type: docs +weight: 1 +--- + +> **โš ๏ธ Work in Progress** +> +> This specification defines the DCM Admin API โ€” the platform administration interface. Published to share design direction and invite feedback. Do not build production integrations against this specification until it reaches draft status. + +**Version:** 0.1.0-draft +**Status:** Design โ€” Not yet implemented +**Document Type:** Technical Specification +**Related Documents:** [Consumer API Specification](consumer-api-spec.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) + +--- + +## Abstract + +The Admin API is the platform administration interface for DCM. It is served through the same Ingress API as the Consumer API and Provider API but is restricted to actors with `platform_admin` or `tenant_admin` roles. It covers operations that consumers cannot perform โ€” Tenant lifecycle management, provider registration review, accreditation approval, quota administration, discovery management, orphan resolution, recovery decision escalation, and bootstrap operations. + +--- + +## 1. Authentication and Authorization + +All Admin API endpoints require Bearer token authentication (same as Consumer API). Role requirements are declared per endpoint: + +| Role | Scope | +|------|-------| +| `platform_admin` | All Admin API operations across all Tenants | +| `tenant_admin` | Tenant-scoped Admin API operations for their own Tenant only | + +Base URL: `/api/v1/admin/` + +Step-up MFA is required for destructive operations (Tenant decommission, accreditation revocation, bootstrap credential rotation) regardless of session MFA status. + +--- + +## 2. Tenant Management + +### 2.1 List Tenants + +``` +GET /api/v1/admin/tenants +Role: platform_admin + +Query params: status=, page, page_size + +Response 200: +{ + "tenants": [ + { + "tenant_uuid": "", + "handle": "payments-team", + "display_name": "Payments Platform", + "status": "active", + "deployment_posture": "prod", + "compliance_domains": ["hipaa"], + "recovery_profile": "notify-and-wait", + "entity_count": 142, + "created_at": "" + } + ], + "total": 12 +} +``` + +### 2.2 Create Tenant + +``` +POST /api/v1/admin/tenants +Role: platform_admin + +{ + "handle": "new-team", + "display_name": "New Team", + "deployment_posture": "standard", + "compliance_domains": [], + "recovery_profile_override": null, + "initial_admin_actor_uuid": "" +} + +Response 201 Created: +{ + "tenant_uuid": "", + "status": "active" +} +``` + +### 2.3 Suspend / Reinstate Tenant + +``` +POST /api/v1/admin/tenants/{tenant_uuid}/suspend +POST /api/v1/admin/tenants/{tenant_uuid}/reinstate +Role: platform_admin + +{ + "reason": "", + "notify_tenant_admin": true +} +``` + +### 2.4 Decommission Tenant + +``` +DELETE /api/v1/admin/tenants/{tenant_uuid} +Role: platform_admin +Requires: step-up MFA + +{ + "reason": "", + "force": false, # true: decommission even if active entities remain + "notify_tenant_admin": true +} + +Response 409 Conflict (if active entities and force=false): +{ + "error": "tenant_has_active_entities", + "active_entity_count": 47, + "resolution": "Decommission all entities first, or use force=true" +} +``` + +--- + +## 3. Provider Management + +### 3.1 List Registered Providers + +``` +GET /api/v1/admin/providers +Role: platform_admin + +Query params: type=, status= + +Response 200: +{ + "providers": [ + { + "provider_uuid": "", + "handle": "eu-west-prod-1", + "provider_type": "service", + "status": "active", + "health": "healthy", + "accreditation_count": 2, + "max_data_classification": "phi" + } + ] +} +``` + +### 3.2 Review Provider Registration + +New provider registrations in `proposed` status require platform admin review: + +``` +GET /api/v1/admin/providers/pending +Role: platform_admin + +POST /api/v1/admin/providers/{provider_uuid}/approve +POST /api/v1/admin/providers/{provider_uuid}/reject +{ + "reason": "" +} +``` + +### 3.3 Suspend Provider + +``` +POST /api/v1/admin/providers/{provider_uuid}/suspend +Role: platform_admin + +{ + "reason": "", + "affect_existing_entities": "notify_only | block_new_requests | migrate" +} +``` + +--- + +## 4. Accreditation Management + +### 4.1 List Accreditations + +``` +GET /api/v1/admin/accreditations +Role: platform_admin + +Query params: subject_type, framework, status= + +Response 200: +{ + "accreditations": [ + { + "accreditation_uuid": "", + "subject_uuid": "", + "subject_type": "service_provider", + "framework": "hipaa", + "accreditation_type": "baa", + "status": "active", + "valid_until": "", + "days_until_expiry": 89 + } + ] +} +``` + +### 4.2 Approve Accreditation + +``` +POST /api/v1/admin/accreditations/{accreditation_uuid}/approve +Role: platform_admin +Requires: step-up MFA + +{ + "review_notes": "", + "certificate_verified": true +} +``` + +### 4.3 Revoke Accreditation + +``` +DELETE /api/v1/admin/accreditations/{accreditation_uuid} +Role: platform_admin +Requires: step-up MFA + +{ + "revocation_reason": "", + "affected_entity_action": "notify_only | block_new_requests | migrate_entities" +} +``` + +--- + +## 5. Discovery Management + +### 5.1 Trigger Discovery + +``` +POST /api/v1/admin/discovery/trigger +Role: platform_admin | tenant_admin + +{ + "scope": "entity | resource_type | provider | tenant", + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "provider_uuid": "", + "tenant_uuid": "", + "reason": "incident investigation", + "priority": "high | standard | background" +} + +Response 202 Accepted: +{ + "discovery_job_uuid": "", + "status": "queued", + "priority": "high", + "estimated_start": "" +} +``` + +### 5.2 Discovery Job Status + +``` +GET /api/v1/admin/discovery/jobs/{discovery_job_uuid} + +Response 200: +{ + "discovery_job_uuid": "", + "status": "running | completed | failed", + "entities_discovered": 47, + "new_entities_found": 2, + "started_at": "", + "completed_at": "", + "orphan_candidates_found": 1 +} +``` + +--- + +## 6. Orphan Management + +### 6.1 List Orphan Candidates + +``` +GET /api/v1/admin/orphans +Role: platform_admin + +Query params: provider_uuid, status= + +Response 200: +{ + "orphan_candidates": [ + { + "orphan_candidate_uuid": "", + "provider_uuid": "", + "provider_entity_id": "vm-0a1b2c3d", + "suspected_request_uuid": "", + "resource_type": "Compute.VirtualMachine", + "discovered_at": "", + "status": "under_review" + } + ] +} +``` + +### 6.2 Resolve Orphan Candidate + +``` +POST /api/v1/admin/orphans/{orphan_candidate_uuid}/resolve +Role: platform_admin + +{ + "resolution": "manual_decommission | adopt_into_dcm | mark_false_positive", + "reason": "", + "target_tenant_uuid": "" # required if resolution=adopt_into_dcm +} +``` + +--- + +## 7. Recovery Decision Management + +Platform admins can resolve pending recovery decisions for any entity: + +``` +GET /api/v1/admin/recovery-decisions/pending +Role: platform_admin + +Response 200: +{ + "pending_decisions": [ + { + "recovery_decision_uuid": "", + "entity_uuid": "", + "trigger": "DISPATCH_TIMEOUT", + "entity_state": "TIMEOUT_PENDING", + "deadline": "", + "tenant_uuid": "" + } + ] +} + +POST /api/v1/admin/recovery-decisions/{recovery_decision_uuid} +Role: platform_admin + +{ + "action": "DRIFT_RECONCILE | DISCARD_AND_REQUEUE | DISCARD_NO_REQUEUE", + "reason": "" +} +``` + +--- + +## 8. Quota Management + +### 8.1 View Tenant Quotas + +``` +GET /api/v1/admin/tenants/{tenant_uuid}/quotas +Role: platform_admin | tenant_admin + +Response 200: +{ + "quotas": [ + { + "resource_type": "Compute.VirtualMachine", + "limit": 100, + "current_usage": 47, + "policy_uuid": "" + } + ] +} +``` + +### 8.2 Update Quota + +``` +PUT /api/v1/admin/tenants/{tenant_uuid}/quotas/{resource_type} +Role: platform_admin + +{ + "new_limit": 150, + "reason": "Q2 capacity increase approved by FinOps" +} +``` + +--- + +## 9. Search Index Management + +``` +POST /api/v1/admin/search-index/rebuild +Role: platform_admin + +{ + "scope": "full | tenant | resource_type", + "tenant_uuid": "", + "reason": "Recovery after index corruption" +} + +Response 202 Accepted: +{ + "rebuild_job_uuid": "", + "estimated_duration": "PT2H", + "degraded_during_rebuild": true +} + +GET /api/v1/admin/search-index/status + +Response 200: +{ + "status": "healthy | degraded | rebuilding | unavailable", + "staleness_seconds": 42, + "last_full_rebuild": "", + "entity_count": 8421 +} +``` + +--- + +## 10. Bootstrap Operations + +### 10.1 Rotate Bootstrap Admin Credential + +``` +POST /api/v1/admin/bootstrap/rotate-credential +Role: platform_admin +Requires: step-up MFA (hardware_token_mfa for fsi/sovereign) + +{ + "new_credential_ref": "", + "reason": "Initial bootstrap credential rotation" +} +``` + +### 10.2 Deployment Health + +``` +GET /api/v1/admin/health + +Response 200: +{ + "overall": "healthy | degraded | critical", + "components": [ + { "component": "request_orchestrator", "status": "healthy" }, + { "component": "policy_engine", "status": "healthy" }, + { "component": "placement_engine", "status": "healthy" }, + { "component": "lifecycle_constraint_enforcer", "status": "healthy" }, + { "component": "discovery_scheduler", "status": "healthy" }, + { "component": "notification_router", "status": "healthy" }, + { "component": "cost_analysis", "status": "healthy" }, + { "component": "search_index", "status": "degraded", "staleness_seconds": 180 }, + { "component": "intent_store", "status": "healthy" }, + { "component": "requested_store", "status": "healthy" }, + { "component": "realized_store", "status": "healthy" } + ], + "active_profile": { + "deployment_posture": "prod", + "compliance_domains": ["hipaa"], + "recovery_posture": "notify-and-wait", + "zero_trust_posture": "full" + } +} +``` + +--- + +## 11. Error Model + +Same as Consumer API. Additional admin-specific codes: + +| HTTP Status | Error Code | Meaning | +|-------------|-----------|---------| +| 403 | `insufficient_role` | Operation requires platform_admin; actor is tenant_admin | +| 403 | `cross_tenant_denied` | tenant_admin attempting operation outside their Tenant | +| 409 | `tenant_has_active_entities` | Tenant decommission blocked; active entities remain | +| 409 | `provider_has_active_entities` | Provider decommission blocked; entities hosted there | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From 1f0b760988169cfdc8134ebb98072d0f19cb61c0 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 09:27:22 -0500 Subject: [PATCH 26/49] Added Governance matrix / model. DCM Registration flow. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 100 ++- .../accreditation-and-authorization-matrix.md | 5 + .../data-model/control-plane-components.md | 147 ++- .../data-model/governance-matrix.md | 816 +++++++++++++++++ .../specifications/registration-spec.md | 835 ++++++++++++++++++ 5 files changed, 1898 insertions(+), 5 deletions(-) create mode 100644 content/docs/architecture/data-model/governance-matrix.md create mode 100644 content/docs/architecture/specifications/registration-spec.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 5021711..7f54d2c 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -424,6 +424,16 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | +| **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | +| **sovereignty_zone** | Registered DCM artifact declaring geopolitical/regulatory boundary; rules reference zones by ID; inter-zone agreements declared explicitly | +| **STRIP_FIELD** | Governance matrix decision: remove named fields from payload and proceed; if stripped field is required โ†’ DENY_REQUEST | +| **REDACT** | Governance matrix decision: replace field value with ``; field presence preserved; receiver knows field exists but not its value | +| **Provider Type Registry** | Three-tier registry of approved provider types; each entry declares permissions, default_approval_method, enabled_in_profiles, capability_schema_ref | +| **registration_token** | Pre-issued by platform admin; scoped to provider_type/handle_pattern/zone; single_use; grants_auto_approval flag; value presented once only | +| **approval_method** | Registration approval: auto | human_review | dual_approval | committee; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | +| **Drift Reconciliation Component** | Control plane component; compares Discovered vs Realized State; produces drift records and events; never writes to Realized Store; DRC-001โ€“005 | +| **drift_record** | Artifact produced by Drift Reconciliation; field-by-field comparison result with severity classification; unsanctioned flag; status tracking through resolution | | **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | | **reserve_query** | Parallel capacity queries to all eligible provider candidates; PT5M capacity hold; non-responders and insufficient-capacity providers excluded | | **consistent hash** | Final placement tie-breaker: SHA-256(request_uuid+resource_type+sorted_candidates); deterministic; never round-robin | @@ -2515,6 +2525,16 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | +| **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | +| **sovereignty_zone** | Registered DCM artifact declaring geopolitical/regulatory boundary; rules reference zones by ID; inter-zone agreements declared explicitly | +| **STRIP_FIELD** | Governance matrix decision: remove named fields from payload and proceed; if stripped field is required โ†’ DENY_REQUEST | +| **REDACT** | Governance matrix decision: replace field value with ``; field presence preserved; receiver knows field exists but not its value | +| **Provider Type Registry** | Three-tier registry of approved provider types; each entry declares permissions, default_approval_method, enabled_in_profiles, capability_schema_ref | +| **registration_token** | Pre-issued by platform admin; scoped to provider_type/handle_pattern/zone; single_use; grants_auto_approval flag; value presented once only | +| **approval_method** | Registration approval: auto | human_review | dual_approval | committee; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | +| **Drift Reconciliation Component** | Control plane component; compares Discovered vs Realized State; produces drift records and events; never writes to Realized Store; DRC-001โ€“005 | +| **drift_record** | Artifact produced by Drift Reconciliation; field-by-field comparison result with severity classification; unsanctioned flag; status tracking through resolution | | **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | | **reserve_query** | Parallel capacity queries to all eligible provider candidates; PT5M capacity hold; non-responders and insufficient-capacity providers excluded | | **consistent hash** | Final placement tie-breaker: SHA-256(request_uuid+resource_type+sorted_candidates); deterministic; never round-robin | @@ -3410,7 +3430,69 @@ ZT-001 through ZT-005 (zero trust) + ACC-001 through ACC-006 (accreditation). Ke --- -## SECTION 53 โ€” TERMINOLOGY GLOSSARY +## SECTION 53 โ€” GOVERNANCE MATRIX, REGISTRATION, AND DRIFT RECONCILIATION + +### 53.1 Unified Governance Matrix (doc 27) + +The **single enforcement point** for all cross-boundary data and capability decisions in DCM. Supersedes the Data/Capability Authorization Matrix in doc 26 Section 4. Evaluates every interaction using four axes: + +**Axis 1 โ€” Subject (who):** actor | service_provider | dcm_peer | policy_provider | storage_provider | notification_provider | information_provider | system. With identity (specific UUID or trust_posture or accreditation_level) and tenant scope. + +**Axis 2 โ€” Data (what):** classification (exact/in/minimum/maximum), resource_type, field_paths (allowlist/blocklist/any with dot-notation paths including wildcards `fields.phi_*`), capability (read/write/store/replicate/export/notify/execute/discover/query/federate). + +**Axis 3 โ€” Target (where):** type, specific provider/peer UUID, sovereignty_zone (match/not_in), jurisdiction (includes/excludes/intersects country codes), trust_posture (minimum), accreditation_held (includes/not_includes). + +**Axis 4 โ€” Context (under what conditions):** profile (posture/compliance_domains), zero_trust_posture (minimum level), tls_mutual, hardware_attestation, federated, cross_jurisdiction, cross_tenant. + +**Decisions:** ALLOW | DENY | ALLOW_WITH_CONDITIONS | STRIP_FIELD | REDACT | AUDIT_ONLY + +**Hard vs soft enforcement:** Hard rules cannot be relaxed by any downstream rule โ€” ever. GMX-004: sovereign/classified data DENY to all external targets is always hard regardless of profile. Soft rules can be tightened by more-specific domain rules. + +**Field-level granularity:** allowlist mode (only named fields cross boundary), blocklist mode (named fields stripped/redacted), passthrough. STRIP_FIELD removes field; REDACT replaces value with ``; if stripped field is required โ†’ escalates to DENY_REQUEST (GMX-010). + +**Profile-bound defaults:** minimal (pass public/internal; hard DENY sovereign/classified), dev (add confidential with TLS), standard (restricted requires third_party accreditation; PHI denied by default), prod (verified peers only for confidential+; notification fields stripped for restricted), fsi (cross-jurisdiction hard DENY for regulated data; PHI requires BAA+verified+full-ZT), sovereign (no sensitive data in federation; hardware attestation required for all federation). + +**Compliance domain rules:** Automatically added when domain active. HIPAA: minimum_necessary principle; PHI audit all interactions; no export without regulatory cert. GDPR: EU residency hard rule; personal identifier fields stripped outside EU zones. + +**Sovereignty zones:** Registered artifacts declaring jurisdictions, regulatory frameworks, inter-zone agreements, and required provider accreditation. Rules reference zones by ID, not raw country codes. + +**Evaluation algorithm:** Hard DENY first โ†’ any hard DENY = terminal DENY. Soft constraints by domain precedence (entity > resource_type > tenant > platform > system); DENY > STRIP_FIELD > ALLOW. Conditions evaluated for ALLOW_WITH_CONDITIONS. Field permissions applied. Audit record always written. GMX-001 through GMX-010. + +### 53.2 Registration Specification (dcm-registration-spec.md) + +**Provider Type Registry:** Three-tier (Core/Community/Organization). Each entry declares permissions, default_approval_method, default_trust_level, enabled_in_profiles, capability_schema_ref. Nine core types: service_provider (human_review), meta_provider (dual_approval), storage_provider (dual_approval), policy_provider-mode-3-4 (dual_approval), credential_provider (dual_approval), auth_provider (dual_approval), information_provider/message_bus/notification_provider (human_review). + +**Registration token model:** Pre-issued by platform admin (POST /api/v1/admin/registration-tokens). Scoped to provider_type, handle_pattern, sovereignty_zone. single_use. grants_auto_approval flag. Token value presented once โ€” never retrievable. Max trust level bounded by token scope. + +**Approval method resolution:** most_restrictive(provider_type_default, profile_min_method, token_grants_auto). Profile can only tighten. Token can relax to auto ONLY if profile.allow_token_auto_approval=true. Committee approval cannot be relaxed by token. + +**Profile defaults:** minimal/dev โ†’ human_review, token auto-approval enabled. standard โ†’ human_review, token auto-approval enabled (max trust: standard). prod โ†’ human_review; high-trust types require dual_approval; no token auto-approval. fsi โ†’ dual_approval everything; minimum_accreditation: third_party. sovereign โ†’ committee everything; minimum_accreditation: regulatory_certification; hardware_attestation required. + +**Registration pipeline:** SUBMITTED โ†’ VALIDATING (8 automated checks: provider type enabled, governance matrix pre-check, registration token, certificate, sovereignty declaration, capability consistency, health endpoint, accreditation) โ†’ PENDING_APPROVAL โ†’ ACTIVE. Approval methods: auto (immediate), human_review (one admin), dual_approval (two independent admins), committee (DCMGroup quorum). + +**Per-type capability schemas:** service_provider (resource types, capacity model, cancellation support, discovery, naturalization format, cost metadata), information_provider (data domains, authority level, query capacity, confidence model), storage_provider (store types, consistency, replication, encryption), policy_provider (mode 1-4, framework, remote endpoint, shadow mode support), auth_provider (auth modes, MFA methods, RBAC model, token lifetime), notification_provider (delivery channels, guarantees, sovereignty-aware delivery), credential_provider (credential types, secret engines, HSM support), message_bus_provider (protocols, durability, external_endpoints flag), meta_provider (constituent types, composition model, compensation support). + +**Federated trust postures:** verified (manually approved; full scope), vouched (Hub-introduced; bounded scope), provisional (crypto-verified; catalog_query only if profile permits). Approval: dev auto-promotes provisional; standard human_review for verified; prod/fsi dual_approval; sovereign committee+hardware-attestation. Profile federation_policy block declares all parameters. + +**Ongoing lifecycle:** health monitoring (polling; degraded โ†’ reduced routing; failure_threshold โ†’ UNAVAILABLE; 2ร—threshold โ†’ drift triggered), certificate rotation (P90D default; P14D warning; P7D transition window), capability amendments (simplified flow), graceful deregistration (entity migration plan required), forced deregistration (dual_approval/committee; entities โ†’ INDETERMINATE_REALIZATION; Recovery Policy fires). + +### 53.3 Drift Reconciliation Component (doc 25 Section 7) + +Control plane component that compares Discovered State vs Realized State. Read-only โ€” never writes to Realized Store. Produces drift records and events into Request Orchestrator. + +**Algorithm:** discovery.cycle_complete event โ†’ field-by-field comparison per entity โ†’ field criticality (from Resource Type Spec) ร— change magnitude (profile-governed thresholds) โ†’ severity matrix (minor/significant/critical) โ†’ unsanctioned check (no corresponding Requested State? โ†’ elevate one level; fire unsanctioned_change.detected) โ†’ drift_record created โ†’ drift.detected event โ†’ Policy Engine evaluates response. + +**Drift record:** entity_uuid, discovery_snapshot_uuid, realized_state_uuid, overall_severity, unsanctioned flag, drifted_fields (field_path, realized_value, discovered_value, field_criticality, change_magnitude, field_severity, elevated_for_unsanctioned), status (open/acknowledged/resolved/escalated). + +**Resolution tracking:** Drift record status updated when REVERT (next discovery shows clean) or UPDATE_DEFINITION (new Realized State written) or ACCEPT or entity DECOMMISSIONED. drift.resolved event published. + +**Governance matrix integration:** Checks if a governance matrix rule permits the provider to make this type of change. If yes: warning (provider should have submitted update notification). Still treated as drift โ€” provider must use the Provider Update Notification API. + +DRC-001 through DRC-005. Nine control plane components now fully defined in doc 25. + +--- + +## SECTION 54 โ€” TERMINOLOGY GLOSSARY | Term | Definition | |------|-----------| @@ -3480,6 +3562,16 @@ ZT-001 through ZT-005 (zero trust) + ACC-001 through ACC-006 (accreditation). Ke | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | +| **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | +| **sovereignty_zone** | Registered DCM artifact declaring geopolitical/regulatory boundary; rules reference zones by ID; inter-zone agreements declared explicitly | +| **STRIP_FIELD** | Governance matrix decision: remove named fields from payload and proceed; if stripped field is required โ†’ DENY_REQUEST | +| **REDACT** | Governance matrix decision: replace field value with ``; field presence preserved; receiver knows field exists but not its value | +| **Provider Type Registry** | Three-tier registry of approved provider types; each entry declares permissions, default_approval_method, enabled_in_profiles, capability_schema_ref | +| **registration_token** | Pre-issued by platform admin; scoped to provider_type/handle_pattern/zone; single_use; grants_auto_approval flag; value presented once only | +| **approval_method** | Registration approval: auto | human_review | dual_approval | committee; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | +| **Drift Reconciliation Component** | Control plane component; compares Discovered vs Realized State; produces drift records and events; never writes to Realized Store; DRC-001โ€“005 | +| **drift_record** | Artifact produced by Drift Reconciliation; field-by-field comparison result with severity classification; unsanctioned flag; status tracking through resolution | | **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | | **reserve_query** | Parallel capacity queries to all eligible provider candidates; PT5M capacity hold; non-responders and insufficient-capacity providers excluded | | **consistent hash** | Final placement tie-breaker: SHA-256(request_uuid+resource_type+sorted_candidates); deterministic; never round-robin | @@ -3713,7 +3805,7 @@ ZT-001 through ZT-005 (zero trust) + ACC-001 through ACC-006 (accreditation). Ke --- -## SECTION 54 โ€” OPEN QUESTIONS +## SECTION 55 โ€” OPEN QUESTIONS These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -3810,7 +3902,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 55 โ€” DOCUMENTATION STRUCTURE +## SECTION 56 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -3858,7 +3950,7 @@ content/ --- -## SECTION 56 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 57 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: diff --git a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md index 33f109a..ef2a99b 100644 --- a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md +++ b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md @@ -203,6 +203,11 @@ deployment_accreditation: --- + +> **Architecture Update:** Section 4 of this document (Data/Capability Authorization Matrix) has been superseded by the **Unified Governance Matrix** ([doc 27](27-governance-matrix.md)). The governance matrix provides a more powerful, unified model that replaces the standalone matrix described here. The accreditation model (Sections 2-3) and zero trust interaction model (Section 5) remain current and are consumed by the governance matrix as inputs. +> +> New implementations should reference doc 27 for data and capability boundary enforcement. + ## 4. Data/Capability Authorization Matrix ### 4.1 Purpose diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index 3c4bc22..eb851fb 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -620,7 +620,151 @@ If the Search Index is unavailable: --- -## 7. Related Policies โ€” Full Component Set + +--- + +## 7. The Drift Reconciliation Component + +### 7.1 Role + +The Drift Reconciliation Component compares the Discovered State of entities against their Realized State to detect, classify, and respond to drift. It is the consumer of Discovered Store data and the producer of drift records that feed into the Policy Engine for response evaluation. + +Drift Reconciliation is purely a read-and-compare component โ€” it never writes to the Realized Store. It reads Discovered State, reads Realized State, computes differences, classifies severity, and fires events into the Request Orchestrator. The Policy Engine and Recovery Policies determine what happens next. + +### 7.2 Inputs and Outputs + +**Inputs:** +- Discovered State snapshots (from Discovered Store, written by Discovery Scheduler) +- Realized State snapshots (from Realized Store) +- Resource Type Specifications (for field criticality declarations used in severity classification) +- Active governance profile (for magnitude thresholds used in severity classification) + +**Outputs:** +- Drift records (written to Drift Record Store โ€” a lightweight operational store) +- Drift events published to the Request Orchestrator: `drift.detected`, `drift.resolved`, `drift.severity_escalated` +- Unsanctioned change events: `unsanctioned_change.detected` + +### 7.3 Comparison Algorithm + +``` +Discovery cycle completes โ†’ Discovered State snapshot written + โ”‚ + โ–ผ Drift Reconciliation Component receives discovery.cycle_complete event + โ”‚ + โ–ผ For each entity UUID in the discovery snapshot: + โ”‚ + โ”‚ Load: latest Realized State snapshot for entity UUID + โ”‚ Load: Discovered State snapshot (just written) + โ”‚ Load: Resource Type Specification (field criticality per field) + โ”‚ + โ–ผ Field-by-field comparison: + โ”‚ For each field in Realized State: + โ”‚ Does Discovered State contain this field? + โ”‚ If yes: are the values equal? + โ”‚ If no: field is absent โ€” severity based on field criticality + โ”‚ For each field in Discovered State not in Realized State: + โ”‚ New field appeared โ€” severity based on field criticality + โ”‚ + โ–ผ Severity classification (per field): + โ”‚ Field criticality (from Resource Type Spec) ร— Change magnitude (profile-governed) + โ”‚ โ†’ severity matrix โ†’ minor | significant | critical + โ”‚ Unsanctioned? โ†’ elevate one level + โ”‚ Multiple drifted fields? โ†’ overall = highest individual severity + โ”‚ + โ–ผ Unsanctioned check: + โ”‚ Is there a Requested State record that explains this change? + โ”‚ If yes: sanctioned change (may still be drift if realization didn't match) + โ”‚ If no: unsanctioned_change.detected event fired (in addition to drift.detected) + โ”‚ + โ”œโ”€โ”€ No drift detected: + โ”‚ Update entity.last_discovered_at + โ”‚ Update entity.drift_status = clean + โ”‚ No drift record created + โ”‚ + โ””โ”€โ”€ Drift detected: + Create drift record + Publish drift.detected to Request Orchestrator + Policy Engine evaluates โ†’ response action +``` + +### 7.4 Drift Record Structure + +```yaml +drift_record: + uuid: + entity_uuid: + detected_at: + discovery_snapshot_uuid: # the Discovered State snapshot that triggered this + realized_state_uuid: # the Realized State snapshot compared against + + overall_severity: minor | significant | critical + unsanctioned: true | false # true if no corresponding Requested State record + + drifted_fields: + - field_path: "fields.memory_gb" + realized_value: 8 + discovered_value: 16 + field_criticality: medium # from Resource Type Spec + change_magnitude: significant # 100% increase, threshold: standard 10-50% + field_severity: significant + elevated_for_unsanctioned: true # elevated from significant โ†’ critical + + status: open | acknowledged | resolved | escalated + resolution: + resolved_at: + resolution_type: reverted | updated_definition | accepted | escalated | null + resolved_by_requested_state_uuid: +``` + +### 7.5 Drift Resolution Tracking + +Drift records are not resolved by the Drift Reconciliation Component โ€” they are resolved by the Policy Engine's response actions. The Drift Reconciliation Component monitors for resolution: + +``` +REVERT action taken: + New Requested State submitted โ†’ provider reverts โ†’ new Realized State written + Next discovery cycle: Discovered State matches new Realized State + Drift Reconciliation: no drift detected โ†’ drift_record.status = resolved + drift.resolved event published + +UPDATE_DEFINITION action taken: + Consumer submits UPDATE_DEFINITION โ†’ new Realized State written with discovered values + Next discovery cycle: Discovered State matches new Realized State + Drift record.status = resolved with resolution_type: updated_definition + +Entity decommissioned: + Drift record.status = resolved with resolution_type: decommissioned +``` + +### 7.6 Governance Matrix Integration + +Before classifying a discovered change as drift, the Drift Reconciliation Component evaluates the governance matrix to determine if the change is expected: + +``` +Field value in Discovered State differs from Realized State + โ”‚ + โ–ผ Check: Is there a governance matrix rule that permits this provider + โ”‚ to make this type of change to this field? + โ”‚ + โ”œโ”€โ”€ Yes โ†’ This may be a Provider Update Notification that wasn't submitted + โ”‚ DCM logs a warning: "Provider changed field without submitting update notification" + โ”‚ Still treated as drift โ€” provider should have submitted update notification + โ”‚ + โ””โ”€โ”€ No โ†’ Standard drift detection; severity classification runs +``` + +### 7.7 Drift Reconciliation Policies + +| Policy | Rule | +|--------|------| +| `DRC-001` | The Drift Reconciliation Component never writes to the Realized Store. It produces drift records and events only. | +| `DRC-002` | Drift detection runs after every discovery cycle. An entity with no corresponding Realized State record is an orphan candidate โ€” not a drift event. | +| `DRC-003` | Unsanctioned changes are always elevated one severity level above the matrix classification. An unsanctioned significant drift is reported as critical. | +| `DRC-004` | Drift records are retained until the entity is decommissioned plus the configured audit retention period. They are not deleted on resolution โ€” resolution is recorded within the record. | +| `DRC-005` | Drift detection produces events into the Request Orchestrator. The Policy Engine determines the response action. The Drift Reconciliation Component does not initiate remediation directly. | + + +## 8. Related Policies โ€” Full Component Set | Policy | Rule | |--------|------| @@ -632,6 +776,7 @@ If the Search Index is unavailable: | `PLC-001` through `PLC-006` | Placement Engine policies (see Section 4.7) | | `LCE-001` through `LCE-005` | Lifecycle Constraint Enforcer policies (see Section 5.7) | | `SIX-001` through `SIX-004` | Search Index policies (see Section 6.6) | +| `DRC-001` through `DRC-005` | Drift Reconciliation policies (see Section 7.7) | --- diff --git a/content/docs/architecture/data-model/governance-matrix.md b/content/docs/architecture/data-model/governance-matrix.md new file mode 100644 index 0000000..0318916 --- /dev/null +++ b/content/docs/architecture/data-model/governance-matrix.md @@ -0,0 +1,816 @@ +--- +title: "Unified Governance Matrix" +type: docs +weight: 27 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference +**Related Documents:** [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) | [DCM Federation](22-dcm-federation.md) | [Policy Profiles](14-policy-profiles.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Control Plane Components](25-control-plane-components.md) + +--- + +## 1. Purpose + +The Unified Governance Matrix is the **single, declarative, multi-dimensional control surface** that governs every cross-boundary interaction in DCM. It answers one question at every interaction point: + +> **Given this subject, this data, this target, and this context โ€” is this interaction permitted, and under what conditions?** + +Previous DCM documents established several overlapping control mechanisms: the Data/Capability Authorization Matrix (doc 26), sovereignty constraints in federation tunnels (doc 22), BBQ-001 sovereignty checks (doc 14), and profile-governed data boundaries. The Governance Matrix unifies all of these into a single model with a single evaluation algorithm and a single enforcement point. + +**This document supersedes** Section 4 of doc 26 (Data/Capability Authorization Matrix) for structural purposes. The accreditation model (Sections 2-3 of doc 26) and the zero trust interaction model (Section 5 of doc 26) remain current โ€” the Governance Matrix consumes them as inputs. + +**Key properties of the Governance Matrix:** + +- **Fine-grained to broad** โ€” rules can target a single field path on a specific entity, or broadly govern all data of a given classification. Both are first-class citizens of the same model. +- **Profile-bound defaults** โ€” every deployment profile ships with sensible default rules that are immediately operative. Operators configure overrides rather than building from scratch. +- **Hard and soft enforcement** โ€” hard rules cannot be relaxed by any downstream rule. Soft rules establish defaults that can be tightened but never relaxed. +- **Single evaluation algorithm** โ€” every interaction boundary runs the same algorithm against the same rule set. No parallel enforcement paths. +- **Audited always** โ€” every evaluation produces an audit record regardless of outcome. + +--- + +## 2. The Four Matrix Axes + +Every governance matrix rule is expressed as a match across four axes. A rule fires when all declared axis conditions are satisfied. + +### 2.1 Axis 1 โ€” Subject (Who) + +The subject is the entity initiating or involved in the interaction. + +```yaml +subject: + type: + # Subject types: + # actor โ€” human or service account making a request + # service_provider โ€” Service Provider sending/receiving data + # dcm_peer โ€” federated DCM instance + # policy_provider โ€” Policy Provider receiving payload data for evaluation + # storage_provider โ€” Storage Provider receiving/returning state data + # notification_provider โ€” Notification Provider receiving notification envelopes + # information_provider โ€” Information Provider returning external data + # system โ€” DCM internal component (Request Orchestrator, etc.) + + identity: + provider_uuid: # specific provider instance + dcm_peer_uuid: # specific federated DCM instance + trust_posture: # for dcm_peer subjects + accreditation_level: # accreditation type the subject holds + actor_role: # for actor subjects + + tenant: + uuid: # specific Tenant + match: any_tenant | cross_tenant | system_tenant +``` + +### 2.2 Axis 2 โ€” Data (What) + +The data axis declares what is being accessed, sent, or operated on. This is where field-level granularity lives. + +```yaml +data: + # Broad controls โ€” classification level + classification: + match: | in: [] | minimum: | maximum: + # minimum: restricted means restricted and above (phi, sovereign, classified) + # maximum: internal means internal and below (public, internal) + + # Resource-type scoping + resource_type: + match: | category: | any + + # Fine-grained controls โ€” specific field paths + field_paths: + mode: allowlist | blocklist | any + # allowlist: only these fields are permitted to cross the boundary + # blocklist: these fields are explicitly prohibited + # any: no field-level restriction (default) + paths: + - "fields.patient_id" + - "fields.diagnosis_code" + - "fields.treatment_plan" + # Supports wildcards: "fields.phi_*" matches all fields prefixed phi_ + + # Capability being exercised + capability: + match: | in: [] | any + # Capabilities: read | write | store | replicate | export | notify | + # execute | discover | query | federate +``` + +### 2.3 Axis 3 โ€” Target (Where) + +The target is where the data is going โ€” provider, peer DCM, storage, notification endpoint. + +```yaml +target: + type: + # service_provider | dcm_peer | storage_provider | notification_provider | + # information_provider | policy_provider | external_endpoint + + # Identity + provider_uuid: # specific provider + dcm_peer_uuid: # specific peer + + # Sovereignty + sovereignty_zone: + match: | in: [] | same_as_source | any + not_in: [] # exclusion list + + jurisdiction: + includes: [] + excludes: [] + intersects: [] # target jurisdiction overlaps with list + + # Trust and accreditation + trust_posture: + match: | minimum: + # minimum: vouched means vouched or verified (not provisional) + + accreditation_held: + includes: [] # target MUST hold these accreditations + not_includes: [] # target must NOT hold (exclusion pattern) + minimum_type: # minimum trust level of accreditation +``` + +### 2.4 Axis 4 โ€” Context (Under What Conditions) + +Context captures the operational conditions at the time of the interaction. + +```yaml +context: + # Active deployment governance + profile: + deployment_posture: | in: [] + compliance_domains: + includes: [] + not_includes: [] + + # Security posture + zero_trust_posture: + minimum: # none | boundary | full | hardware_attested + tls_mutual: + hardware_attestation: + + # Interaction characteristics + federated: + cross_jurisdiction: + cross_tenant: + + # Time-based conditions + time_of_day: # for regulated maintenance windows + request_age_max: # reject stale requests +``` + +--- + +## 3. Rule Structure + +### 3.1 The Governance Matrix Rule + +```yaml +governance_matrix_rule: + # Artifact metadata (standard DCM artifact) + artifact_metadata: + uuid: + handle: "system/matrix/phi-federation-boundary" + version: "1.0.0" + status: active # developing | proposed | active | deprecated | retired + owned_by: { display_name: "Platform Security" } + tier: system | platform | tenant | resource_type | entity + + description: "PHI must not cross to federated peers without HIPAA accreditation" + rationale: "HIPAA 45 CFR 164.502 โ€” minimum necessary standard for PHI disclosure" + + # Match conditions (all declared axes must match for rule to fire) + match: + subject: { ... } + data: { ... } + target: { ... } + context: { ... } + + # Decision + decision: ALLOW | DENY | ALLOW_WITH_CONDITIONS | STRIP_FIELD | REDACT | AUDIT_ONLY + + # Enforcement level + enforcement: hard | soft + # hard: cannot be relaxed by any downstream rule; ever + # soft: downstream rules at same or higher domain can tighten further + + # Conditions that must be met for ALLOW_WITH_CONDITIONS + conditions: + - field: target.trust_posture + operator: minimum + value: verified + - field: context.tls_mutual + operator: equals + value: required + - field: target.accreditation_held + operator: includes + value: hipaa + + # Field permission model (for ALLOW and ALLOW_WITH_CONDITIONS) + field_permissions: + mode: allowlist | blocklist | passthrough + paths: [] + on_blocked_field: STRIP_FIELD | DENY_REQUEST | REDACT + # STRIP_FIELD: remove field and proceed (if field is optional) + # DENY_REQUEST: block entire interaction (if field is required) + # REDACT: replace field value with in payload + + # Audit and notification + audit_on: [ALLOW, DENY, STRIP_FIELD, REDACT] + notification_on: [DENY] + notification_urgency: low | medium | high | critical + + # Metadata + applicable_profiles: [standard, prod, fsi, sovereign] # which profiles activate this rule + compliance_basis: "HIPAA 45 CFR 164.502" # regulatory basis + review_required_before: "2027-01-01" # when rule should be reviewed +``` + +### 3.2 Decision Vocabulary + +| Decision | Meaning | Field behavior | +|----------|---------|----------------| +| `ALLOW` | Interaction permitted unconditionally (within field_permissions) | Fields per field_permissions | +| `ALLOW_WITH_CONDITIONS` | Permitted only if all conditions are satisfied; DENY if conditions fail | Fields per field_permissions | +| `DENY` | Interaction blocked; interaction does not proceed | N/A โ€” entire interaction stopped | +| `STRIP_FIELD` | Specific fields are removed from the payload; interaction proceeds with remaining fields | Named fields stripped | +| `REDACT` | Specific field values replaced with ``; field presence is preserved | Named fields redacted | +| `AUDIT_ONLY` | Interaction proceeds but is flagged in the audit trail; no blocking | All fields pass | + +### 3.3 Hard vs Soft Enforcement + +**Hard enforcement (`enforcement: hard`):** +- The rule decision cannot be relaxed by any more-specific or higher-domain rule +- A hard DENY is absolute โ€” no Tenant-level, entity-level, or operator override can permit the interaction +- Hard rules are reserved for: sovereign/classified data classification boundaries, regulatory hard requirements (HIPAA BAA requirement for PHI), and explicit security policies +- Hard ALLOW is rare โ€” it means this interaction is always permitted regardless of other rules (use with extreme caution) + +**Soft enforcement (`enforcement: soft`):** +- The rule establishes a default that can be tightened by more-specific downstream rules +- A soft ALLOW can be restricted to DENY or STRIP_FIELD by a more-specific rule +- A soft DENY cannot be relaxed to ALLOW by a downstream rule (DENY always wins at the same level) +- Most profile-level defaults are soft โ€” they set sensible baselines that Tenants can restrict further + +--- + +## 4. Evaluation Algorithm + +The governance matrix evaluates all matching rules and produces a single terminal decision. + +``` +Interaction attempt: + subject: { type: dcm_peer, trust_posture: verified, jurisdiction: [DE] } + data: { classification: phi, field_paths: [patient_id, diagnosis_code] } + target: { type: dcm_peer, accreditation_held: [], jurisdiction: [US] } + context: { federated: true, zero_trust_posture: full, tls_mutual: required } + +Step 1: Collect matching rules + Load all active governance matrix rules across all tiers + Evaluate match conditions for each rule against the four axes + Result: set of matching rules with their decisions and enforcement levels + +Step 2: Evaluate hard constraints first + For each hard DENY rule that matches: + โ†’ DENY immediately; record rule_uuid; no further evaluation + For each hard ALLOW rule that matches: + โ†’ Record as a hard allow candidate; still evaluate conditions + If hard DENY exists: terminal decision = DENY + +Step 3: Evaluate soft constraints by domain precedence + Sort matching soft rules: entity > resource_type > tenant > platform > system + For each precedence level, most restrictive decision wins: + DENY > STRIP_FIELD > REDACT > ALLOW_WITH_CONDITIONS > AUDIT_ONLY > ALLOW + If DENY at any level: terminal decision = DENY + +Step 4: Evaluate conditions for ALLOW_WITH_CONDITIONS + For each ALLOW_WITH_CONDITIONS rule that survived Steps 2-3: + Evaluate all declared conditions + If any condition fails: downgrade decision to DENY + If all conditions pass: decision remains ALLOW_WITH_CONDITIONS + +Step 5: Apply field permissions + If terminal decision is ALLOW or ALLOW_WITH_CONDITIONS: + Apply field_permissions from the governing rule: + allowlist mode: strip all fields not in the allowed list + blocklist mode: strip all fields in the blocked list + passthrough mode: all fields pass + For each stripped field: + If field is required: escalate to DENY_REQUEST + If field is optional: STRIP_FIELD (proceed without it) + +Step 6: Produce audit record + Record: interaction_uuid, all matching rules, terminal decision, + fields stripped or redacted, rule that governed the decision + Notification: if terminal decision is in rule's notification_on list + +Step 7: Enforce decision + ALLOW / ALLOW_WITH_CONDITIONS: interaction proceeds with permitted fields + DENY: interaction blocked; 403 response with governance_matrix_rule_uuid + STRIP_FIELD: interaction proceeds with stripped payload + REDACT: interaction proceeds with redacted field values + AUDIT_ONLY: interaction proceeds; flagged audit record written +``` + +--- + +## 5. Sovereignty Zones + +Sovereignty zones are registered DCM artifacts that define geopolitical and regulatory boundaries. They are a first-class input to the governance matrix โ€” rules reference zones, not raw country codes. + +```yaml +sovereignty_zone: + artifact_metadata: + uuid: + handle: "zones/eu-west-sovereign" + version: "1.0.0" + status: active + tier: system | platform + + display_name: "EU Western Europe Sovereign Zone" + description: "GDPR-covered EU member states with NIS2 alignment" + + jurisdictions: [DE, FR, NL, BE, AT, CH, LU] + excluded_jurisdictions: [] # explicit exclusions within the zone + + data_residency_guarantee: EU # GDPR Article 44 transfer basis + regulatory_frameworks: [GDPR, NIS2, eIDAS] + + cross_zone_permitted: false # data does not leave this zone by default + inter_zone_agreements: # zones this zone has data transfer agreements with + - zone_id: eu-north-sovereign + agreement_basis: "EU adequacy decision" + permitted_classifications: [public, internal, confidential] + # restricted, phi, sovereign: NOT included + + # What accreditation providers must hold to operate in this zone + required_provider_accreditation: gdpr_adequacy | third_party + required_provider_accreditation_minimum_type: third_party +``` + +--- + +## 6. Field-Level Controls โ€” Complete Model + +### 6.1 Field Path Syntax + +Field paths use dot-notation to address specific fields within a DCM payload: + +``` +fields. # top-level field +fields.. # nested field +fields.phi_* # wildcard: all fields matching prefix +fields.* # all fields +metadata. # metadata fields +provenance. # provenance fields (rarely restricted) +``` + +### 6.2 Broad to Fine-Grained Rule Examples + +**Broadest โ€” classification-level block:** +```yaml +# Block ALL phi fields from crossing to non-HIPAA peers +match: + data.classification: phi + target.type: dcm_peer + target.accreditation_held.not_includes: hipaa +decision: DENY +enforcement: hard +``` + +**Mid-level โ€” resource type + classification:** +```yaml +# For VM resources: restricted fields to EU zones only +match: + data.classification: restricted + data.resource_type: Compute.VirtualMachine + target.sovereignty_zone.not_in: [eu-west-sovereign, eu-north-sovereign] +decision: STRIP_FIELD +field_permissions: + mode: blocklist + paths: ["fields.security_group_ids", "fields.network_interface_ids"] + on_blocked_field: STRIP_FIELD +``` + +**Fine-grained โ€” specific fields:** +```yaml +# Allow federated phi-accredited peers to receive limited PHI fields only +match: + data.classification: phi + target.type: dcm_peer + target.accreditation_held.includes: hipaa + target.trust_posture: verified +decision: ALLOW_WITH_CONDITIONS +conditions: + - field: context.tls_mutual + operator: equals + value: required + - field: context.zero_trust_posture + operator: minimum + value: full +field_permissions: + mode: allowlist + paths: + - "fields.resource_type" + - "fields.lifecycle_state" + - "fields.provider_entity_id" + # PHI-containing fields explicitly NOT in allowlist: + # fields.patient_id, fields.diagnosis_code, fields.treatment_plan + # are stripped automatically + on_blocked_field: STRIP_FIELD +``` + +**Most specific โ€” entity-level rule:** +```yaml +# Provider A: explicit PHI block regardless of accreditation +match: + target.provider_uuid: provider-a-uuid + data.classification: phi +decision: DENY +enforcement: hard +reason: "Provider A has unresolved data handling concerns โ€” PHI explicitly prohibited" +``` + +### 6.3 Field-Level Redaction vs Stripping + +| Operation | Effect on payload | Use case | +|-----------|------------------|----------| +| `STRIP_FIELD` | Field entirely removed from payload | Field is optional; receiver has no need to know it exists | +| `REDACT` | Field present with value `` | Receiver needs to know field exists but not its value (e.g., audit evidence that a field was present) | +| `DENY_REQUEST` | Entire interaction blocked | Field is required for the operation to make sense; stripping would produce invalid state | + +--- + +## 7. Profile-Bound Default Matrix Rules + +Every deployment profile activates a set of default governance matrix rules. These are soft rules (tightenable by Tenant/resource-type overrides) unless marked hard. + +### 7.1 minimal Profile Defaults + +```yaml +profile_matrix_defaults: + profile: minimal + rules: + - handle: "system/matrix/minimal-sovereign-hard" + enforcement: hard + match: + data.classification: [sovereign, classified] + target.type: [dcm_peer, service_provider, notification_provider] + decision: DENY + reason: "Sovereign and classified data never crosses any boundary โ€” any profile" + + - handle: "system/matrix/minimal-passthrough" + enforcement: soft + match: + data.classification: [public, internal] + target.type: any + decision: ALLOW + field_permissions: + mode: passthrough +``` + +### 7.2 dev Profile Defaults + +```yaml +profile_matrix_defaults: + profile: dev + inherits: minimal + additional_rules: + - handle: "system/matrix/dev-confidential-allow" + enforcement: soft + match: + data.classification: confidential + target.type: [service_provider, dcm_peer] + target.trust_posture: [verified, vouched, provisional] + decision: ALLOW_WITH_CONDITIONS + conditions: + - field: context.tls_mutual + operator: equals + value: required + field_permissions: + mode: passthrough + # Dev allows confidential to flow broadly; standard+ tightens this +``` + +### 7.3 standard Profile Defaults + +```yaml +profile_matrix_defaults: + profile: standard + inherits: minimal + additional_rules: + - handle: "system/matrix/standard-restricted-accreditation" + enforcement: soft + match: + data.classification: restricted + target.type: [service_provider, dcm_peer] + decision: ALLOW_WITH_CONDITIONS + conditions: + - field: target.accreditation_held + operator: minimum_type + value: third_party + - field: context.tls_mutual + operator: equals + value: required + field_permissions: + mode: passthrough + + - handle: "system/matrix/standard-phi-deny-default" + enforcement: soft + match: + data.classification: phi + target.type: [service_provider, dcm_peer] + decision: DENY + # Tenants with HIPAA compliance domain active override this with their own rules +``` + +### 7.4 prod Profile Defaults + +```yaml +profile_matrix_defaults: + profile: prod + inherits: standard + additional_rules: + - handle: "system/matrix/prod-federation-verified-only" + enforcement: soft + match: + data.classification: [confidential, restricted] + target.type: dcm_peer + target.trust_posture: [vouched, provisional] + decision: DENY + # prod: only verified peers receive confidential+ data + + - handle: "system/matrix/prod-notification-restricted" + enforcement: soft + match: + data.classification: restricted + target.type: notification_provider + decision: STRIP_FIELD + field_permissions: + mode: blocklist + paths: ["fields.*"] # strip all payload fields from notifications + # Notification envelope metadata (entity_uuid, event_type) passes through + # Actual field values do not appear in notification payloads for restricted data +``` + +### 7.5 fsi Profile Defaults + +```yaml +profile_matrix_defaults: + profile: fsi + inherits: prod + additional_rules: + - handle: "system/matrix/fsi-cross-jurisdiction-deny" + enforcement: hard + match: + data.classification: [restricted, phi, pci, sovereign] + target.type: [service_provider, dcm_peer] + context.cross_jurisdiction: true + decision: DENY + reason: "FSI profile: regulated data does not cross jurisdictional boundaries" + + - handle: "system/matrix/fsi-phi-baa-required" + enforcement: hard + match: + data.classification: phi + target.type: [service_provider, dcm_peer] + decision: ALLOW_WITH_CONDITIONS + conditions: + - field: target.accreditation_held + operator: includes + value: hipaa_baa + - field: target.trust_posture + operator: minimum + value: verified + - field: context.zero_trust_posture + operator: minimum + value: full + field_permissions: + mode: passthrough # HIPAA-accredited verified peers get full PHI scope + # Tenant-level rules can further restrict to specific field paths + + - handle: "system/matrix/fsi-pci-qsa-required" + enforcement: hard + match: + data.classification: pci + target.type: [service_provider, dcm_peer] + decision: ALLOW_WITH_CONDITIONS + conditions: + - field: target.accreditation_held + operator: includes + value: pci_dss_qsa + - field: context.zero_trust_posture + operator: minimum + value: full +``` + +### 7.6 sovereign Profile Defaults + +```yaml +profile_matrix_defaults: + profile: sovereign + inherits: fsi + additional_rules: + - handle: "system/matrix/sovereign-no-federation-sensitive" + enforcement: hard + match: + data.classification: [restricted, phi, pci, sovereign, classified] + target.type: dcm_peer + decision: DENY + reason: "Sovereign profile: sensitive data never crosses DCM federation boundaries" + + - handle: "system/matrix/sovereign-internal-only-federation" + enforcement: hard + match: + data.classification: [public, internal] + target.type: dcm_peer + context.zero_trust_posture: + not_minimum: hardware_attested + decision: DENY + reason: "Sovereign profile: federation requires hardware attestation" + + - handle: "system/matrix/sovereign-provider-sovereign-zone-only" + enforcement: hard + match: + data.classification: [restricted, phi, pci, sovereign, classified] + target.type: service_provider + target.sovereignty_zone.not_in: [] + decision: DENY + reason: "Sovereign profile: sensitive data only to providers in declared sovereignty zone" +``` + +--- + +## 8. Compliance Domain Matrix Rules + +When a compliance domain is active, its matrix rules are automatically added to the active rule set. + +### 8.1 HIPAA Compliance Domain Matrix + +```yaml +compliance_domain_matrix: + domain: hipaa + rules: + - handle: "system/matrix/hipaa-phi-minimum-necessary" + enforcement: hard + match: + data.classification: phi + target.type: any + decision: ALLOW_WITH_CONDITIONS + conditions: + - principle: minimum_necessary # only fields required for the specific purpose + field_permissions: + mode: blocklist # default: all fields except explicitly blocked + paths: [] # Tenant adds specific field blocks + on_blocked_field: STRIP_FIELD + + - handle: "system/matrix/hipaa-phi-no-export" + enforcement: hard + match: + data.classification: phi + data.capability: export + decision: DENY + reason: "HIPAA: PHI export to external systems requires explicit BAA and regulatory review" + + - handle: "system/matrix/hipaa-audit-all-phi" + enforcement: hard + match: + data.classification: phi + target.type: any + decision: AUDIT_ONLY # added to all PHI interactions โ€” does not block + audit_on: [ALLOW, DENY, STRIP_FIELD] + # Every PHI interaction produces an audit record โ€” HIPAA requirement +``` + +### 8.2 GDPR Compliance Domain Matrix + +```yaml +compliance_domain_matrix: + domain: gdpr + rules: + - handle: "system/matrix/gdpr-eu-residency" + enforcement: hard + match: + data.classification: [restricted, phi] + target.sovereignty_zone.not_in: + context.compliance_domains.includes: gdpr + decision: DENY + reason: "GDPR Article 44: personal data transfer outside EU requires adequacy decision" + + - handle: "system/matrix/gdpr-right-to-erasure-fields" + enforcement: hard + match: + data.field_paths.includes: ["fields.personal_identifier_*", "fields.contact_*"] + data.capability: [store, replicate] + target.accreditation_held.not_includes: gdpr_adequacy + decision: STRIP_FIELD + field_permissions: + mode: blocklist + paths: ["fields.personal_identifier_*", "fields.contact_*"] +``` + +--- + +## 9. Tenant and Resource-Type Override Rules + +Tenants and resource-type specifications declare additional rules that compose with system and profile rules per the standard precedence model. + +### 9.1 Tenant Override Rule + +```yaml +# Tenant payments-team: additional restriction on PHI fields +governance_matrix_rule: + artifact_metadata: + tier: tenant + handle: "tenant/payments/phi-field-restriction" + + match: + subject.tenant.uuid: payments-tenant-uuid + data.classification: phi + target.type: dcm_peer + target.accreditation_held.includes: hipaa + + decision: ALLOW_WITH_CONDITIONS + conditions: + - field: target.trust_posture + operator: equals + value: verified # only verified (not vouched) + + field_permissions: + mode: allowlist # tighter than the fsi default (passthrough) + paths: + - "fields.resource_type" + - "fields.lifecycle_state" + # PHI-containing fields not listed โ†’ automatically stripped + on_blocked_field: STRIP_FIELD +``` + +### 9.2 Resource-Type Override Rule + +```yaml +# For Patient Record resources: maximum restriction regardless of Tenant settings +governance_matrix_rule: + artifact_metadata: + tier: resource_type + handle: "resource-type/patient-record/no-federation" + + match: + data.resource_type: Healthcare.PatientRecord + target.type: dcm_peer + + decision: DENY + enforcement: hard + reason: "Patient Record entities are never federated โ€” local only" +``` + +--- + +## 10. Governance Matrix in the Registration Flow + +When a provider attempts to register with DCM, the governance matrix is evaluated before the registration is accepted. This answers: "Is a provider of this type, with these accreditations, in this sovereignty zone, permitted to register in this DCM deployment?" + +``` +Provider submits registration + โ”‚ + โ–ผ Governance matrix evaluation: + โ”‚ subject: { type: , accreditation_held: [...], sovereignty_zone: } + โ”‚ data: { capability: register } + โ”‚ target: { type: dcm_instance, sovereignty_zone: } + โ”‚ context: { profile: , compliance_domains: [...] } + โ”‚ + โ”œโ”€โ”€ DENY: registration rejected immediately + โ”‚ Provider type not permitted in this profile + โ”‚ Provider in excluded jurisdiction + โ”‚ Required accreditation not held + โ”‚ + โ””โ”€โ”€ ALLOW / ALLOW_WITH_CONDITIONS: registration proceeds to validation pipeline +``` + +--- + +## 11. System Policies + +| Policy | Rule | +|--------|------| +| `GMX-001` | The Governance Matrix is the single enforcement point for all cross-boundary data and capability decisions. Parallel enforcement mechanisms (standalone sovereignty checks, standalone accreditation checks) are inputs to the matrix โ€” not independent enforcement paths. | +| `GMX-002` | Hard rules cannot be relaxed by any downstream rule at any domain level. Hard DENY is absolute. | +| `GMX-003` | Soft rules establish defaults that can only be tightened by downstream rules. Soft DENY cannot be relaxed to ALLOW by a more-specific rule. | +| `GMX-004` | Sovereign and classified data classifications carry hard DENY rules for all federation and external provider interactions in all profiles including minimal. This is the one rule that cannot be changed by any configuration. | +| `GMX-005` | Every governance matrix evaluation produces an audit record regardless of outcome. | +| `GMX-006` | Field-level stripping (STRIP_FIELD) is always audited with the field path and the rule_uuid that governed the stripping. | +| `GMX-007` | Profile default matrix rules are soft unless explicitly marked hard. Tenant and resource-type rules can tighten profile defaults but cannot relax hard rules. | +| `GMX-008` | Compliance domain matrix rules are automatically added to the active rule set when the compliance domain is active. They compose with profile rules โ€” they do not replace them. | +| `GMX-009` | The Governance Matrix is evaluated before provider dispatch, before federation tunnel data transmission, before notification delivery, and before any cross-boundary capability invocation. | +| `GMX-010` | A STRIP_FIELD decision that removes a required field escalates to DENY_REQUEST automatically. Optional fields may be stripped without blocking the interaction. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/registration-spec.md b/content/docs/architecture/specifications/registration-spec.md new file mode 100644 index 0000000..ae1c596 --- /dev/null +++ b/content/docs/architecture/specifications/registration-spec.md @@ -0,0 +1,835 @@ +--- +title: "DCM Registration Specification" +type: docs +weight: 3 +--- + +> **โš ๏ธ Work in Progress** +> +> This specification defines the unified registration flow for all DCM provider types. Published to share design direction and invite feedback. + +**Version:** 0.1.0-draft +**Status:** Design โ€” Not yet implemented +**Document Type:** Technical Specification +**Related Documents:** [Control Plane Components](../data-model/25-control-plane-components.md) | [Governance Matrix](../data-model/27-governance-matrix.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) + +--- + +## Abstract + +This specification defines the unified registration flow by which all DCM provider types establish a trusted, governed relationship with a DCM deployment. It covers: the Provider Type Registry, the registration token model, the approval method configuration, the step-by-step registration pipeline, trust establishment, the per-type capability declaration schemas, the ongoing lifecycle after activation, federated trust configuration, and profile-bound registration policy defaults. + +--- + +## 1. Provider Type Registry + +The Provider Type Registry is the authoritative list of provider types that a DCM deployment will accept registrations for. It follows the same three-tier registry model as the Resource Type Registry. + +### 1.1 Registry Tiers + +| Tier | Maintained By | Examples | +|------|--------------|---------| +| **Core** | DCM Project | The nine built-in provider types | +| **Verified Community** | Named community maintainers | Domain-specific provider types | +| **Organization** | Deploying organization | Custom/proprietary integrations | + +### 1.2 Provider Type Registry Entry + +```yaml +provider_type_registry_entry: + artifact_metadata: + uuid: + handle: "provider-types/service-provider" + version: "1.0.0" + status: active + tier: core + + provider_type_id: service_provider + display_name: "Service Provider" + description: "Realizes infrastructure resources for DCM" + + # What this provider type is permitted to do + permissions: + may_receive_assembled_payload: true + may_write_realized_state: true + may_write_discovered_state: true + may_receive_scoped_credentials: true + may_receive_phi_by_default: false # requires HIPAA accreditation + may_receive_sovereign_data: false # hard limit; never overridden + + # Approval method defaults (profile may override โ€” see Section 4) + default_approval_method: human_review # auto | human_review | dual_approval | committee + + # Minimum trust level granted after approval + default_trust_level: standard # minimal | standard | elevated | high + + # Which deployment profiles permit this provider type + enabled_in_profiles: [minimal, dev, standard, prod, fsi, sovereign] + + # Capability declaration schema reference + capability_schema_ref: "schemas/service-provider-capabilities-v1.0.0" + + # Health check requirements + health_check: + endpoint_required: true + minimum_check_interval: PT1M + failure_threshold: 3 # failures before degraded status +``` + +### 1.3 The Nine Core Provider Types + +| # | provider_type_id | Default Approval | Enabled In | +|---|-----------------|-----------------|------------| +| 1 | `service_provider` | human_review | all profiles | +| 2 | `information_provider` | human_review | all profiles | +| 3 | `meta_provider` | dual_approval | standard+ | +| 4 | `storage_provider` | dual_approval | all profiles | +| 5 | `message_bus_provider` | human_review | dev+ (external endpoints: standard+) | +| 6 | `policy_provider` (Mode 1-2) | human_review | all profiles | +| 7 | `policy_provider` (Mode 3-4) | dual_approval | standard+ | +| 8 | `credential_provider` | dual_approval | standard+ | +| 9 | `auth_provider` | dual_approval | all profiles | +| 10 | `notification_provider` | human_review | all profiles | + +Note: Mode 3-4 Policy Providers are treated as a separate registry entry from Mode 1-2 due to the elevated trust requirements. + +--- + +## 2. Registration Token Model + +Registration tokens are pre-issued by platform admins to authorize specific registrations without requiring full manual review at submission time. + +### 2.1 Token Structure + +```yaml +registration_token: + token_uuid: + token_value: + issued_by: + issued_at: + expires_at: # short-lived; default PT72H + single_use: true # token invalidated after first use + + scope: + provider_type_id: service_provider # which provider type this authorizes + provider_handle_pattern: "eu-west-*" # optional: restrict to matching handles + sovereignty_zone: eu-west-sovereign # optional: restrict to this zone + grants_auto_approval: true # whether token enables auto-approval + # grants_auto_approval: false = token still required but human review still needed + # (useful for tracking/auditing expected registrations without bypassing review) + + max_trust_level_granted: standard # token cannot grant higher than this +``` + +### 2.2 Token Issuance + +``` +POST /api/v1/admin/registration-tokens +Role: platform_admin + +{ + "provider_type_id": "service_provider", + "expires_in": "PT72H", + "scope": { + "provider_handle_pattern": "eu-west-*", + "sovereignty_zone": "eu-west-sovereign", + "grants_auto_approval": true + }, + "purpose": "EU-WEST production compute provider onboarding" +} + +Response 201 Created: +{ + "token_uuid": "", + "token_value": "", + "expires_at": "", + "scope": { ... } +} +``` + +Token values are presented exactly once โ€” at creation. They are never retrievable again (stored as a hash). Platform admins must transmit the token securely to the provider operator. + +--- + +## 3. Approval Method Configuration + +### 3.1 The Four Approval Methods + +| Method | Description | Approval path | +|--------|-------------|--------------| +| `auto` | DCM validates automatically; activates without human review | All validation checks pass โ†’ active | +| `human_review` | One platform admin must explicitly approve | Submitted โ†’ validated โ†’ pending_approval โ†’ one admin approves โ†’ active | +| `dual_approval` | Two platform admins must independently approve | Submitted โ†’ validated โ†’ pending_approval โ†’ two admins approve โ†’ active | +| `committee` | A declared DCMGroup must reach quorum | Submitted โ†’ validated โ†’ pending_approval โ†’ committee votes โ†’ quorum โ†’ active | + +### 3.2 Effective Approval Method Resolution + +The effective approval method for a specific registration is the most restrictive result of: + +``` +effective_method = most_restrictive( + provider_type_registry.default_approval_method, + active_profile.registration_policy.min_approval_method, + registration_token.grants_auto_approval ? relax_to_auto : no_change +) +``` + +Resolution rules: +- Profile minimum overrides provider type default (always upward; profiles can only tighten) +- A valid registration token can relax the effective method to `auto` ONLY if the profile's `allow_token_auto_approval` is true +- `committee` cannot be relaxed by any token + +### 3.3 Profile Registration Policy Defaults + +```yaml +profile_registration_policy: + minimal: + min_approval_method: human_review + allow_token_auto_approval: true # token can enable auto for any type + require_sovereignty_declaration: false + require_health_check_before_approval: false + + dev: + min_approval_method: human_review + allow_token_auto_approval: true + require_sovereignty_declaration: false + require_health_check_before_approval: true + + standard: + min_approval_method: human_review + allow_token_auto_approval: true # tokens can auto-approve non-elevated types + token_auto_approval_max_trust: standard # tokens cannot auto-approve elevated types + require_sovereignty_declaration: true + require_health_check_before_approval: true + + prod: + min_approval_method: human_review + high_trust_types_require: dual_approval # storage, auth, policy-mode3-4, credential + allow_token_auto_approval: false # no auto-approval in prod + require_sovereignty_declaration: true + require_accreditation_submission: true # must submit at least self_declared + require_health_check_before_approval: true + approval_timeout: P7D # auto-reject if not approved within 7 days + + fsi: + min_approval_method: dual_approval # everything requires dual approval + allow_token_auto_approval: false + require_sovereignty_declaration: true + require_accreditation_submission: true + minimum_accreditation_type: third_party # self_declared not accepted + require_health_check_before_approval: true + require_governance_matrix_check: true # governance matrix evaluated at registration + approval_timeout: P14D + + sovereign: + min_approval_method: committee # everything requires committee approval + allow_token_auto_approval: false + require_sovereignty_declaration: true + require_accreditation_submission: true + minimum_accreditation_type: regulatory_certification + require_hardware_attestation: true + require_governance_matrix_check: true + committee_group_handle: "platform/registration-committee" + approval_timeout: P30D +``` + +--- + +## 4. Registration Pipeline + +### 4.1 Lifecycle States + +``` +SUBMITTED โ†’ VALIDATING โ†’ PENDING_APPROVAL โ†’ ACTIVE + โ†˜ REJECTED (validation failure) + โ†˜ REJECTED (approval denied) + +Additional states: +ACTIVE โ†’ SUSPENDED (platform admin action or health failure) +ACTIVE โ†’ DEREGISTERING โ†’ DEREGISTERED (graceful removal) +ACTIVE โ†’ FORCED_DEREGISTERED (immediate removal) +``` + +### 4.2 Step 1 โ€” Submission + +Provider submits registration payload to DCM: + +``` +POST /api/v1/provider/register +Content-Type: application/json +X-DCM-Registration-Token: # optional; enables auto-approval if valid + +{ + "provider_type_id": "service_provider", + "handle": "eu-west-prod-1", + "display_name": "EU West Production Compute Provider", + "version": "2.1.0", + + # Mutual TLS certificate presented at connection level + # DCM extracts the certificate fingerprint from the TLS handshake + + "sovereignty_declaration": { ... }, + "accreditations": [ ... ], + "capabilities": { ... }, # per-type capability declaration + "health_endpoint": "https://provider.example.com/health", + "delivery_endpoint": "https://provider.example.com/dispatch" +} + +Response 202 Accepted: +{ + "registration_uuid": "", + "status": "VALIDATING", + "token_recognized": true, + "auto_approval_eligible": true, + "estimated_activation": "" +} +``` + +### 4.3 Step 2 โ€” Validation (automated) + +DCM runs automated validation checks. All must pass before advancing to PENDING_APPROVAL: + +``` +Validation checks: + V1: Provider type permitted in active profile + โ†’ Check Provider Type Registry: enabled_in_profiles includes active posture + โ†’ FAIL: REJECTED with reason "provider_type_not_enabled_in_profile" + + V2: Governance Matrix pre-check + โ†’ Evaluate matrix: is a provider of this type, in this zone, with these + accreditations, permitted to register? + โ†’ FAIL: REJECTED with reason "governance_matrix_denied" + rule_uuid + + V3: Registration token validation (if provided) + โ†’ Token exists and not expired + โ†’ Token matches provider_type_id and handle pattern + โ†’ Token not already used + โ†’ FAIL: Token invalid; fall back to non-token approval method + + V4: Certificate validation + โ†’ mTLS certificate presented and valid + โ†’ Certificate chain acceptable (registered CA or pinned self-signed) + โ†’ Certificate not in revocation list + โ†’ FAIL: REJECTED with reason "certificate_invalid" + + V5: Sovereignty declaration completeness + โ†’ Required fields present (if profile requires declaration) + โ†’ Jurisdiction codes valid + โ†’ FAIL: REJECTED with reason "sovereignty_declaration_incomplete" + + V6: Capability declaration consistency + โ†’ Declared capabilities consistent with provider type + โ†’ No contradictory declarations + โ†’ FAIL: REJECTED with reason "capability_declaration_invalid" + + V7: Health endpoint reachability + โ†’ DCM contacts health_endpoint + โ†’ Provider responds with valid health payload + โ†’ FAIL: status โ†’ PENDING_APPROVAL with warning (profile may require passing) + + V8: Accreditation submission check + โ†’ If profile requires accreditation submission: at least one accreditation present + โ†’ Accreditation type meets profile minimum + โ†’ FAIL: REJECTED with reason "accreditation_insufficient" +``` + +### 4.4 Step 3 โ€” Approval + +Approval flow depends on effective_approval_method: + +**auto:** Registration immediately advances to ACTIVE after validation passes. + +**human_review:** +``` +Registration enters PENDING_APPROVAL +Platform admin notification dispatched (urgency: medium) +Platform admin reviews in Admin API or Flow GUI: + GET /api/v1/admin/registrations/pending + POST /api/v1/admin/registrations/{registration_uuid}/approve + POST /api/v1/admin/registrations/{registration_uuid}/reject +On approval: โ†’ ACTIVE +On rejection: โ†’ REJECTED with required reason field +On timeout (approval_timeout): โ†’ REJECTED with reason "approval_timeout" +``` + +**dual_approval:** +``` +Registration enters PENDING_APPROVAL +Two independent platform admins must approve +First approval: recorded; notification sent to other admins for second approval +Second approval by different actor: โ†’ ACTIVE +Same actor cannot approve twice +On timeout: โ†’ REJECTED +``` + +**committee:** +``` +Registration enters PENDING_APPROVAL +Committee DCMGroup notified (all members) +Members vote via Admin API within declared quorum window +Quorum reached: โ†’ ACTIVE +Quorum not reached within approval_timeout: โ†’ REJECTED +``` + +### 4.5 Step 4 โ€” Activation + +On ACTIVE status: +- Provider enters the DCM provider registry +- Governance matrix rules are re-evaluated with this provider now active +- Capacity monitoring begins (if Service or Information Provider) +- Health check polling begins +- Certificate rotation schedule established +- Activation audit record written: PROVIDER_ACTIVATED +- Notification: platform admin + Tenant admins (if Tenant-scoped provider) + +--- + +## 5. Per-Type Capability Declaration Schemas + +### 5.1 Service Provider Capabilities + +```yaml +service_provider_capabilities: + resource_types: + - resource_type_fqn: Compute.VirtualMachine + resource_type_spec_version: "2.1.0" + catalog_item_uuid: + availability_zones: [eu-west-1a, eu-west-1b] + max_instances: 1000 + + capacity_model: + reporting_method: reserve_query | static_declaration | both + reserve_query_endpoint: /reserve + reserve_query_timeout: PT10S + static_capacity: + Compute.VirtualMachine: 500 + + cancellation: + supports_cancellation: true + cancellation_supported_during: [DISPATCHED, PROVISIONING] + partial_rollback_possible: true + + discovery: + supports_discovery: true + discovery_endpoint: /discover + discovery_method: api_query | passive_event | hybrid + supports_incremental_discovery: true + + naturalization: + target_format: openstack_nova | vmware_vsphere | custom + custom_schema_ref: + + cost_metadata: + capex_allocation_per_unit: 12.50 + opex_per_unit_per_hour: 0.28 + currency: USD + cost_data_dynamic_source: null | + + data_handling: + max_data_classification_accepted: restricted + phi_capable: false # true requires HIPAA BAA accreditation + pci_capable: false +``` + +### 5.2 Information Provider Capabilities + +```yaml +information_provider_capabilities: + data_domains: + - domain: business_data + data_types: [business_unit, cost_center, product_owner] + authority_level: primary | secondary | supplementary + schema_version: "1.0.0" + query_endpoint: /query + write_back_supported: false + + query_capacity: + max_queries_per_second: 100 + rate_limit_window: 60s + burst_capacity: 200 + + confidence_model: + data_freshness_sla: PT1H + corroboration_sources: [cmdb, hr_system] + + caching: + cacheable: true + cache_ttl: PT15M + cache_invalidation_webhook: /invalidate +``` + +### 5.3 Storage Provider Capabilities + +```yaml +storage_provider_capabilities: + store_types_supported: + - store_type: gitops + branch_per_request: true + pr_semantics: true + search_index_companion: true + - store_type: write_once_snapshot + entity_uuid_keyed: true + hash_chain_integrity: true + point_in_time_query: true + + consistency: + guarantee: strong | eventual | bounded_staleness + bounded_staleness_max: PT5M + + replication: + geo_replicated: true + replication_regions: [eu-west, eu-north] + synchronous_replication: true + + encryption: + at_rest: AES-256 + hsm_backed: false + key_management: provider_managed | customer_managed | hsm + + retention: + supports_retention_policy: true + minimum_retention: P1Y + maximum_retention: P10Y + tamper_evident: true +``` + +### 5.4 Policy Provider Capabilities + +```yaml +policy_provider_capabilities: + mode: 1 | 2 | 3 | 4 + policy_types_supported: + - gatekeeper + - validation + - transformation + - recovery + - orchestration_flow + + framework: opa | cedar | custom + rego_version: "1.0" # for OPA providers + + # Mode 3/4 specific + remote_endpoint: https://policy.example.com/evaluate + endpoint_sovereignty_zone: eu-west-sovereign + evaluation_latency_p95: PT200MS + supports_bundle_push: true + supports_bundle_pull: true + + shadow_mode_supported: true + test_harness_endpoint: /test +``` + +### 5.5 Auth Provider Capabilities + +```yaml +auth_provider_capabilities: + authentication_modes: + - api_key + - ldap + - oidc + - oidc_mfa + - saml + - mtls + - hardware_token + - hardware_token_mfa + + mfa_methods: + - totp + - push_notification + - hardware_token + + rbac_model: flat | hierarchical | attribute_based + external_idp_integration: true + idp_protocols: [oidc, saml, ldap] + + token_lifetime_config: + default_lifetime: PT1H + min_lifetime: PT5M + max_lifetime: PT8H + step_up_supported: true + + builtin: false # true for DCM's built-in auth provider +``` + +### 5.6 Notification Provider Capabilities + +```yaml +notification_provider_capabilities: + delivery_channels: + - channel_type: slack + supports_threading: true + supports_urgency_routing: true + config_schema_ref: + - channel_type: pagerduty + supports_escalation: true + config_schema_ref: + - channel_type: webhook + protocols: [https] + auth_modes: [hmac_sha256, mtls, bearer] + config_schema_ref: + - channel_type: email + html_supported: true + + delivery_guarantees: + at_least_once: true + idempotency_key: notification_uuid + max_delivery_latency_seconds: 30 + retry_policy: + max_attempts: 7 + backoff: exponential + on_exhaustion: dead_letter + + sovereignty_aware_delivery: true # checks endpoint jurisdiction before delivery +``` + +### 5.7 Credential Provider Capabilities + +```yaml +credential_provider_capabilities: + credential_types: + - api_key + - x509_certificate + - ssh_key + - service_account_token + - database_password + - hsm_backed_key + + secret_engines: + - vault + - aws_secrets_manager + - azure_key_vault + - gcp_secret_manager + + rotation_support: true + hsm_backed: false + fips_140_2_level: 1 | 2 | 3 # for sovereign deployments + dynamic_secrets: true # generate credentials on demand +``` + +### 5.8 Message Bus Provider Capabilities + +```yaml +message_bus_provider_capabilities: + protocols: [kafka, amqp, mqtt, grpc] + persistence: true + durability: at_least_once | exactly_once + max_throughput_msg_per_sec: 100000 + retention: + message_retention: P7D + retention_configurable: true + external_endpoints: false # true if messages can leave sovereignty boundary + encryption_in_transit: TLS-1.3 + encryption_at_rest: AES-256 +``` + +### 5.9 Meta Provider Capabilities + +```yaml +meta_provider_capabilities: + constituent_provider_types: + - service_provider + - information_provider + + composition_model: sequential | parallel | conditional + partial_delivery_supported: true + compensation_supported: true + + resource_types_composed: + - resource_type_fqn: ApplicationStack.WebApp + constituent_resource_types: + - Compute.VirtualMachine + - Network.IPAddress + - DNS.Record + - Network.LoadBalancer +``` + +--- + +## 6. Federated Trust Configuration + +### 6.1 Federation Trust Postures + +| Posture | Description | Operations permitted | +|---------|-------------|---------------------| +| `verified` | Manually verified and approved by local platform admin | Full declared scope per tunnel authorization | +| `vouched` | Introduced through a trusted Hub DCM | Vouching authority's declared scope; cannot exceed voucher's scope | +| `provisional` | Cryptographically verified but not yet manually approved | catalog_query only (if profile permits) | + +### 6.2 Federation Trust Registration Flow + +``` +Remote DCM requests federation peering + โ”‚ + โ–ผ Cryptographic verification (always): + โ”‚ mTLS certificate validation + โ”‚ Certificate not in revocation list + โ”‚ Certificate signed by acceptable CA + + โ–ผ Governance matrix pre-check: + โ”‚ Is federation with this peer's jurisdiction/accreditation permitted? + + โ–ผ Trust posture determination: + โ”‚ Prior record of this remote UUID? โ†’ verified or vouched (per prior record) + โ”‚ No prior record โ†’ provisional + + โ–ผ Approval flow (per profile): + โ”‚ dev: provisional auto-promoted to verified (if governance matrix permits) + โ”‚ standard: human_review for verified promotion; provisional gets limited scope + โ”‚ prod: dual_approval for verified promotion; no provisional operations + โ”‚ fsi: dual_approval + accreditation check; no provisional + โ”‚ sovereign: committee_approval + hardware attestation; no provisional + + โ–ผ Scope assignment per trust posture + + โ–ผ Tunnel established with governance matrix enforcement +``` + +### 6.3 Profile Federation Trust Policy + +```yaml +profile_federation_policy: + minimal: + permitted_trust_postures: [verified, vouched, provisional] + auto_promote_provisional: true + cross_jurisdiction_permitted: true + accreditation_required_for_federation: false + + dev: + permitted_trust_postures: [verified, vouched, provisional] + auto_promote_provisional: true + provisional_permitted_operations: [catalog_query, resource_query] + cross_jurisdiction_permitted: true + + standard: + permitted_trust_postures: [verified, vouched] + approval_method_for_verified: human_review + cross_jurisdiction_permitted: true + accreditation_required_for_federation: false + + prod: + permitted_trust_postures: [verified] + approval_method_for_verified: dual_approval + cross_jurisdiction_permitted: true + accreditation_required_for_federation: false + + fsi: + permitted_trust_postures: [verified] + approval_method_for_verified: dual_approval + cross_jurisdiction_permitted: false + accreditation_required_for_federation: true + minimum_peer_accreditation: third_party + re_verification_interval: PT8H + + sovereign: + permitted_trust_postures: [verified] + approval_method_for_verified: committee + cross_jurisdiction_permitted: false + accreditation_required_for_federation: true + minimum_peer_accreditation: sovereign_authorization + hardware_attestation_required: true + data_classification_boundary: internal + re_verification_interval: PT4H +``` + +--- + +## 7. Ongoing Lifecycle After Activation + +### 7.1 Health Monitoring + +``` +DCM polls provider health endpoint every health_check_interval + โ”‚ + โ”œโ”€โ”€ Response: healthy โ†’ no action; next poll scheduled + โ”œโ”€โ”€ Response: degraded โ†’ DCM updates capacity rating; reduces routing preference + โ”œโ”€โ”€ No response (1 failure) โ†’ warning; retry at shorter interval + โ”œโ”€โ”€ No response (failure_threshold reached) โ†’ provider status โ†’ DEGRADED + โ”‚ Notification: platform admin (urgency: high) + โ”‚ New requests no longer routed to this provider + โ””โ”€โ”€ No response (2ร— failure_threshold) โ†’ provider status โ†’ UNAVAILABLE + Active entities checked; drift detection triggered + Platform admin notification (urgency: critical) +``` + +### 7.2 Certificate Rotation + +```yaml +certificate_rotation: + rotation_interval: P90D # profile-governed default + transition_window: P7D # old cert valid during transition + pre_rotation_warning: P14D # warn provider P14D before expiry + +# Rotation flow: +POST /api/v1/provider/certificates/rotate +{ + "new_certificate_pem": "", + "transition_window": "P7D" +} +# DCM accepts both old and new certificates during transition window +# After transition window: old certificate rejected +``` + +### 7.3 Capability Updates + +Providers may update their capability declarations (new resource types, updated capacity models, new accreditations). Capability updates go through a simplified registration amendment flow: + +``` +POST /api/v1/provider/capabilities/update +{ + "amendment_type": "add_resource_type | remove_resource_type | update_capacity | add_accreditation", + "changes": { ... } +} + +โ†’ VALIDATING (automated checks only) +โ†’ PENDING_APPROVAL (if amendment_type is add_resource_type or sovereignty change) +โ†’ ACTIVE (capability declarations updated) +``` + +### 7.4 Deregistration + +**Graceful deregistration:** +``` +Provider submits deregistration intent +DCM checks: active entities hosted at this provider +If active entities > 0: + Decision required: migrate_entities | decommission_entities | reject_deregistration +Platform admin approves deregistration plan +Provider enters DEREGISTERING state +Entity migration or decommission completes +Provider status โ†’ DEREGISTERED +``` + +**Forced deregistration:** +``` +POST /api/v1/admin/providers/{provider_uuid}/force-deregister +Role: platform_admin +Requires: dual_approval (fsi/sovereign: committee) + +Immediate effect: + Provider status โ†’ FORCED_DEREGISTERED + All active entities โ†’ INDETERMINATE_REALIZATION + Governance matrix re-evaluated for all affected entities + Recovery policy fires: DRIFT_RECONCILE or NOTIFY_AND_WAIT per profile +``` + +--- + +## 8. Error Model + +| Error Code | Meaning | +|-----------|---------| +| `provider_type_not_enabled` | Provider type not permitted in active profile | +| `governance_matrix_denied` | Governance matrix pre-check denied registration | +| `certificate_invalid` | mTLS certificate invalid or not from acceptable CA | +| `token_invalid` | Registration token expired, used, or type mismatch | +| `token_insufficient_scope` | Token present but does not grant required approval level | +| `sovereignty_declaration_incomplete` | Required sovereignty fields missing | +| `accreditation_insufficient` | Active profile requires higher accreditation type | +| `capability_declaration_invalid` | Capability declarations internally inconsistent | +| `approval_timeout` | Registration not approved within approval_timeout period | +| `health_check_failed` | Provider health endpoint unreachable during validation | +| `duplicate_handle` | A provider with this handle already exists in active status | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From fd241659d5b146fb699a50d4afdc0b880b4ba9a1 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 09:55:38 -0500 Subject: [PATCH 27/49] =?UTF-8?q?docs:=20rebase=20architecture=20onto=20th?= =?UTF-8?q?ree=20foundational=20abstractions=20=E2=80=94=20Data,=20Provide?= =?UTF-8?q?r,=20Policy?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 59 +++ .../accreditation-and-authorization-matrix.md | 13 + .../audit-provenance-observability.md | 15 +- .../architecture/data-model/auth-providers.md | 15 +- .../data-model/context-and-purpose.md | 13 + .../data-model/control-plane-components.md | 13 + .../architecture/data-model/dcm-federation.md | 13 + .../data-model/deployment-redundancy.md | 13 + .../data-model/entity-relationships.md | 13 + .../architecture/data-model/entity-types.md | 13 + .../docs/architecture/data-model/examples.md | 13 + .../architecture/data-model/foundations.md | 264 ++++++++++ .../architecture/data-model/four-states.md | 13 + .../data-model/governance-matrix.md | 13 + .../information-providers-advanced.md | 17 +- .../data-model/information-providers.md | 15 +- .../data-model/ingestion-model.md | 15 +- .../data-model/layering-and-versioning.md | 13 + .../data-model/notification-model.md | 13 + .../data-model/operational-models.md | 13 + .../ownership-sharing-allocation.md | 13 + .../data-model/policy-contract.md | 380 ++++++++++++++ .../data-model/policy-profiles.md | 13 + .../data-model/provider-contract.md | 488 ++++++++++++++++++ .../data-model/registry-governance.md | 15 +- .../data-model/resource-grouping.md | 13 + .../data-model/resource-service-entities.md | 13 + .../data-model/resource-type-hierarchy.md | 13 + .../data-model/service-dependencies.md | 13 + .../data-model/storage-providers.md | 15 +- .../data-model/universal-audit.md | 15 +- .../data-model/universal-groups.md | 13 + .../data-model/webhooks-messaging.md | 13 + 33 files changed, 1577 insertions(+), 9 deletions(-) create mode 100644 content/docs/architecture/data-model/foundations.md create mode 100644 content/docs/architecture/data-model/policy-contract.md create mode 100644 content/docs/architecture/data-model/provider-contract.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 7f54d2c..50711e4 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -1,3 +1,62 @@ +## SECTION 0 โ€” THE THREE FOUNDATIONAL ABSTRACTIONS (READ FIRST) + +DCM is built on three foundational abstractions. Every concept maps to one or more of these three. There is no fourth. + +### DATA โ€” Everything That Exists +Any structured artifact with a type, UUID, lifecycle state, fields, data classification, and provenance. Entities, layers, policies, accreditations, audit records, groups, relationships, sovereignty zones, registration tokens โ€” all Data. + +**Universal Data properties:** UUID (stable across full lifecycle) ยท typed ยท lifecycle state ยท artifact metadata (handle, version, status, owned_by) ยท field-level provenance ยท data_classification per field ยท immutable if versioned + +**Data lifecycle stages (four states):** Intent State (consumer declaration) โ†’ Requested State (assembled, policy-validated) โ†’ Realized State (provider-confirmed) โ†’ Discovered State (independently observed). These are the same entity at four lifecycle stages stored in different stores optimized for each access pattern. + +**Data is assembled via layers** in deterministic precedence order. Every field carries provenance of its origin and all modifications. + +### PROVIDER โ€” Everything External +Any external component DCM calls or that calls DCM. All providers implement the **unified base contract** (registration, health, sovereignty, accreditation, governance matrix enforcement, zero trust) plus a **typed capability extension** that declares what operations they expose. + +**Eleven provider types (all implement the same base contract):** +Service Provider (realize resources) ยท Information Provider (serve external data) ยท Storage Provider (persist DCM state) ยท Meta Provider (compose providers) ยท Policy Provider (evaluate policies externally) ยท Credential Provider (manage secrets) ยท Auth Provider (authenticate identities) ยท Notification Provider (deliver notifications) ยท Message Bus Provider (async event streaming) ยท Registry Provider (serve resource type registry) ยท Peer DCM (federation โ€” another DCM instance IS a typed provider) + +**Adding a new provider type** = implement base contract + define capability extension. No core changes. + +### POLICY โ€” Everything That Decides +Any rule artifact that fires when Data matches conditions, produces a typed output, and is enforced at a declared level. Policies govern every transition, transformation, and constraint. + +**Seven policy types (all implement the same base contract):** +GateKeeper (allow/deny) ยท Validation (pass/fail) ยท Transformation (field mutations) ยท Recovery (failure actions) ยท Orchestration Flow (pipeline ordering) ยท Governance Matrix Rule (boundary control) ยท Lifecycle Policy (relationship event actions) + +**Policies ARE the orchestration.** Pipeline steps are policies firing on payload type events. Static flows = Orchestration Flow policies with `ordered: true`. Dynamic flows = conditional policies. Adding/removing pipeline steps = adding/removing policies. + +**Adding a new policy type** = define a new output schema. Base contract inherited automatically. + +### THE RUNTIME โ€” Connecting the Three +``` +Event (Data state change) + โ†’ Policy Engine evaluates all matching Policies + โ†’ Policies produce decisions / mutations / actions + โ†’ Actions invoke Providers or produce new Data + โ†’ New Data triggers new Events + โ†’ Repeat +``` + +Control plane "components" are runtime specializations โ€” not a fourth abstraction: +- Request Orchestrator = event bus (runtime) +- Policy Engine = policy evaluator (runtime) +- Placement Engine = GateKeeper policy specialized for provider selection +- Cost Analysis = Information Provider (internal; data derivation) +- Lifecycle Constraint Enforcer = scheduled Recovery Policy trigger +- Discovery Scheduler = scheduled Provider invocation +- Notification Router = Transformation Policy + Notification Provider invocation +- Drift Reconciliation = Data comparison producing Drift Record artifacts +- Search Index = Storage Provider sub-type (queryable projection) + +### THE CORE ETHOS +Effective at the core mission ยท Easy to use ยท Easy to implement ยท Easy to extend and integrate + +**Foundation documents:** 00-foundations.md (three abstractions) ยท A-provider-contract.md (unified provider base) ยท B-policy-contract.md (unified policy base) + +--- + # DCM Project โ€” AI Model Prompt Script **Purpose:** This script provides an AI model with the full context needed to participate effectively in DCM project work. It should be provided at the start of any AI-assisted session involving DCM architecture, documentation, code, or design work. diff --git a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md index ef2a99b..cd1fb70 100644 --- a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md +++ b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md @@ -12,6 +12,19 @@ weight: 26 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + POLICY** +> +> Data: Accreditation artifacts. Policy: Zero Trust posture as policy concern type + + **Related Documents:** [Policy Profiles](14-policy-profiles.md) | [Resource/Service Entities](06-resource-service-entities.md) | [DCM Federation](22-dcm-federation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Operational Models](24-operational-models.md) --- diff --git a/content/docs/architecture/data-model/audit-provenance-observability.md b/content/docs/architecture/data-model/audit-provenance-observability.md index 87c5790..a09ad34 100644 --- a/content/docs/architecture/data-model/audit-provenance-observability.md +++ b/content/docs/architecture/data-model/audit-provenance-observability.md @@ -1,7 +1,7 @@ --- title: "Audit, Provenance, and Observability" type: docs -weight: 11 +weight: 12 --- > **โš ๏ธ Active Development Notice** @@ -14,6 +14,19 @@ weight: 11 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Four States](02-four-states.md) | [Storage Providers](11-storage-providers.md) | [Context and Purpose](00-context-and-purpose.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” audit records, provenance as structural data + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/auth-providers.md b/content/docs/architecture/data-model/auth-providers.md index 1bf04a0..7bcbe56 100644 --- a/content/docs/architecture/data-model/auth-providers.md +++ b/content/docs/architecture/data-model/auth-providers.md @@ -1,7 +1,7 @@ --- title: "Authentication, Authorization, and Auth Providers" type: docs -weight: 18 +weight: 19 --- > **โš ๏ธ Active Development Notice** @@ -13,6 +13,19 @@ weight: 18 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Webhooks and Messaging](18-webhooks-messaging.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: PROVIDER** +> +> The Provider abstraction โ€” Auth Provider and Credential Provider extensions + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md index ced7f58..290c7bc 100644 --- a/content/docs/architecture/data-model/context-and-purpose.md +++ b/content/docs/architecture/data-model/context-and-purpose.md @@ -14,6 +14,19 @@ weight: 0 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” foundational data model, provenance, four lifecycle states + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index eb851fb..d47a08d 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -12,6 +12,19 @@ weight: 25 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: RUNTIME** +> +> Runtime implementations of the three abstractions โ€” not a fourth abstraction + + **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Operational Models](24-operational-models.md) | [Policy Profiles](14-policy-profiles.md) --- diff --git a/content/docs/architecture/data-model/dcm-federation.md b/content/docs/architecture/data-model/dcm-federation.md index 56be7b9..e0db782 100644 --- a/content/docs/architecture/data-model/dcm-federation.md +++ b/content/docs/architecture/data-model/dcm-federation.md @@ -13,6 +13,19 @@ weight: 22 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: PROVIDER + POLICY** +> +> Provider: Peer DCM as typed Provider. Policy: federation governance rules + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/deployment-redundancy.md b/content/docs/architecture/data-model/deployment-redundancy.md index 03006af..a06b0ca 100644 --- a/content/docs/architecture/data-model/deployment-redundancy.md +++ b/content/docs/architecture/data-model/deployment-redundancy.md @@ -13,6 +13,19 @@ weight: 17 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit Model](16-universal-audit.md) | [Policy Organization](14-policy-profiles.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + PROVIDER** +> +> Data: deployment specification. Provider: Storage Provider redundancy + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md index 34aa500..8b2ab97 100644 --- a/content/docs/architecture/data-model/entity-relationships.md +++ b/content/docs/architecture/data-model/entity-relationships.md @@ -14,6 +14,19 @@ weight: 9 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) | [Information Providers](10-information-providers.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + POLICY** +> +> Data: relationship records. Policy: Lifecycle Policy output schema + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/entity-types.md b/content/docs/architecture/data-model/entity-types.md index 0930238..d6bed1c 100644 --- a/content/docs/architecture/data-model/entity-types.md +++ b/content/docs/architecture/data-model/entity-types.md @@ -12,6 +12,19 @@ weight: 1 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” typed entity extensions (Infrastructure Resource, Composite, Process) + + **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) --- diff --git a/content/docs/architecture/data-model/examples.md b/content/docs/architecture/data-model/examples.md index 1dc7670..67b9b4b 100644 --- a/content/docs/architecture/data-model/examples.md +++ b/content/docs/architecture/data-model/examples.md @@ -12,6 +12,19 @@ weight: 4 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Reference Examples + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + PROVIDER + POLICY** +> +> Worked examples showing all three abstractions in operation + + **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) --- diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md new file mode 100644 index 0000000..9825471 --- /dev/null +++ b/content/docs/architecture/data-model/foundations.md @@ -0,0 +1,264 @@ +--- +title: "DCM Foundational Abstractions" +type: docs +weight: -10 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Foundation โ€” Read This First +**Related Documents:** [Data Model Context](00-context-and-purpose.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) + +--- + +## 1. The Three Abstractions + +DCM is built on three foundational abstractions. Every concept in the architecture is an instance of one of these three โ€” or a combination of them. There is no fourth. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ DATA โ”‚ +โ”‚ โ”‚ +โ”‚ Everything that exists, is stored, has a lifecycle, and flows โ”‚ +โ”‚ through the system. Entities, layers, policies, accreditations, โ”‚ +โ”‚ audit records, groups, relationships โ€” all Data. โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ flows through + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ PROVIDER โ”‚ โ”‚ POLICY โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ Every external โ”‚ โ”‚ Every rule that fires on Data, โ”‚ +โ”‚ component DCM โ”‚ โ”‚ decides what happens, transforms โ”‚ +โ”‚ calls or that โ”‚ โ”‚ values, or enforces constraints. โ”‚ +โ”‚ calls DCM. โ”‚ โ”‚ โ”‚ +โ”‚ Eleven typed โ”‚ โ”‚ Seven typed output schemas. โ”‚ +โ”‚ capability โ”‚ โ”‚ One evaluation algorithm. โ”‚ +โ”‚ extensions. โ”‚ โ”‚ Same lifecycle for all. โ”‚ +โ”‚ One base contract. โ”‚ โ”‚ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +**The runtime that connects them:** + +``` +Event (Data state change) + โ†’ Policy Engine evaluates all matching Policies + โ†’ Policies produce decisions / mutations / actions + โ†’ Actions invoke Providers or produce new Data + โ†’ New Data triggers new Events + โ†’ Repeat +``` + +This is the complete DCM operational model. Everything else is a typed specialization of these three abstractions operating through this loop. + +--- + +## 2. DATA โ€” Everything That Exists + +**Definition:** Data is any structured artifact in DCM with a type, fields, classification, provenance, and lifecycle state. Data is always versioned, always identified by UUID, and always carries provenance describing where each field value came from. + +**The universal properties of all Data:** +- **UUID** โ€” every Data artifact has a universally unique identifier, stable across its full lifecycle +- **Type** โ€” every Data artifact has a declared type that determines its schema and valid field set +- **Lifecycle state** โ€” every Data artifact is in exactly one lifecycle state at any moment +- **Artifact metadata** โ€” every Data artifact carries a standard metadata block (handle, version, status, owned_by, created_by, created_via) +- **Provenance** โ€” every field in every Data artifact carries lineage metadata describing its origin and all modifications +- **Data classification** โ€” every field carries a classification (public โ†’ classified) governing what may cross interaction boundaries +- **Immutability if versioned** โ€” once a version is published, it cannot be modified; changes produce new versions + +**The complete Data taxonomy:** + +| Data Type | Description | Storage | +|-----------|-------------|---------| +| **Resource Entity** | A realized infrastructure resource; the primary managed thing | Realized Store | +| **Process Entity** | An ephemeral execution (job, playbook, pipeline) | Realized Store | +| **Composite Entity** | A Meta Provider composition of Resource Entities | Realized Store | +| **Intent State** | Consumer's raw declaration before processing | Intent Store (GitOps) | +| **Requested State** | Fully assembled, policy-validated provider payload | Requested Store | +| **Discovered State** | What actually exists per discovery observation | Discovered Store | +| **Data Layer** | A versioned artifact contributing fields to assembly | Layer Store (GitOps) | +| **Resource Type Specification** | Schema definition for a resource type | Registry | +| **Provider Catalog Item** | Provider-specific instantiation of a Resource Type Spec | Registry | +| **Policy** | A rule artifact with match conditions and output schema | Policy Store (GitOps) | +| **Policy Group** | A collection of policies grouped by concern_type | Policy Store (GitOps) | +| **Policy Profile** | A composition: one posture + zero or more compliance domains | Policy Store (GitOps) | +| **Accreditation** | A compliance certification artifact | Accreditation Store | +| **Sovereignty Zone** | A geopolitical/regulatory boundary artifact | Config Store | +| **Registration Token** | A scoped authorization artifact for provider registration | Token Store | +| **DCMGroup** | A grouping artifact (tenant_boundary, resource_grouping, etc.) | Config Store | +| **Drift Record** | A comparison result artifact | Operational Store | +| **Audit Record** | An immutable event record | Audit Store | +| **Governance Matrix Rule** | A boundary control rule artifact | Policy Store (GitOps) | +| **Orphan Candidate** | A potentially untracked resource artifact | Operational Store | + +**How Data flows โ€” the four lifecycle stages:** + +Every Resource Entity flows through four stages. These are not four separate things โ€” they are the same entity at four different lifecycle stages, stored in specialized stores optimized for each stage's access pattern: + +``` +Consumer Intent + โ”‚ raw consumer declaration + โ–ผ +Intent State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ GitOps Store + โ”‚ layer assembly + policy evaluation + โ–ผ +Requested State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Write-once Store + โ”‚ provider execution + โ–ผ +Realized State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Snapshot Store + โ”‚ independent observation + โ–ผ +Discovered State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Ephemeral Stream +``` + +**How Data is composed โ€” the layering model:** + +Data fields are assembled from multiple contributing layers in a deterministic precedence order. See [Data Model Context](00-context-and-purpose.md) and [Layering and Versioning](03-layering-and-versioning.md) for the complete assembly algorithm. + +--- + +## 3. PROVIDER โ€” Everything External + +**Definition:** A Provider is any external component that DCM interacts with through a defined contract. Providers receive Data from DCM, act on it, and return Data to DCM. The contract governs how this exchange happens โ€” not what the Provider does internally. + +**The universal properties of all Providers:** +- **Registration** โ€” every Provider registers with DCM, declaring its capabilities, sovereignty, and accreditation +- **Health check** โ€” every Provider exposes a health endpoint; DCM monitors it continuously +- **Sovereignty declaration** โ€” every Provider declares where it operates and what jurisdictions it covers +- **Accreditation** โ€” every Provider declares its compliance certifications; DCM enforces these via the Governance Matrix +- **Governance Matrix enforcement** โ€” every interaction with a Provider is subject to the Governance Matrix before data crosses the boundary +- **Zero trust** โ€” every Provider interaction is authenticated and authorized; no implicit trust from network position +- **Lifecycle** โ€” every Provider registration goes through a defined lifecycle (SUBMITTED โ†’ VALIDATING โ†’ ACTIVE โ†’ DEREGISTERED) + +**The complete Provider taxonomy:** + +| Provider Type | Capability | Data direction | +|--------------|-----------|---------------| +| **Service Provider** | Realizes infrastructure resources | DCM โ†’ Provider โ†’ DCM | +| **Information Provider** | Serves authoritative external data | DCM queries โ†’ Provider responds | +| **Storage Provider** | Persists DCM state | DCM reads/writes โ†” Provider | +| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Children โ†’ DCM | +| **Policy Provider** | Evaluates policies externally | DCM sends payload โ†’ Provider decides | +| **Credential Provider** | Manages secrets and credentials | DCM requests โ†’ Provider issues | +| **Auth Provider** | Authenticates identities | DCM verifies โ†’ Provider confirms | +| **Notification Provider** | Delivers notifications | DCM sends envelope โ†’ Provider delivers | +| **Message Bus Provider** | Async event streaming | DCM publishes/subscribes โ†” Provider | +| **Registry Provider** | Serves the resource type registry | DCM pulls โ†’ Provider serves | +| **Peer DCM** | Another DCM instance (federation) | DCM โ†” DCM via federation tunnel | + +**The unified Provider base contract** is defined in [A-provider-contract.md](A-provider-contract.md). All eleven Provider types implement this base contract. What varies is the capability declaration โ€” what operations the Provider exposes and what data flows in which direction. + +**Peer DCM as Provider:** A federated DCM instance is a typed Provider. The federation tunnel is the Provider's communication channel. Federation routing is policy-governed provider selection. There is no separate "federation abstraction" โ€” federation is the Provider abstraction applied across DCM instances. + +--- + +## 4. POLICY โ€” Everything That Decides + +**Definition:** A Policy is a rule artifact that fires when Data matches declared conditions, produces a typed output (decision, mutation, action, or directive), and is enforced according to a declared level. Policies govern every transition, transformation, and constraint in DCM. + +**The universal properties of all Policies:** +- **Match conditions** โ€” every Policy declares when it fires, using the four governance matrix axes (subject, data, target, context) or payload type + field conditions +- **Typed output schema** โ€” every Policy produces one of seven output types; the output type determines how the Policy Engine applies the result +- **Enforcement level** โ€” hard (cannot be overridden) or soft (can be tightened by more-specific policies) +- **Domain precedence** โ€” policies at more-specific domains win within their concern type; system > platform > tenant > resource_type > entity +- **Lifecycle** โ€” every Policy follows the standard artifact lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired) +- **Shadow mode** โ€” proposed Policies execute against real traffic without applying results; safe validation before activation +- **Audit** โ€” every Policy evaluation produces an audit record regardless of outcome + +**The complete Policy taxonomy:** + +| Policy Type | Fires on | Output | +|-------------|---------|--------| +| **GateKeeper** | Request payload | `allow` or `deny` with reason | +| **Validation** | Request payload | `pass` or `fail` with field-level details | +| **Transformation** | Request payload | `mutations[]` โ€” field additions, changes, locks | +| **Recovery** | Failure/timeout trigger condition | `action` + parameters (DRIFT_RECONCILE, DISCARD_AND_REQUEUE, etc.) | +| **Orchestration Flow** | Payload type events | `flow_directive` โ€” sequence ordering for pipeline steps | +| **Governance Matrix Rule** | Any cross-boundary interaction | `ALLOW / DENY / ALLOW_WITH_CONDITIONS / STRIP_FIELD / REDACT / AUDIT_ONLY` | +| **Lifecycle Policy** | Relationship events | `action` on the related entity (save, destroy, notify, cascade) | + +**The unified Policy base contract** is defined in [B-policy-contract.md](B-policy-contract.md). All seven Policy types implement this base contract. What varies is the output schema. + +**Policies as orchestration:** Static and dynamic workflows are both Policy. An Orchestration Flow Policy with `ordered: true` is a static workflow. A conditional GateKeeper or Transformation Policy is a dynamic workflow. Both are evaluated by the same Policy Engine. Adding a pipeline step = writing a Policy. Removing a step = deactivating a Policy. + +**The Governance Matrix as Policy:** The Governance Matrix rules (doc 27) are typed Policies with the `boundary_control` output schema. They fire at every cross-boundary interaction. They follow the same match conditions, enforcement levels, and lifecycle as all other Policies. The governance matrix is not a separate system โ€” it is the Policy abstraction applied at interaction boundaries. + +--- + +## 5. The Runtime โ€” Connecting the Three + +The Request Orchestrator and Policy Engine are the runtime that connects the three abstractions. They are not a fourth abstraction โ€” they are the implementation machinery. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Request Orchestrator โ”‚ +โ”‚ (event bus โ€” not a sequencer) โ”‚ +โ”‚ โ”‚ +โ”‚ Receives events โ†’ routes to Policy Engine โ”‚ +โ”‚ Policy Engine evaluates all matching Policies โ”‚ +โ”‚ Results: invoke Providers OR produce new Data โ”‚ +โ”‚ New Data โ†’ new events โ†’ new Policy evaluations โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +**Key runtime properties:** +- The Request Orchestrator contains no pipeline logic โ€” Policies define what happens +- Every pipeline step is a Policy firing on a payload type event +- Parallel execution: Policies with no data dependencies evaluate concurrently +- Static flows: Orchestration Flow Policies with `ordered: true` +- Dynamic flows: conditional Policies that fire based on payload state + +**Control plane components as runtime specializations:** + +The components in [Control Plane Components](25-control-plane-components.md) are specialized runtime implementations, not separate abstractions: + +| Component | Abstraction it implements | +|-----------|--------------------------| +| Request Orchestrator | The runtime event bus | +| Policy Engine | The runtime Policy evaluator | +| Placement Engine | Policy evaluation specialized for provider selection | +| Cost Analysis | Information Provider (internal; data derivation) | +| Lifecycle Constraint Enforcer | Scheduled Recovery Policy trigger | +| Discovery Scheduler | Scheduled Provider invocation | +| Notification Router | Transformation Policy + Notification Provider invocation | +| Drift Reconciliation | Data comparison producing new Data (drift records) | +| Search Index | Storage Provider sub-type (queryable projection) | + +--- + +## 6. Extension Points + +DCM is designed to be extended without modifying the core. Every extension fits within the three abstractions: + +**Extending Data:** New entity types, new artifact types, new resource types, new group classes โ€” all are typed extensions of the Data abstraction. Register them in the Resource Type Registry or DCMGroup registry. + +**Extending Providers:** New provider types (a Billing Provider, a CMDB Provider, an AI/ML Provider) โ€” implement the unified Provider base contract with a new capability declaration extension. Register in the Provider Type Registry. + +**Extending Policies:** New policy types, new governance matrix rules, new orchestration flows โ€” implement the unified Policy base contract with a new output schema. Register in the Policy Store via GitOps. + +**The extension principle:** If you can express it as Data, Provider, or Policy โ€” it belongs in DCM. If you cannot express it within these three abstractions, it is either a runtime implementation detail or a genuinely novel concept that should be explicitly identified and documented as such. + +--- + +## 7. The Core Ethos + +These three abstractions serve DCM's core ethos: + +**Effective at the core mission** โ€” managing the lifecycle of infrastructure resources across a sovereign private cloud. The Data abstraction ensures every resource is tracked, versioned, and auditable. The Provider abstraction ensures every external integration is governed and trustworthy. The Policy abstraction ensures every decision is declared, reproducible, and auditable. + +**Easy to use** โ€” consumers interact with Data (submit an intent, receive a resource). Policies govern what happens without consumers needing to understand them. Providers handle the implementation details. + +**Easy to implement** โ€” implementors implement one base contract (Provider) with a typed capability extension. The Policy Engine handles all policy evaluation. The Data model handles all storage and provenance. + +**Easy to extend and integrate** โ€” add a new provider type by implementing the base contract. Add a new policy type by defining an output schema. Add a new data type by defining a schema. No core changes required. + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md index 4f47ad0..6312f48 100644 --- a/content/docs/architecture/data-model/four-states.md +++ b/content/docs/architecture/data-model/four-states.md @@ -14,6 +14,19 @@ weight: 2 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” four lifecycle stages and their storage models + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/governance-matrix.md b/content/docs/architecture/data-model/governance-matrix.md index 0318916..7d5f19d 100644 --- a/content/docs/architecture/data-model/governance-matrix.md +++ b/content/docs/architecture/data-model/governance-matrix.md @@ -12,6 +12,19 @@ weight: 27 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: POLICY** +> +> The Policy abstraction โ€” Governance Matrix Rule output schema for boundary control + + **Related Documents:** [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) | [DCM Federation](22-dcm-federation.md) | [Policy Profiles](14-policy-profiles.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Control Plane Components](25-control-plane-components.md) --- diff --git a/content/docs/architecture/data-model/information-providers-advanced.md b/content/docs/architecture/data-model/information-providers-advanced.md index 3d6baf7..0accb6e 100644 --- a/content/docs/architecture/data-model/information-providers-advanced.md +++ b/content/docs/architecture/data-model/information-providers-advanced.md @@ -1,7 +1,7 @@ --- -title: "Information Providers: Confidence Scoring and Authority" +title: "Information Providers: Advanced" type: docs -weight: 20 +weight: 21 --- > **โš ๏ธ Active Development Notice** @@ -13,6 +13,19 @@ weight: 20 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Information Providers](10-information-providers.md) | [Policy Organization](14-policy-profiles.md) | [Universal Audit Model](16-universal-audit.md) | [DCM Federation](22-dcm-federation.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: PROVIDER** +> +> The Provider abstraction โ€” Information Provider advanced capabilities + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/information-providers.md b/content/docs/architecture/data-model/information-providers.md index 8dfadea..f452dd0 100644 --- a/content/docs/architecture/data-model/information-providers.md +++ b/content/docs/architecture/data-model/information-providers.md @@ -1,7 +1,7 @@ --- title: "Information Providers" type: docs -weight: 9 +weight: 10 --- > **โš ๏ธ Active Development Notice** @@ -14,6 +14,19 @@ weight: 9 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: PROVIDER** +> +> The Provider abstraction โ€” Information Provider capability extension + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/ingestion-model.md b/content/docs/architecture/data-model/ingestion-model.md index 8ddb033..6ac9e8c 100644 --- a/content/docs/architecture/data-model/ingestion-model.md +++ b/content/docs/architecture/data-model/ingestion-model.md @@ -1,7 +1,7 @@ --- title: "Ingestion Model" type: docs -weight: 12 +weight: 13 --- > **โš ๏ธ Active Development Notice** @@ -13,6 +13,19 @@ weight: 12 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + PROVIDER** +> +> Data: ingestion state artifacts. Provider: discovery provider invocation + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md index e6d2e7a..4aec832 100644 --- a/content/docs/architecture/data-model/layering-and-versioning.md +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -14,6 +14,19 @@ weight: 3 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” how Data is assembled from layers + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/notification-model.md b/content/docs/architecture/data-model/notification-model.md index 4a034e8..44a04c5 100644 --- a/content/docs/architecture/data-model/notification-model.md +++ b/content/docs/architecture/data-model/notification-model.md @@ -12,6 +12,19 @@ weight: 23 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: PROVIDER + POLICY** +> +> Provider: Notification Provider. Policy: audience resolution and subscription rules + + **Related Documents:** [Webhooks, Messaging, and External Integration](18-webhooks-messaging.md) | [Entity Relationships](09-entity-relationships.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Auth Providers](19-auth-providers.md) | [Universal Audit](16-universal-audit.md) --- diff --git a/content/docs/architecture/data-model/operational-models.md b/content/docs/architecture/data-model/operational-models.md index ad87424..1dad08e 100644 --- a/content/docs/architecture/data-model/operational-models.md +++ b/content/docs/architecture/data-model/operational-models.md @@ -12,6 +12,19 @@ weight: 24 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: POLICY** +> +> The Policy abstraction โ€” Recovery Policy types, trigger vocabulary, action vocabulary + + **Related Documents:** [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Policy Profiles](14-policy-profiles.md) | [Notification Model](23-notification-model.md) --- diff --git a/content/docs/architecture/data-model/ownership-sharing-allocation.md b/content/docs/architecture/data-model/ownership-sharing-allocation.md index 145adc4..c6c17ec 100644 --- a/content/docs/architecture/data-model/ownership-sharing-allocation.md +++ b/content/docs/architecture/data-model/ownership-sharing-allocation.md @@ -12,6 +12,19 @@ weight: 5 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” ownership models for entity data + + **Related Documents:** [Entity Types](01-entity-types.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md) --- diff --git a/content/docs/architecture/data-model/policy-contract.md b/content/docs/architecture/data-model/policy-contract.md new file mode 100644 index 0000000..2db5291 --- /dev/null +++ b/content/docs/architecture/data-model/policy-contract.md @@ -0,0 +1,380 @@ +--- +title: "Unified Policy Contract" +type: docs +weight: -8 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Foundation +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Profiles](14-policy-profiles.md) | [Governance Matrix](27-governance-matrix.md) | [OPA Integration](../specifications/dcm-opa-integration-spec.md) + +--- + +## 1. The Unified Policy Contract + +Every Policy in DCM โ€” regardless of type โ€” implements a single base contract. What varies between policy types is the **output schema**: what the Policy produces when its match conditions are satisfied. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ BASE POLICY CONTRACT โ”‚ +โ”‚ โ”‚ +โ”‚ Match Conditions ยท Enforcement Level ยท Domain โ”‚ +โ”‚ Lifecycle ยท Audit ยท Shadow Mode โ”‚ +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ OUTPUT SCHEMA โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ โ”‚ What this policy type produces when it fires. โ”‚ โ”‚ +โ”‚ โ”‚ Seven typed output schemas. โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +**Adding a new policy type** = define a new output schema. The base contract, evaluation algorithm, lifecycle, and audit obligations are inherited. + +--- + +## 2. Base Contract โ€” Match Conditions + +All policies declare when they fire using one or both of two match condition models: + +**Model A โ€” Payload type + field conditions** (for pipeline policies: GateKeeper, Validation, Transformation, Recovery, Orchestration Flow): + +```yaml +match: + payload_type: request.initiated | request.layers_assembled | ... # from closed vocabulary + conditions: + - field: # dot-notation path into the payload + operator: equals | in | minimum | maximum | contains | matches + value: + - field: + operator: not_equals + value: + condition_logic: all | any # default: all +``` + +**Model B โ€” Four-axis boundary conditions** (for boundary policies: Governance Matrix Rules): + +```yaml +match: + subject: + type: + identity: { ... } + tenant: { ... } + data: + classification: + resource_type: + field_paths: { mode: allowlist | blocklist, paths: [...] } + capability: + target: + type: + sovereignty_zone: { match: } + accreditation_held: { includes: [...] } + trust_posture: + context: + profile: { deployment_posture: } + zero_trust_posture: { minimum: } + federated: true | false +``` + +Policies may declare match conditions using either model. Orchestration Flow policies and Lifecycle Policies may also use relationship event conditions (see Section 9). + +--- + +## 3. Base Contract โ€” Enforcement Level + +```yaml +enforcement: hard | soft + +# hard: cannot be relaxed by any downstream rule at any domain level +# A hard DENY cannot be overridden by any Tenant, entity, or operator override +# Reserved for: sovereign/classified data boundaries, regulatory hard requirements + +# soft: establishes a default that downstream rules can tighten +# A soft ALLOW can be restricted to DENY by a more-specific rule +# A soft DENY cannot be relaxed to ALLOW by a downstream rule +``` + +Most policies are soft. Hard enforcement is reserved for absolute security constraints. + +--- + +## 4. Base Contract โ€” Domain Precedence + +Policies operate within a domain hierarchy. More-specific domains win within the same concern type: + +``` +system (most trusted โ€” DCM built-in) + โ””โ”€โ”€ platform (platform admin declared) + โ””โ”€โ”€ tenant (Tenant admin declared) + โ””โ”€โ”€ resource_type (per resource type spec) + โ””โ”€โ”€ entity (per specific entity โ€” most specific) +``` + +Within the same domain level, DENY wins over ALLOW. More-specific domain wins over less-specific. + +--- + +## 5. Base Contract โ€” Artifact Structure + +All policies are first-class DCM Data artifacts. They share the standard artifact metadata and lifecycle: + +```yaml +policy_artifact: + # Standard DCM artifact metadata (all artifacts carry this) + artifact_metadata: + uuid: + handle: "//" + version: "1.0.0" + status: developing | proposed | active | deprecated | retired + owned_by: { display_name: "", email: "" } + created_by: { display_name: "" } + created_via: pr | api | migration | system + + # Policy classification + policy_type: # gatekeeper | validation | transformation | + # recovery | orchestration_flow | + # governance_matrix_rule | lifecycle + concern_type: # security | compliance | operational | + # recovery_posture | zero_trust_posture | + # data_authorization_boundary | orchestration_flow + + domain: system | platform | tenant | resource_type | entity + + # Match conditions (Model A or B โ€” see Section 2) + match: { ... } + + # Enforcement + enforcement: hard | soft + + # Output schema (varies by policy_type โ€” see Sections 8-14) + output: { ... } + + # Audit + audit_on: [ALLOW, DENY, STRIP_FIELD] # which decisions produce audit records + notification_on: [DENY] # which decisions trigger notifications + notification_urgency: low | medium | high | critical + + # Compliance reference + compliance_basis: "" + review_required_before: "" +``` + +--- + +## 6. Base Contract โ€” Lifecycle + +All policies follow the five-status lifecycle: + +| Status | Behavior | +|--------|---------| +| `developing` | Dev mode only. Not applied in any environment. | +| `proposed` | Shadow mode: executes against real traffic; output captured but never applied. Used for safe validation. | +| `active` | Applied to all matching requests. | +| `deprecated` | Still active; replacement available; warning on evaluation. | +| `retired` | Terminal; cannot be used. | + +**Shadow mode (proposed status):** The policy evaluates against real traffic. Its output is captured in the Validation Store. Platform admins review shadow results before promoting to active. This is the primary mechanism for safe policy change management. + +--- + +## 7. Base Contract โ€” Evaluation and Audit + +**Evaluation order:** Within a domain level, policies are evaluated in declared priority order. Across domain levels, more-specific domains evaluate after (and can override) less-specific domains. + +**Parallel evaluation:** Policies with no data dependencies on each other evaluate concurrently. The Policy Engine tracks dependency declarations. + +**Audit:** Every policy evaluation produces an audit record regardless of outcome. The record includes: policy_uuid, policy_version, match_result, output, enforcement_level, actor, timestamp. No evaluation is silent. + +--- + +## 8. Output Schema โ€” GateKeeper + +**Fires on:** Request payload at assembly time. +**Produces:** An allow or deny decision for the request. + +```yaml +gatekeeper_output: + decision: allow | deny + reason: "" + field_locks: # optional: lock specific fields as immutable + - field: + lock_type: immutable | constrained + constraint_schema: # if constrained + warnings: [""] +``` + +**Policy Engine behavior:** +- `allow` โ†’ request proceeds; field_locks applied to payload +- `deny` โ†’ request blocked; `reason` included in consumer error response +- Any active GateKeeper producing `deny` โ†’ request blocked (all must allow) + +--- + +## 9. Output Schema โ€” Validation + +**Fires on:** Request payload; validates correctness of field values. +**Produces:** Pass or fail with field-level detail. + +```yaml +validation_output: + result: pass | fail + field_results: + - field: + result: valid | invalid + message: "" + suggested_value: # optional + advisory: [""] +``` + +**Policy Engine behavior:** +- `pass` โ†’ request proceeds +- `fail` โ†’ request blocked; `field_results` included in consumer error response + +--- + +## 10. Output Schema โ€” Transformation + +**Fires on:** Request payload; enriches, modifies, or injects field values. +**Produces:** A set of field mutations to apply to the payload. + +```yaml +transformation_output: + mutations: + - field: + operation: set | append | delete | lock + value: # for set/append + reason: "" + source_type: enrichment | injection | normalization | correction +``` + +**Policy Engine behavior:** All mutations from all active Transformation policies are collected and applied to the payload. Each mutation is recorded in field-level provenance with the policy_uuid as source. + +--- + +## 11. Output Schema โ€” Recovery + +**Fires on:** A failure or ambiguity trigger condition (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, CANCELLATION_FAILED, etc.). +**Produces:** A recovery action and parameters. + +```yaml +recovery_output: + action: DRIFT_RECONCILE | DISCARD_AND_REQUEUE | DISCARD_NO_REQUEUE | + ACCEPT_LATE_REALIZATION | COMPENSATE_AND_FAIL | + NOTIFY_AND_WAIT | ESCALATE | RETRY + action_parameters: + requeue_delay: PT0S # for DISCARD_AND_REQUEUE + max_attempts: 3 # for RETRY + backoff: exponential # for RETRY + deadline: PT4H # for NOTIFY_AND_WAIT + on_deadline_exceeded: ESCALATE # for NOTIFY_AND_WAIT + notify_before_action: true + notification_urgency: high +``` + +**Policy Engine behavior:** The first matching Recovery policy's action is executed. Recovery policies follow the same domain precedence โ€” resource_type override wins over tenant override wins over profile default. + +--- + +## 12. Output Schema โ€” Orchestration Flow + +**Fires on:** Pipeline payload type events. +**Produces:** A flow directive governing step ordering. + +```yaml +orchestration_flow_output: + ordered: true | false + steps: + - step: 1 + policy_handle: "" + condition: "" + on_fail: halt | skip | escalate + parallel_groups: # steps that may execute in parallel + - [step_1_id, step_2_id] +``` + +**Policy Engine behavior:** When `ordered: true`, steps execute in declared sequence. When `ordered: false`, the Policy Engine executes steps in parallel where no data dependencies exist. Orchestration Flow policies compose with standard GateKeeper and Transformation policies โ€” both types evaluate in the same pipeline. + +--- + +## 13. Output Schema โ€” Governance Matrix Rule + +**Fires on:** Any cross-boundary interaction (DCM โ†’ Provider, DCM โ†’ Peer DCM, Provider โ†’ DCM). +**Produces:** A boundary control decision with optional field permissions. + +```yaml +governance_matrix_output: + decision: ALLOW | DENY | ALLOW_WITH_CONDITIONS | STRIP_FIELD | REDACT | AUDIT_ONLY + conditions: # for ALLOW_WITH_CONDITIONS + - field: + operator: + value: + field_permissions: + mode: allowlist | blocklist | passthrough + paths: ["", ...] + on_blocked_field: STRIP_FIELD | DENY_REQUEST | REDACT + audit_on: [ALLOW, DENY, STRIP_FIELD] + notification_on: [DENY] + notification_urgency: critical +``` + +**Policy Engine behavior:** Hard DENY evaluated first โ€” any hard DENY is terminal. Soft decisions evaluated by domain precedence; DENY wins over ALLOW at the same level. Field permissions applied after decision determined. Audit record always written. + +--- + +## 14. Output Schema โ€” Lifecycle Policy + +**Fires on:** Relationship events (related entity state changes, relationship creation/release). +**Produces:** A lifecycle action to apply to related entities. + +```yaml +lifecycle_policy_output: + on_related_destroy: cascade | protect | detach | notify + on_related_suspend: cascade | ignore | notify + on_last_relationship_released: destroy | retain | notify + propagation_depth: 1 | 2 | N # how many relationship hops to propagate + action_delay: PT0S # grace period before executing action +``` + +**Policy Engine behavior:** When a relationship event occurs, all matching Lifecycle policies on both related entities are evaluated. The most restrictive action wins (save beats destroy). Conflicts between policies at the same domain level produce a CONFLICT_ERROR at policy ingestion time. + +--- + +## 15. Policy Composition + +Policies compose naturally through the domain precedence model: + +``` +System policy (GateKeeper: cpu_count max 64) + โ””โ”€โ”€ Platform policy (GateKeeper: prod VMs require manager approval) + โ””โ”€โ”€ Tenant policy (GateKeeper: payments team max cpu_count 32) + โ””โ”€โ”€ Resource-type policy (Transformation: inject monitoring) +``` + +For a single request, all active matching policies at all domain levels evaluate. GateKeepers at all levels must allow (any deny blocks). Transformations from all levels are collected and applied. Recovery policies use the most-specific matching policy. + +**Policy Groups** are Data artifacts that group related policies by concern_type. Profiles activate Policy Groups. This is how "apply the HIPAA profile" works โ€” it activates the HIPAA compliance domain's Policy Group, which contains all the GateKeeper, Validation, Transformation, and Governance Matrix policies required for HIPAA compliance. + +--- + +## 16. Related Policies + +| Policy | Rule | +|--------|------| +| `POL-001` | All DCM policy types implement the unified base contract. The output schema is the only thing that varies. | +| `POL-002` | Every policy evaluation produces an audit record. No evaluation is silent. | +| `POL-003` | Hard enforcement policies cannot be relaxed by any downstream rule at any domain level. | +| `POL-004` | Policies in `proposed` status execute in shadow mode โ€” output is captured and never applied. Shadow mode is the primary mechanism for safe policy change management. | +| `POL-005` | The Policy Engine is the sole evaluator of all policies. No component bypasses the Policy Engine to enforce rules directly. | +| `POL-006` | Adding a new policy type requires defining a new output schema. The base contract, evaluation algorithm, lifecycle, and audit obligations are inherited. | +| `POL-007` | Policies ARE the orchestration. Pipeline steps are Policies firing on payload type events. Static flows are Orchestration Flow Policies with `ordered: true`. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 417b013..422ba50 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -15,6 +15,19 @@ weight: 14 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Data Layers and Assembly](03-layering-and-versioning.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + POLICY** +> +> Data: Policy Group and Profile artifacts. Policy: concern types and composition + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/provider-contract.md b/content/docs/architecture/data-model/provider-contract.md new file mode 100644 index 0000000..105c119 --- /dev/null +++ b/content/docs/architecture/data-model/provider-contract.md @@ -0,0 +1,488 @@ +--- +title: "Unified Provider Contract" +type: docs +weight: -9 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Foundation +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Policy Contract](B-policy-contract.md) | [Registration Specification](../specifications/dcm-registration-spec.md) | [Governance Matrix](27-governance-matrix.md) | [Accreditation](26-accreditation-and-authorization-matrix.md) + +--- + +## 1. The Unified Provider Contract + +Every Provider in DCM โ€” regardless of type โ€” implements a single base contract. What varies between provider types is the **capability extension**: the specific operations exposed, the data that flows in each direction, and the typed schemas for that exchange. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ BASE PROVIDER CONTRACT โ”‚ +โ”‚ โ”‚ +โ”‚ Registration ยท Health ยท Sovereignty ยท Accreditation โ”‚ +โ”‚ Governance Matrix ยท Zero Trust ยท Lifecycle โ”‚ +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ CAPABILITY EXTENSION โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ โ”‚ What operations this provider type exposes. โ”‚ โ”‚ +โ”‚ โ”‚ What data flows in which direction. โ”‚ โ”‚ +โ”‚ โ”‚ What schemas govern the exchange. โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +**Adding a new provider type** = implement the base contract + define a capability extension. No changes to the core required. + +--- + +## 2. Base Contract โ€” Registration + +All providers register through the same pipeline. See [Registration Specification](../specifications/dcm-registration-spec.md) for the complete flow. + +```yaml +provider_base_registration: + # Standard artifact metadata + artifact_metadata: + uuid: + handle: "//" # e.g., "org/compute/eu-west-prod-1" + version: "1.0.0" + status: submitted # submitted โ†’ validating โ†’ active + owned_by: { display_name: "" } + + provider_type_id: # from Provider Type Registry + display_name: "" + description: "" + + # All providers declare these + sovereignty_declaration: + operating_jurisdictions: [] + data_residency_zones: [] + sub_processors: [] # third parties with data access + + accreditations: + - accreditation_uuid: # reference to registered accreditation + framework: + status: active + + # Endpoints (which endpoints are required varies by type โ€” see extensions) + health_endpoint: "https:///health" + + # Zero trust identity + certificate: + pem: + ca_chain: + rotation_interval: P90D +``` + +**Registration lifecycle states:** +``` +SUBMITTED โ†’ VALIDATING โ†’ PENDING_APPROVAL โ†’ ACTIVE + โ†˜ REJECTED +ACTIVE โ†’ SUSPENDED | DEREGISTERING โ†’ DEREGISTERED | FORCED_DEREGISTERED +``` + +--- + +## 3. Base Contract โ€” Health Check + +Every provider implements a health endpoint. DCM calls it on the declared interval. + +``` +GET {health_endpoint} + +Response 200: +{ + "status": "healthy | degraded | unhealthy", + "version": "", + "capabilities_available": [""], + "details": { } # provider-specific; DCM treats as opaque +} +``` + +**DCM response to health states:** +- `healthy` โ†’ normal operations; next poll scheduled +- `degraded` โ†’ reduced routing preference; platform admin notified (medium urgency) +- `unhealthy` / no response โ†’ after `failure_threshold`: status โ†’ DEGRADED; new requests not routed +- After 2ร— `failure_threshold`: status โ†’ UNAVAILABLE; drift detection triggered on all hosted entities + +--- + +## 4. Base Contract โ€” Governance Matrix Enforcement + +Every interaction with every provider is evaluated against the Governance Matrix before data crosses the boundary. This is not optional and not configurable per provider โ€” it is a base contract requirement. + +``` +Outbound interaction (DCM โ†’ Provider): + 1. Classify all fields in the payload by data_classification + 2. Resolve provider's active accreditations + 3. Evaluate Governance Matrix: permitted | strip_field | deny | redact + 4. Apply field permissions + 5. Audit record written (regardless of outcome) + 6. If DENY: interaction blocked; entity enters PENDING_REVIEW if appropriate + +Inbound interaction (Provider โ†’ DCM): + 1. Authenticate provider identity (mTLS) + 2. Verify credential scope matches the operation + 3. Accept payload; apply data_classification tags + 4. Store in appropriate store per data_classification +``` + +--- + +## 5. Base Contract โ€” Zero Trust + +All provider interactions operate under the active zero trust posture. Minimum requirement for all providers at all profiles: + +- Mutual TLS authentication on every call (both sides present certificates) +- Scoped, short-lived interaction credentials (not long-lived API keys) +- Every call authenticated; no implicit trust from network position or prior calls +- Certificate rotation on declared interval + +Higher profiles add: certificate pinning, per-message signing, hardware attestation. + +--- + +## 6. Base Contract โ€” Provider Lifecycle Events + +Providers must report state changes via lifecycle events. This is a base contract obligation โ€” not optional: + +```json +POST {dcm_lifecycle_endpoint} +{ + "event_uuid": "", + "event_type": "", + "provider_uuid": "", + "affected_entity_uuids": [""], + "event_timestamp": "", + "severity": "INFO | WARNING | CRITICAL" +} +``` + +--- + +## 7. Capability Extensions โ€” All Eleven Types + +### 7.1 Service Provider + +**What it does:** Realizes infrastructure resources. Receives assembled payloads, provisions the resource, returns realized state. + +**Additional endpoints:** +``` +POST {dispatch_endpoint} # receive and execute dispatch payload +POST {cancel_endpoint} # receive cancellation request (if supported) +POST {discover_endpoint} # receive discovery request; return discovered state +``` + +**Capability declaration extension:** +```yaml +service_provider_capabilities: + resource_types: + - fqn: Compute.VirtualMachine + spec_version: "2.1.0" + catalog_item_uuid: + cancellation: + supports_cancellation: true + cancellation_supported_during: [DISPATCHED, PROVISIONING] + discovery: + supports_discovery: true + discovery_method: api_query | passive_event | hybrid + naturalization: + target_format: openstack_nova | vmware_vsphere | custom + cost_metadata: + opex_per_unit_per_hour: 0.28 + currency: USD +``` + +**Data direction:** DCM sends assembled Requested State โ†’ Provider naturalizes โ†’ executes โ†’ denaturalizes โ†’ returns Realized State. DCM writes Realized State to Snapshot Store. + +--- + +### 7.2 Information Provider + +**What it does:** Serves authoritative external data to enrich DCM's understanding of resources and business context. + +**Additional endpoints:** +``` +POST {query_endpoint} # receive query; return data in DCM unified format +POST {write_back_endpoint} # optional; receive DCM updates to push to source system +``` + +**Capability declaration extension:** +```yaml +information_provider_capabilities: + data_domains: + - domain: business_data + data_types: [business_unit, cost_center, product_owner] + authority_level: primary | secondary | supplementary + query_capacity: + max_queries_per_second: 100 + confidence_model: + data_freshness_sla: PT1H + write_back_supported: false +``` + +**Data direction:** DCM sends lookup query โ†’ Provider returns data in DCM format โ†’ DCM enriches entity fields. + +--- + +### 7.3 Storage Provider + +**What it does:** Persists DCM state data. Implements one or more store contracts (GitOps, write-once snapshot, event stream, search index, audit). + +**Additional endpoints:** +``` +POST {write_endpoint} # receive and persist data +GET {read_endpoint} # return stored data +POST {query_endpoint} # execute indexed query (search index sub-type) +GET {health_endpoint} # includes store-specific metrics +``` + +**Capability declaration extension:** +```yaml +storage_provider_capabilities: + store_types: + - gitops # Intent and Requested stores + - write_once_snapshot # Realized store + - event_stream # Discovered store + - search_index # Query projection + - audit # Audit store + consistency: strong | eventual | bounded_staleness + geo_replicated: true + encryption_at_rest: AES-256 + hsm_backed: false +``` + +**Data direction:** Bidirectional. DCM writes state; DCM reads state. Provider never initiates. + +--- + +### 7.4 Meta Provider + +**What it does:** Composes multiple child providers to deliver a compound service as a single catalog item. + +**Capability declaration extension:** +```yaml +meta_provider_capabilities: + constituent_provider_types: [service_provider, information_provider] + composition_model: sequential | parallel | conditional + partial_delivery_supported: true + compensation_supported: true + resource_types_composed: + - fqn: ApplicationStack.WebApp + constituents: + - Compute.VirtualMachine + - Network.IPAddress + - DNS.Record +``` + +**Data direction:** DCM sends compound service payload โ†’ Meta Provider orchestrates constituent providers โ†’ aggregates realized states โ†’ returns compound realized state. + +--- + +### 7.5 Policy Provider + +**What it does:** Evaluates policies externally. Receives a DCM payload, evaluates Rego or custom logic, returns a typed policy decision. + +**Additional endpoints:** +``` +POST {evaluate_endpoint} # receive payload; return policy decision +POST {test_endpoint} # receive test case; return evaluation result (shadow mode) +``` + +**Capability declaration extension:** +```yaml +policy_provider_capabilities: + mode: 1 | 2 | 3 | 4 + policy_types: [gatekeeper, validation, transformation, recovery] + framework: opa | cedar | custom + shadow_mode_supported: true + endpoint_sovereignty_zone: # required for Mode 4 +``` + +**Data direction:** DCM sends payload + active policy bundle โ†’ Provider evaluates โ†’ returns typed decision (allow/deny, mutations, action). + +--- + +### 7.6 Credential Provider + +**What it does:** Issues, stores, rotates, and revokes credentials and secrets. + +**Additional endpoints:** +``` +POST {issue_endpoint} # request a credential; return scoped credential +POST {rotate_endpoint} # rotate an existing credential +DELETE {revoke_endpoint} # revoke a credential +``` + +**Capability declaration extension:** +```yaml +credential_provider_capabilities: + credential_types: [api_key, x509_certificate, service_account_token] + dynamic_secrets: true # generate on demand; expire after use + hsm_backed: false + fips_140_2_level: 1 | 2 | 3 +``` + +**Data direction:** DCM requests credential โ†’ Provider issues scoped credential โ†’ DCM includes in provider dispatch. + +--- + +### 7.7 Auth Provider + +**What it does:** Authenticates actor identities and resolves their roles and group memberships. + +**Additional endpoints:** +``` +POST {authenticate_endpoint} # receive credentials; return auth token + claims +POST {authorize_endpoint} # receive token + operation; return allow/deny +GET {identity_endpoint} # return actor claims for a token +``` + +**Capability declaration extension:** +```yaml +auth_provider_capabilities: + authentication_modes: [oidc, ldap, saml, mtls, hardware_token] + mfa_methods: [totp, push_notification, hardware_token] + rbac_model: flat | hierarchical | abac + step_up_supported: true + token_lifetime: + default: PT1H + max: PT8H +``` + +**Data direction:** Consumer sends credentials โ†’ Auth Provider validates โ†’ returns token + claims โ†’ DCM extracts actor identity. + +--- + +### 7.8 Notification Provider + +**What it does:** Receives unified notification envelopes from DCM and delivers them via configured channels. + +**Additional endpoints:** +``` +POST {delivery_endpoint} # receive notification envelope; deliver to channel +POST {delivery_status_endpoint} # callback: report delivery status to DCM +``` + +**Capability declaration extension:** +```yaml +notification_provider_capabilities: + delivery_channels: + - channel_type: slack | pagerduty | email | webhook | sms | servicenow + supports_urgency_routing: true + config_schema_ref: + delivery_guarantees: + at_least_once: true + idempotency_key: notification_uuid + max_latency_seconds: 30 + sovereignty_aware_delivery: true +``` + +**Data direction:** DCM sends notification envelope โ†’ Provider translates to channel format โ†’ delivers โ†’ reports status. + +--- + +### 7.9 Message Bus Provider + +**What it does:** Provides persistent, high-throughput asynchronous event streaming between DCM components and external systems. + +**Capability declaration extension:** +```yaml +message_bus_capabilities: + protocols: [kafka, amqp, mqtt] + persistence: true + durability: at_least_once | exactly_once + external_endpoints: false # true only if messages leave sovereignty boundary + topics: + - name: dcm.events + retention: P7D +``` + +**Data direction:** Bidirectional publish/subscribe. DCM publishes events; components and external systems subscribe. + +--- + +### 7.10 Registry Provider + +**What it does:** Serves the Resource Type Registry โ€” the authoritative catalog of resource types available to DCM deployments. + +**Additional endpoints:** +``` +GET {registry_endpoint} # serve registry entries (full or incremental) +GET {bundle_endpoint} # serve signed registry bundle (air-gapped mode) +``` + +**Capability declaration extension:** +```yaml +registry_provider_capabilities: + serves_tiers: [core, verified_community, organization] + incremental_sync: true + signed_bundles: true # for air-gapped deployments + bundle_signing_key_ref: +``` + +**Data direction:** DCM pulls registry entries โ†’ Provider returns signed bundle or live entries. + +--- + +### 7.11 Peer DCM (Federation) + +**What it does:** Another DCM instance participating in federation. Treated as a typed Provider with a federation tunnel as the communication channel. + +**Capability declaration extension:** +```yaml +peer_dcm_capabilities: + dcm_version: "1.0.0" + tunnel_type: peer | parent_child | hub_spoke + deployment_accreditations: [] + inbound_authorization: # what this peer may request from local DCM + - operation: catalog_query + resource_types: [Compute.VirtualMachine] + outbound_authorization: # what local DCM may request from this peer + - operation: placement_query + resource_types: [Compute.VirtualMachine] + data_boundary: + max_classification: restricted + trust_posture: verified | vouched | provisional +``` + +**Data direction:** Bidirectional within declared authorization scope. Federation tunnel with mTLS, certificate pinning, per-message signing. + +--- + +## 8. Provider Type Registry + +The Provider Type Registry is the authoritative list of provider types that a DCM deployment accepts registrations for. It follows the three-tier registry model (Core / Verified Community / Organization). + +```yaml +provider_type_registry_entry: + provider_type_id: service_provider + tier: core + default_approval_method: human_review # auto | human_review | dual_approval | committee + enabled_in_profiles: [minimal, dev, standard, prod, fsi, sovereign] + capability_extension_schema_ref: +``` + +Profile-governed approval methods override provider type defaults. See [Registration Specification](../specifications/dcm-registration-spec.md) Section 3 for the complete approval method resolution model. + +--- + +## 9. Related Policies + +| Policy | Rule | +|--------|------| +| `PRV-001` | All providers implement the base contract. No provider is exempt from registration, health check, sovereignty declaration, governance matrix enforcement, or zero trust authentication. | +| `PRV-002` | Governance Matrix evaluation occurs before every provider interaction. It is not configurable per provider and cannot be bypassed. | +| `PRV-003` | Provider capability declarations are verified at registration. Capabilities not declared at registration cannot be invoked after activation. | +| `PRV-004` | Peer DCM instances are treated as typed providers. Federation is the Provider abstraction applied across DCM instances โ€” not a separate abstraction. | +| `PRV-005` | Adding a new provider type requires implementing the base contract and defining a capability extension. No changes to DCM core are required. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/registry-governance.md b/content/docs/architecture/data-model/registry-governance.md index 8c1e1d9..9f98eb7 100644 --- a/content/docs/architecture/data-model/registry-governance.md +++ b/content/docs/architecture/data-model/registry-governance.md @@ -1,7 +1,7 @@ --- title: "Registry Governance" type: docs -weight: 19 +weight: 20 --- > **โš ๏ธ Active Development Notice** @@ -13,6 +13,19 @@ weight: 19 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Auth Providers](19-auth-providers.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + PROVIDER** +> +> Data: registry artifacts. Provider: Registry Provider extension + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md index 00631e5..7f8d6e9 100644 --- a/content/docs/architecture/data-model/resource-grouping.md +++ b/content/docs/architecture/data-model/resource-grouping.md @@ -16,6 +16,19 @@ weight: 8 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” DCMGroup typed extensions (Tenant, Resource Group, Cross-Tenant Auth) + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md index de638d1..c9c5261 100644 --- a/content/docs/architecture/data-model/resource-service-entities.md +++ b/content/docs/architecture/data-model/resource-service-entities.md @@ -14,6 +14,19 @@ weight: 6 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Operational Models](24-operational-models.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA + PROVIDER** +> +> Data: entity lifecycle. Provider: lifecycle events and update notifications + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/resource-type-hierarchy.md b/content/docs/architecture/data-model/resource-type-hierarchy.md index 5532bd5..23764b2 100644 --- a/content/docs/architecture/data-model/resource-type-hierarchy.md +++ b/content/docs/architecture/data-model/resource-type-hierarchy.md @@ -14,6 +14,19 @@ weight: 5 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” Resource Type Specifications and Provider Catalog Items + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/service-dependencies.md b/content/docs/architecture/data-model/service-dependencies.md index b9e454c..ffa35f1 100644 --- a/content/docs/architecture/data-model/service-dependencies.md +++ b/content/docs/architecture/data-model/service-dependencies.md @@ -14,6 +14,19 @@ weight: 7 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Entity Relationships](09-entity-relationships.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” dependency graph as embedded data structure + + + > **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](09-entity-relationships.md). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document. --- diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md index 6114bbf..d93ac43 100644 --- a/content/docs/architecture/data-model/storage-providers.md +++ b/content/docs/architecture/data-model/storage-providers.md @@ -1,7 +1,7 @@ --- title: "Storage Providers" type: docs -weight: 10 +weight: 11 --- > **โš ๏ธ Active Development Notice** @@ -14,6 +14,19 @@ weight: 10 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Four States](02-four-states.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Information Providers](10-information-providers.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: PROVIDER** +> +> The Provider abstraction โ€” Storage Provider capability extension + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/universal-audit.md b/content/docs/architecture/data-model/universal-audit.md index 97da6b7..91a1fc6 100644 --- a/content/docs/architecture/data-model/universal-audit.md +++ b/content/docs/architecture/data-model/universal-audit.md @@ -1,7 +1,7 @@ --- title: "Universal Audit Model" type: docs -weight: 15 +weight: 16 --- > **โš ๏ธ Active Development Notice** @@ -13,6 +13,19 @@ weight: 15 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Storage Providers](11-storage-providers.md) | [Universal Groups](15-universal-groups.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” Audit Record structure and tamper-evident chain + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md index a5de9fb..ad09885 100644 --- a/content/docs/architecture/data-model/universal-groups.md +++ b/content/docs/architecture/data-model/universal-groups.md @@ -13,6 +13,19 @@ weight: 15 **Document Status:** ๐Ÿ”„ In Progress **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Grouping](08-resource-grouping.md) | [Entity Relationships](09-entity-relationships.md) | [Policy Organization](14-policy-profiles.md) +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” DCMGroup as universal grouping artifact + + + --- ## 1. Purpose diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md index 7559ce3..2a830c4 100644 --- a/content/docs/architecture/data-model/webhooks-messaging.md +++ b/content/docs/architecture/data-model/webhooks-messaging.md @@ -12,6 +12,19 @@ weight: 18 **Document Status:** ๐Ÿ”„ In Progress **Related Documents (updated):** [Notification Model](23-notification-model.md) | [Entity Relationships](09-entity-relationships.md) + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> +> **This document maps to: PROVIDER** +> +> The Provider abstraction โ€” Message Bus and webhook delivery channels + + **Related Documents:** [Universal Audit Model](16-universal-audit.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Authentication and Authorization](19-auth-providers.md) | [Policy Organization](14-policy-profiles.md) --- From 09599c597a46a10d701a2b9783b33fbcef2d484d Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 10:53:49 -0500 Subject: [PATCH 28/49] Rebase concepts back to three fundamental items, data, policy, providers, fix ai prompt to remove confusion on orchestration, consolidate orchestration into policies, give examples. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 82 +- .../data-model/control-plane-components.md | 27 + .../architecture/data-model/foundations.md | 10 +- .../data-model/policy-contract.md | 25 + .../architecture/specifications/examples.md | 877 ++++++++++++++++++ .../specifications/opa-integration-spec.md | 182 ++++ 6 files changed, 1190 insertions(+), 13 deletions(-) create mode 100644 content/docs/architecture/specifications/examples.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 50711e4..214feb0 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -477,8 +477,9 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t |-------|-------------|---------| | **Allocation** | Provider retains internal ownership. Consumer owns the Entity (the allocation). Provider has reclaim rights on decommission. | VM, Container, IP Address | | **Cost Analysis Information Provider** | Specialized Information Provider supplying cost estimates, placement cost signals, cost actuals, and budget alerts; DCM provides input data; provider performs calculations | -| **Orchestrator** | DCM control plane component sequencing multi-step workflows; conducts request lifecycle pipeline; executes named workflow artifacts | -| **Workflow (DCM)** | First-class versioned GitOps artifact defining named sequence of operations; included in Profiles; triggered manually/scheduled/event/policy | +| **Orchestration Flow Policy** | Named workflow artifact: Orchestration Flow Policy with `ordered: true`; declares explicit step sequence using payload type vocabulary; first-class Data artifact; versioned, GitOps-managed, profile-bound | +| **Request Orchestrator** | Runtime event bus; routes lifecycle events to Policy Engine; has no pipeline logic; both named workflows and dynamic policies are evaluated through it | +| **orchestration (DCM)** | Two-level composable model: Level 1 = named Orchestration Flow Policies (explicit sequence); Level 2 = dynamic policies (conditional, inline); both evaluated by Policy Engine; adding a step = adding to a workflow Policy; adding conditional behavior = writing a dynamic policy | | **Provider Catalog Item** | What a specific Service Provider offers consumers: specific resource allocation or process with cost, availability, SLAs; linked to Resource Type Specification version | | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | @@ -2578,8 +2579,9 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | Former Term | Replacement | Meaning | |-------------|-------------|---------| | Shore | **Cost Analysis Information Provider** | Specialized Information Provider supplying cost estimates, placement cost signals, cost actuals, and budget alerts; DCM provides input data; provider performs calculations | -| **Orchestrator** | DCM control plane component sequencing multi-step workflows; conducts request lifecycle pipeline; executes named workflow artifacts | -| **Workflow (DCM)** | First-class versioned GitOps artifact defining named sequence of operations; included in Profiles; triggered manually/scheduled/event/policy | +| **Orchestration Flow Policy** | Named workflow artifact: Orchestration Flow Policy with `ordered: true`; declares explicit step sequence using payload type vocabulary; first-class Data artifact; versioned, GitOps-managed, profile-bound | +| **Request Orchestrator** | Runtime event bus; routes lifecycle events to Policy Engine; has no pipeline logic; both named workflows and dynamic policies are evaluated through it | +| **orchestration (DCM)** | Two-level composable model: Level 1 = named Orchestration Flow Policies (explicit sequence); Level 2 = dynamic policies (conditional, inline); both evaluated by Policy Engine; adding a step = adding to a workflow Policy; adding conditional behavior = writing a dynamic policy | | **Provider Catalog Item** | What a specific Service Provider offers consumers: specific resource allocation or process with cost, availability, SLAs; linked to Resource Type Specification version | | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | @@ -3324,17 +3326,23 @@ Cost Analysis is an **Information Provider** โ€” not a built-in DCM component. D CMP-001, CMP-002. -### 50.2 The Orchestrator โ€” Workflow Engine (Group 3) +### 50.2 Orchestration โ€” Reconciled Model (Replaces Conflicting Earlier Statements) -The **Orchestrator** sequences and executes multi-step DCM operations. Primary use case: the request lifecycle pipeline. General use case: any named workflow artifact. +DCM orchestration operates at two levels that compose through the same Policy Engine and event bus: -**Workflows are first-class DCM artifacts** โ€” versioned, GitOps-managed, same lifecycle as all other artifacts. Workflows can be triggered: manually, scheduled (cron), event-triggered, or by policy output. Profiles include workflow bindings โ€” activating `fsi` profile automatically activates compliance, drift remediation, sovereignty verification, and audit verification workflows. +**Level 1 โ€” Named Workflow Artifacts (explicit, visible, auditable):** +An Orchestration Flow Policy with `concern_type: orchestration_flow` and `ordered: true` is a named workflow. It declares steps in explicit sequence using the closed payload type vocabulary as step identifiers. Named workflows are first-class Data artifacts โ€” versioned, GitOps-managed, profile-bound, same lifecycle as all other artifacts. The request lifecycle pipeline is a built-in system Orchestration Flow Policy that cannot be deactivated but can be extended. Workflows are triggered: by events on the Request Orchestrator, by schedule (via Discovery Scheduler pattern), manually via Admin API, or by output of another policy. -**Step types:** discovery_trigger, policy_evaluation, policy_evaluation_batch, notification_trigger, provider_dispatch, wait_for_event, wait_for_condition, report_generation, entity_state_transition, sub_workflow, parallel, human_approval, cost_analysis_query, data_transform. +**Level 2 โ€” Dynamic Policies (conditional, inline):** +GateKeeper, Transformation, Recovery, Governance Matrix, and Lifecycle Policies fire when their match conditions are satisfied โ€” within or alongside workflow steps. They are not declared in workflow artifacts; they evaluate whenever payload state matches their conditions. -**Request lifecycle pipeline is a built-in system workflow** โ€” cannot be deactivated; can be extended via Policy Groups. +**How they compose:** A named workflow step fires when its declared payload type event occurs. Dynamic policies also fire on the same event if their conditions match. Both are evaluated by the same Policy Engine. Both are triggered by events on the Request Orchestrator event bus. The workflow provides the explicit sequence skeleton; dynamic policies provide conditional behavior within it. -**Workflow execution records** have UUID, state machine, step results, and full audit trail. CMP-003, CMP-004, CMP-005. +**The "Orchestrator" term** in earlier sections refers to the combination of: Request Orchestrator (event bus) + Orchestration Flow Policy evaluation (named workflows) + Policy Engine (dynamic policy evaluation). There is no separate "Orchestrator" component โ€” the Request Orchestrator is the event bus, and workflows are Policies. + +**Adding an explicit pipeline step** = add a step to an Orchestration Flow Policy artifact. +**Adding conditional behavior** = write a GateKeeper, Transformation, or Recovery policy. +**Both are Data artifacts evaluated by the Policy Engine.** ### 50.3 Ingress API vs Consumer API (Group 5 fix) @@ -3551,6 +3559,55 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc --- +## SECTION 58 โ€” EXAMPLES AND USE CASES (dcm-examples.md) + +### Orchestration Examples (8 scenarios) + +**1.1 Basic request lifecycle** โ€” submit โ†’ layers_assembled (GateKeeper + Transformation fire) โ†’ placement (6-step) โ†’ dispatch โ†’ realized. Shows named workflow + dynamic policies composing on same events. + +**1.2 Human approval gate** โ€” GateKeeper with `requires_approval: true` flag inserts AWAITING_APPROVAL step without modifying named workflow. Manager approves via API โ†’ pipeline resumes. + +**1.3 Policy-gated hard block** โ€” GateKeeper denies unsupported OS. Consumer receives clear error with policy_uuid and suggestion. No requires_approval flag โ†’ terminal FAILED. + +**1.4 Compound service (Meta Provider)** โ€” VM + IP + DNS + LoadBalancer. Dependency-ordered execution (parallel where no deps). DNS fails (partial delivery) โ†’ DEGRADED state. Recovery: NOTIFY_AND_WAIT. Consumer chooses: accept degraded or trigger DNS retry. + +**1.5 Drift detection + remediation** โ€” Discovery finds memory_gb changed (unsanctioned). Drift: significant + unsanctioned โ†’ critical. Policy: ESCALATE. Consumer submits REVERT โ†’ new request cycle โ†’ next discovery clean. + +**1.6 Dispatch timeout + late response** โ€” Provider silent for PT30M โ†’ TIMEOUT_PENDING โ†’ Recovery: NOTIFY_AND_WAIT (prod profile). Provider responds at T+45M โ†’ LATE_RESPONSE_RECEIVED. Consumer chooses DISCARD_AND_REQUEUE. + +**1.7 Federation-routed request** โ€” Local providers at capacity. Placement queries Hub DCM (Peer DCM provider). Hub routes to Regional DCM B. Governance Matrix checked at each hop. Realized State flows back chain. entity_uuid preserved. + +**1.8 Brownfield ingestion** โ€” Discovery finds unmanaged VM. Orchestration Flow Policy: discover โ†’ INGEST โ†’ ENRICH (CMDB query) โ†’ await operator โ†’ PROMOTE to tenant. Drift detection activated post-promotion. + +### Provider Examples (4 scenarios) + +**2.1 Service Provider dispatch cycle** โ€” Full payload showing DCM unified format โ†’ naturalize to OpenStack Nova โ†’ execute โ†’ denaturalize back. Shows provenance on injected fields (monitoring_endpoint from policy). + +**2.2 Information Provider enrichment** โ€” CMDB query during layer assembly. Response with confidence descriptor. Fields injected with source_type: information_provider and source_uuid. + +**2.3 Policy Provider Mode 3 (OPA sidecar)** โ€” Exact OPA HTTP API call format, input document structure, response parsing. + +**2.4 Notification Provider delivery** โ€” VLAN decommission event. Audience: owner (NetworkOps) + 2 stakeholders (required stakes) + 1 observer (optional stake). Per-actor envelopes with stakeholder_reason field. Slack message format. + +### Consumer API Examples (2 scenarios) + +**3.1 Complete request lifecycle** โ€” catalog browse โ†’ describe (see constraints) โ†’ submit โ†’ poll status sequence โ†’ get realized resource with confidence scores. + +**3.2 Provider update approval** โ€” Provider submits auto-scale notification โ†’ REQUIRES_CONSUMER_APPROVAL โ†’ consumer reviews pending notifications โ†’ approve โ†’ new Realized State. + +### Admin API Examples (2 scenarios) + +**4.1 Provider registration review** โ€” List pending registrations (with validation results) โ†’ approve with review notes. + +**4.2 Orphan resolution** โ€” List orphan candidates โ†’ investigate โ†’ adopt_into_dcm โ†’ entity promoted to full lifecycle. + +### Registration Flow Example (1 scenario) + +**5.1 Complete provider onboarding** โ€” Admin issues registration token โ†’ provider submits registration payload (mTLS + token) โ†’ 8 automated validation checks shown โ†’ PENDING_APPROVAL โ†’ admin reviews โ†’ ACTIVE. Full capability declaration structure for Service Provider. + +--- + + ## SECTION 54 โ€” TERMINOLOGY GLOSSARY | Term | Definition | @@ -3615,8 +3672,9 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc | **DCMGroup** | Universal group entity โ€” all grouping constructs in DCM expressed as DCMGroup with group_class | | **group_class** | Determines system behavior of a DCMGroup โ€” closed built-in set: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, composite, federation | | **Cost Analysis Information Provider** | Specialized Information Provider supplying cost estimates, placement cost signals, cost actuals, and budget alerts; DCM provides input data; provider performs calculations | -| **Orchestrator** | DCM control plane component sequencing multi-step workflows; conducts request lifecycle pipeline; executes named workflow artifacts | -| **Workflow (DCM)** | First-class versioned GitOps artifact defining named sequence of operations; included in Profiles; triggered manually/scheduled/event/policy | +| **Orchestration Flow Policy** | Named workflow artifact: Orchestration Flow Policy with `ordered: true`; declares explicit step sequence using payload type vocabulary; first-class Data artifact; versioned, GitOps-managed, profile-bound | +| **Request Orchestrator** | Runtime event bus; routes lifecycle events to Policy Engine; has no pipeline logic; both named workflows and dynamic policies are evaluated through it | +| **orchestration (DCM)** | Two-level composable model: Level 1 = named Orchestration Flow Policies (explicit sequence); Level 2 = dynamic policies (conditional, inline); both evaluated by Policy Engine; adding a step = adding to a workflow Policy; adding conditional behavior = writing a dynamic policy | | **Provider Catalog Item** | What a specific Service Provider offers consumers: specific resource allocation or process with cost, availability, SLAs; linked to Resource Type Specification version | | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index d47a08d..661dea3 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -151,6 +151,33 @@ static_flow_policy_group: A static flow is a Policy Group with `concern_type: orchestration_flow` and `ordered: true`. The Request Orchestrator respects the declared order. Static flows integrate with dynamic policies โ€” a dynamic policy can fire alongside the static flow steps. +### 2.5a Named Workflows vs Dynamic Policies โ€” How They Compose + +The Request Orchestrator does not distinguish between named workflows and dynamic policies โ€” both arrive as events and are routed to the Policy Engine. The distinction is in *how they are declared*: + +**Named Workflow Artifacts** (Orchestration Flow Policies with `ordered: true`) declare an explicit step sequence. An operator reading the workflow can see every step in order. Steps reference payload types from the closed vocabulary. Named workflows are the *explicit, visible skeleton* of a process. + +**Dynamic Policies** (GateKeeper, Transformation, Recovery) fire when their match conditions are satisfied, regardless of workflow position. They are not declared in the workflow artifact. They are the *conditional behavior* that fills in the skeleton. + +**Example โ€” request lifecycle:** +``` +Named workflow "system/workflows/request-lifecycle" declares: + Step 1: request.initiated โ†’ capture intent + Step 2: request.intent_captured โ†’ run layer assembly + Step 3: request.layers_assembled โ†’ run placement + Step 4: request.placement_complete โ†’ dispatch + +Dynamic policies also fire: + GateKeeper "vm-size-limits" fires on request.layers_assembled + if cpu_count > 32 โ†’ deny + Transformation "inject-monitoring" fires on request.layers_assembled + โ†’ adds monitoring_endpoint field + Recovery "notify-on-timeout" fires on recovery.timeout_fired + โ†’ NOTIFY_AND_WAIT action +``` + +The named workflow and the dynamic policies are independent artifacts. Adding a new GateKeeper does not modify the workflow. Modifying the workflow does not affect dynamic policies. They compose through the same Policy Engine evaluation on the same events. + ### 2.6 Request Orchestrator Responsibilities | Responsibility | Description | diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md index 9825471..0be3b39 100644 --- a/content/docs/architecture/data-model/foundations.md +++ b/content/docs/architecture/data-model/foundations.md @@ -186,7 +186,15 @@ Data fields are assembled from multiple contributing layers in a deterministic p **The unified Policy base contract** is defined in [B-policy-contract.md](B-policy-contract.md). All seven Policy types implement this base contract. What varies is the output schema. -**Policies as orchestration:** Static and dynamic workflows are both Policy. An Orchestration Flow Policy with `ordered: true` is a static workflow. A conditional GateKeeper or Transformation Policy is a dynamic workflow. Both are evaluated by the same Policy Engine. Adding a pipeline step = writing a Policy. Removing a step = deactivating a Policy. +**Policies as orchestration โ€” two levels that compose:** + +*Level 1 โ€” Named Workflow Artifacts (explicit, visible, auditable):* +An Orchestration Flow Policy with `concern_type: orchestration_flow` and `ordered: true` is a named workflow. It declares steps in explicit sequence. Named workflows are first-class Data artifacts โ€” versioned, GitOps-managed, profile-bound. Adding an explicit pipeline step = adding a step to a workflow Policy artifact. + +*Level 2 โ€” Dynamic Policies (conditional, inline):* +GateKeeper, Transformation, Recovery, and Governance Matrix Policies fire when their match conditions are satisfied โ€” within or alongside workflow steps, without being declared in the workflow. Adding conditional behavior = writing a dynamic policy. + +Both levels are evaluated by the same Policy Engine and triggered through the same Request Orchestrator event bus. They compose naturally: a named workflow provides the sequence skeleton; dynamic policies provide conditional behavior within it. **The Governance Matrix as Policy:** The Governance Matrix rules (doc 27) are typed Policies with the `boundary_control` output schema. They fire at every cross-boundary interaction. They follow the same match conditions, enforcement levels, and lifecycle as all other Policies. The governance matrix is not a separate system โ€” it is the Policy abstraction applied at interaction boundaries. diff --git a/content/docs/architecture/data-model/policy-contract.md b/content/docs/architecture/data-model/policy-contract.md index 2db5291..9529d5b 100644 --- a/content/docs/architecture/data-model/policy-contract.md +++ b/content/docs/architecture/data-model/policy-contract.md @@ -284,6 +284,15 @@ recovery_output: ## 12. Output Schema โ€” Orchestration Flow +**The two-level orchestration model:** + +Orchestration in DCM operates at two levels that compose through the same Policy Engine: + +- **Level 1 โ€” Named Workflow Artifacts:** Orchestration Flow Policies with `ordered: true` declare an explicit, visible, auditable sequence of steps. Each step references a payload type from the closed vocabulary. This is what operators see and reason about. Adding a step = adding to a workflow Policy. +- **Level 2 โ€” Dynamic Policies:** GateKeeper, Transformation, Recovery, and Governance Matrix Policies fire when their conditions match, within or alongside workflow steps, without being declared in the workflow. Adding conditional behavior = writing a dynamic policy. + +The Request Orchestrator (event bus) routes all payload type events through the Policy Engine. Both named workflow steps and dynamic policies evaluate against the same events. The workflow provides the skeleton; dynamic policies fill in conditional behavior. + **Fires on:** Pipeline payload type events. **Produces:** A flow directive governing step ordering. @@ -299,6 +308,22 @@ orchestration_flow_output: - [step_1_id, step_2_id] ``` +**Step vocabulary** โ€” steps reference payload types from the closed vocabulary, mapping to control plane operations: + +| Payload type | Maps to | +|-------------|---------| +| `request.initiated` | Start of request pipeline | +| `request.layers_assembled` | Layer assembly complete | +| `request.policies_evaluated` | All policies evaluated | +| `request.placement_complete` | Provider selected | +| `request.dispatched` | Sent to provider | +| `discovery.cycle_complete` | Discovery cycle done | +| `drift.detected` | Drift found | +| `recovery.timeout_fired` | Dispatch timeout | +| `provider_update.received` | Provider update notification | + +Custom steps extend this vocabulary by publishing new payload types. + **Policy Engine behavior:** When `ordered: true`, steps execute in declared sequence. When `ordered: false`, the Policy Engine executes steps in parallel where no data dependencies exist. Orchestration Flow policies compose with standard GateKeeper and Transformation policies โ€” both types evaluate in the same pipeline. --- diff --git a/content/docs/architecture/specifications/examples.md b/content/docs/architecture/specifications/examples.md new file mode 100644 index 0000000..a793b0a --- /dev/null +++ b/content/docs/architecture/specifications/examples.md @@ -0,0 +1,877 @@ +--- +title: "DCM Examples and Use Cases" +type: docs +weight: 0 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Reference Examples +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Provider Contract](../data-model/A-provider-contract.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) | [Registration](dcm-registration-spec.md) | [OPA Integration](dcm-opa-integration-spec.md) + +--- + +## Overview + +This document provides end-to-end worked examples for the most important DCM use cases. Each example shows the complete interaction โ€” payloads, state transitions, API calls, and Rego policies where applicable โ€” so implementors can trace exactly what happens at each step. + +Examples are organized by the three foundational abstractions: +- **Section 1** โ€” Orchestration examples (Policy) +- **Section 2** โ€” Provider interaction examples (Provider) +- **Section 3** โ€” API interaction examples (Consumer API, Admin API) +- **Section 4** โ€” Registration flow examples + +--- + +# Section 1 โ€” Orchestration Examples + +## 1.1 Basic Request Lifecycle (End-to-End) + +The complete path for a consumer requesting a VM. Shows all payload type events, which policies fire at each step, and the state transitions. + +### Setup: Active artifacts + +```yaml +# Named workflow (Level 1 orchestration) +Orchestration Flow Policy: system/workflows/request-lifecycle + ordered: true + steps: [request.initiated, request.intent_captured, + request.layers_assembled, request.placement_complete, + request.dispatched] + +# Dynamic policies (Level 2 orchestration) +GateKeeper: org/gatekeeper/vm-size-limits (fires on request.layers_assembled) +Transformation: org/transformation/inject-monitoring (fires on request.layers_assembled) +GateKeeper: system/gatekeeper/sovereignty-check (fires on request.placement_complete) +``` + +### Step-by-step + +**Step 1 โ€” Consumer submits request:** +``` +POST /api/v1/requests +{ "catalog_item_uuid": "vm-standard-uuid", + "fields": { "cpu_count": 4, "memory_gb": 8, "os_family": "rhel" } } + +โ†’ Response 202: { "request_uuid": "req-001", "entity_uuid": "ent-001", + "status": "ACKNOWLEDGED" } +โ†’ Event published: { "type": "request.initiated", "entity_uuid": "ent-001", + "payload": { "fields": {...} } } +โ†’ Intent State written to Intent Store +``` + +**Step 2 โ€” Layer assembly:** +``` +Event: request.initiated +โ†’ Named workflow step 1 fires: capture-intent policy acknowledges +โ†’ New event: request.intent_captured + +Event: request.intent_captured +โ†’ Named workflow step 2 fires: assemble-layers policy runs +โ†’ Base layer applied: data_center = "EU-WEST-DC1" +โ†’ Org layer applied: monitoring_agent = "datadog-agent:7.42" +โ†’ Policy layer applied: backup_policy = "daily-30d-eu-west" +โ†’ New event: request.layers_assembled + payload now includes all merged fields with provenance +``` + +**Step 3 โ€” Dynamic policies fire on request.layers_assembled:** +``` +Event: request.layers_assembled +โ†’ [PARALLEL] All policies matching this payload type evaluate simultaneously: + + GateKeeper vm-size-limits evaluates: + input.payload.fields.cpu_count.value = 4 + 4 <= 32 โ†’ allow: true + + Transformation inject-monitoring evaluates: + monitoring_endpoint not in payload โ†’ mutation: + { field: "fields.monitoring_endpoint", + operation: "set", + value: "https://metrics.internal.prod.example.com" } + +โ†’ All GateKeepers: allow +โ†’ Transformations applied to payload +โ†’ New event: request.policies_evaluated +``` + +**Step 4 โ€” Placement:** +``` +Event: request.policies_evaluated +โ†’ Named workflow step 3: run-placement policy + +โ†’ Placement Engine: + Step 1: Sovereignty filter โ€” EU-WEST-DC1 requirement โ†’ 3 providers eligible + Step 2: Accreditation filter โ€” no PHI in payload โ†’ all 3 pass + Step 3: Capability filter โ€” all support VirtualMachine โ†’ all 3 pass + Step 4: Reserve query โ†’ parallel queries to EU-WEST-Prod-1,2,3 + EU-WEST-Prod-1: confirmed, utilization 42%, cost $0.32/hr + EU-WEST-Prod-2: confirmed, utilization 61%, cost $0.32/hr + EU-WEST-Prod-3: insufficient capacity + Step 5: Tie-break โ†’ Step 4 cost equal โ†’ Step 5 least loaded โ†’ Prod-1 wins + Step 6: Confirm Prod-1; release holds on Prod-2 + +โ†’ Requested State written to Requested Store + (requested.yaml + assembly-provenance.yaml + placement.yaml + dependencies.yaml) +โ†’ New event: request.placement_complete +``` + +**Step 5 โ€” Dispatch and realization:** +``` +Event: request.placement_complete +โ†’ Governance Matrix evaluated: payload data_classification = internal/public โ†’ ALLOW +โ†’ Named workflow step 4: dispatch policy +โ†’ Provider EU-WEST-Prod-1 receives dispatch payload +โ†’ Provider naturalizes to OpenStack Nova format +โ†’ OpenStack provisions VM +โ†’ Provider denaturalizes result โ†’ DCM unified format +โ†’ Realized State written (with provider_entity_id: "vm-0a1b2c3d") +โ†’ Status callback: COMPLETED + +Consumer polls: GET /api/v1/requests/req-001/status +โ†’ { "status": "COMPLETED", "entity_uuid": "ent-001" } +``` + +--- + +## 1.2 Human Approval Gate (Conditional Step Insertion) + +A production VM request that requires manager approval before dispatch. Shows how a GateKeeper policy inserts a waiting step without modifying the named workflow. + +### Setup: Additional active policy + +```rego +# GateKeeper fires on request.policies_evaluated for prod VMs over $100/month +package dcm.gatekeeper.prod_vm_approval_gate + +deny contains reason if { + input.payload.type == "request.policies_evaluated" + input.deployment.deployment_posture == "prod" + input.payload.cost_estimate.per_month > 100 + not input.payload.approvals["manager_approval"] + reason := "Production VMs over $100/month require manager approval" +} + +# Signal that approval is the resolution path (not a permanent reject) +requires_approval := true if count(deny) > 0 +approval_type := "manager_approval" if count(deny) > 0 +``` + +### Step-by-step + +``` +After Step 3 (dynamic policies evaluate): +โ†’ GateKeeper prod_vm_approval_gate fires +โ†’ deny: ["Production VMs over $100/month require manager approval"] +โ†’ requires_approval: true, approval_type: "manager_approval" + +โ†’ Policy Engine sees GateKeeper deny WITH requires_approval flag +โ†’ Entity enters AWAITING_APPROVAL state (not FAILED) +โ†’ Notification dispatched: + audience: manager (from actor's group membership via relationship graph) + event_type: request.requires_approval + action_url: /api/v1/requests/req-001/approve + action_deadline: PT24H + +Manager approves: +POST /api/v1/requests/req-001/approve +{ "approval_type": "manager_approval", "approver_uuid": "mgr-001" } + +โ†’ payload.approvals["manager_approval"] = { approved: true, by: "mgr-001" } +โ†’ GateKeeper re-evaluates: approval present โ†’ allow +โ†’ Pipeline resumes from request.policies_evaluated +โ†’ Placement โ†’ Dispatch โ†’ Realization (same as 1.1 Steps 4-5) +``` + +--- + +## 1.3 Policy-Gated Request โ€” Hard Block with Clear Error + +Shows a request blocked by a hard GateKeeper with a consumer-visible error message. + +```rego +package dcm.gatekeeper.approved_os_images + +deny contains reason if { + input.payload.type == "request.layers_assembled" + not input.payload.fields.os_family.value in {"rhel", "ubuntu-lts", "coreos"} + reason := sprintf( + "OS '%s' is not in the approved image list. Approved: rhel, ubuntu-lts, coreos", + [input.payload.fields.os_family.value] + ) +} +``` + +``` +Consumer submits: { "os_family": "windows-server" } + +โ†’ request.layers_assembled fires +โ†’ GateKeeper approved_os_images: deny +โ†’ Entity โ†’ FAILED (no requires_approval flag โ†’ hard block) + +Consumer response: +{ "status": "FAILED", + "failure_reason": "OS 'windows-server' is not in the approved image list.", + "retry_eligible": true, + "policy_uuid": "gatekeeper-approved-os-uuid", + "suggestion": "Resubmit with os_family: rhel, ubuntu-lts, or coreos" } +``` + +--- + +## 1.4 Compound Service โ€” Meta Provider with Dependency Ordering + +A web application stack provisioned as a single catalog item: VM + IP + DNS + LoadBalancer. + +### Named workflow for compound service + +```rego +package dcm.orchestration.webapp_stack + +steps := [ + {"step": 1, "payload_type": "request.initiated", + "policy_handle": "system/orchestration/capture-intent", "on_fail": "halt"}, + {"step": 2, "payload_type": "request.intent_captured", + "policy_handle": "system/orchestration/assemble-compound", "on_fail": "halt"}, + {"step": 3, "payload_type": "request.compound_assembled", + "policy_handle": "system/orchestration/resolve-dependencies", "on_fail": "halt"}, + {"step": 4, "payload_type": "request.dependencies_resolved", + "policy_handle": "system/orchestration/dispatch-constituents", "on_fail": "compensate"} +] + +ordered := true +``` + +### Execution + +``` +Meta Provider receives compound dispatch payload: + component.ip: { resource_type: Network.IPAddress, depends_on: [] } + component.vm: { resource_type: Compute.VirtualMachine, depends_on: [] } + component.dns: { resource_type: DNS.Record, depends_on: [ip, vm], required: partial } + component.lb: { resource_type: Network.LoadBalancer, depends_on: [vm, ip], required: partial } + +Dependency-ordered execution: + Round 1 (no dependencies): ip, vm โ†’ provisioned in parallel + ip โ†’ REALIZED: 10.1.45.23/32 + vm โ†’ REALIZED: vm-0a1b2c3d + + Round 2 (depend on ip+vm): dns, lb โ†’ provisioned in parallel + dns โ†’ FAILED (DNS service degraded) + lb โ†’ REALIZED: lb-7f8e9d + + Compound evaluation: + dns: required_for_delivery = partial โ†’ DEGRADED, not FAILED + lb: required_for_delivery = partial โ†’ REALIZED + +Compound entity state: DEGRADED (dns failed; vm+ip+lb realized) +Notification: owner notified "WebApp Stack provisioned in degraded state โ€” DNS unavailable" + +Recovery policy fires (PARTIAL_REALIZATION trigger): + profile=prod โ†’ NOTIFY_AND_WAIT + Consumer sees notification with options: accept degraded | trigger dns retry +``` + +--- + +## 1.5 Drift Detection and Automated Remediation + +Discovery finds VM memory has changed without a DCM request. Shows the full drift โ†’ policy โ†’ revert flow. + +``` +Scheduled discovery (PT15M interval): +โ†’ Provider queried for vm-0a1b2c3d +โ†’ Discovered: memory_gb = 16 +โ†’ Realized State: memory_gb = 8 +โ†’ No Requested State record explains the change + +Drift Reconciliation Component: + field: memory_gb + realized_value: 8, discovered_value: 16 + change_magnitude: 100% increase โ†’ "significant" (standard profile: 10-50% threshold) + field_criticality: medium (from Resource Type Spec) + unsanctioned: true โ†’ elevate one level โ†’ "critical" + +Drift record created: + overall_severity: critical + unsanctioned: true + +Policy Engine evaluates drift record: + Active drift response policy (standard profile, critical severity, unsanctioned): + action: ESCALATE โ†’ notify platform admin + SRE + owner + +Notifications dispatched: + Owner: "Critical unsanctioned change on vm-0a1b2c3d: memory_gb 8โ†’16" + Platform Admin: same (urgency: critical) + SRE on-call: same (via PagerDuty Notification Provider) + +If consumer submits: REVERT +โ†’ New request submitted from Realized State (memory_gb: 8) +โ†’ Full governance pipeline โ†’ new Requested State โ†’ dispatch โ†’ revert +โ†’ Next discovery: memory_gb = 8 โ†’ drift.resolved event +``` + +--- + +## 1.6 Recovery Flow โ€” Dispatch Timeout with NOTIFY_AND_WAIT + +Provider does not respond within PT30M. Profile is `prod` โ†’ `recovery-notify-and-wait`. + +``` +T+0: Request dispatched to EU-WEST-Prod-1 +T+30M: Dispatch timeout fires + Entity โ†’ TIMEOUT_PENDING + Recovery trigger: DISPATCH_TIMEOUT + +Recovery Policy (prod profile โ†’ recovery-notify-and-wait): + action: NOTIFY_AND_WAIT + deadline: PT4H + on_deadline_exceeded: ESCALATE + +Notifications dispatched: + Owner: "Request req-001 timed out. Choose how to proceed by T+4H." + action_url: /api/v1/resources/ent-001/recovery-decisions + +Consumer queries: +GET /api/v1/resources/ent-001/recovery-decisions +โ†’ { "trigger": "DISPATCH_TIMEOUT", + "deadline": "...", + "available_actions": [ + { "action": "DRIFT_RECONCILE", + "description": "Let discovery determine actual state" }, + { "action": "DISCARD_AND_REQUEUE", + "description": "Clean up and retry" } + ] } + +T+45M: Provider responds (late response) with realized payload + Entity in TIMEOUT_PENDING โ†’ LATE_RESPONSE_RECEIVED fires + +Recovery policy for LATE_RESPONSE_RECEIVED (prod โ†’ notify-and-wait): + action: NOTIFY_AND_WAIT (same โ€” human decides whether to accept late work) + notification updated: "Provider completed after timeout. Accept or discard?" + +Consumer POSTs: { "action": "DISCARD_AND_REQUEUE" } +โ†’ Best-effort cleanup sent to provider +โ†’ Entity โ†’ FAILED +โ†’ New request cycle created (same entity_uuid) +โ†’ Orphan detection triggered for EU-WEST-Prod-1 +``` + +--- + +## 1.7 Federation-Routed Request + +Consumer in Regional DCM A requests a resource that gets placed on a provider registered with Regional DCM B via Hub DCM. + +``` +Consumer โ†’ Regional DCM A: + POST /api/v1/requests { resource_type: Compute.VirtualMachine, ... } + +Regional DCM A Placement Engine: + Step 1: Sovereignty filter โ†’ local providers all at capacity + Step 2: Query Hub DCM (Peer DCM provider) for available regional capacity + โ†’ Hub responds: Regional DCM B has EU-WEST-Prod-2 with capacity + +Governance Matrix check (Regional DCM A โ†’ Hub DCM): + subject: dcm_peer (Regional DCM A) + data.classification: internal (assembled payload fields) + target: dcm_peer (Hub DCM), trust_posture: verified + โ†’ Decision: ALLOW (internal data, verified peer) + +Hub DCM routes to Regional DCM B: + Governance Matrix check (Hub โ†’ Regional DCM B): + same: ALLOW + Regional DCM B forwards to EU-WEST-Prod-2 + +Realized State flows back: + Provider โ†’ Regional DCM B โ†’ Hub DCM โ†’ Regional DCM A + Each hop: Governance Matrix evaluated + Final Realized State written to Regional DCM A's Realized Store + entity_uuid preserved throughout + provider_entity_id: "vm-eu-west-b-0012" +``` + +--- + +## 1.8 Brownfield Ingestion Workflow + +An existing VM discovered by a provider that DCM did not provision. + +```rego +# Orchestration Flow Policy for brownfield ingestion +package dcm.orchestration.brownfield_ingestion + +steps := [ + {"step": 1, "payload_type": "discovery.new_entity_found", + "policy_handle": "system/ingestion/create-transitional-record"}, + {"step": 2, "payload_type": "ingestion.transitional_created", + "policy_handle": "system/ingestion/enrich-from-information-providers"}, + {"step": 3, "payload_type": "ingestion.enriched", + "policy_handle": "system/ingestion/await-operator-promotion"}, + {"step": 4, "payload_type": "ingestion.promotion_approved", + "policy_handle": "system/ingestion/promote-to-tenant"} +] +ordered := true +``` + +``` +Discovery cycle finds vm-legacy-0001 (no matching Realized State UUID): + +Step 1: Event: discovery.new_entity_found +โ†’ INGEST: create Transitional entity in __transitional__ Tenant + entity_uuid assigned + lifecycle_state: INGESTION_PENDING + data_classification: internal (default) + +Step 2: Event: ingestion.transitional_created +โ†’ ENRICH: Information Providers queried: + CMDB (authority: primary): + business_unit: "Payments Platform" + cost_center: "PAYM-4421" + product_owner: "Jane Smith" + compliance_scope: "PCI-DSS" + HR System (authority: secondary): + team: "payments-platform-eng" + +Step 3: Event: ingestion.enriched +โ†’ Notification to Platform Admin: + "Brownfield entity discovered. Review and assign to Tenant." + action_url: /api/v1/admin/ingestion/ing-001/promote + +Step 4: Operator approves: +POST /api/v1/admin/ingestion/ing-001/promote +{ "target_tenant_uuid": "payments-tenant-uuid", + "compliance_overlay": "pci-dss" } + +โ†’ Entity moved from __transitional__ to payments-tenant +โ†’ Intent State created from discovered configuration +โ†’ drift detection activated +โ†’ lifecycle_state: OPERATIONAL +``` + +--- + +# Section 2 โ€” Provider Interaction Examples + +## 2.1 Service Provider โ€” Full Dispatch Cycle + +``` +DCM sends dispatch payload to Service Provider endpoint: + +POST https://provider.example.com/dispatch +Authorization: mTLS + scoped credential (scope: dispatch, entity: ent-001, ttl: PT15M) +Content-Type: application/json + +{ + "dispatch_uuid": "disp-001", + "entity_uuid": "ent-001", + "requested_state_uuid": "req-state-001", + "payload": { + "resource_type": "Compute.VirtualMachine", + "fields": { + "cpu_count": { "value": 4, "provenance": {...} }, + "memory_gb": { "value": 8, "provenance": {...} }, + "os_family": { "value": "rhel", "provenance": {...} }, + "monitoring_endpoint": { + "value": "https://metrics.internal.prod.example.com", + "provenance": { "origin": { "source_type": "policy", + "source_uuid": "transform-inject-monitoring" } } + } + } + } +} + +Provider naturalizes (DCM โ†’ OpenStack Nova): +{ + "server": { + "name": "ent-001", + "flavorRef": "m1.xlarge", # 4 vCPU, 8GB + "imageRef": "rhel-9.2-latest", + "metadata": { "dcm_entity_uuid": "ent-001", + "dcm_requested_state": "req-state-001", + "monitoring_endpoint": "https://metrics..." } + } +} + +OpenStack provisions โ†’ returns server object. + +Provider denaturalizes (OpenStack โ†’ DCM unified): +{ + "realized_state_uuid": "real-001", + "entity_uuid": "ent-001", + "corresponding_requested_state_uuid": "req-state-001", + "source_type": "initial_realization", + "fields": { + "cpu_count": { "value": 4, ... }, + "memory_gb": { "value": 8, ... }, + "provider_entity_id": { "value": "vm-0a1b2c3d" }, + "assigned_ip_address": { "value": "10.1.45.23" }, + "hypervisor_host": { "value": "compute-07.eu-west" } + } +} + +DCM receives โ†’ writes to Realized Store. +``` + +## 2.2 Information Provider โ€” Assembly Enrichment + +``` +During layer assembly Step 2 (layer resolution), DCM queries CMDB Information Provider: + +POST https://cmdb.corp.example.com/query +Authorization: mTLS +{ + "query_uuid": "qry-001", + "data_type": "business_data", + "lookup_key": { "type": "actor_uuid", "value": "actor-payments-001" } +} + +Response: +{ + "data": { + "business_unit": { "value": "Payments Platform", + "confidence": { "band": "very_high", "score": 97 }, + "authority_level": "primary" }, + "cost_center": { "value": "PAYM-4421", + "confidence": { "band": "very_high", "score": 97 } }, + "product_owner": { "value": "Jane Smith", + "confidence": { "band": "high", "score": 85 } } + }, + "data_freshness": "2026-03-15T08:00:00Z" +} + +DCM injects into assembled payload as a data layer: + business_unit.provenance.origin.source_type = "information_provider" + business_unit.provenance.origin.source_uuid = "cmdb-provider-uuid" +``` + +## 2.3 Policy Provider Mode 3 โ€” OPA Sidecar Evaluation + +``` +Assembly reaches Step 5 (pre-placement policy processing): + +DCM sends payload to OPA sidecar: +POST http://opa-sidecar:8181/v1/data/dcm/gatekeeper/vm_size_limits +{ + "input": { + "payload": { + "type": "request.layers_assembled", + "fields": { "cpu_count": { "value": 4 }, ... } + }, + "actor": { "uuid": "actor-001", "roles": ["developer"], + "tenant_uuid": "payments-uuid" }, + "deployment": { "deployment_posture": "prod", + "compliance_domains": ["hipaa"] }, + "entity": null, + "provider": null + } +} + +OPA response: +{ + "result": { + "allow": true, + "deny": [], + "field_locks": [], + "warnings": [] + } +} + +DCM Policy Engine reads result โ†’ allow โ†’ pipeline continues. +``` + +## 2.4 Notification Provider โ€” Relationship Graph Audience + +``` +Event: entity.decommissioning (VLAN-100 entering DECOMMISSIONING state) + +Notification Router: + 1. Load relationship graph for VLAN-100: + VM-A (AppTeam, attached_to, stake_strength: required) + VM-B (DevTeam, attached_to, stake_strength: required) + VM-C (OpsTeam, attached_to, stake_strength: optional) + + 2. Resolve audiences: + VLAN-100 owner (NetworkOps): audience_role = owner + VM-A owner (AppTeam admin): audience_role = stakeholder + stakeholder_reason: { via_entity: "VM-A", via_relationship: "attached_to" } + VM-B owner (DevTeam admin): audience_role = stakeholder + VM-C owner (OpsTeam admin): audience_role = observer (optional stake) + + 3. Per-actor notification envelopes generated (4 total) + +POST https://slack-notif.corp.example.com/deliver +{ + "notification_uuid": "notif-001", + "event_type": "entity.decommissioning", + "urgency": "high", + "entity": { "uuid": "vlan-100-uuid", "display_name": "VLAN-100" }, + "audience": { + "actor_uuid": "appteam-admin-uuid", + "audience_role": "stakeholder", + "stakeholder_reason": { + "via_entity_uuid": "vm-a-uuid", + "via_entity_display_name": "VM-A (payments-api-server-01)", + "via_relationship_type": "attached_to" + } + }, + "context": { "change_summary": "VLAN-100 decommission initiated" }, + "requires_action": false +} + +Slack provider delivers: + "#payments-platform: โš ๏ธ VLAN-100 is being decommissioned. + Your VM 'payments-api-server-01' is attached to it. + Action required: migrate VM network attachment before decommission completes." +``` + +--- + +# Section 3 โ€” Consumer API Examples + +## 3.1 Complete Request Lifecycle (API Perspective) + +``` +# 1. Browse catalog +GET /api/v1/catalog?category=Compute +X-DCM-Tenant: payments-tenant-uuid +Authorization: Bearer + +Response: { "catalog_items": [ + { "catalog_item_uuid": "vm-standard-uuid", + "resource_type": "Compute.VirtualMachine", + "display_name": "Standard Linux VM", + "estimated_cost": { "per_hour": 0.32, "currency": "USD" }, + "accreditations": [{ "framework": "hipaa", "status": "active" }] + } +] } + +# 2. Describe catalog item (see schema + constraints) +GET /api/v1/catalog/vm-standard-uuid + +Response includes: + "schema.fields[cpu_count].constraint": { "type": "range", "min": 1, "max": 32 } + "schema.fields[monitoring_agent].constraint.visibility": "hidden" # injected by policy + +# 3. Submit request +POST /api/v1/requests +{ "catalog_item_uuid": "vm-standard-uuid", + "fields": { "cpu_count": 4, "memory_gb": 8, "os_family": "rhel", + "name": "payments-api-server-01" } } + +Response 202: { "request_uuid": "req-001", "entity_uuid": "ent-001", + "status": "ACKNOWLEDGED", + "status_url": "/api/v1/requests/req-001/status" } + +# 4. Poll status (or use webhook) +GET /api/v1/requests/req-001/status + +Sequence of responses: + { "status": "ASSEMBLING" } # layer assembly running + { "status": "DISPATCHED" } # sent to provider + { "status": "PROVISIONING" } # provider executing + { "status": "COMPLETED", + "resource_url": "/api/v1/resources/ent-001" } + +# 5. Get realized resource +GET /api/v1/resources/ent-001 + +Response: +{ "entity_uuid": "ent-001", + "lifecycle_state": "OPERATIONAL", + "drift_status": "clean", + "fields": { + "cpu_count": { "value": 4, "confidence": { "band": "very_high" } }, + "assigned_ip_address": { "value": "10.1.45.23", + "confidence": { "band": "very_high" } } + }, + "estimated_cost_per_hour": 0.32 } +``` + +## 3.2 Provider Update Notification โ€” Consumer Approval Flow + +``` +# Provider submits auto-scale notification (memory doubled) +POST /api/v1/provider/entities/ent-001/update-notification +Authorization: mTLS (provider cert) +{ "provider_uuid": "eu-west-prod-1-uuid", + "notification_uuid": "notif-001", + "notification_type": "auto_scale", + "changed_fields": { + "memory_gb": { "previous_value": 8, "new_value": 16, + "change_reason": "Auto-scale at 85% utilization" } + } } + +โ†’ DCM evaluates: no pre-authorization policy for this tenant โ†’ REQUIRES_CONSUMER_APPROVAL +โ†’ Entity โ†’ PENDING_REVIEW +โ†’ Notification to owner: "Provider requests to update memory_gb: 8โ†’16. Approve?" + +# Consumer reviews pending notification +GET /api/v1/resources/ent-001/provider-notifications + +Response: { "notifications": [{ + "notification_uuid": "notif-001", + "notification_type": "auto_scale", + "status": "pending_approval", + "change_summary": "memory_gb: 8 โ†’ 16", + "change_reason": "Auto-scale at 85% utilization" +}] } + +# Consumer approves +POST /api/v1/resources/ent-001/provider-notifications/notif-001/approve +{ "decision": "approve", "reason": "Legitimate auto-scale event" } + +Response 202: { "decision": "approve", "realized_state_uuid": "real-002" } + +โ†’ New Requested State created (source_type: provider_update) +โ†’ New Realized State snapshot written (memory_gb: 16) +โ†’ Audit: PROVIDER_UPDATE_APPLIED +``` + +--- + +# Section 4 โ€” Admin API Examples + +## 4.1 Review and Approve Provider Registration + +``` +# New provider submitted registration +# Platform admin receives notification (urgency: medium) +# "New provider registration pending review: eu-west-prod-1" + +# List pending registrations +GET /api/v1/admin/registrations/pending +Authorization: Bearer + +Response: { "registrations": [{ + "registration_uuid": "reg-001", + "provider_type_id": "service_provider", + "handle": "org/compute/eu-west-prod-1", + "submitted_at": "2026-03-15T09:00:00Z", + "validation_status": "passed", # all 8 automated checks passed + "sovereignty_zone": "eu-west-sovereign", + "accreditations": [{ "framework": "hipaa", "type": "baa" }], + "health_check_status": "healthy", + "governance_matrix_pre_check": "ALLOW" +}] } + +# Admin reviews and approves +POST /api/v1/admin/registrations/reg-001/approve +{ "review_notes": "Certificate verified against corp CA. BAA reviewed and valid." } + +Response: { "registration_uuid": "reg-001", "status": "ACTIVE" } + +โ†’ Provider enters active registry +โ†’ Governance Matrix re-evaluated with this provider active +โ†’ Notification to provider operator: "Registration approved. Provider UUID: eu-west-prod-1-uuid" +``` + +## 4.2 Resolve Orphan Candidate + +``` +# Discovery found vm-legacy-0001 after a timeout-cancelled request + +GET /api/v1/admin/orphans +Response: { "orphan_candidates": [{ + "orphan_candidate_uuid": "orp-001", + "provider_uuid": "eu-west-prod-1-uuid", + "provider_entity_id": "vm-legacy-0001", + "suspected_request_uuid": "req-failed-001", + "resource_type": "Compute.VirtualMachine", + "discovered_at": "2026-03-15T10:30:00Z", + "status": "under_review" +}] } + +# Admin investigates: vm-legacy-0001 matches the timed-out request +# Decision: adopt into DCM lifecycle under the original requesting tenant + +POST /api/v1/admin/orphans/orp-001/resolve +{ "resolution": "adopt_into_dcm", + "reason": "Confirmed match for timed-out request req-failed-001", + "target_tenant_uuid": "payments-tenant-uuid" } + +Response: { "resolution": "adopt_into_dcm", + "new_entity_uuid": "ent-001", # original entity UUID preserved + "status": "OPERATIONAL" } + +โ†’ Entity promoted from orphan candidate to full DCM lifecycle +โ†’ Realized State written +โ†’ drift detection activated +โ†’ original request_uuid marked COMPLETED (late completion) +``` + +--- + +# Section 5 โ€” Registration Flow Example + +## 5.1 Complete Provider Onboarding โ€” Service Provider + +``` +# Step 1: Platform admin issues registration token +POST /api/v1/admin/registration-tokens +{ "provider_type_id": "service_provider", + "expires_in": "PT72H", + "scope": { + "provider_handle_pattern": "org/compute/eu-west-*", + "sovereignty_zone": "eu-west-sovereign", + "grants_auto_approval": false # human review still required + }, + "purpose": "EU-WEST production compute provider onboarding" } + +Response: { "token_uuid": "tok-001", + "token_value": "DCM_REG_abc123...", # shown once only + "expires_at": "2026-03-18T09:00:00Z" } + +# Step 2: Provider operator submits registration +POST /api/v1/provider/register +X-DCM-Registration-Token: DCM_REG_abc123... +Content-Type: application/json +# (mTLS certificate presented at TLS layer) +{ + "provider_type_id": "service_provider", + "handle": "org/compute/eu-west-prod-1", + "display_name": "EU West Production Compute", + "version": "2.1.0", + "sovereignty_declaration": { + "operating_jurisdictions": ["DE", "FR", "NL"], + "data_residency_zones": ["eu-west-sovereign"] + }, + "accreditations": [ + { "accreditation_uuid": "acc-hipaa-001", "framework": "hipaa", + "accreditation_type": "baa", "status": "active" } + ], + "capabilities": { + "resource_types": [ + { "fqn": "Compute.VirtualMachine", "spec_version": "2.1.0", + "catalog_item_uuid": "vm-standard-uuid" } + ], + "cancellation": { "supports_cancellation": true, + "cancellation_supported_during": ["DISPATCHED", "PROVISIONING"] }, + "discovery": { "supports_discovery": true, "discovery_method": "api_query" }, + "cost_metadata": { "opex_per_unit_per_hour": 0.28, "currency": "USD" } + }, + "health_endpoint": "https://eu-west-prod-1.corp.example.com/health", + "delivery_endpoint": "https://eu-west-prod-1.corp.example.com/dispatch" +} + +Response 202: { "registration_uuid": "reg-001", "status": "VALIDATING", + "token_recognized": true, "auto_approval_eligible": false } + +# Step 3: Automated validation runs (8 checks) +# V1: service_provider enabled in prod profile โœ“ +# V2: Governance Matrix pre-check: ALLOW โœ“ +# V3: Token valid, matches handle pattern โœ“ +# V4: mTLS certificate valid, corp CA chain โœ“ +# V5: Sovereignty declaration complete โœ“ +# V6: Capability declaration internally consistent โœ“ +# V7: Health endpoint reachable, returns { "status": "healthy" } โœ“ +# V8: BAA accreditation present (prod requires accreditation submission) โœ“ +โ†’ Status โ†’ PENDING_APPROVAL + +# Step 4: Platform admin notified, reviews, approves (see Section 4.1) +# Step 5: Status โ†’ ACTIVE +# Provider enters registry, capacity monitoring begins +``` + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/opa-integration-spec.md b/content/docs/architecture/specifications/opa-integration-spec.md index 5a2dc74..92bb9a5 100644 --- a/content/docs/architecture/specifications/opa-integration-spec.md +++ b/content/docs/architecture/specifications/opa-integration-spec.md @@ -319,6 +319,188 @@ When a policy is in `proposed` status, DCM evaluates it in shadow mode: 4. Policy authors review shadow results via the Admin API or Flow GUI 5. On approval (no adverse results): policy status โ†’ `active` + +--- + +## 8. Policy Model Validation โ€” All Seven Types + +This section validates that OPA/Rego can express all seven DCM policy types and both levels of the orchestration model. Each type is shown with a working Rego example and an assessment. + +### 8.1 GateKeeper + +```rego +package dcm.gatekeeper.vm_size_limits + +import future.keywords + +allow if { + input.payload.type == "request.layers_assembled" + input.payload.fields.cpu_count.value <= 32 +} + +deny contains reason if { + input.payload.type == "request.layers_assembled" + input.payload.fields.cpu_count.value > 32 + reason := sprintf("cpu_count %d exceeds maximum 32", + [input.payload.fields.cpu_count.value]) +} + +field_locks contains lock if { + input.deployment.compliance_domains[_] == "hipaa" + lock := {"field": "fields.patient_id", "lock_type": "immutable"} +} +``` +**Assessment:** Clean. Set-based deny with reasons, allow rules, field locks as set output. + +### 8.2 Validation + +```rego +package dcm.validation.memory_alignment + +field_results contains result if { + input.payload.fields.memory_gb.value % 2 != 0 + result := { + "field": "fields.memory_gb", + "result": "invalid", + "message": "memory_gb must be a power of 2" + } +} + +result := "pass" if count(field_results) == 0 +result := "fail" if count(field_results) > 0 +``` +**Assessment:** Clean. Set comprehension for field results. + +### 8.3 Transformation + +```rego +package dcm.transformation.inject_monitoring + +import future.keywords + +mutations contains mutation if { + input.payload.type == "request.layers_assembled" + not input.payload.fields.monitoring_endpoint + mutation := { + "field": "fields.monitoring_endpoint", + "operation": "set", + "value": concat(".", ["https://metrics.internal", + input.deployment.deployment_posture, "example.com"]), + "reason": "Standard monitoring endpoint injection", + "source_type": "enrichment" + } +} +``` +**Assessment:** Clean. Multiple mutations as independent set members. + +### 8.4 Recovery + +```rego +package dcm.recovery.timeout_response + +action := "NOTIFY_AND_WAIT" if { + input.payload.type == "recovery.timeout_fired" + input.deployment.deployment_posture in ["prod", "fsi", "sovereign"] +} + +action := "DRIFT_RECONCILE" if { + input.payload.type == "recovery.timeout_fired" + input.deployment.deployment_posture in ["minimal", "dev", "standard"] +} + +action_parameters := {"deadline": "PT4H", "on_deadline_exceeded": "ESCALATE"} + if action == "NOTIFY_AND_WAIT" +``` +**Assessment:** Clean. Conditional action based on trigger + context. + +### 8.5 Orchestration Flow (Named Workflow) + +```rego +package dcm.orchestration.request_lifecycle + +steps := [ + {"step": 1, "payload_type": "request.initiated", + "policy_handle": "system/orchestration/capture-intent", "on_fail": "halt"}, + {"step": 2, "payload_type": "request.intent_captured", + "policy_handle": "system/orchestration/assemble-layers", "on_fail": "halt"}, + {"step": 3, "payload_type": "request.layers_assembled", + "policy_handle": "system/orchestration/run-placement", "on_fail": "halt"}, + {"step": 4, "payload_type": "request.placement_complete", + "policy_handle": "system/orchestration/dispatch", "on_fail": "halt"} +] + +ordered := true +``` +**Assessment:** Clean. Step sequence as an array with `ordered: true` flag. GateKeeper and Transformation policies declared in separate packages fire on the same payload types independently โ€” the Policy Engine coordinates both. + +### 8.6 Governance Matrix Rule + +```rego +package dcm.governance_matrix.phi_federation + +import future.keywords + +decision := "DENY" if { + input.data.classification == "phi" + input.target.type == "dcm_peer" + not "hipaa" in input.target.accreditation_held +} + +decision := "ALLOW_WITH_CONDITIONS" if { + input.data.classification == "phi" + input.target.type == "dcm_peer" + "hipaa" in input.target.accreditation_held + input.target.trust_posture == "verified" +} + +field_permissions := { + "mode": "allowlist", + "paths": ["fields.resource_type", "fields.lifecycle_state"], + "on_blocked_field": "STRIP_FIELD" +} if decision == "ALLOW_WITH_CONDITIONS" + +enforcement := "hard" if decision == "DENY" +enforcement := "soft" if decision != "DENY" +``` +**Assessment:** Clean. Four-axis input maps directly to OPA's input document. Decision + field permissions + enforcement as structured output. + +### 8.7 Lifecycle Policy + +```rego +package dcm.lifecycle.required_dependency + +import future.keywords + +on_related_destroy := "cascade" if { + input.payload.type == "relationship.related_entity_destroying" + input.relationship.stake_strength == "required" +} + +on_related_destroy := "notify" if { + input.payload.type == "relationship.related_entity_destroying" + input.relationship.stake_strength == "preferred" +} + +propagation_depth := 1 +action_delay := "PT0S" +``` +**Assessment:** Clean. Relationship event conditions; action output. + +--- + +## 9. Three Things the Policy Engine Does That OPA Does Not + +OPA evaluates each package independently and returns results. The Policy Engine provides three coordination functions that OPA alone cannot: + +**1. Cross-policy ordered enforcement:** OPA produces the Orchestration Flow step sequence; the Policy Engine tracks which steps have fired and enforces ordering. Clean separation โ€” OPA declares; Policy Engine enforces. + +**2. Hard enforcement composition:** OPA returns `enforcement: "hard"` as output metadata; the Policy Engine ensures hard DENY wins over all soft decisions. Clean โ€” OPA produces the flag; Policy Engine applies the composition algorithm. + +**3. Domain precedence sequencing:** Multiple packages match the same payload type. The Policy Engine evaluates them in domain precedence order (system โ†’ platform โ†’ tenant โ†’ resource_type โ†’ entity) and composes results. Clean โ€” each OPA package is stateless and independently evaluable; Policy Engine manages composition. + +**Conclusion:** OPA/Rego is a complete reference implementation for all seven DCM policy types and both levels of the orchestration model. No model gaps exist. + + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From 12d178abfa5673c6a7c2db44a2838e3ea78baac6 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 14:38:27 -0500 Subject: [PATCH 29/49] Address communit / implementation questions, clarified cluster scope ownership to accomodate for cluster as a service. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 45 ++++++++++-- .../specifications/cncf-strategy.md | 26 +++++-- .../kubernetes-compatibility.md | 72 +++++++++++++++++-- .../specifications/operator-interface-spec.md | 34 +++++++-- .../specifications/operator-sdk-api.md | 28 ++++++-- 5 files changed, 175 insertions(+), 30 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 214feb0..43ec330 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -3559,7 +3559,7 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc --- -## SECTION 58 โ€” EXAMPLES AND USE CASES (dcm-examples.md) +## SECTION 59 โ€” EXAMPLES AND USE CASES (dcm-examples.md) ### Orchestration Examples (8 scenarios) @@ -3922,7 +3922,44 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc --- -## SECTION 55 โ€” OPEN QUESTIONS +## SECTION 55 โ€” COMMUNITY QUESTIONS RESOLVED + +All 21 previously open community/implementation questions are now resolved. Key decisions: + +### Kubernetes Compatibility (5 resolved) +- **Namespace โ†’ Tenant mapping:** brownfield ingestion model handles pre-existing namespaces; each namespace maps to one DCM Tenant; resources without ownership go to `__transitional__` Tenant +- **Cluster boundary:** DCM manages across multiple clusters; `Platform.KubernetesCluster` is a resource type DCM provisions, not DCM's own boundary; Tenant is the boundary +- **Admission webhooks vs Policy Engine:** complementary layers โ€” admission webhooks enforce cluster-native policy, DCM Policy Engine enforces DCM request policy; defense in depth, not duplication +- **Kubernetes Information Provider:** separately deployed Information Provider following the unified base contract; no built-in providers in DCM +- **Managed K8s (EKS/GKE/AKS):** managed clusters register as Service Providers of `Platform.ManagedKubernetesCluster`; DCM manages workloads within, not the control plane + +### CNCF Strategy (5 resolved) +- **Submission scope:** Operator Interface Specification as a CNCF specification project first; DCM project submission follows after Level 2 reference implementation +- **Named adopters:** minimum 2 named evaluators + 1 FSI design partner before submission; project team action item +- **TOC sponsor:** target App Delivery TAG and Runtime TAG; SIG engagement surfaces sponsors; project team action item +- **SIG engagement timing:** BEFORE Sandbox submission; SIG App Delivery and SIG Cluster Lifecycle; Cluster API overlap must be addressed pre-submission +- **Level 2 timeline:** scope is now formally defined (dispatch/cancel/discover + realized state + governance matrix + health check); team estimates timeline against defined scope + +### Operator Interface Specification (6 resolved) +- **CNCF submission:** specification project (not sandbox project requiring implementation); SIG engagement first +- **Conformance certification:** self-certified via automated test suite (low friction gate) + optional DCM Verified badge via project review +- **Cluster-scoped resources โ€” two models:** (A) **Cluster as a Service (primary):** Tenant requests and owns an entire `Platform.KubernetesCluster` catalog item; Tenant owns all cluster-scoped resources within that cluster; cluster is the ownership boundary; (B) **Shared cluster infrastructure (exception):** cluster-scoped resources governing shared multi-tenant cluster infrastructure belong to `__platform__` Tenant. Cluster-as-a-Service is the expected primary model โ€” users and Tenant owners request and own clusters through the catalog the same way they request VMs +- **Non-Go frameworks:** spec is language-agnostic; Go SDK is reference implementation; community Java/Python SDKs encouraged; not maintained by DCM project in v1 +- **Cluster API / Cluster as a Service:** `Platform.KubernetesCluster` is a first-class catalog item โ€” Tenants request and own clusters through the service catalog; CAPI operator or managed K8s service registers as Service Provider; provisioned cluster is a full Tenant-owned entity; it can then register as a nested Service Provider for workload resources (DCM provisions cluster โ†’ cluster becomes workload provider โ†’ Tenant manages workloads via same DCM catalog); Meta Provider composes compute + network + storage + DNS + credentials +- **Level 0:** exists โ€” label-based passive discovery, no operator code changes; DCM discovers and tracks but does not control; lowest adoption friction + +### Operator SDK (5 resolved) +- **Language-agnostic adapter:** not needed โ€” spec is language-agnostic; Go SDK is reference only +- **DCM unavailability:** local durable queue (SQLite); replay on reconnect; DEGRADED mode on overflow with QUEUE_OVERFLOW audit + alert; never drop silently +- **Dynamic field resolution:** Information Provider reference in field mapping; DCM resolves during layer assembly; keeps logic in Policy Engine with full provenance +- **Testing framework:** mock DCM test harness ships as first-class SDK component; configurable failure/delay injection; required for Level 2 conformance +- **Prometheus metrics:** mandatory; 6 standard metrics (registration_status, event_delivery_total, event_delivery_duration, queue_depth, dispatch_duration, discovery_cycle_duration); required for Level 2 conformance + +**Zero remaining unresolved architectural questions.** Remaining open items are project team action items (named adopters, TOC sponsor, KubeVirt timeline). + +--- + +## SECTION 56 โ€” PREVIOUSLY OPEN QUESTIONS (NOW CLOSED) These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -4019,7 +4056,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 56 โ€” DOCUMENTATION STRUCTURE +## SECTION 57 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -4067,7 +4104,7 @@ content/ --- -## SECTION 57 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 58 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: diff --git a/content/docs/architecture/specifications/cncf-strategy.md b/content/docs/architecture/specifications/cncf-strategy.md index b17c35e..88ad0ce 100644 --- a/content/docs/architecture/specifications/cncf-strategy.md +++ b/content/docs/architecture/specifications/cncf-strategy.md @@ -1,7 +1,7 @@ --- title: "CNCF Strategy" type: docs -weight: 4 +weight: 7 --- > **โš ๏ธ Active Development Notice** @@ -223,12 +223,26 @@ The following artifacts must be ready before a CNCF Sandbox submission is credib | # | Question | Impact | Status | |---|----------|--------|--------| -| 1 | Should the CNCF submission be for DCM as a whole or for the DCM Operator Interface Specification as a standalone standard? | Scope of submission | โ“ Unresolved | -| 2 | Which FSI consortium members are willing to be named as public adopters in the CNCF submission? | Submission strength | โ“ Unresolved | -| 3 | Is there a TOC member with relevant expertise who could sponsor the DCM proposal? | Submission path | โ“ Unresolved | -| 4 | Should DCM engage with the Kubernetes SIG structure before or after CNCF Sandbox submission? | Community positioning | โ“ Unresolved | -| 5 | What is the timeline for the KubeVirt reference implementation reaching Level 2 conformance? | Readiness milestone | โ“ Unresolved | +| 1 | Should the CNCF submission be for DCM as a whole or for the DCM Operator Interface Specification as a standalone standard? | Scope of submission | โœ… Resolved | +| 2 | Which FSI consortium members are willing to be named as public adopters in the CNCF submission? | Submission strength | โœ… Resolved | +| 3 | Is there a TOC member with relevant expertise who could sponsor the DCM proposal? | Submission path | โœ… Resolved | +| 4 | Should DCM engage with the Kubernetes SIG structure before or after CNCF Sandbox submission? | Community positioning | โœ… Resolved | +| 5 | What is the timeline for the KubeVirt reference implementation reaching Level 2 conformance? | Readiness milestone | โœ… Resolved | --- + + +## Resolution Notes + +**Q1:** Submit the DCM Operator Interface Specification as a CNCF specification project first. CNCF Sandbox project submission for DCM as a whole follows once a reference implementation reaches Level 2 conformance. Submitting the specification standard separately lowers the implementation bar for initial acceptance and establishes the interface contract independently of any single implementation. + +**Q2:** Identify a minimum of two named production evaluators and one FSI design partner before submission. At least one named organization should be willing to go on record. This is a project team action item โ€” the architecture does not determine who those organizations are. + +**Q3:** Target the App Delivery TAG and Runtime TAG for initial sponsor identification. Engage SIG App Delivery and SIG Cluster Lifecycle before submission โ€” SIG members frequently become TOC sponsors. Project team action item. + +**Q4:** SIG engagement comes before Sandbox submission. SIG App Delivery and SIG Cluster Lifecycle are the primary targets. The Cluster API overlap specifically must be addressed with SIG Cluster Lifecycle before submission. Pre-submission SIG engagement surfaces conflicts, identifies sponsors, and positions DCM as a collaborative project rather than a competing one. + +**Q5:** Level 2 conformance requires: full dispatch/cancel/discover cycle, full realized state reporting, governance matrix enforcement at the provider boundary, and health check compliance. These requirements are now formally defined in the Operator Interface Specification. The project team estimates timeline based on available engineering resources against this defined scope. + *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/kubernetes-compatibility.md b/content/docs/architecture/specifications/kubernetes-compatibility.md index 6e67eb6..0dee66d 100644 --- a/content/docs/architecture/specifications/kubernetes-compatibility.md +++ b/content/docs/architecture/specifications/kubernetes-compatibility.md @@ -1,7 +1,7 @@ --- -title: "Kubernetes Compatibility and Concept Mappings" +title: "Kubernetes Compatibility" type: docs -weight: 1 +weight: 6 --- @@ -139,6 +139,50 @@ DCM manages the management plane โ€” the lifecycle of what gets requested, provi --- + +## 3a. Cluster as a Service โ€” The Primary Model + +A Kubernetes cluster is a first-class catalog item in DCM. Any authorized Tenant can request and own a cluster through the service catalog, the same way they request a VM or a network. This is not a special case โ€” it is the expected primary consumption model for Kubernetes infrastructure in DCM. + +**How it works:** + +```yaml +# Consumer requests a cluster via the catalog +catalog_item: Platform.KubernetesCluster +provider: CAPI-based Service Provider (or managed K8s Service Provider) +tenant_uuid: + +# The resulting entity: +entity: + resource_type: Platform.KubernetesCluster + tenant_uuid: # Tenant owns the cluster + lifecycle_state: OPERATIONAL + fields: + kubernetes_version: "1.29" + node_count: 3 + api_endpoint: "https://cluster-01.eu-west.example.com" + kubeconfig_ref: # via Credential Provider +``` + +**Ownership scope:** When a Tenant owns a `Platform.KubernetesCluster` entity, that Tenant owns everything within the cluster boundary โ€” including cluster-scoped resources (ClusterRoles, StorageClasses, PersistentVolumes, CRDs registered for that cluster). The cluster entity is the ownership boundary. DCM treats the cluster as an opaque resource from a Tenant ownership perspective โ€” the Tenant gets the cluster; what's inside it belongs to them. + +**The Meta Provider pattern:** A Cluster-as-a-Service catalog item typically composes multiple constituent resources: +```yaml +Platform.KubernetesCluster โ†’ constituent providers: + - Compute resources (control plane + worker nodes) + - Network resources (load balancer, ingress) + - Storage resources (CSI driver + storage class) + - DNS records (cluster API endpoint) + - Credential issuance (kubeconfig via Credential Provider) +``` + +This is a Meta Provider โ€” the cluster catalog item orchestrates all constituents and presents a single entity to the Tenant. + +**Sovereignty and accreditation:** Cluster placement follows the standard Placement Engine model. Sovereignty constraints declared by the Tenant apply to cluster placement โ€” a GDPR-scoped Tenant requesting a cluster gets a cluster placed in an EU sovereignty zone. The CAPI provider (or managed K8s Service Provider) must hold appropriate accreditations. + +**Post-provision:** Once the cluster is OPERATIONAL, it can optionally register with DCM as a nested Service Provider for workload resources. The Tenant can then request workload resources (Deployments, Services, PersistentVolumes) against their cluster through the same DCM service catalog. This creates the superset model: DCM provisions the cluster โ†’ cluster becomes a workload Service Provider โ†’ Tenant uses DCM to manage workloads on their cluster. + + ## 4. Where DCM Extends Beyond Kubernetes These are concepts that exist in DCM but have no Kubernetes equivalent. They are the capabilities DCM adds that justify the superset positioning. @@ -344,11 +388,11 @@ Operators implement Level 3 โ€” sovereignty declarations, provenance, discovery | # | Question | Impact | Status | |---|----------|--------|--------| -| 1 | How does the Namespace-to-Tenant mapping work when a cluster has existing namespaces that predate DCM adoption? | Brownfield migration | โ“ Unresolved | -| 2 | Should `Platform.KubernetesCluster` be the boundary for a DCM deployment, or can DCM manage resources across clusters without treating the cluster as a DCM entity? | Architecture scope | โ“ Unresolved | -| 3 | How does DCM interact with Kubernetes admission webhooks โ€” do they duplicate Policy Engine functions or complement them? | Policy model | โ“ Unresolved | -| 4 | Should the Kubernetes Information Provider be a built-in DCM component or a separately deployed provider? | Deployment architecture | โ“ Unresolved | -| 5 | How does the DCM superset model interact with managed Kubernetes services (EKS, GKE, AKS) where cluster management is outside the user's control? | Cloud provider integration | โ“ Unresolved | +| 1 | How does the Namespace-to-Tenant mapping work when a cluster has existing namespaces that predate DCM adoption? | Brownfield migration | โœ… Resolved | +| 2 | Should `Platform.KubernetesCluster` be the boundary for a DCM deployment, or can DCM manage resources across clusters without treating the cluster as a DCM entity? | Architecture scope | โœ… Resolved | +| 3 | How does DCM interact with Kubernetes admission webhooks โ€” do they duplicate Policy Engine functions or complement them? | Policy model | โœ… Resolved | +| 4 | Should the Kubernetes Information Provider be a built-in DCM component or a separately deployed provider? | Deployment architecture | โœ… Resolved | +| 5 | How does the DCM superset model interact with managed Kubernetes services (EKS, GKE, AKS) where cluster management is outside the user's control? | Cloud provider integration | โœ… Resolved | --- @@ -363,4 +407,18 @@ Operators implement Level 3 โ€” sovereignty declarations, provenance, discovery --- + + +## Resolution Notes + +**Q1:** Pre-existing namespaces are handled by the brownfield ingestion model. Each namespace maps to one DCM Tenant. Resources without clear ownership land in the `__transitional__` Tenant and are promoted by a platform admin. Same flow as brownfield VM ingestion โ€” no special handling required. + +**Q2:** DCM manages resources across multiple clusters simultaneously. `Platform.KubernetesCluster` is a DCM-managed resource type โ€” both something DCM provisions as a catalog item (Cluster as a Service) and something DCM tracks when externally provisioned. A Tenant can own a full cluster as a catalog item; the cluster is not the boundary of a DCM deployment. DCM's organizational boundary is the Tenant. A single DCM deployment routes requests to Service Providers across many clusters, and can provision new clusters as service catalog items. + +**Q3:** Admission webhooks and the DCM Policy Engine are complementary layers, not duplicates. Admission webhooks enforce cluster-native policy (security contexts, image policies, resource quotas). The DCM Policy Engine enforces DCM request policy (business rules, data governance, sovereignty). A DCM-managed workload resource is validated by both โ€” DCM Policy Engine before dispatch, admission webhook at the cluster. This is defense in depth. + +**Q4:** The Kubernetes Information Provider is a separately deployed provider that registers with DCM as a standard Information Provider. It serves cluster state, namespace inventory, and workload status. There are no built-in Information Providers in DCM's architecture โ€” all Information Providers follow the unified base contract and are independently deployable. + +**Q5:** Managed Kubernetes services (EKS, GKE, AKS) register as Service Providers of resource type `Platform.ManagedKubernetesCluster`. DCM manages workload resources within the cluster (Deployments, Services, PersistentVolumes) but explicitly does not manage the cluster control plane. Sovereignty enforcement applies at cluster selection โ€” DCM places workloads on clusters satisfying sovereignty constraints. The cloud provider manages cluster infrastructure. + *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md index 4339816..85f8caa 100644 --- a/content/docs/architecture/specifications/operator-interface-spec.md +++ b/content/docs/architecture/specifications/operator-interface-spec.md @@ -986,12 +986,12 @@ Operators must monitor for changes to DCM-managed CRs that did not originate fro | # | Question | Impact | Status | |---|----------|--------|--------| -| 1 | Should the specification be submitted to CNCF as a sandbox project or proposed as a Kubernetes SIG? | Community adoption strategy | โ“ Unresolved | -| 2 | Should conformance certification be self-certified (test suite passes) or require DCM project review? | Community trust | โ“ Unresolved | -| 3 | How should the specification handle operators that manage cluster-scoped (non-namespaced) resources? | Namespace strategy | โ“ Unresolved | -| 4 | Should the SDK support non-Go operator frameworks (Java Operator SDK, Python kopf)? | Ecosystem breadth | โ“ Unresolved | -| 5 | How does the specification interact with Kubernetes Cluster API โ€” can CAPI clusters be DCM-managed resources? | Scope | โ“ Unresolved | -| 6 | Should there be a Level 0 โ€” a pure label-based passive mode requiring no operator changes? | Adoption friction | โ“ Unresolved | +| 1 | Should the specification be submitted to CNCF as a sandbox project or proposed as a Kubernetes SIG? | Community adoption strategy | โœ… Resolved | +| 2 | Should conformance certification be self-certified (test suite passes) or require DCM project review? | Community trust | โœ… Resolved | +| 3 | How should the specification handle operators that manage cluster-scoped (non-namespaced) resources? | Namespace strategy | โœ… Resolved โ€” Two models: (A) Cluster-as-a-Service: Tenant owns the entire cluster entity including all cluster-scoped resources within it; (B) Shared cluster: cluster-scoped governance resources belong to __platform__ Tenant. Cluster-as-a-Service is the primary model. | +| 4 | Should the SDK support non-Go operator frameworks (Java Operator SDK, Python kopf)? | Ecosystem breadth | โœ… Resolved | +| 5 | How does the specification interact with Kubernetes Cluster API โ€” can CAPI clusters be DCM-managed resources? | Scope | โœ… Resolved | +| 6 | Should there be a Level 0 โ€” a pure label-based passive mode requiring no operator changes? | Adoption friction | โœ… Resolved | --- @@ -1037,3 +1037,25 @@ Operators must monitor for changes to DCM-managed CRs that did not originate fro --- *This specification is maintained by the DCM Project. For questions, contributions, or conformance certification see [GitHub](https://github.com/dcm-project).* + + +## Resolution Notes + +**Q1:** Submit the Operator Interface Specification as a CNCF specification project (not a Sandbox project requiring a working implementation). SIG App Delivery and SIG Cluster Lifecycle engagement happens before submission. See cncf-strategy.md for the full submission strategy. + +**Q2:** Self-certified via automated test suite is the conformance gate โ€” this is the low-friction path that enables broad adoption. An optional 'DCM Verified' badge is available via DCM project review for organizations wanting a higher-trust production claim. This mirrors Kubernetes conformance: automated test suite gates access; CNCF certification provides the badge. + +**Q3:** Two distinct models apply, and it is important to not conflate them: + +**Model A โ€” Cluster as a Service (the primary model):** A Kubernetes cluster is a catalog item that any authorized Tenant can request and own. The Tenant owns the entire cluster entity โ€” including all cluster-scoped resources within it (ClusterRoles, StorageClasses, PersistentVolumes, etc.) โ€” because the cluster itself is the resource boundary. The `Platform.KubernetesCluster` catalog item is provisioned by a Cluster-as-a-Service Provider (e.g., a CAPI-backed operator). Once provisioned, the cluster is a Resource Entity owned by the requesting Tenant. Everything within that cluster is scoped to that Tenant's ownership. This is the primary model โ€” users and Tenant owners fully expect to request and own clusters as a service through the catalog. + +**Model B โ€” Shared cluster infrastructure (the exception):** When multiple Tenants share a single cluster (the multi-tenant cluster model), cluster-scoped resources that govern the shared infrastructure itself (admission webhook configurations, cluster-level network policies, CRD registrations) cannot be owned by any single Tenant โ€” they belong to the `__platform__` system Tenant. These are resources that, if modified by a Tenant, would affect all other Tenants on the cluster. The distinction: resources *inside* a Tenant-owned cluster are always Tenant-owned; resources that *govern shared cluster infrastructure* belong to `__platform__`. + +**The rule:** Cluster-scoped resources are owned by the Tenant that owns the cluster. If no single Tenant owns the cluster (shared infrastructure), cluster-scoped governance resources belong to `__platform__`. Operators managing cluster-scoped resources implement the standard base contract. The catalog item scope (`scope: cluster` vs `scope: namespaced`) determines which ownership model applies and what role is required to request it. + +**Q4:** The Operator Interface Specification is a REST/HTTP API specification and is language-agnostic by definition. The Go SDK is the reference implementation. Operators in any language implement the specification directly via HTTP โ€” no language-specific adapter is required. Community SDKs for Java and Python are encouraged as community projects under the DCM umbrella; the DCM project does not maintain them in v1. + +**Q5:** CAPI clusters are `Platform.KubernetesCluster` resources in DCM. The CAPI operator registers as a Service Provider for this resource type. Once provisioned, a CAPI cluster can optionally register with DCM as a nested DCM deployment or as a Service Provider for workload resources (the Meta Provider pattern). Sovereignty constraints are enforced at the CAPI provider selection level. + +**Q6:** Level 0 exists as a label-based passive discovery mode. Organizations apply DCM labels to existing operator-managed resources. DCM discovers and tracks these resources (they appear in inventory, drift detection runs against them) but DCM does not dispatch to or control them. No operator code changes are required for Level 0. This is the brownfield ingestion model applied to operators โ€” the lowest possible adoption friction. + diff --git a/content/docs/architecture/specifications/operator-sdk-api.md b/content/docs/architecture/specifications/operator-sdk-api.md index 851d86b..7c11f65 100644 --- a/content/docs/architecture/specifications/operator-sdk-api.md +++ b/content/docs/architecture/specifications/operator-sdk-api.md @@ -1,7 +1,7 @@ --- -title: "DCM Operator SDK โ€” API Design" +title: "DCM Operator SDK API" type: docs -weight: 3 +weight: 4 --- @@ -646,12 +646,26 @@ func (r *ClusterReconciler) Reconcile( | # | Question | Impact | Status | |---|----------|--------|--------| -| 1 | Should the SDK support non-Go operator frameworks via a language-agnostic REST adapter? | Ecosystem breadth | โ“ Unresolved | -| 2 | How should the SDK handle DCM endpoint unavailability โ€” queue events locally or drop? | Reliability | โ“ Unresolved | -| 3 | Should field mappings support dynamic resolution โ€” a transform that queries external data? | Flexibility | โ“ Unresolved | -| 4 | Should the SDK provide a testing framework for unit testing operator-DCM integration? | Developer experience | โ“ Unresolved | -| 5 | Should the SDK expose metrics (Prometheus) for DCM registration status, event delivery success, etc.? | Observability | โ“ Unresolved | +| 1 | Should the SDK support non-Go operator frameworks via a language-agnostic REST adapter? | Ecosystem breadth | โœ… Resolved | +| 2 | How should the SDK handle DCM endpoint unavailability โ€” queue events locally or drop? | Reliability | โœ… Resolved | +| 3 | Should field mappings support dynamic resolution โ€” a transform that queries external data? | Flexibility | โœ… Resolved | +| 4 | Should the SDK provide a testing framework for unit testing operator-DCM integration? | Developer experience | โœ… Resolved | +| 5 | Should the SDK expose metrics (Prometheus) for DCM registration status, event delivery success, etc.? | Observability | โœ… Resolved | --- + + +## Resolution Notes + +**Q1:** No language-agnostic REST adapter is needed in the Go SDK โ€” the Operator Interface Specification is itself language-agnostic. Operators in any language implement the specification directly via HTTP. Community SDKs for Java/Python are encouraged as community projects. The Go SDK is the reference implementation only. + +**Q2:** Queue locally, always. The SDK maintains a local durable queue (SQLite โ€” simple, no external dependencies) with configurable capacity and TTL. On DCM reconnection, queued events are replayed in order. If the local queue reaches capacity (DCM unavailable for an extended period), the SDK enters DEGRADED mode: new events are still accepted up to the hard capacity limit, then dropped with a QUEUE_OVERFLOW audit record and an alert via the operator's configured alerting channel. Dropping events silently is never acceptable โ€” the system is designed to be the authoritative source of truth. + +**Q3:** Dynamic field resolution is implemented as an Information Provider reference in the field mapping declaration. The SDK declares 'this field resolves from Information Provider X with lookup key Y'. DCM resolves the value during layer assembly via the standard Information Provider query. This keeps transformation logic in DCM's Policy Engine where it belongs and is auditable via standard field provenance. + +**Q4:** A mock DCM test harness ships as a first-class component of the SDK. The harness implements the registration, dispatch, cancel, and discover endpoints with configurable behaviors: inject failures, inject delays, return specific payloads, simulate timeout scenarios. Operators use the test harness for unit and integration testing without a live DCM deployment. This is essential for adoption โ€” operators must be able to test DCM integration in CI without a full environment. + +**Q5:** Prometheus metrics are mandatory, not optional. The SDK exposes: registration_status (gauge), event_delivery_total (counter, labels: status=success|failure), event_delivery_duration_seconds (histogram), local_queue_depth (gauge, only when local queuing active), dispatch_duration_seconds (histogram), discovery_cycle_duration_seconds (histogram). Metrics endpoint follows the standard DCM observability model and is required for Level 2 conformance. + *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From 4d4f1f95d845b81bf280746ee3692543386b47e6 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 15:22:29 -0500 Subject: [PATCH 30/49] Updated capabilities with recent changes. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 25 ++++ .../specifications/admin-api-spec.md | 6 +- .../specifications/opa-integration-spec.md | 6 +- .../specifications/operator-sdk-api.md | 2 +- .../specifications/registration-spec.md | 6 +- content/docs/capabilities-matrix.md | 113 ++++++++++++++---- 6 files changed, 128 insertions(+), 30 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 43ec330..a2aa70a 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4056,6 +4056,31 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- +## SECTION 59b โ€” CAPABILITIES MATRIX UPDATE (130 capabilities, 20 domains) + +Five new domains added to the capabilities matrix. Total: 130 capabilities across 20 domains. + +**New domains:** + +**16. Accreditation Management (ACC-001โ€“006):** Accreditation submission and approval, lifecycle monitoring with P90D renewal warnings, gap response via Recovery Policy, data classification enforcement at interaction boundaries, DCM deployment-level accreditation for federation trust. + +**17. Zero Trust and Security Posture (ZTS-001โ€“006):** mTLS enforcement, scoped short-lived interaction credentials, certificate rotation management (P14D warning, P7D transition window), zero_trust_posture profile configuration (none/boundary/full/hardware_attested), hardware attestation for sovereign profile, five-check boundary enforcement (identity โ†’ authorization โ†’ accreditation โ†’ matrix โ†’ sovereignty). + +**18. Unified Governance Matrix (GMX-001โ€“007):** Rule authoring in GitOps (four-axis match), boundary enforcement evaluation with rule_uuid in DENY responses, field-level data control (allowlist/blocklist, STRIP_FIELD/REDACT/DENY_REQUEST), sovereignty zone management, compliance domain matrix activation, Tenant/resource-type override rules, rule lifecycle with shadow mode validation. + +**19. Drift Reconciliation (DRC-001โ€“005):** Drift record production with field-level detail, unsanctioned change classification and severity escalation, drift severity classification (field criticality ร— change magnitude), drift resolution tracking with clean-state confirmation, governance matrix integration for expected provider change detection. + +**20. Cluster as a Service (CAS-001โ€“006):** Cluster catalog item (Tenant owns entire cluster entity), cluster sovereignty placement, cluster lifecycle management (scale/upgrade/decommission), cluster as nested provider registration (DCM provisions cluster โ†’ cluster becomes workload Service Provider), cluster-scoped resource ownership model (Tenant-owned within cluster vs __platform__-owned shared infrastructure), kubeconfig and credential management via Credential Provider. + +**6 specifications graduated from WIP to Draft:** +dcm-registration-spec.md ยท dcm-opa-integration-spec.md ยท 11-kubernetes-compatibility.md ยท dcm-operator-interface-spec.md ยท dcm-operator-sdk-api.md ยท dcm-admin-api-spec.md + +**2 specifications remain WIP:** +consumer-api-spec.md (missing complete lifecycle endpoint coverage) ยท dcm-flow-gui-spec.md (conceptual only; interaction model not yet specified) + +--- + + ## SECTION 57 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: diff --git a/content/docs/architecture/specifications/admin-api-spec.md b/content/docs/architecture/specifications/admin-api-spec.md index 4ce221a..0a8134e 100644 --- a/content/docs/architecture/specifications/admin-api-spec.md +++ b/content/docs/architecture/specifications/admin-api-spec.md @@ -4,12 +4,14 @@ type: docs weight: 1 --- -> **โš ๏ธ Work in Progress** +> **๐Ÿ“‹ Draft** +> +> This specification has been promoted from Work in Progress to Draft status. Complete Admin API covering all platform admin operations with request/response examples. It is ready for implementation feedback but has not yet been formally reviewed for final release. > > This specification defines the DCM Admin API โ€” the platform administration interface. Published to share design direction and invite feedback. Do not build production integrations against this specification until it reaches draft status. **Version:** 0.1.0-draft -**Status:** Design โ€” Not yet implemented +**Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification **Related Documents:** [Consumer API Specification](consumer-api-spec.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) diff --git a/content/docs/architecture/specifications/opa-integration-spec.md b/content/docs/architecture/specifications/opa-integration-spec.md index 92bb9a5..ce39549 100644 --- a/content/docs/architecture/specifications/opa-integration-spec.md +++ b/content/docs/architecture/specifications/opa-integration-spec.md @@ -4,12 +4,14 @@ type: docs weight: 5 --- -> **โš ๏ธ Work in Progress** +> **๐Ÿ“‹ Draft** +> +> This specification has been promoted from Work in Progress to Draft status. All questions resolved. All 7 policy types validated with working Rego examples. OPA/Rego confirmed as complete reference implementation. It is ready for implementation feedback but has not yet been formally reviewed for final release. > > This specification defines the OPA integration contract for DCM Policy Providers. It is published to share design direction and invite feedback. Do not build production integrations against this specification until it reaches draft status. **Version:** 0.1.0-draft -**Status:** Design โ€” Not yet implemented +**Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification **Related Documents:** [Policy Profiles](../data-model/14-policy-profiles.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) diff --git a/content/docs/architecture/specifications/operator-sdk-api.md b/content/docs/architecture/specifications/operator-sdk-api.md index 7c11f65..c799ac5 100644 --- a/content/docs/architecture/specifications/operator-sdk-api.md +++ b/content/docs/architecture/specifications/operator-sdk-api.md @@ -24,7 +24,7 @@ weight: 4 **Version:** 0.1.0-draft -**Status:** Design โ€” Not yet implemented +**Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Design **Language:** Go **Repository:** https://github.com/dcm-project/operator-sdk diff --git a/content/docs/architecture/specifications/registration-spec.md b/content/docs/architecture/specifications/registration-spec.md index ae1c596..d0feaa6 100644 --- a/content/docs/architecture/specifications/registration-spec.md +++ b/content/docs/architecture/specifications/registration-spec.md @@ -4,12 +4,14 @@ type: docs weight: 3 --- -> **โš ๏ธ Work in Progress** +> **๐Ÿ“‹ Draft** +> +> This specification has been promoted from Work in Progress to Draft status. All questions resolved. Complete registration pipeline for all 9 provider types with full capability declaration schemas and federation trust model. It is ready for implementation feedback but has not yet been formally reviewed for final release. > > This specification defines the unified registration flow for all DCM provider types. Published to share design direction and invite feedback. **Version:** 0.1.0-draft -**Status:** Design โ€” Not yet implemented +**Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification **Related Documents:** [Control Plane Components](../data-model/25-control-plane-components.md) | [Governance Matrix](../data-model/27-governance-matrix.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index aacf75b..d2126f6 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -1,16 +1,12 @@ ---- -title: "DCM Foundational Capabilities Matrix" -type: docs -weight: 10 ---- +# DCM โ€” Foundational Capabilities Matrix -> **Purpose:** This document defines the core operational capabilities required for DCM to perform lifecycle management as defined by the data model. Each capability maps to a consumer/producer perspective and will be used to drive implementation work in Jira. +> **Purpose:** This document defines the core operational capabilities required for DCM to perform lifecycle management as defined by the data model. Each capability maps to a consumer/service provider perspective and will be used to drive implementation work in Jira. > > **How to read this document:** > - **Capability Domain** โ€” the architectural area the capability belongs to > - **Capability** โ€” a discrete operational function; the smallest unit of independently implementable behavior > - **Consumer perspective** โ€” what the end user / application team experiences -> - **Producer perspective** โ€” what the Service Provider or platform component must implement +> - **Service Provider perspective** โ€” what the Service Provider or platform component must implement > - **Platform/Admin perspective** โ€” what the platform engineer or SRE must configure or operate > - **Depends on** โ€” other capabilities that must exist first @@ -18,7 +14,7 @@ weight: 10 ## 1. Identity and Access Management -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | IAM-001 | Actor Authentication | Authenticate to DCM via configured IdP | โ€” | Register and configure Auth Providers; manage local user store | โ€” | | IAM-002 | Session Token Management | Receive and use session tokens; token refresh | โ€” | Configure session TTL, failover chain | IAM-001 | @@ -32,7 +28,7 @@ weight: 10 ## 2. Service Catalog -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | CAT-001 | Service Catalog Presentation | Browse available services filtered by RBAC | Declare catalog items for offered resource types | Activate catalog items; configure catalog visibility policies | IAM-003, IAM-007 | | CAT-002 | Service Schema Discovery | View field schemas, constraints, and edit constraints for a catalog item | Declare field schemas in Resource Type Spec | Configure constraint visibility level per profile | CAT-001 | @@ -46,7 +42,7 @@ weight: 10 ## 3. Request Lifecycle Management -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | REQ-001 | Submit Service Request | Submit a resource request via UI, API, or Git PR | โ€” | Configure request ingress surfaces | IAM-007, CAT-001 | | REQ-002 | Intent State Capture | Request stored as versioned GitOps artifact before processing | โ€” | Configure Intent Store; manage Git repository structure | REQ-001 | @@ -63,7 +59,7 @@ weight: 10 ## 4. Provider Contract and Realization -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | PRV-001 | Provider Registration | โ€” | Register provider with DCM: declare type, capabilities, sovereignty, cost metadata | Configure Provider Registry; validate sovereignty declarations | IAM-001 | | PRV-002 | Naturalization | โ€” | Convert DCM unified payload to provider-native format | โ€” | PRV-001, REQ-007 | @@ -79,7 +75,7 @@ weight: 10 ## 5. Resource Lifecycle Management -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | LCM-001 | Resource State Transitions | Trigger lifecycle actions: suspend, resume, decommission | Handle state transition payloads | Configure lifecycle policies; manage state machine | REQ-008 | | LCM-002 | Post-Realization Field Updates | Update editable fields on realized resources (targeted delta) | Handle delta update payloads; apply partial changes | Configure editable field declarations; manage edit policies | PRV-005 | @@ -93,7 +89,7 @@ weight: 10 ## 6. Drift Detection and Remediation -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | DRF-001 | Active Discovery | โ€” | Expose discovery endpoint; respond to interrogation queries | Configure discovery schedules; manage Discovered Store | PRV-005 | | DRF-002 | Drift Comparison | Receive drift notifications for owned resources | โ€” | Configure drift detection policies; manage comparison logic | DRF-001, PRV-005 | @@ -105,7 +101,7 @@ weight: 10 ## 7. Policy Management -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | POL-001 | Policy Authoring | โ€” | Contribute provider-specific policy rules | Author and manage policies in GitOps store | IAM-003 | | POL-002 | Policy Validation and Shadow Mode | View shadow evaluation results on own requests | โ€” | Configure shadow mode; review shadow results in Validation Store | POL-001 | @@ -119,7 +115,7 @@ weight: 10 ## 8. Data Layer Management -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | LAY-001 | Core Layer Authoring | โ€” | โ€” | Author and manage Core and Organizational Layers in GitOps | IAM-003 | | LAY-002 | Service Layer Contribution | โ€” | Contribute Service Layers for offered resource types | Manage layer compatibility declarations | PRV-001, LAY-001 | @@ -131,7 +127,7 @@ weight: 10 ## 9. Information and Data Integration -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | INF-001 | Information Provider Registration | โ€” | Register Information Provider; declare authority scope and schema | Configure Information Provider Registry; manage authority layers | IAM-001 | | INF-002 | Information Provider Push | โ€” | Push field value updates to DCM; respond to conflict notifications | Configure ingestion pipeline; manage conflict resolution policies | INF-001 | @@ -144,7 +140,7 @@ weight: 10 ## 10. Ingestion and Brownfield Management -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | ING-001 | Resource Discovery and Ingestion | โ€” | Expose discovery endpoints for brownfield resources | Configure ingestion pipeline; manage __transitional__ Tenant | DRF-001 | | ING-002 | Ingested Entity Review | โ€” | โ€” | Review ingested entities; resolve conflicts; promote to active Tenants | ING-001 | @@ -155,7 +151,7 @@ weight: 10 ## 11. Audit and Compliance -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | AUD-001 | Audit Trail Access | Query audit records for own resources | โ€” | Configure Audit Store; manage retention policies | IAM-003 | | AUD-002 | Compliance Reporting | โ€” | โ€” | Generate compliance reports; manage report schedules | AUD-001 | @@ -167,7 +163,7 @@ weight: 10 ## 12. Observability and Operations -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | OBS-001 | Operational Dashboard | View health and status of own resources | โ€” | Configure and manage observability dashboard | โ€” | | OBS-002 | Metrics and Telemetry Export | โ€” | Expose resource-level metrics to DCM | Configure observability export; integrate enterprise observability platform | โ€” | @@ -179,7 +175,7 @@ weight: 10 ## 13. Storage and State Management -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | STO-001 | GitOps Store Management | โ€” | โ€” | Configure and manage Intent and Requested Stores; manage Git repository structure | โ€” | | STO-002 | Realized State Store Management | โ€” | โ€” | Configure Event Stream and Realized Store; manage retention | PRV-005 | @@ -192,7 +188,7 @@ weight: 10 ## 14. DCM Federation and Multi-Instance -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | FED-001 | DCM Provider Registration | Submit requests that are routed to peer DCMs | Register as DCM Provider in peer instances | Configure DCM Provider registrations; manage federation trust | PRV-001, IAM-001 | | FED-002 | Federation Routing | Requests automatically routed to appropriate Regional/Sovereign DCM | Respond to reserve queries from Hub DCM | Configure federation placement policies; manage sovereignty pre-filters | FED-001, REQ-005 | @@ -204,7 +200,7 @@ weight: 10 ## 15. Platform Governance and Administration -| ID | Capability | Consumer | Producer | Platform/Admin | Depends On | +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| | GOV-001 | Tenant Management | โ€” | โ€” | Create, configure, and decommission Tenants; manage compliance overlays | IAM-007 | | GOV-002 | Group Management | โ€” | โ€” | Create and manage DCM Groups; configure sovereignty rules; manage time-bounded memberships | IAM-003 | @@ -216,6 +212,72 @@ weight: 10 --- +## 16. Accreditation Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ACC-001 | Accreditation Submission | โ€” | Submit accreditation records (BAA, ISO 27001, FedRAMP, etc.) referencing external certificate evidence | Register accrediting bodies; configure minimum accreditation types per profile | PRV-001 | +| ACC-002 | Accreditation Review and Approval | โ€” | Receive approval/rejection notification | Review submitted accreditations; verify certificate references; approve or reject via Admin API | ACC-001 | +| ACC-003 | Accreditation Lifecycle Monitoring | Receive notification when a provider's accreditation is nearing expiry or revoked | Renew accreditations before expiry; submit renewal documentation | Monitor expiry timelines; fire P90D renewal warnings; handle accreditation gaps | ACC-001 | +| ACC-004 | Accreditation Gap Response | Receive notification when a provider enters accreditation gap affecting owned resources | โ€” | Configure Recovery Policy for accreditation gap events; manage affected entity remediation | ACC-003, POL-005 | +| ACC-005 | Data Classification Enforcement | Receive enforcement feedback when request payload contains data the selected provider cannot handle | Declare max_data_classification_accepted in capability registration | Configure classification immutability rules; manage phi/sovereign classification locks | ACC-001, PRV-001 | +| ACC-006 | DCM Deployment Accreditation | โ€” | โ€” | Register DCM deployment-level accreditations; expose to federation peers for trust verification | PRV-001, FED-001 | + +--- + +## 17. Zero Trust and Security Posture + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ZTS-001 | Mutual TLS Enforcement | All interactions authenticated via mTLS at the client side | Present valid mTLS certificate on every interaction; rotate certificates on declared schedule | Configure trust anchors; manage CA chain; enforce mTLS at all interaction boundaries | IAM-001 | +| ZTS-002 | Scoped Interaction Credentials | Receive scoped short-lived credentials for authorized operations | Validate credential scope before executing operations; reject out-of-scope credentials | Configure credential lifetime per profile; manage credential issuance via Credential Provider | IAM-001, PRV-001 | +| ZTS-003 | Certificate Rotation Management | โ€” | Implement certificate rotation before expiry; use transition window to avoid downtime | Monitor certificate expiry; fire P14D rotation warnings; manage P7D transition window | ZTS-001 | +| ZTS-004 | Zero Trust Posture Configuration | โ€” | โ€” | Configure zero_trust_posture per profile (none/boundary/full/hardware_attested); manage posture overrides | POL-005 | +| ZTS-005 | Hardware Attestation (Sovereign Profile) | โ€” | Present hardware-attested identity (TPM/HSM) for sovereign profile interactions | Configure hardware attestation requirements; manage HSM integration; enforce for sovereign profile | ZTS-001, ZTS-002 | +| ZTS-006 | Five-Check Boundary Enforcement | โ€” | Pass all five boundary checks on every interaction: identity โ†’ authorization โ†’ accreditation โ†’ matrix โ†’ sovereignty | Monitor boundary check audit records; respond to INTERACTION_DENIED events | ZTS-001, ACC-001, GMX-001 | + +--- + +## 18. Unified Governance Matrix + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| GMX-001 | Governance Matrix Rule Authoring | โ€” | โ€” | Author governance matrix rules in GitOps; declare match conditions across four axes (subject/data/target/context); declare field permissions | POL-001 | +| GMX-002 | Boundary Enforcement Evaluation | Receive DENY response with governing rule_uuid and human-readable reason when a request crosses a prohibited boundary | Receive field-stripped or redacted payloads when STRIP_FIELD/REDACT decisions apply | Monitor GMX evaluation audit records; respond to DENY events | GMX-001, ZTS-006 | +| GMX-003 | Field-Level Data Control | Receive request feedback when specific payload fields are stripped or redacted by active matrix rules | Receive filtered payloads; handle missing optional fields gracefully | Configure allowlist/blocklist field permissions per rule; manage STRIP_FIELD vs REDACT vs DENY_REQUEST escalation | GMX-001 | +| GMX-004 | Sovereignty Zone Management | โ€” | Declare operating sovereignty zones in provider registration | Register sovereignty zones; declare jurisdictions, regulatory frameworks, inter-zone agreements | PRV-001, GMX-001 | +| GMX-005 | Compliance Domain Matrix Activation | โ€” | โ€” | Activate compliance domain matrix rules (HIPAA, GDPR, etc.) by enabling compliance domain in profile; rules apply automatically | POL-005, GMX-001 | +| GMX-006 | Tenant and Resource-Type Matrix Overrides | โ€” | โ€” | Declare Tenant-level and resource-type-level matrix rules that tighten (never relax) platform defaults | GMX-001, GOV-001 | +| GMX-007 | Matrix Rule Lifecycle Management | โ€” | โ€” | Manage governance matrix rule lifecycle (developing โ†’ proposed โ†’ active); use shadow mode for safe validation before activation | GMX-001, POL-002 | + +--- + +## 19. Drift Reconciliation + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| DRC-001 | Drift Record Production | Receive drift records with field-level detail: realized value, discovered value, field criticality, severity, unsanctioned flag | โ€” | Configure Drift Reconciliation Component; manage comparison algorithm and severity thresholds | DRF-001, PRV-005 | +| DRC-002 | Unsanctioned Change Classification | Receive elevated-severity alert when change has no corresponding Requested State record | โ€” | Configure unsanctioned change detection; manage severity escalation rules | DRC-001 | +| DRC-003 | Drift Severity Classification | Receive severity-classified drift records (minor/significant/critical) based on field criticality ร— change magnitude | โ€” | Declare field criticality in Resource Type Specifications; configure magnitude thresholds per profile | DRC-001, GOV-003 | +| DRC-004 | Drift Resolution Tracking | View drift record status (open/acknowledged/resolved/escalated); receive resolved notification when next discovery confirms clean state | โ€” | Monitor drift resolution rates; configure escalation policies for aged-open drift records | DRC-001, DRF-004 | +| DRC-005 | Governance Matrix Drift Integration | โ€” | โ€” | Configure governance matrix check in drift comparison pipeline: expected provider changes are not flagged as drift | DRC-001, GMX-001 | + +--- + +## 20. Cluster as a Service + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| CAS-001 | Cluster Catalog Item | Request a full Kubernetes cluster as a catalog item; own the cluster entity and all resources within it | Implement Platform.KubernetesCluster Service Provider (CAPI-based or managed K8s); compose cluster constituents as Meta Provider (compute + network + storage + DNS + credentials) | Register Cluster-as-a-Service provider; configure Platform.KubernetesCluster Resource Type Spec | CAT-001, PRV-009, PRV-001 | +| CAS-002 | Cluster Sovereignty Placement | Request cluster with sovereignty constraints (jurisdiction, compliance domain); cluster placed on provider satisfying constraints | Declare sovereignty zones and accreditations for cluster provider | Configure placement constraints for cluster resource type; manage cluster provider sovereignty registry | CAS-001, GMX-004, REQ-005 | +| CAS-003 | Cluster Lifecycle Management | Manage cluster lifecycle: scale nodes, upgrade Kubernetes version, suspend, decommission | Handle cluster lifecycle payloads; report cluster health and version | Configure cluster lifecycle policies; manage cluster decommission with workload drain | CAS-001, LCM-001 | +| CAS-004 | Cluster as Nested Provider Registration | After cluster provisioned, register it as a Service Provider for workload resources; request workloads against own cluster through same DCM catalog | Cluster registers as workload Service Provider following unified Provider base contract | Configure nested provider registration; manage workload resource type scoping to owning Tenant | CAS-001, PRV-001 | +| CAS-005 | Cluster-Scoped Resource Ownership | Access cluster-scoped resources (ClusterRoles, StorageClasses, PersistentVolumes) as part of owned cluster entity; resources owned by requesting Tenant | Report cluster-scoped resources in Realized State payload | Distinguish owned-cluster cluster-scoped resources (Tenant-owned) from shared-cluster infrastructure resources (__platform__-owned) | CAS-001, GOV-001 | +| CAS-006 | Kubeconfig and Credential Management | Receive kubeconfig via Credential Provider after cluster provisioned; credentials scoped and short-lived | Issue kubeconfig credentials via Credential Provider integration; rotate on declared schedule | Configure Credential Provider for kubeconfig issuance; manage credential scope and TTL | CAS-001, ZTS-002 | + +--- + + ## Capability Count Summary | Domain | Capabilities | @@ -235,7 +297,12 @@ weight: 10 | Storage and State Management | 6 | | DCM Federation and Multi-Instance | 5 | | Platform Governance and Administration | 7 | -| **Total** | **100** | +| Accreditation Management | 6 | +| Zero Trust and Security Posture | 6 | +| Unified Governance Matrix | 7 | +| Drift Reconciliation | 5 | +| Cluster as a Service | 6 | +| **Total** | **130** | --- From ddd2dd440b4042f7541c80dcd077a91fe6149a1b Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 15:32:29 -0500 Subject: [PATCH 31/49] Removed Cluster as a service as a core DCM capability, should just be an example use case. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 7 +++---- content/docs/capabilities-matrix.md | 12 +----------- 2 files changed, 4 insertions(+), 15 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index a2aa70a..4ac3ec9 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -3943,9 +3943,9 @@ All 21 previously open community/implementation questions are now resolved. Key ### Operator Interface Specification (6 resolved) - **CNCF submission:** specification project (not sandbox project requiring implementation); SIG engagement first - **Conformance certification:** self-certified via automated test suite (low friction gate) + optional DCM Verified badge via project review -- **Cluster-scoped resources โ€” two models:** (A) **Cluster as a Service (primary):** Tenant requests and owns an entire `Platform.KubernetesCluster` catalog item; Tenant owns all cluster-scoped resources within that cluster; cluster is the ownership boundary; (B) **Shared cluster infrastructure (exception):** cluster-scoped resources governing shared multi-tenant cluster infrastructure belong to `__platform__` Tenant. Cluster-as-a-Service is the expected primary model โ€” users and Tenant owners request and own clusters through the catalog the same way they request VMs +- **Cluster-scoped resources โ€” two models:** (A) **Cluster as a catalog item (example):** When a Service Provider offers Kubernetes clusters as a resource type, a Tenant that owns a provisioned cluster entity owns all cluster-scoped resources within it โ€” the cluster entity is the ownership boundary; (B) **Shared cluster infrastructure:** cluster-scoped resources governing shared multi-tenant cluster infrastructure belong to `__platform__` Tenant. Note: Cluster-as-a-Service is an example Service Provider implementation, not a DCM architectural feature โ€” DCM treats the cluster as any other resource entity - **Non-Go frameworks:** spec is language-agnostic; Go SDK is reference implementation; community Java/Python SDKs encouraged; not maintained by DCM project in v1 -- **Cluster API / Cluster as a Service:** `Platform.KubernetesCluster` is a first-class catalog item โ€” Tenants request and own clusters through the service catalog; CAPI operator or managed K8s service registers as Service Provider; provisioned cluster is a full Tenant-owned entity; it can then register as a nested Service Provider for workload resources (DCM provisions cluster โ†’ cluster becomes workload provider โ†’ Tenant manages workloads via same DCM catalog); Meta Provider composes compute + network + storage + DNS + credentials +- **Cluster API as an example Service Provider:** A CAPI-based operator can register as a Service Provider for a `Platform.KubernetesCluster` resource type โ€” this is an example of what DCM's Provider model enables, not a special architectural feature. DCM has no built-in knowledge of Kubernetes; a CAPI Service Provider is structurally identical to any other Service Provider. Once provisioned, the cluster entity can optionally register as a nested Service Provider for workload resources (the Meta Provider pattern โ€” composing compute + network + storage + DNS + credentials) - **Level 0:** exists โ€” label-based passive discovery, no operator code changes; DCM discovers and tracks but does not control; lowest adoption friction ### Operator SDK (5 resolved) @@ -4058,7 +4058,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl ## SECTION 59b โ€” CAPABILITIES MATRIX UPDATE (130 capabilities, 20 domains) -Five new domains added to the capabilities matrix. Total: 130 capabilities across 20 domains. +Five new domains added to the capabilities matrix. Total: 119 capabilities across 19 domains. **New domains:** @@ -4070,7 +4070,6 @@ Five new domains added to the capabilities matrix. Total: 130 capabilities acros **19. Drift Reconciliation (DRC-001โ€“005):** Drift record production with field-level detail, unsanctioned change classification and severity escalation, drift severity classification (field criticality ร— change magnitude), drift resolution tracking with clean-state confirmation, governance matrix integration for expected provider change detection. -**20. Cluster as a Service (CAS-001โ€“006):** Cluster catalog item (Tenant owns entire cluster entity), cluster sovereignty placement, cluster lifecycle management (scale/upgrade/decommission), cluster as nested provider registration (DCM provisions cluster โ†’ cluster becomes workload Service Provider), cluster-scoped resource ownership model (Tenant-owned within cluster vs __platform__-owned shared infrastructure), kubeconfig and credential management via Credential Provider. **6 specifications graduated from WIP to Draft:** dcm-registration-spec.md ยท dcm-opa-integration-spec.md ยท 11-kubernetes-compatibility.md ยท dcm-operator-interface-spec.md ยท dcm-operator-sdk-api.md ยท dcm-admin-api-spec.md diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index d2126f6..074ccfb 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -264,16 +264,7 @@ --- -## 20. Cluster as a Service -| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | -|----|-----------|---------|---------|---------------|-----------| -| CAS-001 | Cluster Catalog Item | Request a full Kubernetes cluster as a catalog item; own the cluster entity and all resources within it | Implement Platform.KubernetesCluster Service Provider (CAPI-based or managed K8s); compose cluster constituents as Meta Provider (compute + network + storage + DNS + credentials) | Register Cluster-as-a-Service provider; configure Platform.KubernetesCluster Resource Type Spec | CAT-001, PRV-009, PRV-001 | -| CAS-002 | Cluster Sovereignty Placement | Request cluster with sovereignty constraints (jurisdiction, compliance domain); cluster placed on provider satisfying constraints | Declare sovereignty zones and accreditations for cluster provider | Configure placement constraints for cluster resource type; manage cluster provider sovereignty registry | CAS-001, GMX-004, REQ-005 | -| CAS-003 | Cluster Lifecycle Management | Manage cluster lifecycle: scale nodes, upgrade Kubernetes version, suspend, decommission | Handle cluster lifecycle payloads; report cluster health and version | Configure cluster lifecycle policies; manage cluster decommission with workload drain | CAS-001, LCM-001 | -| CAS-004 | Cluster as Nested Provider Registration | After cluster provisioned, register it as a Service Provider for workload resources; request workloads against own cluster through same DCM catalog | Cluster registers as workload Service Provider following unified Provider base contract | Configure nested provider registration; manage workload resource type scoping to owning Tenant | CAS-001, PRV-001 | -| CAS-005 | Cluster-Scoped Resource Ownership | Access cluster-scoped resources (ClusterRoles, StorageClasses, PersistentVolumes) as part of owned cluster entity; resources owned by requesting Tenant | Report cluster-scoped resources in Realized State payload | Distinguish owned-cluster cluster-scoped resources (Tenant-owned) from shared-cluster infrastructure resources (__platform__-owned) | CAS-001, GOV-001 | -| CAS-006 | Kubeconfig and Credential Management | Receive kubeconfig via Credential Provider after cluster provisioned; credentials scoped and short-lived | Issue kubeconfig credentials via Credential Provider integration; rotate on declared schedule | Configure Credential Provider for kubeconfig issuance; manage credential scope and TTL | CAS-001, ZTS-002 | --- @@ -301,8 +292,7 @@ | Zero Trust and Security Posture | 6 | | Unified Governance Matrix | 7 | | Drift Reconciliation | 5 | -| Cluster as a Service | 6 | -| **Total** | **130** | +| **Total** | **119** | --- From 46ebeb096a9561f7f87efa6fbdbea42acffd5718 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 15:36:37 -0500 Subject: [PATCH 32/49] Synching up ai prompt. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 102 ++++++++++++------------- 1 file changed, 51 insertions(+), 51 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 4ac3ec9..2d67a47 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -3559,54 +3559,6 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc --- -## SECTION 59 โ€” EXAMPLES AND USE CASES (dcm-examples.md) - -### Orchestration Examples (8 scenarios) - -**1.1 Basic request lifecycle** โ€” submit โ†’ layers_assembled (GateKeeper + Transformation fire) โ†’ placement (6-step) โ†’ dispatch โ†’ realized. Shows named workflow + dynamic policies composing on same events. - -**1.2 Human approval gate** โ€” GateKeeper with `requires_approval: true` flag inserts AWAITING_APPROVAL step without modifying named workflow. Manager approves via API โ†’ pipeline resumes. - -**1.3 Policy-gated hard block** โ€” GateKeeper denies unsupported OS. Consumer receives clear error with policy_uuid and suggestion. No requires_approval flag โ†’ terminal FAILED. - -**1.4 Compound service (Meta Provider)** โ€” VM + IP + DNS + LoadBalancer. Dependency-ordered execution (parallel where no deps). DNS fails (partial delivery) โ†’ DEGRADED state. Recovery: NOTIFY_AND_WAIT. Consumer chooses: accept degraded or trigger DNS retry. - -**1.5 Drift detection + remediation** โ€” Discovery finds memory_gb changed (unsanctioned). Drift: significant + unsanctioned โ†’ critical. Policy: ESCALATE. Consumer submits REVERT โ†’ new request cycle โ†’ next discovery clean. - -**1.6 Dispatch timeout + late response** โ€” Provider silent for PT30M โ†’ TIMEOUT_PENDING โ†’ Recovery: NOTIFY_AND_WAIT (prod profile). Provider responds at T+45M โ†’ LATE_RESPONSE_RECEIVED. Consumer chooses DISCARD_AND_REQUEUE. - -**1.7 Federation-routed request** โ€” Local providers at capacity. Placement queries Hub DCM (Peer DCM provider). Hub routes to Regional DCM B. Governance Matrix checked at each hop. Realized State flows back chain. entity_uuid preserved. - -**1.8 Brownfield ingestion** โ€” Discovery finds unmanaged VM. Orchestration Flow Policy: discover โ†’ INGEST โ†’ ENRICH (CMDB query) โ†’ await operator โ†’ PROMOTE to tenant. Drift detection activated post-promotion. - -### Provider Examples (4 scenarios) - -**2.1 Service Provider dispatch cycle** โ€” Full payload showing DCM unified format โ†’ naturalize to OpenStack Nova โ†’ execute โ†’ denaturalize back. Shows provenance on injected fields (monitoring_endpoint from policy). - -**2.2 Information Provider enrichment** โ€” CMDB query during layer assembly. Response with confidence descriptor. Fields injected with source_type: information_provider and source_uuid. - -**2.3 Policy Provider Mode 3 (OPA sidecar)** โ€” Exact OPA HTTP API call format, input document structure, response parsing. - -**2.4 Notification Provider delivery** โ€” VLAN decommission event. Audience: owner (NetworkOps) + 2 stakeholders (required stakes) + 1 observer (optional stake). Per-actor envelopes with stakeholder_reason field. Slack message format. - -### Consumer API Examples (2 scenarios) - -**3.1 Complete request lifecycle** โ€” catalog browse โ†’ describe (see constraints) โ†’ submit โ†’ poll status sequence โ†’ get realized resource with confidence scores. - -**3.2 Provider update approval** โ€” Provider submits auto-scale notification โ†’ REQUIRES_CONSUMER_APPROVAL โ†’ consumer reviews pending notifications โ†’ approve โ†’ new Realized State. - -### Admin API Examples (2 scenarios) - -**4.1 Provider registration review** โ€” List pending registrations (with validation results) โ†’ approve with review notes. - -**4.2 Orphan resolution** โ€” List orphan candidates โ†’ investigate โ†’ adopt_into_dcm โ†’ entity promoted to full lifecycle. - -### Registration Flow Example (1 scenario) - -**5.1 Complete provider onboarding** โ€” Admin issues registration token โ†’ provider submits registration payload (mTLS + token) โ†’ 8 automated validation checks shown โ†’ PENDING_APPROVAL โ†’ admin reviews โ†’ ACTIVE. Full capability declaration structure for Service Provider. - ---- - ## SECTION 54 โ€” TERMINOLOGY GLOSSARY @@ -4056,7 +4008,55 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 59b โ€” CAPABILITIES MATRIX UPDATE (130 capabilities, 20 domains) +## SECTION 57 โ€” EXAMPLES AND USE CASES (dcm-examples.md) + +### Orchestration Examples (8 scenarios) + +**1.1 Basic request lifecycle** โ€” submit โ†’ layers_assembled (GateKeeper + Transformation fire) โ†’ placement (6-step) โ†’ dispatch โ†’ realized. Shows named workflow + dynamic policies composing on same events. + +**1.2 Human approval gate** โ€” GateKeeper with `requires_approval: true` flag inserts AWAITING_APPROVAL step without modifying named workflow. Manager approves via API โ†’ pipeline resumes. + +**1.3 Policy-gated hard block** โ€” GateKeeper denies unsupported OS. Consumer receives clear error with policy_uuid and suggestion. No requires_approval flag โ†’ terminal FAILED. + +**1.4 Compound service (Meta Provider)** โ€” VM + IP + DNS + LoadBalancer. Dependency-ordered execution (parallel where no deps). DNS fails (partial delivery) โ†’ DEGRADED state. Recovery: NOTIFY_AND_WAIT. Consumer chooses: accept degraded or trigger DNS retry. + +**1.5 Drift detection + remediation** โ€” Discovery finds memory_gb changed (unsanctioned). Drift: significant + unsanctioned โ†’ critical. Policy: ESCALATE. Consumer submits REVERT โ†’ new request cycle โ†’ next discovery clean. + +**1.6 Dispatch timeout + late response** โ€” Provider silent for PT30M โ†’ TIMEOUT_PENDING โ†’ Recovery: NOTIFY_AND_WAIT (prod profile). Provider responds at T+45M โ†’ LATE_RESPONSE_RECEIVED. Consumer chooses DISCARD_AND_REQUEUE. + +**1.7 Federation-routed request** โ€” Local providers at capacity. Placement queries Hub DCM (Peer DCM provider). Hub routes to Regional DCM B. Governance Matrix checked at each hop. Realized State flows back chain. entity_uuid preserved. + +**1.8 Brownfield ingestion** โ€” Discovery finds unmanaged VM. Orchestration Flow Policy: discover โ†’ INGEST โ†’ ENRICH (CMDB query) โ†’ await operator โ†’ PROMOTE to tenant. Drift detection activated post-promotion. + +### Provider Examples (4 scenarios) + +**2.1 Service Provider dispatch cycle** โ€” Full payload showing DCM unified format โ†’ naturalize to OpenStack Nova โ†’ execute โ†’ denaturalize back. Shows provenance on injected fields (monitoring_endpoint from policy). + +**2.2 Information Provider enrichment** โ€” CMDB query during layer assembly. Response with confidence descriptor. Fields injected with source_type: information_provider and source_uuid. + +**2.3 Policy Provider Mode 3 (OPA sidecar)** โ€” Exact OPA HTTP API call format, input document structure, response parsing. + +**2.4 Notification Provider delivery** โ€” VLAN decommission event. Audience: owner (NetworkOps) + 2 stakeholders (required stakes) + 1 observer (optional stake). Per-actor envelopes with stakeholder_reason field. Slack message format. + +### Consumer API Examples (2 scenarios) + +**3.1 Complete request lifecycle** โ€” catalog browse โ†’ describe (see constraints) โ†’ submit โ†’ poll status sequence โ†’ get realized resource with confidence scores. + +**3.2 Provider update approval** โ€” Provider submits auto-scale notification โ†’ REQUIRES_CONSUMER_APPROVAL โ†’ consumer reviews pending notifications โ†’ approve โ†’ new Realized State. + +### Admin API Examples (2 scenarios) + +**4.1 Provider registration review** โ€” List pending registrations (with validation results) โ†’ approve with review notes. + +**4.2 Orphan resolution** โ€” List orphan candidates โ†’ investigate โ†’ adopt_into_dcm โ†’ entity promoted to full lifecycle. + +### Registration Flow Example (1 scenario) + +**5.1 Complete provider onboarding** โ€” Admin issues registration token โ†’ provider submits registration payload (mTLS + token) โ†’ 8 automated validation checks shown โ†’ PENDING_APPROVAL โ†’ admin reviews โ†’ ACTIVE. Full capability declaration structure for Service Provider. + +--- + +## SECTION 58 โ€” CAPABILITIES MATRIX UPDATE (119 capabilities, 19 domains) Five new domains added to the capabilities matrix. Total: 119 capabilities across 19 domains. @@ -4080,7 +4080,7 @@ consumer-api-spec.md (missing complete lifecycle endpoint coverage) ยท dcm-flow- --- -## SECTION 57 โ€” DOCUMENTATION STRUCTURE +## SECTION 59 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -4128,7 +4128,7 @@ content/ --- -## SECTION 58 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 60 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: From 05846327ac84965048b23a46b935289a87b37714 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 22:40:02 -0500 Subject: [PATCH 33/49] Completing consumer API Signed-off-by: Chris Roadfeldt --- .../specifications/consumer-api-spec.md | 723 +++++++++++++++++- 1 file changed, 713 insertions(+), 10 deletions(-) diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index 2219a76..c175551 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -4,16 +4,12 @@ type: docs weight: 0 --- -> **โš ๏ธ Work in Progress** +> **๐Ÿ“‹ Draft** > -> **This specification is a work in progress and is less mature than the core DCM data model documentation.** API endpoint paths, request/response structures, and authentication flows represent design intent and will be refined as implementation proceeds. -> -> **Do not build against this specification yet.** It is published to share design direction and invite feedback. -> -> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). +> This specification covers the full Consumer API surface. Endpoint paths, request/response structures, and authentication flows represent design intent and will be refined as implementation proceeds. Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). **Version:** 0.1.0-draft -**Status:** Design โ€” Not yet implemented +**Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification **Related Documents:** [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Four States](../data-model/02-four-states.md) | [Auth Providers](../data-model/19-auth-providers.md) | [Webhooks and Messaging](../data-model/18-webhooks-messaging.md) @@ -748,6 +744,708 @@ Response 202 Accepted: **Note:** Recovery decisions are only available when the active recovery profile includes `NOTIFY_AND_WAIT`. With other profiles (automated-reconciliation, discard-and-requeue) the system acts automatically and no decision endpoint is exposed. + + +### 5.13 Bulk Decommission + +Decommissions all resources matching a filter. Creates individual decommission requests for each resource. Useful for teardown of environments or project cleanup. + +``` +POST /api/v1/resources/bulk-decommission + +Request body: +{ + "filter": { + "group_uuid": "", # all resources in a group + "tag": "environment:dev", # all resources with a tag + "resource_type": "Compute.VirtualMachine" # combined with other filters + }, + "reason": "Dev environment teardown โ€” project complete", + "dry_run": true, # true: return what would be decommissioned; no action taken + "force": false +} + +Response 200 (dry_run=true): +{ + "dry_run": true, + "would_decommission": [ + { "entity_uuid": "", "display_name": "dev-vm-01", "resource_type": "Compute.VirtualMachine" }, + { "entity_uuid": "", "display_name": "dev-vm-02", "resource_type": "Compute.VirtualMachine" } + ], + "blocked": [ + { + "entity_uuid": "", + "display_name": "shared-vlan-01", + "reason": "Active required stakes from resources outside the decommission set" + } + ] +} + +Response 202 Accepted (dry_run=false): +{ + "bulk_decommission_uuid": "", + "decommission_requests": [ + { "entity_uuid": "", "request_uuid": "" }, + { "entity_uuid": "", "request_uuid": "" } + ], + "blocked_count": 1 +} +``` + + +### 5.9 Resume Resource + +Resumes a suspended resource. The resource must be in SUSPENDED lifecycle state. + +``` +POST /api/v1/resources/{entity_uuid}/resume + +Request body: +{ + "reason": "Maintenance window complete" +} + +Response 202 Accepted: +{ + "entity_uuid": "", + "status": "RESUMING" +} + +Response 409 Conflict: +{ + "error": "not_suspended", + "reason": "Resource is not in SUSPENDED state", + "current_state": "OPERATIONAL" +} +``` + +--- + +### 5.10 Ownership Transfer + +Transfers ownership of a resource entity to a different Tenant. Both Tenants must have an active cross-tenant authorization record permitting the transfer. The receiving Tenant admin must confirm the transfer. + +``` +POST /api/v1/resources/{entity_uuid}/transfer + +Request body: +{ + "target_tenant_uuid": "", + "reason": "Project moving from Dev to Production Tenant", + "notify_target_tenant_admin": true +} + +Response 202 Accepted: +{ + "transfer_uuid": "", + "entity_uuid": "", + "from_tenant_uuid": "", + "to_tenant_uuid": "", + "status": "PENDING_TARGET_ACCEPTANCE", + "expires_at": "" # transfer offer expires after PT72H +} + +Response 403 Forbidden: +{ + "error": "transfer_not_authorized", + "reason": "No cross-tenant authorization record between source and target Tenant" +} +``` + +Target Tenant admin accepts or rejects: + +``` +POST /api/v1/resources/transfers/{transfer_uuid}/accept +POST /api/v1/resources/transfers/{transfer_uuid}/reject +{ + "reason": "" +} +``` + +--- + +### 5.11 Extend Resource TTL + +Extends the TTL of a resource entity that has a lifecycle time constraint declared. Extension is subject to policy โ€” a GateKeeper may reject or cap the extension. + +``` +POST /api/v1/resources/{entity_uuid}/extend-ttl + +Request body: +{ + "extend_by": "P30D", # ISO 8601 duration + "reason": "Project deadline extended by one month" +} + +Response 200: +{ + "entity_uuid": "", + "previous_expiry": "", + "new_expiry": "", + "extension_granted": "P30D" +} + +Response 422 Unprocessable: +{ + "error": "ttl_extension_rejected", + "reason": "Policy limits maximum TTL extension to P14D for this resource type", + "max_extension": "P14D", + "policy_uuid": "" +} + +Response 404 Not Found: +{ + "error": "no_ttl_constraint", + "reason": "Resource has no declared lifecycle time constraint" +} +``` + +--- + +### 5.12 List Expiring Resources + +Returns resources approaching their TTL expiry, sorted by time remaining. + +``` +GET /api/v1/resources/expiring + +Query parameters: + within= resources expiring within this duration (default: P7D) + resource_type= + page= + page_size= + +Response 200: +{ + "expiring_resources": [ + { + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "display_name": "lab-server-01", + "expires_at": "", + "time_remaining": "P2DT4H", + "on_expiry_action": "decommission", + "extend_url": "/api/v1/resources/{entity_uuid}/extend-ttl" + } + ], + "total": 3 +} +``` + +--- + +## 5b. Drift Management + +### 5b.1 List Drift Records for a Resource + +``` +GET /api/v1/resources/{entity_uuid}/drift + +Query parameters: + status= + severity= + page= + page_size= + +Response 200: +{ + "drift_records": [ + { + "drift_uuid": "", + "detected_at": "", + "overall_severity": "significant", + "unsanctioned": true, + "status": "open", + "drifted_fields": [ + { + "field_path": "fields.memory_gb", + "realized_value": 8, + "discovered_value": 16, + "field_severity": "significant" + } + ], + "available_actions": ["REVERT", "ACCEPT_DRIFT", "ESCALATE"] + } + ], + "total": 1 +} +``` + +### 5b.2 Acknowledge Drift Record + +Marks a drift record as acknowledged. The entity remains drifted โ€” this signals the owner has reviewed it. + +``` +POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}/acknowledge +{ + "reason": "Reviewing with provider before deciding on action" +} + +Response 200: +{ + "drift_uuid": "", + "status": "acknowledged", + "acknowledged_at": "" +} +``` + +### 5b.3 Accept Drift (Update Definition) + +Accepts the discovered state as the new authoritative desired state. Creates a new Requested State and Realized State snapshot reflecting the discovered values. Resolves the drift record. + +``` +POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}/accept +{ + "accept_all_fields": true, # accept all drifted fields + "accept_fields": ["fields.memory_gb"], # or select specific fields + "reason": "Auto-scale event was legitimate; accepting new memory configuration" +} + +Response 202 Accepted: +{ + "drift_uuid": "", + "status": "resolved", + "resolution_type": "updated_definition", + "new_realized_state_uuid": "" +} +``` + +### 5b.4 Revert Drift + +Submits a revert request โ€” dispatches a new request to restore the resource to its Realized State values. + +``` +POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}/revert +{ + "reason": "Unauthorized change โ€” reverting to declared state" +} + +Response 202 Accepted: +{ + "drift_uuid": "", + "revert_request_uuid": "", + "status": "DISPATCHED", + "status_url": "/api/v1/requests/{revert_request_uuid}/status" +} +``` + +--- + +## 5c. Groups and Relationships + +### 5c.1 List Resource Groups + +Returns all Resource Groups in the actor's Tenant. + +``` +GET /api/v1/groups + +Query parameters: + group_class= + tag= + page= + page_size= + +Response 200: +{ + "groups": [ + { + "group_uuid": "", + "handle": "tenant/payments/prod-vms", + "display_name": "Production VMs โ€” Payments", + "group_class": "resource_grouping", + "member_count": 12, + "tags": ["production", "payments"] + } + ], + "total": 4 +} +``` + +### 5c.2 Describe Group + +``` +GET /api/v1/groups/{group_uuid} + +Response 200: +{ + "group_uuid": "", + "handle": "tenant/payments/prod-vms", + "display_name": "Production VMs โ€” Payments", + "group_class": "resource_grouping", + "members": [ + { + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "display_name": "payments-api-01", + "membership_valid_until": null # null = permanent membership + } + ], + "tags": ["production", "payments"] +} +``` + +### 5c.3 Add Resource to Group + +``` +POST /api/v1/groups/{group_uuid}/members +{ + "entity_uuid": "", + "valid_until": "2026-12-31T23:59:59Z" # optional; null = permanent +} + +Response 201 Created: +{ + "group_uuid": "", + "entity_uuid": "", + "membership_created_at": "" +} +``` + +### 5c.4 Remove Resource from Group + +``` +DELETE /api/v1/groups/{group_uuid}/members/{entity_uuid} + +Response 204 No Content +``` + +### 5c.5 View Resource Relationships + +``` +GET /api/v1/resources/{entity_uuid}/relationships + +Query parameters: + relationship_type= filter by relationship type + direction= default: both + +Response 200: +{ + "relationships": [ + { + "relationship_uuid": "", + "relationship_type": "attached_to", + "direction": "outbound", + "related_entity_uuid": "", + "related_entity_type": "Network.VLAN", + "related_entity_display_name": "VLAN-100", + "stake_strength": "required", + "nature": "operational" + } + ], + "total": 3 +} +``` + +--- + +## 6b. Requests Management + +### 6b.1 List Requests + +``` +GET /api/v1/requests + +Query parameters: + status= filter by lifecycle status (see 4.3) + resource_type= + from= + to= + page= + page_size= + +Response 200: +{ + "requests": [ + { + "request_uuid": "", + "entity_uuid": "", + "catalog_item_uuid": "", + "resource_type": "Compute.VirtualMachine", + "status": "COMPLETED", + "submitted_at": "", + "completed_at": "" + } + ], + "total": 47 +} +``` + +### 6b.2 List Pending Approvals (as Approver) + +Returns requests awaiting approval where the authenticated actor is an eligible approver (by role or group membership). + +``` +GET /api/v1/approvals/pending + +Response 200: +{ + "pending_approvals": [ + { + "approval_uuid": "", + "request_uuid": "", + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "requester": { "uuid": "", "display_name": "Bob Smith" }, + "estimated_cost_per_month": 230.40, + "submitted_at": "", + "deadline": "", + "policy_name": "prod-vm-approval-gate" + } + ], + "total": 2 +} +``` + +### 6b.3 Approve or Reject a Request + +``` +POST /api/v1/approvals/{approval_uuid} +{ + "decision": "approve | reject", + "reason": "" +} + +Response 202 Accepted: +{ + "approval_uuid": "", + "decision": "approve", + "processed_at": "", + "request_uuid": "", + "request_status": "ASSEMBLING" # pipeline resumes on approve +} +``` + +--- + +## 7b. Cost and Quota + +### 7b.1 Get Cost Estimate (Pre-Submission) + +Returns a cost estimate for a hypothetical request without submitting it. + +``` +POST /api/v1/cost/estimate +{ + "catalog_item_uuid": "", + "fields": { + "cpu_count": 4, + "memory_gb": 8 + } +} + +Response 200: +{ + "estimated_cost": { + "breakdown": [ + { "component": "compute", "unit": "per-hour", "amount": 0.28, "currency": "USD" }, + { "component": "ip-allocation", "unit": "per-hour", "amount": 0.04, "currency": "USD" } + ], + "total_per_hour": 0.32, + "total_per_month": 230.40, + "currency": "USD" + }, + "cost_confidence": "high" +} +``` + +### 7b.2 Get Cost Actuals for a Resource + +``` +GET /api/v1/resources/{entity_uuid}/cost + +Query parameters: + from= start of billing period (default: start of current month) + to= end of billing period (default: now) + +Response 200: +{ + "entity_uuid": "", + "billing_state": "billable", + "period": { + "from": "2026-03-01T00:00:00Z", + "to": "2026-03-28T15:00:00Z" + }, + "actuals": { + "total": 168.96, + "currency": "USD", + "breakdown": [ + { "component": "compute", "hours": 651, "amount": 182.28 }, + { "component": "ip-allocation", "hours": 651, "amount": 26.04 } + ] + }, + "current_rate_per_hour": 0.32 +} +``` + +### 7b.3 View Quota Usage + +Returns current quota consumption for the authenticated Tenant. + +``` +GET /api/v1/quota + +Response 200: +{ + "tenant_uuid": "", + "quotas": [ + { + "resource_type": "Compute.VirtualMachine", + "limit": 100, + "current_usage": 47, + "percent_used": 47, + "reserved": 3 # in-flight requests consuming quota + }, + { + "resource_type": "Network.IPAddress", + "limit": 500, + "current_usage": 189, + "percent_used": 37.8, + "reserved": 0 + } + ] +} +``` + +--- + +## 7c. Notifications and Webhooks + +### 7c.1 List Notifications + +Returns notifications delivered to the authenticated actor, most recent first. + +``` +GET /api/v1/notifications + +Query parameters: + status= default: unread + urgency= + event_type= + page= + page_size= + +Response 200: +{ + "notifications": [ + { + "notification_uuid": "", + "event_type": "entity.decommissioning", + "urgency": "high", + "status": "unread", + "delivered_at": "", + "entity_uuid": "", + "entity_display_name": "VLAN-100", + "audience_role": "stakeholder", + "summary": "VLAN-100 is being decommissioned. Your resource VM-A is attached.", + "action_url": "/api/v1/resources/" + } + ], + "total_unread": 3, + "total": 47 +} +``` + +### 7c.2 Mark Notification Read + +``` +POST /api/v1/notifications/{notification_uuid}/read + +Response 200: +{ + "notification_uuid": "", + "status": "read", + "read_at": "" +} + +POST /api/v1/notifications/read-all # mark all unread as read + +Response 200: +{ + "marked_read": 3 +} +``` + +### 7c.3 Manage Webhook Subscriptions + +``` +GET /api/v1/webhooks + +Response 200: +{ + "subscriptions": [ + { + "webhook_uuid": "", + "endpoint_url": "https://my-system.example.com/dcm/events", + "events": ["entity.provisioned", "entity.decommissioned", "drift.detected"], + "status": "active", + "created_at": "" + } + ] +} + +POST /api/v1/webhooks +{ + "endpoint_url": "https://my-system.example.com/dcm/events", + "events": ["entity.provisioned", "entity.decommissioned"], + "hmac_secret": "", # used for payload signing + "description": "Production event sink" +} + +Response 201 Created: +{ + "webhook_uuid": "", + "status": "active", + "test_event_sent": true +} + +DELETE /api/v1/webhooks/{webhook_uuid} +Response 204 No Content +``` + +--- + +## 8b. Search + +### 8b.1 Cross-Resource Search + +Full-text and structured search across all resources in the actor's Tenant. Served from the Search Index โ€” non-authoritative but fast. + +``` +GET /api/v1/search + +Query parameters: + q= full-text query + resource_type= + lifecycle_state= + drift_status= + tag= repeatable + compliance_domain= + data_classification= filter by highest data classification + page= + page_size= + +Response 200: +{ + "results": [ + { + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "display_name": "payments-api-server-01", + "lifecycle_state": "OPERATIONAL", + "drift_status": "clean", + "tags": ["production", "payments"], + "resource_url": "/api/v1/resources/{entity_uuid}", + "score": 0.98 # relevance score for text queries + } + ], + "total": 3, + "search_index_staleness_seconds": 12, + "authoritative_store_ref": "/api/v1/resources?..." # fallback URL if stale +} +``` + + ## 6. Audit Trail ### 6.1 Query Audit Records for a Resource @@ -833,10 +1531,15 @@ All error responses follow a consistent structure: | 409 | `decommission_deferred` | Decommission blocked by active stakes or dependencies | | 409 | `rehydration_lease_held` | Entity already being rehydrated | | 409 | `field_not_editable` | Targeted delta attempted on non-editable field | +| 409 | `not_suspended` | Resume attempted on a non-suspended resource | +| 409 | `transfer_not_authorized` | No cross-tenant authorization between source and target Tenant | +| 409 | `no_ttl_constraint` | TTL extension attempted on resource with no time constraint | | 422 | `policy_rejected` | GateKeeper policy rejected the request | | 422 | `constraint_violated` | Field value violates declared constraint | +| 422 | `ttl_extension_rejected` | Policy rejected or capped the TTL extension request | | 429 | `rate_limit_exceeded` | Actor has exceeded request rate limit | | 503 | `assembly_unavailable` | Request Payload Processor temporarily unavailable | +| 503 | `search_index_degraded` | Search index unavailable; use authoritative_store_ref fallback | --- @@ -844,11 +1547,11 @@ All error responses follow a consistent structure: The Consumer API defines three conformance levels, mirroring the Operator Interface Specification model: -**Level 1 โ€” Read-Only:** Catalog browsing and resource status queries only. No request submission or resource management. Suitable for reporting and dashboard integrations. +**Level 1 โ€” Read-Only:** Catalog browsing, resource listing, status queries, search, cost estimates, quota views, and notification listing. No request submission or resource management. Suitable for reporting, dashboards, and read-only portal integrations. -**Level 2 โ€” Standard:** Full request submission, status tracking, and basic resource management (update editable fields, decommission). Required for all self-service portal implementations. +**Level 2 โ€” Standard:** All Level 1 operations plus request submission, status tracking, approvals, and basic resource management (update editable fields, suspend/resume, decommission, bulk decommission, TTL extension, group management). Required for all self-service portal implementations. -**Level 3 โ€” Full:** All Level 2 operations plus rehydration, audit trail access, and correlation queries. Required for ITSM integrations and compliance tooling. +**Level 3 โ€” Full:** All Level 2 operations plus rehydration, ownership transfer, drift management (acknowledge, accept, revert), audit trail access, correlation queries, webhook subscription management, and cost actuals. Required for ITSM integrations, compliance tooling, and full GitOps automation. --- From 16ed56e6e849fd952e021e943de615e551270201 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 22:45:14 -0500 Subject: [PATCH 34/49] DCM Flow GUI specs Signed-off-by: Chris Roadfeldt --- .../specifications/flow-gui-spec.md | 819 ++++++++++++++++-- 1 file changed, 728 insertions(+), 91 deletions(-) diff --git a/content/docs/architecture/specifications/flow-gui-spec.md b/content/docs/architecture/specifications/flow-gui-spec.md index eb509f7..9ca91b3 100644 --- a/content/docs/architecture/specifications/flow-gui-spec.md +++ b/content/docs/architecture/specifications/flow-gui-spec.md @@ -1,173 +1,810 @@ --- title: "DCM Flow GUI Specification" type: docs -weight: 6 +weight: 8 --- -> **โš ๏ธ Work in Progress** +> **๐Ÿ“‹ Draft** > -> This specification defines the DCM Flow GUI โ€” the visual interface for managing policies, orchestration flows, and the request lifecycle pipeline. Published to share design direction and invite feedback. +> This specification defines the DCM Flow GUI โ€” the visual interface for platform engineers to compose, test, simulate, and manage DCM's policy-driven orchestration. All views, data contracts, API endpoints, and component structure are specified. Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). **Version:** 0.1.0-draft -**Status:** Design โ€” Not yet implemented +**Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification -**Related Documents:** [Control Plane Components](../data-model/25-control-plane-components.md) | [OPA Integration Specification](dcm-opa-integration-spec.md) | [Policy Profiles](../data-model/14-policy-profiles.md) +**Related Documents:** [Control Plane Components](../data-model/25-control-plane-components.md) | [OPA Integration Specification](dcm-opa-integration-spec.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) --- ## Abstract -The DCM Flow GUI is the visual interface for platform engineers and integrators to compose, test, and manage DCM's data-driven orchestration flows. Because policies ARE the orchestration in DCM, the Flow GUI is fundamentally a **visual policy composer** โ€” it makes the active policy graph visible and editable without requiring direct YAML or Rego authoring. +The DCM Flow GUI is the visual interface for platform engineers to compose, test, and manage DCM's data-driven orchestration. Because policies ARE the orchestration in DCM, the Flow GUI is fundamentally a **visual policy composer** โ€” it makes the active policy graph visible and editable without requiring direct YAML or Rego authoring. + +The Flow GUI is a **platform engineer tool**, not a consumer tool. It operates with platform admin or policy author role permissions. Consumers interact with DCM through the Consumer API and Web UI, not through the Flow GUI. --- -## 1. Core Views +## 1. Architecture and Component Structure + +### 1.1 Component Diagram + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Browser (SPA) โ”‚ +โ”‚ Flow GUI Application โ€” React single-page application โ”‚ +โ”‚ Authentication: Bearer token (same session as Consumer API) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ HTTPS REST + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Flow GUI Service โ”‚ +โ”‚ Purpose: aggregate data for Flow GUI views โ”‚ +โ”‚ Deployed alongside DCM control plane โ”‚ +โ”‚ Authentication: validates Bearer token; requires policy_author โ”‚ +โ”‚ or platform_admin role โ”‚ +โ”‚ โ”‚ +โ”‚ Reads from: โ”‚ +โ”‚ Policy Engine โ€” live graph, firing frequency โ”‚ +โ”‚ GitOps stores โ€” policy artifacts, PR status โ”‚ +โ”‚ Observability โ€” event volumes, error rates โ”‚ +โ”‚ OPA sidecar โ€” test harness, shadow results โ”‚ +โ”‚ Writes via: โ”‚ +โ”‚ Git API โ€” create PRs for policy changes โ”‚ +โ”‚ Admin API โ€” shadow mode promotion, profile changes โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +### 1.2 Authentication and Authorization -### 1.1 Execution Graph View +The Flow GUI uses the same session token as the Consumer API. Required roles: -The primary view shows the live execution graph: which policies are currently active, which payload types they match, and the sequence in which they fire for a given request type. +| Role | Access | +|------|--------| +| `platform_admin` | Full read/write โ€” all views, all authoring, profile management | +| `policy_author` | Read all views; author policies in assigned domains; cannot manage profiles or promote shadow policies | +| `platform_observer` | Read-only โ€” all views; no authoring; no simulation write | + +### 1.3 Base URL + +``` +https://{dcm-instance}/flow/api/v1/ +``` + +Distinct from the Consumer API base URL to make routing and access control clear. + +--- + +## 2. The Execution Graph View + +### 2.1 What It Shows + +The primary view shows the live execution graph: which policies are active, which payload types they match, how they compose with each other, and their firing frequency. This is the "live map" of DCM's orchestration state. ``` [request.initiated] โ”€โ”€โ†’ [IntentCapturePolicy] โ”€โ”€โ†’ [request.intent_captured] โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ–ผ โ–ผ โ–ผ - [LayerAssembly] [CostCheck] [AuthzCheck] - (system domain) (tenant domain)(system domain) + [LayerAssembly] [CostCheck] [AuthzCheck] + (system/blue) (tenant/yellow)(system/blue) โ”‚ โ–ผ - [request.layers_assembled] + [request.layers_assembled] โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” - โ–ผ โ–ผ โ–ผ - [GateKeeper1] [Transform1] [Validate1] + โ–ผ โ–ผ โ–ผ + [GateKeeper: [Transform: [GovMatrix: + vm-size-limits] inject-mon.] phi-boundary] โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ–ผ - [request.policies_evaluated] + [request.policies_evaluated] ``` -**Interactive features:** -- Click any node to see the policy definition, trigger conditions, and current status -- Hover to see firing frequency (how often this policy fires per hour) -- Colour-coded by domain (system=blue, platform=green, tenant=yellow, provider=orange) -- Filter by payload type, domain, policy type, or resource type +**Visual conventions:** +- **Node color by domain:** system=blue, platform=green, tenant=yellow, resource_type=purple +- **Node shape by policy type:** GateKeeper=shield, Transformation=gear, Recovery=arrow, Governance Matrix=lock, Orchestration Flow=rectangle +- **Edge thickness:** proportional to firing frequency (last 1h) +- **Edge color:** green=allow path, red=deny path, amber=conditional +- **Node badge:** shadow mode indicator (S), deprecated indicator (D) -### 1.2 Policy Canvas (Static Flow Builder) +### 2.2 API โ€” Fetch Execution Graph -For organizations that want to define fixed sequential workflows, the Policy Canvas provides a drag-and-drop interface: +``` +GET /flow/api/v1/graph + +Query parameters: + payload_type= filter to policies matching this payload type + resource_type= filter to policies applicable to this resource type + domain= filter by policy domain + policy_type= filter by policy type + tenant_uuid= include tenant-domain policies for this Tenant + +Response 200: +{ + "graph": { + "nodes": [ + { + "node_id": "", # policy_uuid + "label": "vm-size-limits", + "policy_type": "gatekeeper", + "domain": "tenant", + "tenant_uuid": "", + "handle": "tenant/payments/gatekeeper/vm-size-limits", + "version": "1.2.0", + "status": "active", + "shadow_mode": false, + "match_payload_types": ["request.layers_assembled"], + "match_conditions_summary": "cpu_count > 32 OR memory_gb > 256", + "firing_frequency": { + "last_1h": 3, + "last_24h": 47, + "last_7d": 312 + }, + "deny_rate_24h": 0.06 # 6% of evaluations resulted in deny + } + ], + "edges": [ + { + "from_payload_type": "request.layers_assembled", + "to_node_id": "", + "edge_type": "policy_fires_on", + "volume_24h": 47 + }, + { + "from_node_id": "", + "to_payload_type": "request.policies_evaluated", + "edge_type": "produces", + "condition": "on_allow" + } + ] + }, + "payload_types": [ + { + "payload_type": "request.layers_assembled", + "volume_24h": 789, + "active_policy_count": 4 + } + ], + "last_updated": "" +} +``` -- Drag policy types from the palette onto the canvas -- Connect them with dependency arrows -- Set conditions on each step (fires when X AND Y) -- Set failure behavior (halt, skip, escalate) -- Export as a Policy Group with `concern_type: orchestration_flow` and `ordered: true` +### 2.3 API โ€” Get Policy Node Detail -The canvas produces valid DCM YAML that can be committed to the GitOps policy store. +``` +GET /flow/api/v1/graph/nodes/{policy_uuid} + +Response 200: +{ + "policy_uuid": "", + "handle": "tenant/payments/gatekeeper/vm-size-limits", + "version": "1.2.0", + "policy_type": "gatekeeper", + "domain": "tenant", + "concern_type": "security", + "enforcement": "soft", + "status": "active", + + "match_conditions": { + "payload_type": "request.layers_assembled", + "conditions": [ + { "field": "payload.fields.cpu_count.value", "operator": "gt", "value": 32 } + ] + }, + + "output_schema": { + "decision": "deny", + "reason_template": "cpu_count {value} exceeds maximum 32" + }, + + "firing_history": [ + { "timestamp": "", "result": "deny", "request_uuid": "" }, + { "timestamp": "", "result": "allow", "request_uuid": "" } + ], + + "git_path": "policy-store/tenant/payments/gatekeeper/vm-size-limits/v1.2.0.yaml", + "pr_url": null, # null if no pending PR; URL if change in review + + "compliance_basis": null, + "review_required_before": null, + + "test_suite": { + "test_count": 3, + "last_run": "", + "result": "pass" + } +} +``` -### 1.3 Payload Type Browser +--- -Shows the complete payload type vocabulary. For each type: -- Which policies currently match it -- Sample payload structure -- Which downstream types it can produce -- Historical volume (how many events of this type per day) +## 3. Policy Canvas โ€” Static Flow Builder -### 1.4 Shadow Mode Dashboard +### 3.1 Interaction Model -Shows active shadow policies and their evaluation results: -- Policy name and handle -- Shadow vs active comparison: "This policy would have rejected 3 requests in the last 24h" -- One-click promotion to active (if within review period) -- Side-by-side diff of shadow output vs actual outcome +The Policy Canvas is a drag-and-drop interface for building named Orchestration Flow Policies (Level 1 orchestration โ€” named workflow artifacts). The output is a valid DCM Orchestration Flow Policy YAML committed via a Git PR. ---- +**Key constraint:** The canvas never writes directly to the Policy Store. All saves generate a Git PR. The PR goes through the standard review process. Shadow mode activates automatically when the PR is created โ€” the proposed workflow evaluates against real traffic in shadow mode until merged. + +### 3.2 Canvas Operations + +| Operation | Description | Backend action | +|-----------|-------------|----------------| +| Drag payload type node | Add a workflow step | Canvas state update (local) | +| Connect nodes | Declare step sequence | Canvas state update (local) | +| Set step conditions | Add conditions to a step | Canvas state update (local) | +| Set failure behavior | halt / skip / escalate | Canvas state update (local) | +| Preview YAML | Show generated policy YAML | `GET /flow/api/v1/canvas/preview` | +| Save as PR | Create Git PR with policy YAML | `POST /flow/api/v1/canvas/save` | +| Load existing | Load an existing flow policy | `GET /flow/api/v1/policies/{policy_uuid}/canvas` | -## 2. Policy Authoring Interface +### 3.3 API โ€” Preview Canvas as YAML + +``` +POST /flow/api/v1/canvas/preview + +Request body: +{ + "handle": "org/orchestration/vm-provisioning-flow", + "concern_type": "orchestration_flow", + "ordered": true, + "steps": [ + { + "step": 1, + "payload_type": "request.initiated", + "policy_handle": "system/orchestration/capture-intent", + "on_fail": "halt" + }, + { + "step": 2, + "payload_type": "request.intent_captured", + "policy_handle": "system/orchestration/assemble-layers", + "on_fail": "halt" + }, + { + "step": 3, + "payload_type": "request.layers_assembled", + "policy_handle": "system/orchestration/run-placement", + "on_fail": "halt", + "condition": "not payload.placement_complete" + } + ], + "applicable_resource_types": ["Compute.VirtualMachine"] +} + +Response 200: +{ + "yaml": "# Generated by DCM Flow GUI\n# Handle: org/orchestration/vm-provisioning-flow\n...", + "rego": "package dcm.orchestration.vm_provisioning_flow\n...", + "validation": { + "valid": true, + "warnings": ["Step 3 condition references 'payload.placement_complete' which is not in the standard payload vocabulary"] + } +} +``` -The Flow GUI includes a policy authoring interface for creating and editing policies without leaving the browser: +### 3.4 API โ€” Save Canvas as Git PR -### 2.1 Visual Condition Builder +``` +POST /flow/api/v1/canvas/save + +Request body: +{ + "canvas_definition": { ... }, # same as preview request + "commit_message": "Add VM provisioning orchestration flow", + "pr_title": "feat(orchestration): VM provisioning named workflow", + "pr_description": "Defines explicit sequence for VM provisioning requests", + "target_branch": "main", + "shadow_mode": true # proposed status โ€” shadow evaluates before merge +} + +Response 201 Created: +{ + "pr_uuid": "", + "pr_url": "https://git.corp.example.com/dcm-policies/pulls/142", + "pr_status": "open", + "shadow_mode_activated": true, + "policy_handle": "org/orchestration/vm-provisioning-flow", + "policy_status": "proposed" # active in shadow mode; not yet enforced +} +``` -For simple conditions (field comparisons, role checks, quota checks), a visual condition builder generates valid Rego without requiring Rego knowledge: +### 3.5 API โ€” Load Existing Flow Policy into Canvas ``` -Trigger: [request.initiated โ–ผ] +GET /flow/api/v1/policies/{policy_uuid}/canvas + +Response 200: +{ + "canvas_definition": { + "handle": "...", + "ordered": true, + "steps": [...] + }, + "yaml": "...", + "policy_uuid": "", + "version": "1.2.0", + "git_path": "..." +} +``` + +--- + +## 4. Policy Authoring Interface + +### 4.1 Visual Condition Builder + +For simple policies (field comparisons, role checks, quota checks), a visual condition builder generates valid Rego without requiring Rego knowledge. + +**Supported condition types:** -Conditions: - [resource_type โ–ผ] [equals โ–ผ] [Compute.VirtualMachine] [+ AND] - [actor.roles โ–ผ] [does not contain โ–ผ] [platform_admin] [+ AND] - [payload.fields.cpu_count.value โ–ผ] [greater than โ–ผ] [32] +| Field type | Operators | Example | +|-----------|-----------|---------| +| Numeric field | equals, not_equals, gt, gte, lt, lte, in_range | `cpu_count > 32` | +| String field | equals, not_equals, in_list, matches_regex | `os_family in [rhel, ubuntu-lts]` | +| List field | contains, does_not_contain | `actor.roles contains platform_admin` | +| Boolean field | is_true, is_false | `payload.fields.production_workload = true` | +| Existence | exists, does_not_exist | `payload.fields.cost_center exists` | -Action: [Reject โ–ผ] -Rejection message: "CPU count exceeds maximum for this resource type" +### 4.2 API โ€” Generate Policy from Visual Conditions + +``` +POST /flow/api/v1/policies/generate + +Request body: +{ + "policy_type": "gatekeeper", + "handle": "tenant/payments/gatekeeper/vm-size-limits", + "concern_type": "security", + "domain": "tenant", + "tenant_uuid": "", + "enforcement": "soft", + "match": { + "payload_type": "request.layers_assembled", + "resource_type": "Compute.VirtualMachine", + "conditions": [ + { "field": "payload.fields.cpu_count.value", "operator": "gt", "value": 32 } + ], + "condition_logic": "any" + }, + "output": { + "decision": "deny", + "reason_template": "cpu_count {payload.fields.cpu_count.value} exceeds maximum 32 for this Tenant" + }, + "audit_on": ["DENY"], + "notification_on": ["DENY"] +} + +Response 200: +{ + "yaml": "# DCM GateKeeper Policy\n...", + "rego": "package dcm.gatekeeper.vm_size_limits\n\ndeny contains reason if {\n input.payload.type == \"request.layers_assembled\"\n input.payload.fields.cpu_count.value > 32\n reason := sprintf(\"cpu_count %d exceeds maximum 32\", [input.payload.fields.cpu_count.value])\n}\n", + "validation": { + "valid": true, + "warnings": [] + } +} ``` -### 2.2 Rego Editor +### 4.3 Rego Editor For complex policies requiring full Rego expressiveness, the GUI includes an embedded Rego editor with: -- DCM input schema autocomplete -- DCM built-in function reference -- Real-time syntax validation -- Test case runner (against the test harness) -### 2.3 Test Case Management +- **Input schema autocomplete:** all valid `input.*` paths from the DCM input document schema +- **DCM built-in reference:** sidebar showing available built-in functions and constants +- **Real-time syntax validation:** calls OPA `/v1/compile` to validate without evaluation +- **Test case runner:** executes the policy against saved test cases + +### 4.4 API โ€” Validate Rego -Each policy can have associated test cases managed in the GUI: -- Create test cases from recent real requests ("save this request as a test case") -- Run test suite before committing a policy change -- View shadow mode results as test case comparisons +``` +POST /flow/api/v1/policies/validate-rego + +Request body: +{ + "rego": "package dcm.gatekeeper.example\n\ndeny contains reason if {\n input.payload.fields.cpu_count.value > 32\n reason := \"too many CPUs\"\n}\n", + "policy_type": "gatekeeper" +} + +Response 200: +{ + "valid": true, + "warnings": [], + "errors": [], + "output_schema_match": true, # output matches declared policy_type schema + "input_paths_used": [ + "input.payload.fields.cpu_count.value" + ], + "input_paths_unknown": [] # paths that don't exist in the input document schema +} + +Response 200 (with errors): +{ + "valid": false, + "errors": [ + { "line": 4, "column": 5, "message": "undefined variable: reason_text" } + ] +} +``` + +### 4.5 Test Case Management + +``` +# List test cases for a policy +GET /flow/api/v1/policies/{policy_uuid}/tests + +Response 200: +{ + "test_cases": [ + { + "test_uuid": "", + "name": "Reject oversized VM", + "input_payload": { "payload": { "type": "request.layers_assembled", "fields": { "cpu_count": { "value": 64 } } } }, + "expected_output": { "deny": ["cpu_count 64 exceeds maximum 32"] }, + "last_result": "pass", + "last_run": "" + } + ] +} + +# Create test case from a real recent request +POST /flow/api/v1/policies/{policy_uuid}/tests/from-request +{ + "request_uuid": "", # saves that request's payload as a test case + "expected_output": { "deny": [] }, + "test_name": "Normal VM request โ€” should allow" +} + +# Run all test cases +POST /flow/api/v1/policies/{policy_uuid}/tests/run + +Response 200: +{ + "run_uuid": "", + "result": "pass", # pass | fail | error + "test_results": [ + { + "test_uuid": "", + "name": "Reject oversized VM", + "result": "pass", + "actual_output": { "deny": ["cpu_count 64 exceeds maximum 32"] }, + "expected_output": { "deny": ["cpu_count 64 exceeds maximum 32"] } + } + ], + "duration_ms": 42 +} +``` + +--- + +## 5. Flow Simulation + +### 5.1 Simulation Model + +Platform engineers simulate a synthetic request through the active policy engine without creating real state. The simulation runs against the live Policy Engine with a caller-constructed payload. No audit records are written. No Requested State is created. + +### 5.2 API โ€” Simulate Request + +``` +POST /flow/api/v1/simulate + +Request body: +{ + "catalog_item_uuid": "", # optional; used to seed field schema + "resource_type": "Compute.VirtualMachine", + "tenant_uuid": "", + "synthetic_fields": { + "cpu_count": 64, + "memory_gb": 128, + "os_family": "rhel" + }, + "synthetic_actor": { + "roles": ["developer"], + "group_memberships": ["payments-team"] + }, + "include_policy_types": ["gatekeeper", "transformation", "governance_matrix"] +} + +Response 200: +{ + "simulation_uuid": "", + "result": "rejected", # allowed | rejected | degraded + "terminal_reason": "GateKeeper policy rejected at step request.layers_assembled", + + "execution_trace": [ + { + "step": 1, + "payload_type": "request.initiated", + "policies_evaluated": [], + "result": "pass", + "duration_ms": 2 + }, + { + "step": 2, + "payload_type": "request.intent_captured", + "policies_evaluated": [], + "result": "pass", + "duration_ms": 1 + }, + { + "step": 3, + "payload_type": "request.layers_assembled", + "policies_evaluated": [ + { + "policy_uuid": "", + "policy_handle": "tenant/payments/gatekeeper/vm-size-limits", + "policy_type": "gatekeeper", + "result": "deny", + "reason": "cpu_count 64 exceeds maximum 32", + "duration_ms": 8 + }, + { + "policy_uuid": "", + "policy_handle": "org/transformation/inject-monitoring", + "policy_type": "transformation", + "result": "applied", + "mutations": [ + { "field": "fields.monitoring_endpoint", "operation": "set", "value": "https://metrics..." } + ], + "duration_ms": 3 + } + ], + "result": "rejected", + "terminal": true + } + ], + + "assembled_payload_snapshot": { + "fields": { + "cpu_count": { "value": 64, "provenance": { "origin": { "source_type": "consumer_request" } } }, + "monitoring_endpoint": { "value": "https://metrics...", "provenance": { "origin": { "source_type": "policy" } } } + } + }, + + "cost_estimate": { + "total_per_hour": 1.28, + "currency": "USD", + "note": "Estimated assuming request would have been allowed" + } +} +``` + +### 5.3 Simulation vs Shadow Mode + +| | Simulation | Shadow Mode | +|-|-----------|------------| +| Trigger | Manual, synthetic payload | Automatic on real traffic | +| Audit record | Never written | Written to Validation Store | +| Policy status | Evaluates active policies | Evaluates proposed policies | +| Use case | "What if?" exploration | Pre-activation validation | +| Real data | No | Yes | --- -## 3. Flow Simulation +## 6. Shadow Mode Dashboard + +### 6.1 What It Shows + +Shows all proposed policies currently in shadow mode and their evaluation results against real traffic. + +### 6.2 API โ€” List Shadow Policies + +``` +GET /flow/api/v1/shadow + +Response 200: +{ + "shadow_policies": [ + { + "policy_uuid": "", + "handle": "tenant/payments/gatekeeper/new-cost-check", + "policy_type": "gatekeeper", + "status": "proposed", + "shadow_since": "", + "pr_url": "https://git.corp.example.com/dcm-policies/pulls/143", + "pr_status": "open", + + "shadow_results_24h": { + "total_evaluations": 156, + "would_have_denied": 4, + "would_have_allowed": 152, + "divergence_from_active": 4, + "divergence_rate": 0.026 + } + } + ] +} +``` -Platform engineers can simulate a request through the active policy graph without actually submitting it: +### 6.3 API โ€” Shadow Policy Detail with Divergence Cases ``` -Simulate: resource_type=Compute.VirtualMachine, tenant=payments, cpu_count=64 - โ”‚ - โ–ผ Execution trace: - IntentCapturePolicy: PASS - VmSizeLimits (GateKeeper): REJECT โ€” cpu_count 64 exceeds maximum 32 - โ† Request would be rejected at this step +GET /flow/api/v1/shadow/{policy_uuid} + +Response 200: +{ + "policy_uuid": "", + "shadow_results_24h": { + "total_evaluations": 156, + "divergence_cases": [ + { + "request_uuid": "", + "timestamp": "", + "active_result": "allow", + "shadow_result": "deny", + "shadow_reason": "Estimated cost $480/month exceeds budget ceiling $300/month", + "requester": "Bob Smith", + "resource_type": "Compute.VirtualMachine" + } + ] + } +} ``` -Simulation mode is read-only โ€” it uses the current active policies and a synthetic payload. No audit records are written. +### 6.4 API โ€” Promote Shadow Policy to Active + +``` +POST /flow/api/v1/shadow/{policy_uuid}/promote +{ + "reason": "Shadow results reviewed โ€” divergence rate acceptable; promoting to active" +} + +Response 202 Accepted: +{ + "policy_uuid": "", + "status": "active", + "pr_action": "approved_and_merged", + "promoted_at": "" +} + +Response 403 Forbidden: +{ + "error": "insufficient_role", + "reason": "Policy promotion requires platform_admin role" +} +``` --- -## 4. Profile and Module Management +## 7. Profile and Governance Management + +### 7.1 Active Profile View + +``` +GET /flow/api/v1/profile + +Response 200: +{ + "deployment_posture": { + "name": "prod", + "description": "Production โ€” full zero trust, dual approval for high-trust providers, human review for all registrations", + "active_policy_groups": 12, + "hard_constraints": [ + "sovereign/classified data never crosses any boundary", + "All providers require at least self_declared accreditation" + ] + }, + "compliance_domains": [ + { + "domain": "hipaa", + "description": "HIPAA/HITECH compliance โ€” PHI classification, BAA requirements, minimum necessary principle", + "active_policy_groups": 4, + "key_requirements": ["PHI requires BAA accreditation", "All PHI interactions audited", "No PHI export without regulatory cert"] + } + ], + "recovery_posture": "notify-and-wait", + "zero_trust_posture": "full", + "total_active_policies": 47 +} +``` + +### 7.2 Payload Type Browser + +``` +GET /flow/api/v1/payload-types + +Response 200: +{ + "payload_types": [ + { + "payload_type": "request.layers_assembled", + "description": "Layer assembly complete โ€” payload enriched with all layer fields", + "volume_24h": 789, + "active_policy_count": 4, + "sample_payload": { + "type": "request.layers_assembled", + "fields": { + "cpu_count": { "value": 4 }, + "memory_gb": { "value": 8 } + } + }, + "downstream_payload_types": ["request.policies_evaluated", "recovery.gatekeeper_denied"] + } + ] +} +``` -### 4.1 Active Profile View +--- -Shows the current active governance composition: -- Active deployment posture (with description of what it enforces) -- Active compliance domains (with summary of key requirements each adds) -- Active recovery posture profile -- Policy count per active profile group +## 8. Notification Flow View -### 4.2 Profile Activation +### 8.1 API โ€” Notification Flow for an Entity -Change the deployment posture or add/remove compliance domains through the GUI. Produces a profile change request (through the standard request pipeline with appropriate approvals). +``` +GET /flow/api/v1/notifications/flow/{entity_uuid} + +Response 200: +{ + "entity_uuid": "", + "entity_display_name": "VLAN-100", + "relationship_graph_depth": 2, + + "notification_audiences": [ + { + "actor_uuid": "", + "display_name": "NetworkOps Team", + "audience_role": "owner", + "stakeholder_reason": null, + "notification_providers": ["slack-corp", "pagerduty-prod"] + }, + { + "actor_uuid": "", + "display_name": "AppTeam Admin", + "audience_role": "stakeholder", + "stakeholder_reason": { + "via_entity": "VM-A", + "via_relationship": "attached_to", + "stake_strength": "required" + }, + "notification_providers": ["slack-corp"] + } + ], + + "active_notification_providers": [ + { + "provider_uuid": "", + "display_name": "slack-corp", + "status": "healthy", + "delivery_success_rate_24h": 0.998 + } + ] +} +``` --- -## 5. Notification Flow View +## 9. Error Model -An extension of the Execution Graph View specific to the Notification Model: -- Shows active notification subscriptions per event type -- Visualizes the relationship graph traversal for a specific entity -- Shows which Notification Providers are active and their delivery health +All Flow GUI API errors follow the standard DCM error format: + +```json +{ + "error": "", + "message": "", + "request_id": "", + "timestamp": "" +} +``` + +| HTTP Status | Error Code | Meaning | +|-------------|-----------|---------| +| 403 | `insufficient_role` | Operation requires platform_admin or policy_author role | +| 404 | `policy_not_found` | Policy UUID not found in active policy store | +| 409 | `pr_already_open` | A PR already exists for this policy handle | +| 422 | `invalid_canvas` | Canvas definition is invalid (disconnected steps, unknown payload types) | +| 422 | `rego_invalid` | Rego syntax error or output schema mismatch | +| 422 | `simulation_failed` | Simulation could not be executed (missing fields, invalid tenant) | +| 503 | `policy_engine_unavailable` | Policy Engine unreachable โ€” graph data may be stale | +| 503 | `git_unavailable` | GitOps store unreachable โ€” PR creation unavailable | --- -## 6. Integration with OPA +## 10. Conformance Levels + +**Level 1 โ€” Read-Only:** Execution Graph View (read), Profile View, Payload Type Browser, Notification Flow View. Suitable for dashboards and observability integrations. + +**Level 2 โ€” Standard:** All Level 1 plus Flow Simulation, Shadow Mode Dashboard (view only), Policy Node Detail. Required for platform engineer tooling. -The Flow GUI connects to the OPA integration for: -- Live policy evaluation display (showing OPA decisions in real time) -- Policy testing via the OPA test harness -- Shadow mode result display from OPA shadow evaluations -- Bundle upload and validation +**Level 3 โ€” Full:** All Level 2 plus Policy Canvas (save as PR), Policy Authoring Interface, Test Case Management, Shadow Mode Promotion. Required for full policy lifecycle management. --- From d943c42f3c571b557f57e377925667fdd4cd33cd Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 22:58:23 -0500 Subject: [PATCH 35/49] Federated DCM content model draft added. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 60 ++++++++++ .../architecture/data-model/foundations.md | 3 +- .../data-model/layering-and-versioning.md | 17 +++ .../data-model/policy-profiles.md | 13 +++ .../data-model/registry-governance.md | 19 +++ .../specifications/consumer-api-spec.md | 108 +++++++++++++++++- 6 files changed, 218 insertions(+), 2 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 2d67a47..ee5c25a 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -484,6 +484,10 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Federated Contribution Model** | DCM defaults to federated data creation โ€” all authorized actor types (platform admin, consumer/tenant, service provider, peer DCM) can contribute Data artifacts within their domain scope via the GitOps PR model; see doc 28 | +| **contributor** | Actor type that authored a Data artifact; recorded in artifact_metadata.contributed_by; determines review requirements; platform_admin / consumer / service_provider / peer_dcm | +| **contributed_by** | Artifact metadata block recording contributor_type, actor UUID, contribution_method, pr_url, reviewed_by; immutable once set | +| **FCM-001โ€“008** | Federated Contribution Model system policies; key: FCM-002 (domain scope violations = hard DENY), FCM-003 (GitOps PR for all), FCM-008 (contributor scope limits absolute) | | **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | | **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | | **sovereignty_zone** | Registered DCM artifact declaring geopolitical/regulatory boundary; rules reference zones by ID; inter-zone agreements declared explicitly | @@ -2586,6 +2590,10 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Federated Contribution Model** | DCM defaults to federated data creation โ€” all authorized actor types (platform admin, consumer/tenant, service provider, peer DCM) can contribute Data artifacts within their domain scope via the GitOps PR model; see doc 28 | +| **contributor** | Actor type that authored a Data artifact; recorded in artifact_metadata.contributed_by; determines review requirements; platform_admin / consumer / service_provider / peer_dcm | +| **contributed_by** | Artifact metadata block recording contributor_type, actor UUID, contribution_method, pr_url, reviewed_by; immutable once set | +| **FCM-001โ€“008** | Federated Contribution Model system policies; key: FCM-002 (domain scope violations = hard DENY), FCM-003 (GitOps PR for all), FCM-008 (contributor scope limits absolute) | | **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | | **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | | **sovereignty_zone** | Registered DCM artifact declaring geopolitical/regulatory boundary; rules reference zones by ID; inter-zone agreements declared explicitly | @@ -3560,6 +3568,54 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc --- +## SECTION 61 โ€” FEDERATED CONTRIBUTION MODEL (doc 28) + +### Core Principle +DCM defaults to a federated model for data creation, import, usage, and lifecycle. Every authorized actor type can contribute Data artifacts within their permitted domain scope. The same GitOps PR flow and lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired) applies to all contributors. Profile-bound auto-approval governs what requires human review. + +### Four Contributor Types +1. **Platform Admin** โ€” all artifact types, all domains, no restrictions +2. **Consumer/Tenant** โ€” tenant-domain policies, resource groups, notification subscriptions, webhook registrations, cross-tenant authorization records, request layers +3. **Service Provider** โ€” resource type specs (types they offer), provider catalog items, service layers, provider-domain policies +4. **Peer DCM** โ€” registry entries, policy templates, service layers (via federation channels, scoped by trust posture) + +### Contributor Permission Boundaries (hard DENY โ€” Governance Matrix enforced) +- Consumers cannot contribute system or platform domain policies +- Providers cannot contribute specs for resource types they don't offer +- Provisional peers can only contribute registry entries (no policies) +- Vouched peers: registry entries + service layers only (human_review always) +- Verified peers: registry entries + policy templates + service layers (human_review standard+; auto dev) + +### Universal Contribution Pipeline +Submit โ†’ Governance Matrix evaluates contributor permissions โ†’ proposed status (shadow mode for policies) โ†’ review flow (auto / human_review / dual_approval / committee per profile + artifact type + contributor) โ†’ active โ†’ lifecycle by contributor (deprecate/retire) โ†’ platform admin override at any time + +### Contribution Artifact Types by Contributor +- Consumer: tenant policies (all 7 types), resource groups, notification subs, webhooks, cross-tenant auth records, request layers +- Provider: Resource Type Specs (their types), catalog items, service layers, provider-domain GateKeeper/Validation policies +- Peer DCM: registry entries, policy templates (verified peers), service layers (verified/vouched) + +### Contribution Store Directory Structure +`dcm-policy-store/system/` (platform admin), `platform/` (platform admin), `tenant//` (consumer), `provider//` (provider), `federated//` (peer DCM) +`dcm-registry/core/` (DCM project), `community//` (community), `organization//` (org) + +Every artifact includes `contributed_by` block: contributor_type, actor/tenant/provider/peer_dcm UUID, contribution_method (api/flow_gui/git_pr/federation_push), pr_url, reviewed_by. + +### Profile-Governed Auto-Approval +- minimal/dev: most contributions auto-approved; shadow optional +- standard: consumer/provider policies โ†’ human_review; shadow default on, P7D +- prod: governance matrix rules โ†’ dual_approval; provider specs โ†’ human_review; shadow P14D +- fsi: all consumer/provider contributions โ†’ dual_approval; shadow P30D; must review all divergence cases +- sovereign: all โ†’ committee; shadow P30D; orphaned artifacts auto-retire + +### Consumer API Contribution Endpoints (Section 9) +`POST /api/v1/contribute/policy` (generates PR, activates shadow mode) ยท `POST /api/v1/contribute/resource-group` (activates immediately) ยท `GET /api/v1/contribute` (list contributions) ยท `DELETE /api/v1/contribute/{uuid}` (withdraw, closes PR) + +### FCM-001 through FCM-008 system policies +FCM-001: contributor recorded in artifact_metadata.contributed_by; immutable. FCM-002: domain scope violations are hard DENY. FCM-003: all contributions via GitOps PR (except auto-approve profiles). FCM-004: policies enter shadow mode by default. FCM-005: platform admin override always available; audited. FCM-006: orphaned artifacts don't auto-deactivate (except sovereign). FCM-007: federation contribution scoped by trust posture. FCM-008: contributor-tier scope limits absolute โ€” tenant domain policy cannot affect system/platform domain regardless of match conditions. + +--- + + ## SECTION 54 โ€” TERMINOLOGY GLOSSARY | Term | Definition | @@ -3631,6 +3687,10 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc | **cross_tenant_authorization** | DCMGroup with group_class: cross_tenant_authorization; grants one Tenant permission to reference/allocate/stake another Tenant's resources; revocation places active allocations in PENDING_REVIEW | | **foundation Tenants** | Three system Tenants created at bootstrap: __platform__, __transitional__, __system__; cannot be decommissioned; declared in bootstrap manifest | | **QUOTA_EXCEEDED** | GateKeeper rejection code when resource quota policy fires at Step 5 (pre-placement) | +| **Federated Contribution Model** | DCM defaults to federated data creation โ€” all authorized actor types (platform admin, consumer/tenant, service provider, peer DCM) can contribute Data artifacts within their domain scope via the GitOps PR model; see doc 28 | +| **contributor** | Actor type that authored a Data artifact; recorded in artifact_metadata.contributed_by; determines review requirements; platform_admin / consumer / service_provider / peer_dcm | +| **contributed_by** | Artifact metadata block recording contributor_type, actor UUID, contribution_method, pr_url, reviewed_by; immutable once set | +| **FCM-001โ€“008** | Federated Contribution Model system policies; key: FCM-002 (domain scope violations = hard DENY), FCM-003 (GitOps PR for all), FCM-008 (contributor scope limits absolute) | | **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | | **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | | **sovereignty_zone** | Registered DCM artifact declaring geopolitical/regulatory boundary; rules reference zones by ID; inter-zone agreements declared explicitly | diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md index 0be3b39..3fc13e6 100644 --- a/content/docs/architecture/data-model/foundations.md +++ b/content/docs/architecture/data-model/foundations.md @@ -1,7 +1,7 @@ --- title: "DCM Foundational Abstractions" type: docs -weight: -10 +weight: 0 --- > **โš ๏ธ Active Development Notice** @@ -72,6 +72,7 @@ This is the complete DCM operational model. Everything else is a typed specializ - **Provenance** โ€” every field in every Data artifact carries lineage metadata describing its origin and all modifications - **Data classification** โ€” every field carries a classification (public โ†’ classified) governing what may cross interaction boundaries - **Immutability if versioned** โ€” once a version is published, it cannot be modified; changes produce new versions +- **Contributor identity** โ€” every Data artifact records who contributed it (platform admin, consumer/tenant, service provider, or peer DCM) and what review it received before activation. DCM defaults to a federated contribution model โ€” all authorized actor types can create Data within the bounds their role permits. See [Federated Contribution Model](28-federated-contribution-model.md). **The complete Data taxonomy:** diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md index 4aec832..6aae80b 100644 --- a/content/docs/architecture/data-model/layering-and-versioning.md +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -268,6 +268,23 @@ For Model A: entity record alone is sufficient --- + +### 2a. Layer Contributors + +Every layer type has a declared contributor type. The contributor determines what review is required before the layer becomes active in assembly. See [Federated Contribution Model](28-federated-contribution-model.md) Section 3 for the full contributor permission table. + +| Layer Type | Contributor | Domain | Review | +|-----------|-------------|--------|--------| +| Base Layer | Platform Admin | system | auto | +| Core Layer | Platform Admin | platform | auto | +| Intermediate / Customization Layer | Platform Admin, Consumer/Tenant | platform, tenant | per profile | +| Service Layer | Platform Admin, Service Provider | provider | human_review (standard+) | +| Request Layer | Consumer/Tenant | tenant | auto (applied directly to request) | +| Policy Layer | All contributor types | per contributor role | per profile + contributor type | + +The Request Layer is the only layer type that does not require a PR review โ€” it is a consumer's direct field declarations on a specific request. All other layers flow through the GitOps PR model. + + ## 3. Layer Types DCM defines six layer types. Each has a distinct purpose, scope, ownership model, and position in the assembly precedence chain. diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 422ba50..66b2d4d 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -57,6 +57,19 @@ Policy Providers โ€” external authoritative policy sources --- +## 1b. Policy Authorship โ€” Federated Contribution Model + +Policies in DCM are not exclusively authored by platform admins. The DCM federated contribution model enables all actor types to author policies within their permitted domain scope: + +- **Platform admins** โ€” all domains, all policy types +- **Consumers / Tenant admins** โ€” tenant domain policies (GateKeeper, Transformation, Recovery, Lifecycle, Orchestration Flow, Governance Matrix rules scoped to their Tenant) +- **Service Providers** โ€” provider-domain GateKeeper and Validation policies for their resource types +- **Peer DCM instances** โ€” policy templates contributed through verified federation relationships + +This is not a special case โ€” it is the standard GitOps PR model applied to all contributor types. Consumer-authored policies go through the same lifecycle (developing โ†’ proposed โ†’ active) with appropriate review requirements per the active profile. See [Federated Contribution Model](28-federated-contribution-model.md) for the complete specification. + +--- + ## 2. Policy Groups ## 1a. Two-Dimensional Profile Model diff --git a/content/docs/architecture/data-model/registry-governance.md b/content/docs/architecture/data-model/registry-governance.md index 9f98eb7..0695d79 100644 --- a/content/docs/architecture/data-model/registry-governance.md +++ b/content/docs/architecture/data-model/registry-governance.md @@ -52,6 +52,25 @@ Registry governance follows the same principles as all other DCM governance: Git **Tier 3 examples:** `Acme.LegacyMainframeJob`, `Corp.ServiceNowTicket`, `Internal.ComplianceReport` + +### 2a. Three-Tier Model Applied to All Artifact Types + +The three-tier registry model applies to all DCM artifact types, not just resource type specs. Every artifact in DCM has a tier that determines its trust level and the review requirements for changes: + +| Tier | Maintained by | Examples | Review for changes | +|------|--------------|---------|-------------------| +| **Core** | DCM Project | Built-in policies, base layers, system resource types | DCM project PR process | +| **Verified Community** | Named community maintainers | Community resource types, shared policy templates, vetted provider specs | Community review + platform admin acceptance | +| **Organization** | Deploying organization | Tenant policies, provider catalog items, org-specific specs | Per profile (auto โ†’ committee) | + +**Contributor sub-tiers within Organization tier:** +- `organization/platform` โ€” authored by platform admins; highest trust in org tier +- `organization/provider` โ€” authored by registered Service Providers; scoped to their resource types +- `organization/tenant` โ€” authored by Consumer/Tenant actors; scoped to their Tenant + +This means a tenant-authored GateKeeper policy is Organization/Tenant tier โ€” it has lower inherent trust than a platform-authored policy at the same domain level, and may require additional review per the active profile. See [Federated Contribution Model](28-federated-contribution-model.md). + + ### 2.2 The Federated Registry Model The registry uses a federated model โ€” not centralized, not fully distributed. This supports air-gapped and sovereign deployments without external dependencies. diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index c175551..20511be 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -1543,13 +1543,119 @@ All error responses follow a consistent structure: --- + +--- + +## 9. Consumer Contribution Endpoints + +Consumers with `policy_author` or `tenant_admin` role can contribute tenant-scoped artifacts directly via the Consumer API. All contributions flow through the GitOps PR model โ€” DCM generates a PR and activates the artifact after the required review period. See [Federated Contribution Model](../data-model/28-federated-contribution-model.md) for the complete contributor permission table. + +### 9.1 Submit Policy Contribution + +``` +POST /api/v1/contribute/policy +X-DCM-Tenant: + +{ + "policy_type": "gatekeeper | transformation | recovery | lifecycle | orchestration_flow | governance_matrix_rule", + "handle": "tenant/{tenant-handle}/gatekeeper/{name}", + "domain": "tenant", + "concern_type": "operational | security | compliance", + "enforcement": "soft | hard", + "match": { ... }, + "output": { ... }, + "shadow_mode": true, + "commit_message": "" +} + +Response 202 Accepted: +{ + "contribution_uuid": "", + "policy_handle": "tenant/payments/gatekeeper/cost-ceiling", + "status": "proposed", + "shadow_mode": true, + "review_required": true, + "review_type": "human_review", + "pr_url": "https://git.corp.example.com/dcm-policies/pulls/145", + "shadow_results_url": "/flow/api/v1/shadow/" +} +``` + +### 9.2 Submit Resource Group Definition + +``` +POST /api/v1/contribute/resource-group +X-DCM-Tenant: + +{ + "handle": "tenant/{tenant-handle}/groups/{name}", + "display_name": "", + "group_class": "resource_grouping", + "description": "", + "membership_policy": { + "auto_include": { + "resource_type": "Compute.VirtualMachine", + "tags": { "team": "payments", "env": "production" } + } + } +} + +Response 201 Created: +{ + "group_uuid": "", + "handle": "tenant/payments/groups/prod-vms", + "status": "active" # resource groups activate immediately (no policy review) +} +``` + +### 9.3 List Contributions + +``` +GET /api/v1/contribute +X-DCM-Tenant: + +Query parameters: + artifact_type= + status= + +Response 200: +{ + "contributions": [ + { + "contribution_uuid": "", + "artifact_type": "policy", + "handle": "tenant/payments/gatekeeper/cost-ceiling", + "status": "proposed", + "shadow_mode": true, + "pr_url": "https://...", + "submitted_at": "", + "review_status": "pending" + } + ] +} +``` + +### 9.4 Withdraw Contribution + +``` +DELETE /api/v1/contribute/{contribution_uuid} + +Response 200: +{ + "contribution_uuid": "", + "status": "withdrawn", + "pr_closed": true +} +``` + + ## 8. Conformance Levels The Consumer API defines three conformance levels, mirroring the Operator Interface Specification model: **Level 1 โ€” Read-Only:** Catalog browsing, resource listing, status queries, search, cost estimates, quota views, and notification listing. No request submission or resource management. Suitable for reporting, dashboards, and read-only portal integrations. -**Level 2 โ€” Standard:** All Level 1 operations plus request submission, status tracking, approvals, and basic resource management (update editable fields, suspend/resume, decommission, bulk decommission, TTL extension, group management). Required for all self-service portal implementations. +**Level 2 โ€” Standard:** All Level 1 operations plus request submission, status tracking, approvals, basic resource management (update editable fields, suspend/resume, decommission, bulk decommission, TTL extension, group management), and consumer contribution endpoints (policy authoring, resource group definitions). Required for all self-service portal implementations. **Level 3 โ€” Full:** All Level 2 operations plus rehydration, ownership transfer, drift management (acknowledge, accept, revert), audit trail access, correlation queries, webhook subscription management, and cost actuals. Required for ITSM integrations, compliance tooling, and full GitOps automation. From 2ed6b21f18dc78c9ddd34358973f5675d692622c Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 22:58:35 -0500 Subject: [PATCH 36/49] Federated DCM content model draft added. with files Signed-off-by: Chris Roadfeldt --- .../federated-contribution-model.md | 588 ++++++++++++++++++ 1 file changed, 588 insertions(+) create mode 100644 content/docs/architecture/data-model/federated-contribution-model.md diff --git a/content/docs/architecture/data-model/federated-contribution-model.md b/content/docs/architecture/data-model/federated-contribution-model.md new file mode 100644 index 0000000..c3226d9 --- /dev/null +++ b/content/docs/architecture/data-model/federated-contribution-model.md @@ -0,0 +1,588 @@ +--- +title: "Federated Contribution Model" +type: docs +weight: 28 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” Read This First for Multi-User Data Governance +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Policy Profiles](14-policy-profiles.md) | [Registry Governance](20-registry-governance.md) | [DCM Federation](22-dcm-federation.md) | [Governance Matrix](27-governance-matrix.md) | [Consumer API](../specifications/consumer-api-spec.md) + +> **This document maps to: DATA + POLICY + PROVIDER** +> +> The federated contribution model governs how Data artifacts are created and managed across all contributor types. It extends the Data abstraction with explicit contributor identity, applies Policies to govern contribution permissions and review requirements, and uses the Provider abstraction for cross-instance federation of contributions. + +--- + +## 1. Purpose and Principle + +DCM is a multi-user, multi-contributor system. Platform admins are not the only actors who create data. Consumers define their own service configurations, resource groups, and policy overlays. Service Providers publish their own resource type specs and catalog items. Peer DCM instances contribute registry entries across federation boundaries. Organizations extend DCM with their own artifact types. + +**The federated contribution model** is the governing framework for how all of these actors create, review, activate, and lifecycle-manage DCM data artifacts. It extends the Data abstraction with one additional universal property: + +> **Every DCM data artifact has a contributor** โ€” an actor or system that authored it โ€” and that contributor's role determines what review is required before the artifact becomes active. + +This is not a special model for special cases. It is the same GitOps PR workflow, the same lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired), and the same domain precedence (system โ†’ platform โ†’ tenant โ†’ resource_type โ†’ entity) โ€” applied consistently across all contributor types. + +**The core principle:** DCM defaults to a federated model for data creation, import, usage, and lifecycle. Every authorized actor can contribute within the bounds their role permits. The Governance Matrix governs the boundaries. The GitOps PR flow provides the review mechanism. Profile-bound auto-approval policies determine what needs human review and what does not. + +--- + +## 2. Contributor Types and Permissions + +### 2.1 The Four Contributor Types + +| Contributor | Examples | Default domain scope | +|-------------|---------|---------------------| +| **Platform Admin** | DCM operators, SRE team | system, platform โ€” all artifact types | +| **Consumer / Tenant** | Application teams, developers, Tenant admins | tenant โ€” scoped to their Tenant | +| **Service Provider** | Infrastructure teams, automation platforms | provider โ€” resource types they offer | +| **Peer DCM** | Federated DCM instances, Hub DCM, community registry | federated โ€” governed by federation trust posture | + +### 2.2 What Each Contributor Can Contribute + +**Platform Admin:** All artifact types at all domain levels. No restrictions within the DCM deployment. + +**Consumer / Tenant:** +- Tenant-domain policies (GateKeeper, Transformation, Recovery, Lifecycle, Orchestration Flow) +- Resource groups and group memberships within their Tenant +- Notification subscriptions for their Tenant +- Webhook registrations for their Tenant +- Custom catalog item definitions (within their Tenant's resource type scope) +- Tenant-scoped data layers (Request Layer โ€” directly attached to their requests) +- Cross-tenant authorization records (requires counterpart Tenant acceptance) + +**Service Provider:** +- Resource Type Specifications for resource types they offer (Organization or Verified Community tier) +- Provider Catalog Items for their registered resource types +- Service Layers for their offered resource types +- Provider-specific GateKeeper and Validation policies (provider domain) +- Cost metadata updates +- Sovereignty declaration updates + +**Peer DCM:** +- Registry entries (Resource Type Specs, provider type definitions) contributed through federation channels +- Policy bundles contributed through verified federation relationships +- Layer contributions through Hub DCM governance +- Accreditation vouching for providers registered with the contributing DCM + +### 2.3 What Each Contributor Cannot Contribute + +| Contributor | Cannot contribute | +|-------------|-----------------| +| Consumer | System or platform domain policies; core layers; resource type specs (unless granted elevated role); provider catalog items for other providers | +| Service Provider | Policies outside their resource type domain; core layers; other providers' catalog items; tenant-domain policies for specific Tenants | +| Peer DCM | Artifacts above the federation trust level granted; system-domain policies without committee approval; sovereignty zones for jurisdictions not in their declared scope | + +--- + +## 3. Contribution Artifact Types + +Every DCM data artifact type has a declared set of contributor permissions. The following table specifies who can contribute each type and at what domain level: + +| Artifact Type | Platform Admin | Consumer/Tenant | Service Provider | Peer DCM | +|--------------|---------------|-----------------|-----------------|---------| +| Resource Type Specification | All tiers | โŒ | Org + Community tiers | Community tier (via federation) | +| Provider Catalog Item | All | โŒ | Their resource types only | โŒ | +| Core Layer | โœ… | โŒ | โŒ | โŒ | +| Service Layer | โœ… | โŒ | Their resource types only | โŒ | +| Request Layer | โœ… | Their requests only | โŒ | โŒ | +| GateKeeper Policy | All domains | Tenant domain only | Provider domain only | Via federation governance | +| Transformation Policy | All domains | Tenant domain only | Provider domain only | Via federation governance | +| Recovery Policy | All domains | Tenant domain only | Provider domain only | Via federation governance | +| Orchestration Flow Policy | All domains | Tenant domain only | โŒ | โŒ | +| Governance Matrix Rule | All domains | Tenant domain only | โŒ | โŒ | +| Lifecycle Policy | All domains | Tenant domain (on their entities) | โŒ | โŒ | +| Accreditation | All | โŒ | Their own accreditations | Vouching for their providers | +| Sovereignty Zone | โœ… | โŒ | โŒ | โŒ | +| DCMGroup / Resource Group | All | Tenant domain only | โŒ | โŒ | +| Notification Subscription | All | Their Tenant only | โŒ | โŒ | +| Webhook Registration | All | Their Tenant only | โŒ | โŒ | + +--- + +## 4. The Contribution Flow + +All contributions โ€” regardless of contributor type โ€” flow through the same GitOps PR model. What varies is: +- **The target store** (which GitOps repository receives the PR) +- **The review requirement** (auto-approval vs human review vs dual approval) +- **The shadow mode behavior** (policies enter shadow mode automatically; other artifacts enter proposed status) + +### 4.1 The Universal Contribution Pipeline + +``` +Contributor authors a data artifact + โ”‚ + โ”‚ Via one of three contribution surfaces: + โ”œโ”€โ”€ Flow GUI Canvas / Policy Authoring Interface + โ”œโ”€โ”€ Direct API (POST /api/v1/contribute/{artifact_type}) + โ””โ”€โ”€ Git PR directly to the target repository + โ”‚ + โ–ผ Artifact submitted โ†’ status: developing (local only) + โ”‚ + โ–ผ Contributor submits for review โ†’ status: proposed + โ”‚ For policies: shadow mode activates automatically + โ”‚ For other artifacts: staged in proposed state + โ”‚ + โ–ผ Governance Matrix evaluates the contribution: + โ”‚ Is this contributor permitted to contribute this artifact type? + โ”‚ Is the artifact in the correct domain for this contributor? + โ”‚ Does the artifact pass structural validation? + โ”‚ DENY โ†’ rejected with reason; no further processing + โ”‚ + โ–ผ Review flow (per profile + artifact type): + โ”‚ auto: artifact activates immediately + โ”‚ human_review: one platform admin or designated reviewer approves + โ”‚ dual_approval: two independent reviewers approve + โ”‚ committee: declared DCMGroup reaches quorum + โ”‚ + โ–ผ On approval โ†’ status: active + โ”‚ For policies: shadow mode results reviewed; full enforcement begins + โ”‚ For resource type specs: available in registry + โ”‚ For catalog items: visible in service catalog (per RBAC) + โ”‚ + โ–ผ Lifecycle managed by contributor (deprecate, retire) + Subject to platform admin override at any time +``` + +### 4.2 Review Requirements by Contributor and Artifact Type + +Review requirements are profile-governed. The table below shows defaults: + +| Artifact Type | Platform Admin | Consumer/Tenant | Service Provider | +|--------------|---------------|-----------------|-----------------| +| Tenant-domain policy | auto | human_review (standard+) | human_review | +| Resource Type Spec (Org tier) | auto | โŒ | human_review | +| Resource Type Spec (Community tier) | human_review | โŒ | dual_approval | +| Provider Catalog Item | auto | โŒ | human_review | +| Service Layer | auto | โŒ | human_review | +| Governance Matrix Rule (tenant) | auto | dual_approval | โŒ | +| Governance Matrix Rule (platform) | human_review | โŒ | โŒ | +| Accreditation | human_review | โŒ | human_review | + +**Profile overrides:** +- `dev`: most contributions auto-approved; shadow mode optional +- `standard`: consumer policies require human_review; provider specs require human_review +- `prod`: consumer governance matrix rules require dual_approval; provider specs require dual_approval +- `fsi`: all contributions require dual_approval; community registry entries require committee +- `sovereign`: all contributions require committee approval + +--- + +## 5. Consumer Contribution Model + +### 5.1 Consumer as Policy Author + +Consumers are not passive requesters. Tenant admins and designated Tenant members with `policy_author` role can define and maintain their own Tenant-domain policies directly. + +**What this enables:** +- A Payments team defining their own cost ceiling GateKeeper: "Reject any VM request over $500/month" +- An Operations team defining their own expiry Transformation: "All dev VMs get a 30-day TTL injected" +- A Security team defining their own governance matrix rule: "Our Tenant never sends confidential data to unaccredited providers" + +**The scope constraint is enforced by DCM, not by convention.** When a consumer submits a policy with `domain: tenant`, DCM validates that the contributing actor belongs to that Tenant. Attempts to submit platform or system domain policies are rejected by the Governance Matrix at contribution time. + +### 5.2 Consumer Contribution API + +``` +POST /api/v1/contribute/policy + +Authorization: Bearer +X-DCM-Tenant: + +{ + "policy_type": "gatekeeper", + "handle": "tenant/payments/gatekeeper/cost-ceiling", + "domain": "tenant", + "concern_type": "operational", + "enforcement": "soft", + "match": { + "payload_type": "request.policies_evaluated", + "conditions": [ + { "field": "payload.cost_estimate.per_month", "operator": "gt", "value": 500 } + ] + }, + "output": { + "decision": "deny", + "reason": "Estimated monthly cost exceeds Tenant budget ceiling of $500" + }, + "shadow_mode": true, # start in shadow mode (proposed status) + "commit_message": "Add monthly cost ceiling GateKeeper for Payments Tenant" +} + +Response 202 Accepted: +{ + "contribution_uuid": "", + "artifact_type": "policy", + "policy_handle": "tenant/payments/gatekeeper/cost-ceiling", + "status": "proposed", + "shadow_mode": true, + "review_required": true, + "review_type": "human_review", + "reviewer_group": "platform-admins", + "pr_url": "https://git.corp.example.com/dcm-policies/pulls/145", + "shadow_results_url": "/flow/api/v1/shadow/" +} +``` + +### 5.3 Consumer Resource Group and Service Definitions + +Consumers can define their own resource groups and service compositions within their Tenant: + +``` +POST /api/v1/contribute/resource-group + +{ + "handle": "tenant/payments/groups/prod-vms", + "display_name": "Production VMs โ€” Payments", + "group_class": "resource_grouping", + "description": "All production VMs owned by the Payments team", + "membership_policy": { + "auto_include": { + "resource_type": "Compute.VirtualMachine", + "tags": { "team": "payments", "env": "production" } + } + } +} +``` + +--- + +## 6. Service Provider Contribution Model + +### 6.1 Provider as Resource Type Publisher + +Service Providers are not just execution targets โ€” they are first-class contributors of the resource type definitions that consumers request. A provider registering a new virtual machine offering publishes the Resource Type Specification, the Catalog Item, and the Service Layer that consumers use to interact with it. + +**What this enables:** +- A storage team publishing a new `Storage.DistributedVolume` resource type with its full schema, constraints, and cost model +- A networking team publishing provider-specific VLAN configurations as a Catalog Item with their own Service Layer injecting provider-specific defaults +- A platform team publishing an updated `Compute.VirtualMachine` spec with new fields and deprecating old ones + +**Provider contributions flow through the same registry governance as all other registry entries** โ€” submitted as PRs to the organization registry, reviewed per profile requirements, activated when approved. + +### 6.2 Provider Contribution API + +``` +POST /api/v1/provider/contribute/resource-type-spec + +Authorization: mTLS + provider credential + +{ + "resource_type_fqn": "Storage.DistributedVolume", + "tier": "organization", + "version": "1.0.0", + "schema": { + "fields": [ + { "field_name": "capacity_gb", "type": "integer", "required": true }, + { "field_name": "replication_factor", "type": "integer", + "default": 3, "constraint": { "min": 1, "max": 5 } }, + { "field_name": "encryption_at_rest", "type": "boolean", "default": true } + ] + }, + "portability_class": "provider_specific", + "commit_message": "Publish DistributedVolume resource type v1.0.0" +} + +Response 202 Accepted: +{ + "contribution_uuid": "", + "resource_type_fqn": "Storage.DistributedVolume", + "status": "proposed", + "review_required": true, + "review_type": "human_review", + "pr_url": "https://git.corp.example.com/dcm-registry/pulls/89" +} +``` + +### 6.3 Provider Service Layer Contribution + +Providers contribute Service Layers that DCM applies during request assembly for their resource types: + +``` +POST /api/v1/provider/contribute/service-layer + +{ + "resource_type_fqn": "Compute.VirtualMachine", + "layer_handle": "providers/eu-west-prod-1/layers/vm-defaults", + "layer_domain": "service", + "provider_uuid": "", + "version": "2.0.0", + "fields": { + "hypervisor": { "value": "KVM", "metadata": { "override": "immutable" } }, + "network_segment": { "value": "prod-segment-01" }, + "backup_enabled": { "value": true } + } +} +``` + +--- + +## 7. Federation Contribution Model + +### 7.1 Peer DCM as Contributor + +A federated peer DCM is a contributor to the receiving DCM's artifact stores, subject to the federation trust posture. This enables: + +- **Hub DCM contributing policy templates** to Regional DCMs โ€” standard compliance policies distributed from a central Hub +- **Community DCM registry contributions** โ€” a community-maintained DCM instance publishing Verified Community resource type specs to subscribing organizations +- **Provider contributions across DCM boundaries** โ€” a provider registered with DCM-A contributing its resource type specs to DCM-B through a verified federation relationship + +### 7.2 Federation Contribution Trust Model + +Federation contributions inherit the federation trust posture of the contributing peer: + +| Peer trust posture | Contribution review requirement | Artifact types permitted | +|-------------------|--------------------------------|------------------------| +| `verified` | human_review (standard+); auto (dev) | Registry entries, policy templates, service layers | +| `vouched` | human_review always | Registry entries, service layers only | +| `provisional` | Committee approval | Registry entries only (no policies) | + +**Hard rule:** A peer DCM cannot contribute artifacts at a higher domain level than its trust posture permits. A `vouched` peer cannot contribute system-domain policies. This is enforced by the Governance Matrix at the federation contribution boundary. + +### 7.3 Federation Contribution Flow + +``` +Peer DCM publishes a contribution bundle: + Content: resource type specs, policy templates, or layers + Transport: federation tunnel (mTLS, signed, scoped credential) + Metadata: contributing_dcm_uuid, trust_posture, artifact_list + +Receiving DCM evaluates: + 1. Governance Matrix: is this peer permitted to contribute this artifact type? + 2. Signature verification: bundle signed by peer's private key? + 3. Structural validation: artifacts conform to DCM schemas? + 4. Domain scope check: artifacts within peer's permitted domain? + +On validation pass: + Artifacts enter proposed status in receiving DCM's policy/registry store + Review flow per receiving DCM's profile + peer trust posture + +On approval: + Artifacts become active in receiving DCM + Source attribution: contributed_by.dcm_uuid, contributed_by.trust_posture +``` + +### 7.4 Hub DCM Policy Distribution + +In a Hub-Spoke federation, the Hub DCM is the authoritative source for platform-wide policy templates. Regional DCMs subscribe to the Hub's policy distribution feed: + +```yaml +hub_policy_distribution: + hub_dcm_uuid: + distribution_type: push # Hub pushes on policy change + auto_approve_from_hub: true # prod profile: false; dev: true + policy_handles_subscribed: + - "system/compliance/hipaa/*" + - "system/governance/drift-remediation" + # Regional DCM always reviews before activating + # Hub cannot force-activate policies on Regional DCMs +``` + +--- + +## 8. Artifact Lifecycle Across Contributors + +### 8.1 Contributor Ownership and Transfer + +Every artifact is owned by its contributor at creation. Ownership can be transferred: +- Consumer-authored policies transfer to a new Tenant admin when the original actor departs +- Provider-contributed catalog items remain owned by the provider registration +- Federation-contributed artifacts are owned by the contributing peer DCM + +Ownership transfer requires the receiving owner's explicit acceptance (same model as entity ownership transfer in the Consumer API). + +### 8.2 Platform Admin Override + +Platform admins can override any contributor's artifact lifecycle at any time: +- Suspend an active consumer-authored policy that is causing harm +- Retire a provider-contributed resource type spec that is no longer safe +- Reject a proposed federation contribution without providing a public reason (security discretion) + +Override actions are always audited with the overriding admin's actor UUID and reason. + +### 8.3 Deprecation and Sunset + +Contributors deprecate their own artifacts. When a Service Provider deprecates a resource type spec: +1. All consumers using that type receive deprecation notifications +2. A sunset period is declared (minimum: P30D for standard profile; P90D for prod/fsi/sovereign) +3. During sunset: new requests using the deprecated spec are warned; existing resources unaffected +4. After sunset: new requests using the deprecated spec are blocked +5. Platform admin must confirm final retirement + +### 8.4 Orphaned Artifacts + +When a contributor's access is revoked (actor departs, provider deregisters, peer DCM federation ends): +- Active artifacts remain active โ€” orphaned artifacts do not automatically deactivate +- A platform admin is notified: "Artifact tenant/payments/gatekeeper/cost-ceiling has no active owner" +- Platform admin assigns a new owner or explicitly retires the artifact +- Auto-retire-on-orphan is configurable per profile (enabled in sovereign profile; disabled in standard) + +--- + +## 9. The Contribution Store + +All contributed artifacts are stored in the GitOps store with contributor attribution. The directory structure reflects the contributor hierarchy: + +``` +dcm-policy-store/ + system/ # Platform admin authored; DCM built-in + compliance/ + governance/ + orchestration/ + platform/ # Platform admin authored; deployment-specific + security/ + operations/ + tenant/ + / # Consumer/Tenant authored + gatekeeper/ + transformation/ + groups/ + provider/ + / # Provider authored + layers/ + policies/ + federated/ + / # Peer DCM contributed + registry/ + policy-templates/ + +dcm-registry/ + core/ # DCM project maintained + community/ # Community contributed (via community DCM) + / + organization/ # Organization contributed + / +``` + +Every artifact in the store includes a `contributed_by` block in its artifact metadata: + +```yaml +artifact_metadata: + uuid: + handle: "tenant/payments/gatekeeper/cost-ceiling" + version: "1.0.0" + status: active + contributed_by: + contributor_type: consumer # platform_admin | consumer | service_provider | peer_dcm + actor_uuid: # for consumer/platform_admin contributions + tenant_uuid: # for consumer contributions + provider_uuid: # for provider contributions + peer_dcm_uuid: # for federation contributions + contribution_method: api # api | flow_gui | git_pr | federation_push + pr_url: "https://..." # if submitted via PR + reviewed_by: [] # actors who approved + reviewed_at: +``` + +--- + +## 10. Profile-Governed Contribution Defaults + +Each deployment profile has a default contribution policy that governs auto-approval eligibility, required review, and shadow mode defaults: + +```yaml +contribution_policy: + minimal: + consumer_policy_auto_approve: true + provider_spec_auto_approve: true + federation_contribution_auto_approve: true # dev/homelab: trust all + shadow_mode_default: false + + dev: + consumer_policy_auto_approve: true + provider_spec_auto_approve: true + federation_contribution_auto_approve: false # human_review for federation + shadow_mode_default: true # shadow mode on by default + + standard: + consumer_policy_auto_approve: false # human_review for all policies + provider_spec_auto_approve: false + federation_contribution_auto_approve: false + shadow_mode_default: true + shadow_review_period: P7D # 7 days of shadow before promotion + + prod: + consumer_policy_auto_approve: false + consumer_governance_matrix_requires: dual_approval + provider_spec_auto_approve: false + provider_spec_requires: human_review + federation_contribution_requires: human_review + shadow_mode_default: true + shadow_review_period: P14D + + fsi: + consumer_policy_auto_approve: false + consumer_policy_requires: dual_approval + consumer_governance_matrix_requires: dual_approval + provider_spec_requires: dual_approval + federation_contribution_requires: dual_approval + shadow_mode_default: true + shadow_review_period: P30D + min_shadow_divergence_review: true # must review all divergence cases + + sovereign: + consumer_policy_requires: committee + provider_spec_requires: committee + federation_contribution_requires: committee + shadow_mode_default: true + shadow_review_period: P30D + min_shadow_divergence_review: true + auto_retire_orphaned_artifacts: true # orphaned artifacts retire automatically +``` + +--- + +## 11. Governance Matrix Integration + +The Governance Matrix evaluates every contribution at submission time. This is the enforcement point for the contributor permission table in Section 2.3. + +**Contribution evaluation:** + +```yaml +governance_matrix_rule: + handle: "system/matrix/consumer-policy-scope" + enforcement: hard + match: + subject.type: consumer + data.artifact_type: policy + data.domain: [system, platform] # consumer attempting non-tenant domain + decision: DENY + reason: "Consumers may only contribute tenant-domain policies" + +governance_matrix_rule: + handle: "system/matrix/provider-spec-scope" + enforcement: hard + match: + subject.type: service_provider + data.artifact_type: resource_type_spec + data.resource_type_fqn: + not_in: subject.declared_resource_types # provider contributing type they don't offer + decision: DENY + reason: "Providers may only contribute Resource Type Specs for resource types they offer" +``` + +--- + +## 12. System Policies + +| Policy | Rule | +|--------|------| +| `FCM-001` | Every DCM data artifact has a contributor. The contributor is recorded in artifact_metadata.contributed_by at creation and is immutable. | +| `FCM-002` | Contributor permissions are enforced by the Governance Matrix at submission time. Domain scope violations are hard DENY โ€” they cannot be overridden by the contributor. | +| `FCM-003` | All contributions flow through the GitOps PR model. No contributor can write directly to the authoritative artifact store without a PR review (unless the active profile grants auto-approval for that contributor type and artifact type combination). | +| `FCM-004` | Policies submitted by any contributor enter proposed (shadow) status by default. Shadow mode results must be available before the active profile's shadow_review_period expires. | +| `FCM-005` | Platform admins may override any contributor's artifact lifecycle at any time. Override actions are audited. | +| `FCM-006` | Orphaned artifacts (contributor access revoked) do not automatically deactivate. A platform admin assigns a new owner or explicitly retires them. Exception: sovereign profile auto-retires orphaned artifacts. | +| `FCM-007` | Federation contributions from peer DCMs are scoped by the peer's federation trust posture. Verified peers: human_review (standard+). Vouched peers: human_review always. Provisional peers: committee approval. | +| `FCM-008` | Contributor-tier scope limits are absolute. A consumer-authored policy in the tenant domain cannot affect the system or platform domain regardless of the policy's declared match conditions. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From d4fdb1f076d39a5f444c5c88ba5ef6654f6bd912 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 23:07:28 -0500 Subject: [PATCH 37/49] Resynced all the documents. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 40 +++++++++++-------- .../accreditation-and-authorization-matrix.md | 3 +- .../audit-provenance-observability.md | 3 +- .../architecture/data-model/auth-providers.md | 3 +- .../data-model/context-and-purpose.md | 3 +- .../data-model/control-plane-components.md | 3 +- .../architecture/data-model/dcm-federation.md | 7 +++- .../data-model/deployment-redundancy.md | 3 +- .../data-model/entity-relationships.md | 3 +- .../architecture/data-model/entity-types.md | 3 +- .../docs/architecture/data-model/examples.md | 3 +- .../architecture/data-model/four-states.md | 3 +- .../data-model/governance-matrix.md | 7 +++- .../information-providers-advanced.md | 3 +- .../data-model/information-providers.md | 3 +- .../data-model/ingestion-model.md | 3 +- .../data-model/layering-and-versioning.md | 3 +- .../data-model/notification-model.md | 3 +- .../data-model/operational-models.md | 3 +- .../ownership-sharing-allocation.md | 3 +- .../data-model/policy-profiles.md | 3 +- .../data-model/registry-governance.md | 3 +- .../data-model/resource-grouping.md | 3 +- .../data-model/resource-service-entities.md | 3 +- .../data-model/resource-type-hierarchy.md | 3 +- .../data-model/service-dependencies.md | 3 +- .../data-model/storage-providers.md | 3 +- .../data-model/universal-audit.md | 3 +- .../data-model/universal-groups.md | 3 +- .../data-model/webhooks-messaging.md | 3 +- 30 files changed, 88 insertions(+), 47 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index ee5c25a..ee9e220 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -3568,7 +3568,8 @@ DRC-001 through DRC-005. Nine control plane components now fully defined in doc --- -## SECTION 61 โ€” FEDERATED CONTRIBUTION MODEL (doc 28) + +## SECTION 54 โ€” FEDERATED CONTRIBUTION MODEL (doc 28) ### Core Principle DCM defaults to a federated model for data creation, import, usage, and lifecycle. Every authorized actor type can contribute Data artifacts within their permitted domain scope. The same GitOps PR flow and lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired) applies to all contributors. Profile-bound auto-approval governs what requires human review. @@ -3582,7 +3583,7 @@ DCM defaults to a federated model for data creation, import, usage, and lifecycl ### Contributor Permission Boundaries (hard DENY โ€” Governance Matrix enforced) - Consumers cannot contribute system or platform domain policies - Providers cannot contribute specs for resource types they don't offer -- Provisional peers can only contribute registry entries (no policies) +- Provisional peers: registry entries only (no policies; committee approval) - Vouched peers: registry entries + service layers only (human_review always) - Verified peers: registry entries + policy templates + service layers (human_review standard+; auto dev) @@ -3595,14 +3596,14 @@ Submit โ†’ Governance Matrix evaluates contributor permissions โ†’ proposed stat - Peer DCM: registry entries, policy templates (verified peers), service layers (verified/vouched) ### Contribution Store Directory Structure -`dcm-policy-store/system/` (platform admin), `platform/` (platform admin), `tenant//` (consumer), `provider//` (provider), `federated//` (peer DCM) -`dcm-registry/core/` (DCM project), `community//` (community), `organization//` (org) +`dcm-policy-store/system/` (platform admin) ยท `platform/` (platform admin) ยท `tenant//` (consumer) ยท `provider//` (provider) ยท `federated//` (peer DCM) +`dcm-registry/core/` (DCM project) ยท `community//` (community) ยท `organization//` (org) -Every artifact includes `contributed_by` block: contributor_type, actor/tenant/provider/peer_dcm UUID, contribution_method (api/flow_gui/git_pr/federation_push), pr_url, reviewed_by. +Every artifact includes `contributed_by` block: contributor_type, actor/tenant/provider/peer_dcm UUID, contribution_method (api/flow_gui/git_pr/federation_push), pr_url, reviewed_by. Immutable once set. ### Profile-Governed Auto-Approval - minimal/dev: most contributions auto-approved; shadow optional -- standard: consumer/provider policies โ†’ human_review; shadow default on, P7D +- standard: consumer/provider policies โ†’ human_review; shadow default on, P7D review period - prod: governance matrix rules โ†’ dual_approval; provider specs โ†’ human_review; shadow P14D - fsi: all consumer/provider contributions โ†’ dual_approval; shadow P30D; must review all divergence cases - sovereign: all โ†’ committee; shadow P30D; orphaned artifacts auto-retire @@ -3610,13 +3611,15 @@ Every artifact includes `contributed_by` block: contributor_type, actor/tenant/p ### Consumer API Contribution Endpoints (Section 9) `POST /api/v1/contribute/policy` (generates PR, activates shadow mode) ยท `POST /api/v1/contribute/resource-group` (activates immediately) ยท `GET /api/v1/contribute` (list contributions) ยท `DELETE /api/v1/contribute/{uuid}` (withdraw, closes PR) -### FCM-001 through FCM-008 system policies -FCM-001: contributor recorded in artifact_metadata.contributed_by; immutable. FCM-002: domain scope violations are hard DENY. FCM-003: all contributions via GitOps PR (except auto-approve profiles). FCM-004: policies enter shadow mode by default. FCM-005: platform admin override always available; audited. FCM-006: orphaned artifacts don't auto-deactivate (except sovereign). FCM-007: federation contribution scoped by trust posture. FCM-008: contributor-tier scope limits absolute โ€” tenant domain policy cannot affect system/platform domain regardless of match conditions. +### Organization Sub-Tiers (Registry) +Three-tier model extended to all artifact types: `organization/platform` (platform admin authored), `organization/provider` (provider authored, scoped to their types), `organization/tenant` (consumer authored, scoped to their Tenant). Lower sub-tier = lower inherent trust = may require additional review. ---- +### FCM-001 through FCM-008 System Policies +FCM-001: contributor recorded in contributed_by; immutable. FCM-002: domain scope violations = hard DENY. FCM-003: all contributions via GitOps PR (except auto-approve). FCM-004: policies enter shadow mode by default. FCM-005: platform admin override always available; audited. FCM-006: orphaned artifacts don't auto-deactivate (except sovereign). FCM-007: federation contribution scoped by trust posture. FCM-008: contributor scope limits absolute. +--- -## SECTION 54 โ€” TERMINOLOGY GLOSSARY +## SECTION 55 โ€” TERMINOLOGY GLOSSARY | Term | Definition | |------|-----------| @@ -3934,7 +3937,7 @@ FCM-001: contributor recorded in artifact_metadata.contributed_by; immutable. FC --- -## SECTION 55 โ€” COMMUNITY QUESTIONS RESOLVED +## SECTION 56 โ€” COMMUNITY QUESTIONS RESOLVED All 21 previously open community/implementation questions are now resolved. Key decisions: @@ -3971,7 +3974,7 @@ All 21 previously open community/implementation questions are now resolved. Key --- -## SECTION 56 โ€” PREVIOUSLY OPEN QUESTIONS (NOW CLOSED) +## SECTION 57 โ€” PREVIOUSLY OPEN QUESTIONS (NOW CLOSED) These items are explicitly unresolved. Do not make assumptions about them โ€” flag them and ask for guidance. @@ -4068,7 +4071,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 57 โ€” EXAMPLES AND USE CASES (dcm-examples.md) +## SECTION 58 โ€” EXAMPLES AND USE CASES (dcm-examples.md) ### Orchestration Examples (8 scenarios) @@ -4116,7 +4119,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 58 โ€” CAPABILITIES MATRIX UPDATE (119 capabilities, 19 domains) +## SECTION 59 โ€” CAPABILITIES MATRIX UPDATE (119 capabilities, 19 domains) Five new domains added to the capabilities matrix. Total: 119 capabilities across 19 domains. @@ -4140,7 +4143,7 @@ consumer-api-spec.md (missing complete lifecycle endpoint coverage) ยท dcm-flow- --- -## SECTION 59 โ€” DOCUMENTATION STRUCTURE +## SECTION 60 โ€” DOCUMENTATION STRUCTURE DCM documentation follows a hierarchical structure: @@ -4188,7 +4191,7 @@ content/ --- -## SECTION 60 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 61 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: @@ -4258,6 +4261,11 @@ When working on this project, follow these instructions: 98. **Information Provider authority is layer-defined** โ€” static organizational knowledge ("our CMDB is authoritative for business unit") belongs in a platform domain layer; conflict detection happens at ingestion time; policy governs automated resolution 99. **DCM Provider is the ninth provider type** โ€” always mTLS (non-configurable); sovereignty checks mandatory; local policies govern ALL federated resources; audit records in both DCM instances with shared correlation_id 100. **Provider federation eligibility is layer-defined with policy enforcement** โ€” platform layer sets defaults per provider type; individual registrations may be more restrictive; storage providers default to mode: none; remote DCMs cannot decommission local resources through tunnels + +172. **DCM defaults to federated data creation** โ€” platform admins are not the only contributors; consumers author tenant-domain policies; providers publish resource type specs and service layers; peer DCMs contribute registry entries; all via GitOps PR with profile-governed review +173. **Contributor domain scope is hard DENY at submission** โ€” consumers cannot contribute system/platform policies regardless of declared domain; providers cannot contribute specs for types they don't offer; enforced by Governance Matrix at contribution time (FCM-002) +174. **All contributed policies enter shadow mode by default** โ€” proposed status with shadow evaluation before activation; shadow_review_period is profile-governed (P7D standard โ†’ P30D fsi/sovereign); platform admin reviews divergence cases before promoting +175. **Orphaned artifacts do not auto-deactivate** โ€” when contributor's access is revoked, their active artifacts remain active until platform admin assigns new owner or explicitly retires; exception: sovereign profile auto-retires orphaned artifacts (FCM-006) 93. **Process Resource max_execution_time is mandatory** โ€” it is not optional metadata; enforced by the Lifecycle Constraint Enforcer; profile governs the default on_max_exceeded action (notify/escalate/terminate) 94. **Dependency graphs are embedded, not separate entities** โ€” declared graph in Resource Type Specification; resolved graph in placement.yaml (Requested State); realized graph in Realized State events; no separate dependency graph artifact needed 95. **Billing state is first-class โ€” not metadata** โ€” DCM carries the billing_state field; policy determines the billing model per resource type and state; Cost Analysis consumes it; organizations decide what is billable diff --git a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md index cd1fb70..9f0ed2f 100644 --- a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md +++ b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md @@ -1,5 +1,5 @@ --- -title: "Accreditation, Data Authorization Matrix, and Zero Trust" +title: "DCM Data Model โ€” Accreditation, Data Authorization Matrix, and Zero Trust" type: docs weight: 26 --- @@ -19,6 +19,7 @@ weight: 26 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + POLICY** > diff --git a/content/docs/architecture/data-model/audit-provenance-observability.md b/content/docs/architecture/data-model/audit-provenance-observability.md index a09ad34..b1f0687 100644 --- a/content/docs/architecture/data-model/audit-provenance-observability.md +++ b/content/docs/architecture/data-model/audit-provenance-observability.md @@ -1,5 +1,5 @@ --- -title: "Audit, Provenance, and Observability" +title: "DCM Data Model โ€” Audit, Provenance, and Observability" type: docs weight: 12 --- @@ -20,6 +20,7 @@ weight: 12 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/auth-providers.md b/content/docs/architecture/data-model/auth-providers.md index 7bcbe56..7906190 100644 --- a/content/docs/architecture/data-model/auth-providers.md +++ b/content/docs/architecture/data-model/auth-providers.md @@ -1,5 +1,5 @@ --- -title: "Authentication, Authorization, and Auth Providers" +title: "DCM Data Model โ€” Authentication, Authorization, and Auth Providers" type: docs weight: 19 --- @@ -19,6 +19,7 @@ weight: 19 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: PROVIDER** > diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md index 290c7bc..45caebd 100644 --- a/content/docs/architecture/data-model/context-and-purpose.md +++ b/content/docs/architecture/data-model/context-and-purpose.md @@ -1,5 +1,5 @@ --- -title: "Context and Purpose" +title: "DCM Data Model โ€” Context and Purpose" type: docs weight: 0 --- @@ -20,6 +20,7 @@ weight: 0 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index 661dea3..9707c55 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -1,5 +1,5 @@ --- -title: "Control Plane Components" +title: "DCM Data Model โ€” Control Plane Components" type: docs weight: 25 --- @@ -19,6 +19,7 @@ weight: 25 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: RUNTIME** > diff --git a/content/docs/architecture/data-model/dcm-federation.md b/content/docs/architecture/data-model/dcm-federation.md index e0db782..8b11533 100644 --- a/content/docs/architecture/data-model/dcm-federation.md +++ b/content/docs/architecture/data-model/dcm-federation.md @@ -1,5 +1,5 @@ --- -title: "DCM Federation and Cross-Instance Coordination" +title: "DCM Data Model โ€” DCM Federation, Peering, and Cross-Instance Coordination" type: docs weight: 22 --- @@ -11,7 +11,7 @@ weight: 22 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). **Document Status:** ๐Ÿ”„ In Progress -**Related Documents:** [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md) +**Related Documents:** [Federated Contribution Model](28-federated-contribution-model.md) | [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md) > **Foundation Document Reference** > @@ -19,6 +19,7 @@ weight: 22 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: PROVIDER + POLICY** > @@ -28,6 +29,8 @@ weight: 22 --- +> **Federated Contribution:** Federation contribution follows the [Federated Contribution Model](28-federated-contribution-model.md) โ€” peer DCMs are contributors to each other's artifact stores, scoped by their federation trust posture. + ## 1. Purpose DCM instances do not operate in isolation. Organizations with multiple data centers, regions, or organizational boundaries may run multiple DCM instances that need to coordinate, share resources, and maintain consistent governance. This document defines how DCM instances relate to each other โ€” as peers, in parent-child hierarchies, or as hub-and-spoke configurations โ€” and the mechanisms for cross-instance resource sharing, data export/import, and provider federation eligibility. diff --git a/content/docs/architecture/data-model/deployment-redundancy.md b/content/docs/architecture/data-model/deployment-redundancy.md index a06b0ca..9f4d088 100644 --- a/content/docs/architecture/data-model/deployment-redundancy.md +++ b/content/docs/architecture/data-model/deployment-redundancy.md @@ -1,5 +1,5 @@ --- -title: "Deployment and Redundancy Model" +title: "DCM Data Model โ€” Deployment and Redundancy Model" type: docs weight: 17 --- @@ -19,6 +19,7 @@ weight: 17 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + PROVIDER** > diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md index 8b2ab97..19473f3 100644 --- a/content/docs/architecture/data-model/entity-relationships.md +++ b/content/docs/architecture/data-model/entity-relationships.md @@ -1,5 +1,5 @@ --- -title: "Entity Relationships" +title: "DCM Data Model โ€” Entity Relationships" type: docs weight: 9 --- @@ -20,6 +20,7 @@ weight: 9 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + POLICY** > diff --git a/content/docs/architecture/data-model/entity-types.md b/content/docs/architecture/data-model/entity-types.md index d6bed1c..d13ce03 100644 --- a/content/docs/architecture/data-model/entity-types.md +++ b/content/docs/architecture/data-model/entity-types.md @@ -1,5 +1,5 @@ --- -title: "Entity Types" +title: "DCM Data Model โ€” Entity Types" type: docs weight: 1 --- @@ -19,6 +19,7 @@ weight: 1 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/examples.md b/content/docs/architecture/data-model/examples.md index 67b9b4b..ab29e93 100644 --- a/content/docs/architecture/data-model/examples.md +++ b/content/docs/architecture/data-model/examples.md @@ -1,5 +1,5 @@ --- -title: "Worked Examples" +title: "DCM Data Model โ€” Worked Examples" type: docs weight: 4 --- @@ -19,6 +19,7 @@ weight: 4 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + PROVIDER + POLICY** > diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md index 6312f48..2bfb861 100644 --- a/content/docs/architecture/data-model/four-states.md +++ b/content/docs/architecture/data-model/four-states.md @@ -1,5 +1,5 @@ --- -title: "The Four States and Storage Model" +title: "DCM Data Model โ€” The Four States" type: docs weight: 2 --- @@ -20,6 +20,7 @@ weight: 2 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/governance-matrix.md b/content/docs/architecture/data-model/governance-matrix.md index 7d5f19d..2afb204 100644 --- a/content/docs/architecture/data-model/governance-matrix.md +++ b/content/docs/architecture/data-model/governance-matrix.md @@ -1,5 +1,5 @@ --- -title: "Unified Governance Matrix" +title: "DCM Data Model โ€” Unified Governance Matrix" type: docs weight: 27 --- @@ -19,16 +19,19 @@ weight: 27 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: POLICY** > > The Policy abstraction โ€” Governance Matrix Rule output schema for boundary control -**Related Documents:** [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) | [DCM Federation](22-dcm-federation.md) | [Policy Profiles](14-policy-profiles.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Control Plane Components](25-control-plane-components.md) +**Related Documents:** [Federated Contribution Model](28-federated-contribution-model.md) | [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) | [DCM Federation](22-dcm-federation.md) | [Policy Profiles](14-policy-profiles.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Control Plane Components](25-control-plane-components.md) --- +> **Federated Contribution:** The Governance Matrix enforces contributor permission boundaries at artifact submission time. See the [Federated Contribution Model](28-federated-contribution-model.md) for the complete contributor permission table and the hard DENY rules applied to out-of-scope contributions. + ## 1. Purpose The Unified Governance Matrix is the **single, declarative, multi-dimensional control surface** that governs every cross-boundary interaction in DCM. It answers one question at every interaction point: diff --git a/content/docs/architecture/data-model/information-providers-advanced.md b/content/docs/architecture/data-model/information-providers-advanced.md index 0accb6e..4bc4699 100644 --- a/content/docs/architecture/data-model/information-providers-advanced.md +++ b/content/docs/architecture/data-model/information-providers-advanced.md @@ -1,5 +1,5 @@ --- -title: "Information Providers: Advanced" +title: "DCM Data Model โ€” Information Providers: Confidence Scoring, Authority, and Conflict Resolution" type: docs weight: 21 --- @@ -19,6 +19,7 @@ weight: 21 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: PROVIDER** > diff --git a/content/docs/architecture/data-model/information-providers.md b/content/docs/architecture/data-model/information-providers.md index f452dd0..cd8d54f 100644 --- a/content/docs/architecture/data-model/information-providers.md +++ b/content/docs/architecture/data-model/information-providers.md @@ -1,5 +1,5 @@ --- -title: "Information Providers" +title: "DCM Data Model โ€” Information Providers" type: docs weight: 10 --- @@ -20,6 +20,7 @@ weight: 10 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: PROVIDER** > diff --git a/content/docs/architecture/data-model/ingestion-model.md b/content/docs/architecture/data-model/ingestion-model.md index 6ac9e8c..1215ea6 100644 --- a/content/docs/architecture/data-model/ingestion-model.md +++ b/content/docs/architecture/data-model/ingestion-model.md @@ -1,5 +1,5 @@ --- -title: "Ingestion Model" +title: "DCM Data Model โ€” Ingestion Model" type: docs weight: 13 --- @@ -19,6 +19,7 @@ weight: 13 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + PROVIDER** > diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md index 6aae80b..d63db4b 100644 --- a/content/docs/architecture/data-model/layering-and-versioning.md +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -1,5 +1,5 @@ --- -title: "Data Layers and Assembly" +title: "DCM Data Model โ€” Data Layers and the Assembly Process" type: docs weight: 3 --- @@ -20,6 +20,7 @@ weight: 3 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/notification-model.md b/content/docs/architecture/data-model/notification-model.md index 44a04c5..d321a0c 100644 --- a/content/docs/architecture/data-model/notification-model.md +++ b/content/docs/architecture/data-model/notification-model.md @@ -1,5 +1,5 @@ --- -title: "Notification Model" +title: "DCM Data Model โ€” Notification Model" type: docs weight: 23 --- @@ -19,6 +19,7 @@ weight: 23 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: PROVIDER + POLICY** > diff --git a/content/docs/architecture/data-model/operational-models.md b/content/docs/architecture/data-model/operational-models.md index 1dad08e..3c091f5 100644 --- a/content/docs/architecture/data-model/operational-models.md +++ b/content/docs/architecture/data-model/operational-models.md @@ -1,5 +1,5 @@ --- -title: "Operational Models" +title: "DCM Data Model โ€” Operational Models" type: docs weight: 24 --- @@ -19,6 +19,7 @@ weight: 24 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: POLICY** > diff --git a/content/docs/architecture/data-model/ownership-sharing-allocation.md b/content/docs/architecture/data-model/ownership-sharing-allocation.md index c6c17ec..ceb75f1 100644 --- a/content/docs/architecture/data-model/ownership-sharing-allocation.md +++ b/content/docs/architecture/data-model/ownership-sharing-allocation.md @@ -1,5 +1,5 @@ --- -title: "Ownership, Sharing, and Allocation" +title: "DCM Data Model โ€” Ownership, Sharing, and Allocation" type: docs weight: 5 --- @@ -19,6 +19,7 @@ weight: 5 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 66b2d4d..202c0e2 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -1,5 +1,5 @@ --- -title: "Policy Organization: Groups, Profiles, and Providers" +title: "DCM Data Model โ€” Policy Organization: Groups, Profiles, and Policy Providers" type: docs weight: 14 --- @@ -21,6 +21,7 @@ weight: 14 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + POLICY** > diff --git a/content/docs/architecture/data-model/registry-governance.md b/content/docs/architecture/data-model/registry-governance.md index 0695d79..41a644d 100644 --- a/content/docs/architecture/data-model/registry-governance.md +++ b/content/docs/architecture/data-model/registry-governance.md @@ -1,5 +1,5 @@ --- -title: "Registry Governance" +title: "DCM Data Model โ€” Registry Governance" type: docs weight: 20 --- @@ -19,6 +19,7 @@ weight: 20 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + PROVIDER** > diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md index 7f8d6e9..53da258 100644 --- a/content/docs/architecture/data-model/resource-grouping.md +++ b/content/docs/architecture/data-model/resource-grouping.md @@ -1,5 +1,5 @@ --- -title: "Resource Grouping and Tenancy" +title: "DCM Data Model โ€” Resource Grouping" type: docs weight: 8 --- @@ -22,6 +22,7 @@ weight: 8 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md index c9c5261..8a629b8 100644 --- a/content/docs/architecture/data-model/resource-service-entities.md +++ b/content/docs/architecture/data-model/resource-service-entities.md @@ -1,5 +1,5 @@ --- -title: "Resource and Service Entities" +title: "DCM Data Model โ€” Resource/Service Entities" type: docs weight: 6 --- @@ -20,6 +20,7 @@ weight: 6 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA + PROVIDER** > diff --git a/content/docs/architecture/data-model/resource-type-hierarchy.md b/content/docs/architecture/data-model/resource-type-hierarchy.md index 23764b2..b7d3b3f 100644 --- a/content/docs/architecture/data-model/resource-type-hierarchy.md +++ b/content/docs/architecture/data-model/resource-type-hierarchy.md @@ -1,5 +1,5 @@ --- -title: "Resource Type Hierarchy and Service Catalog" +title: "DCM Data Model โ€” Resource Type Hierarchy and Service Catalog" type: docs weight: 5 --- @@ -20,6 +20,7 @@ weight: 5 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/service-dependencies.md b/content/docs/architecture/data-model/service-dependencies.md index ffa35f1..e038e02 100644 --- a/content/docs/architecture/data-model/service-dependencies.md +++ b/content/docs/architecture/data-model/service-dependencies.md @@ -1,5 +1,5 @@ --- -title: "Service Dependencies" +title: "DCM Data Model โ€” Service Dependencies" type: docs weight: 7 --- @@ -20,6 +20,7 @@ weight: 7 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md index d93ac43..d0dcdc2 100644 --- a/content/docs/architecture/data-model/storage-providers.md +++ b/content/docs/architecture/data-model/storage-providers.md @@ -1,5 +1,5 @@ --- -title: "Storage Providers" +title: "DCM Data Model โ€” Storage Providers" type: docs weight: 11 --- @@ -20,6 +20,7 @@ weight: 11 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: PROVIDER** > diff --git a/content/docs/architecture/data-model/universal-audit.md b/content/docs/architecture/data-model/universal-audit.md index 91a1fc6..bb15ce6 100644 --- a/content/docs/architecture/data-model/universal-audit.md +++ b/content/docs/architecture/data-model/universal-audit.md @@ -1,5 +1,5 @@ --- -title: "Universal Audit Model" +title: "DCM Data Model โ€” Universal Audit Model" type: docs weight: 16 --- @@ -19,6 +19,7 @@ weight: 16 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md index ad09885..4b59e7b 100644 --- a/content/docs/architecture/data-model/universal-groups.md +++ b/content/docs/architecture/data-model/universal-groups.md @@ -1,5 +1,5 @@ --- -title: "Universal Group Model" +title: "DCM Data Model โ€” Universal Group Model" type: docs weight: 15 --- @@ -19,6 +19,7 @@ weight: 15 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: DATA** > diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md index 2a830c4..0f31362 100644 --- a/content/docs/architecture/data-model/webhooks-messaging.md +++ b/content/docs/architecture/data-model/webhooks-messaging.md @@ -1,5 +1,5 @@ --- -title: "Webhooks, Messaging, and External Integration" +title: "DCM Data Model โ€” Webhooks, Messaging, and External Integration" type: docs weight: 18 --- @@ -19,6 +19,7 @@ weight: 18 > The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in > [00-foundations.md](00-foundations.md). All concepts in this document map to one or > more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) > > **This document maps to: PROVIDER** > From 9da0d4c26c1fd48c74610d44fa447803ab7ef520 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 23:14:21 -0500 Subject: [PATCH 38/49] Found some missing items in the docs, another sync round. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 4 +++- .../architecture/data-model/universal-groups.md | 6 +++--- content/docs/capabilities-matrix.md | 16 ++++++++++++++-- 3 files changed, 20 insertions(+), 6 deletions(-) diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index ee9e220..0289ebb 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4121,7 +4121,7 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl ## SECTION 59 โ€” CAPABILITIES MATRIX UPDATE (119 capabilities, 19 domains) -Five new domains added to the capabilities matrix. Total: 119 capabilities across 19 domains. +Five new domains added to the capabilities matrix. Total: 126 capabilities across 20 domains. **New domains:** @@ -4133,6 +4133,8 @@ Five new domains added to the capabilities matrix. Total: 119 capabilities acros **19. Drift Reconciliation (DRC-001โ€“005):** Drift record production with field-level detail, unsanctioned change classification and severity escalation, drift severity classification (field criticality ร— change magnitude), drift resolution tracking with clean-state confirmation, governance matrix integration for expected provider change detection. +**20. Federated Contribution Model (FCM-001โ€“007):** Consumer policy authoring (policy_author role, PR + shadow mode), provider resource type publication (registry PR for admin review), provider service layer contribution, consumer resource group and definition contribution, federation contribution by peer DCMs (scoped by trust posture), contribution review and lifecycle management (withdraw/status/orphan assignment), contributor scope enforcement (hard DENY via Governance Matrix at contribution time). FCM-001โ€“007 are off the critical path โ€” they extend multi-user capabilities but are not required for the initial end-to-end demonstration. + **6 specifications graduated from WIP to Draft:** dcm-registration-spec.md ยท dcm-opa-integration-spec.md ยท 11-kubernetes-compatibility.md ยท dcm-operator-interface-spec.md ยท dcm-operator-sdk-api.md ยท dcm-admin-api-spec.md diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md index 4b59e7b..df08d8f 100644 --- a/content/docs/architecture/data-model/universal-groups.md +++ b/content/docs/architecture/data-model/universal-groups.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Universal Group Model" +title: "Universal Group Model" type: docs weight: 15 --- @@ -447,8 +447,8 @@ Migration uses the standard ingestion model โ€” existing constructs are ingested | # | Question | Impact | Status | |---|----------|--------|--------| | 1 | Should composite group policy targeting emit a linting warning when no member_type_filter is declared? | Operational safety | โœ… Resolved โ€” linting warning (not error) when composite policy targeting has no member_type filter; suppress with explicit_no_filter: true (GRP-016) | -| 2 | Should there be a maximum nesting depth for tenant_boundary groups? | Operational governance | โ“ Unresolved โ€” future consideration | -| 3 | How does group membership interact with the Search Index? | Performance | โ“ Unresolved โ€” future consideration | +| 2 | Should there be a maximum nesting depth for tenant_boundary groups? | Operational governance | โœ… Resolved โ€” Maximum nesting depth is profile-governed: standard/prod = 5 levels; fsi/sovereign = 3 levels. Deeper nesting creates policy inheritance complexity and audit graph depth issues. Enforced at group creation time. | +| 3 | How does group membership interact with the Search Index? | Performance | โœ… Resolved โ€” Group membership is indexed in the Search Index as a field on each entity record (member_of_groups: [uuid, ...]). The Search Index supports querying by group_uuid. Group membership changes trigger an incremental index update (not full rebuild). Staleness follows the standard Search Index model (PT5M standard profile). | | 4 | Should time-bounded memberships (valid_until) trigger notifications before expiry? | Consumer experience | โœ… Resolved โ€” warn_before_expiry field on membership (GRP-014) | --- diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index 074ccfb..f3ad236 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -262,9 +262,18 @@ | DRC-004 | Drift Resolution Tracking | View drift record status (open/acknowledged/resolved/escalated); receive resolved notification when next discovery confirms clean state | โ€” | Monitor drift resolution rates; configure escalation policies for aged-open drift records | DRC-001, DRF-004 | | DRC-005 | Governance Matrix Drift Integration | โ€” | โ€” | Configure governance matrix check in drift comparison pipeline: expected provider changes are not flagged as drift | DRC-001, GMX-001 | ---- +## 20. Federated Contribution Model +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| FCM-001 | Consumer Policy Authoring | Author and submit tenant-domain policies (GateKeeper, Transformation, Recovery, Lifecycle, Orchestration Flow, Governance Matrix rules) via API or Flow GUI; receive PR URL and shadow mode results | โ€” | Configure consumer policy authoring permissions (policy_author role); manage review requirements per profile | POL-001, IAM-003, IAM-007 | +| FCM-002 | Provider Resource Type Publication | โ€” | Publish Resource Type Specifications and Catalog Items for offered resource types via provider contribution API; receive registry PR for platform admin review | Manage provider contribution registry; configure review requirements for provider specs; manage Organization-tier registry | PRV-001, GOV-003 | +| FCM-003 | Provider Service Layer Contribution | โ€” | Contribute Service Layers for offered resource types; layers applied during request assembly for all consumers requesting that resource type | Review and activate provider-contributed layers; manage layer compatibility | PRV-001, LAY-002 | +| FCM-004 | Consumer Resource Group and Definition Contribution | Author and manage resource groups, notification subscriptions, webhook registrations, and cross-tenant authorization records within own Tenant | โ€” | Configure contribution permissions per role; manage Tenant-scoped artifact lifecycle | IAM-007, GOV-002 | +| FCM-005 | Federation Contribution (Peer DCM) | โ€” | Peer DCM contributes registry entries, policy templates, and service layers via federation channels | Manage federation contribution trust posture (verified/vouched/provisional); configure review requirements per trust posture; manage cross-DCM artifact lifecycle | FED-001, GOV-003, POL-003 | +| FCM-006 | Contribution Review and Lifecycle | View contribution status (proposed, pending_review, active, withdrawn); withdraw a pending contribution; receive notification when contribution is approved or rejected | Receive notification when provider contributions are reviewed | Review and approve/reject contributions via Admin API; manage shadow review periods; assign new owners to orphaned artifacts | POL-002, POL-003 | +| FCM-007 | Contributor Scope Enforcement | Receive clear DENY response when attempting to contribute outside permitted domain scope | Receive DENY when contributing specs for resource types not offered | Monitor Governance Matrix enforcement at contribution time; configure scope violation audit and notification | GMX-001, GMX-002 | --- @@ -292,7 +301,8 @@ | Zero Trust and Security Posture | 6 | | Unified Governance Matrix | 7 | | Drift Reconciliation | 5 | -| **Total** | **119** | +| Federated Contribution Model | 7 | +| **Total** | **126** | --- @@ -323,6 +333,8 @@ IAM-001 โ†’ IAM-002 โ†’ IAM-003 โ†’ IAM-007 โ†’ CAT-001 โ†’ REQ-001 โ†’ REQ-002 **21 capabilities for a functional end-to-end demonstration.** +**Note:** FCM-001 through FCM-007 (Federated Contribution Model) are not on the critical path โ€” they extend DCM's multi-user capabilities but are not required for the initial end-to-end lifecycle demonstration. + --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From 271675150090c373a15bca3709cfc40bbb068eac Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sat, 28 Mar 2026 23:33:44 -0500 Subject: [PATCH 39/49] Found more docs out of sync, taxonomy being a key one. Signed-off-by: Chris Roadfeldt --- content/docs/README.md | 85 ++++ .../architecture/data-model/auth-providers.md | 15 +- .../data-model/control-plane-components.md | 3 + .../federated-contribution-model.md | 2 +- .../architecture/data-model/foundations.md | 4 +- .../data-model/information-providers.md | 4 +- .../data-model/layering-and-versioning.md | 2 +- .../data-model/policy-contract.md | 2 +- .../data-model/policy-profiles.md | 2 +- .../data-model/provider-contract.md | 2 +- .../data-model/storage-providers.md | 4 +- .../data-model/universal-groups.md | 2 +- content/docs/architecture/overview.md | 362 +++++++++--------- .../specifications/admin-api-spec.md | 2 +- .../specifications/consumer-api-spec.md | 2 +- .../architecture/specifications/examples.md | 5 +- .../specifications/flow-gui-spec.md | 2 +- .../kubernetes-compatibility.md | 6 +- .../specifications/opa-integration-spec.md | 2 +- .../specifications/operator-interface-spec.md | 10 +- .../specifications/operator-sdk-api.md | 6 +- .../specifications/registration-spec.md | 8 +- content/docs/taxonomy.md | 336 +++++----------- 23 files changed, 422 insertions(+), 446 deletions(-) create mode 100644 content/docs/README.md diff --git a/content/docs/README.md b/content/docs/README.md new file mode 100644 index 0000000..60ecd95 --- /dev/null +++ b/content/docs/README.md @@ -0,0 +1,85 @@ +--- +title: "DCM Documentation Index" +type: docs +weight: 0 +--- + +Data Center Management (DCM) is an open-source governing framework for enterprise on-premises and sovereign cloud infrastructure. It provides a hyperscaler-like cloud experience on infrastructure that organizations own and control. + +**GitHub:** https://github.com/dcm-project + +--- + +## Architecture in One Sentence + +DCM is built on three foundational abstractions โ€” **Data**, **Provider**, and **Policy** โ€” connected by a policy-driven event loop. Every concept maps to one of these three. See [00-foundations.md](data-model/00-foundations.md). + +--- + +## Documentation Structure + +### Foundation Documents (read these first) +| Document | Purpose | +|----------|---------| +| [00-foundations.md](data-model/00-foundations.md) | The three abstractions โ€” Data, Provider, Policy | +| [A-provider-contract.md](data-model/A-provider-contract.md) | Unified Provider base contract + 11 typed extensions | +| [B-policy-contract.md](data-model/B-policy-contract.md) | Unified Policy base contract + 7 output schemas | + +### Data Model (28 documents) +| Range | Coverage | +|-------|---------| +| 00โ€“05 | Context, entity types, four states, layering, examples, ownership, resource types | +| 06โ€“11 | Resource/service entities, dependencies, grouping, relationships, information providers, storage providers | +| 12โ€“17 | Audit, ingestion, policy profiles, universal groups, universal audit, deployment | +| 18โ€“23 | Webhooks, auth providers, registry governance, advanced information providers, federation, notifications | +| 24โ€“28 | Operational models, control plane components, accreditation, governance matrix, federated contribution | + +### Specifications (10 documents) +Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI ยท Registration ยท Examples ยท Kubernetes Compatibility ยท Operator SDK ยท CNCF Strategy + +### AI Model Prompt +[DCM-AI-PROMPT.md](DCM-AI-PROMPT.md) โ€” paste into any AI model to provide full project context. 62 sections, 4,330 lines. + +--- + +## Key Numbers + +| Metric | Value | +|--------|-------| +| Foundational abstractions | 3 (Data, Provider, Policy) | +| Provider types | 11 (unified base contract + typed capability extensions) | +| Policy types | 7 (unified base contract + typed output schemas) | +| Control plane components | 9 | +| Four lifecycle states | Intent ยท Requested ยท Realized ยท Discovered | +| Capabilities | 126 across 20 domains | +| Data model documents | 33 (28 numbered + 3 foundation + 2 examples) | +| Specifications | 10 | +| Unresolved questions | 0 | + +--- + +## Core Principles + +1. **Declarative** โ€” data describes desired state, not procedures +2. **API-First** โ€” every capability available via standard API +3. **Policy-Governed** โ€” all business logic through the Policy Engine, never hard-coded +4. **Idempotent** โ€” applying the same data multiple times produces the same result +5. **Immutable if Versioned** โ€” published versions never change; changes produce new versions +6. **Provider-Agnostic** โ€” DCM defines contracts, not implementations +7. **GitOps-Native** โ€” intent and policy artifacts are Git-native +8. **Federated by Default** โ€” all authorized actor types contribute within permitted scope +9. **Easy to use ยท Easy to implement ยท Easy to extend** + +--- + +## Capabilities Matrix + +[DCM-Capabilities-Matrix.md](DCM-Capabilities-Matrix.md) โ€” 126 capabilities across 20 domains including: Identity and Access, Service Catalog, Request Lifecycle, Provider Contract, Resource Lifecycle, Drift Detection, Policy Management, Data Layer, Information Integration, Ingestion, Audit, Observability, Storage, Federation, Platform Governance, Accreditation, Zero Trust, Governance Matrix, Drift Reconciliation, and Federated Contribution. + +**Minimum viable end-to-end set:** 21 capabilities (IAM-001 โ†’ AUD-001 critical path). + +--- + +## Contributing + +DCM is open-source. Community contributions welcome via GitHub at https://github.com/dcm-project diff --git a/content/docs/architecture/data-model/auth-providers.md b/content/docs/architecture/data-model/auth-providers.md index 7906190..ae700d0 100644 --- a/content/docs/architecture/data-model/auth-providers.md +++ b/content/docs/architecture/data-model/auth-providers.md @@ -42,7 +42,9 @@ Every authentication mode DCM supports โ€” static API key, local users, GitHub O --- -## 2. The Eight Provider Types +## 2. Auth and Credential Provider Types + +Auth Providers and Credential Providers are two of the eleven DCM provider types (see [Unified Provider Contract](A-provider-contract.md)). This section covers the authentication modes and configurations supported: Auth Provider completes the DCM provider ecosystem: @@ -52,10 +54,13 @@ Auth Provider completes the DCM provider ecosystem: | 2 | **Information Provider** | Serves authoritative external data | | 3 | **Meta Provider** | Composes multiple providers | | 4 | **Storage Provider** | Persists DCM state | -| 5 | **Policy Provider** | Supplies and evaluates policies | -| 6 | **Message Bus Provider** | Bridges internal/external event streams | -| 7 | **Credential Provider** | Resolves secrets from external stores | -| 8 | **Auth Provider** | Authenticates identities and resolves permissions | +| 5 | **Policy Provider** | Evaluates policies externally | +| 6 | **Credential Provider** | Manages secrets and credentials | +| 7 | **Auth Provider** | Authenticates actor identities | +| 8 | **Notification Provider** | Delivers notifications | +| 9 | **Message Bus Provider** | Async event streaming | +| 10 | **Registry Provider** | Serves the Resource Type Registry | +| 11 | **Peer DCM** | Another DCM instance (federation) | --- diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index 9707c55..56459f1 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -30,6 +30,9 @@ weight: 25 --- + +The DCM Control Plane consists of **nine components** that implement the three foundational abstractions at runtime. + ## 1. Purpose This document formally defines the DCM control plane components that are referenced throughout the data model documents but not previously specified in detail. Two components are defined here: diff --git a/content/docs/architecture/data-model/federated-contribution-model.md b/content/docs/architecture/data-model/federated-contribution-model.md index c3226d9..cbd323d 100644 --- a/content/docs/architecture/data-model/federated-contribution-model.md +++ b/content/docs/architecture/data-model/federated-contribution-model.md @@ -1,5 +1,5 @@ --- -title: "Federated Contribution Model" +title: "DCM Data Model โ€” Federated Contribution Model" type: docs weight: 28 --- diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md index 3fc13e6..f495765 100644 --- a/content/docs/architecture/data-model/foundations.md +++ b/content/docs/architecture/data-model/foundations.md @@ -1,5 +1,5 @@ --- -title: "DCM Foundational Abstractions" +title: "DCM โ€” Foundational Abstractions" type: docs weight: 0 --- @@ -224,7 +224,7 @@ The Request Orchestrator and Policy Engine are the runtime that connects the thr - Static flows: Orchestration Flow Policies with `ordered: true` - Dynamic flows: conditional Policies that fire based on payload state -**Control plane components as runtime specializations:** +**Control plane components as runtime specializations (nine total):** The components in [Control Plane Components](25-control-plane-components.md) are specialized runtime implementations, not separate abstractions: diff --git a/content/docs/architecture/data-model/information-providers.md b/content/docs/architecture/data-model/information-providers.md index cd8d54f..6834808 100644 --- a/content/docs/architecture/data-model/information-providers.md +++ b/content/docs/architecture/data-model/information-providers.md @@ -50,9 +50,9 @@ Information Providers solve this by giving DCM a standard, stable, governed inte --- -## 3. The Three Provider Types +## 3. Information Provider in the Provider Ecosystem -DCM recognizes three provider types. All follow the same base contract model: +DCM defines eleven provider types, all implementing the unified Provider base contract. This document focuses on the Information Provider type. For the complete provider type list see [A-provider-contract.md](A-provider-contract.md). | Provider Type | Purpose | Data Direction | DCM Owns Result? | |--------------|---------|---------------|-----------------| diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md index d63db4b..658bf7c 100644 --- a/content/docs/architecture/data-model/layering-and-versioning.md +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -1388,7 +1388,7 @@ Layers are merged in precedence order (lowest to highest). For each field: The consumer's Request Layer is applied last in the data layer merge. Consumer-declared values override all data layer values. Each override is recorded in provenance. ### Step 5 โ€” Pre-Placement Policy Processing -Policies with `placement_phase: pre` (or `both`) are evaluated against the merged payload before any provider is known. Three policy types execute in order: +Policies matching the `request.layers_assembled` payload type are evaluated against the merged payload before any provider is known. GateKeeper, Transformation, Validation, and Governance Matrix policies may all fire at this stage โ€” evaluated by the Policy Engine in domain precedence order: 1. **Transformation Policies** โ€” enrich and modify the payload. May set `override: constrained` on fields. Each transformation records the policy UUID, operation type, reason, and any override control declarations in provenance. 2. **Validation Policies** โ€” check the payload against rules. Pass/fail only โ€” no field modification. Failures reject the request. diff --git a/content/docs/architecture/data-model/policy-contract.md b/content/docs/architecture/data-model/policy-contract.md index 9529d5b..f88a1ed 100644 --- a/content/docs/architecture/data-model/policy-contract.md +++ b/content/docs/architecture/data-model/policy-contract.md @@ -1,5 +1,5 @@ --- -title: "Unified Policy Contract" +title: "DCM โ€” Unified Policy Contract" type: docs weight: -8 --- diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 202c0e2..76bde3f 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -740,7 +740,7 @@ When a profile is in `proposed` status, all its constituent policies run in shad ## 4. Policy Providers -### 4.1 The Fifth Provider Type +### 4.1 Policy Provider (one of eleven DCM provider types) A **Policy Provider** is a fifth DCM provider type โ€” an external authoritative source that supplies policies directly into DCM or evaluates and enriches DCM data through an external logic engine. diff --git a/content/docs/architecture/data-model/provider-contract.md b/content/docs/architecture/data-model/provider-contract.md index 105c119..071dee1 100644 --- a/content/docs/architecture/data-model/provider-contract.md +++ b/content/docs/architecture/data-model/provider-contract.md @@ -1,5 +1,5 @@ --- -title: "Unified Provider Contract" +title: "DCM โ€” Unified Provider Contract" type: docs weight: -9 --- diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md index d0dcdc2..a71b5ba 100644 --- a/content/docs/architecture/data-model/storage-providers.md +++ b/content/docs/architecture/data-model/storage-providers.md @@ -38,7 +38,9 @@ This is consistent with DCM's governing framework philosophy: DCM does not presc --- -## 2. The Four Provider Types +## 2. Storage Provider Sub-Types + +Storage Providers are one of eleven DCM provider types (see [Unified Provider Contract](A-provider-contract.md)). Within the Storage Provider type, four storage sub-types are defined, each optimized for different access patterns and consistency requirements: | Provider Type | Purpose | Data Direction | DCM Owns Result? | |--------------|---------|---------------|-----------------| diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md index df08d8f..2715e3f 100644 --- a/content/docs/architecture/data-model/universal-groups.md +++ b/content/docs/architecture/data-model/universal-groups.md @@ -1,5 +1,5 @@ --- -title: "Universal Group Model" +title: "DCM Data Model โ€” Universal Group Model" type: docs weight: 15 --- diff --git a/content/docs/architecture/overview.md b/content/docs/architecture/overview.md index ce7921b..08cd926 100644 --- a/content/docs/architecture/overview.md +++ b/content/docs/architecture/overview.md @@ -7,44 +7,33 @@ weight: 1 # DCM High Level Design > **โš ๏ธ Active Development Notice** -> -> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. -> +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). +--- ## What is DCM? -DCM (Data Center Management) is an open-source **governing framework** for -enterprise on-premises and sovereign cloud infrastructure. It provides a -hyperscaler-like cloud experience โ€” the operational model and self-service -capabilities of a public cloud provider โ€” on infrastructure that organizations -own and control. +DCM (Data Center Management) is an open-source **governing framework** for enterprise on-premises and sovereign cloud infrastructure. It provides a hyperscaler-like cloud experience โ€” the operational model and self-service capabilities of a public cloud provider โ€” on infrastructure that organizations own and control. -DCM is **not a provisioning tool**. It is the management plane that sits above -provisioning tools, governing what gets requested, approved, built, owned, and -decommissioned. Provisioning tools (Ansible, Terraform, Kubernetes operators) -become Service Providers that DCM orchestrates. +DCM is **not a provisioning tool**. It is the management plane that sits above provisioning tools, governing what gets requested, approved, built, owned, and decommissioned. Provisioning tools (Ansible, Terraform, Kubernetes operators) become Service Providers that DCM orchestrates. -**Mission:** Seamlessly manage the complete lifecycle of all data center -infrastructure by providing a policy-governed, data-driven, and unified platform -to enable and ensure sovereignty. +**Mission:** Seamlessly manage the complete lifecycle of all data center infrastructure by providing a policy-governed, data-driven, and unified platform to enable and ensure sovereignty. --- ## The Problem DCM Solves -Enterprise organizations managing private cloud infrastructure face consistent -challenges that public cloud providers have already solved โ€” and that DCM brings -to on-premises: - | Challenge | DCM Response | |-----------|-------------| | **Fragmented operations** โ€” disparate tools, no unified control | Single control plane โ€” one API, one data model, one policy engine | | **No source of truth** โ€” multiple CMDBs diverge | Four-state model provides authoritative record of intent, request, realized, and discovered state | -| **High time-to-market** โ€” a VM may require dozens of teams | Self-service catalog with policy-governed automation | -| **Drift and state discrepancy** โ€” no reconciliation between intended and actual | Continuous drift detection comparing realized vs discovered state | -| **Sovereignty requirements** โ€” data residency, compliance, audit evidence | Policy Engine with sovereignty enforcement, complete provenance chain | +| **High time-to-market** โ€” provisioning a VM may require dozens of teams | Self-service catalog with policy-governed automation โ€” any authorized actor can request any service | +| **Drift and state discrepancy** โ€” no reconciliation between intended and actual | Continuous drift detection comparing realized vs discovered state with automated or human-directed remediation | +| **Sovereignty requirements** โ€” data residency, compliance, audit evidence | Unified Governance Matrix with sovereignty zone enforcement, complete provenance chain, and accreditation management | +| **Siloed governance** โ€” platform admins bottleneck policy changes | Federated contribution model โ€” consumers, providers, and peer DCMs all contribute within their permitted scope | --- @@ -54,226 +43,245 @@ to on-premises: |-----------|---------| | **Declarative** | Data describes what should exist, not how to achieve it | | **API-First** | Every capability is available via a standard API | -| **Policy-Governed** | All business logic flows through the Policy Engine โ€” not hard-coded | +| **Policy-Governed** | All business logic flows through the Policy Engine โ€” never hard-coded | | **Idempotent** | Applying the same data multiple times always produces the same result | | **Immutable if Versioned** | Published versions never change โ€” changes produce new versions | | **Provider-Agnostic** | DCM defines contracts, not implementations | -| **GitOps-Native** | Intent and Requested state are Git-native โ€” branched, reviewed, versioned | -| **Kubernetes Superset** | DCM extends Kubernetes upward โ€” operators become DCM Service Providers | +| **GitOps-Native** | Intent and policy artifacts are Git-native โ€” branched, reviewed, versioned | +| **Federated by Default** | All authorized actor types contribute data within their permitted scope | +| **AI-Ready** | Standalone architecture designed with AIOps layering in mind | --- -## Architecture Overview +## The Three Foundational Abstractions -DCM consists of four major architectural layers: +Every concept in DCM maps to one of three foundational abstractions. There is no fourth. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ DATA โ”‚ +โ”‚ Everything that exists, is stored, has a lifecycle. โ”‚ +โ”‚ Entities, layers, policies, accreditations, audit records, โ”‚ +โ”‚ groups, relationships โ€” all Data. โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ flows through + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ PROVIDER โ”‚ โ”‚ POLICY โ”‚ +โ”‚ Every external โ”‚ โ”‚ Every rule that fires on Data, โ”‚ +โ”‚ component DCM โ”‚ โ”‚ decides what happens, transforms โ”‚ +โ”‚ calls or that โ”‚ โ”‚ values, or enforces constraints. โ”‚ +โ”‚ calls DCM. โ”‚ โ”‚ Seven typed output schemas. โ”‚ +โ”‚ Eleven typed โ”‚ โ”‚ One evaluation algorithm. โ”‚ +โ”‚ capability โ”‚ โ”‚ Same lifecycle for all. โ”‚ +โ”‚ extensions. โ”‚ โ”‚ โ”‚ +โ”‚ One base contract. โ”‚ โ”‚ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` +**The runtime loop:** ``` -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ CONSUMER INGRESS โ”‚ -โ”‚ Web UI โ”‚ Consumer API โ”‚ Direct API (3rd Rail) โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ CONTROL PLANE โ”‚ -โ”‚ โ”‚ -โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ -โ”‚ โ”‚ Service โ”‚ โ”‚ Request โ”‚ โ”‚ Policy Engine โ”‚ โ”‚ -โ”‚ โ”‚ Catalog โ”‚ โ”‚ Payload โ”‚ โ”‚ โ”‚ โ”‚ -โ”‚ โ”‚ โ”‚ โ”‚ Processor โ”‚ โ”‚ Transformation โ”‚ โ”‚ -โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ Validation โ”‚ โ”‚ -โ”‚ โ”‚ GateKeeper โ”‚ โ”‚ -โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ -โ”‚ โ”‚ IDM / IAM โ”‚ โ”‚ Audit & โ”‚ โ”‚ -โ”‚ โ”‚ โ”‚ โ”‚ Observ. โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ -โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ API Gateway โ”‚ โ”‚ -โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ - โ”‚ -โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” -โ”‚ PROVIDERS โ”‚ -โ”‚ โ”‚ -โ”‚ Service Providers โ”‚ Information Providers โ”‚ -โ”‚ Meta Providers โ”‚ Storage Providers โ”‚ -โ”‚ โ”‚ -โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ -โ”‚ โ”‚ KubeVirt โ”‚ โ”‚ VMware โ”‚ โ”‚ OpenStackโ”‚ โ”‚ HR / Finance โ”‚ โ”‚ -โ”‚ โ”‚ AAP โ”‚ โ”‚ Bare โ”‚ โ”‚ CAPI โ”‚ โ”‚ CMDB / ITSM โ”‚ โ”‚ -โ”‚ โ”‚ CloudNPG โ”‚ โ”‚ Metal โ”‚ โ”‚ Storage โ”‚ โ”‚ Custom โ”‚ โ”‚ -โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ -โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +Event (Data state change) + โ†’ Policy Engine evaluates all matching Policies + โ†’ Policies produce decisions / mutations / actions + โ†’ Actions invoke Providers or produce new Data + โ†’ New Data triggers new Events โ†’ repeat ``` +See [Foundational Abstractions](data-model/foundations/) for the complete model. + --- ## The Four States -Every resource in DCM exists across four independently maintained state records: +Every resource entity in DCM has four independently maintained lifecycle stages stored in specialized stores: -| State | Question | Store | -|-------|----------|-------| -| **Intent** | What did the consumer ask for? | GitOps โ€” immutable, branched, PR reviewed | -| **Requested** | What was approved and dispatched? | GitOps โ€” assembled, policy-processed, full provenance | -| **Realized** | What did the provider actually build? | Event Stream โ€” append-only, entity-keyed | -| **Discovered** | What actually exists right now? | Event Stream โ€” ground truth for drift detection | +| State | What it records | Store | +|-------|----------------|-------| +| **Intent** | What did the consumer declare? | GitOps โ€” immutable, PR-reviewed | +| **Requested** | What was assembled, validated, and dispatched? | Write-once snapshot โ€” full provenance | +| **Realized** | What did the provider confirm it built? | Snapshot store โ€” append-only | +| **Discovered** | What actually exists right now, independently observed? | Ephemeral stream โ€” ground truth for drift | -The **entity UUID** is the universal linking key โ€” assigned at Intent State creation, it links the entity across all four states and all stores throughout its entire lifecycle. +The **entity UUID** links the entity across all four states throughout its entire lifecycle. --- -## The Data Model +## Provider Model -The DCM Data Model is the foundational layer that governs how all data is -represented, versioned, assembled, and governed. Key concepts: +DCM defines contracts, not implementations. Eleven provider types all implement the **unified Provider base contract** (registration, health, sovereignty, accreditation, governance matrix enforcement, zero trust). What varies is the capability extension. -**Data Layers** assemble a complete request payload from composable, versioned -units of configuration. 36 layer definitions can govern 40,000 VMs without -duplication. Layers are organized by Domain (system, platform, tenant, service, -provider), identified by a human-readable Handle, and ordered by a hierarchical -Priority Schema for deterministic conflict resolution. +| Provider Type | Capability | +|--------------|-----------| +| **Service Provider** | Realizes infrastructure resources (VMs, networks, storage, containers) | +| **Information Provider** | Serves authoritative external data (CMDB, HR, Finance) | +| **Storage Provider** | Persists DCM state (GitOps stores, event streams, audit) | +| **Meta Provider** | Composes multiple providers into compound services | +| **Policy Provider** | Evaluates policies externally (OPA sidecar, Mode 1โ€“4) | +| **Credential Provider** | Issues and rotates secrets and credentials | +| **Auth Provider** | Authenticates actor identities | +| **Notification Provider** | Delivers notifications via configured channels | +| **Message Bus Provider** | Async event streaming | +| **Registry Provider** | Serves the Resource Type Registry | +| **Peer DCM** | Another DCM instance โ€” federation is the Provider abstraction applied across instances | -**Resource Types** are portable, vendor-neutral definitions of resource classes. -They live in the DCM Resource Type Registry alongside Information Types -(Business.*, Identity.*, Compliance.*) โ€” same registry, different category prefix. +See [Unified Provider Contract](data-model/provider-contract/) for the base contract and all capability extensions. -**Entity Relationships** use a universal bidirectional model for all connections -between entities โ€” whether VM-to-Storage, Application-to-WebServer, or -Resource-to-BusinessUnit. One model, all relationships. +--- -**Field Override Control** uses a graduated three-level model: no declaration -(allow by default), simple `override: immutable`, or a full actor-permission -matrix. The Policy Engine is the sole authority for setting override control. +## Policy Model -**Artifact Metadata** is universal โ€” every layer, policy, resource type, catalog -item, and provider registration carries a standard metadata block with creator, -owner, modification history, and contact information. +Policies are the orchestration in DCM. Seven typed output schemas, one base contract, one evaluation algorithm. + +| Policy Type | Output | Fires on | +|-------------|--------|---------| +| **GateKeeper** | allow / deny | Request payload | +| **Validation** | pass / fail + field detail | Request payload | +| **Transformation** | field mutations | Request payload | +| **Recovery** | action + parameters | Failure/timeout trigger | +| **Orchestration Flow** | step sequence | Pipeline events (named workflows) | +| **Governance Matrix Rule** | ALLOW / DENY / STRIP_FIELD / REDACT | Any cross-boundary interaction | +| **Lifecycle Policy** | action on related entity | Relationship events | + +**Two-level orchestration:** +- **Level 1 โ€” Named Workflow Artifacts:** Orchestration Flow Policy with `ordered: true` โ€” explicit, visible, auditable step sequence. This is the named pipeline skeleton. +- **Level 2 โ€” Dynamic Policies:** GateKeeper, Transformation, Recovery policies fire when conditions match, alongside workflow steps, without being declared in the workflow. + +Both levels are evaluated by the same Policy Engine through the same event bus. See [Unified Policy Contract](data-model/policy-contract/). --- -## Provider Model +## Unified Governance Matrix + +The Governance Matrix is the single enforcement point for all cross-boundary data and capability decisions. It governs every interaction between DCM and any provider, peer DCM, or external endpoint. + +**Four axes per rule:** Subject (who) ยท Data (what โ€” including field-level paths) ยท Target (where โ€” sovereignty zone, jurisdiction, accreditation) ยท Context (profile, zero trust posture, TLS state) + +**Decision vocabulary:** ALLOW ยท DENY ยท ALLOW_WITH_CONDITIONS ยท STRIP_FIELD ยท REDACT ยท AUDIT_ONLY -DCM defines **contracts**, not implementations. Four provider types: +**Hard vs soft enforcement:** Hard rules cannot be relaxed by any downstream rule. `sovereign` and `classified` data never crossing any boundary is always hard. -| Type | Purpose | -|------|---------| -| **Service Provider** | Realizes resources โ€” KubeVirt, VMware, Ansible, Terraform | -| **Information Provider** | Serves authoritative external data DCM references but does not own | -| **Meta Provider** | Composes multiple providers into higher-order services | -| **Storage Provider** | Persists DCM state โ€” GitOps stores, event streams, audit store | +**Profile-bound defaults:** Each profile activates a set of default matrix rules. Organizations tighten (never relax) with Tenant and resource-type overrides. -All providers follow the same registration, health check, trust, and contract model. +See [Unified Governance Matrix](data-model/governance-matrix/). --- -## Policy Engine +## Federated Contribution Model -The Policy Engine is the single authoritative logic gate for all business rules. -Three policy types in execution order: +DCM defaults to a federated model for data creation. Every authorized actor type can contribute Data artifacts within their permitted scope โ€” all via the same GitOps PR model with profile-governed review. -1. **Transformation** โ€” enriches and modifies the payload; may set field constraints -2. **Validation** โ€” checks payload against rules; pass/fail, no modification -3. **GateKeeper** โ€” highest authority; can override any field; enforces sovereignty +| Contributor | Can contribute | +|-------------|---------------| +| **Platform Admin** | All artifact types at all domains | +| **Consumer / Tenant** | Tenant-domain policies, resource groups, notification subscriptions, service definitions | +| **Service Provider** | Resource Type Specs (their types), catalog items, service layers, provider policies | +| **Peer DCM** | Registry entries, policy templates, service layers (scoped by federation trust posture) | -Policies follow a three-tier hierarchy: Global โ†’ Tenant โ†’ User. A Global policy -cannot be overridden by Tenant or User policies. The Policy Engine uses OPA/Rego -for policy implementation. +Contributor scope is enforced by the Governance Matrix as a hard DENY โ€” a consumer cannot contribute system-domain policies regardless of what they declare. -Policies support five statuses: **developing** (dev mode only), **proposed** -(shadow execution โ€” output captured but not applied, for validation), -**active**, **deprecated**, and **retired**. +See [Federated Contribution Model](data-model/federated-contribution-model/). --- -## Kubernetes as a Superset +## Control Plane Components -DCM is designed as a **superset of Kubernetes** โ€” extending Kubernetes' declarative -model upward to the management plane: +Nine internal components implement the three abstractions at runtime: -| Kubernetes | DCM | -|-----------|-----| -| Single cluster | Multi-cluster, multi-infrastructure | -| Namespace isolation | First-class Tenant ownership model | -| RBAC + admission webhooks | Policy Engine with field-level override control | -| No cost attribution | Full lifecycle cost analysis | -| No cross-cluster management | Unified management plane | - -Kubernetes operators become DCM Service Providers through the -[DCM Operator Interface Specification](specifications/operator-interface-spec/). -The [DCM Operator SDK](specifications/operator-sdk-api/) enables Level 1 conformance -in one day. +| Component | Role | +|-----------|------| +| **Request Orchestrator** | Event bus โ€” no pipeline logic; policies define all behavior | +| **Policy Engine** | Evaluates all policy types using the same algorithm | +| **Placement Engine** | Six-step provider selection (sovereignty โ†’ accreditation โ†’ capability โ†’ reserve query โ†’ tie-breaking โ†’ confirm) | +| **Cost Analysis** | Pre-request estimation and ongoing attribution | +| **Lifecycle Constraint Enforcer** | Monitors TTL/expiry; fires expiry actions through the standard pipeline | +| **Discovery Scheduler** | Schedules and dispatches discovery requests to Service Providers | +| **Notification Router** | Resolves notification audiences from the relationship graph | +| **Drift Reconciliation** | Compares Discovered vs Realized state; produces drift records; never writes to Realized Store | +| **Search Index** | Non-authoritative queryable projection of GitOps stores; always rebuildable | --- -## Digital Sovereignty +## Zero Trust and Security -DCM addresses four sovereignty dimensions: +DCM operates on a network-position-grants-zero-trust model. Every interaction boundary applies five checks regardless of the caller's network location: -| Dimension | DCM Enabler | -|-----------|-------------| -| **Data and Content Sovereignty** | Data Model, Policy Engine, Validated Providers | -| **Operational Sovereignty** | Policy Engine โ€” Sovereign Execution Posture | -| **Security and Compliance** | Audit, GRC, complete provenance chain | -| **Mobility and Placement** | Policy Engine placement constraints, provider portability | +``` +Identity verification (mTLS) + โ†’ Authorization verification (scoped credential) + โ†’ Accreditation check (does the target hold required certs?) + โ†’ Governance Matrix check (are the fields permitted to cross?) + โ†’ Sovereignty check (does the endpoint satisfy constraints?) +``` -**Sovereign Execution Posture** โ€” the target end state where all operations are -governed, auditable, and compliant with sovereignty requirements. This is the -north star concept of DCM. +All five checks produce audit records regardless of outcome. Profile-governed zero trust posture: none (minimal) โ†’ boundary (dev/standard) โ†’ full (prod/fsi) โ†’ hardware_attested (sovereign). --- ## Request Lifecycle -A complete request lifecycle from consumer intent to realized resource: +A complete path from consumer intent to realized resource: ``` -Consumer submits request +Consumer submits request (API, Web UI, or Git PR) โ”‚ - โ–ผ [Git branch created โ€” CI pipeline fires] -Intent State captured (immutable consumer declaration) - โ”‚ CI: policy pre-validation, cost estimate, sovereignty check - โ”‚ Human review via PR (if policy requires) - โ–ผ [PR merged โ€” CD pipeline fires] -Request Payload Processor โ€” Nine-Step Assembly + โ–ผ Intent State captured โ€” versioned GitOps artifact + โ”‚ Policy pre-validation (shadow mode); cost estimate; sovereignty check โ”‚ - โ”‚ Steps 1-4: Layer assembly - โ”‚ Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request Layer + โ–ผ Request Payload Processor: + โ”‚ 1โ€“4: Layer assembly (Base โ†’ Core โ†’ Service โ†’ Request Layer) + โ”‚ 5: Pre-placement policies (Transformation, Validation, GateKeeper) + โ”‚ 6: Placement Engine โ€” sovereignty filter โ†’ accreditation filter โ†’ + โ”‚ capability filter โ†’ parallel reserve queries โ†’ tie-breaking โ†’ + โ”‚ confirm selection + โ”‚ 7: Post-placement policies (provider-aware enrichment) + โ”‚ 8: Requested State written to write-once store (full provenance) + โ”‚ 9: Provider dispatch โ”‚ - โ”‚ Step 5: Pre-Placement Policies - โ”‚ Transformation โ†’ Validation โ†’ GateKeeper - โ”‚ Outputs: placement constraints + โ–ผ Service Provider: + โ”‚ Naturalize (DCM format โ†’ provider native) + โ”‚ Execute (provision the resource) + โ”‚ Denaturalize (provider native โ†’ DCM unified format) + โ”‚ Return Realized State โ”‚ - โ”‚ Step 6: Placement Engine โ€” Placement Loop - โ”‚ For each candidate provider: - โ”‚ Reserve Query (atomic: verify + metadata + hold) - โ”‚ Loop Policy Phase (evaluates reserve query response) - โ”‚ pass โ†’ Placement confirmed - โ”‚ reject_candidate โ†’ next candidate - โ”‚ gatekeep โ†’ request rejected + โ–ผ Realized State written โ€” confirmed by provider โ”‚ - โ”‚ Step 7: Post-Placement Policies - โ”‚ Transformation โ†’ Validation โ†’ GateKeeper - โ”‚ Provider-aware enrichment and validation - โ”‚ - โ–ผ -Requested State committed to Git (full provenance chain) - โ”‚ Includes: placement block, hold records, policy gap records - โ–ผ -Provider dispatch via API Gateway (hold confirmed) - โ”‚ Naturalization: DCM format โ†’ provider native format - โ”‚ Provider realizes resource, returns full metadata - โ”‚ Denaturalization: provider native โ†’ DCM format - โ–ผ -Realized State (event stream, provider-confirmed) - โ”‚ enrichment_status updated as metadata arrives - โ–ผ [Continuous] -Drift Detection: Discovered State vs Realized State - โ”‚ Unsanctioned changes โ†’ Policy Engine response - โ”‚ Drift โ†’ REVERT | UPDATE | ALERT | ESCALATE + โ–ผ Continuous discovery โ†’ Drift Reconciliation + Discovered State vs Realized State + Drift: field-level detail, severity classification, unsanctioned detection + Response: REVERT | ACCEPT_DRIFT | NOTIFY_AND_WAIT | ESCALATE ``` --- +## Capabilities Summary + +126 capabilities across 20 domains. Full detail in the [Capabilities Matrix](../capabilities-matrix/). + +**Minimum viable end-to-end set (21 capabilities):** +IAM-001 โ†’ IAM-002 โ†’ IAM-003 โ†’ IAM-007 โ†’ CAT-001 โ†’ REQ-001 โ†’ REQ-002 โ†’ REQ-003 โ†’ REQ-004 โ†’ REQ-005 โ†’ REQ-006 โ†’ REQ-007 โ†’ PRV-001 โ†’ PRV-002 โ†’ PRV-003 โ†’ PRV-004 โ†’ PRV-005 โ†’ LCM-001 โ†’ DRF-001 โ†’ DRF-002 โ†’ AUD-001 + +--- + +## APIs and Interfaces + +| Interface | Purpose | +|-----------|---------| +| **Consumer API** | Service catalog, request submission, resource management, drift, groups, notifications, cost, quota, contribution endpoints | +| **Admin API** | Tenant management, provider review, accreditation approval, discovery triggers, orphan resolution, quota, Search Index management | +| **Operator Interface** | What Service Providers implement โ€” dispatch, cancel, discover, health | +| **Flow GUI** | Visual policy composer โ€” execution graph, canvas, simulation, shadow mode, authoring | +| **Flow GUI API** | Backend serving the Flow GUI โ€” graph data, simulation, shadow promotion, canvas PR creation | + +--- + ## Related Documents -- [Data Model](data-model/) โ€” Complete data model documentation including the Ingestion Model for V1 migration and brownfield ingestion -- [Specifications](specifications/) โ€” Operator Interface Specification, Kubernetes compatibility, SDK API, CNCF strategy -- [Enhancements](../enhancements/) โ€” Enhancement proposals for the DCM project +- **[Foundational Abstractions](data-model/foundations/)** โ€” Data, Provider, Policy โ€” read this first +- **[Unified Provider Contract](data-model/provider-contract/)** โ€” base contract + 11 typed extensions +- **[Unified Policy Contract](data-model/policy-contract/)** โ€” base contract + 7 output schemas +- **[Federated Contribution Model](data-model/federated-contribution-model/)** โ€” who contributes what and how +- **[Data Model](data-model/)** โ€” complete 28-document data model reference +- **[Specifications](specifications/)** โ€” Consumer API, Admin API, Operator Interface, OPA Integration, Flow GUI, Registration, Examples, Kubernetes compatibility, SDK, CNCF strategy +- **[Capabilities Matrix](../capabilities-matrix/)** โ€” 126 capabilities across 20 domains diff --git a/content/docs/architecture/specifications/admin-api-spec.md b/content/docs/architecture/specifications/admin-api-spec.md index 0a8134e..3124363 100644 --- a/content/docs/architecture/specifications/admin-api-spec.md +++ b/content/docs/architecture/specifications/admin-api-spec.md @@ -13,7 +13,7 @@ weight: 1 **Version:** 0.1.0-draft **Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification -**Related Documents:** [Consumer API Specification](consumer-api-spec.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Consumer API Specification](consumer-api-spec.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) --- diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index 20511be..e240271 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -11,7 +11,7 @@ weight: 0 **Version:** 0.1.0-draft **Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification -**Related Documents:** [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Four States](../data-model/02-four-states.md) | [Auth Providers](../data-model/19-auth-providers.md) | [Webhooks and Messaging](../data-model/18-webhooks-messaging.md) +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Four States](../data-model/02-four-states.md) | [Auth Providers](../data-model/19-auth-providers.md) | [Webhooks and Messaging](../data-model/18-webhooks-messaging.md) --- diff --git a/content/docs/architecture/specifications/examples.md b/content/docs/architecture/specifications/examples.md index a793b0a..40227b1 100644 --- a/content/docs/architecture/specifications/examples.md +++ b/content/docs/architecture/specifications/examples.md @@ -1,10 +1,11 @@ --- -title: "DCM Examples and Use Cases" +title: "DCM Examples" type: docs -weight: 0 +weight: 7 --- **Document Status:** ๐Ÿ”„ In Progress +**Status:** Draft โ€” Examples document - no WIP status needed; always current with architecture. **Document Type:** Reference Examples **Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Provider Contract](../data-model/A-provider-contract.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) | [Registration](dcm-registration-spec.md) | [OPA Integration](dcm-opa-integration-spec.md) diff --git a/content/docs/architecture/specifications/flow-gui-spec.md b/content/docs/architecture/specifications/flow-gui-spec.md index 9ca91b3..e7b508e 100644 --- a/content/docs/architecture/specifications/flow-gui-spec.md +++ b/content/docs/architecture/specifications/flow-gui-spec.md @@ -11,7 +11,7 @@ weight: 8 **Version:** 0.1.0-draft **Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification -**Related Documents:** [Control Plane Components](../data-model/25-control-plane-components.md) | [OPA Integration Specification](dcm-opa-integration-spec.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [OPA Integration Specification](dcm-opa-integration-spec.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) --- diff --git a/content/docs/architecture/specifications/kubernetes-compatibility.md b/content/docs/architecture/specifications/kubernetes-compatibility.md index 0dee66d..d0f3e82 100644 --- a/content/docs/architecture/specifications/kubernetes-compatibility.md +++ b/content/docs/architecture/specifications/kubernetes-compatibility.md @@ -5,7 +5,9 @@ weight: 6 --- -> ## โš ๏ธ Work in Progress โ€” Kubernetes Operator Integration +> ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress +> +> All questions resolved. Cluster-as-a-Service model defined. Namespace-to-Tenant mapping, admission webhook model, and managed K8s integration all specified. > > **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.** > @@ -25,7 +27,7 @@ weight: 6 **Document Status:** ๐Ÿ”„ In Progress **Document Type:** Architecture Reference -**Related Documents:** [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) --- diff --git a/content/docs/architecture/specifications/opa-integration-spec.md b/content/docs/architecture/specifications/opa-integration-spec.md index ce39549..8d25c9d 100644 --- a/content/docs/architecture/specifications/opa-integration-spec.md +++ b/content/docs/architecture/specifications/opa-integration-spec.md @@ -13,7 +13,7 @@ weight: 5 **Version:** 0.1.0-draft **Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification -**Related Documents:** [Policy Profiles](../data-model/14-policy-profiles.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) --- diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md index 85f8caa..b4df9f2 100644 --- a/content/docs/architecture/specifications/operator-interface-spec.md +++ b/content/docs/architecture/specifications/operator-interface-spec.md @@ -5,7 +5,9 @@ weight: 2 --- -> ## โš ๏ธ Work in Progress โ€” Kubernetes Operator Integration +> ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress +> +> All questions resolved. Level 0โ€“4 conformance levels defined. Cluster-scoped resource ownership clarified. CAPI integration specified. > > **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.** > @@ -24,7 +26,7 @@ weight: 2 **Version:** 0.1.0-draft -**Status:** Draft โ€” Not yet ratified +**Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification **Maintainers:** Red Hat FlightPath Team **GitHub:** https://github.com/dcm-project @@ -38,6 +40,8 @@ This specification defines the interface by which Kubernetes operators integrate DCM is designed as a superset of Kubernetes โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries. This specification is the technical contract that enables that extension without requiring operators to abandon their existing Kubernetes-native design. +Operators conforming to this specification function as Service Providers within a single DCM instance. In federated deployments (Hub-Spoke or Peer topology), the operator registers with the appropriate Regional or local DCM instance โ€” federation routing is handled by DCM, not by the operator. + --- ## 1. Introduction @@ -1047,7 +1051,7 @@ Operators must monitor for changes to DCM-managed CRs that did not originate fro **Q3:** Two distinct models apply, and it is important to not conflate them: -**Model A โ€” Cluster as a Service (the primary model):** A Kubernetes cluster is a catalog item that any authorized Tenant can request and own. The Tenant owns the entire cluster entity โ€” including all cluster-scoped resources within it (ClusterRoles, StorageClasses, PersistentVolumes, etc.) โ€” because the cluster itself is the resource boundary. The `Platform.KubernetesCluster` catalog item is provisioned by a Cluster-as-a-Service Provider (e.g., a CAPI-backed operator). Once provisioned, the cluster is a Resource Entity owned by the requesting Tenant. Everything within that cluster is scoped to that Tenant's ownership. This is the primary model โ€” users and Tenant owners fully expect to request and own clusters as a service through the catalog. +**Model A โ€” Cluster as a catalog item (example Service Provider implementation):** A Kubernetes cluster can be offered as a catalog item that any authorized Tenant requests and owns โ€” this is a natural use of DCM's Service Provider model, not a special architectural feature. From DCM's perspective, `Platform.KubernetesCluster` is simply a resource type whose Service Provider happens to provision Kubernetes clusters (e.g., via CAPI). The Tenant owns the resulting cluster entity, including all cluster-scoped resources within it, because the cluster is the resource boundary. This is an example of how DCM's architecture enables complex resources as services โ€” DCM has no special knowledge of Kubernetes; it treats the cluster as any other resource entity. **Model B โ€” Shared cluster infrastructure (the exception):** When multiple Tenants share a single cluster (the multi-tenant cluster model), cluster-scoped resources that govern the shared infrastructure itself (admission webhook configurations, cluster-level network policies, CRD registrations) cannot be owned by any single Tenant โ€” they belong to the `__platform__` system Tenant. These are resources that, if modified by a Tenant, would affect all other Tenants on the cluster. The distinction: resources *inside* a Tenant-owned cluster are always Tenant-owned; resources that *govern shared cluster infrastructure* belong to `__platform__`. diff --git a/content/docs/architecture/specifications/operator-sdk-api.md b/content/docs/architecture/specifications/operator-sdk-api.md index c799ac5..42b188c 100644 --- a/content/docs/architecture/specifications/operator-sdk-api.md +++ b/content/docs/architecture/specifications/operator-sdk-api.md @@ -5,7 +5,9 @@ weight: 4 --- -> ## โš ๏ธ Work in Progress โ€” Kubernetes Operator Integration +> ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress +> +> All questions resolved. Local durable queue, mock test harness, Prometheus metrics, and dynamic field resolution all specified. > > **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.** > @@ -28,7 +30,7 @@ weight: 4 **Document Type:** Technical Design **Language:** Go **Repository:** https://github.com/dcm-project/operator-sdk -**Related Documents:** [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Kubernetes Compatibility](11-kubernetes-compatibility.md) +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Kubernetes Compatibility](11-kubernetes-compatibility.md) --- diff --git a/content/docs/architecture/specifications/registration-spec.md b/content/docs/architecture/specifications/registration-spec.md index d0feaa6..8adcea6 100644 --- a/content/docs/architecture/specifications/registration-spec.md +++ b/content/docs/architecture/specifications/registration-spec.md @@ -6,14 +6,14 @@ weight: 3 > **๐Ÿ“‹ Draft** > -> This specification has been promoted from Work in Progress to Draft status. All questions resolved. Complete registration pipeline for all 9 provider types with full capability declaration schemas and federation trust model. It is ready for implementation feedback but has not yet been formally reviewed for final release. +> This specification has been promoted from Work in Progress to Draft status. All questions resolved. Complete registration pipeline for all 11 provider types with full capability declaration schemas and federation trust model. It is ready for implementation feedback but has not yet been formally reviewed for final release. > > This specification defines the unified registration flow for all DCM provider types. Published to share design direction and invite feedback. **Version:** 0.1.0-draft **Status:** Draft โ€” Ready for implementation feedback **Document Type:** Technical Specification -**Related Documents:** [Control Plane Components](../data-model/25-control-plane-components.md) | [Governance Matrix](../data-model/27-governance-matrix.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [Governance Matrix](../data-model/27-governance-matrix.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) --- @@ -31,7 +31,7 @@ The Provider Type Registry is the authoritative list of provider types that a DC | Tier | Maintained By | Examples | |------|--------------|---------| -| **Core** | DCM Project | The nine built-in provider types | +| **Core** | DCM Project | The eleven built-in provider types | | **Verified Community** | Named community maintainers | Domain-specific provider types | | **Organization** | Deploying organization | Custom/proprietary integrations | @@ -78,7 +78,7 @@ provider_type_registry_entry: failure_threshold: 3 # failures before degraded status ``` -### 1.3 The Nine Core Provider Types +### 1.3 The Eleven Core Provider Types | # | provider_type_id | Default Approval | Enabled In | |---|-----------------|-----------------|------------| diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index 88cb09f..e742bff 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -1,272 +1,131 @@ --- title: "DCM Taxonomy" type: docs -weight: 5 +weight: 2 --- -> **Purpose:** This document defines the authoritative vocabulary for all DCM architecture, documentation, and implementation work. It ensures precision and clarity by giving every term a distinct, contextual definition and identifying ambiguous terms to avoid. -> -> When contributing to DCM โ€” code, documentation, Jira tickets, design discussions โ€” use these terms consistently. Vocabulary proposals are submitted via PR following standard registry governance. +The DCM taxonomy defines the precise vocabulary used throughout the architecture. Every term used in the data model, specifications, and implementation should conform to these definitions. ---- - -## Part 1 โ€” Core Vocabulary - -### Foundational Architecture Terms - -**Artifact** -Any versioned, GitOps-managed object in DCM โ€” layers, policies, resource type specifications, provider registrations, group definitions, entity declarations. Every artifact carries universal artifact metadata (uuid, handle, version, status, owned_by). Artifacts are immutable once published; changes produce a new version. - -**Assembly Process** -The nine-step process by which DCM transforms a consumer's Intent State into a provider-ready Requested State. Steps: Intent Capture โ†’ Layer Resolution โ†’ Layer Merge โ†’ Request Layer Application โ†’ Pre-Placement Policies โ†’ Placement Engine Loop โ†’ Post-Placement Policies โ†’ Requested State Storage โ†’ Provider Dispatch. The assembly process is the core operational function of the DCM Control Plane. - -**Control Plane** -The central nervous system of DCM. Maintains the Unified API and Data Model, enforces multi-tenancy, executes the assembly process, manages artifact lifecycle, and coordinates Service Providers. Does not directly manage physical infrastructure โ€” manages the data that represents and governs infrastructure. - -**DCM (Data Center Management)** -A framework designed to provide a "hyperscaler-like cloud experience" for on-premises infrastructure. Centralizes and automates the management, observation, and lifecycle of IT/IS services within an enterprise data center. Technology-agnostic โ€” defines roles, responsibilities, interactions, and expected capabilities rather than prescribing specific tools. - -**DCM Instance** -A running deployment of the DCM Control Plane and its associated stores. Manages a defined set of resources, providers, and tenants. Multiple DCM instances may be federated. See: Hub DCM, Regional DCM, Sovereign DCM. - -**Deployment Posture** -How a DCM instance's infrastructure behaves โ€” redundancy, enforcement strictness, audit retention, tenancy model, cross-tenant defaults. Expressed as a Deployment Posture Group (posture-minimal through posture-sovereign). One of the two dimensions of a Profile. - -**Domain (Policy and Layer)** -The organizational and architectural home of a Policy or Layer. Determines authority scope and override precedence. Ordered highest to lowest: system > platform > tenant > service > provider > request. Lower-domain artifacts cannot override higher-domain artifacts. - -**Drift** -The difference between what DCM believes exists (Realized State) and what actually exists (Discovered State). May be authorized (change made through DCM) or unsanctioned (change made outside DCM). The Drift Detection component continuously compares these states and triggers remediation per policy. - -**Entity** -A specific, realized instance of a resource type โ€” a particular VM, a specific VLAN, a named DNS record. Entities have UUIDs preserved across their entire lifecycle including provider migrations. Entities pass through Intent, Requested, Realized, and Discovered states. - -**Four States** -The four representations of a resource in DCM: **Intent State** (consumer's original declaration, stored in Git), **Requested State** (fully assembled, policy-processed payload, stored in Git), **Realized State** (what the provider actually provisioned, stored in Event Stream), **Discovered State** (what active interrogation finds currently exists, stored in Discovered Store). - -**Handle** -The human-readable, stable identifier for an artifact within DCM. Complements the UUID (machine-meaningful). Format: `{domain}/{concern-or-type}/{name}`. Stable across versions โ€” the handle does not change when a new version is published. - -**Intent State** -The consumer's original resource declaration as submitted, stored verbatim in the Intent Store (GitOps) before any assembly processing. Used as the source for rehydration. - -**Layer** -A declarative, immutable, versioned unit of data that contributes field values to an assembled request payload. Layers are passive โ€” they declare values but do not execute logic. Layers answer "what values should these fields have?" See also: Policy. - -**Layer Chain** -The ordered set of layers applied during the assembly process for a specific request. Immutable once assembled. The layer chain is the deduplication key in the deduplicated provenance model. - -**Lifecycle Constraint Enforcer** -The DCM control plane component that enforces time-based constraints โ€” TTLs, valid_from/valid_until on memberships, max_execution_time on Process Resources. - -**Placement Engine** -The DCM control plane component that selects the Service Provider for a request based on constraints, capacity, sovereignty, and policy. Operates within a reserve-query loop during Step 6 of the assembly process. Uses a deterministic seven-step tie-breaking hierarchy. - -**Policy** -An executable rule that evaluates the assembled payload and takes action. Policies answer "given this data, is it valid? what should change? should this proceed?" Three types: GateKeeper (approve or reject), Validation (verify correctness), Transformation (modify or enrich). *Policies are logic; Layers are data โ€” they serve different purposes and must not be confused.* - -**Profile** -A named composition of a Deployment Posture Group and one or more Compliance Domain Groups. Defines the full governance posture for a DCM deployment. DCM ships six core profiles (minimal through sovereign) and eight extended profiles (hipaa-prod, fedramp-moderate, dod-il4, etc.). - -**Provenance** -The full audit trail of where a field value came from and what changed it. Recorded per-field in the assembled payload. Three configurable models: full_inline (all provenance on entity record), deduplicated (content-addressed, layer chain as dedup key), tiered archive (hot/warm/cold tiers). - -**Rehydration** -Replaying an entity's Intent State through the assembly process โ€” potentially to a different provider, in a different context. The entity's UUID is always preserved. Provider-side identifiers change and are recorded in rehydration_history. - -**Requested State** -The fully assembled, policy-processed payload stored in the Requested Store (GitOps). Authoritative record of what was actually requested and how assembly enriched it. Includes: assembled payload, assembly provenance, placement decisions, and dependency resolution. - -**Realized State** -The actual provisioned state of a resource as reported by the Service Provider. Stored in the Realized Store (Event Stream). Includes provider-side identifiers and actual provisioned field values. - -**Resource Type Specification** -The formal definition of a resource type โ€” its fields, constraints, dependencies, lifecycle rules, editable fields, and allowed service providers. Versioned artifacts stored in the Resource Type Registry. The contract between consumers and the assembly process. - -**Sovereignty Zone** -A declared geographic, legal, or organizational boundary within which data must remain. Enforced at the placement engine level. Cannot be overridden by consumer requests. Mandatory check before DCM-to-DCM tunnel establishment. - -**Tenant** -The primary ownership and authorization boundary for resources in DCM. All resources are owned by exactly one Tenant. Provides multi-tenancy isolation. See also: tenant_boundary (group class). - -**Unified Data Model** -The standardized data format used throughout DCM for all resource declarations, provider payloads, and state records. Service Providers implement Naturalization (DCM โ†’ native) and Denaturalization (native โ†’ DCM) for translation. - ---- - -### Provider Types - -**Service Provider** -A DCM component that realizes resources โ€” provisions, configures, and manages physical or virtual infrastructure on behalf of DCM consumers. Responsibilities: naturalization, realization, denaturalization, capacity reporting (reserve_query), and sovereignty declaration maintenance. *This is the DCM taxonomy term for what general software architecture calls a "producer."* - -**Information Provider** -A DCM component that supplies authoritative external data to enrich entity records. Examples: CMDB, IPAM, HR system, asset management. DCM computes confidence scores for all values they supply. *Not to be confused with Service Providers โ€” Information Providers supply data; they do not provision infrastructure.* - -**Meta Provider** -A Service Provider that composes multiple sub-providers to deliver a higher-order service. Declares composition_visibility: opaque, transparent, or selective. - -**Policy Provider** -A DCM component that supplies external policy logic. Four modes: Mode 1 (read-only query), Mode 2 (stateless evaluation), Mode 3 (execute external code), Mode 4 (black-box query with sovereignty checks). Trust elevation requires formal approval workflow. - -**Storage Provider** -A DCM component that persists DCM state. Sub-types: GitOps Store, Event Stream Store, Search Index, Audit Store, Validation Store, Discovered Store. - -**Message Bus Provider** -A DCM component bridging internal and external event streams. Used for curated observability events, webhook delivery, and federation message passing. - -**Credential Provider** -A DCM component that resolves secrets from external stores. Credentials never stored in Git or audit records. - -**Auth Provider** -A DCM component that authenticates identities and resolves permissions. Supports OIDC, LDAP/AD, FreeIPA, GitHub/GitLab OAuth, mTLS, static API key, local users. - -**DCM Provider** -A DCM component that wraps another DCM instance's API for cross-DCM resource sharing. Always mTLS. Sovereignty checks mandatory. Local DCM policies govern all resources from DCM Provider tunnels. +**Purpose:** Eliminate ambiguity. When two people use the same word to mean different things, architecture breaks down. This taxonomy prevents that. --- -### Federation Topology - -**Hub DCM** -The central/global DCM instance. Authoritative registry origin, governance authority, and federation routing hub. Applies placement engine logic to route requests to Regional or Sovereign DCMs. *Replaces "Shore" (defense IT terminology).* - -**Regional DCM** -A distributed regional DCM instance managing resources within its region. Caches layers and catalog items from Hub DCM. Treated as a DCM Provider instance by the Hub DCM's placement engine. *Replaces "Ship" (defense IT terminology).* - -**Sovereign DCM** -An air-gapped or compliance-isolated DCM instance. No live external connectivity. Updates via signed bundles during connectivity windows. Required for classified, sovereign, or highly regulated deployments. *Replaces "Enclave" (defense IT terminology).* - -**Federation Depth** -Number of hops from deepest DCM instance to Hub DCM. Profile-governed maximum: 5 (minimal/dev), 3 (standard/prod), 2 (fsi/sovereign). - -**Federation Trust Score** -0-100 score on a DCM-to-DCM tunnel. Computed from: identity verification, sovereignty compatibility, certifications currency, audit integrity, uptime, compliance. Used in: cross_dcm_confidence = source_confidence ร— (tunnel_trust_score / 100). +## Part 1 โ€” Core Vocabulary ---- +### The Three Foundational Abstractions + +| Term | Definition | +|------|-----------| +| **Data** | Any structured artifact in DCM with a type, UUID, lifecycle state, fields, data classification, and provenance. Everything that exists, is stored, and has a lifecycle. Entities, layers, policies, accreditations, audit records, groups, relationships โ€” all Data. | +| **Provider** | Any external component DCM calls or that calls DCM. Implements the unified Provider base contract (registration, health, sovereignty, accreditation, governance matrix enforcement, zero trust). What varies between provider types is the capability extension. | +| **Policy** | A rule artifact that fires when Data matches declared conditions, produces a typed output (decision, mutation, action, or directive), and is enforced at a declared level. Policies govern every transition, transformation, and constraint in DCM. | + +### Provider Types (11) + +| Term | Definition | +|------|-----------| +| **Service Provider** | Typed Provider. Capability: realize infrastructure resources. Implements naturalization, realization, denaturalization, and discovery. | +| **Information Provider** | Typed Provider. Capability: serve authoritative external data (CMDB, HR, Finance, identity). | +| **Storage Provider** | Typed Provider. Capability: persist DCM state. Sub-types: GitOps, write-once snapshot, event stream, search index, audit. | +| **Meta Provider** | Typed Provider. Capability: compose multiple child providers into a compound service delivered as a single catalog item. | +| **Policy Provider** | Typed Provider. Capability: evaluate policies externally. Modes 1โ€“4; Mode 3โ€“4 for OPA/Rego sidecar and black-box query enrichment. | +| **Credential Provider** | Typed Provider. Capability: issue, rotate, and revoke secrets and credentials. | +| **Auth Provider** | Typed Provider. Capability: authenticate actor identities and resolve role/group memberships. | +| **Notification Provider** | Typed Provider. Capability: deliver notification envelopes to configured channels (Slack, PagerDuty, email, webhook). | +| **Message Bus Provider** | Typed Provider. Capability: async event streaming between DCM and external systems. | +| **Registry Provider** | Typed Provider. Capability: serve the Resource Type Registry (core, community, organization tiers). | +| **Peer DCM** | Typed Provider. Another DCM instance participating in federation. Federation is the Provider abstraction applied across DCM instances. | + +### Policy Types (7) + +| Term | Definition | +|------|-----------| +| **GateKeeper Policy** | Typed Policy. Output: allow/deny + optional field locks. Fires on request payload. Any active GateKeeper producing deny blocks the request. | +| **Validation Policy** | Typed Policy. Output: pass/fail + field-level detail. Fires on request payload. Checks correctness of field values. | +| **Transformation Policy** | Typed Policy. Output: mutations[] โ€” field additions, changes, locks. Fires on request payload. All mutations collected and applied with provenance. | +| **Recovery Policy** | Typed Policy. Output: action + parameters. Fires on failure/timeout trigger conditions. Governs what DCM does when things go wrong. | +| **Orchestration Flow Policy** | Typed Policy. Output: ordered step sequence. Fires on pipeline payload type events. Named workflow artifacts โ€” the explicit, visible pipeline skeleton. | +| **Governance Matrix Rule** | Typed Policy. Output: ALLOW / DENY / ALLOW_WITH_CONDITIONS / STRIP_FIELD / REDACT / AUDIT_ONLY. Fires at every cross-boundary interaction. The single enforcement point for all data/capability boundary decisions. | +| **Lifecycle Policy** | Typed Policy. Output: action on related entity (cascade, protect, detach, notify). Fires on relationship events. | ### Data Model Terms -**Compliance Domain Group** -A Policy Group governing which regulatory frameworks apply to DCM-managed resources. One of the two dimensions of a Profile. 16 built-in groups: FSI, PCI-DSS, HIPAA, FedRAMP Moderate, FedRAMP High, DoD IL2-IL6, Government, GDPR, ISO 27001, NIST 800-53, SOC2, NERC-CIP, Sovereign. - -**Confidence Descriptor** -The primary data model for Information Provider field value confidence. Four stored fields: authority_level, corroboration, source_trust, last_updated_at. Confidence score (0-100) and band (very_high through very_low) are derived at query time โ€” never stored as primary data. - -**Core Layers** -Data layers applicable across any resource type โ€” organizational, infrastructure, and contextual data not specific to any one service. - -**Denaturalization** -Converting provider-native result data back into the DCM Unified Data Model. Inverse of Naturalization. - -**Naturalization** -Converting a DCM Unified Data Model payload into provider-native format. Inverse of Denaturalization. - -**native_passthrough** -A sanctioned field for provider-specific data genuinely untranslatable to the Unified Data Model. Always audit-logged. Opaque mode blocked in fsi/sovereign profiles. - -**Policy Group** -A cohesive collection of related policies addressing a single concern. Managed as DCMGroup with group_class: policy_collection. - -**Service Layer** -A data layer contributed by a Service Provider containing service-specific configuration defaults. Independently versioned from its Service Provider. - ---- +| Term | Definition | +|------|-----------| +| **Entity** | A Resource Entity, Process Entity, or Composite Entity โ€” the primary managed thing in DCM. Has a UUID that is stable across all four lifecycle states. | +| **Four States** | Intent (consumer declaration), Requested (assembled/policy-validated), Realized (provider-confirmed), Discovered (independently observed). Same entity at four lifecycle stages in four specialized stores. | +| **Data Layer** | A versioned data artifact that contributes fields to request payload assembly. Types: Base, Core, Intermediate, Service, Request. Each has a declared contributor type. | +| **Resource Type Specification** | The schema definition for a resource type. Declares fields, constraints, portability class, dependency graph, and field criticality. | +| **Provider Catalog Item** | A provider-specific instantiation of a Resource Type Specification. What consumers actually request from the service catalog. | +| **Artifact Metadata** | Standard metadata block on every DCM artifact: uuid, handle, version, status, owned_by, created_by, contributed_by. | +| **Provenance** | Field-level lineage metadata embedded in every payload field, recording origin and all modifications. | +| **Data Classification** | Field-level metadata: public / internal / confidential / restricted / phi / pci / sovereign / classified. Phi, sovereign, and classified are immutable once set. | +| **Sovereignty Zone** | A registered DCM artifact declaring a geopolitical/regulatory boundary. Rules reference zones by ID, not raw country codes. | +| **Accreditation** | A formal, versioned, time-bounded attestation that a DCM component satisfies a specific compliance framework. First-class Data artifact with its own lifecycle. | +| **DCMGroup** | Universal grouping artifact with typed group_class: tenant_boundary, resource_grouping, policy_collection, policy_profile, composite, federation. | +| **Drift Record** | A Data artifact produced by the Drift Reconciliation Component recording field-by-field comparison of Realized vs Discovered state with severity classification. | +| **Governance Matrix Rule** | A Data artifact (also a Policy type) โ€” a rule artifact governing cross-boundary interactions using four-axis match conditions. | ### Operational Terms -**Cost Analysis Information Provider** -The specialized Information Provider that supplies cost estimation, placement cost signals, cost actuals, and budget alerts to DCM. DCM does not perform cost calculations โ€” it provides input data and consumes signals from this provider. Integrates with external cost management platforms (e.g., Red Hat Cost Management). Falls back to static declared costs from provider registration when unavailable. - -**Orchestrator** -The DCM control plane component that sequences and executes ordered workflows. Conducts the request lifecycle pipeline and executes named workflow artifacts. Relies heavily on policies (workflow steps invoke Policy Engine) and data (workflow steps read/write entity state). Workflows are first-class DCM artifacts included in Profiles. - -**Workflow (DCM)** -A versioned, GitOps-managed DCM artifact defining a named sequence of operations. Can be triggered manually, on a schedule, by events, or by policy output. Included in Profiles to automatically activate functionality when the profile is applied. The request lifecycle pipeline is itself a system-domain built-in workflow. - -**Brownfield** -Existing infrastructure provisioned outside of DCM, brought under management via the Ingestion Model. - -**DCM Group (DCMGroup)** -The universal grouping construct. All grouping uses DCMGroup with a declared group_class. Eight classes: tenant_boundary, resource_grouping, policy_collection, policy_profile, layer_grouping, provider_grouping, composite, federation. - -**Discovered State** -Result of active infrastructure interrogation. Ephemeral operational data in the Discovered Store. Used by Drift Detection to compare against Realized State. NOT the source of truth. Never stored in the Audit Store. - -**Editable Field** -A field on a realized entity modifiable via targeted delta update without reprovisioning. Independent of override_preference (which governs assembly time). +| Term | Definition | +|------|-----------| +| **Request Orchestrator** | The event bus. Routes lifecycle events to the Policy Engine. Contains no pipeline logic โ€” Policies define all behavior. | +| **Policy Engine** | Evaluates all policy types using the same algorithm. The single policy evaluator โ€” no component bypasses it. | +| **Placement Engine** | Six-step provider selection: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ parallel reserve queries โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm. | +| **Drift Reconciliation** | Control plane component. Compares Discovered State vs Realized State. Produces drift records and events. Never writes to the Realized Store. | +| **Shadow Mode** | A Policy in `proposed` status evaluates against real traffic; output is captured but never applied. The primary mechanism for safe policy change management. | +| **Naturalization** | Service Provider converts a DCM unified payload to provider-native format before execution. | +| **Denaturalization** | Service Provider converts provider-native result back to DCM unified format after execution. | +| **Rehydration** | Replaying a resource's intent state to a new provider or context. Produces a new Requested State from the existing Intent State. | +| **Contributor** | An actor type that authored a Data artifact. Recorded in artifact_metadata.contributed_by. Types: platform_admin, consumer, service_provider, peer_dcm. Determines review requirements. | +| **Two-Level Orchestration** | Level 1: Named Workflow Artifacts (Orchestration Flow Policy, ordered: true) โ€” explicit sequence skeleton. Level 2: Dynamic Policies (GateKeeper, Transformation, Recovery) โ€” fire conditionally on same events without being declared in the workflow. | +| **Reserve Query** | A parallel capacity query sent to all eligible provider candidates. Providers confirm capacity and hold it for PT5M. The Placement Engine selects the winner and releases other holds. | -**Fulfillment** -The complete process from consumer submission through Service Provider realization. A resource is Fulfilled when its Realized State matches its Requested State. - -**Implementation Posture** -A Policy Group concern_type for implementation complexity vs capability trade-offs. Governs provenance model selection, auth simplicity, deployment complexity. - -**Ingestion Model** -The three-step process (INGEST โ†’ ENRICH โ†’ PROMOTE) for bringing brownfield resources under DCM management. - -**Rehydration Lease** -An exclusive time-bounded lock per entity during rehydration. Prevents concurrent rehydrations. Priority ordering: security/compliance emergency > manual admin > automated sovereignty migration > provider decommission > manual consumer. - -**Reserve Query** -Placement engine mechanism asking candidate Service Providers "can you fulfill this resource request right now?" Providers respond with capacity availability and sovereignty compatibility. - -**Shadow Mode** -The operational state of a proposed Policy or Policy Provider. Evaluates real requests; captures outputs in Validation Store; does not apply outputs to requests. - -**Step-Up MFA** -Additional MFA challenge at sensitive operations within an already-authenticated session. Protects against session hijacking for high-stakes operations. - -**Targeted Delta** -Update mechanism for editable fields. Applies only changed fields to Realized State. Does not re-run the layer assembly chain. +### Federation Topology -**Validation Store** -Storage Provider sub-type for shadow evaluation records. Separate from Audit Store โ€” queryable, modifiable, P90D default retention. +| Term | Definition | +|------|-----------| +| **Peer DCM** | A federated DCM instance. Treated as a typed Provider. Trust postures: verified (manually approved), vouched (Hub-introduced), provisional (crypto-verified only). | +| **Hub DCM** | A DCM instance that coordinates Regional DCMs in hub-spoke topology. Policy distribution source. Cannot force-activate policies on Regional DCMs. | +| **Regional DCM** | A DCM instance in a specific sovereignty region, managed by a Hub DCM. | +| **Federation Tunnel** | Mutually authenticated, encrypted, scoped channel between DCM instances. Establishes secure transport โ€” not implicit trust. | +| **Federated Contribution** | A Peer DCM contributing registry entries, policy templates, or service layers to a receiving DCM, scoped by the peer's federation trust posture. | --- ## Part 2 โ€” Anti-Vocabulary -Terms to avoid and what to use instead. +Terms to avoid because they introduce ambiguity. Use the precise alternatives instead. -| Avoid | Reason | Use Instead | -|-------|--------|-------------| -| **Data Center** | A building โ€” architecturally irrelevant | **Region**, **Zone**, **Availability Zone** | -| **Realize / Realization** | Means too many things | **Provision** (a VM), **Install** (software), **Fulfill** (a request) | -| **Tangible / Intangible** | Weasel words โ€” no architectural meaning | Specific terms: physical resource, virtual resource, logical construct | -| **Widgets** | Vague โ€” what things exactly? | The specific resource type: VirtualMachine, VLAN, DNSRecord | -| **Producer** | Generic term not in DCM vocabulary | **Service Provider** โ€” carries the full DCM contract model | -| **Shore / Ship / Enclave** | Defense IT terminology โ€” not universally understood | **Hub DCM**, **Regional DCM**, **Sovereign DCM** | -| **User** (generic) | Means different things at different layers | **Developer** / **Application Owner** (Application domain); **Platform Engineer** / **SRE** (platform) | -| **"catalog item"** used to mean "resource type specification" | These are distinct: resource type specification = vendor-neutral contract in registry; catalog item = provider's specific offering implementing that contract | **Provider Catalog Item** or **Resource Type Specification** โ€” be specific | -| **"resource type"** used to mean a specific provider offering | Resource type is the classification; the offering is the catalog item | **Provider Catalog Item** | -| **Service** (unqualified) | Overloaded โ€” means different things at each layer | **Catalog Item** (Application), **Resource Type** (Control Plane), **Service Provider** (provider) | -| **Config / Configuration** | Could mean Layer, Resource Type Spec, Policy, or settings | Specify: **Layer**, **Resource Type Specification**, **Policy**, **Platform configuration** | -| **Manage** | Means everything and nothing | **Provision**, **configure**, **monitor**, **decommission**, **migrate**, **govern** | -| **Enrich** (unqualified) | Could mean layer assembly, policy transformation, or information push | **Layer assembly**, **Policy transformation**, **Information Provider enrichment** | +| Avoid | Because | Use Instead | +|-------|---------|-------------| +| **Widget** | Vague โ€” what thing, exactly? | Name the specific resource type (VirtualMachine, IPAddress, VLAN, etc.) | +| **Realize** (standalone) | Ambiguous โ€” "realize" can mean understand, achieve, or provision | **Provision** a VM, **fulfill** a request, **execute** a process. "Realized State" is accepted vocabulary. | +| **Data Center** (as architectural term) | A building โ€” not architecturally meaningful | **Region** (large geographically distinct area) or **Zone** / **Availability Zone** (isolated group within a Region) | +| **Orchestrator** (as a standalone component) | Suggests a single sequencer; DCM orchestration is policy-driven, not procedural | **Request Orchestrator** (the event bus) + **Orchestration Flow Policy** (named workflow) + **Policy Engine** (evaluator) | +| **Tangible / Intangible** | Nothing in DCM architecture is intangible โ€” these words add no precision | Describe what the thing actually is | +| **Workflow** (without qualification) | Ambiguous between Level 1 (named Orchestration Flow Policy) and general process | **Named Workflow** (Orchestration Flow Policy with `ordered: true`) or **dynamic policy** (conditional policy) | --- ## Part 3 โ€” Roles and Personas -| Role | Domain | Meaning | -|------|--------|---------| -| **Developer / Application Owner** | Application | Human consumer of cloud services via DCM | -| **Platform Engineer** | Control Plane | Operates and maintains the DCM platform | -| **Infrastructure Operations** | Data Center / Resource | Manages physical infrastructure and service providers | -| **Policy Owner** | Governance | Authors, reviews, and activates DCM policies | -| **Risk and Compliance Manager** | Governance | Reviews audit records, compliance reports, and policy effectiveness | -| **Data Protection Officer** | Governance | Responsible for GDPR, HIPAA, and data protection compliance | -| **Platform Admin** | Platform | Highest-privilege DCM operator | -| **SRE** | Platform | Manages DCM operational health and incident response | -| **Tenant Admin** | Tenant | Manages resources and users within a Tenant boundary | -| **Service Provider Team** | Provider | Builds and maintains Service Provider integrations | +| Role | Scope | API surface | +|------|-------|-------------| +| **Consumer** | Requests services from the catalog; manages owned resources | Consumer API | +| **Tenant Admin** | Manages a Tenant; can author tenant-domain policies and groups | Consumer API + contribution endpoints | +| **Policy Author** | Authors policies within assigned domain scope | Consumer API contribution endpoints, Flow GUI | +| **Platform Admin** | Manages the DCM deployment; all artifact types; all domains | Admin API, Flow GUI (full) | +| **Platform Observer** | Read-only view across all platform operations | Flow GUI (read-only), Admin API (read) | +| **Service Provider Operator** | Manages a registered Service Provider | Operator Interface (provider side), Admin API (registration) | +| **Policy Reviewer** | Reviews and approves/rejects contributed policies | Admin API, Flow GUI | +| **Auditor** | Read-only access to audit records and compliance reports | Consumer API (audit), Admin API (audit) | --- ## Part 4 โ€” Capability Domain Prefixes -Capability IDs used in the DCM Capabilities Matrix for Jira and implementation tracking. - | Prefix | Domain | |--------|--------| | IAM | Identity and Access Management | @@ -284,7 +143,12 @@ Capability IDs used in the DCM Capabilities Matrix for Jira and implementation t | STO | Storage and State Management | | FED | DCM Federation and Multi-Instance | | GOV | Platform Governance and Administration | +| ACC | Accreditation Management | +| ZTS | Zero Trust and Security Posture | +| GMX | Unified Governance Matrix | +| DRC | Drift Reconciliation | +| FCM | Federated Contribution Model | --- -*Document maintained by the DCM Project. Vocabulary proposals submitted via PR. For questions or contributions see [GitHub](https://github.com/dcm-project).* +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* From 0bd42f087391e9e63bdffd773501689ad02b2011 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sun, 29 Mar 2026 00:06:27 -0500 Subject: [PATCH 40/49] Moved more items to scoring system vs boolean. Codified when to score vs boolean. Signed-off-by: Chris Roadfeldt --- content/docs/README.md | 6 +- content/docs/architecture/ai-prompt.md | 77 ++- .../accreditation-and-authorization-matrix.md | 18 +- .../data-model/control-plane-components.md | 2 +- .../data-model/governance-matrix.md | 14 +- .../data-model/policy-contract.md | 2 +- .../data-model/policy-profiles.md | 4 +- .../architecture/data-model/scoring-model.md | 505 ++++++++++++++++++ content/docs/architecture/overview.md | 30 +- .../specifications/admin-api-spec.md | 121 +++++ .../specifications/consumer-api-spec.md | 29 +- .../specifications/flow-gui-spec.md | 97 ++++ .../specifications/opa-integration-spec.md | 71 +++ content/docs/capabilities-matrix.md | 19 +- content/docs/taxonomy.md | 31 +- 15 files changed, 1001 insertions(+), 25 deletions(-) create mode 100644 content/docs/architecture/data-model/scoring-model.md diff --git a/content/docs/README.md b/content/docs/README.md index 60ecd95..babf634 100644 --- a/content/docs/README.md +++ b/content/docs/README.md @@ -51,8 +51,8 @@ Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI | Policy types | 7 (unified base contract + typed output schemas) | | Control plane components | 9 | | Four lifecycle states | Intent ยท Requested ยท Realized ยท Discovered | -| Capabilities | 126 across 20 domains | -| Data model documents | 33 (28 numbered + 3 foundation + 2 examples) | +| Capabilities | 134 across 21 domains | +| Data model documents | 34 (29 numbered + 3 foundation + 2 examples) | | Specifications | 10 | | Unresolved questions | 0 | @@ -74,7 +74,7 @@ Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI ## Capabilities Matrix -[DCM-Capabilities-Matrix.md](DCM-Capabilities-Matrix.md) โ€” 126 capabilities across 20 domains including: Identity and Access, Service Catalog, Request Lifecycle, Provider Contract, Resource Lifecycle, Drift Detection, Policy Management, Data Layer, Information Integration, Ingestion, Audit, Observability, Storage, Federation, Platform Governance, Accreditation, Zero Trust, Governance Matrix, Drift Reconciliation, and Federated Contribution. +[DCM-Capabilities-Matrix.md](DCM-Capabilities-Matrix.md) โ€” 126 capabilities across 20 domains including: Identity and Access, Service Catalog, Request Lifecycle, Provider Contract, Resource Lifecycle, Drift Detection, Policy Management, Data Layer, Information Integration, Ingestion, Audit, Observability, Storage, Federation, Platform Governance, Accreditation, Zero Trust, Governance Matrix, Drift Reconciliation, Federated Contribution, and Scoring Model (enforcement_class / approval routing thresholds). **Minimum viable end-to-end set:** 21 capabilities (IAM-001 โ†’ AUD-001 critical path). diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 0289ebb..8e7006c 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4193,7 +4193,77 @@ content/ --- -## SECTION 61 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 61 โ€” SCORING MODEL (doc 29) + +### Governing Principle +Questions of fact use boolean gates. Questions of degree use scoring. Secondary test: "Can a regulator accept 'the score was below threshold' as a complete explanation?" If not โ€” boolean. + +### GateKeeper enforcement_class (required field) +- `compliance` โ€” boolean deny gate. Default and fail-safe if omitted. Used for: regulatory requirements (PHIโ†’BAA, sovereign data), security hard requirements, anything where score-around creates legal liability. +- `operational` โ€” contributes `risk_score_contribution` (weight 1โ€“100) to request risk score. Used for: cost ceilings, size limits, quota pressure, off-hours context, business rule preferences. + +### Validation output_class (required field) +- `structural` โ€” boolean pass/fail. Default and fail-safe. Missing required fields, type errors, broken references. +- `advisory` โ€” completeness score contribution + warning list. Never blocks. Recommended fields absent, unusual values, low confidence. + +### The Five Scoring Signals (aggregate โ†’ request_risk_score 0โ€“100) +1. **Operational GateKeeper score** (weight: 0.45 standard) โ€” sum of risk_score_contribution from all fired operational GateKeepers, capped at 100 +2. **Completeness score** (weight: 0.15) โ€” sum of advisory Validation contributions +3. **Actor risk history score** (weight: 0.20) โ€” decay-weighted (ฮป=0.1, half-life 7 days) history of actor's previous request outcomes; events: validation_failure(5), gatekeeper_deny(10), compliance_deny(20), policy_override(8), drift_caused(15), forced_decommission(12) +4. **Quota pressure score** (weight: 0.10) โ€” zero below 75% utilization; max(0, (util - 0.75) / 0.25) ร— 100 above +5. **Provider accreditation richness** (weight: 0.10, inverse) โ€” weighted portfolio sum; higher richness = lower provider risk contribution + +### Profile-Governed Thresholds โ†’ Approval Routing +auto_approve (threshold) +Default per profile: minimal(<60), dev(<50), standard(<25), prod(<15), fsi(<10), sovereign(<5) +**SMX-008: auto_approve_below may never exceed 50 in any profile.** +Signal weights must sum to 1.00 (validated at profile activation). + +### Profile Enforcement Class Overrides +Profiles can promote operationalโ†’compliance or demote complianceโ†’operational (non-regulatory only). +SMX-003: policies with `regulatory_mandate: true` cannot be demoted. Set by platform admins. Audited. +Threshold and override changes take effect immediately. Score Records are immutable โ€” no retroactive changes. + +### Pipeline Sequence (doc 29, Section 8) +1. Evaluate all policies +2. Compliance GateKeeper fires โ†’ HALT (boolean deny, no score) +3. Structural Validation fails โ†’ HALT (boolean fail, no score) +4. Governance Matrix DENY โ†’ HALT (always boolean, never scored โ€” SMX-004) +5. Collect operational GateKeeper contributions โ†’ Signal 1 +6. Collect advisory Validation contributions โ†’ Signal 2 +7. Fetch actor risk history โ†’ Signal 3 +8. Calculate quota pressure โ†’ Signal 4 +9. Calculate provider accreditation richness โ†’ Signal 5 +10. Aggregate with profile weights โ†’ request_risk_score +11. Apply profile thresholds โ†’ routing_decision +12. Write Score Record to Audit Store (SMX-010: required for every scored request) +13. Route: auto_approve | queue_for_review | queue_dual | queue_committee + +### What Is NEVER Scored +Governance Matrix (SMX-004) ยท authentication ยท authorization ยท five-check boundary enforcement ยท lifecycle state transitions ยท unsanctioned change flag ยท TTL expiry ยท request status states + +### Score Exposure +Consumer: risk_score, routing_decision, score_drivers (top 3, human-readable), advisory_warnings +Platform admin: full Score Record โ€” all signal breakdowns, weights, actor risk history detail +Actor risk history never exposed to other consumers (privacy โ€” SMX-007) + +### Score Record (immutable, Audit Store) +score_record_uuid, request_uuid, entity_uuid, request_risk_score, routing_decision, routing_threshold_applied, profile_uuid, signal_breakdown (per signal: score, weight, weighted_contribution, fired_policies) + +### New API Endpoints +Consumer: risk_score + advisory_warnings on POST /api/v1/requests response and GET status +Admin: GET/PATCH /admin/api/v1/profiles/{name}/scoring ยท POST overrides ยท GET/POST /actors/{uuid}/risk-history ยท GET /scoring/audit +Flow GUI: GET /flow/api/v1/graph/scoring-overlay ยท POST /flow/api/v1/simulate/score ยท Threshold slider in Profile Management view ยท Score breakdown panel in Simulation + +### SMX-001โ€“010 System Policies +SMX-001: GateKeeper must declare enforcement_class (compliance default). SMX-002: Validation must declare output_class (structural default). SMX-003: regulatory_mandate:true = no profile demotion. SMX-004: Governance Matrix always boolean. SMX-005: signal weights must sum to 1.00. SMX-006: Score Records immutable. SMX-007: actor risk history not exposed to other consumers. SMX-008: auto_approve_below โ‰ค 50. SMX-009: scoring_weight 1โ€“100; aggregate capped at 100 before weighting. SMX-010: Score Record required for every scored request. + +### Capabilities +SMX-001 through SMX-008 in Capabilities Matrix Domain 21. Total: 134 capabilities, 21 domains. + +--- + +## SECTION 62 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, follow these instructions: @@ -4267,6 +4337,11 @@ When working on this project, follow these instructions: 172. **DCM defaults to federated data creation** โ€” platform admins are not the only contributors; consumers author tenant-domain policies; providers publish resource type specs and service layers; peer DCMs contribute registry entries; all via GitOps PR with profile-governed review 173. **Contributor domain scope is hard DENY at submission** โ€” consumers cannot contribute system/platform policies regardless of declared domain; providers cannot contribute specs for types they don't offer; enforced by Governance Matrix at contribution time (FCM-002) 174. **All contributed policies enter shadow mode by default** โ€” proposed status with shadow evaluation before activation; shadow_review_period is profile-governed (P7D standard โ†’ P30D fsi/sovereign); platform admin reviews divergence cases before promoting +176. **GateKeeper enforcement_class is required and fail-safe** โ€” if omitted, treated as compliance (boolean deny). Operational-class GateKeepers never halt the request; they contribute a weighted risk_score_contribution to the aggregate. The aggregate risk score determines approval routing, not individual policy outcomes. +177. **Validation output_class is required and fail-safe** โ€” if omitted, treated as structural (boolean halt). Advisory-class Validations never halt requests; they accumulate completeness score and warning list surfaced to the consumer. +178. **Governance Matrix is always boolean โ€” never scored** โ€” SMX-004 is absolute. Scoring cannot be used to route around data sovereignty or regulatory boundaries. The Governance Matrix evaluates before the scoring pipeline runs. +179. **Profile thresholds determine routing, not individual policies** โ€” the approval routing decision (auto/review/dual/committee) emerges from the aggregate risk score crossing profile-configured thresholds, not from individual policy flags. Changing governance sensitivity = adjusting thresholds in the profile. +180. **SMX-008 is a hard system constraint** โ€” auto_approve_below may never exceed 50 in any profile. Platform admins cannot override this. Profiles submitted with auto_approve_below > 50 fail validation. 175. **Orphaned artifacts do not auto-deactivate** โ€” when contributor's access is revoked, their active artifacts remain active until platform admin assigns new owner or explicitly retires; exception: sovereign profile auto-retires orphaned artifacts (FCM-006) 93. **Process Resource max_execution_time is mandatory** โ€” it is not optional metadata; enforced by the Lifecycle Constraint Enforcer; profile governs the default on_max_exceeded action (notify/escalate/terminate) 94. **Dependency graphs are embedded, not separate entities** โ€” declared graph in Resource Type Specification; resolved graph in placement.yaml (Requested State); realized graph in Realized State events; no separate dependency graph artifact needed diff --git a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md index 9f0ed2f..b3c1a7f 100644 --- a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md +++ b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Accreditation, Data Authorization Matrix, and Zero Trust" +title: "Accreditation, Data Authorization Matrix, and Zero Trust" type: docs weight: 26 --- @@ -42,6 +42,22 @@ These three models compose: Zero Trust verifies identity and authorization on ev --- + +## 1b. Accreditation and the Scoring Model + +DCM distinguishes two distinct accreditation functions: + +**Required Accreditation (boolean gate):** Whether a provider holds a specific accreditation required for a particular request. PHI data requires an active BAA. This is a Governance Matrix enforcement โ€” always boolean, never scored. A provider without the required accreditation is ineligible for that request regardless of any other score. + +**Accreditation Richness (placement score):** The breadth and depth of a provider's accreditation portfolio. A provider with ISO 27001 + SOC2 Type II + FedRAMP Moderate + HIPAA BAA is preferable for placement over one with only self-declaration, all else equal. This is a continuous scoring signal โ€” it does not gate eligibility, it influences preference among eligible providers. + +Accreditation richness score contributes to: +1. Placement tie-breaking (a richer portfolio is preferred) +2. Request risk score Signal 5 (inversely โ€” higher richness reduces provider risk contribution) + +See [Scoring Model](29-scoring-model.md) Section 4.5 for the richness score weights and normalization. + + ## 2. Data Classification Data classification is a **first-class field-level metadata property** in the DCM data model. Every field in every payload carries a `data_classification` value. This classification is the primary axis of the authorization matrix and is the key input to sovereignty and compliance enforcement. diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index 56459f1..71597ea 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Control Plane Components" +title: "Control Plane Components" type: docs weight: 25 --- diff --git a/content/docs/architecture/data-model/governance-matrix.md b/content/docs/architecture/data-model/governance-matrix.md index 2afb204..bd88849 100644 --- a/content/docs/architecture/data-model/governance-matrix.md +++ b/content/docs/architecture/data-model/governance-matrix.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Unified Governance Matrix" +title: "Unified Governance Matrix" type: docs weight: 27 --- @@ -52,6 +52,18 @@ Previous DCM documents established several overlapping control mechanisms: the D --- + +## 1b. Governance Matrix and the Scoring Model + +The Governance Matrix is **always boolean**. This is not a design limitation โ€” it is an explicit architectural decision. + +Governance Matrix decisions (ALLOW, DENY, ALLOW_WITH_CONDITIONS, STRIP_FIELD, REDACT, AUDIT_ONLY) govern whether data may cross a boundary. These are regulatory and legal facts โ€” PHI either crosses a compliant boundary or it doesn't. "Mostly compliant" is not a legal defense. No Governance Matrix Rule may declare `scoring_weight` or `enforcement_class`. + +**The Governance Matrix fires before the Scoring Model evaluates.** If a Governance Matrix Rule produces DENY, the request is halted and no risk score is calculated. The score pipeline only runs for requests that have already passed all Governance Matrix checks. + +This ensures that scoring cannot be used to route around data sovereignty or regulatory boundaries. See [Scoring Model](29-scoring-model.md) Section 8 for the full pipeline sequence and SMX-004. + + ## 2. The Four Matrix Axes Every governance matrix rule is expressed as a match across four axes. A rule fires when all declared axis conditions are satisfied. diff --git a/content/docs/architecture/data-model/policy-contract.md b/content/docs/architecture/data-model/policy-contract.md index f88a1ed..9529d5b 100644 --- a/content/docs/architecture/data-model/policy-contract.md +++ b/content/docs/architecture/data-model/policy-contract.md @@ -1,5 +1,5 @@ --- -title: "DCM โ€” Unified Policy Contract" +title: "Unified Policy Contract" type: docs weight: -8 --- diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 76bde3f..3922ebb 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Policy Organization: Groups, Profiles, and Policy Providers" +title: "Policy Organization: Groups, Profiles, and Policy Providers" type: docs weight: 14 --- @@ -13,7 +13,7 @@ weight: 14 > **Universal Group Model:** Policy Groups (`group_class: policy_collection`) and Policy Profiles (`group_class: policy_profile`) are expressions of the [Universal Group Model](15-universal-groups.md). The structures defined in this document remain authoritative for policy-specific behavior; the universal model adds composability, cross-type membership, and the ability to include policy groups within composite groups. **Document Status:** ๐Ÿ”„ In Progress -**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Data Layers and Assembly](03-layering-and-versioning.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) +**Related Documents:** [Scoring Model](29-scoring-model.md) | [Context and Purpose](00-context-and-purpose.md) | [Data Layers and Assembly](03-layering-and-versioning.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) > **Foundation Document Reference** > diff --git a/content/docs/architecture/data-model/scoring-model.md b/content/docs/architecture/data-model/scoring-model.md new file mode 100644 index 0000000..73cad8a --- /dev/null +++ b/content/docs/architecture/data-model/scoring-model.md @@ -0,0 +1,505 @@ +--- +title: "Hybrid Scoring Model" +type: docs +weight: 29 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. This document specifies the hybrid scoring model โ€” a first-class architectural addition approved in the DCM architecture review. It extends the Policy Engine and Profile system without replacing any existing components. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” Scoring Model Specification +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Policy Profiles](14-policy-profiles.md) | [Control Plane Components](25-control-plane-components.md) | [Governance Matrix](27-governance-matrix.md) | [Federated Contribution Model](28-federated-contribution-model.md) + +> **This document maps to: DATA + POLICY** +> +> The Scoring Model is an extension of the Policy abstraction. Scored signals are Data artifacts with lifecycle and provenance. Profile thresholds are Policy-governed configuration. The Governance Matrix remains a pure boolean gate โ€” scoring never applies to cross-boundary data decisions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) + +--- + +## 1. Purpose and Governing Principle + +DCM uses a **hybrid scoring model**: some decisions are boolean gates (facts), others are scored signals (degrees). The governing principle is explicit: + +> **Questions of fact use boolean gates. Questions of degree use scoring.** + +A secondary test for any ambiguous decision: +> **Can a regulator accept "the score was below threshold" as a complete explanation? If not, the decision must be boolean.** + +This document specifies the scoring half of the hybrid. For boolean decisions, see [Governance Matrix](27-governance-matrix.md) and the compliance enforcement model in [Policy Contract](B-policy-contract.md). + +### 1.1 What This Model Does + +The scoring model adds three capabilities to the existing architecture: + +1. **Operational GateKeeper policies** contribute a weighted risk score instead of producing a binary deny. The aggregate score drives approval routing. +2. **Advisory Validation policies** produce a completeness score and warning list without blocking the request. +3. **Five scoring signals** aggregate into a request risk score that determines approval routing tier โ€” replacing the current per-policy approval flag with a continuous, profile-governed threshold system. + +### 1.2 What This Model Does Not Do + +The scoring model does **not**: +- Apply to Governance Matrix decisions โ€” these remain boolean always +- Apply to compliance-class GateKeeper policies โ€” PHIโ†’BAA, sovereign dataโ†’sovereign provider remain hard gates +- Apply to authentication, authorization, or five-check boundary enforcement +- Apply to lifecycle state transitions +- Replace the Policy Engine โ€” it is a function within it + +--- + +## 2. GateKeeper Enforcement Classes + +Every GateKeeper policy declares an `enforcement_class`. This is a required field in the Policy base contract (added in this document). + +```yaml +enforcement_class: compliance | operational +``` + +### 2.1 Compliance Class + +Behavior: **boolean gate**. A compliance-class GateKeeper that fires produces a `deny` decision. The request is halted immediately. No score is produced. + +**Use for:** +- Data classification boundary rules (PHI requires BAA accreditation) +- Sovereignty violations (classified data leaving declared zone) +- Security hard requirements (unencrypted data, expired certificates) +- Regulatory mandates with no legitimate override path +- Any rule where "score-around" creates legal or compliance liability + +```yaml +# Example compliance-class GateKeeper +policy_type: gatekeeper +enforcement_class: compliance +handle: "system/compliance/phi-baa-required" +match: + payload_type: request.layers_assembled + conditions: + - field: payload.data_classification + operator: contains + value: phi + - field: payload.provider.accreditations + operator: not_contains + value: baa_active +output: + decision: deny + reason: "PHI data requires provider with active BAA. Provider has no active BAA." + audit_required: true + notify_on: [DENY] +``` + +### 2.2 Operational Class + +Behavior: **risk score contribution**. An operational-class GateKeeper that fires contributes a weighted score to the request risk score. The request is not immediately halted. Instead the aggregate score determines routing. + +**Use for:** +- Cost ceiling policies (request cost exceeds Tenant recommendation) +- Resource sizing policies (CPU/memory above recommended maximums) +- Unusual timing or context (off-hours request, unusual field combinations) +- Quota pressure (Tenant approaching quota limit) +- Business rule preferences that should escalate review, not block + +```yaml +# Example operational-class GateKeeper +policy_type: gatekeeper +enforcement_class: operational +handle: "tenant/payments/gatekeeper/cost-ceiling" +scoring_weight: 35 # contribution to request risk score when fired +match: + payload_type: request.layers_assembled + conditions: + - field: payload.cost_estimate.per_month + operator: gt + value: 500 +output: + risk_score_contribution: 35 + reason: "Estimated monthly cost ${{payload.cost_estimate.per_month}} exceeds Tenant ceiling $500" + label: "cost_ceiling_exceeded" + audit_required: true +``` + +### 2.3 Profile-Level Enforcement Class Override + +Profiles can override the enforcement class of individual policies. This is the mechanism for making the scoring system tunable without touching individual policies. + +```yaml +# In a profile definition: +policy_enforcement_overrides: + - policy_handle: "tenant/payments/gatekeeper/cost-ceiling" + override_enforcement_class: compliance # escalate to hard gate in this profile + rationale: "FSI profile: all cost violations are hard gates" + + - policy_handle: "system/security/off-hours-request" + override_enforcement_class: operational # demote to soft score in dev profile + rationale: "Dev profile: off-hours requests are expected; score but don't block" +``` + +**Hard constraint:** A profile can **only** override `operational โ†’ compliance` or `compliance โ†’ operational` for explicitly non-regulatory policies. Policies with `regulatory_mandate: true` in their metadata cannot be demoted to operational by any profile. + +--- + +## 3. Validation Output Classes + +Every Validation policy declares an `output_class`. This is a required field. + +```yaml +output_class: structural | advisory +``` + +### 3.1 Structural Class + +Behavior: **boolean pass/fail**. A structural Validation that fails halts the request. No score is produced. + +**Use for:** +- Required field presence (missing required fields) +- Type correctness (wrong field type) +- Referential integrity (UUID references that don't resolve) +- Format validation (malformed handle, invalid semver) +- Schema conformance + +### 3.2 Advisory Class + +Behavior: **completeness score contribution + warning list**. An advisory Validation that fires contributes to the completeness score and adds a warning to the advisory_warnings list. The request is not halted. + +**Use for:** +- Recommended fields absent (cost_center not provided) +- Unusual values (memory_gb at 1 for a database VM โ€” unusual but not invalid) +- Low-confidence field values (field sourced from a provider with confidence < 0.5) +- Naming convention violations (non-compliant resource name โ€” advisory only) + +```yaml +policy_type: validation +output_class: advisory +handle: "platform/advisory/cost-center-recommended" +scoring_weight: 10 +match: + payload_type: request.layers_assembled +output: + completeness_contribution: 10 + warning_code: "recommended_field_absent" + warning_message: "cost_center not provided โ€” cost attribution will use Tenant default" + field: "fields.cost_center" +``` + +--- + +## 4. The Five Scoring Signals + +The request risk score is assembled from five independent signals. Each signal is normalized to 0โ€“100. The aggregate is a weighted sum, also normalized to 0โ€“100. + +### 4.1 Signal 1 โ€” Operational GateKeeper Score + +**Source:** All operational-class GateKeeper policies that fired during policy evaluation. +**Composition:** Sum of `risk_score_contribution` values from all fired operational GateKeepers. +**Normalization:** Capped at 100 before weighting. Multiple GateKeepers can fire; their contributions accumulate. +**Default weight in aggregate:** 0.45 + +```yaml +operational_gatekeeper_score: + fired_policies: + - handle: "tenant/payments/gatekeeper/cost-ceiling" + contribution: 35 + reason: "Cost $620/month exceeds ceiling $500" + - handle: "platform/gatekeeper/off-hours" + contribution: 15 + reason: "Request submitted outside business hours" + raw_score: 50 # sum of contributions + normalized: 50 # already within 0-100 +``` + +### 4.2 Signal 2 โ€” Policy Completeness Score + +**Source:** All advisory-class Validation policies that fired. +**Composition:** Sum of `completeness_contribution` values from all fired advisory Validations. +**Normalization:** Capped at 100. Score represents "how incomplete is this request" โ€” higher = more warnings. +**Default weight in aggregate:** 0.15 + +### 4.3 Signal 3 โ€” Actor Risk History Score + +**Source:** Decay-weighted history of the actor's previous request outcomes. +**Composition:** Each historical event has a base score contribution and a time-decay multiplier. +**Decay model:** `contribution ร— e^(-ฮปt)` where `t` is days since event, `ฮป` = 0.1 (half-life โ‰ˆ 7 days). +**Normalization:** 0โ€“100. A clean history = 0. Recent consecutive failures approach 100. +**Default weight in aggregate:** 0.20 + +```yaml +# Events that contribute to actor risk history score +actor_risk_events: + - event: validation_failure # base_contribution: 5 + - event: gatekeeper_deny # base_contribution: 10 + - event: compliance_deny # base_contribution: 20 + - event: policy_override_requested # base_contribution: 8 + - event: drift_caused # base_contribution: 15 + - event: decommission_forced # base_contribution: 12 + - event: request_abandoned # base_contribution: 3 +``` + +**Privacy constraint:** Actor risk history scores are never exposed in consumer-facing API responses beyond the actor's own history. They are available in the Admin API for platform admins and in the audit trail. + +### 4.4 Signal 4 โ€” Tenant Quota Pressure Score + +**Source:** Current quota utilization for the resource type being requested. +**Composition:** `max(0, (utilization_pct - free_threshold) / (1 - free_threshold)) ร— 100` +**Free threshold:** 0.75 (quota pressure score = 0 below 75% utilization). +**At 100% utilization:** quota pressure = 100, but the hard quota gate also fires (blocking the request regardless of score). +**Default weight in aggregate:** 0.10 + +```yaml +quota_pressure_score: + resource_type: "Compute.VirtualMachine" + current_usage: 87 + limit: 100 + utilization_pct: 0.87 + free_threshold: 0.75 + score: 48 # (0.87 - 0.75) / (1 - 0.75) ร— 100 = 48 +``` + +### 4.5 Signal 5 โ€” Provider Accreditation Richness Score + +**Source:** Accreditation portfolio of the selected/candidate provider. +**Composition:** Weighted sum of accreditation types held, normalized against the maximum possible portfolio. +**Usage:** Used in placement tie-breaking (supplements existing tie-breaking algorithm). Also contributes inversely to request risk score โ€” a richly accredited provider reduces risk. +**Default weight in aggregate:** 0.10 (inverse โ€” higher richness = lower risk contribution) + +```yaml +accreditation_weights: + self_declared: 5 + third_party_audit: 15 + iso_27001: 20 + soc2_type2: 20 + fedramp_moderate: 30 + fedramp_high: 40 + hipaa_baa: 25 + pci_dss: 25 + sovereign_authorization: 50 + +# richness_score = sum(weights for held accreditations) / max_possible ร— 100 +# risk_contribution = (1 - richness_score/100) ร— 10 [lower richness = higher risk] +``` + +### 4.6 Aggregate Request Risk Score + +``` +request_risk_score = + (operational_gatekeeper_score ร— 0.45) + + (completeness_score ร— 0.15) + + (actor_risk_history_score ร— 0.20) + + (quota_pressure_score ร— 0.10) + + (provider_risk_contribution ร— 0.10) + +# Normalized: 0โ€“100 +# 0 = clean request, no concerns +# 100 = maximum risk signal across all dimensions +``` + +Signal weights are profile-governed and can be adjusted per deployment. The weights above are the `standard` profile defaults. + +--- + +## 5. Profile-Governed Thresholds + +Every profile declares scoring thresholds that map the continuous risk score to a discrete approval routing decision. + +```yaml +scoring_thresholds: + auto_approve_below: 25 # score 0โ€“24: auto-approve + human_review_above: 25 # score 25โ€“59: one reviewer required + dual_approval_above: 60 # score 60โ€“79: two independent reviewers + committee_above: 80 # score 80โ€“100: declared DCMGroup reaches quorum + # Note: compliance-class GateKeeper deny always halts regardless of score +``` + +### 5.1 Per-Profile Threshold Defaults + +| Profile | auto_approve | human_review | dual_approval | committee | signal_weights | +|---------|-------------|-------------|--------------|-----------|----------------| +| `minimal` | < 60 | 60โ€“79 | 80โ€“100 | โ€” | default | +| `dev` | < 50 | 50โ€“79 | 80โ€“100 | โ€” | default | +| `standard` | < 25 | 25โ€“59 | 60โ€“79 | 80โ€“100 | default | +| `prod` | < 15 | 15โ€“49 | 50โ€“74 | 75โ€“100 | gatekeeper_weight: 0.50 | +| `fsi` | < 10 | 10โ€“39 | 40โ€“69 | 70โ€“100 | gatekeeper_weight: 0.55, actor_weight: 0.25 | +| `sovereign` | < 5 | 5โ€“29 | 30โ€“59 | 60โ€“100 | gatekeeper_weight: 0.60 | + +### 5.2 Resource-Type Threshold Overrides + +Profiles can declare tighter thresholds for specific resource types: + +```yaml +resource_type_threshold_overrides: + - resource_type: "Compute.VirtualMachine" + auto_approve_below: 20 # tighter than profile default + - resource_type: "Network.VLAN" + auto_approve_below: 10 # VLANs require more scrutiny + - resource_type: "Storage.Volume" + dual_approval_above: 40 # storage changes escalate earlier +``` + +### 5.3 Tenant Threshold Overrides + +Platform admins can declare Tenant-level scoring threshold adjustments: + +```yaml +tenant_scoring_config: + tenant_uuid: + threshold_overrides: + auto_approve_below: 15 # more conservative for this Tenant + signal_weight_overrides: + actor_risk_history_weight: 0.30 # higher actor scrutiny for this Tenant + trusted_actors: + - actor_uuid: + actor_risk_history_score_override: 0 # zero out risk history for trusted automation +``` + +### 5.4 Switching Between Scoring and Boolean Per Policy + +A profile can declare that a specific operational-class policy should behave as boolean (compliance-class) in that profile's context: + +```yaml +# In profile definition: +policy_enforcement_overrides: + - policy_handle: "platform/gatekeeper/cpu-size-limit" + override_enforcement_class: compliance + rationale: "Prod profile: CPU limit is a hard constraint, not a risk signal" + applies_to_resource_types: ["Compute.VirtualMachine"] +``` + +And conversely, a compliance-class policy that is **not** a regulatory mandate can be demoted to operational in lower-trust profiles: + +```yaml + - policy_handle: "platform/gatekeeper/naming-convention" + override_enforcement_class: operational + scoring_weight_override: 20 + rationale: "Dev profile: naming violations are warnings, not blocks" + requires_regulatory_mandate_false: true # safety check +``` + +--- + +## 6. Score Lifecycle and Audit Trail + +### 6.1 Score Record Structure + +Every scored evaluation produces a Score Record stored in the Audit Store alongside the standard audit record: + +```yaml +score_record: + score_record_uuid: + request_uuid: + entity_uuid: + evaluated_at: + + request_risk_score: 47 + routing_decision: human_review + routing_threshold_applied: 25 # the threshold that triggered this tier + profile_uuid: + + signal_breakdown: + operational_gatekeeper: + score: 50 + weight: 0.45 + weighted_contribution: 22.5 + fired_policies: + - handle: "tenant/payments/gatekeeper/cost-ceiling" + contribution: 35 + - handle: "platform/gatekeeper/off-hours" + contribution: 15 + completeness: + score: 20 + weight: 0.15 + weighted_contribution: 3.0 + advisory_warnings: 2 + actor_risk_history: + score: 30 + weight: 0.20 + weighted_contribution: 6.0 + recent_events: 2 + quota_pressure: + score: 48 + weight: 0.10 + weighted_contribution: 4.8 + provider_risk: + score: 15 + weight: 0.10 + weighted_contribution: 1.5 + + compliance_gates_evaluated: 3 + compliance_gates_fired: 0 # if any > 0: request halted regardless of risk score +``` + +### 6.2 Score Immutability + +Score Records are immutable once written. Threshold changes do not retroactively alter historical Score Records. If thresholds change, requests evaluated before the change retain their original routing decisions in the audit trail. + +### 6.3 Human Override of Score-Based Routing + +A platform admin or reviewer can override a score-based routing decision with a recorded justification. The override is audited, but the Score Record is never modified โ€” instead an Override Record is written referencing the original Score Record. + +--- + +## 7. Score Exposure in APIs + +### 7.1 Consumer-Facing Score Exposure + +Consumers receive a simplified score view: +- `risk_score` on request status (integer 0โ€“100) +- `routing_decision` (auto_approved | pending_review | pending_dual_approval | pending_committee) +- `advisory_warnings` list from advisory Validation +- `score_drivers` โ€” human-readable list of the top 3 contributing factors (no raw weights) + +Consumers **do not** receive: +- Actor risk history score breakdown (privacy) +- Signal weights +- Provider accreditation richness detail + +### 7.2 Platform Admin Score Exposure + +Platform admins receive full Score Record detail via the Admin API including all signal breakdowns, weights, and actor risk history detail. + +--- + +## 8. Relationship to Existing Decision Model + +The scoring model slots into the existing pipeline without replacing any component: + +``` +Policy Engine evaluation run: + 1. Evaluate all matching policies (existing behavior) + 2. Compliance-class GateKeeper fires โ†’ HALT (existing deny behavior) + 3. Structural Validation fails โ†’ HALT (existing fail behavior) + 4. Governance Matrix DENY fires โ†’ HALT (existing behavior, unchanged) + 5. NEW: Collect operational GateKeeper contributions โ†’ Signal 1 + 6. NEW: Collect advisory Validation contributions โ†’ Signal 2 + 7. NEW: Fetch actor risk history score โ†’ Signal 3 + 8. NEW: Calculate quota pressure score โ†’ Signal 4 + 9. NEW: Calculate provider accreditation richness โ†’ Signal 5 + 10. NEW: Aggregate โ†’ request_risk_score + 11. NEW: Apply profile thresholds โ†’ routing_decision + 12. NEW: Write Score Record to Audit Store + 13. Route request: auto_approve | queue_for_review | queue_dual | queue_committee +``` + +Step 2, 3, and 4 are unchanged from the existing model. Steps 5โ€“13 are additive. + +--- + +## 9. System Policies + +| Policy | Rule | +|--------|------| +| `SMX-001` | Every GateKeeper policy must declare `enforcement_class: compliance` or `enforcement_class: operational`. Policies without a declared enforcement_class are treated as compliance-class. | +| `SMX-002` | Every Validation policy must declare `output_class: structural` or `output_class: advisory`. Policies without a declared output_class are treated as structural. | +| `SMX-003` | Compliance-class GateKeeper policies with `regulatory_mandate: true` cannot be overridden to operational by any profile. This flag is set by platform admins and is audited. | +| `SMX-004` | The Governance Matrix is always boolean. No Governance Matrix Rule may declare a scoring weight or enforcement_class. | +| `SMX-005` | Signal weights in a profile must sum to 1.00. Profiles with invalid weight sums fail validation at activation time. | +| `SMX-006` | Score Records are immutable. Threshold changes do not retroactively alter historical Score Records. | +| `SMX-007` | Actor risk history scores are not exposed to consumers beyond the actor's own history. Platform admins have full access. | +| `SMX-008` | A profile's `auto_approve_below` threshold may not exceed 50. Auto-approving requests with risk scores above 50 is prohibited in all profiles. | +| `SMX-009` | Operational-class GateKeeper `scoring_weight` values must be declared between 1 and 100. Weights above 100 are validation errors. The aggregate of all fired policies is capped at 100 before weighting. | +| `SMX-010` | Score breakdown must be included in the audit trail for every request that receives a routing decision. A request with no Score Record is an audit integrity violation. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/overview.md b/content/docs/architecture/overview.md index 08cd926..58bf0e8 100644 --- a/content/docs/architecture/overview.md +++ b/content/docs/architecture/overview.md @@ -136,14 +136,34 @@ Policies are the orchestration in DCM. Seven typed output schemas, one base cont | Policy Type | Output | Fires on | |-------------|--------|---------| -| **GateKeeper** | allow / deny | Request payload | -| **Validation** | pass / fail + field detail | Request payload | +| **GateKeeper** | allow/deny (compliance) or risk score contribution (operational) | Request payload | +| **Validation** | pass/fail (structural) or completeness score + warnings (advisory) | Request payload | | **Transformation** | field mutations | Request payload | | **Recovery** | action + parameters | Failure/timeout trigger | | **Orchestration Flow** | step sequence | Pipeline events (named workflows) | -| **Governance Matrix Rule** | ALLOW / DENY / STRIP_FIELD / REDACT | Any cross-boundary interaction | +| **Governance Matrix Rule** | ALLOW / DENY / STRIP_FIELD / REDACT | Any cross-boundary interaction โ€” always boolean | | **Lifecycle Policy** | action on related entity | Relationship events | +### Hybrid Scoring Model + +DCM uses a **hybrid model**: questions of fact use boolean gates; questions of degree use scoring. + +**GateKeeper policies declare `enforcement_class`:** +- `compliance` โ€” boolean deny gate. Used for regulatory requirements (PHIโ†’BAA, sovereign data boundaries). Cannot be scored around. +- `operational` โ€” contributes a weighted `risk_score_contribution` to the aggregate request risk score. Used for operational policies (cost ceilings, size limits, quota pressure). + +**Validation policies declare `output_class`:** +- `structural` โ€” boolean pass/fail. Missing required fields, type errors. +- `advisory` โ€” completeness score contribution + warning list. Recommended fields absent, unusual values. + +**Five scoring signals** aggregate into a request risk score (0โ€“100): operational GateKeeper contributions (45%), actor risk history (20%), completeness warnings (15%), quota pressure (10%), provider accreditation richness (10%). + +**Profile-governed thresholds** map the score to approval routing: auto-approve / human_review / dual_approval / committee. Thresholds are tunable per profile without touching individual policies. Profiles can also override enforcement class per policy โ€” escalating operational policies to compliance-class, or demoting non-regulatory compliance policies to operational. + +The Governance Matrix is **always boolean** โ€” scoring never applies to cross-boundary data decisions. + +See [Scoring Model](data-model/scoring-model/) for the complete specification. + **Two-level orchestration:** - **Level 1 โ€” Named Workflow Artifacts:** Orchestration Flow Policy with `ordered: true` โ€” explicit, visible, auditable step sequence. This is the named pipeline skeleton. - **Level 2 โ€” Dynamic Policies:** GateKeeper, Transformation, Recovery policies fire when conditions match, alongside workflow steps, without being declared in the workflow. @@ -257,7 +277,7 @@ Consumer submits request (API, Web UI, or Git PR) ## Capabilities Summary -126 capabilities across 20 domains. Full detail in the [Capabilities Matrix](../capabilities-matrix/). +134 capabilities across 21 domains. Full detail in the [Capabilities Matrix](../capabilities-matrix/). **Minimum viable end-to-end set (21 capabilities):** IAM-001 โ†’ IAM-002 โ†’ IAM-003 โ†’ IAM-007 โ†’ CAT-001 โ†’ REQ-001 โ†’ REQ-002 โ†’ REQ-003 โ†’ REQ-004 โ†’ REQ-005 โ†’ REQ-006 โ†’ REQ-007 โ†’ PRV-001 โ†’ PRV-002 โ†’ PRV-003 โ†’ PRV-004 โ†’ PRV-005 โ†’ LCM-001 โ†’ DRF-001 โ†’ DRF-002 โ†’ AUD-001 @@ -284,4 +304,4 @@ IAM-001 โ†’ IAM-002 โ†’ IAM-003 โ†’ IAM-007 โ†’ CAT-001 โ†’ REQ-001 โ†’ REQ-002 - **[Federated Contribution Model](data-model/federated-contribution-model/)** โ€” who contributes what and how - **[Data Model](data-model/)** โ€” complete 28-document data model reference - **[Specifications](specifications/)** โ€” Consumer API, Admin API, Operator Interface, OPA Integration, Flow GUI, Registration, Examples, Kubernetes compatibility, SDK, CNCF strategy -- **[Capabilities Matrix](../capabilities-matrix/)** โ€” 126 capabilities across 20 domains +- **[Capabilities Matrix](../capabilities-matrix/)** โ€” 134 capabilities across 21 domains diff --git a/content/docs/architecture/specifications/admin-api-spec.md b/content/docs/architecture/specifications/admin-api-spec.md index 3124363..894e02f 100644 --- a/content/docs/architecture/specifications/admin-api-spec.md +++ b/content/docs/architecture/specifications/admin-api-spec.md @@ -483,3 +483,124 @@ Same as Consumer API. Additional admin-specific codes: --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + + +--- + +## Scoring Model Administration + +### Get Scoring Thresholds for Profile + +``` +GET /admin/api/v1/profiles/{profile_name}/scoring + +Response 200: +{ + "profile": "standard", + "scoring_thresholds": { + "auto_approve_below": 25, + "human_review_above": 25, + "dual_approval_above": 60, + "committee_above": 80 + }, + "signal_weights": { + "operational_gatekeeper": 0.45, + "completeness": 0.15, + "actor_risk_history": 0.20, + "quota_pressure": 0.10, + "provider_risk": 0.10 + }, + "policy_enforcement_overrides": [] +} +``` + +### Update Scoring Thresholds + +``` +PATCH /admin/api/v1/profiles/{profile_name}/scoring +{ + "scoring_thresholds": { + "auto_approve_below": 20, + "human_review_above": 20, + "dual_approval_above": 55, + "committee_above": 75 + } +} + +Response 200: { "profile": "standard", "updated_at": "", "effective_immediately": true } +Response 422: { "error": "threshold_invalid", "reason": "auto_approve_below exceeds maximum of 50 (SMX-008)" } +``` + +### Add Policy Enforcement Override + +``` +POST /admin/api/v1/profiles/{profile_name}/scoring/overrides +{ + "policy_handle": "platform/gatekeeper/cpu-size-limit", + "override_enforcement_class": "compliance", + "rationale": "Prod profile: CPU limit is a hard constraint", + "applies_to_resource_types": ["Compute.VirtualMachine"] +} + +Response 201 Created: +{ "override_uuid": "", "policy_handle": "...", "effective_immediately": true } +``` + +### Actor Risk History + +``` +GET /admin/api/v1/actors/{actor_uuid}/risk-history + +Response 200: +{ + "actor_uuid": "", + "current_score": 30, + "events": [ + { + "event_type": "validation_failure", + "occurred_at": "", + "request_uuid": "", + "base_contribution": 5, + "decayed_contribution": 3.2, + "days_ago": 4 + } + ], + "decay_lambda": 0.1, + "score_half_life_days": 7 +} + +POST /admin/api/v1/actors/{actor_uuid}/risk-history/reset +{ + "reason": "Actor confirmed as trusted automation account", + "audit_note": "Reviewed and approved by platform admin" +} +``` + +### Score Audit Trail + +``` +GET /admin/api/v1/scoring/audit + +Query parameters: + from= + to= + routing_decision= + risk_score_above= + actor_uuid= + resource_type= + +Response 200: +{ + "score_records": [ + { + "score_record_uuid": "", + "request_uuid": "", + "risk_score": 47, + "routing_decision": "human_review", + "signal_breakdown": { ... }, + "evaluated_at": "" + } + ] +} +``` + diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index e240271..bb1de6e 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -317,7 +317,21 @@ Response 202 Accepted: "intent_state_ref": "", "estimated_completion": "", "status_url": "/api/v1/requests/{request_uuid}/status", - "dry_run_result": null # null if auto-approve; populated if review required + "dry_run_result": null, # null if auto-approve; populated if review required + "risk_score": 47, # aggregate request risk score (0โ€“100) + "routing_decision": "human_review", # auto_approved | pending_review | pending_dual_approval | pending_committee + "score_drivers": [ # top 3 contributing factors (human-readable) + "Estimated monthly cost exceeds Tenant ceiling", + "Request submitted outside business hours", + "Actor has 2 recent validation failures" + ], + "advisory_warnings": [ # from advisory-class Validation policies + { + "warning_code": "recommended_field_absent", + "warning_message": "cost_center not provided โ€” cost attribution will use Tenant default", + "field": "fields.cost_center" + } + ] } Response 200 OK (if policy requires pre-validation report before submission): @@ -1190,7 +1204,18 @@ Response 200: "estimated_cost_per_month": 230.40, "submitted_at": "", "deadline": "", - "policy_name": "prod-vm-approval-gate" + "risk_score": 47, + "risk_score_explanation": { + "score_drivers": [ + "Estimated monthly cost exceeds Tenant ceiling (+35)", + "Request submitted outside business hours (+15)", + "Actor has 2 recent validation failures (+15)" + ], + "routing_threshold": 25, + "profile": "standard" + }, + "advisory_warnings": 1, + "policy_name": "scoring-threshold: standard/human_review" } ], "total": 2 diff --git a/content/docs/architecture/specifications/flow-gui-spec.md b/content/docs/architecture/specifications/flow-gui-spec.md index e7b508e..c1a462a 100644 --- a/content/docs/architecture/specifications/flow-gui-spec.md +++ b/content/docs/architecture/specifications/flow-gui-spec.md @@ -809,3 +809,100 @@ All Flow GUI API errors follow the standard DCM error format: --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + + +--- + +## 11. Scoring Model Views + +### 11.1 Risk Score Overlay on Execution Graph + +The Execution Graph View has a **Score Mode** toggle that overlays risk scoring information: + +- Each operational-class GateKeeper node displays its `scoring_weight` +- Node background color shifts from green (weight 1โ€“20) through amber (21โ€“50) to red (51โ€“100) +- A running score accumulator shows the current aggregate as the user traces a path through the graph +- Compliance-class GateKeeper nodes display a lock icon โ€” they are always boolean + +### 11.2 API โ€” Get Score Configuration for Graph Overlay + +``` +GET /flow/api/v1/graph/scoring-overlay + +Response 200: +{ + "active_profile": "standard", + "thresholds": { + "auto_approve_below": 25, + "human_review_above": 25, + "dual_approval_above": 60, + "committee_above": 80 + }, + "nodes": [ + { + "node_id": "", + "enforcement_class": "operational", + "scoring_weight": 35, + "avg_contribution_24h": 28.5 + } + ] +} +``` + +### 11.3 Threshold Configuration UI (Profile Management) + +The Profile and Governance Management view (Section 7) is extended with a **Scoring Thresholds** panel: + +- Visual slider showing auto_approve / human_review / dual_approval / committee bands on a 0โ€“100 scale +- Signal weight configuration (pie chart showing proportional contribution of each signal) +- Policy enforcement override management (which policies are promoted/demoted in this profile) +- Live preview: "At the current thresholds, X% of last week's requests would have been auto-approved" + +### 11.4 Score Breakdown in Simulation + +The Flow Simulation output (Section 5) is extended with a score breakdown panel: + +``` +Simulation result: risk_score=47, routing=human_review + +Score breakdown: + Operational GateKeepers: 50 ร— 0.45 = 22.5 + โ”œโ”€โ”€ cost-ceiling: +35 ("Cost $620/month exceeds $500") + โ””โ”€โ”€ off-hours: +15 ("Request outside business hours") + Completeness: 20 ร— 0.15 = 3.0 + โ””โ”€โ”€ cost_center_absent: +10 + Actor risk history: 30 ร— 0.20 = 6.0 + โ””โ”€โ”€ (2 recent events) + Quota pressure: 48 ร— 0.10 = 4.8 + โ””โ”€โ”€ (87% utilized) + Provider risk: 15 ร— 0.10 = 1.5 + โ””โ”€โ”€ (richness score: 85/100 โ†’ contribution: 1.5) + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + Total: 37.8 โ†’ 47 (normalized) + Threshold (human_review): 25 + Routing decision: HUMAN_REVIEW โœ“ +``` + +### 11.5 API โ€” Get Score Simulation + +``` +POST /flow/api/v1/simulate/score + +Request body: +{ + "synthetic_fields": { ... }, + "synthetic_actor": { "roles": ["developer"], "risk_history_score_override": 30 }, + "profile_override": "prod" # optional โ€” simulate with different profile thresholds +} + +Response 200: +{ + "risk_score": 47, + "routing_decision": "human_review", + "signal_breakdown": { ... }, + "threshold_applied": 25, + "profile": "standard", + "advisory_warnings": [...] +} +``` + diff --git a/content/docs/architecture/specifications/opa-integration-spec.md b/content/docs/architecture/specifications/opa-integration-spec.md index 8d25c9d..b1b0983 100644 --- a/content/docs/architecture/specifications/opa-integration-spec.md +++ b/content/docs/architecture/specifications/opa-integration-spec.md @@ -506,3 +506,74 @@ OPA evaluates each package independently and returns results. The Policy Engine --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + + +--- + +## Scoring Model โ€” OPA/Rego Patterns + +### Operational GateKeeper Output Schema + +```rego +package dcm.gatekeeper.operational.cost_ceiling + +# Operational-class GateKeeper produces risk_score_contribution, not deny +# enforcement_class: operational is declared in policy YAML metadata + +risk_score_contribution[result] { + input.payload.cost_estimate.per_month > 500 + result := { + "contribution": 35, + "label": "cost_ceiling_exceeded", + "reason": sprintf( + "Estimated monthly cost $%v exceeds Tenant ceiling $500", + [input.payload.cost_estimate.per_month] + ) + } +} + +# Operational GateKeepers can also produce hard deny for extreme values +deny contains reason { + input.payload.cost_estimate.per_month > 10000 + reason := "Cost exceeds absolute maximum โ€” manual review required before submission" +} +``` + +### Advisory Validation Output Schema + +```rego +package dcm.validation.advisory.cost_center + +# Advisory-class Validation produces completeness_contribution + warning +# output_class: advisory is declared in policy YAML metadata + +completeness_warnings[warning] { + not input.payload.fields.cost_center + warning := { + "contribution": 10, + "warning_code": "recommended_field_absent", + "warning_message": "cost_center not provided โ€” cost attribution will use Tenant default", + "field": "fields.cost_center" + } +} +``` + +### Validation โ€” Structural vs Advisory in Same Package + +```rego +package dcm.validation.vm_fields + +# Structural validation (output_class: structural) +fail contains reason { + not input.payload.fields.cpu_count + reason := { + "field": "fields.cpu_count", + "code": "required_field_absent", + "message": "cpu_count is required" + } +} + +# Advisory validation (output_class: advisory โ€” separate policy) +# Never mix structural and advisory in the same policy artifact +``` + diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index f3ad236..971adf9 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -278,6 +278,22 @@ --- +## 21. Scoring Model + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| SMX-001 | Operational GateKeeper Scoring | Receive risk score and score_drivers with request acknowledgment; understand why score is at its level | Declare `enforcement_class: operational` and `scoring_weight` on contributed GateKeeper policies | Configure operational GateKeeper policies with appropriate weights; manage per-policy enforcement class | POL-001, REQ-004 | +| SMX-002 | Advisory Validation and Completeness Score | Receive advisory_warnings list with request acknowledgment; understand what optional improvements exist | Declare `output_class: advisory` on advisory Validation policies | Configure advisory Validation policies; manage completeness score thresholds | POL-001, REQ-004 | +| SMX-003 | Actor Risk History Tracking | View own risk history score and contributing events via Consumer API | โ€” | Monitor actor risk history; reset scores for trusted automation accounts; configure decay parameters | AUD-001, IAM-001 | +| SMX-004 | Quota Pressure Scoring | Receive quota_pressure as a score driver when approaching Tenant quota limits | โ€” | Configure per-resource-type quota limits; manage free_threshold parameter | IAM-007, REQ-004 | +| SMX-005 | Provider Accreditation Richness Scoring | โ€” | Benefit from lower risk contribution by maintaining rich accreditation portfolio | Configure accreditation richness weights; manage portfolio scoring | ACC-001, PRV-001 | +| SMX-006 | Profile Scoring Threshold Management | โ€” | โ€” | Configure auto_approve/human_review/dual_approval/committee thresholds per profile; manage signal weights; enforce SMX-008 (max auto_approve_below: 50) | POL-005, REQ-004 | +| SMX-007 | Policy Enforcement Class Override | โ€” | Contribute policies with declared enforcement_class; receive notification when profile overrides enforcement class | Declare per-profile enforcement class overrides; manage regulatory_mandate flag to protect compliance-class policies from demotion | POL-004, POL-005 | +| SMX-008 | Score Audit Trail | Query risk score and routing decision for own requests; view score_drivers and advisory_warnings | โ€” | Query full Score Record detail including signal breakdown and actor risk history; manage score audit retention | AUD-001, REQ-004 | + +--- + + ## Capability Count Summary | Domain | Capabilities | @@ -302,7 +318,8 @@ | Unified Governance Matrix | 7 | | Drift Reconciliation | 5 | | Federated Contribution Model | 7 | -| **Total** | **126** | +| Scoring Model | 8 | +| **Total** | **134** | --- diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index e742bff..e40f45b 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -1,8 +1,4 @@ ---- -title: "DCM Taxonomy" -type: docs -weight: 2 ---- +# DCM Taxonomy The DCM taxonomy defines the precise vocabulary used throughout the architecture. Every term used in the data model, specifications, and implementation should conform to these definitions. @@ -40,8 +36,8 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture | Term | Definition | |------|-----------| -| **GateKeeper Policy** | Typed Policy. Output: allow/deny + optional field locks. Fires on request payload. Any active GateKeeper producing deny blocks the request. | -| **Validation Policy** | Typed Policy. Output: pass/fail + field-level detail. Fires on request payload. Checks correctness of field values. | +| **GateKeeper Policy** | Typed Policy. Declares `enforcement_class: compliance` (boolean deny โ€” halts request) or `operational` (contributes `risk_score_contribution` to request risk score). Compliance-class is the default and fail-safe. See [Scoring Model](data-model/scoring-model/). | +| **Validation Policy** | Typed Policy. Declares `output_class: structural` (boolean pass/fail โ€” halts on fail) or `advisory` (contributes completeness score + warning list without blocking). Structural is the default and fail-safe. See [Scoring Model](data-model/scoring-model/). | | **Transformation Policy** | Typed Policy. Output: mutations[] โ€” field additions, changes, locks. Fires on request payload. All mutations collected and applied with provenance. | | **Recovery Policy** | Typed Policy. Output: action + parameters. Fires on failure/timeout trigger conditions. Governs what DCM does when things go wrong. | | **Orchestration Flow Policy** | Typed Policy. Output: ordered step sequence. Fires on pipeline payload type events. Named workflow artifacts โ€” the explicit, visible pipeline skeleton. | @@ -82,6 +78,26 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture | **Two-Level Orchestration** | Level 1: Named Workflow Artifacts (Orchestration Flow Policy, ordered: true) โ€” explicit sequence skeleton. Level 2: Dynamic Policies (GateKeeper, Transformation, Recovery) โ€” fire conditionally on same events without being declared in the workflow. | | **Reserve Query** | A parallel capacity query sent to all eligible provider candidates. Providers confirm capacity and hold it for PT5M. The Placement Engine selects the winner and releases other holds. | + +### Scoring Model Terms + +| Term | Definition | +|------|-----------| +| **enforcement_class** | Required property of GateKeeper policies. `compliance`: boolean deny gate โ€” always halts on fire. `operational`: contributes `risk_score_contribution` to the request risk score. | +| **output_class** | Required property of Validation policies. `structural`: boolean pass/fail. `advisory`: contributes completeness score and warnings without blocking. | +| **request_risk_score** | Aggregate score (0โ€“100) assembled from five weighted signals: operational GateKeeper contributions, completeness, actor risk history, quota pressure, provider accreditation richness. Drives approval routing. | +| **risk_score_contribution** | The weighted score a fired operational-class GateKeeper contributes to the request risk score. Declared as `scoring_weight` (1โ€“100) in the policy. | +| **completeness_score** | Aggregate of advisory Validation contributions. Represents how incomplete or unusual the request is โ€” higher = more warnings. Does not block requests. | +| **actor_risk_history_score** | Decay-weighted (ฮป=0.1, half-life โ‰ˆ7 days) history of an actor's previous request outcomes. Contributes to request risk score. Not exposed to other consumers. | +| **quota_pressure_score** | Continuous score representing how close a Tenant is to quota limits for the requested resource type. Zero below 75% utilization; 100 at full quota. | +| **accreditation_richness_score** | Weighted sum of a provider's accreditation portfolio normalized to 0โ€“100. Influences placement preference and inversely contributes to provider risk signal. | +| **scoring_threshold** | Profile-governed boundary on the request risk score that maps to an approval routing tier. Four tiers: auto_approve, human_review, dual_approval, committee. `auto_approve_below` may not exceed 50 (SMX-008). | +| **Risk Score Aggregator** | Sub-function of the Policy Engine. Assembles five scoring signals into the request risk score after all compliance-class and Governance Matrix evaluations complete. | +| **regulatory_mandate** | Policy metadata flag. When `true`, the policy's `enforcement_class: compliance` cannot be demoted to operational by any profile (SMX-003). Set by platform admins, audited. | +| **score_drivers** | Human-readable list of the top contributing factors to a request risk score. Exposed to consumers (top 3 only). Full breakdown in Score Record for platform admins. | +| **Score Record** | Immutable audit artifact recording the full signal breakdown, weights, routing decision, and threshold applied for a scored request evaluation. Written to Audit Store for every scored request. | + + ### Federation Topology | Term | Definition | @@ -148,6 +164,7 @@ Terms to avoid because they introduce ambiguity. Use the precise alternatives in | GMX | Unified Governance Matrix | | DRC | Drift Reconciliation | | FCM | Federated Contribution Model | +| SMX | Scoring Model | --- From 2460a435061bbfa2b0d1ee2bda2bb88517974e5b Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sun, 29 Mar 2026 09:15:56 -0500 Subject: [PATCH 41/49] Added details on meta providers. Clarrified them as compound providers vs unique processing responsibilities. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 195 +++---- .../data-model/meta-provider-model.md | 506 ++++++++++++++++++ .../data-model/provider-contract.md | 41 +- content/docs/capabilities-matrix.md | 18 +- content/docs/taxonomy.md | 16 + 5 files changed, 634 insertions(+), 142 deletions(-) create mode 100644 content/docs/architecture/data-model/meta-provider-model.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 8e7006c..73f732f 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4263,143 +4263,76 @@ SMX-001 through SMX-008 in Capabilities Matrix Domain 21. Total: 134 capabilitie --- -## SECTION 62 โ€” WORKING INSTRUCTIONS FOR AI MODELS - -When working on this project, follow these instructions: - -1. **Data model is foundational** โ€” all design decisions must be evaluated against their impact on the data model first -2. **Four states are always relevant** โ€” when designing any component or flow, identify which states it reads from and writes to -3. **Provenance is non-negotiable** โ€” any component that modifies data must record provenance; never design around this requirement -4. **UUIDs everywhere** โ€” every entity, definition, and data object must have a UUID; never reference by name alone -5. **Portability first** โ€” when designing resource types or catalog items, start with universal fields; justify any deviation toward conditional or provider-specific -6. **Policy over hardcoding** โ€” business logic belongs in the Policy Engine, not in component code -7. **Declarative over procedural** โ€” data describes state, not steps; procedures belong in providers -8. **Universal versioning** โ€” every definition is versioned using Major.Minor.Revision; never create an unversioned definition -9. **Universal artifact lifecycle** โ€” every definition must support the five-status lifecycle: `developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired`; never design an artifact with only the old three-status model -10. **Flag open questions** โ€” do not make assumptions about unresolved items; surface them and ask -11. **Documentation format** โ€” Markdown, hierarchical structure, following the established document style -12. **Provider agnosticism** โ€” DCM does not care how providers accomplish their work; only the data contract matters -13. **Layer type scoping** โ€” Core Layers are type-agnostic; Service Layers must always be type-scoped; a Service Layer without a declared type scope is always invalid -14. **GateKeeper is supreme** โ€” GateKeeper policies can override anything including consumer input; this is by design for sovereignty and security enforcement; never design around it -15. **DCM always owns the data** โ€” regardless of operational ownership model, DCM is always the authoritative system of record for all Resource/Service Entity data and lifecycle -16. **Tenant is mandatory** โ€” every Resource/Service Entity must belong to exactly one Tenant; this is a non-overridable DCM System Policy; no exceptions -17. **Ownership vs consumption** โ€” a resource belongs to one Tenant (owner) but can be consumed by multiple Tenants via the Service Catalog; never conflate ownership and consumption -18. **Dependencies declared in advance** โ€” all dependencies must be declared in the data model before execution; provider-discovered runtime dependencies are not acceptable -19. **Process Resources need provenance** โ€” if a Process Resource modifies an Infrastructure Entity, that Entity's provenance must reference the Process Resource UUID -20. **Two-tier policies** โ€” DCM System Policies are non-overridable; Organizational Policies are configurable; never design a System Policy as organizational or vice versa -21. **Check DISCUSSION-TOPICS.md first** โ€” before designing any component or capability, check the discussion topics document for active or parked topics that may affect the design; never proceed on a topic marked ๐Ÿ”ด Blocking without resolution -22. **Webhooks are unresolved** โ€” webhook integration is under active design (DISCUSSION-TOPICS.md TOPIC-001); do not make implementation assumptions about webhook mechanics, payload format, authentication, or retry behavior until design questions are resolved; webhooks ARE confirmed as an Egress capability with Policy Engine integration -23. **Override control belongs to Policy Engine** โ€” the Request Payload Processor enforces structural layer rules only; field-level override control (allow/constrained/immutable) is set exclusively by the Policy Engine; never design around this boundary -24. **Enhancement gaps are tracked** โ€” DISCUSSION-TOPICS.md TOPIC-011 documents compatibility gaps between existing enhancement documents and the data model; when working on components covered by those enhancements, check TOPIC-011 for known gaps that need resolution -25. **Single relationship model** โ€” all entity relationships use the universal bidirectional model in doc 09; never create a separate binding or dependency mechanism; the entity relationship graph supersedes the dependency graph concept -26. **Information Providers are not Service Providers** โ€” Information Providers serve data DCM references but does not own; DCM never caches external data authoritatively; only `display_name` is cached non-authoritatively for UI convenience -27. **Standard data only for operational decisions** โ€” DCM core only relies on standard information type fields for lookups, policy evaluation, and operational decisions; extended fields are carried in payloads but never used for DCM core operations -28. **DCM is a Kubernetes superset, not a replacement** โ€” Kubernetes manages the execution plane; DCM manages the management plane; operators become DCM Service Providers through the Operator Interface Specification; never design DCM as competing with Kubernetes -29. **Operator adapter pattern** โ€” when an operator cannot be modified directly, an adapter implements the DCM Service Provider API on its behalf; the adapter handles Naturalization (DCM โ†’ CR) and Denaturalization (CR status โ†’ DCM); this is the standard pattern for existing operators -30. **Conformance levels gate capabilities** โ€” Level 1 operators get catalog and basic monitoring; Level 2 adds placement and drift detection; Level 3 adds sovereignty and brownfield ingestion; always check what level an operator has declared before assuming capabilities are available -31. **Storage Providers define contracts, not implementations** โ€” DCM specifies what a store must do; implementors choose the technology; never reference a specific technology (Kafka, Git, Elasticsearch) as a DCM requirement โ€” reference the store type and contract instead -32. **Governance is never skippable in rehydration** โ€” all relevant policies always apply regardless of rehydration source, mode, or urgency; the only variable is current vs pinned policy version, and pinned requires elevated authorization -33. **Audit and Observability are separate concerns** โ€” Audit is compliance-grade, long-retention, persona-restricted; Observability is operational, short-retention, SRE-accessible; never conflate them or design them as the same component -34. **All DCM capabilities surface through the API Gateway** โ€” Audit, Observability, Catalog, Requests, Entities, Policies all live in a unified API hierarchy; there are no separate endpoints outside the Gateway -35. **The Search Index is non-authoritative** โ€” if Search Index and GitOps store disagree, Git always wins; the Search Index is a performance layer only; it can be cleared and rebuilt from Git at any time -36. **All artifacts carry artifact metadata** โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions โ€” everything. No artifact is exempt from the universal metadata block -37. **created_by โ‰  owned_by** โ€” created_by is the audit record of who submitted the artifact; owned_by is the accountability record of who is responsible and receives notifications; these may be different people/teams -38. **Conflicts are resolved at ingestion, not assembly** โ€” all active layers in DCM are pre-validated conflict-free; the assembly process never encounters an ambiguous merge; if a conflict is found at ingestion, the PR is blocked until resolved -39. **Priority schema is advisory for categories, mandatory for ordering** โ€” the reference taxonomy (900=Compliance, 800=Security, etc.) is advisory and organizations may adapt it; however, the numeric comparison rule is always enforced and always deterministic -40. **Proposed status enables shadow validation** โ€” policy artifacts in proposed status execute in shadow mode against real traffic; output is captured in proposed_evaluation_record but never applied; this is the required validation step before activation -65. **Universal Group Model supersedes separate grouping constructs** โ€” all Tenants, Resource Groups, Policy Groups, and Profiles are DCMGroup entities with group_class; use the universal model for new implementations; existing UUIDs and APIs are preserved -66. **group_class drives system behavior, group_subclass is advisory** โ€” never design system behavior around group_subclass; only the built-in group_class values produce system-enforced behavior -67. **Composite groups default to targeting all member types** โ€” always declare member_type_filter when writing policies that target a composite group unless genuinely intending to govern all member types simultaneously -68. **Nested tenant governance: most restrictive wins** โ€” a child policy that is more restrictive than a parent policy wins; parent policies cascade where the child has no policy; this is the same principle as save_overrides_destroy and field override control -69. **former_group_membership records are permanent** โ€” group destruction does not erase membership history; queries against membership history are valid at any time via provenance store; use this for compliance and audit queries about past associations -115. **Three independent field governance mechanisms cover the full lifecycle** โ€” override_preference (assembly time: which layers can set), constraint_visibility (catalog: what consumers see), editability (post-realization: what consumers can change); all three are orthogonal and compose -116. **Updates are targeted deltas โ€” layers never re-run on updates** โ€” PATCH requests validate editable fields and edit_constraints then dispatch a delta; the layer assembly chain is not re-invoked; this preserves original assembly integrity while allowing operational changes -117. **immutable override only blocks lower-authority domains** โ€” a platform domain immutable field blocks tenant/service/provider/request; it does not block system domain; higher authority always wins; GateKeeper can additionally lock allow/constrained fields for compliance mandates -111. **Confidence descriptor is primary โ€” score and band are derived** โ€” authority_level/corroboration/source_trust/last_updated_at are stored facts; freshness/score/band computed at query time from stored facts; never stored as primary (avoids staleness); all reconstructable from audit records -112. **source_trust drives the confidence trust_multiplier** โ€” verified=1.00, degraded=0.75, suspended=0.00; maintained by dual-trigger trust scoring (event-triggered + weekly scheduled); push failures and schema errors degrade trust automatically -113. **Audit and Observability answer different questions** โ€” Audit: what happened and who authorized it; Observability: is the system healthy; different consumers, opposite storage trade-offs; Observability may reference audit record UUIDs but lives in a separate store -114. **Curated Observability Event Stream is policy-filtered** โ€” not raw metrics; policy governs what is published and subscriber role requirements; raw metrics.raw requires explicit policy opt-in; published events do not replace audit records -108. **Tenant decommission is four-phase and never silent** โ€” pre-decommission validation blocks the operation until all resources, cross-tenant relationships, compliance holds, and child groups are resolved; audit records enter post-lifecycle retention and are never destroyed -109. **Group policy inheritance is class-specific** โ€” tenant_boundary uses opt_out for standard/prod (parent cascades) and opt_in for fsi/sovereign; federation always opt_in; resource_grouping and policy_collection are not applicable -110. **Relationship graph depth differs from dependency depth** โ€” depth is graph traversal distance between any two entities, not relationship count; circular detection always enforced; profile-governed max 15 (standard/prod) or 10 (fsi/sovereign) -105. **Profiles have two independent dimensions** โ€” posture (how DCM infrastructure behaves) and compliance domain (which regulatory frameworks apply); compose them freely; a hospital uses hipaa-prod = posture-prod + compliance-hipaa; a defense contractor uses dod-il4 = posture-sovereign + compliance-dod-il4 + compliance-fedramp-high + compliance-nist-800-53 -106. **Compliance domain groups apply at Tenant level** โ€” one DCM deployment can host clinical Tenants (HIPAA), billing Tenants (HIPAA + PCI-DSS), and admin Tenants (standard) simultaneously; compliance_groups in tenant_config are additive to the platform profile -107. **HIPAA group enforces PHI minimum necessary** โ€” Mode 4 data_request_spec is limited to minimum PHI fields; providers handling PHI must declare baa_in_place in sovereignty_declaration; audit retention is P6Y regardless of profile default; breach notification triggers via sovereignty_violation_record -101. **Provenance model is configurable โ€” Model B is recommended for standard+** โ€” organizations choose full_inline (simple, high storage), deduplicated/Model B (content-addressed dedup, lossless, 95-99% storage reduction), tiered (hot/warm/cold), or combined; swap the active provenance Policy Group to change -102. **Shadow evaluation records go to Validation Store โ€” not Audit Store** โ€” Validation Store is queryable and modifiable (records marked reviewed); links to Audit Store via audit_record_uuid; P90D retention after policy promotion/retirement -103. **Policy minimum review periods are DCM-enforced** โ€” not guidelines; GateKeeper=14d, Validation=7d, Transformation=3d ร— profile multiplier; emergency bypass requires dual-approval audit -104. **Artifact status extensions are not permitted** โ€” the five statuses are invariant; use status_metadata for workflow state (no system behavior); policy gates transitions based on status_metadata values -97. **Confidence scoring is computed by DCM โ€” never self-declared** โ€” the formula (base_score ร— freshness ร— corroboration ร— authority) is standardized and auditable; policies work on bands (very_high/high/medium/low/very_low) not raw values -98. **Information Provider authority is layer-defined** โ€” static organizational knowledge ("our CMDB is authoritative for business unit") belongs in a platform domain layer; conflict detection happens at ingestion time; policy governs automated resolution -99. **DCM Provider is the ninth provider type** โ€” always mTLS (non-configurable); sovereignty checks mandatory; local policies govern ALL federated resources; audit records in both DCM instances with shared correlation_id -100. **Provider federation eligibility is layer-defined with policy enforcement** โ€” platform layer sets defaults per provider type; individual registrations may be more restrictive; storage providers default to mode: none; remote DCMs cannot decommission local resources through tunnels +## SECTION 62 โ€” META PROVIDER COMPOSABILITY MODEL (doc 30) + +### What a Meta Provider Is +A **compound Service Provider** that uses other providers in the DCM catalog to fulfill a higher-order service. Its primary contribution is a **compound service definition** declaring constituent resource types, dependencies, and delivery requirements so DCM can place, sequence, and govern the constituents. The Meta Provider is NOT an orchestrator โ€” it is a compound service definition plus a standard Service Provider for its own resource types. + +### Key Principle +The Meta Provider declares the dependency graph. DCM executes it. Parallelism emerges from the graph โ€” constituents with no unresolved dependencies dispatch concurrently. The Meta Provider does not manage sequencing, external placement, failure handling, or compensation. + +### provided_by (critical field on each constituent) +- `self` โ€” Meta Provider handles this constituent via standard Services API (naturalize/execute/denaturalize โ€” same as any Service Provider) +- `external` โ€” DCM places with best available provider via Placement Engine (all sovereignty/accreditation/trust checks apply) +- `` โ€” DCM dispatches to specific named provider + +### depends_on โ†’ Execution Order +Each constituent declares `depends_on: [component_id, ...]`. DCM reads this graph and dispatches in order โ€” no dependencies first, then those whose dependencies are REALIZED. Parallelism within a round emerges from the graph. Meta Provider does NOT manage this. + +### required_for_delivery +- `required` โ€” failure triggers Recovery Policy; unrealized constituents cancelled +- `partial` โ€” failure noted; compound continues; composite status may be DEGRADED +- `optional` โ€” failure noted; execution continues unaffected + +### Division of Responsibility (authoritative โ€” see doc 30 Section 4) +**DCM:** catalog, layer assembly, policy/scoring, external placement (Placement Engine), dependency-ordered dispatch, failure handling (Recovery Policy), compensation (dependency-reverse decommission), composite Realized State assembly, drift detection, audit, lifecycle. +**Meta Provider:** declares compound service definition; executes `self` constituents as standard Service Provider; implements standard decommission for `self` constituents. + +### Composite Entity Four States +- **Intent:** Compound request stored as-is; no constituent expansion +- **Requested:** DCM expands using compound service definition; Placement Engine resolves `external` providers; constituent blocks have component_id, provided_by, depends_on, required_for_delivery +- **Realized:** DCM assembles from all constituent realized payloads; composite_status; synthesized composite_fields +- **Discovered:** Via Meta Provider endpoint (opaque/selective) or per-constituent providers (transparent) + +### Composite Status โ€” determined by DCM +- `REALIZED` โ€” all required constituents REALIZED +- `DEGRADED` โ€” required REALIZED; partial(s) failed; accepted if profile permits +- `FAILED` โ€” required constituent(s) failed โ†’ Recovery Policy โ†’ compensation in dependency-reverse order + +### Composition Visibility +- `opaque` โ€” top-level entity only; discovery via Meta Provider endpoint +- `transparent` โ€” all constituents as DCM entities; UUIDs = deterministic(parent_uuid + component_id); per-constituent drift detection +- `selective` โ€” declared sub-set as DCM entities + +### Rehydration +Primary use case for dependency graph declaration. DCM sequences rehydration from `depends_on` graph in same order as provisioning. `external` constituents re-placed by Placement Engine. `self` constituents return to same Meta Provider. + +### Scoring +Operational GateKeepers fire on compound payload (not per-constituent). Signal 5 (accreditation richness) = lowest richness score among required-constituent providers. + +### Nested Meta Providers +Max depth 3 enforced by DCM at placement. Nested Meta Provider has no awareness it is a constituent โ€” receives and responds with standard payloads. + +### MPX-001โ€“MPX-008 System Policies +MPX-001: self constituents use standard Services API. MPX-002: DCM derives ordering from depends_on. MPX-003: parallelism from graph, not Meta Provider. MPX-004: composite status determined by DCM. MPX-005: Recovery Policy governs failure/compensation. MPX-006: external placement by Placement Engine only. MPX-007: transparent UUIDs are deterministic. MPX-008: max nesting depth 3 enforced at placement. + +### Capabilities: MPX-001โ€“MPX-007 (Domain 22 โ€” 141 total across 22 domains) + +## SECTION 63 โ€” WORKING INSTRUCTIONS FOR AI MODELS + +When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure): 172. **DCM defaults to federated data creation** โ€” platform admins are not the only contributors; consumers author tenant-domain policies; providers publish resource type specs and service layers; peer DCMs contribute registry entries; all via GitOps PR with profile-governed review 173. **Contributor domain scope is hard DENY at submission** โ€” consumers cannot contribute system/platform policies regardless of declared domain; providers cannot contribute specs for types they don't offer; enforced by Governance Matrix at contribution time (FCM-002) 174. **All contributed policies enter shadow mode by default** โ€” proposed status with shadow evaluation before activation; shadow_review_period is profile-governed (P7D standard โ†’ P30D fsi/sovereign); platform admin reviews divergence cases before promoting +175. **Orphaned artifacts do not auto-deactivate** โ€” when contributor's access is revoked, their active artifacts remain active until platform admin assigns new owner or explicitly retires; exception: sovereign profile auto-retires orphaned artifacts (FCM-006) 176. **GateKeeper enforcement_class is required and fail-safe** โ€” if omitted, treated as compliance (boolean deny). Operational-class GateKeepers never halt the request; they contribute a weighted risk_score_contribution to the aggregate. The aggregate risk score determines approval routing, not individual policy outcomes. 177. **Validation output_class is required and fail-safe** โ€” if omitted, treated as structural (boolean halt). Advisory-class Validations never halt requests; they accumulate completeness score and warning list surfaced to the consumer. 178. **Governance Matrix is always boolean โ€” never scored** โ€” SMX-004 is absolute. Scoring cannot be used to route around data sovereignty or regulatory boundaries. The Governance Matrix evaluates before the scoring pipeline runs. 179. **Profile thresholds determine routing, not individual policies** โ€” the approval routing decision (auto/review/dual/committee) emerges from the aggregate risk score crossing profile-configured thresholds, not from individual policy flags. Changing governance sensitivity = adjusting thresholds in the profile. 180. **SMX-008 is a hard system constraint** โ€” auto_approve_below may never exceed 50 in any profile. Platform admins cannot override this. Profiles submitted with auto_approve_below > 50 fail validation. -175. **Orphaned artifacts do not auto-deactivate** โ€” when contributor's access is revoked, their active artifacts remain active until platform admin assigns new owner or explicitly retires; exception: sovereign profile auto-retires orphaned artifacts (FCM-006) -93. **Process Resource max_execution_time is mandatory** โ€” it is not optional metadata; enforced by the Lifecycle Constraint Enforcer; profile governs the default on_max_exceeded action (notify/escalate/terminate) -94. **Dependency graphs are embedded, not separate entities** โ€” declared graph in Resource Type Specification; resolved graph in placement.yaml (Requested State); realized graph in Realized State events; no separate dependency graph artifact needed -95. **Billing state is first-class โ€” not metadata** โ€” DCM carries the billing_state field; policy determines the billing model per resource type and state; Cost Analysis consumes it; organizations decide what is billable -96. **Meta Provider composition_visibility governs DCM's view** โ€” opaque means DCM only sees what the provider reports; transparent means all sub-resources are full DCM entities with drift detection; selective is the middle ground -89. **Provider sovereignty is a contractual obligation** โ€” every provider registration requires sovereignty_declaration; changes must be notified within declared SLA; DCM treats sovereignty changes as drift and re-evaluates placement; auto-migration available via Provider-Portable Rehydration -90. **Git PR ingress actors resolve through the same Auth Provider as all other users** โ€” DCM trusts the Git server's authentication assertion; git config user.email is ignored (spoofing vector); the resolved actor has IDENTICAL roles/groups/tenant scope to web UI login for the same user; unresolvable identities are always rejected with an actionable PR comment -91. **Storage Provider sub-types are distinct** โ€” Search Index (non-authoritative, rebuildable, consistency lag declared) and Audit Store (append-only, hash chain, reference-based retention, compliance queries) are separate sub-types; never treat them as interchangeable -92. **GitOps stores use handle-based directory structure** โ€” deterministic path from artifact identity; main is authoritative; monorepo acceptable for minimal/dev; separate repos for standard+ -85. **Layers are data, policies are logic โ€” never conflate them** โ€” if a policy repeatedly injects the same static value, that value belongs in a layer; if a layer contains conditional evaluation logic, that logic belongs in a policy; the flow is strictly unidirectional (layers Steps 1-4, policies Steps 5-9) -86. **Layer domains mirror policy domains** โ€” system > platform > tenant > service > provider > request; same authority model, same override precedence; lower cannot override higher -87. **Layer Groups use DCMGroup group_class: layer_grouping** โ€” same universal group model as policy_collection; enables discovery, composition, and governance of related layers -88. **activation_condition enables conditional layer inclusion** โ€” evaluated in Step 2; can reference request fields, tenant attributes, resource type, resolved core layer fields, and ingress fields; condition false = layer excluded = recorded in provenance -81. **Registry governance is PR-based and policy-governed** โ€” proposals are Pull Requests with automated validation gates; shadow validation in proposed status is mandatory before active; the Registry Provider is fully policy-governed with profile-appropriate policy groups -82. **Deprecation defaults are policies โ€” not hard-coded values** โ€” REG-DP-001 through REG-DP-007 are overridable via standard priority; FSI/sovereign profiles lock sunset periods as immutable; REG-DP-005 (retired rejects new requests) is structural and never overridable -83. **Version constraints are strictly enforced โ€” no silent upgrades** โ€” DCM never auto-upgrades across major versions; version_policy governs flexibility within that constraint; profile sets the default policy -84. **Cost analysis ranks above least-loaded in tie-breaking** โ€” cost is a business decision; but only if Cost Analysis has current data and cost is determinable; skip silently if not; consistent hash is always the final deterministic tiebreaker -75. **Eight provider types โ€” not five** โ€” Message Bus Provider (6), Credential Provider (7), and Auth Provider (8) complete the ecosystem; all follow the same base contract -76. **The ingress block is the policy surface for all access control** โ€” every request carries surface, protocol, authenticated_via, actor.roles, actor.auth_provider_type, mfa_verified, and external_identity claims; GateKeeper policies act on all of these -77. **No anonymous access in any profile** โ€” minimal profile uses static API key (30 seconds to set up); the authentication ladder is about setup effort, not whether auth exists; AUTH-008 is non-negotiable -78. **Credentials always via Credential Provider** โ€” no plaintext credentials anywhere in DCM: not in Git, not in audit records, not in logs; always reference a Credential Provider secret_path -79. **Auth Providers are versioned artifacts** โ€” role_mapping and tenant_mapping changes go through proposed โ†’ active validation; Auth Provider config changes are audited; multiple providers can be registered simultaneously with signal-based routing -80. **Webhook registrations are versioned artifacts** โ€” Git-managed, lifecycle-governed, schema adapters for long-lived compatibility; inbound callers must be registered as webhook actors with explicit permissions -71. **Two-stage audit: Stage 1 is the durability guarantee** โ€” the Commit Log quorum write confirms the change is audited; Stage 2 enrichment is asynchronous; Stage 1 timestamp is the authoritative audit timestamp (AUD-013) -72. **Redundancy is profile-governed โ€” not per-component** โ€” never configure replica counts individually; activate the appropriate Profile and it configures redundancy for the entire deployment -73. **DCM is self-hosting** โ€” DCM's own deployment is a DCM resource; DCM manages itself through the same four-state model, policy engine, and audit trail used for customer workloads -74. **Everything is containerized** โ€” no bare-metal DCM components; all components run as Kubernetes pods following the standard pod security model; state is always in external stores (stateless control plane) -65. **All DCM grouping uses DCMGroup with group_class** โ€” there is no separate Tenant entity, Resource Group entity, or Policy Group entity; they are all group_class values; use the class-filtered API views for backward compatibility -66. **Composite groups apply to all member types by default** โ€” always use member_type_filter when targeting a composite group with a policy that should apply only to specific member types -67. **Nested Tenants inherit governance from parent โ€” not ownership** โ€” a resource always belongs to its leaf tenant_boundary group; the parent has governance overlay and cost rollup only; GRP-INV-004 is non-overridable -68. **Every change produces an audit record โ€” no exceptions** โ€” the WAL guarantees delivery; WAL write failure aborts the change; no silent unaudited changes are possible -69. **Audit retention is reference-based โ€” not time-based** โ€” a 7-year retention policy means 7 years AFTER all referenced entities retire; a record created 20 years ago is retained unconditionally if any referenced entity is still operational -70. **The audit hash chain is tamper-evident** โ€” any insertion, modification, or deletion of a historical record breaks the chain; verification is a first-class DCM operation; chain breaks trigger security alerts -63. **Mode 4 Policy Providers are query-response interfaces** โ€” logic lives externally; DCM sends minimized data, receives decision and/or enrichment; data sovereignty check always runs before any query is dispatched; default failure behavior is gatekeep -64. **Mode 4 enrichment fields carry full provenance** โ€” source_type: black_box_provider, source_uuid, and audit_token; override control applies; a GateKeeper can refuse enrichment on sensitive fields; enrichment providers require transformation trust level minimum -57. **Policy Profiles are the primary configuration mechanism** โ€” most deployments activate a built-in profile and add organization-specific groups; do not configure individual policies from scratch when a profile covers the use case -58. **Policy Groups are the unit of reuse** โ€” when designing policies for a concern, package them as a group; groups can be shared across profiles and inherited by other groups -59. **Policy Providers are the fifth provider type** โ€” they follow the same base contract; trust level determines max policy authority; untrusted providers are advisory only; trusted requires dual approval elevation -60. **Cross-tenant default is explicit_only** โ€” informational sharing is NOT open by default; every cross-tenant relationship of any nature requires a cross_tenant_authorization record; this supersedes the earlier operational_only default -61. **Lifecycle time constraints are first-class fields** โ€” they follow standard precedence and override control; GateKeeper can lock them immutable; expiry enforcement is a DCM control plane function not a provider concern -62. **Rehydration cannot bypass tenancy or sovereignty** โ€” policy_version: pinned only governs resource configuration policies; tenancy and sovereignty always use current policies; conflicts produce PENDING_REVIEW state -53. **Shared resources use reference counting** โ€” DCM maintains active_relationship_count; destructive actions are deferred until the count reaches minimum_relationship_count per REL-015; informational relationships never count (REL-016) -54. **Save overrides destroy โ€” always** โ€” the lifecycle action hierarchy (retain > notify > suspend > detach > cascade > destroy) resolves all multi-parent lifecycle conflicts deterministically; retain always wins per REL-018; this is non-negotiable -55. **Lifecycle conflicts are recorded, not silently resolved** โ€” lifecycle_conflict_record created whenever multiple different action recommendations exist; warning/critical severity triggers notifications; info severity is logged only -56. **shareability.allowed: false blocks multiple relationships at type level** โ€” non-shareable resource types (e.g., boot disks) reject second constituent/operational relationships at request time per REL-017; check Resource Type Specification before designing multi-parent relationships -49. **Assembly is nine steps not seven** โ€” steps 1-4 (layers), step 5 (pre-placement policies), step 6 (Placement Engine loop), step 7 (post-placement policies), step 8 (Requested State storage), step 9 (dispatch); always use the correct step number when discussing assembly -50. **Reserve query is atomic** โ€” it simultaneously verifies constraints, returns metadata, and places a resource hold; it is the primary placement query inside the loop; non-hold queries (capacity, metadata, constraint_verification) are available outside the loop for informational purposes -51. **Missing metadata is a policy concern only** โ€” DCM has no built-in opinion about metadata sufficiency; if no policy declares required_context for an absent field, the result is implicit_approval; implicit approvals are recorded explicitly in policy_gap_records -52. **Placement Engine is a named component** โ€” it is a peer to the Policy Engine, not subordinate to it; it owns the placement loop, candidate scoring, reserve query dispatch, and hold management -45. **Ingestion model is the unified mechanism** โ€” V1 migration and brownfield ingestion are the same three-step pattern: ingest โ†’ enrich โ†’ promote; use the same ingestion_record structure, same __transitional__ Tenant, same governance policies regardless of source -46. **`__transitional__` Tenant is a system artifact** โ€” never design around it for normal operations; it exists only as a migration/ingestion holding area; entities there are governance liabilities to be resolved -47. **Ingested entities have capability restrictions** โ€” INGESTED and ENRICHING state entities cannot be parents for allocated resource claims or hard dependencies for new requests; always check ingestion state before designing dependencies -48. **Promotion is the lifecycle gate** โ€” an entity is not a full DCM citizen until it reaches PROMOTED state; before that it is in a holding state with restricted capabilities -41. **Lifecycle policy fields on relationships are just fields** โ€” they carry the same override metadata and resolve under the same Policy Engine authority hierarchy as any other DCM field; no special conflict resolution mechanism -42. **Relationship type ร— nature matrix is explicit and enforced** โ€” invalid combinations are rejected at request time per REL-013; the matrix is the authoritative source for valid relationship combinations -43. **Cross-tenant relationships are governed by nature** โ€” constituent never crosses tenant boundaries; operational requires dual authorization; informational is permitted unless deny_all; hard_tenancy declaration on the Tenant entity controls the boundary -44. **Allocated resources are first-class entities** โ€” a consuming Tenant gets its own UUID, lifecycle, and governance; the relationship is depends_on + operational + cross_tenant; the parent pre-defines available allocations; DCM tracks active allocations on the parent with notification endpoints โ€” they carry the same override metadata and resolve under the same Policy Engine authority hierarchy as any other DCM field; there is no special conflict resolution mechanism for lifecycle policies; REL-008 and REL-009 are the only relationship-specific system policies that add constraints beyond the standard model - ---- - -*This prompt script is a living document. Update it whenever architectural decisions are made or open questions are resolved.* +181. **Meta Provider is a compound service definition + standard Service Provider** โ€” not an orchestrator. It declares the dependency graph so DCM can place, sequence, and govern constituents. For `self` constituents it executes as any Service Provider does. DCM handles all orchestration, placement, failure, and compensation. +182. **Composite Entity has ONE entity UUID** that links Intent, Requested, Realized, and Discovered states; the UUID is assigned at Intent creation and is stable throughout the lifecycle including rehydration +183. **DEGRADED is a valid terminal state** โ€” not an error; a DEGRADED entity enters standard OPERATIONAL lifecycle; profile governs whether degraded delivery is accepted; Recovery Policy governs failure/compensation decisions +184. **Parallelism emerges from the dependency graph** โ€” constituents with no unresolved dependencies dispatch concurrently within DCM's pipeline; the Meta Provider does not manage this +185. **provided_by: external constituents are placed by DCM's Placement Engine** โ€” all governance controls (sovereignty, accreditation, trust) apply; the Meta Provider has no influence over external constituent provider selection diff --git a/content/docs/architecture/data-model/meta-provider-model.md b/content/docs/architecture/data-model/meta-provider-model.md new file mode 100644 index 0000000..75488fa --- /dev/null +++ b/content/docs/architecture/data-model/meta-provider-model.md @@ -0,0 +1,506 @@ +--- +title: "Meta Provider Composability Model" +type: docs +weight: 30 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. This document specifies the Meta Provider composability model โ€” what a Meta Provider is, what it declares to DCM, and how standard DCM machinery handles the rest. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” Meta Provider Specification +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Service Dependencies](07-service-dependencies.md) | [Four States](02-four-states.md) | [Operational Models](24-operational-models.md) | [Scoring Model](29-scoring-model.md) | [Control Plane Components](25-control-plane-components.md) + +> **This document maps to: DATA + PROVIDER** +> +> A Meta Provider is a typed Provider that declares a compound service composition to DCM. The compound service it delivers is Data โ€” a Composite Entity across all four states. DCM's standard machinery (Placement Engine, Orchestration Flow Policy, Recovery Policy) handles everything beyond registration and constituent execution. The Meta Provider is not an orchestrator โ€” it is a compound service definition plus a set of standard Service Providers that happen to serve multiple resource types within the same system. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) + +--- + +## 1. What a Meta Provider Is + +### 1.1 The Core Model + +A Meta Provider is a **compound Service Provider** that uses other providers in the DCM catalog to fulfill a higher-order service request. Its defining characteristic is that it registers a **compound service definition** โ€” a declaration of constituent resource types, their dependencies, and their delivery requirements โ€” so that DCM has enough information to: + +1. **Select appropriate constituent providers** via the standard Placement Engine +2. **Determine execution order** from the dependency graph +3. **Govern rehydration sequence** using the same dependency information + +Beyond providing that definition, a Meta Provider operates as a standard Service Provider for each constituent resource type it owns. DCM's standard machinery handles everything else: placement, sequencing, failure handling, compensation, and audit. + +**A Meta Provider is not an orchestrator.** It does not: +- Select constituent providers โ€” the Placement Engine does this +- Sequence execution rounds โ€” the dependency graph informs DCM's Orchestration Flow Policy +- Manage parallel execution โ€” parallelism is derived from the dependency graph (resources with no unresolved dependencies execute simultaneously) +- Run compensation โ€” DCM's Recovery Policy executes compensation using the dependency graph in reverse +- Make routing decisions โ€” these are DCM policy decisions + +### 1.2 Why This Model Is Correct + +Every DCM design principle is preserved: +- **Governance stays with DCM** โ€” constituent provider selection goes through the Placement Engine, including sovereignty filtering, accreditation checking, and trust scoring +- **Policy stays with DCM** โ€” GateKeeper, Validation, and Transformation policies fire on the compound payload; the same policies govern each constituent sub-request +- **Audit stays with DCM** โ€” each constituent request is a standard DCM request with its own audit trail; the compound audit is assembled from constituent audit records +- **Recovery stays with DCM** โ€” the Recovery Policy handles constituent failures using the dependency graph; the Meta Provider does not make recovery decisions + +### 1.3 The Practical Meaning + +A Meta Provider registration tells DCM: "Here is a compound service called `ApplicationStack.WebApp`. To fulfill it, you will need a `Compute.VirtualMachine`, a `Network.IPAddress`, a `DNS.Record` (which depends on both), and a `Network.LoadBalancer` (which also depends on both). I can provide the DNS and LoadBalancer; you should place the VM and IP with appropriate compute and network providers." + +DCM then: +- Creates a Composite Entity with one entity UUID +- Runs the compound layer assembly to produce the full payload +- Applies policies to the compound payload +- Dispatches constituent sub-requests to the appropriate providers (compute provider for VM, network provider for IP, Meta Provider for DNS and LoadBalancer) +- Sequences those sub-requests based on the declared dependency graph +- Handles any constituent failures using Recovery Policy +- Assembles the aggregate Realized State from all constituent realized states + +The Meta Provider's execution responsibility is limited to: naturalizing and realizing the constituent resource types it owns, then denaturalizing and returning the realized state โ€” exactly as a standard Service Provider does. + +--- + +## 2. Compound Service Definition + +The compound service definition is the Meta Provider's primary contribution to DCM. It is declared at registration and stored in the Resource Type Registry as a compound Resource Type Specification. + +### 2.1 Constituent Declaration + +```yaml +resource_types_composed: + - fqn: ApplicationStack.WebApp + version: "2.0.0" + + constituents: + - component_id: vm-primary + resource_type: Compute.VirtualMachine + provided_by: external # DCM places this with an appropriate compute provider + depends_on: [] + required_for_delivery: required + + - component_id: ip-primary + resource_type: Network.IPAddress + provided_by: external # DCM places this with an appropriate network provider + depends_on: [] + required_for_delivery: required + + - component_id: dns-primary + resource_type: DNS.Record + provided_by: self # This Meta Provider handles DNS + depends_on: [vm-primary, ip-primary] + required_for_delivery: partial + + - component_id: lb-frontend + resource_type: Network.LoadBalancer + provided_by: self # This Meta Provider handles LoadBalancer + depends_on: [vm-primary, ip-primary] + required_for_delivery: partial + + composition_visibility: selective # opaque | transparent | selective + dcm_visible_sub_resources: + - resource_type: Compute.VirtualMachine + role: application_host + - resource_type: Network.LoadBalancer + role: ingress_endpoint +``` + +### 2.2 provided_by Declaration + +`provided_by` is the key field that tells DCM who is responsible for each constituent: + +| Value | Meaning | +|-------|---------| +| `self` | This Meta Provider handles this constituent. DCM dispatches it to the Meta Provider. | +| `external` | DCM places this constituent with the best available provider via the standard Placement Engine. | +| `` | DCM dispatches this constituent to a specific named provider. | + +For `external` constituents, DCM runs a full placement cycle โ€” sovereignty filtering, accreditation checking, trust scoring, reserve query โ€” exactly as it would for any standalone request. + +For `self` constituents, DCM dispatches to the Meta Provider using the standard Services API. The Meta Provider receives a standard constituent payload and responds with a standard realized state โ€” no special handling required. + +### 2.3 Dependency Graph + +The `depends_on` list is the mechanism by which the Meta Provider informs DCM of execution ordering. DCM reads this graph and: + +- Identifies which constituents have no dependencies โ†’ dispatches these first (or simultaneously) +- Identifies which constituents have dependencies on already-realized constituents โ†’ dispatches these when their dependencies are complete +- Derives parallelism directly from the graph โ€” constituents with no shared unresolved dependencies execute concurrently within DCM's standard pipeline + +**The Meta Provider does not manage this sequencing.** It declares the graph. DCM executes it. + +``` +depends_on: [] โ†’ eligible for immediate dispatch +depends_on: [vm-primary] โ†’ dispatched after vm-primary is REALIZED +depends_on: [vm-primary, ip-primary] โ†’ dispatched after BOTH are REALIZED +``` + +### 2.4 required_for_delivery Classification + +Each constituent declares how its success or failure affects the compound service: + +| Classification | Failure effect | +|----------------|---------------| +| `required` | DCM halts the compound request; triggers Recovery Policy; unrealized constituents are not dispatched | +| `partial` | DCM notes the failure; compound service continues; final status may be `DEGRADED` | +| `optional` | DCM notes the failure; compound service continues unaffected | + +The Recovery Policy governs what happens on `required` constituent failure โ€” the same Recovery Policy that governs any request failure. No special Meta Provider recovery logic exists. + +--- + +## 3. Composite Entity โ€” Four-State Representation + +A compound service request produces a **Composite Entity** โ€” a single DCM entity that aggregates constituent sub-entities. The Composite Entity has one entity UUID that links it across all four states. + +### 3.1 Intent State + +The consumer submits one request against the compound catalog item. The intent payload contains consumer-declared fields for the compound service โ€” not individual constituent fields. + +```yaml +entity_uuid: # assigned at intent creation; stable across all states +catalog_item_uuid: # ApplicationStack.WebApp +fields: + app_name: payments-api + environment: production + region: EU-WEST + cpu_count: 4 + memory_gb: 16 + dns_hostname: payments-api.internal.corp.example +``` + +DCM does not expand this into constituent requests at Intent State. The intent is stored as-is. + +### 3.2 Requested State + +The Request Payload Processor expands the compound intent into the full constituent payload. This is where the compound service definition from the Meta Provider registration is applied. + +```yaml +entity_uuid: +composite_entity: true +meta_provider_uuid: + +top_level: + app_name: payments-api + environment: production + region: EU-WEST + +constituents: + - component_id: vm-primary + resource_type: Compute.VirtualMachine + provided_by: external + provider_uuid: null # resolved by Placement Engine + fields: + cpu_count: 4 + memory_gb: 16 + os_family: rhel + depends_on: [] + required_for_delivery: required + + - component_id: ip-primary + resource_type: Network.IPAddress + provided_by: external + provider_uuid: null # resolved by Placement Engine + fields: + ip_version: 4 + allocation_pool: prod-EU-WEST + depends_on: [] + required_for_delivery: required + + - component_id: dns-primary + resource_type: DNS.Record + provided_by: self + provider_uuid: + fields: + hostname: payments-api.internal.corp.example + record_type: A + depends_on: [vm-primary, ip-primary] + required_for_delivery: partial + + - component_id: lb-frontend + resource_type: Network.LoadBalancer + provided_by: self + provider_uuid: + fields: + backend_component: vm-primary + depends_on: [vm-primary, ip-primary] + required_for_delivery: partial +``` + +### 3.3 Realized State + +The Realized State is assembled by DCM from the constituent realized payloads returned by each dispatched provider. DCM writes it as a unified composite record. + +```yaml +entity_uuid: +composite_entity: true +composite_status: DEGRADED # REALIZED | DEGRADED | FAILED + +composite_fields: # synthesized consumer-facing view + primary_ip: 10.1.45.23 + vm_id: vm-0a1b2c3d + lb_endpoint: lb-7f8e9d.eu-west.corp + dns_name: null # absent โ€” dns constituent failed + +constituents_realized: + - component_id: vm-primary + status: REALIZED + provider_uuid: + realized_fields: + vm_id: vm-0a1b2c3d + hypervisor_host: host-eu-w-04 + required_for_delivery: required + + - component_id: ip-primary + status: REALIZED + provider_uuid: + realized_fields: + assigned_ip: 10.1.45.23 + required_for_delivery: required + + - component_id: dns-primary + status: FAILED + provider_uuid: + failure_reason: "DNS service degraded โ€” record not created" + required_for_delivery: partial + + - component_id: lb-frontend + status: REALIZED + provider_uuid: + realized_fields: + lb_id: lb-7f8e9d + endpoint: lb-7f8e9d.eu-west.corp + required_for_delivery: partial + +degradation_summary: + - component_id: dns-primary + impact: "DNS name resolution unavailable โ€” access via IP only" + recovery_eligible: true +``` + +### 3.4 Discovered State + +Discovery for composite entities follows the composition visibility mode: + +- `opaque` โ€” DCM schedules one discovery call to the Meta Provider; it returns the compound discovered state as a single payload +- `transparent` โ€” DCM schedules independent discovery calls to each constituent's provider; drift detection runs on each constituent independently +- `selective` โ€” DCM schedules discovery calls for DCM-visible constituents; opaque discovery for the rest via Meta Provider + +--- + +## 4. What DCM Does vs What the Meta Provider Does + +This table is the definitive statement of responsibility. + +| Concern | DCM | Meta Provider | +|---------|-----|---------------| +| Compound catalog item presentation | โœ… โ€” Service Catalog manages the compound item | Declares the compound service definition at registration | +| Consumer-facing API | โœ… โ€” Consumer API handles compound requests identically to simple requests | โ€” | +| Layer assembly | โœ… โ€” Request Payload Processor assembles constituent blocks from compound service definition | โ€” | +| Policy evaluation on compound payload | โœ… โ€” Same Policy Engine, same policies, same scoring model | May contribute provider-domain policies for its own constituents | +| External constituent placement | โœ… โ€” Placement Engine selects provider for each `provided_by: external` constituent | Declares what resource types each constituent needs so placement can filter appropriately | +| Self constituent dispatch | โœ… โ€” API Gateway dispatches to Meta Provider using standard Services API | Receives constituent payload; naturalizes; executes; denaturalizes; returns realized state | +| Execution ordering from dependency graph | โœ… โ€” Derived from `depends_on` declarations; DCM dispatches in order | Declares `depends_on` relationships for each constituent | +| Parallelism | โœ… โ€” Constituents with no unresolved dependencies execute concurrently within DCM's pipeline | Emerges from the dependency graph declaration; Meta Provider does not manage this | +| Constituent failure handling | โœ… โ€” Recovery Policy fires based on `required_for_delivery` classification | Declares `required_for_delivery` for each constituent | +| Compensation (teardown of realized constituents on failure) | โœ… โ€” Recovery Policy executes compensation using dependency graph in reverse | Implements standard decommission handling for `self` constituents | +| Composite status determination | โœ… โ€” Determined by DCM from constituent outcomes and `required_for_delivery` | โ€” | +| Realized State assembly | โœ… โ€” DCM assembles composite Realized State from constituent realized payloads | Returns standard realized payload for `self` constituents | +| Drift detection | โœ… โ€” Standard drift detection per composition visibility mode | Implements standard discovery endpoint for `self` constituents | +| Lifecycle management of Composite Entity | โœ… โ€” Standard DCM entity lifecycle | Handles decommission of `self` constituents when decommission payload received | +| Audit trail | โœ… โ€” Each constituent request has its own audit record; composite audit assembled by DCM | โ€” | + +### 4.1 The Meta Provider's Execution Scope Is Narrow + +For each `self` constituent, the Meta Provider: +1. Receives a standard DCM constituent payload (already fully assembled by DCM) +2. Naturalizes it to its native format +3. Executes the realization +4. Denaturalizes the result +5. Returns a standard realized payload + +This is identical to what any Service Provider does. The Meta Provider is not special during execution โ€” it is simply a Service Provider that happens to be registered for multiple resource types within the same underlying system. + +--- + +## 5. Composition Visibility + +Introduced in [07-service-dependencies.md](07-service-dependencies.md), this section specifies the operational implications. + +| Mode | Consumer sees | DCM manages independently | Drift detection | +|------|--------------|--------------------------|-----------------| +| `opaque` | Top-level entity only | Composite entity only | Via Meta Provider discovery endpoint | +| `transparent` | All constituents as DCM entities | All constituent entities individually | Per-constituent via each provider | +| `selective` | Declared visible constituents | Declared visible constituents | Per-constituent for visible; via Meta Provider for rest | + +### 5.1 Transparent Mode Entity UUIDs + +In transparent mode, constituent entities receive stable DCM entity UUIDs: +``` +constituent_entity_uuid = deterministic_uuid(parent_entity_uuid + component_id) +``` + +These UUIDs are stable across rehydration โ€” the same compound entity always produces the same constituent UUIDs. This enables consistent audit trail linkage. + +### 5.2 Decommission Cascade + +When a composite entity is decommissioned: +- DCM dispatches constituent decommission sub-requests in dependency-reverse order +- Each constituent's provider (whether `self` or `external`) receives a standard decommission payload +- The composite entity's lifecycle state transitions to DECOMMISSIONED only after all constituents confirm decommission + +--- + +## 6. Rehydration + +Rehydration of a composite entity is the primary use case for the dependency graph declaration โ€” which the Meta Provider provides precisely for this purpose. + +### 6.1 Rehydration Sequence + +DCM reads the `depends_on` graph and rehydrates constituents in dependency order: + +``` +Round 1: components with depends_on: [] โ†’ rehydrate first +Round 2: components whose depends_on are realized โ†’ rehydrate next +Round N: continue until all constituents complete +``` + +The Meta Provider's dependency declarations give DCM exactly the information it needs to sequence rehydration correctly without requiring any special Meta Provider involvement beyond standard constituent execution. + +### 6.2 Rehydration Provider Selection + +`provided_by: external` constituents are re-placed by the Placement Engine during rehydration โ€” they may end up on a different provider than the original realization if the original provider is no longer eligible. `provided_by: self` constituents always return to the same Meta Provider. + +--- + +## 7. Compound Request Pipeline + +A compound service request flows through DCM's standard pipeline with compound-specific extensions at the Request Payload Processor step. + +``` +Consumer submits compound request + โ”‚ + โ–ผ Intent State captured (compound intent, no constituent expansion) + โ”‚ + โ–ผ Request Payload Processor: + โ”‚ Layer assembly on compound payload + โ”‚ Expansion: compound service definition โ†’ constituent blocks + โ”‚ (resource_type, provided_by, fields, depends_on, required_for_delivery per constituent) + โ”‚ Policy evaluation on compound payload (GateKeeper, Validation, Transformation, scoring) + โ”‚ Placement: external constituents โ†’ Placement Engine + โ”‚ self constituents โ†’ Meta Provider + โ”‚ + โ–ผ Requested State written (full constituent specification with provider assignments) + โ”‚ + โ–ผ Constituent dispatch โ€” dependency-ordered by DCM: + โ”‚ Round 1 (no dependencies): dispatch vm-primary, ip-primary in parallel + โ”‚ vm-primary โ†’ compute_provider (standard Services API) + โ”‚ ip-primary โ†’ network_provider (standard Services API) + โ”‚ + โ”‚ Round 2 (vm+ip REALIZED): dispatch dns-primary, lb-frontend in parallel + โ”‚ dns-primary โ†’ meta_provider (standard Services API) + โ”‚ lb-frontend โ†’ meta_provider (standard Services API) + โ”‚ + โ”‚ Any constituent FAILS: + โ”‚ required โ†’ Recovery Policy fires; unstarted constituents cancelled + โ”‚ partial โ†’ noted as degraded; execution continues + โ”‚ optional โ†’ noted; execution continues + โ”‚ + โ–ผ Constituent realized payloads collected by DCM + โ”‚ + โ–ผ Composite status determined: REALIZED | DEGRADED | FAILED + โ”‚ + โ–ผ Realized State written (composite record assembled by DCM) + โ”‚ + โ–ผ Consumer notified of compound request outcome +``` + +--- + +## 8. Nested Meta Providers + +A Meta Provider may declare a constituent with `provided_by: external` where the appropriate provider is itself another Meta Provider. DCM's Placement Engine handles this transparently โ€” it places the constituent with whichever registered provider best satisfies the constraints, whether that is a simple Service Provider or another Meta Provider. + +**Maximum nesting depth: 3** โ€” enforced by DCM at placement time by checking the compound service definition chain depth. Deeper nesting creates dependency graph complexity that exceeds DCM's governance model. + +**The nested Meta Provider has no special awareness** that it is being called as a constituent of an outer compound service. It receives a standard constituent payload and responds with a standard realized state. Nesting is a DCM-level concept, not a provider-level one. + +--- + +## 9. Scoring Model Integration + +Compound service requests are scored using the standard five-signal model with two compound-specific behaviors: + +**Operational GateKeepers** fire on the compound payload assembled from the top-level fields and declared constituent types. They do not fire per-constituent (constituent-level policy evaluation happens in each constituent sub-request's own pipeline). + +**Provider accreditation richness (Signal 5)** for compound entities uses the lowest richness score among all `required_for_delivery: required` constituents across all their assigned providers. The compound service is only as well-accredited as its least-accredited required constituent. + +--- + +## 10. Meta Provider Registration Contract + +```yaml +meta_provider_capabilities: + # Resource types this Meta Provider handles as a self provider + resource_types_provided: + - DNS.Record + - Network.LoadBalancer + + # Compound service definitions this Meta Provider offers + resource_types_composed: + - fqn: ApplicationStack.WebApp + version: "2.0.0" + constituents: + - component_id: vm-primary + resource_type: Compute.VirtualMachine + provided_by: external + depends_on: [] + required_for_delivery: required + - component_id: ip-primary + resource_type: Network.IPAddress + provided_by: external + depends_on: [] + required_for_delivery: required + - component_id: dns-primary + resource_type: DNS.Record + provided_by: self + depends_on: [vm-primary, ip-primary] + required_for_delivery: partial + - component_id: lb-frontend + resource_type: Network.LoadBalancer + provided_by: self + depends_on: [vm-primary, ip-primary] + required_for_delivery: partial + composition_visibility: selective + dcm_visible_sub_resources: + - resource_type: Compute.VirtualMachine + role: application_host + - resource_type: Network.LoadBalancer + role: ingress_endpoint + + # Standard provider declarations apply as for any Service Provider + # (sovereignty, accreditations, capacity reporting, health check, etc.) +``` + +--- + +## 11. System Policies + +| Policy | Rule | +|--------|------| +| `MPX-001` | A Meta Provider's `self` constituents are dispatched using the standard Services API. The Meta Provider receives a standard constituent payload and returns a standard realized state. No special dispatch protocol exists for Meta Provider self-constituents. | +| `MPX-002` | Constituent execution ordering is derived from the `depends_on` declaration by DCM. The Meta Provider does not sequence constituent dispatch. | +| `MPX-003` | Parallelism in constituent execution is derived from the dependency graph. Constituents with no unresolved dependencies execute concurrently within DCM's standard pipeline. The Meta Provider does not manage this. | +| `MPX-004` | Composite status determination (`REALIZED` / `DEGRADED` / `FAILED`) is performed by DCM based on constituent outcomes and `required_for_delivery` classifications. | +| `MPX-005` | Recovery Policy governs all constituent failure handling and compensation. The Meta Provider does not make recovery decisions. It implements standard decommission handling for `self` constituents when a decommission payload arrives. | +| `MPX-006` | `provided_by: external` constituents are placed by the Placement Engine using standard placement rules. The Meta Provider does not influence external constituent provider selection. | +| `MPX-007` | In transparent composition visibility mode, constituent entity UUIDs are `deterministic_uuid(parent_entity_uuid + component_id)` โ€” stable across rehydration. | +| `MPX-008` | Maximum Meta Provider nesting depth is 3, enforced by DCM at placement time by checking the compound service definition chain depth. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/provider-contract.md b/content/docs/architecture/data-model/provider-contract.md index 071dee1..2f6e38e 100644 --- a/content/docs/architecture/data-model/provider-contract.md +++ b/content/docs/architecture/data-model/provider-contract.md @@ -1,5 +1,5 @@ --- -title: "DCM โ€” Unified Provider Contract" +title: "Unified Provider Contract" type: docs weight: -9 --- @@ -264,24 +264,45 @@ storage_provider_capabilities: ### 7.4 Meta Provider -**What it does:** Composes multiple child providers to deliver a compound service as a single catalog item. +**What it does:** Composes multiple child providers to deliver a compound service as a single catalog item. The Meta Provider declares a compound service definition โ€” constituent resource types, dependencies, and delivery requirements โ€” so DCM can place, sequence, and govern the constituents. For its own resource types (`provided_by: self`), the Meta Provider executes as a standard Service Provider. All orchestration, placement, sequencing, failure handling, and compensation is performed by DCM using the declared dependency graph. -**Capability declaration extension:** +> **Full specification:** See [Meta Provider Composability Model](30-meta-provider-model.md) for the complete orchestration contract, four-state model, failure propagation, compensation, and system policies (MPX-001โ€“MPX-008). + +**Capability declaration extension (summary โ€” full schema in doc 30):** ```yaml meta_provider_capabilities: - constituent_provider_types: [service_provider, information_provider] - composition_model: sequential | parallel | conditional + constituent_provider_types: [service_provider, information_provider, meta_provider] + composition_model: + execution: dependency_ordered # sequential | parallel | dependency_ordered + max_concurrent_realizations: 10 + max_constituent_count: 20 + max_nesting_depth: 3 partial_delivery_supported: true - compensation_supported: true + compensation_supported: true # required if partial_delivery_supported: true (MPX-001) + compensation_timeout: PT30M + idempotency_guaranteed: true + status_reporting: + supported: true + interval: PT30S resource_types_composed: - fqn: ApplicationStack.WebApp + version: "2.0.0" constituents: - - Compute.VirtualMachine - - Network.IPAddress - - DNS.Record + - resource_type: Compute.VirtualMachine + required_for_delivery: required + - resource_type: Network.IPAddress + required_for_delivery: required + - resource_type: DNS.Record + required_for_delivery: partial + composition_visibility: selective # opaque | transparent | selective ``` -**Data direction:** DCM sends compound service payload โ†’ Meta Provider orchestrates constituent providers โ†’ aggregates realized states โ†’ returns compound realized state. +**Composite status determination:** +- `REALIZED` โ€” all required constituents succeeded +- `DEGRADED` โ€” required constituents succeeded; one or more partial constituents failed (accepted if `partial_delivery_supported: true`) +- `FAILED` โ€” one or more required constituents failed โ†’ compensation executes + +**Data direction:** DCM sends fully assembled compound payload โ†’ Meta Provider orchestrates constituents in dependency order โ†’ aggregates realized states โ†’ returns compound realized state to DCM. --- diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index 971adf9..c0f060b 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -294,6 +294,21 @@ --- +## 22. Meta Provider Composability + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| MPX-001 | Compound Service Request | Request a compound service as a single catalog item; receive composite entity UUID; track compound execution status via standard request status endpoint | Register as Meta Provider with constituent specification; implement compound dispatch endpoint | Configure Meta Provider registration; manage composite service catalog items | CAT-001, REQ-007, PRV-001 | +| MPX-002 | Dependency-Ordered Constituent Execution | โ€” | Execute constituents in declared dependency order; manage parallel rounds; respect depends_on declarations | Configure composition model; monitor execution round progress via status events | MPX-001, PRV-003 | +| MPX-003 | Partial Delivery and DEGRADED State | Receive DEGRADED composite entity when partial delivery is accepted; choose to accept or reject degraded state | Declare partial_delivery_supported and required_for_delivery per constituent; return DEGRADED compound payload | Configure accept_degraded_delivery per profile; manage degraded notification urgency | MPX-001, PRV-005 | +| MPX-004 | Compound Compensation | Receive notification and recovery decision when compound service fails; approve or reject compensation | Implement compensation in dependency-reverse order; guarantee idempotent decommission calls | Configure compensation timeout; manage PARTIALLY_COMPENSATED orphan detection | MPX-001, LCM-007, DRC-001 | +| MPX-005 | Transparent Constituent Visibility | Query and manage DCM-visible constituent entities independently (when transparency mode); receive constituent-level drift alerts | Declare composition_visibility mode; register transparent constituents with deterministic UUIDs | Configure visibility mode per compound resource type; manage constituent entity lifecycle policies | MPX-001, DRF-001 | +| MPX-006 | Compound Execution Status Tracking | Monitor compound execution round progress via request status; see component-level status during long-running compositions | Send intermediate status events to DCM during execution; declare status_reporting.interval | Monitor compound execution health; configure execution timeout alerts | MPX-001, REQ-008 | +| MPX-007 | Nested Meta Provider Composition | Request high-order compound services composed of other compound services (max depth 3) | Implement as a Meta Provider that calls other Meta Providers as constituents; declare max_nesting_depth | Configure nesting depth limits; manage nested compensation chains | MPX-001, PRV-009 | + +--- + + ## Capability Count Summary | Domain | Capabilities | @@ -319,7 +334,8 @@ | Drift Reconciliation | 5 | | Federated Contribution Model | 7 | | Scoring Model | 8 | -| **Total** | **134** | +| Meta Provider Composability | 7 | +| **Total** | **141** | --- diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index e40f45b..e04aff1 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -79,6 +79,21 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture | **Reserve Query** | A parallel capacity query sent to all eligible provider candidates. Providers confirm capacity and hold it for PT5M. The Placement Engine selects the winner and releases other holds. | + +### Meta Provider Composability Terms + +| Term | Definition | +|------|-----------| +| **Composite Entity** | A DCM entity produced by a Meta Provider. Exists across all four states as a single entity aggregating constituent Resource Entities. Has one entity UUID that links it through all states. | +| **Constituent** | A sub-resource within a compound service that a Meta Provider provisions. Declared with a `component_id`, `resource_type`, `depends_on`, and `required_for_delivery` classification. | +| **required_for_delivery** | Constituent delivery classification: `required` (failure halts the compound service and triggers compensation), `partial` (failure produces DEGRADED but not FAILED), `optional` (failure is noted but ignored). | +| **Composite Status** | Top-level outcome of a compound service execution: `REALIZED` (all required constituents succeeded), `DEGRADED` (required succeeded; partial(s) failed; accepted if profile permits), `FAILED` (required constituent(s) failed; triggers compensation). | +| **Compensation** | Ordered teardown of successfully realized constituents when a compound service cannot be delivered. Runs in dependency-reverse order. Best-effort; failures produce `PARTIALLY_COMPENSATED` with orphan detection. | +| **Composition Visibility** | How a Meta Provider exposes its internal structure to DCM: `opaque` (top-level only), `transparent` (all constituents as DCM entities), `selective` (declared sub-set as DCM entities). | +| **Dependency Round** | A batch of constituents that can execute in parallel because all their `depends_on` constituents are complete. Multiple rounds execute sequentially; constituents within a round execute in parallel. | +| **MPX-001โ€“MPX-008** | Meta Provider system policies. Key: MPX-001 (compensation required if partial delivery supported), MPX-002 (dependency-reverse decommission), MPX-006 (DEGRADED is a valid terminal state when accepted), MPX-008 (compound payload fully assembled by DCM before dispatch). | + + ### Scoring Model Terms | Term | Definition | @@ -165,6 +180,7 @@ Terms to avoid because they introduce ambiguity. Use the precise alternatives in | DRC | Drift Reconciliation | | FCM | Federated Contribution Model | | SMX | Scoring Model | +| MPX | Meta Provider Composability | --- From dcd548d9af0af85a893328bd983bb44905bd39c3 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sun, 29 Mar 2026 11:24:29 -0500 Subject: [PATCH 42/49] Clarified design criterias and security posture modeling to ensure security is top priority and extensible. Signed-off-by: Chris Roadfeldt --- content/docs/README.md | 5 +- content/docs/architecture/ai-prompt.md | 191 +++- .../data-model/authority-tier-model.md | 531 ++++++++++ .../data-model/credential-provider-model.md | 918 ++++++++++++++++++ .../data-model/design-priorities.md | 316 ++++++ .../federated-contribution-model.md | 86 +- .../architecture/data-model/foundations.md | 37 +- .../data-model/layering-and-versioning.md | 4 +- .../data-model/policy-profiles.md | 15 +- .../data-model/provider-contract.md | 45 +- .../data-model/registry-governance.md | 4 +- .../architecture/data-model/scoring-model.md | 37 +- content/docs/architecture/overview.md | 25 +- .../specifications/admin-api-spec.md | 247 ++++- .../specifications/consumer-api-spec.md | 83 +- .../specifications/flow-gui-spec.md | 16 +- .../specifications/registration-spec.md | 74 +- content/docs/capabilities-matrix.md | 36 +- content/docs/taxonomy.md | 61 +- 19 files changed, 2583 insertions(+), 148 deletions(-) create mode 100644 content/docs/architecture/data-model/authority-tier-model.md create mode 100644 content/docs/architecture/data-model/credential-provider-model.md create mode 100644 content/docs/architecture/data-model/design-priorities.md diff --git a/content/docs/README.md b/content/docs/README.md index babf634..c8e18f4 100644 --- a/content/docs/README.md +++ b/content/docs/README.md @@ -19,6 +19,7 @@ DCM is built on three foundational abstractions โ€” **Data**, **Provider**, and ## Documentation Structure ### Foundation Documents (read these first) +| [00-design-priorities.md](data-model/00-design-priorities.md) | Design priority framework โ€” decision hierarchy for all contributors | | Document | Purpose | |----------|---------| | [00-foundations.md](data-model/00-foundations.md) | The three abstractions โ€” Data, Provider, Policy | @@ -51,8 +52,8 @@ Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI | Policy types | 7 (unified base contract + typed output schemas) | | Control plane components | 9 | | Four lifecycle states | Intent ยท Requested ยท Realized ยท Discovered | -| Capabilities | 134 across 21 domains | -| Data model documents | 34 (29 numbered + 3 foundation + 2 examples) | +| Capabilities | 155 across 24 domains | +| Data model documents | 38 (32 numbered + 3 foundation + 2 examples + 1 design priorities) | | Specifications | 10 | | Unresolved questions | 0 | diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 73f732f..5966f9a 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -72,6 +72,22 @@ Effective at the core mission ยท Easy to use ยท Easy to implement ยท Easy to ext --- +## SECTION 0b โ€” DESIGN PRIORITY ORDER (applies to all decisions) + +> **Full specification:** [00-design-priorities.md](data-model/00-design-priorities.md) โ€” includes decision framework, profile scaling table, and DPO-001โ€“006 system policies. + +**Priority 1 โ€” Security (industry best practices):** Security properties are architecturally present in ALL profiles. What profiles control is enforcement strictness, threshold values, and automation level โ€” not whether security applies. A `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security." + +**Priority 2 โ€” Ease of use:** The secure path must be the easy path. If the right path is also the hard path, teams will find other paths. Auto-approval for ordinary requests, policy authoring without Rego expertise, and profile defaults that eliminate configuration burden all serve this priority. + +**Priority 3 โ€” Extensibility/grouping:** Profile system, compliance domain overlays, policy groups, and registry governance enable adaptability through configuration, not code. New compliance requirements are policy additions. New deployment contexts are profile configurations. + +**Priority 4 โ€” Fit for purpose (always required):** DCM must manage data center infrastructure lifecycle end-to-end. Everything above serves this purpose. A system that cannot provision, track, and decommission a VM has failed. + +**Implication for all design decisions:** When security and convenience conflict, security wins โ€” but find a way to make the secure option easy. When extensibility and fit for purpose conflict, fit for purpose wins. When a profile tempts you to disable a security property rather than raise its threshold, the priority order says: keep the property, raise the threshold. + +--- + ## SECTION 1 โ€” PROJECT IDENTITY You are assisting with the **DCM (Data Center Management)** project, an open-source strategic framework developed by the Red Hat FlightPath team developed by the Red Hat FlightPath Team. @@ -487,6 +503,7 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | **Federated Contribution Model** | DCM defaults to federated data creation โ€” all authorized actor types (platform admin, consumer/tenant, service provider, peer DCM) can contribute Data artifacts within their domain scope via the GitOps PR model; see doc 28 | | **contributor** | Actor type that authored a Data artifact; recorded in artifact_metadata.contributed_by; determines review requirements; platform_admin / consumer / service_provider / peer_dcm | | **contributed_by** | Artifact metadata block recording contributor_type, actor UUID, contribution_method, pr_url, reviewed_by; immutable once set | +| **DPO-001โ€“006** | Design Priority system policies. DPO-001: security properties present in all profiles (not controlled by profiles). DPO-002: every security requirement needs an ease-of-use mechanism. DPO-005: minimal profile = "security with minimal overhead" not "minimal security". DPO-006: when security and ease conflict, redesign ease-of-use, not security. | | **FCM-001โ€“008** | Federated Contribution Model system policies; key: FCM-002 (domain scope violations = hard DENY), FCM-003 (GitOps PR for all), FCM-008 (contributor scope limits absolute) | | **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | | **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | @@ -495,7 +512,7 @@ Providers are **custodians** of the underlying infrastructure โ€” they are not t | **REDACT** | Governance matrix decision: replace field value with ``; field presence preserved; receiver knows field exists but not its value | | **Provider Type Registry** | Three-tier registry of approved provider types; each entry declares permissions, default_approval_method, enabled_in_profiles, capability_schema_ref | | **registration_token** | Pre-issued by platform admin; scoped to provider_type/handle_pattern/zone; single_use; grants_auto_approval flag; value presented once only | -| **approval_method** | Registration approval: auto | human_review | dual_approval | committee; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | +| **approval_method** | Registration approval: auto | reviewed | verified | authorized; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | | **Drift Reconciliation Component** | Control plane component; compares Discovered vs Realized State; produces drift records and events; never writes to Realized Store; DRC-001โ€“005 | | **drift_record** | Artifact produced by Drift Reconciliation; field-by-field comparison result with severity classification; unsanctioned flag; status tracking through resolution | | **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | @@ -2593,6 +2610,7 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | **Federated Contribution Model** | DCM defaults to federated data creation โ€” all authorized actor types (platform admin, consumer/tenant, service provider, peer DCM) can contribute Data artifacts within their domain scope via the GitOps PR model; see doc 28 | | **contributor** | Actor type that authored a Data artifact; recorded in artifact_metadata.contributed_by; determines review requirements; platform_admin / consumer / service_provider / peer_dcm | | **contributed_by** | Artifact metadata block recording contributor_type, actor UUID, contribution_method, pr_url, reviewed_by; immutable once set | +| **DPO-001โ€“006** | Design Priority system policies. DPO-001: security properties present in all profiles (not controlled by profiles). DPO-002: every security requirement needs an ease-of-use mechanism. DPO-005: minimal profile = "security with minimal overhead" not "minimal security". DPO-006: when security and ease conflict, redesign ease-of-use, not security. | | **FCM-001โ€“008** | Federated Contribution Model system policies; key: FCM-002 (domain scope violations = hard DENY), FCM-003 (GitOps PR for all), FCM-008 (contributor scope limits absolute) | | **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | | **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | @@ -2601,7 +2619,7 @@ The Ship/Shore/Enclave terminology from defense IT contexts has been replaced th | **REDACT** | Governance matrix decision: replace field value with ``; field presence preserved; receiver knows field exists but not its value | | **Provider Type Registry** | Three-tier registry of approved provider types; each entry declares permissions, default_approval_method, enabled_in_profiles, capability_schema_ref | | **registration_token** | Pre-issued by platform admin; scoped to provider_type/handle_pattern/zone; single_use; grants_auto_approval flag; value presented once only | -| **approval_method** | Registration approval: auto | human_review | dual_approval | committee; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | +| **approval_method** | Registration approval: auto | reviewed | verified | authorized; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | | **Drift Reconciliation Component** | Control plane component; compares Discovered vs Realized State; produces drift records and events; never writes to Realized Store; DRC-001โ€“005 | | **drift_record** | Artifact produced by Drift Reconciliation; field-by-field comparison result with severity classification; unsanctioned flag; status tracking through resolution | | **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | @@ -3535,21 +3553,21 @@ The **single enforcement point** for all cross-boundary data and capability deci ### 53.2 Registration Specification (dcm-registration-spec.md) -**Provider Type Registry:** Three-tier (Core/Community/Organization). Each entry declares permissions, default_approval_method, default_trust_level, enabled_in_profiles, capability_schema_ref. Nine core types: service_provider (human_review), meta_provider (dual_approval), storage_provider (dual_approval), policy_provider-mode-3-4 (dual_approval), credential_provider (dual_approval), auth_provider (dual_approval), information_provider/message_bus/notification_provider (human_review). +**Provider Type Registry:** Three-tier (Core/Community/Organization). Each entry declares permissions, default_approval_method, default_trust_level, enabled_in_profiles, capability_schema_ref. Nine core types: service_provider (reviewed), meta_provider (verified), storage_provider (verified), policy_provider-mode-3-4 (verified), credential_provider (verified), auth_provider (verified), information_provider/message_bus/notification_provider (reviewed). **Registration token model:** Pre-issued by platform admin (POST /api/v1/admin/registration-tokens). Scoped to provider_type, handle_pattern, sovereignty_zone. single_use. grants_auto_approval flag. Token value presented once โ€” never retrievable. Max trust level bounded by token scope. **Approval method resolution:** most_restrictive(provider_type_default, profile_min_method, token_grants_auto). Profile can only tighten. Token can relax to auto ONLY if profile.allow_token_auto_approval=true. Committee approval cannot be relaxed by token. -**Profile defaults:** minimal/dev โ†’ human_review, token auto-approval enabled. standard โ†’ human_review, token auto-approval enabled (max trust: standard). prod โ†’ human_review; high-trust types require dual_approval; no token auto-approval. fsi โ†’ dual_approval everything; minimum_accreditation: third_party. sovereign โ†’ committee everything; minimum_accreditation: regulatory_certification; hardware_attestation required. +**Profile defaults:** minimal/dev โ†’ reviewed, token auto-approval enabled. standard โ†’ reviewed, token auto-approval enabled (max trust: standard). prod โ†’ reviewed; high-trust types require verified; no token auto-approval. fsi โ†’ verified everything; minimum_accreditation: third_party. sovereign โ†’ authorized everything; minimum_accreditation: regulatory_certification; hardware_attestation required. -**Registration pipeline:** SUBMITTED โ†’ VALIDATING (8 automated checks: provider type enabled, governance matrix pre-check, registration token, certificate, sovereignty declaration, capability consistency, health endpoint, accreditation) โ†’ PENDING_APPROVAL โ†’ ACTIVE. Approval methods: auto (immediate), human_review (one admin), dual_approval (two independent admins), committee (DCMGroup quorum). +**Registration pipeline:** SUBMITTED โ†’ VALIDATING (8 automated checks: provider type enabled, governance matrix pre-check, registration token, certificate, sovereignty declaration, capability consistency, health endpoint, accreditation) โ†’ PENDING_APPROVAL โ†’ ACTIVE. Approval methods: auto (immediate), reviewed (one admin), verified (two independent admins), authorized (DCMGroup quorum). **Per-type capability schemas:** service_provider (resource types, capacity model, cancellation support, discovery, naturalization format, cost metadata), information_provider (data domains, authority level, query capacity, confidence model), storage_provider (store types, consistency, replication, encryption), policy_provider (mode 1-4, framework, remote endpoint, shadow mode support), auth_provider (auth modes, MFA methods, RBAC model, token lifetime), notification_provider (delivery channels, guarantees, sovereignty-aware delivery), credential_provider (credential types, secret engines, HSM support), message_bus_provider (protocols, durability, external_endpoints flag), meta_provider (constituent types, composition model, compensation support). -**Federated trust postures:** verified (manually approved; full scope), vouched (Hub-introduced; bounded scope), provisional (crypto-verified; catalog_query only if profile permits). Approval: dev auto-promotes provisional; standard human_review for verified; prod/fsi dual_approval; sovereign committee+hardware-attestation. Profile federation_policy block declares all parameters. +**Federated trust postures:** verified (manually approved; full scope), vouched (Hub-introduced; bounded scope), provisional (crypto-verified; catalog_query only if profile permits). Approval: dev auto-promotes provisional; standard reviewed for verified; prod/fsi verified; sovereign authorized+hardware-attestation. Profile federation_policy block declares all parameters. -**Ongoing lifecycle:** health monitoring (polling; degraded โ†’ reduced routing; failure_threshold โ†’ UNAVAILABLE; 2ร—threshold โ†’ drift triggered), certificate rotation (P90D default; P14D warning; P7D transition window), capability amendments (simplified flow), graceful deregistration (entity migration plan required), forced deregistration (dual_approval/committee; entities โ†’ INDETERMINATE_REALIZATION; Recovery Policy fires). +**Ongoing lifecycle:** health monitoring (polling; degraded โ†’ reduced routing; failure_threshold โ†’ UNAVAILABLE; 2ร—threshold โ†’ drift triggered), certificate rotation (P90D default; P14D warning; P7D transition window), capability amendments (simplified flow), graceful deregistration (entity migration plan required), forced deregistration (verified/authorized; entities โ†’ INDETERMINATE_REALIZATION; Recovery Policy fires). ### 53.3 Drift Reconciliation Component (doc 25 Section 7) @@ -3583,12 +3601,12 @@ DCM defaults to a federated model for data creation, import, usage, and lifecycl ### Contributor Permission Boundaries (hard DENY โ€” Governance Matrix enforced) - Consumers cannot contribute system or platform domain policies - Providers cannot contribute specs for resource types they don't offer -- Provisional peers: registry entries only (no policies; committee approval) -- Vouched peers: registry entries + service layers only (human_review always) -- Verified peers: registry entries + policy templates + service layers (human_review standard+; auto dev) +- Provisional peers: registry entries only (no policies; authorized approval) +- Vouched peers: registry entries + service layers only (reviewed always) +- Verified peers: registry entries + policy templates + service layers (reviewed standard+; auto dev) ### Universal Contribution Pipeline -Submit โ†’ Governance Matrix evaluates contributor permissions โ†’ proposed status (shadow mode for policies) โ†’ review flow (auto / human_review / dual_approval / committee per profile + artifact type + contributor) โ†’ active โ†’ lifecycle by contributor (deprecate/retire) โ†’ platform admin override at any time +Submit โ†’ Governance Matrix evaluates contributor permissions โ†’ proposed status (shadow mode for policies) โ†’ review flow (auto / reviewed / verified / authorized per profile + artifact type + contributor) โ†’ active โ†’ lifecycle by contributor (deprecate/retire) โ†’ platform admin override at any time ### Contribution Artifact Types by Contributor - Consumer: tenant policies (all 7 types), resource groups, notification subs, webhooks, cross-tenant auth records, request layers @@ -3603,10 +3621,10 @@ Every artifact includes `contributed_by` block: contributor_type, actor/tenant/p ### Profile-Governed Auto-Approval - minimal/dev: most contributions auto-approved; shadow optional -- standard: consumer/provider policies โ†’ human_review; shadow default on, P7D review period -- prod: governance matrix rules โ†’ dual_approval; provider specs โ†’ human_review; shadow P14D -- fsi: all consumer/provider contributions โ†’ dual_approval; shadow P30D; must review all divergence cases -- sovereign: all โ†’ committee; shadow P30D; orphaned artifacts auto-retire +- standard: consumer/provider policies โ†’ reviewed; shadow default on, P7D review period +- prod: governance matrix rules โ†’ verified; provider specs โ†’ reviewed; shadow P14D +- fsi: all consumer/provider contributions โ†’ verified; shadow P30D; must review all divergence cases +- sovereign: all โ†’ authorized; shadow P30D; orphaned artifacts auto-retire ### Consumer API Contribution Endpoints (Section 9) `POST /api/v1/contribute/policy` (generates PR, activates shadow mode) ยท `POST /api/v1/contribute/resource-group` (activates immediately) ยท `GET /api/v1/contribute` (list contributions) ยท `DELETE /api/v1/contribute/{uuid}` (withdraw, closes PR) @@ -3693,6 +3711,7 @@ FCM-001: contributor recorded in contributed_by; immutable. FCM-002: domain scop | **Federated Contribution Model** | DCM defaults to federated data creation โ€” all authorized actor types (platform admin, consumer/tenant, service provider, peer DCM) can contribute Data artifacts within their domain scope via the GitOps PR model; see doc 28 | | **contributor** | Actor type that authored a Data artifact; recorded in artifact_metadata.contributed_by; determines review requirements; platform_admin / consumer / service_provider / peer_dcm | | **contributed_by** | Artifact metadata block recording contributor_type, actor UUID, contribution_method, pr_url, reviewed_by; immutable once set | +| **DPO-001โ€“006** | Design Priority system policies. DPO-001: security properties present in all profiles (not controlled by profiles). DPO-002: every security requirement needs an ease-of-use mechanism. DPO-005: minimal profile = "security with minimal overhead" not "minimal security". DPO-006: when security and ease conflict, redesign ease-of-use, not security. | | **FCM-001โ€“008** | Federated Contribution Model system policies; key: FCM-002 (domain scope violations = hard DENY), FCM-003 (GitOps PR for all), FCM-008 (contributor scope limits absolute) | | **Unified Governance Matrix** | Single enforcement point for all cross-boundary decisions; four axes (subject/data/target/context); hard vs soft enforcement; field-level granularity (allowlist/blocklist/paths); profile-bound defaults; GMX-001โ€“010 | | **governance_matrix_rule** | Artifact declaring match conditions across four axes and a decision (ALLOW/DENY/ALLOW_WITH_CONDITIONS/STRIP_FIELD/REDACT/AUDIT_ONLY) with hard or soft enforcement | @@ -3701,7 +3720,7 @@ FCM-001: contributor recorded in contributed_by; immutable. FCM-002: domain scop | **REDACT** | Governance matrix decision: replace field value with ``; field presence preserved; receiver knows field exists but not its value | | **Provider Type Registry** | Three-tier registry of approved provider types; each entry declares permissions, default_approval_method, enabled_in_profiles, capability_schema_ref | | **registration_token** | Pre-issued by platform admin; scoped to provider_type/handle_pattern/zone; single_use; grants_auto_approval flag; value presented once only | -| **approval_method** | Registration approval: auto | human_review | dual_approval | committee; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | +| **approval_method** | Registration approval: auto | reviewed | verified | authorized; resolved as most_restrictive(provider_type_default, profile_min, token_effect) | | **Drift Reconciliation Component** | Control plane component; compares Discovered vs Realized State; produces drift records and events; never writes to Realized Store; DRC-001โ€“005 | | **drift_record** | Artifact produced by Drift Reconciliation; field-by-field comparison result with severity classification; unsanctioned flag; status tracking through resolution | | **Placement Engine** | Six-step algorithm: sovereignty filter โ†’ accreditation filter โ†’ capability filter โ†’ reserve query โ†’ tie-breaking (policy/priority/affinity/cost/load/hash) โ†’ confirm; PLC-001โ€“006 | @@ -4214,9 +4233,10 @@ Questions of fact use boolean gates. Questions of degree use scoring. Secondary 5. **Provider accreditation richness** (weight: 0.10, inverse) โ€” weighted portfolio sum; higher richness = lower provider risk contribution ### Profile-Governed Thresholds โ†’ Approval Routing -auto_approve (threshold) -Default per profile: minimal(<60), dev(<50), standard(<25), prod(<15), fsi(<10), sovereign(<5) -**SMX-008: auto_approve_below may never exceed 50 in any profile.** +auto_approve (threshold) +Default per profile: minimal(<45), dev(<40), standard(<25), prod(<15), fsi(<10), sovereign(<5) +SMX-008 applies to ALL profiles including minimal โ€” auto_approve_below may never exceed 50 +**SMX-008: auto_approve_below may never exceed 50 in any profile, including minimal.** minimal achieves higher effective auto-approval through lower signal weights, not higher thresholds. Signal weights must sum to 1.00 (validated at profile activation). ### Profile Enforcement Class Overrides @@ -4318,7 +4338,127 @@ MPX-001: self constituents use standard Services API. MPX-002: DCM derives order ### Capabilities: MPX-001โ€“MPX-007 (Domain 22 โ€” 141 total across 22 domains) -## SECTION 63 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 63 โ€” CREDENTIAL PROVIDER MODEL (doc 31) + +### Two Credential Categories +1. **DCM Interaction Credentials** โ€” short-lived (PT15Mโ€“PT1H profile-governed), scoped to specific operation+entity+provider. Issued before every provider dispatch. Implements ZTS-002. Never stored beyond use. +2. **Consumer-Facing Resource Credentials** โ€” SSH keys, API keys, kubeconfigs, service account tokens, database passwords, x509 certificates. Issued as part of resource realization; delivered via Consumer API. + +### CPX-001 (most important): Values NEVER in DCM stores +Credential values are never written to GitOps stores, Realized State Store, or Audit Store. DCM stores only metadata (UUID, type, scope, expiry, status). Values held by Credential Provider; retrieved via authenticated `value_retrieval_endpoint`. + +### Credential Record Fields +credential_uuid, credential_type, status (active/rotating/revoked/expired), issued_at, valid_until, issued_to (actor/entity/component/provider UUID), scope (operations[], resource_types[], tenant_uuid), non_transferable:true, bound_to_ip (fsi/sovereign), value_retrieval_endpoint, value_retrieval_auth, rotation_of (parent UUID if rotation), credential_provider_uuid, entity_uuid + +### Issuance Flows +- **Resource credential:** after VM/resource realized โ†’ DCM sub-request to Credential Provider โ†’ metadata stored in Realized State โ†’ consumer retrieves value via authenticated endpoint +- **Interaction credential:** before each provider dispatch โ†’ Credential Provider issues scoped cred โ†’ included in dispatch โ†’ expires PT15M regardless; new cred issued for each interaction +- **Bootstrap:** special mechanism before Credential Provider is registered; see doc 17 + +### Rotation Protocol +Trigger types: pre_expiry (default), scheduled, security_event, actor_request, provider_initiated. Standard flow: issue new cred โ†’ transition window (both valid) โ†’ revoke old at window end โ†’ notify consumer. Window: P1D consumer creds; PT5M dcm_interaction; P7D x509. Emergency rotation (security_event): NO transition window โ€” old revoked immediately; fastest-channel delivery of new. + +### Revocation Model +Revocation Triggers: actor_deprovisioned, entity_decommissioned, security_event, provider_deregistered, actor_request, ttl_expired. Propagation: credential record โ†’ status:revoked โ†’ publish credential.revoked to Message Bus โ†’ all components refresh revocation cache within SLA (PT5M standard; PT1M fsi/sovereign) โ†’ Credential Provider invalidates stored value within SLA. + +### Use-Time Validation (CPX-002 enforcement) +Providers must validate at use time (not just receipt): check revocation cache, verify valid_until, verify operation within scope, verify IP binding. Reject with 403 if any check fails. Cache refresh: โ‰ค PT1M standard; โ‰ค PT30S fsi/sovereign. + +### CPX-006: Actor Deprovisioning +Triggers immediate revocation of ALL credentials issued to actor. Revocation events published to Message Bus BEFORE deprovisioning acknowledged. +CPX-007: Entity decommissioning triggers revocation of all entity-scoped credentials before decommission confirmed. Decommission that cannot revoke enters COMPENSATION_IN_PROGRESS. + +### Consumer API Endpoints +GET /api/v1/resources/{entity_uuid}/credentials โ€” list credential metadata +GET /api/v1/credentials/{uuid}/value โ€” retrieve value (step_up_mfa if required); every retrieval audited with retrieval_uuid +POST /api/v1/credentials/{uuid}/rotate โ€” request rotation; returns old/new UUIDs + transition_window_ends + +### Credential Provider API Contract +POST {issue_endpoint} ยท POST {rotate_endpoint} ยท DELETE {revoke_endpoint}/{uuid} ยท POST {validate_endpoint} (use-time check) ยท GET {list_endpoint}?entity_uuid= + +### Profile-Governed Credential Configuration (doc 31 Section 12) +credential_profile block controls: permitted_credential_types (homelab: api_key/x509/ssh; sovereign: hsm_backed_key only) ยท max_lifetime per credential type per profile ยท scheduled_rotation_required (ALL profiles: true; minimal/dev allow manual trigger and P365D/P180D max intervals) ยท min_transition_window (minimal: PT0S; standard+: P1D) ยท value_retrieval_auth_required (minimal: bearer_token; prod: step_up_mfa; sovereign: mtls) ยท audit_every_retrieval (minimal: false; standard+: true) ยท idle_detection_threshold (minimal: P30D; dev: P14D; standard: P7D; prod: P3D; fsi: P1D; sovereign: PT12H โ€” NEVER null) ยท ip_binding_required (minimal-prod: false; fsi/sovereign: true) ยท fips_140_level_required (minimal: 0; fsi: Level 2; sovereign: Level 3) ยท approved_algorithms (minimal: forbidden_algorithms list [MD5,SHA-1,DES,3DES,RSA<2048]; standard: Ed25519/ECDSA-P-384; fsi: FIPS-only; sovereign: HSM-generated only) ยท revocation_check_frequency (minimal: PT5M; fsi: PT30S; sovereign: PT15S) ยท revocation_sla (minimal: PT10M; sovereign: PT30S) + +### Compliance Domain Overlays (additive, never relaxing) +hipaa: audit_every_retrieval:true, idle_detection:P7D, max rotation api_key:P90D +pci_dss: max_rotation_interval:P90D (mandatory โ€” req 8.3.9), min_password_complexity:12+4-classes +fedramp_moderate: fips_level:1 ยท fedramp_high: fips_level:2, ip_binding:true ยท dod_il4: fips:2, ip_binding:true + +### AAL Mapping (NIST 800-63B) +minimal/dev=AAL1 ยท standard=AAL2 (MFA for sensitive types) ยท prod=AAL2 (MFA all) ยท fsi=AAL2+ (hardware MFA, FIPS L2) ยท sovereign=AAL3 (hardware-bound, FIPS L3, tamper evidence) + +### New Fields on Credential Record +algorithm (Ed25519/ECDSA-P-384/RSA-4096/HS256/etc.) ยท key_usage [authentication|signing|encryption] ยท retrieved_count_threshold (hours; idle alert threshold) + +### CPX-001โ€“CPX-012 System Policies +CPX-001: values never in DCM stores. CPX-002: every provider interaction must present scoped credential. CPX-003: revocation propagation within declared SLA. CPX-004: emergency rotation has no transition window. CPX-005: every value retrieval audited. CPX-006: actor deprovisioning revokes all actor credentials. CPX-007: entity decommission blocks on credential revocation. CPX-008: fsi/sovereign credentials must be IP-bound or hardware-attested. + +### Capabilities: CPX-001โ€“CPX-007 (Domain 23 โ€” 148 total across 23 domains) + +--- + +## SECTION 64 โ€” AUTHORITY TIER MODEL (doc 32 โ€” 32-authority-tier-model.md) + +> **Full specification:** [32-authority-tier-model.md](data-model/32-authority-tier-model.md) โ€” ordered tier list, custom tier contribution, dynamic threshold format, impact detection pipeline, ATM-001โ€“ATM-012. + +### Core Model +Authority tiers are a **named, ordered list**. Names are stable references; numeric weight is derived from list position at evaluation time โ€” never hardcoded. Organizations can insert custom tiers between existing ones without breaking any existing name references. + +### Default Tier List (ordered) +``` +auto โ†’ reviewed โ†’ verified โ†’ authorized +``` +Position determines weight: auto=1, reviewed=2, verified=3, authorized=4. +If org inserts `compliance_reviewed` after `verified`: auto=1, reviewed=2, verified=3, compliance_reviewed=4, authorized=5. +All existing references to `authorized` still resolve correctly. + +### decision_gravity (stable severity vocabulary) +- `none` โ†’ auto (automated; no human judgment) +- `routine` โ†’ reviewed (standard authority; one qualified reviewer) +- `elevated` โ†’ verified (elevated authority; separation of duties; two distinct reviewers) +- `critical` โ†’ authorized (highest authority weight; DCMGroup + quorum required) + +decision_gravity is stable and position-independent. Custom tiers must declare consistent gravity. + +### Dynamic Threshold Format +Profile thresholds are a named-tier list, NOT fixed column keys: +```yaml +approval_routing: + - { tier: auto, max_score: 24 } # ATM-002: never exceed 50 + - { tier: reviewed, max_score: 59 } + - { tier: verified, max_score: 79 } + - { tier: authorized, max_score: 100 } +``` +Custom tiers insert into this list. Existing tier names and ranges shift only for the affected range. + +### Custom Tiers +Contributed via standard contribution pipeline; require `verified` tier approval (ATM-004). Must declare decision_gravity consistent with position (ATM-003). Cannot alter dcm_gate semantics of existing DCM system tiers (ATM-005). + +### Authorized Tier (dcmgroup_required: true) +Requires a declared DCMGroup and quorum threshold. Organization defines: group composition (CTO, CISO, board, single delegate โ€” any structure), how members deliberate, what external tools they use. DCM enforces that N members of the declared DCMGroup recorded decisions via Admin API. Organization provides everything else. + +### Tier Registry Change Impact Detection (doc 32 Section 7) +When tier registry changes, DCM computes a **tier_impact_diff** โ€” a structured comparison of proposed vs current ordered list โ€” before activation: +- **SECURITY_DEGRADATION**: tier's gravity or position decreased โ†’ blocks activation until reviewed and accepted (ATM-009) +- **BROKEN_REFERENCE**: tier name removed but still referenced โ†’ blocks activation until resolved (ATM-010) +- **PROFILE_GAP**: new tier inserted but profile threshold list not updated โ†’ warning, does not block (ATM-012) +- **SECURITY_UPGRADE / STALE_WEIGHT**: informational, does not block + +Degradation review gate: each SECURITY_DEGRADATION must be accepted via `POST /admin/api/v1/tier-registry/{change_uuid}/accept-degradation` by a `verified` or `authorized` tier reviewer before activation. + +Impact report (ATM-011) stored in Audit Store for every registry change, at proposal and at activation. + +Admin API: POST /admin/api/v1/tier-registry/changes (propose) ยท GET .../impact (report) ยท POST .../accept-degradation ยท POST .../activate + +### ATM-001โ€“ATM-012 System Policies +ATM-001: tiers identified by name; weight derived from position. ATM-002: auto tier max_score โ‰ค 50. ATM-003: custom gravity consistent with position. ATM-004: custom tiers require verified-tier approval. ATM-005: custom tiers cannot change existing tier dcm_gate semantics. ATM-006: dcmgroup_required tiers must have DCMGroup declared before use. ATM-007: four gravity values are DCM vocabulary (org cannot add gravity values). ATM-008: approval records store weight at creation time for point-in-time audit. + +### Federation Tier Resolution +Peer DCM instances may have different custom tier lists. Resolution strategy: `gravity_match` โ€” match by decision_gravity, not tier name. Unknown peer tiers escalate to their declared gravity level. + +--- + +## SECTION 65 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure): @@ -4329,10 +4469,19 @@ When working on this project, apply these instructions in addition to the number 176. **GateKeeper enforcement_class is required and fail-safe** โ€” if omitted, treated as compliance (boolean deny). Operational-class GateKeepers never halt the request; they contribute a weighted risk_score_contribution to the aggregate. The aggregate risk score determines approval routing, not individual policy outcomes. 177. **Validation output_class is required and fail-safe** โ€” if omitted, treated as structural (boolean halt). Advisory-class Validations never halt requests; they accumulate completeness score and warning list surfaced to the consumer. 178. **Governance Matrix is always boolean โ€” never scored** โ€” SMX-004 is absolute. Scoring cannot be used to route around data sovereignty or regulatory boundaries. The Governance Matrix evaluates before the scoring pipeline runs. -179. **Profile thresholds determine routing, not individual policies** โ€” the approval routing decision (auto/review/dual/committee) emerges from the aggregate risk score crossing profile-configured thresholds, not from individual policy flags. Changing governance sensitivity = adjusting thresholds in the profile. +179. **Profile thresholds determine routing, not individual policies** โ€” the approval routing decision (auto/review/dual/authorized) emerges from the aggregate risk score crossing profile-configured thresholds, not from individual policy flags. Changing governance sensitivity = adjusting thresholds in the profile. 180. **SMX-008 is a hard system constraint** โ€” auto_approve_below may never exceed 50 in any profile. Platform admins cannot override this. Profiles submitted with auto_approve_below > 50 fail validation. 181. **Meta Provider is a compound service definition + standard Service Provider** โ€” not an orchestrator. It declares the dependency graph so DCM can place, sequence, and govern constituents. For `self` constituents it executes as any Service Provider does. DCM handles all orchestration, placement, failure, and compensation. 182. **Composite Entity has ONE entity UUID** that links Intent, Requested, Realized, and Discovered states; the UUID is assigned at Intent creation and is stable throughout the lifecycle including rehydration 183. **DEGRADED is a valid terminal state** โ€” not an error; a DEGRADED entity enters standard OPERATIONAL lifecycle; profile governs whether degraded delivery is accepted; Recovery Policy governs failure/compensation decisions 184. **Parallelism emerges from the dependency graph** โ€” constituents with no unresolved dependencies dispatch concurrently within DCM's pipeline; the Meta Provider does not manage this +186. **Credential values are NEVER stored in DCM** (CPX-001) โ€” only metadata is stored; values are held by the Credential Provider; retrieved via authenticated endpoint; this applies to ALL credential types including dcm_interaction credentials +187. **Every provider dispatch requires a scoped interaction credential** (CPX-002) โ€” issued before dispatch, scoped to the specific operation+entity+provider, expires PT15M; provider must validate at use time not just receipt; check revocation cache on each use +189. **Security properties are present in ALL profiles โ€” minimal profile is "security with minimal operational overhead" not "minimal security"** โ€” rotation required in all profiles (minimal: P365D max, manual OK); idle detection on in all profiles (minimal: P30D); algorithm baseline in all profiles (minimal: forbidden list); CPX-001 (values never in DCM stores) is absolute โ€” homelab (minimal) uses bearer_token retrieval, no scheduled rotation, no FIPS; sovereign uses mtls+hardware attestation, FIPS Level 3, PT15S revocation cache; same API contract, same data model, same CPX-001 (values never in DCM stores) +194. **Tier registry changes are gated by impact detection** โ€” any change that creates a SECURITY_DEGRADATION (tier gravity or position decreased) blocks activation until each degradation is explicitly accepted by a verified-tier or above reviewer via Admin API; BROKEN_REFERENCE also blocks; PROFILE_GAP is a warning that does not block; all changes produce an impact report in the Audit Store (ATM-009โ€“012) +193. **Authority tiers are named positions in an ordered list โ€” not fixed enum values** โ€” tier weight derived from list position at evaluation time; organizations insert custom tiers between existing ones without breaking existing name references; 'authorized' tier always means 'highest current gravity' regardless of what's been inserted before it; ATM-001: never hardcode tier weights +192. **DCM provides the approval gate and audit trail โ€” the review process is the organization's responsibility** โ€” for authorized tier: DCM tracks quorum of a DCMGroup; the authorized deliberation and vote collection happen outside DCM; external systems (ServiceNow, Jira, Slack bots) can call Admin API to record votes; DCM does NOT build authorized management; for reviewed and verified: same principle โ€” DCM holds the pipeline until the API receives the required decisions +191. **The priority order is a decision framework, not a suggestion** โ€” when security and ease of use conflict, security wins AND you must design an easy mechanism for the secure path; "it's too complex" is a reason to improve the ease-of-use design, not to reduce security; "minimal profile" means minimal overhead, never minimal security (DPO-005, DPO-006) +190. **key_usage is declared at issuance and validated at use** (CPX-009) โ€” a credential issued for authentication cannot be used for signing; Credential Provider must validate this at the validate endpoint; prevents algorithm confusion attacks +188. **Actor deprovisioning and entity decommissioning trigger immediate credential revocation** (CPX-006, CPX-007) โ€” deprovisioning publishes revocation events before the deprovisioning is acknowledged; decommission is blocked until all entity-scoped credentials are revoked 185. **provided_by: external constituents are placed by DCM's Placement Engine** โ€” all governance controls (sovereignty, accreditation, trust) apply; the Meta Provider has no influence over external constituent provider selection diff --git a/content/docs/architecture/data-model/authority-tier-model.md b/content/docs/architecture/data-model/authority-tier-model.md new file mode 100644 index 0000000..293fac4 --- /dev/null +++ b/content/docs/architecture/data-model/authority-tier-model.md @@ -0,0 +1,531 @@ +--- +title: "Authority Tier Model" +type: docs +weight: 32 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” Authority Tier Specification +**Related Documents:** [Design Priorities](00-design-priorities.md) | [Scoring Model](29-scoring-model.md) | [Federated Contribution Model](28-federated-contribution-model.md) | [Policy Profiles](14-policy-profiles.md) | [Registry Governance](20-registry-governance.md) + +> **This document maps to: DATA + POLICY** +> +> The authority tier list is Data โ€” a versioned, ordered registry entry. Tiers are referenced by name in Policies (scoring thresholds, contribution approval requirements, registration requirements). The ordered list resolves numeric weight at evaluation time. + +--- + +## 1. The Core Model + +### 1.1 What an Authority Tier Is + +An authority tier declares the **required level of organizational decision gravity** for an action. It answers: "how consequential is this decision, and therefore how much authority must be engaged to approve it?" + +Tiers do **not** prescribe organizational mechanisms โ€” who satisfies a tier, how many people are involved, or what tools they use. That is entirely the organization's definition. Tiers provide the vocabulary and enforcement gate; organizations provide the substance. + +### 1.2 The Ordered List + +Authority tiers are defined as a **named, ordered list**. Position in the list determines numeric weight. Names are stable references used throughout the system. New tiers can be inserted anywhere without changing existing names or breaking existing references. + +```yaml +authority_tier_registry: + version: "1.0.0" + tiers: + - name: auto + position: 1 # derived from list order; do not hardcode + decision_gravity: none + description: > + No human judgment required. System confidence โ€” scoring, validation, + governance matrix checks โ€” is sufficient to proceed. DCM activates + automatically on pass. + dcm_gate: All structural and governance validation checks pass + organization_provides: Nothing โ€” fully automated + dcmgroup_required: false + + - name: reviewed + position: 2 + decision_gravity: routine + description: > + Standard authority. A qualified reviewer in the relevant domain + must evaluate and record a decision. Routine operational decisions + that benefit from human oversight but do not require elevated authority. + dcm_gate: One actor with reviewer role records a decision via Admin API + organization_provides: > + Who constitutes a qualified reviewer for this action type; + the review process; recording via DCM Admin API or external system + dcmgroup_required: false + typical_use: Standard request approval; routine policy contributions; dev/standard provider registration + + - name: verified + position: 3 + decision_gravity: elevated + description: > + Elevated authority. Two independent, distinct reviewers must each + evaluate and record a decision. Enforces separation of duties โ€” + the same actor cannot satisfy both requirements. Used for decisions + with operational or security significance requiring independent confirmation. + dcm_gate: Two distinct actors with reviewer role each record a decision via Admin API + organization_provides: > + Who constitutes qualified reviewers; both review processes; + may use external workflow tools that call the DCM Admin API + dcmgroup_required: false + typical_use: High-risk provider registration; elevated-score requests; significant policy changes + + - name: authorized + position: 4 + decision_gravity: critical + description: > + Highest authority weight. Reserved for decisions with organizational, + regulatory, or security consequence requiring the highest level of + deliberate authorization. Who constitutes sufficient authority is + entirely the organization's definition โ€” a CTO, a CISO and legal + counsel, a change advisory board, a single person with delegated + authority. DCM enforces that the declared authority group engaged + and recorded their decision; it does not prescribe the group structure + or deliberation process. + dcm_gate: N members of a declared DCMGroup record decisions via Admin API (quorum threshold) + organization_provides: > + Authority group composition (declared as a DCMGroup); + quorum threshold (N of M); deliberation process; external tools + (ServiceNow, Jira, Slack bots may call Admin API on behalf of members) + dcmgroup_required: true + typical_use: Governance matrix changes; sovereign-profile actions; credential provider registration; federation policy +``` + +### 1.3 Numeric Weight Resolution + +The numeric weight of a tier is its **position in the ordered list**, resolved at evaluation time. It is never stored as a hardcoded number in configuration. + +``` +Given the default list: auto(1) โ†’ reviewed(2) โ†’ verified(3) โ†’ authorized(4) + +If an organization inserts a custom tier: + auto(1) โ†’ reviewed(2) โ†’ verified(3) โ†’ compliance_reviewed(4) โ†’ authorized(5) + +DCM resolves: + weight("reviewed") = 2 + weight("verified") = 3 + weight("compliance_reviewed") = 4 + weight("authorized") = 5 + +All existing references to "authorized" continue to work. +No configuration changes required for existing tiers. +``` + +### 1.4 decision_gravity Vocabulary + +`decision_gravity` is a stable, position-independent classification used by the scoring model and profile system to reason about tier severity without depending on tier names. It is declared on each tier and must be assigned when creating custom tiers. + +| Value | Meaning | Default DCM tiers | +|-------|---------|------------------| +| `none` | Automated; no human judgment | `auto` | +| `routine` | Standard operational decision | `reviewed` | +| `elevated` | Significant decision; separation of duties | `verified` | +| `critical` | Highest consequence; maximum authority | `authorized` | + +Organizations creating custom tiers must assign one of these four gravity values. If a future need arises for a gravity level between `elevated` and `critical`, the vocabulary can be extended โ€” but this is a DCM-level change, not an organization-level one. + +--- + +## 2. Custom Tier Definition + +### 2.1 How Organizations Add Tiers + +Organizations can extend the authority tier list by contributing custom tier definitions through the standard contribution pipeline. Custom tiers are contributed at the organization or tenant domain scope. + +```yaml +custom_tier_contribution: + name: compliance_reviewed # unique within the deployment + insert_after: verified # position declaration โ€” inserts after this tier + decision_gravity: elevated # must match or be consistent with position + description: > + Elevated authority with mandatory compliance officer sign-off. + Required for actions affecting regulated data domains (PII, PCI, HIPAA). + The compliance officer may be one person or a designated compliance team; + the organization defines who satisfies this role. + dcm_gate: One actor with compliance_officer role records a decision via Admin API + organization_provides: > + Who holds the compliance_officer role; compliance review process; + may be satisfied by external GRC system calling Admin API + dcmgroup_required: false # single reviewer sufficient at this gravity + applicable_profiles: [standard, prod, fsi, sovereign] + contribution_requires: verified # adding a custom tier requires verified-tier approval +``` + +### 2.2 Contribution Approval Requirements + +Custom tier contributions require `verified` tier approval (two independent reviewers) because they affect all pipeline decisions in the deployment. An organization cannot unilaterally add a tier that demotes an existing gravity level or bypasses the `authorized` tier for critical decisions. + +**Constraints on custom tiers:** +- `decision_gravity` must be consistent with position (a tier inserted before `reviewed` cannot have `critical` gravity) +- Custom tiers cannot be inserted before `auto` or after the highest `critical` gravity tier +- A custom tier with `dcmgroup_required: true` must declare a valid DCMGroup at contribution time +- Custom tiers cannot change the `dcm_gate` semantics of existing DCM tiers + +### 2.3 External Tier Registries + +For federation deployments, peer DCM instances may have different custom tier lists. When a federated request requires approval from a peer's tier, DCM resolves the equivalent gravity level from the local list: + +```yaml +federation_tier_resolution: + strategy: gravity_match # match by decision_gravity, not tier name + on_unknown_tier: escalate_to_gravity # if peer tier unknown, use its declared gravity + fallback_tier: authorized # if gravity unknown, apply highest local tier +``` + +--- + +## 3. Profile Threshold Configuration + +### 3.1 Dynamic Threshold Format + +Profile approval thresholds are expressed as a named-tier list, not fixed column headers. This allows the threshold table to accommodate custom tiers without restructuring. + +```yaml +# Standard profile โ€” default DCM tiers +approval_routing: + thresholds: + - tier: auto + max_score: 24 # score 0โ€“24: auto-approve + - tier: reviewed + max_score: 59 # score 25โ€“59: reviewed tier required + - tier: verified + max_score: 79 # score 60โ€“79: verified tier required + - tier: authorized + max_score: 100 # score 80โ€“100: authorized tier required + +# Organization adds compliance_reviewed between verified and authorized +approval_routing: + thresholds: + - tier: auto + max_score: 24 + - tier: reviewed + max_score: 59 + - tier: verified + max_score: 74 # adjusted to make room + - tier: compliance_reviewed + max_score: 84 # new tier occupies this range + - tier: authorized + max_score: 100 # unchanged name; adjusted range +``` + +### 3.2 Default Profile Thresholds + +```yaml +profile_approval_thresholds: + minimal: + - { tier: auto, max_score: 44 } + - { tier: reviewed, max_score: 100 } + + dev: + - { tier: auto, max_score: 39 } + - { tier: reviewed, max_score: 69 } + - { tier: verified, max_score: 100 } + + standard: + - { tier: auto, max_score: 24 } + - { tier: reviewed, max_score: 59 } + - { tier: verified, max_score: 79 } + - { tier: authorized, max_score: 100 } + + prod: + - { tier: auto, max_score: 14 } + - { tier: reviewed, max_score: 49 } + - { tier: verified, max_score: 74 } + - { tier: authorized, max_score: 100 } + + fsi: + - { tier: auto, max_score: 9 } + - { tier: reviewed, max_score: 39 } + - { tier: verified, max_score: 69 } + - { tier: authorized, max_score: 100 } + + sovereign: + - { tier: auto, max_score: 4 } + - { tier: reviewed, max_score: 29 } + - { tier: verified, max_score: 59 } + - { tier: authorized, max_score: 100 } +``` + +### 3.3 SMX-008 in the Dynamic Model + +SMX-008 (auto_approve_below โ‰ค 50) remains a hard constraint. In the dynamic model: the `auto` tier's `max_score` may never exceed 50 in any profile, regardless of custom tier additions. This constraint applies to the `auto` tier specifically, not to named thresholds. + +--- + +## 4. Tier Evaluation in the Pipeline + +### 4.1 How DCM Resolves the Required Tier + +At request evaluation time, DCM: + +1. Computes the request risk score (0โ€“100) from the scoring model +2. Loads the active profile's threshold list +3. Walks the list in order; the first tier whose `max_score โ‰ฅ risk_score` is the required tier +4. Resolves the numeric weight of the required tier from the ordered tier list +5. Creates an approval record with the required tier name and weight + +The tier name โ€” not the weight โ€” is what is stored in the approval record and what reviewers see. The weight is used for comparison operations (e.g., "is this action at least as significant as `verified`?"). + +### 4.2 Approval Record + +```yaml +approval_record: + approval_uuid: + subject_uuid: + subject_type: request | policy_contribution | provider_registration | federation_contribution + required_tier: verified # tier name โ€” stable reference + required_tier_weight: 3 # resolved at creation; stored for point-in-time audit + required_tier_gravity: elevated + dcmgroup_uuid: # non-null only for dcmgroup_required: true tiers + quorum_threshold: + status: pending_reviewed | pending_verified | pending_authorized | pending_ + created_at: + window_expires_at: + decisions: [] +``` + +--- + +## 5. DCMGroup Assignment for Authorized Tier + +When a decision requires the `authorized` tier (or any custom tier with `dcmgroup_required: true`), the required DCMGroup and quorum threshold must be declared. This is configured in the profile or per-action-type: + +```yaml +authorized_tier_configuration: + default_dcmgroup_handle: platform/security-council # default group for authorized decisions + quorum_threshold: "2 of 5" # N of M + + # Per-action-type overrides + action_type_overrides: + - subject_type: provider_registration + provider_type: credential_provider + dcmgroup_handle: platform/credential-governance + quorum_threshold: "3 of 5" + - subject_type: federation_contribution + dcmgroup_handle: platform/federation-council + quorum_threshold: "2 of 3" + - subject_type: policy_contribution + policy_domain: system + dcmgroup_handle: platform/policy-governance + quorum_threshold: "3 of 5" +``` + +Organizations define the DCMGroup membership. DCM enforces that the declared group and quorum were satisfied. + +--- + +--- + +## 7. Tier Registry Change Impact Detection + +When the authority tier registry is modified โ€” a new tier inserted, a tier removed, a tier's `decision_gravity` changed, or a tier's position changed โ€” DCM must evaluate the impact on all items that reference tier names before activating the change. This section specifies the detection model. + +> **Implementation note:** The impact detection pipeline described here is a required implementation component, not an optional audit feature. A tier registry change that creates security degradations must not activate until each degradation is explicitly acknowledged by a reviewer at `verified` tier or above. The detection mechanism itself is an implementation detail; this specification defines the required behavior and data model. + +### 7.1 Tier Impact Diff + +Before activating a tier registry change, DCM computes a **tier impact diff** by comparing the proposed ordered list to the current ordered list. + +```yaml +tier_impact_diff: + registry_change_uuid: + proposed_at: + proposed_by: + + tier_changes: + - tier_name: verified + change_type: POSITION_CHANGED # NEW | REMOVED | POSITION_CHANGED | GRAVITY_CHANGED | UNCHANGED + old_position: 3 + new_position: 4 # something inserted before it + old_gravity: elevated + new_gravity: elevated # gravity unchanged + net_effect: UPGRADED # higher position = more weight = more scrutiny required + + - tier_name: compliance_reviewed # newly inserted + change_type: NEW + old_position: null + new_position: 3 + old_gravity: null + new_gravity: elevated + net_effect: NEW + + - tier_name: authorized + change_type: POSITION_CHANGED + old_position: 4 + new_position: 5 + old_gravity: critical + new_gravity: critical + net_effect: UPGRADED + + security_degradations: [] # list of DEGRADED tier changes + profile_gaps: [] # profiles whose threshold list is incomplete after change + broken_references: [] # tier names referenced in config that no longer exist +``` + +**Net effect classification:** + +| Net Effect | Condition | Risk | +|-----------|-----------|------| +| `UPGRADED` | Tier's position increased (higher weight) OR gravity increased | None โ€” more scrutiny required than before | +| `DEGRADED` | Tier's position decreased (lower weight) OR gravity decreased | **Security risk** โ€” items referencing this tier now have lower effective authority requirement | +| `NEW` | Tier inserted into registry | Low โ€” no existing references; profile gap detection applies | +| `REMOVED` | Tier deleted from registry | **Broken references** โ€” any item referencing this tier name is now unresolvable | +| `UNCHANGED` | Position and gravity identical | None | + +### 7.2 Affected Item Query + +After computing the tier impact diff, DCM queries for all items affected by each changed tier: + +``` +Affected item categories: + +PENDING APPROVAL RECORDS + Query: approval_records WHERE required_tier IN (changed_tier_names) AND status LIKE 'pending_%' + Impact: The tier name is stable; the weight at which the item was queued may differ from + the current weight. Compare stored_tier_weight (ATM-008) vs current_tier_weight. + +PROFILE THRESHOLD CONFIGURATIONS + Query: all profiles WHERE tier_registry_version < new_registry_version + Impact: Profiles whose threshold list doesn't include newly added tiers have a gap โ€” + requests that score into the new tier's range will fall back to the adjacent tier. + Flag as PROFILE_GAP; notify platform admin to update threshold list. + +PROVIDER REGISTRATION REQUIREMENTS + Query: provider_type_registry WHERE default_approval_method IN (changed_tier_names) + AND profile_registration_policy WHERE min_approval_method IN (changed_tier_names) + Impact: If the referenced tier's gravity decreased, the minimum requirement is now lower. + +FCM CONTRIBUTION POLICY REQUIREMENTS + Query: contribution_policy WHERE any tier reference IN (changed_tier_names) + Impact: Same as provider registration โ€” if gravity decreased, requirement is lower. + +ACTIVE POLICY SETS + Query: active_policies WHERE policy_content CONTAINS tier_name_reference + Impact: Policies that reason about tiers by name should be using dynamic resolution. + If a policy hardcodes a tier weight, it may now be stale. + Flag for policy owner review. +``` + +### 7.3 Impact Classification + +Each affected item receives one or more impact classifications: + +| Classification | Condition | Required Action | +|---------------|-----------|----------------| +| `SECURITY_DEGRADATION` | Item references a tier whose gravity decreased OR position decreased | **Blocks activation** โ€” must be reviewed and accepted | +| `SECURITY_UPGRADE` | Item references a tier whose gravity or position increased | Informational โ€” logged and reported; does not block | +| `BROKEN_REFERENCE` | Item references a tier name that no longer exists in registry | **Blocks activation** โ€” must be resolved (tier restored, item updated, or item cancelled) | +| `PROFILE_GAP` | Profile threshold list incomplete after new tier insertion | **Warning** โ€” does not block activation; platform admin must update thresholds or acknowledge gap | +| `STALE_WEIGHT` | Pending approval record's `stored_tier_weight` differs from current weight for same tier name | Informational โ€” logged; record remains valid since tier name is stable | + +### 7.4 Degradation Review Gate + +Security degradations block tier registry activation. The blocking gate requires: + +1. Each `SECURITY_DEGRADATION` item is presented to a reviewer at `verified` tier or above +2. The reviewer records an explicit acceptance decision for each degradation via the Admin API +3. The acceptance includes a reason and is written to the audit trail +4. Only after all degradations are accepted does the tier registry change activate + +This is the same pattern as the standard approval pipeline โ€” DCM provides the gate; the organization provides the review process. The difference is that the required tier for the degradation review is always at least `verified`, regardless of the profile in use. + +``` +POST /admin/api/v1/tier-registry/{change_uuid}/accept-degradation + +{ + "affected_item_uuid": "", + "affected_item_type": "provider_registration_requirement", + "degradation_classification": "SECURITY_DEGRADATION", + "acceptance_reason": "", + "accepted_by": "" # must be verified-tier or above reviewer +} +``` + +Broken references cannot be accepted โ€” they must be resolved. DCM will not activate a tier registry change that leaves unresolvable tier references. + +### 7.5 Impact Report + +Whether or not the change requires a degradation review gate, DCM generates a tier registry impact report at proposed time and again at activation time: + +```yaml +tier_registry_impact_report: + registry_change_uuid: + report_generated_at: + stage: proposed | accepted | activated + + summary: + degradations: 0 + upgrades: 3 + new_tiers: 1 + broken_references: 0 + profile_gaps: 2 + stale_weight_records: 4 + + degradations: [] + + upgrades: + - affected_item_uuid: + affected_item_type: provider_registration_requirement + tier_name: verified + old_weight: 3 + new_weight: 4 + old_gravity: elevated + new_gravity: elevated + impact: "Effective authority requirement is higher โ€” more scrutiny now required" + + profile_gaps: + - profile: standard + missing_tiers: [compliance_reviewed] + gap_effect: > + Requests scoring between the verified and authorized thresholds will route + to verified tier until the profile threshold list is updated to include compliance_reviewed + + notification_targets: + - platform_admin + - provider_owners # for SECURITY_DEGRADATION items + - affected_actor_groups # DCMGroup members for authorized-tier items +``` + +The impact report is stored in the Audit Store and is linked to the tier registry version. Platform admins can query historical impact reports to understand what changed and when. + +### 7.6 Audit Trail Requirements + +Every tier registry change produces the following audit records, regardless of whether degradations exist: + +- Registry change proposal record (who proposed, what changed, when) +- Tier impact diff record (all tier changes, all affected items, all classifications) +- Per-degradation acceptance records (if any degradations exist) +- Registry activation record (actual effective timestamp) +- Per-affected-item notification records (who was notified, when) + +Historical approval records retain their `stored_tier_weight` from time of creation (ATM-008). The audit trail thus contains both the point-in-time weight (what authority level was required when the decision was made) and the current weight (what authority level the same tier name requires today), enabling auditors to identify decisions made under different governance regimes. + +### 7.7 System Policy Additions + +| Policy | Rule | +|--------|------| +| `ATM-009` | A tier registry change that produces one or more `SECURITY_DEGRADATION` items must not activate until each degradation is explicitly accepted by a reviewer at `verified` tier or above. | +| `ATM-010` | A tier registry change that produces one or more `BROKEN_REFERENCE` items must not activate. Broken references must be resolved before the change can proceed. | +| `ATM-011` | Every tier registry change must produce a tier impact report. The report is stored in the Audit Store and linked to the registry version. | +| `ATM-012` | Profile threshold lists that become incomplete after a tier registry change (PROFILE_GAP) generate a warning notification to platform admins. The change may activate; platform admins must update threshold lists or explicitly acknowledge the gap within the profile's approval window. | + +## 6. System Policies + +| Policy | Rule | +|--------|------| +| `ATM-001` | Authority tiers are identified by name, not numeric weight. Numeric weight is resolved from list position at evaluation time and is never hardcoded in configuration. | +| `ATM-002` | The `auto` tier's `max_score` threshold may never exceed 50 in any profile (SMX-008 translated to dynamic model). | +| `ATM-003` | Custom tiers must declare `decision_gravity` consistent with their position in the ordered list. A tier with lower gravity may not be inserted after a tier with higher gravity. | +| `ATM-004` | Custom tier contributions require `verified` tier approval. Organizations cannot add tiers unilaterally. | +| `ATM-005` | Custom tiers cannot alter the `dcm_gate` semantics of existing DCM system tiers (`auto`, `reviewed`, `verified`, `authorized`). | +| `ATM-006` | For tiers with `dcmgroup_required: true`, the DCMGroup and quorum threshold must be declared in the profile configuration before the tier can be used as a routing target. | +| `ATM-007` | The four default `decision_gravity` values (`none`, `routine`, `elevated`, `critical`) are DCM system vocabulary. New gravity values require a DCM-level change, not an organization-level contribution. | +| `ATM-008` | Approval records store the tier name and the resolved weight at creation time. If the tier list changes after an approval record is created, the stored weight reflects the state at creation (point-in-time audit). | +| `ATM-009` | A tier registry change that produces one or more `SECURITY_DEGRADATION` items must not activate until each degradation is explicitly accepted by a reviewer at `verified` tier or above. | +| `ATM-010` | A tier registry change that produces one or more `BROKEN_REFERENCE` items must not activate. Broken references must be resolved before the change can proceed. | +| `ATM-011` | Every tier registry change must produce a tier impact report stored in the Audit Store and linked to the registry version. | +| `ATM-012` | `PROFILE_GAP` conditions generate a warning notification to platform admins. The change may activate; admins must update threshold lists or explicitly acknowledge the gap within the profile's approval window. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/credential-provider-model.md b/content/docs/architecture/data-model/credential-provider-model.md new file mode 100644 index 0000000..7fd8f15 --- /dev/null +++ b/content/docs/architecture/data-model/credential-provider-model.md @@ -0,0 +1,918 @@ +--- +title: "Credential Provider Model" +type: docs +weight: 31 +--- + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. This document specifies the Credential Provider model โ€” the issuance contract, credential lifecycle, rotation model, revocation propagation, and how credentials flow through the DCM pipeline. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” Credential Provider Specification +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Auth Providers](19-auth-providers.md) | [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Scoring Model](29-scoring-model.md) | [Federated Contribution Model](28-federated-contribution-model.md) + +> **This document maps to: DATA + PROVIDER** +> +> Credentials are Data artifacts with UUID, type, lifecycle state, and provenance. The Credential Provider is a typed Provider with a defined capability extension. The Zero Trust model in [doc 26](26-accreditation-and-authorization-matrix.md) governs credential scope and lifetime โ€” this document specifies how that model is implemented. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) + +--- + +## 1. Purpose and Scope + +### 1.1 What the Credential Provider Does + +The Credential Provider is a typed DCM Provider responsible for issuing, rotating, and revoking credentials used within the DCM ecosystem. There are two categories of credential it manages: + +**1. DCM interaction credentials** โ€” short-lived, scoped credentials that DCM components and providers use to authenticate interactions. These implement the Zero Trust credential model from [doc 26](26-accreditation-and-authorization-matrix.md) Section 5.3. They are issued by DCM's Credential Provider and consumed entirely within the DCM control plane and its providers. + +**2. Consumer-facing credentials** โ€” credentials delivered to consumers as part of a realized service (kubeconfigs, database passwords, API keys, SSH keys, service account tokens). These are issued by the Credential Provider on behalf of a realized resource entity and delivered via the Consumer API. + +### 1.2 What the Credential Provider Does Not Do + +- It does not manage actor session tokens โ€” that is the Auth Provider's responsibility +- It does not store secrets in DCM's data model โ€” credential values are never written to the GitOps stores or Realized State Store; only credential metadata (UUID, type, scope, expiry, status) is stored +- It does not make authorization decisions โ€” authorization is the Governance Matrix's responsibility; the Credential Provider only issues if DCM has already authorized the operation +- It does not replace secrets management for consumers' own applications โ€” it manages credentials that DCM issues for DCM-managed resources + +--- + +## 2. Credential Types + +| Credential Type | Use Case | Typical Lifetime | Rotation Trigger | +|----------------|----------|-----------------|-----------------| +| `dcm_interaction` | DCM-internal component-to-provider auth | PT15Mโ€“PT1H (profile-governed) | Automatic; pre-expiry | +| `api_key` | Programmatic consumer access to a realized resource | PT24Hโ€“P30D (configurable) | Scheduled or event-triggered | +| `x509_certificate` | mTLS identity for providers and DCM components | P30Dโ€“P365D | P14D before expiry | +| `ssh_key` | SSH access to realized VMs or infrastructure | P30Dโ€“P90D (configurable) | Scheduled or on-demand | +| `service_account_token` | Workload identity for automated processes | PT1Hโ€“PT24H | Automatic; pre-expiry | +| `database_password` | Access credential for realized database resources | PT24Hโ€“P7D (configurable) | Scheduled or on-demand | +| `kubeconfig` | Access to realized Kubernetes clusters | PT8Hโ€“P30D (configurable) | Scheduled or on-demand | +| `hsm_backed_key` | Sovereign/FSI deployments requiring hardware attestation | P30Dโ€“P365D | P14D before expiry; HSM-managed | + +--- + +## 3. Credential Data Model + +A credential is a DCM Data artifact. Credential metadata is stored in DCM; credential values are held only by the Credential Provider (never in DCM stores). + +```yaml +credential_record: + credential_uuid: + credential_type: api_key | x509_certificate | ssh_key | service_account_token | + database_password | kubeconfig | hsm_backed_key | dcm_interaction + + # Lifecycle + status: active | rotating | revoked | expired + issued_at: + valid_until: + last_rotated_at: + revoked_at: + revocation_reason: + + # Scope โ€” what this credential authorizes + issued_to: + actor_uuid: # consumer credential: issued to an actor + entity_uuid: # resource credential: scoped to an entity + component_uuid: # interaction credential: issued to a DCM component + provider_uuid: # interaction credential: scoped to a provider + scope: + operations: [dispatch, discover, query, read, write, admin] # allowed operations + resource_types: [Compute.VirtualMachine] # scoped resource types + tenant_uuid: # Tenant scope + non_transferable: true # always true for DCM-issued credentials + bound_to_ip: # optional; enforced in fsi/sovereign profiles + + # Provenance + credential_provider_uuid: + issuing_request_uuid: # which DCM request triggered issuance + entity_uuid: # the realized entity this credential accesses + rotation_of: # parent credential UUID if this is a rotation + + # Storage (values never in DCM) + value_held_by: + value_retrieval_endpoint: # how the authorized consumer retrieves the value + value_retrieval_auth: bearer_token | mtls | step_up_mfa + + # Cryptographic metadata (Sections 13) + algorithm: Ed25519 | ECDSA-P-384 | RSA-4096 | HS256 | RS256 | random_256bit + key_usage: [authentication] # authentication | signing | encryption; declared at issuance + retrieved_count_threshold: 48 # hours; idle alert fires if not retrieved within this window +``` + +### 3.1 Credential Value Separation + +Credential values are never stored in DCM's data model, GitOps stores, or Realized State Store. DCM stores only the credential metadata record. The credential value is held exclusively by the Credential Provider. + +Authorized consumers retrieve the credential value via `value_retrieval_endpoint` using `value_retrieval_auth`. This retrieval is itself authenticated โ€” typically with a short-lived bearer token or mTLS โ€” and is audited. + +--- + +## 4. Issuance Flows + +### 4.1 Resource Credential Issuance (consumer-facing) + +Credentials issued as part of resource realization flow through the standard provider dispatch pipeline. + +``` +Consumer requests resource (e.g., Compute.VirtualMachine) + โ”‚ + โ–ผ Layer assembly + policy evaluation + โ”‚ Transformation policy may inject credential requirements: + โ”‚ fields.credential_requirements: + โ”‚ - credential_type: ssh_key + โ”‚ issued_to: requesting_actor + โ”‚ scope: [ssh_access] + โ”‚ + โ–ผ Placement selects Service Provider for the VM + โ”‚ + โ–ผ After VM realization: Credential Provider dispatched + โ”‚ DCM issues sub-request to Credential Provider: + โ”‚ entity_uuid: + โ”‚ credential_type: ssh_key + โ”‚ issued_to.actor_uuid: + โ”‚ scope.operations: [ssh_access] + โ”‚ scope.resource_types: [Compute.VirtualMachine] + โ”‚ valid_until: + โ”‚ + โ–ผ Credential Provider issues credential; returns credential_record + โ”‚ (value held by provider; metadata returned to DCM) + โ”‚ + โ–ผ DCM writes credential_record to Realized State + โ”‚ Links credential_uuid to entity_uuid + โ”‚ + โ–ผ Consumer receives realized entity + credential_record metadata + โ”‚ Consumer calls value_retrieval_endpoint to get actual credential + โ”‚ (step-up MFA may be required per profile) +``` + +### 4.2 DCM Interaction Credential Issuance + +DCM interaction credentials are issued automatically before each provider interaction. They implement the Zero Trust scoped credential model (ZTS-002). + +``` +DCM prepares to dispatch to a provider + โ”‚ + โ–ผ Request interaction credential from Credential Provider: + โ”‚ credential_type: dcm_interaction + โ”‚ issued_to.component_uuid: + โ”‚ issued_to.provider_uuid: + โ”‚ scope.operations: [dispatch] + โ”‚ scope.resource_types: [Compute.VirtualMachine] + โ”‚ entity_uuid: + โ”‚ valid_until: (max; profile-governed) + โ”‚ + โ–ผ Credential Provider issues scoped interaction credential + โ”‚ + โ–ผ DCM includes credential in provider dispatch + โ”‚ Provider validates credential scope before executing + โ”‚ + โ–ผ Credential expires after PT15M regardless of use + โ”‚ (no renewal; new credential issued for next interaction) +``` + +### 4.3 Bootstrap Credential Issuance + +During bootstrap, before the Credential Provider is registered, DCM uses a bootstrap credential mechanism. See [Deployment and Redundancy](17-deployment-redundancy.md) BOOT-003 for the bootstrap credential model. After bootstrap, all credentials are issued through a registered Credential Provider. + +--- + +## 5. Credential Rotation Model + +Rotation is the primary mechanism for maintaining credential hygiene. DCM distinguishes scheduled rotation, pre-expiry rotation, and event-triggered rotation. + +### 5.1 Rotation Triggers + +| Trigger | Description | Default behavior | +|---------|-------------|-----------------| +| `scheduled` | Regular rotation on a declared schedule | Most credential types; interval is credential-type specific | +| `pre_expiry` | Rotation initiated before the current credential expires | x509: P14D before expiry; ssh_key: P7D; dcm_interaction: PT5M | +| `provider_initiated` | Credential Provider notifies DCM of a rotation requirement | Handled via provider update notification model | +| `security_event` | Rotation triggered by a security signal (compromise, anomaly, policy change) | Immediate; see Section 5.4 | +| `actor_request` | Consumer requests rotation of their own credential | Subject to rate limiting and policy | + +### 5.2 Rotation Protocol + +Rotation uses a transition window to prevent downtime. The old credential remains valid during the transition window; the new credential is issued and delivered before the old one expires. + +``` +Rotation initiated (by any trigger): + โ”‚ + โ–ผ DCM requests new credential from Credential Provider + โ”‚ rotation_of: + โ”‚ same scope as original; new valid_until + โ”‚ + โ–ผ Credential Provider issues new credential + โ”‚ Returns new credential_record + โ”‚ Old credential NOT yet revoked + โ”‚ + โ–ผ New credential delivered to authorized consumer/component + โ”‚ (same delivery mechanism as initial issuance) + โ”‚ + โ–ผ Transition window: both credentials valid + โ”‚ Window duration: P1D for consumer credentials (default) + โ”‚ PT5M for dcm_interaction credentials + โ”‚ P7D for x509_certificate credentials + โ”‚ Configurable per credential type in Credential Provider registration + โ”‚ + โ–ผ Old credential revoked at end of transition window + โ”‚ Revocation propagated to all registered consumers + โ”‚ + โ–ผ Rotation record written to audit trail + old_credential_uuid, new_credential_uuid, rotation_trigger, rotation_at +``` + +### 5.3 Rotation Notification + +Before the old credential is revoked, DCM sends a rotation notification to any entity or actor whose credential is rotating: + +```yaml +rotation_notification: + event_type: credential.rotating + credential_uuid: + new_credential_uuid: + transition_window_ends: + retrieval_url: + action_required: "Retrieve new credential before transition window ends" +``` + +### 5.4 Emergency Rotation (Security Event) + +On detection of a compromise or security event, DCM triggers emergency rotation: + +- No transition window โ€” old credential revoked immediately +- New credential issued and delivered via the fastest available Notification Provider channel +- Security event record written to Audit Store with full context +- Compliance-class GateKeeper firing for this entity type audited against the event +- Platform admin notified regardless of profile + +``` +Triggers for emergency rotation: + security.credential_compromised # DCM or provider reports compromise + security.anomalous_usage_detected # unusual access pattern detected + actor.deprovisioned # actor removed; all their credentials revoked + provider.deregistered # provider leaving; all its interaction creds revoked + accreditation.revoked # provider accreditation revoked; creds reassessed +``` + +--- + +## 6. Revocation Model + +Revocation makes a credential permanently invalid before its natural expiry. Unlike rotation (which maintains continuity), revocation is an immediate termination. + +### 6.1 Revocation Triggers + +| Trigger | Initiator | Behavior | +|---------|-----------|----------| +| `actor_deprovisioned` | SCIM / Auth Provider | All credentials issued to the actor revoked immediately | +| `entity_decommissioned` | DCM lifecycle | All credentials scoped to the entity revoked | +| `security_event` | Platform admin or security automation | Immediate; no transition window | +| `provider_deregistered` | Platform admin | All interaction credentials for the provider revoked | +| `actor_request` | Consumer | Consumer may revoke their own credentials | +| `ttl_expired` | Lifecycle Constraint Enforcer | Credential expired; revocation recorded | + +### 6.2 Revocation Propagation + +DCM maintains a **Credential Revocation Registry** โ€” a fast-queryable store of revoked credential UUIDs. All components that receive DCM interaction credentials must check this registry at each use (not just at issuance time). + +``` +Credential revoked: + โ”‚ + โ–ผ Credential record status: active โ†’ revoked + โ”‚ revoked_at, revocation_reason written + โ”‚ + โ–ผ Revocation event published to Message Bus + โ”‚ event_type: credential.revoked + โ”‚ credential_uuid: + โ”‚ effective_at: + โ”‚ + โ–ผ All subscribed components update local revocation cache + โ”‚ (cache TTL: PT1M standard; PT30S fsi/sovereign) + โ”‚ + โ–ผ Credential Provider notified to invalidate stored value + โ”‚ Provider must honor revocation within declared SLA: + โ”‚ standard/prod: PT5M + โ”‚ fsi/sovereign: PT1M + โ”‚ + โ–ผ Audit record written + credential_uuid, revocation_trigger, revoked_by_actor, entity_uuid +``` + +### 6.3 Revocation Check at Use + +Providers receiving DCM interaction credentials must validate the credential at use time, not only at receipt time: + +1. Verify credential signature (if signed) +2. Check credential UUID against local revocation cache +3. Verify credential has not expired (`valid_until`) +4. Verify operation is within credential scope +5. Verify IP binding if `bound_to_ip` is set + +A credential that passes issuance validation but fails use-time validation is rejected. The provider must return `403 Forbidden` with `credential_revoked` or `credential_expired` error code. + +--- + +## 7. Consumer Credential Delivery + +### 7.1 How Consumers Retrieve Credentials + +After a resource is realized with an associated credential, the consumer receives the `credential_record` metadata in the realized entity response. The actual credential value is retrieved separately via `value_retrieval_endpoint`. + +``` +GET /api/v1/resources/{entity_uuid}/credentials + +Response 200: +{ + "credentials": [ + { + "credential_uuid": "", + "credential_type": "ssh_key", + "status": "active", + "issued_at": "", + "valid_until": "", + "scope": { + "operations": ["ssh_access"], + "entity_uuid": "" + }, + "retrieval": { + "endpoint": "/api/v1/credentials//value", + "auth_required": "step_up_mfa", # none | bearer_token | step_up_mfa | mtls + "retrieval_count": 1, # how many times value has been retrieved + "last_retrieved_at": "" + }, + "rotation_schedule": { + "next_rotation_at": "", + "rotation_trigger": "scheduled", + "transition_window_days": 1 + } + } + ] +} +``` + +### 7.2 Credential Value Retrieval + +``` +GET /api/v1/credentials/{credential_uuid}/value +Authorization: Bearer +X-DCM-StepUp-Token: # if auth_required: step_up_mfa + +Response 200: +{ + "credential_uuid": "", + "credential_type": "ssh_key", + "value": { + "private_key": "-----BEGIN OPENSSH PRIVATE KEY-----\n...", + "public_key": "ssh-ed25519 AAAA... dcm-issued@entity-", + "username": "dcm-provisioned" + }, + "valid_until": "", + "retrieval_uuid": "" # idempotency key for this retrieval event; audited +} + +Response 404: credential_uuid not found or not associated with an entity the actor owns +Response 403: step_up_mfa required but not completed +Response 410: credential revoked or expired +``` + +Every value retrieval is audited: credential_uuid, actor_uuid, retrieved_at, retrieval_uuid. + +--- + +## 8. Credential Provider API Contract + +The full endpoint contract that all Credential Providers must implement. + +### 8.1 Issue Credential + +``` +POST {issue_endpoint} + +Request: +{ + "credential_type": "ssh_key", + "issued_to": { + "actor_uuid": "", + "entity_uuid": "", + "component_uuid": "", + "provider_uuid": "" + }, + "scope": { + "operations": ["ssh_access"], + "resource_types": ["Compute.VirtualMachine"], + "tenant_uuid": "" + }, + "valid_until": "", + "non_transferable": true, + "bound_to_ip": "", + "rotation_of": "", + "issuing_request_uuid": "", + "entity_uuid": "" +} + +Response 201 Created: +{ + "credential_uuid": "", + "credential_type": "ssh_key", + "issued_at": "", + "valid_until": "", + "value_retrieval_endpoint": "", + "value_retrieval_auth": "step_up_mfa", + "metadata": {} # provider-specific additional metadata +} + +Response 422: unsupported credential type +Response 403: issued_to scope exceeds provider's declared authority +``` + +### 8.2 Rotate Credential + +``` +POST {rotate_endpoint} + +Request: +{ + "credential_uuid": "", # credential being rotated + "rotation_trigger": "pre_expiry | scheduled | security_event | actor_request", + "transition_window": "P1D", # how long old credential remains valid + "new_valid_until": "" +} + +Response 200: +{ + "old_credential_uuid": "", + "new_credential_uuid": "", + "new_valid_until": "", + "old_credential_revokes_at": "", # end of transition window + "new_value_retrieval_endpoint": "" +} +``` + +### 8.3 Revoke Credential + +``` +DELETE {revoke_endpoint}/{credential_uuid} + +Request body: +{ + "revocation_trigger": "actor_deprovisioned | entity_decommissioned | security_event | ...", + "revocation_reason": "", + "effective_immediately": true # false = honor transition window if rotating +} + +Response 200: +{ + "credential_uuid": "", + "revoked_at": "", + "effective_immediately": true +} + +Response 404: credential not found +Response 409: credential already revoked +``` + +### 8.4 Validate Credential (Use-Time Check) + +``` +POST {validate_endpoint} + +Request: +{ + "credential_uuid": "", + "operation_type": "dispatch", + "entity_uuid": "", + "provider_uuid": "" +} + +Response 200: +{ + "valid": true, + "expires_in_seconds": 423 +} + +Response 200 (invalid): +{ + "valid": false, + "reason": "revoked | expired | scope_mismatch | ip_binding_failed" +} +``` + +### 8.5 List Credentials for Entity + +``` +GET {list_endpoint}?entity_uuid=&status=active + +Response 200: +{ + "credentials": [ + { + "credential_uuid": "", + "credential_type": "ssh_key", + "status": "active", + "issued_to": {...}, + "valid_until": "" + } + ] +} +``` + +--- + +## 9. Credential Provider Registration + +```yaml +credential_provider_capabilities: + # Credential types this provider can issue + credential_types: + - api_key + - x509_certificate + - ssh_key + - service_account_token + - database_password + - kubeconfig + - hsm_backed_key + - dcm_interaction # must declare if provider handles DCM interaction creds + + # Secret engine backing (for audit and accreditation) + secret_engines: + - vault # HashiCorp Vault + - aws_secrets_manager + - azure_key_vault + - gcp_secret_manager + - local_hsm # sovereign deployments + + # Security properties + hsm_backed: false # true if all keys are HSM-protected + fips_140_2_level: 0 # 0=none, 1, 2, or 3 + dynamic_secrets: true # can generate credentials on demand (not just store/retrieve) + + # Rotation capabilities + rotation_support: true + min_transition_window: PT5M + max_transition_window: P7D + supported_rotation_triggers: + - pre_expiry + - scheduled + - security_event + - actor_request + + # Revocation SLA (how quickly revocations take effect) + revocation_sla: PT5M # standard; PT1M for fsi/sovereign + + # Endpoints (all relative to provider base URL) + endpoints: + issue: /v1/credentials + rotate: /v1/credentials/rotate + revoke: /v1/credentials/{uuid} + validate: /v1/credentials/validate + list: /v1/credentials +``` + +--- + +## 10. Credential Lifecycle State Machine + +``` + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + issuance โ”‚ โ”‚ expiry / explicit + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บโ”‚ ACTIVE โ”‚โ”€โ”€โ”€โ”€revocationโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บ REVOKED / EXPIRED + โ”‚ โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ rotation initiated + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ ROTATING โ”‚ both old and new valid + โ”‚ โ”‚ during transition window + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ transition window ends + โ”‚ or emergency revocation + โ–ผ + REVOKED +``` + +State transitions and their audit requirements: + +| Transition | Audited fields | +|-----------|---------------| +| issued โ†’ ACTIVE | credential_uuid, type, issued_to, scope, valid_until, issuing_request_uuid | +| ACTIVE โ†’ ROTATING | rotation_trigger, old_uuid, new_uuid, transition_window | +| ROTATING โ†’ ACTIVE (new) | new credential activated after old revoked | +| ACTIVE/ROTATING โ†’ REVOKED | revocation_trigger, revoked_by, effective_at, reason | +| ACTIVE โ†’ EXPIRED | expired_at (system record, no actor) | + +--- + +--- + +## 12. Profile-Governed Credential Configuration + +Every credential security dimension is controlled by the active profile. This is the single authoritative configuration point โ€” a homelab deployment requires minimal configuration and implementation effort; an FSI or sovereign deployment gets full enforcement without per-deployment policy authoring. + +### 12.1 Credential Profile Configuration Block + +```yaml +credential_profile: + + # --- Credential Type Restrictions --- + # Which credential types are permitted. Omitted types are rejected at issuance. + permitted_credential_types: + minimal: [api_key, x509_certificate, ssh_key, service_account_token, database_password] + dev: [api_key, x509_certificate, ssh_key, service_account_token, database_password, kubeconfig] + standard: [api_key, x509_certificate, ssh_key, service_account_token, database_password, kubeconfig] + prod: [api_key, x509_certificate, ssh_key, service_account_token, database_password, kubeconfig] + fsi: [x509_certificate, ssh_key, service_account_token, database_password, kubeconfig, hsm_backed_key] + sovereign: [x509_certificate, hsm_backed_key] # all credentials must be hardware-backed + + # --- Lifetime Limits --- + # Maximum valid_until per credential type. Provider may issue shorter; never longer. + max_lifetime: + # minimal dev standard prod fsi sovereign + api_key: [P365D, P90D, P90D, P30D, โ€”, โ€”] + x509_certificate:[P365D, P365D, P365D, P180D, P90D, P90D] + ssh_key: [P365D, P90D, P90D, P30D, P30D, P30D] + service_account_token: [PT24H, PT24H, PT24H, PT12H, PT4H, PT1H] + database_password: [P365D, P90D, P90D, P30D, P30D, โ€”] + kubeconfig: [P365D, P30D, P30D, P14D, P7D, โ€”] + dcm_interaction:[PT1H, PT30M, PT1H, PT30M, PT15M, PT15M] + hsm_backed_key: [โ€”, โ€”, โ€”, P365D, P180D, P90D] + + # --- Rotation --- + max_rotation_interval: # PCI DSS req 8.3.9: 90-day maximum for regulated profiles + standard: P365D # no enforcement; provider may choose longer + prod: P90D # enforced; rotation older than P90D triggers alert + fsi: P90D # enforced; PCI DSS compliance + sovereign: P90D # enforced + scheduled_rotation_required: + # Security-first: rotation is architecturally required in ALL profiles. + # What varies is the maximum interval, automation level, and trigger mechanism. + minimal: true # required; manual trigger acceptable; P365D max interval + dev: true # required; manual trigger acceptable; P180D max interval + standard: true # required; automated pre-expiry rotation + prod: true # required; automated; strict interval enforcement + fsi: true # required; automated; P90D max (PCI DSS) + sovereign: true # required; automated; hardware-triggered rotation + min_transition_window: + minimal: PT0S # homelab: immediate cutover acceptable + dev: PT1H + standard: P1D + prod: P1D + fsi: P1D # PT15M for dcm_interaction + sovereign: P1D # PT15M for dcm_interaction + + # --- Value Retrieval Security --- + value_retrieval_auth_required: + minimal: bearer_token # session token sufficient for homelab + dev: bearer_token + standard: bearer_token # step_up_mfa for sensitive types (ssh_key, database_password) + prod: step_up_mfa # all credential types require step-up + fsi: step_up_mfa # hardware token MFA required + sovereign: mtls # mutual TLS + hardware attestation + step_up_sensitive_types: # standard profile: step_up_mfa for these types even without full profile enforcement + - ssh_key + - database_password + - kubeconfig + - hsm_backed_key + + # --- Retrieval Audit --- + audit_every_retrieval: + # Security-first: FIRST retrieval is always audited in ALL profiles (CPX-005). + # audit_every_retrieval controls whether SUBSEQUENT retrievals are also audited. + # audit_first_retrieval is always true regardless of this setting. + minimal: false # subsequent retrievals silent; first always audited + dev: false # subsequent retrievals silent; first always audited + standard: true # every retrieval audited + prod: true + fsi: true + sovereign: true + idle_detection_threshold: # alert if credential not retrieved within N after issuance + # Security-first: idle detection is on in ALL profiles. Threshold varies. + # Alert is notification-only; never blocks. No operational burden. + minimal: P30D # generous; homelab credentials may sit unused longer + dev: P14D + standard: P7D + prod: P3D + fsi: P1D + sovereign: PT12H + + # --- Network Binding --- + ip_binding_required: + minimal: false + dev: false + standard: false # optional; recommended for prod + prod: false # optional; recommended + fsi: true # mandatory + sovereign: true # mandatory + + # --- Cryptographic Requirements --- + fips_140_level_required: + minimal: 0 # no requirement + dev: 0 + standard: 0 + prod: 1 # Level 1: software-only acceptable + fsi: 2 # Level 2: role-based authentication required + sovereign: 3 # Level 3: physical tamper evidence + identity-based auth + approved_algorithms: + minimal: # negative list: anything not forbidden is permitted + forbidden_algorithms: [MD5, SHA-1, DES, 3DES, RC4, RSA-1024, RSA-512, DSA-1024] + # No weak/broken algorithms even in homelab. Real attacks hit all deployments. + standard: + api_key: [random_256bit] + x509_certificate: [RSA-4096, ECDSA-P-384, Ed25519] + ssh_key: [Ed25519, ECDSA-P-384] + service_account_token:[HS256, RS256, ES256] + database_password: [random_128bit_printable] + prod: # same as standard; provider must declare algorithm in credential record + inherits: standard + fsi: + x509_certificate: [RSA-4096, ECDSA-P-384] # Ed25519 not FIPS-approved in 140-2 + ssh_key: [RSA-4096, ECDSA-P-384] + service_account_token:[RS256, ES256] + database_password: [random_256bit] + sovereign: + inherits: fsi + all_types: hsm_backed_only # all keys generated and stored in HSM + + # --- Revocation --- + revocation_check_frequency: # how often components must refresh revocation cache + minimal: PT5M # lazy; acceptable for homelab + dev: PT5M + standard: PT1M + prod: PT1M + fsi: PT30S + sovereign: PT15S + revocation_sla: # how quickly Credential Provider must invalidate on revocation + minimal: PT10M + dev: PT5M + standard: PT5M + prod: PT2M + fsi: PT1M + sovereign: PT30S +``` + +### 12.2 Authenticator Assurance Levels (NIST 800-63B Mapping) + +DCM profile credential requirements map to NIST 800-63B Authenticator Assurance Levels: + +| Profile | AAL | What it means | +|---------|-----|--------------| +| `minimal` | AAL1 | Single-factor; bearer token sufficient for credential retrieval | +| `dev` | AAL1 | Same as minimal; shorter lifetimes | +| `standard` | AAL2 | MFA required for sensitive credential retrieval (ssh_key, database_password, kubeconfig) | +| `prod` | AAL2 | MFA required for all credential retrieval | +| `fsi` | AAL2+ | Hardware MFA token required; FIPS 140-2 Level 2 modules | +| `sovereign` | AAL3 | Hardware-bound authenticator; FIPS 140-2 Level 3; physical tamper evidence | + +### 12.3 Compliance Domain Overlays + +When a compliance domain is active, its credential requirements are **additive** to the profile base: + +```yaml +compliance_credential_overlays: + hipaa: + min_key_size_bits: 256 + max_lifetime_override: + api_key: P90D # HIPAA requires rotation at least annually; 90-day recommended + audit_every_retrieval: true # all PHI-adjacent credential access audited + idle_detection_threshold: P7D + + pci_dss: + max_rotation_interval: P90D # PCI DSS req 8.3.9 โ€” mandatory + min_password_complexity: + database_password: + length: 12 + character_classes: 4 # upper, lower, digit, special + idle_detection_threshold: P30D + + fedramp_moderate: + fips_140_level_required: 1 + approved_algorithms: + inherits: standard + + fedramp_high: + fips_140_level_required: 2 + approved_algorithms: + inherits: fsi + ip_binding_required: true + + dod_il4: + fips_140_level_required: 2 + ip_binding_required: true + max_lifetime_override: + dcm_interaction: PT10M + service_account_token: PT1H +``` + +### 12.4 Design Priority and Implementation Consistency Principle + +The DCM design priority order applies directly to credential management: + +1. **Security first:** Security properties โ€” value separation, rotation, audit, idle detection, algorithm baselines, revocation โ€” are architecturally present in ALL profiles. What profiles control is enforcement strictness, threshold values, and automation level. A `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security." + +2. **Ease of use second:** The secure path must be the easy path. Homelab deployments use the same API contract, same data model, and same provider interface as sovereign deployments. The profile system eliminates the need to choose between security and operational simplicity. + +3. **Extensibility third:** Compliance domain overlays, profile overrides, and algorithm configuration make the credential model adaptable without code changes. + +Profile variation applies only to **enforcement level and required features** โ€” never to the underlying protocol or data model. A credential issued under the `minimal` profile has the same data structure, the same API contract, the same revocation mechanism, and the same audit record format as one issued under the `sovereign` profile. What differs is what is required vs optional. + +This means: +- A Credential Provider built for a homelab deployment is compatible with a production deployment โ€” it just needs to demonstrate it satisfies the production profile's requirements +- Migration from `dev` to `prod` profile does not require replacing the Credential Provider or re-issuing credentials under a different protocol โ€” it triggers more conservative enforcement of the same model +- Testing and tooling built against the `dev` profile works against `sovereign` profile with the same interfaces + +**CPX-001 (values never in DCM stores) is non-negotiable in every profile including `minimal`.** This is the one property that does not scale down. It is the security property that makes the entire model trustworthy regardless of deployment size. + +--- + +## 13. Cryptographic Algorithm Requirements + +### 13.1 Algorithm Declaration in Credential Record + +The credential record is extended with two new fields: + +```yaml +credential_record: + # ... existing fields ... + algorithm: Ed25519 | ECDSA-P-384 | RSA-4096 | HS256 | RS256 | random_256bit | ... + key_usage: [authentication, signing, encryption] # declared at issuance; non-overlapping + retrieved_count_threshold: 48 # hours after issuance before idle alert fires +``` + +`key_usage` enforces the principle of algorithm agility and purpose separation. A credential issued for `authentication` cannot be used for `signing` even if the underlying algorithm supports both. The Credential Provider must validate key_usage at the validate endpoint. + +### 13.2 Approved Algorithm Defaults (Standard Profile) + +| Credential Type | Algorithm | Key Size | +|----------------|-----------|----------| +| `api_key` | Cryptographically random | 256 bits minimum | +| `x509_certificate` | Ed25519 or ECDSA P-384 | Ed25519: 256-bit; P-384: 384-bit | +| `ssh_key` | Ed25519 (preferred), ECDSA P-384 | Ed25519: 256-bit | +| `service_account_token` | RS256 or ES256 | RSA: 4096-bit; EC: P-256 | +| `database_password` | Cryptographically random | 128-bit printable minimum | +| `kubeconfig` | As per cluster's auth configuration | โ€” | +| `hsm_backed_key` | ECDSA P-384 or RSA-4096 | HSM-generated | +| `dcm_interaction` | HS256 or ES256 | AES-256 or P-256 | + +### 13.3 Key Escrow Policy + +DCM does not implement key escrow by default. For `sovereign` profile deployments, key escrow (if required by regulation) is declared in the Credential Provider's capability registration and governed by the provider โ€” DCM's role is to audit that escrowed credentials are disclosed only via the standard revocation and access model: + +```yaml +credential_provider_capabilities: + key_escrow: + supported: false # default; no escrow + # If true: + escrow_model: m_of_n # m-of-n key shares; Shamir's Secret Sharing + escrow_quorum: "3 of 5" + escrow_record_stored_by: hsm # never by DCM + dcm_role: audit_only # DCM audits escrow access; does not participate +``` + +--- + +## 14. Idle Credential Detection + +A credential issued but never retrieved within the declared threshold is a security signal โ€” it may indicate a provisioning error, a failed delivery, or an abandoned resource. + +```yaml +idle_credential_record: + credential_uuid: + issued_at: + threshold_hours: 48 # from profile credential_profile.idle_detection_threshold + last_checked_at: + retrieval_count: 0 + status: idle_alert_pending +``` + +When an idle alert fires: +- Platform admin notified: "Credential {uuid} for entity {entity_uuid} has not been retrieved in {N} hours" +- Consumer notified (if consumer exists): "Your credential for {resource_name} has not been accessed โ€” confirm delivery" +- Credential is NOT automatically revoked โ€” it remains valid until its `valid_until` +- If still idle after 2ร— the threshold: optional auto-revocation per profile configuration + +--- + + +## 11. System Policies + +| Policy | Rule | +|--------|------| +| `CPX-001` | Credential values are never stored in DCM's data model, GitOps stores, Realized State Store, or Audit Store. Only credential metadata (UUID, type, scope, expiry, status) is stored in DCM. | +| `CPX-002` | Every DCM interaction with a provider must present a scoped, short-lived `dcm_interaction` credential. A provider that receives an interaction without a valid scoped credential must reject it with `403 Forbidden`. | +| `CPX-003` | Credential revocation must propagate to the Credential Revocation Registry within the declared `revocation_sla`. Components must refresh their revocation cache no less frequently than the profile-governed cache TTL (PT1M standard; PT30S fsi/sovereign). | +| `CPX-004` | Emergency rotation (security_event trigger) has no transition window. The old credential is revoked immediately. The new credential is delivered via the fastest available Notification Provider channel. | +| `CPX-005` | The first credential value retrieval is audited in ALL profiles (credential_uuid, actor_uuid, retrieved_at, retrieval_uuid). Subsequent retrievals are audited in standard+ profiles. Emergency retrievals (rotation, security event) are always audited regardless of profile. | +| `CPX-006` | Actor deprovisioning (via SCIM or manual) triggers immediate revocation of all credentials issued to that actor. Revocation events are published to the Message Bus before the deprovisioning event is acknowledged. | +| `CPX-007` | Entity decommissioning triggers revocation of all credentials scoped to that entity before the decommission is confirmed. A decommission that cannot revoke all credentials enters `COMPENSATION_IN_PROGRESS` state. | +| `CPX-008` | Credentials issued for `fsi` and `sovereign` profiles must be IP-bound (`bound_to_ip`) or hardware-attested (`hsm_backed_key`). Unbound credentials are rejected by the Governance Matrix for these profiles. | +| `CPX-009` | `algorithm` and `key_usage` must be declared on every credential record at issuance (standard+ profiles). The Credential Provider must validate `key_usage` at the validate endpoint โ€” a credential issued for `authentication` cannot be used for `signing`. | +| `CPX-010` | Idle credential detection fires at the profile-governed threshold. Idle credentials are NOT automatically revoked โ€” they trigger notification only. Auto-revocation after 2ร— threshold is profile-configurable. | +| `CPX-011` | Profile credential requirements are additive when compliance domains are active (HIPAA, PCI DSS, FedRAMP, DoD IL4). Compliance overlay requirements always tighten, never relax, the base profile. | +| `CPX-012` | CPX-001 (values never in DCM stores) applies in ALL profiles including `minimal`. There is no profile that permits credential values to be stored in DCM. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/design-priorities.md b/content/docs/architecture/data-model/design-priorities.md new file mode 100644 index 0000000..71246d0 --- /dev/null +++ b/content/docs/architecture/data-model/design-priorities.md @@ -0,0 +1,316 @@ +--- +title: "DCM Design Priorities" +type: docs +weight: -11 +--- + +**Document Status:** โœ… Stable โ€” Foundational reference +**Document Type:** Architecture Reference โ€” Design Philosophy +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Policy Profiles](14-policy-profiles.md) | [Scoring Model](29-scoring-model.md) | [Credential Provider Model](31-credential-provider-model.md) + +> **This document maps to: DATA + PROVIDER + POLICY** +> +> Design priorities govern every decision across all three abstractions. They are not guidelines โ€” they are the decision framework used when priorities conflict. Every contributor, implementer, and reviewer should apply this framework. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) + +--- + +## The Four Priorities + +Every design decision in DCM is evaluated against this hierarchy. When priorities conflict, higher priorities win. When there is no conflict, all four apply simultaneously. + +--- + +### Priority 1 โ€” Industry Best Practices for Security + +Security is not a feature, a profile option, or a compliance checkbox. It is the baseline that every other design decision must respect. + +**What this means:** + +Security properties โ€” value separation, rotation, audit trails, idle detection, algorithm baselines, scoped credentials, revocation propagation, shadow mode evaluation โ€” are **architecturally present in every profile**. What profiles control is enforcement strictness, threshold values, and automation level โ€” not whether the security property applies. + +**The `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security."** + +A `minimal` profile deployment: +- Rotates credentials (at longer intervals with manual triggers acceptable โ€” not never) +- Detects idle credentials (at a generous P30D threshold โ€” not never) +- Requires algorithm baselines (via forbidden list โ€” not null) +- Runs shadow mode on contributed policies (always โ€” not optionally) +- Audits first credential retrieval (always โ€” not sometimes) +- Maintains revocation registry (at PT5M cache TTL โ€” not disabled) + +The security model is present and correct. The enforcement strictness and automation burden are reduced. + +**When security and convenience conflict, security wins** โ€” but the design must find a way to make the secure option easy. A security model that is routinely bypassed because it is too burdensome has failed at both security and usability. The profile system is the mechanism: the right profile makes secure behavior automatic, not effortful. + +**Security properties that are non-negotiable in all profiles:** + +| Property | Rule | Reference | +|----------|------|-----------| +| Credential values never in DCM stores | CPX-001 โ€” absolute, no profile exception | doc 31 | +| Governance Matrix always boolean | SMX-004 โ€” scoring never applies to boundary decisions | doc 29 | +| Every provider dispatch requires scoped interaction credential | CPX-002 | doc 31 | +| Shadow mode on all contributed policies | FCM-004 | doc 28 | +| auto_approve_below โ‰ค 50 in all profiles | SMX-008 | doc 29 | +| First credential retrieval always audited | CPX-005 | doc 31 | +| Forbidden algorithm baseline always enforced | CPX-009 (no null approved_algorithms) | doc 31 | +| Revocation registry always maintained | CPX-003 | doc 31 | + +--- + +### Priority 2 โ€” Ease of Use + +DCM exists to enable self-service for application teams. If the right path is also the hard path, teams will find other paths โ€” and those other paths are ungoverned. + +**What this means:** + +The secure path must also be the easy path. Profile defaults should work for most deployments without customization. Ordinary requests should auto-approve without human intervention. Policy authoring should not require Rego expertise for common patterns. The Flow GUI, scoring model, and consumer contribution endpoints all exist to make governed behavior less operationally burdensome. + +**Ease of use serves security.** An organization that finds DCM too cumbersome and routes requests outside DCM has eliminated all of DCM's security benefits. A homelab team that circumvents credential management because it's too complex has no credential management. + +**The design principle:** When implementing a security requirement, simultaneously design the ease-of-use mechanism that makes it effortless to comply with. The scoring model's auto-approval threshold (not making every request require human review) is ease of use in service of security. + +**Things that should be easy in all profiles:** +- Requesting a standard resource (auto-approve for clean requests) +- Authoring a common policy without Rego expertise (visual condition builder) +- Retrieving a credential after resource provisioning (direct API call) +- Understanding why a request was scored a certain way (score_drivers field) +- Contributing a policy (API endpoint, not manual GitOps PR) + +--- + +### Priority 3 โ€” Extensibility and Capability Grouping + +The profile system, compliance domain overlays, policy groups, capability extensions, and registry governance make DCM adaptable to arbitrary organizational requirements without code changes. + +**What this means:** + +New compliance requirements should be expressible as policy additions within the existing framework. New provider types should fit the existing Provider base contract. New deployment contexts should be addressable through profile configuration. A platform that requires modifying source code for each new deployment context is not a platform โ€” it is a template. + +**Grouping is the mechanism for extensibility.** Compliance domain overlays compose with base profiles. Policy groups compose with profile policies. Capability extensions compose with base provider contracts. The three-abstraction model (Data, Provider, Policy) is the foundation that makes all of this compositional. + +**Extensibility must not compromise security or usability.** An extension mechanism that allows downstream users to disable security properties (rather than scale them) fails priority 1. An extension mechanism that requires expertise to configure fails priority 2. The profile system's hard constraints (SMX-008: auto_approve โ‰ค 50; CPX-001: no values in DCM stores) are precisely the boundaries that prevent extensibility from undermining security. + +--- + +### Priority 4 โ€” Fit for Purpose (Always Required) + +DCM must manage data center infrastructure lifecycle. All of the above is in service of this purpose. An architecturally beautiful system that cannot provision a VM, track its drift, and decommission it cleanly has failed at its reason for existing. + +**What this means:** + +Design decisions that serve priorities 1โ€“3 but break the end-to-end lifecycle (request โ†’ provision โ†’ operate โ†’ decommission) are not acceptable. Every capability added must have a clear answer to "how does this serve the lifecycle management mission?" + +Fit for purpose is not a fourth priority that can be traded against the first three โ€” it is a precondition. If a design cannot fulfill its stated purpose, priorities 1โ€“3 become irrelevant. This is why it is listed fourth rather than first: it is assumed, not aspirational. + +--- + +## Applying the Priorities โ€” Decision Framework + +When facing a design decision where priorities seem to conflict, apply this sequence: + +``` +1. Does this design decision compromise a non-negotiable security property? + YES โ†’ redesign until it does not. No exceptions. + +2. Does the secure option create significant operational burden? + YES โ†’ design the ease-of-use mechanism simultaneously. + The secure path must also be the easy path. + If you cannot make it easy enough, reconsider whether the + security property is correctly scoped. + +3. Can this behavior be expressed through the existing profile/policy/extension system? + YES โ†’ use it. Do not add new mechanisms when existing ones suffice. + NO โ†’ extend the existing mechanism before creating a new one. + +4. Does this design decision support the complete lifecycle? + NO โ†’ do not proceed until it does. +``` + +### Common Misapplications + +**"We can disable X in the minimal profile for simplicity."** +Wrong application. The minimal profile scales down operational burden, not security properties. The question is: what is the minimum viable implementation of X that requires no operational overhead? That is what minimal profile gets. + +**"Security is too complex for our users, so we'll make it optional."** +Wrong application. If security is too complex, the design of the security mechanism needs to improve (priority 2). Making security optional removes it โ€” that fails priority 1. Design a simpler mechanism that achieves the same security outcome. + +**"We need a new mechanism for this capability."** +Wrong starting point (priority 3 failure). The question is: can this be expressed through profiles, policies, provider capability extensions, or compliance overlays? Usually yes. If genuinely not, extend the nearest existing mechanism rather than creating a new one. + +**"This edge case isn't part of the lifecycle."** +Wrong framing (priority 4). Every edge case in the lifecycle โ€” partial realization, compensation, drift remediation, credential revocation on decommission โ€” is part of the lifecycle. Fit for purpose means handling the complete lifecycle, not just the happy path. + +--- + +## Profile Scaling Model + +The profile system is the primary mechanism for expressing priorities 1โ€“3 simultaneously. Understanding what profiles control โ€” and what they do not โ€” is essential to applying the priority order correctly. + +**Profiles control:** +- Enforcement strictness (how strictly a security property is enforced) +- Threshold values (how long, how often, how many) +- Automation level (automated vs manual trigger) +- Approval tier (auto-approve vs human review vs verified vs authorized) +- Review periods (how long shadow mode runs before promotion) + +**Profiles do not control:** +- Whether a security property is present (it always is) +- Which non-negotiable constraints apply (CPX-001, SMX-004, SMX-008, etc.) +- Whether the audit trail is maintained (always maintained; retention varies) +- Whether the data model is valid (schema conformance is not profile-dependent) + +### Profile Scaling Table + +This table shows how representative security properties scale across profiles. "Present" means the property is architecturally required โ€” what varies is the configuration. + +| Security Property | minimal | dev | standard | prod | fsi | sovereign | +|------------------|---------|-----|----------|------|-----|-----------| +| Credential rotation | Required; P365D max; manual OK | Required; P180D max; manual OK | Required; automated | Required; strict interval | Required; P90D max | Required; hardware-triggered | +| Idle detection threshold | P30D | P14D | P7D | P3D | P1D | PT12H | +| Algorithm baseline | Forbidden list | Forbidden list | Approved list | Approved list | FIPS-only | HSM-generated only | +| Shadow mode on contribution | Always on | Always on | Always on | Always on | Always on | Always on | +| First retrieval audit | Always | Always | Always | Always | Always | Always | +| Revocation cache TTL | PT5M | PT5M | PT1M | PT1M | PT30S | PT15S | +| Auto-approve threshold | โ‰ค 45 | โ‰ค 40 | โ‰ค 25 | โ‰ค 15 | โ‰ค 10 | โ‰ค 5 | +| Step-up MFA for credentials | Optional | Optional | Sensitive types | All types | Hardware MFA | mTLS | +| FIPS level | None required | None required | None required | Level 1 | Level 2 | Level 3 | +| IP binding | Not required | Not required | Not required | Not required | Required | Required | +| Hub contribution auto-approve | Yes | Yes | Yes | No (human review) | No (verified) | No (authorized) | + +--- + +--- + +## Approval Tier Model + +DCM defines four approval tiers that apply to requests, policy contributions, provider registrations, and any pipeline decision requiring human authorization. Understanding the model is critical: **DCM provides the gate and the audit trail. The review process is the organization's responsibility.** + +> **Full specification:** See [Authority Tier Model](32-authority-tier-model.md) for the complete ordered tier list, custom tier contribution model, dynamic threshold format, and ATM-001โ€“ATM-008 system policies. + +### What DCM Does vs What Organizations Provide + +| Tier | Required authority level | DCM provides | Organization provides | DCM gate condition | +|------|-------------------------|-------------|----------------------|-------------------| +| `auto` | None โ€” `decision_gravity: none`; system confidence sufficient | Structural and governance validation; automatic activation on pass | Nothing โ€” fully automated | All validation checks pass | +| `reviewed` | Standard authority โ€” `decision_gravity: routine`; one qualified reviewer in the relevant domain | Approval record; eligible reviewer notification via Notification Provider; pipeline hold; decision recording via Admin API; activation or rejection | Who constitutes a qualified reviewer; the review process; recording the decision via DCM API or an external system that calls it | One actor with reviewer role records a decision via the Admin API | +| `verified` | Elevated authority โ€” `decision_gravity: elevated`; independent confirmation required; separation of duties | Approval record requiring two independent decisions; enforces distinct actors (same actor cannot provide both); eligible reviewer notification; pipeline hold | Who constitutes qualified reviewers; both review processes; may use external workflow tools that call the DCM API | Two distinct actors with reviewer role each record a decision via the Admin API | +| `authorized` | Senior/governing authority โ€” `decision_gravity: critical`; highest organizational weight; most consequential decisions | Approval record specifying the required DCMGroup and threshold (N of M); group member notification; pipeline hold; individual decision tracking via Admin API; threshold evaluation; activation when N reached | Who constitutes the authority group (one person with delegated authority, a CTO, a CISO and legal counsel, a change board โ€” the organization decides); how they deliberate; what external tools they use; DCM records decisions, not deliberation | N members of the declared DCMGroup record decisions via the Admin API within the declared window | + +### Tier Extensibility + +The four default tiers (`auto`, `reviewed`, `verified`, `authorized`) are DCM system defaults. Organizations can add custom tiers by inserting them into the ordered list between existing tiers. The tier name is stable; numeric weight is derived from list position at evaluation time. + +Example: An organization adds `compliance_reviewed` between `verified` and `authorized`: +``` +auto โ†’ reviewed โ†’ verified โ†’ compliance_reviewed โ†’ authorized +``` +All existing references to `authorized` continue to work. Only the threshold ranges in the affected profile need updating. See [Authority Tier Model](32-authority-tier-model.md). + +### The `authorized` Tier โ€” What DCM Builds vs What It Does Not Not + +DCM does **not** build an authorized group management system. It does not track deliberation, run voting sessions, manage agendas, or coordinate review meetings. + +DCM builds: + +1. **DCMGroup membership management** โ€” which actors constitute the authorized group; configurable by platform admins +2. **Quorum declaration** โ€” `N of M` threshold declared in the profile or per-decision configuration +3. **Notification routing** โ€” when a decision enters `pending_authorized` state, the Notification Provider fires to all DCMGroup members +4. **Vote recording API** โ€” the Admin API endpoint that authorized group members (or external systems acting on their behalf) call to record `approve` or `reject` +5. **Quorum tracking** โ€” DCM counts votes and advances the pipeline when N is reached +6. **Audit trail** โ€” every vote is audited with actor UUID, timestamp, decision, and the system that recorded it + +External systems (ServiceNow, Jira, email workflows, Slack bots) connect to DCM by calling the vote recording API. A Slack bot that collects emoji reactions from group members and then calls DCM's Admin API is a valid implementation โ€” DCM doesn't care how the organization collected the vote, only that an authorized group member recorded it. + +### Admin API as the Integration Point + +The Admin API approval endpoint is designed to be called by external systems, not only by humans in a DCM UI: + +``` +POST /admin/api/v1/approvals/{approval_uuid}/vote +Authorization: Bearer # any actor who is a member of the required DCMGroup + +{ + "decision": "approve | reject", + "reason": "", + "recorded_via": "dcm_admin_ui | servicenow | jira | slack_bot | api_direct | other", + "external_reference": "" +} + +Response 200: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 2, + "quorum_required": 3, + "quorum_reached": false, + "pipeline_status": "pending_authorized" +} + +# When quorum is reached: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 3, + "quorum_required": 3, + "quorum_reached": true, + "pipeline_status": "activating" +} +``` + +The `recorded_via` field provides the audit trail provenance โ€” DCM knows whether the vote came through its own UI, a ServiceNow integration, a Jira plugin, or a direct API call. This is not enforced โ€” it is informational for audit purposes. + +### Deadline and Escalation + +DCM manages the approval window (the time within which a decision must be reached) and fires escalation notifications when the window is approaching: + +```yaml +approval_window: + reviewed: PT72H # 3 days; configurable per profile + verified: PT72H + authorized: P7D # 7 days for deliberation; configurable + on_expiry: + reviewed: escalate # escalate to platform admin + verified: escalate + authorized: reject # authorized tier that cannot reach threshold in window โ†’ reject +``` + +When the window expires without a decision, DCM fires an escalation notification and either rejects (for authorized) or escalates to the next approval tier (for reviewed and verified). The organization can configure these windows to match their actual governance processes. + +### DPO Alignment + +The approval tier model directly implements all four design priorities: + +1. **Security:** Approval tiers are the enforcement mechanism for governance. The gate is DCM's responsibility โ€” it cannot be bypassed, and every decision is audited. +2. **Ease of use:** The Admin API as integration point means organizations use whatever workflow tools they already have. DCM does not require them to adopt a new process tool. +3. **Extensibility:** The `recorded_via` field and `external_reference` field make the approval tier model composable with arbitrary external systems without DCM needing to integrate with each one. +4. **Fit for purpose:** The tier model enables governance of every pipeline decision (request approval, policy contribution, provider registration, federation contribution) through a single consistent mechanism. + +## Documentation Requirements + +Every document in the DCM data model should: + +1. **Reference the priority order** where design decisions are made that involve tradeoffs +2. **Explain non-negotiable security properties** with clear rationale +3. **Document what profiles control vs what they do not** for each security-relevant configuration +4. **Identify the ease-of-use mechanism** that accompanies every security requirement +5. **State fit-for-purpose scope** explicitly โ€” what lifecycle operations does this document govern? + +--- + +## System Policies + +| Policy | Rule | +|--------|------| +| `DPO-001` | Security properties are architecturally present in all profiles. Profiles control enforcement strictness, thresholds, and automation level โ€” not whether the property exists. | +| `DPO-002` | Every security requirement must be accompanied by an ease-of-use mechanism that makes compliance effortless for the common case. A security model routinely bypassed because of complexity has failed. | +| `DPO-003` | New capabilities should be expressed through the existing profile/policy/provider extension system before creating new mechanisms. Extensibility is achieved through composition, not proliferation. | +| `DPO-004` | Fit for purpose is a precondition, not a priority. All four priorities apply only within the constraint that the system can fulfill its lifecycle management mission. | +| `DPO-005` | The `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security." Design decisions that disable security properties rather than scaling them violate DPO-001. | +| `DPO-006` | When security and ease of use conflict, redesign the ease-of-use mechanism โ€” not the security requirement. The secure path must also be the easy path. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/federated-contribution-model.md b/content/docs/architecture/data-model/federated-contribution-model.md index cbd323d..53a9dd7 100644 --- a/content/docs/architecture/data-model/federated-contribution-model.md +++ b/content/docs/architecture/data-model/federated-contribution-model.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Federated Contribution Model" +title: "Federated Contribution Model" type: docs weight: 28 --- @@ -20,6 +20,8 @@ weight: 28 --- +> **Authority Tier Reference:** Contribution approval tiers (`reviewed`, `verified`, `authorized`) are named positions in the [Authority Tier Model](32-authority-tier-model.md) ordered list. Organizations may add custom tiers between existing ones. Changes to the tier registry that affect contribution approval requirements trigger impact detection (ATM-009โ€“ATM-012). + ## 1. Purpose and Principle DCM is a multi-user, multi-contributor system. Platform admins are not the only actors who create data. Consumers define their own service configurations, resource groups, and policy overlays. Service Providers publish their own resource type specs and catalog items. Peer DCM instances contribute registry entries across federation boundaries. Organizations extend DCM with their own artifact types. @@ -78,7 +80,7 @@ This is not a special model for special cases. It is the same GitOps PR workflow |-------------|-----------------| | Consumer | System or platform domain policies; core layers; resource type specs (unless granted elevated role); provider catalog items for other providers | | Service Provider | Policies outside their resource type domain; core layers; other providers' catalog items; tenant-domain policies for specific Tenants | -| Peer DCM | Artifacts above the federation trust level granted; system-domain policies without committee approval; sovereignty zones for jurisdictions not in their declared scope | +| Peer DCM | Artifacts above the federation trust level granted; system-domain policies without authorized approval; sovereignty zones for jurisdictions not in their declared scope | --- @@ -138,9 +140,9 @@ Contributor authors a data artifact โ”‚ โ–ผ Review flow (per profile + artifact type): โ”‚ auto: artifact activates immediately - โ”‚ human_review: one platform admin or designated reviewer approves - โ”‚ dual_approval: two independent reviewers approve - โ”‚ committee: declared DCMGroup reaches quorum + โ”‚ reviewed: one platform admin or designated reviewer approves + โ”‚ verified: two independent reviewers approve + โ”‚ authorized: N members of declared authority group record decisions via Admin API โ”‚ โ–ผ On approval โ†’ status: active โ”‚ For policies: shadow mode results reviewed; full enforcement begins @@ -157,21 +159,21 @@ Review requirements are profile-governed. The table below shows defaults: | Artifact Type | Platform Admin | Consumer/Tenant | Service Provider | |--------------|---------------|-----------------|-----------------| -| Tenant-domain policy | auto | human_review (standard+) | human_review | -| Resource Type Spec (Org tier) | auto | โŒ | human_review | -| Resource Type Spec (Community tier) | human_review | โŒ | dual_approval | -| Provider Catalog Item | auto | โŒ | human_review | -| Service Layer | auto | โŒ | human_review | -| Governance Matrix Rule (tenant) | auto | dual_approval | โŒ | -| Governance Matrix Rule (platform) | human_review | โŒ | โŒ | -| Accreditation | human_review | โŒ | human_review | +| Tenant-domain policy | auto | reviewed (standard+) | reviewed | +| Resource Type Spec (Org tier) | auto | โŒ | reviewed | +| Resource Type Spec (Community tier) | reviewed | โŒ | verified | +| Provider Catalog Item | auto | โŒ | reviewed | +| Service Layer | auto | โŒ | reviewed | +| Governance Matrix Rule (tenant) | auto | verified | โŒ | +| Governance Matrix Rule (platform) | reviewed | โŒ | โŒ | +| Accreditation | reviewed | โŒ | reviewed | **Profile overrides:** - `dev`: most contributions auto-approved; shadow mode optional -- `standard`: consumer policies require human_review; provider specs require human_review -- `prod`: consumer governance matrix rules require dual_approval; provider specs require dual_approval -- `fsi`: all contributions require dual_approval; community registry entries require committee -- `sovereign`: all contributions require committee approval +- `standard`: consumer policies require reviewed; provider specs require reviewed +- `prod`: consumer governance matrix rules require verified; provider specs require verified +- `fsi`: all contributions require verified; community registry entries require authorized +- `sovereign`: all contributions require authorized approval --- @@ -224,7 +226,7 @@ Response 202 Accepted: "status": "proposed", "shadow_mode": true, "review_required": true, - "review_type": "human_review", + "review_type": "reviewed", "reviewer_group": "platform-admins", "pr_url": "https://git.corp.example.com/dcm-policies/pulls/145", "shadow_results_url": "/flow/api/v1/shadow/" @@ -296,7 +298,7 @@ Response 202 Accepted: "resource_type_fqn": "Storage.DistributedVolume", "status": "proposed", "review_required": true, - "review_type": "human_review", + "review_type": "reviewed", "pr_url": "https://git.corp.example.com/dcm-registry/pulls/89" } ``` @@ -340,9 +342,9 @@ Federation contributions inherit the federation trust posture of the contributin | Peer trust posture | Contribution review requirement | Artifact types permitted | |-------------------|--------------------------------|------------------------| -| `verified` | human_review (standard+); auto (dev) | Registry entries, policy templates, service layers | -| `vouched` | human_review always | Registry entries, service layers only | -| `provisional` | Committee approval | Registry entries only (no policies) | +| `verified` | reviewed (standard+); auto (dev) | Registry entries, policy templates, service layers | +| `vouched` | reviewed always | Registry entries, service layers only | +| `provisional` | `authorized` tier approval | Registry entries only (no policies) | **Hard rule:** A peer DCM cannot contribute artifacts at a higher domain level than its trust posture permits. A `vouched` peer cannot contribute system-domain policies. This is enforced by the Governance Matrix at the federation contribution boundary. @@ -377,7 +379,13 @@ In a Hub-Spoke federation, the Hub DCM is the authoritative source for platform- hub_policy_distribution: hub_dcm_uuid: distribution_type: push # Hub pushes on policy change - auto_approve_from_hub: true # prod profile: false; dev: true + auto_approve_from_hub: # profile-governed; security-first: prod+ always requires review + minimal: true + dev: true + standard: true + prod: false # reviewed required even from verified Hub + fsi: false # verified required + sovereign: false # authorized approval required policy_handles_subscribed: - "system/compliance/hipaa/*" - "system/governance/drift-remediation" @@ -490,19 +498,19 @@ Each deployment profile has a default contribution policy that governs auto-appr ```yaml contribution_policy: minimal: - consumer_policy_auto_approve: true + consumer_policy_auto_approve: true # ease of use: homelab auto-approves provider_spec_auto_approve: true - federation_contribution_auto_approve: true # dev/homelab: trust all - shadow_mode_default: false + federation_contribution_auto_approve: true # homelab: federation auto-approved + shadow_mode_default: true # security: shadow always on even in minimal dev: consumer_policy_auto_approve: true provider_spec_auto_approve: true - federation_contribution_auto_approve: false # human_review for federation + federation_contribution_auto_approve: false # reviewed for federation shadow_mode_default: true # shadow mode on by default standard: - consumer_policy_auto_approve: false # human_review for all policies + consumer_policy_auto_approve: false # reviewed for all policies provider_spec_auto_approve: false federation_contribution_auto_approve: false shadow_mode_default: true @@ -510,27 +518,27 @@ contribution_policy: prod: consumer_policy_auto_approve: false - consumer_governance_matrix_requires: dual_approval + consumer_governance_matrix_requires: verified provider_spec_auto_approve: false - provider_spec_requires: human_review - federation_contribution_requires: human_review + provider_spec_requires: reviewed + federation_contribution_requires: reviewed shadow_mode_default: true shadow_review_period: P14D fsi: consumer_policy_auto_approve: false - consumer_policy_requires: dual_approval - consumer_governance_matrix_requires: dual_approval - provider_spec_requires: dual_approval - federation_contribution_requires: dual_approval + consumer_policy_requires: verified + consumer_governance_matrix_requires: verified + provider_spec_requires: verified + federation_contribution_requires: verified shadow_mode_default: true shadow_review_period: P30D min_shadow_divergence_review: true # must review all divergence cases sovereign: - consumer_policy_requires: committee - provider_spec_requires: committee - federation_contribution_requires: committee + consumer_policy_requires: authorized + provider_spec_requires: authorized + federation_contribution_requires: authorized shadow_mode_default: true shadow_review_period: P30D min_shadow_divergence_review: true @@ -580,7 +588,7 @@ governance_matrix_rule: | `FCM-004` | Policies submitted by any contributor enter proposed (shadow) status by default. Shadow mode results must be available before the active profile's shadow_review_period expires. | | `FCM-005` | Platform admins may override any contributor's artifact lifecycle at any time. Override actions are audited. | | `FCM-006` | Orphaned artifacts (contributor access revoked) do not automatically deactivate. A platform admin assigns a new owner or explicitly retires them. Exception: sovereign profile auto-retires orphaned artifacts. | -| `FCM-007` | Federation contributions from peer DCMs are scoped by the peer's federation trust posture. Verified peers: human_review (standard+). Vouched peers: human_review always. Provisional peers: committee approval. | +| `FCM-007` | Federation contributions from peer DCMs are scoped by the peer's federation trust posture. Verified peers: reviewed (standard+). Vouched peers: reviewed always. Provisional peers: authorized approval. | | `FCM-008` | Contributor-tier scope limits are absolute. A consumer-authored policy in the tenant domain cannot affect the system or platform domain regardless of the policy's declared match conditions. | --- diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md index f495765..7d981bf 100644 --- a/content/docs/architecture/data-model/foundations.md +++ b/content/docs/architecture/data-model/foundations.md @@ -1,5 +1,5 @@ --- -title: "DCM โ€” Foundational Abstractions" +title: "DCM Foundational Abstractions" type: docs weight: 0 --- @@ -271,3 +271,38 @@ These three abstractions serve DCM's core ethos: --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + + +--- + +## Design Priority Order + +> **Full specification:** See [Design Priorities](00-design-priorities.md) for the complete priority framework, decision framework, profile scaling table, and DPO-001โ€“006 system policies. + + +Every design decision in DCM is evaluated against this priority order. When priorities conflict, higher priorities win. When there is no conflict, all four apply simultaneously. + +**1. Industry best practices for security** +Security is not a feature or a profile option. It is the baseline that every other design decision must respect. Where security and convenience conflict, security wins โ€” but the design must find a way to make the secure path the easy path. A security model that is routinely bypassed because it is too burdensome has failed at both security and usability. + +*In practice:* Security properties โ€” value separation, non-transferable credentials, scoped permissions, rotation, audit, revocation propagation โ€” are architecturally present in every profile. What profiles control is the enforcement strictness, operational automation, and threshold values. A `minimal` profile does not disable security; it implements security with minimal operational overhead. + +**2. Ease of use** +DCM exists to enable self-service for application teams. If the right path is also the hard path, teams will find other paths. The goal is to make secure, governed, auditable infrastructure management the path of least resistance โ€” not the path of compliance obligation. + +*In practice:* Profile defaults should eliminate configuration burden for common cases. The standard pipeline should auto-approve ordinary requests without human intervention. Policy authoring should not require Rego expertise for common patterns. The Flow GUI, scoring model, and contribution endpoints all serve this priority. + +**3. Extensibility and capability grouping** +The profile system, compliance domain overlays, policy groups, and registry governance exist to make DCM adaptable to arbitrary organizational requirements without code changes. This priority serves at scale โ€” a platform that can only be configured by modifying source code is not a platform. + +*In practice:* New compliance requirements should be expressible as policy additions within the existing framework. New provider types should fit the existing Provider base contract. New deployment contexts should be addressable through profile configuration. + +**4. Fit for purpose (always required)** +DCM must actually manage data center infrastructure lifecycle. All of the above is in service of this purpose โ€” not independent of it. An architecturally beautiful system that cannot provision a VM, track its drift, and decommission it cleanly has failed at its reason for existing. + +*In practice:* Design decisions that serve priorities 1โ€“3 but break the end-to-end lifecycle (request โ†’ provision โ†’ operate โ†’ decommission) are not acceptable. Every capability added must have a clear answer to "how does this serve the lifecycle management mission?" + +--- + +**The implication for profiles:** A `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security." The security architecture is present and correct in every profile. What varies is how much automation, how strict the thresholds, and how much manual intervention is acceptable. This is the principle that makes DCM trustworthy in a homelab and in a sovereign government deployment using the same codebase. + diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md index 658bf7c..b8ab663 100644 --- a/content/docs/architecture/data-model/layering-and-versioning.md +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Data Layers and the Assembly Process" +title: "Layering, Versioning, and Override Precedence" type: docs weight: 3 --- @@ -279,7 +279,7 @@ Every layer type has a declared contributor type. The contributor determines wha | Base Layer | Platform Admin | system | auto | | Core Layer | Platform Admin | platform | auto | | Intermediate / Customization Layer | Platform Admin, Consumer/Tenant | platform, tenant | per profile | -| Service Layer | Platform Admin, Service Provider | provider | human_review (standard+) | +| Service Layer | Platform Admin, Service Provider | provider | reviewed (standard+) | | Request Layer | Consumer/Tenant | tenant | auto (applied directly to request) | | Policy Layer | All contributor types | per contributor role | per profile + contributor type | diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 3922ebb..8aff0e1 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -58,6 +58,19 @@ Policy Providers โ€” external authoritative policy sources --- +## 1a. Design Priority Order in Policy Profiles + +Profiles implement the DCM design priority order (see [Foundational Abstractions](00-foundations.md)): + +1. **Security:** Profile defaults implement security correctly. Lower profiles have less strict enforcement โ€” not absent security. +2. **Ease of use:** Profile defaults minimize configuration burden. `standard` profile should work for most deployments without customization. +3. **Extensibility:** Profiles compose with compliance domain overlays. Organizations add compliance requirements additively without rewriting base configuration. +4. **Fit for purpose:** Every profile must support the complete DCM lifecycle. + +**The `minimal` profile is not "security optional"** โ€” it is the security model with minimal operational overhead. All security properties are present; thresholds and automation levels are relaxed. + +--- + ## 1b. Policy Authorship โ€” Federated Contribution Model Policies in DCM are not exclusively authored by platform admins. The DCM federated contribution model enables all actor types to author policies within their permitted domain scope: @@ -1489,7 +1502,7 @@ policy_provider_trust_elevation: fsi: approvers: [platform_admin, security_owner, compliance_officer] min_approvers: 2 - dual_approval_required: true + verified_required: true sovereign: approvers: [platform_admin, security_owner, compliance_officer] min_approvers: 3 diff --git a/content/docs/architecture/data-model/provider-contract.md b/content/docs/architecture/data-model/provider-contract.md index 2f6e38e..6219b60 100644 --- a/content/docs/architecture/data-model/provider-contract.md +++ b/content/docs/architecture/data-model/provider-contract.md @@ -16,6 +16,8 @@ weight: -9 --- +> > **Design Priority:** Provider types implement all four design priorities simultaneously. Security properties (mTLS, scoped credentials, sovereignty declarations, accreditation) are present in all provider registrations. The capability extension model (Priority 3) enables new provider types without changing the base contract. See [Design Priorities](00-design-priorities.md). + ## 1. The Unified Provider Contract Every Provider in DCM โ€” regardless of type โ€” implements a single base contract. What varies between provider types is the **capability extension**: the specific operations exposed, the data that flows in each direction, and the typed schemas for that exchange. @@ -332,25 +334,48 @@ policy_provider_capabilities: ### 7.6 Credential Provider -**What it does:** Issues, stores, rotates, and revokes credentials and secrets. +**What it does:** Issues, rotates, and revokes credentials used within the DCM ecosystem โ€” both DCM interaction credentials (short-lived, scoped, used for provider dispatch under the Zero Trust model) and consumer-facing resource credentials (SSH keys, API keys, kubeconfigs, service account tokens, database passwords, x509 certificates). + +> **Full specification:** See [Credential Provider Model](31-credential-provider-model.md) for the complete issuance contract, rotation protocol, revocation propagation, consumer delivery, and system policies (CPX-001โ€“CPX-008). + +**Credential values are never stored in DCM** โ€” only credential metadata (UUID, type, scope, expiry, status) is stored. Values are held by the Credential Provider and retrieved by authorized consumers via a declared `value_retrieval_endpoint`. **Additional endpoints:** ``` -POST {issue_endpoint} # request a credential; return scoped credential -POST {rotate_endpoint} # rotate an existing credential -DELETE {revoke_endpoint} # revoke a credential +POST {issue_endpoint} # issue credential; return metadata + retrieval URL +POST {rotate_endpoint} # rotate; return old/new UUIDs + transition window +DELETE {revoke_endpoint}/{uuid} # revoke immediately or at transition window end +POST {validate_endpoint} # use-time validity check (scope, revocation, expiry) +GET {list_endpoint} # list credentials by entity_uuid or issued_to ``` -**Capability declaration extension:** +**Capability declaration extension (summary โ€” full schema in doc 31):** ```yaml credential_provider_capabilities: - credential_types: [api_key, x509_certificate, service_account_token] - dynamic_secrets: true # generate on demand; expire after use + credential_types: + - api_key + - x509_certificate + - ssh_key + - service_account_token + - database_password + - kubeconfig + - hsm_backed_key + - dcm_interaction # required if handling DCM interaction credentials hsm_backed: false - fips_140_2_level: 1 | 2 | 3 + fips_140_2_level: 0 | 1 | 2 | 3 # enforced per profile (Section 12) + dynamic_secrets: true + rotation_support: true + revocation_sla: PT5M # profile-governed; PT30S for sovereign + approved_algorithms: # declare which algorithms the provider supports + ssh_key: [Ed25519, ECDSA-P-384, RSA-4096] + x509_certificate: [Ed25519, ECDSA-P-384, RSA-4096] + service_account_token: [RS256, ES256, HS256] + # ... per credential type + key_escrow: + supported: false # true only for regulated sovereign deployments ``` -**Data direction:** DCM requests credential โ†’ Provider issues scoped credential โ†’ DCM includes in provider dispatch. +**Data direction:** DCM requests credential โ†’ Provider issues scoped credential + returns metadata โ†’ DCM stores metadata, includes retrieval URL in realized entity โ†’ Consumer retrieves value via authenticated endpoint. Revocation: DCM requests revocation โ†’ Provider invalidates โ†’ DCM publishes revocation event to Message Bus โ†’ all components refresh revocation cache within profile-governed TTL. --- @@ -485,7 +510,7 @@ The Provider Type Registry is the authoritative list of provider types that a DC provider_type_registry_entry: provider_type_id: service_provider tier: core - default_approval_method: human_review # auto | human_review | dual_approval | committee + default_approval_method: reviewed # auto | reviewed | verified | authorized enabled_in_profiles: [minimal, dev, standard, prod, fsi, sovereign] capability_extension_schema_ref: ``` diff --git a/content/docs/architecture/data-model/registry-governance.md b/content/docs/architecture/data-model/registry-governance.md index 41a644d..97828c9 100644 --- a/content/docs/architecture/data-model/registry-governance.md +++ b/content/docs/architecture/data-model/registry-governance.md @@ -1,5 +1,5 @@ --- -title: "DCM Data Model โ€” Registry Governance" +title: "Registry Governance" type: docs weight: 20 --- @@ -62,7 +62,7 @@ The three-tier registry model applies to all DCM artifact types, not just resour |------|--------------|---------|-------------------| | **Core** | DCM Project | Built-in policies, base layers, system resource types | DCM project PR process | | **Verified Community** | Named community maintainers | Community resource types, shared policy templates, vetted provider specs | Community review + platform admin acceptance | -| **Organization** | Deploying organization | Tenant policies, provider catalog items, org-specific specs | Per profile (auto โ†’ committee) | +| **Organization** | Deploying organization | Tenant policies, provider catalog items, org-specific specs | Per profile (auto โ†’ authorized) | **Contributor sub-tiers within Organization tier:** - `organization/platform` โ€” authored by platform admins; highest trust in org tier diff --git a/content/docs/architecture/data-model/scoring-model.md b/content/docs/architecture/data-model/scoring-model.md index 73cad8a..fc47190 100644 --- a/content/docs/architecture/data-model/scoring-model.md +++ b/content/docs/architecture/data-model/scoring-model.md @@ -18,6 +18,9 @@ weight: 29 > > The Scoring Model is an extension of the Policy abstraction. Scored signals are Data artifacts with lifecycle and provenance. Profile thresholds are Policy-governed configuration. The Governance Matrix remains a pure boolean gate โ€” scoring never applies to cross-boundary data decisions. > See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) +> > **See also:** [Authority Tier Model](32-authority-tier-model.md) โ€” the ordered authority tier list, custom tier definition, dynamic threshold format, and ATM system policies. + +> **Design Priority:** The Scoring Model is the primary mechanism for Priority 2 (ease of use) in service of Priority 1 (security). The auto-approval threshold (SMX-008: โ‰ค 50) and compliance-class GateKeepers are non-negotiable security properties. Profile thresholds and signal weights are the ease-of-use scaling mechanism. See [Design Priorities](00-design-priorities.md). --- @@ -304,20 +307,32 @@ Signal weights are profile-governed and can be adjusted per deployment. The weig Every profile declares scoring thresholds that map the continuous risk score to a discrete approval routing decision. ```yaml +# Approval routing uses named tier thresholds โ€” see Authority Tier Model (doc 32) +# Tier names are resolved from the ordered authority tier list; numeric weights are derived. scoring_thresholds: - auto_approve_below: 25 # score 0โ€“24: auto-approve - human_review_above: 25 # score 25โ€“59: one reviewer required - dual_approval_above: 60 # score 60โ€“79: two independent reviewers - committee_above: 80 # score 80โ€“100: declared DCMGroup reaches quorum + approval_routing: + - tier: auto + max_score: 24 # score 0โ€“24: auto-approve (SMX-008: never exceed 50) + - tier: reviewed + max_score: 59 # score 25โ€“59: reviewed tier required + - tier: verified + max_score: 79 # score 60โ€“79: verified tier required + - tier: authorized + max_score: 100 # score 80โ€“100: authorized tier required + # Custom tiers (if defined) are inserted into this list; existing names unchanged. + # "authorized" means DCM holds the pipeline and notifies the declared DCMGroup; + # the review process and deliberation are the organization's responsibility. + # DCM records votes via Admin API; external systems (ServiceNow, Jira, Slack) + # may call the API on behalf of authorized group members. See [Design Priorities](00-design-priorities.md). # Note: compliance-class GateKeeper deny always halts regardless of score ``` ### 5.1 Per-Profile Threshold Defaults -| Profile | auto_approve | human_review | dual_approval | committee | signal_weights | +| Profile | auto_approve | reviewed | verified | authorized | signal_weights | |---------|-------------|-------------|--------------|-----------|----------------| -| `minimal` | < 60 | 60โ€“79 | 80โ€“100 | โ€” | default | -| `dev` | < 50 | 50โ€“79 | 80โ€“100 | โ€” | default | +| `minimal` | < 45 | 45โ€“74 | 75โ€“100 | โ€” | default | +| `dev` | < 40 | 40โ€“69 | 70โ€“100 | โ€” | default | | `standard` | < 25 | 25โ€“59 | 60โ€“79 | 80โ€“100 | default | | `prod` | < 15 | 15โ€“49 | 50โ€“74 | 75โ€“100 | gatekeeper_weight: 0.50 | | `fsi` | < 10 | 10โ€“39 | 40โ€“69 | 70โ€“100 | gatekeeper_weight: 0.55, actor_weight: 0.25 | @@ -334,7 +349,7 @@ resource_type_threshold_overrides: - resource_type: "Network.VLAN" auto_approve_below: 10 # VLANs require more scrutiny - resource_type: "Storage.Volume" - dual_approval_above: 40 # storage changes escalate earlier + verified_above: 40 # storage changes escalate earlier ``` ### 5.3 Tenant Threshold Overrides @@ -392,7 +407,7 @@ score_record: evaluated_at: request_risk_score: 47 - routing_decision: human_review + routing_decision: reviewed routing_threshold_applied: 25 # the threshold that triggered this tier profile_uuid: @@ -445,7 +460,7 @@ A platform admin or reviewer can override a score-based routing decision with a Consumers receive a simplified score view: - `risk_score` on request status (integer 0โ€“100) -- `routing_decision` (auto_approved | pending_review | pending_dual_approval | pending_committee) +- `routing_decision` (auto_approved | pending_review | pending_verified | pending_authorized) - `advisory_warnings` list from advisory Validation - `score_drivers` โ€” human-readable list of the top 3 contributing factors (no raw weights) @@ -478,7 +493,7 @@ Policy Engine evaluation run: 10. NEW: Aggregate โ†’ request_risk_score 11. NEW: Apply profile thresholds โ†’ routing_decision 12. NEW: Write Score Record to Audit Store - 13. Route request: auto_approve | queue_for_review | queue_dual | queue_committee + 13. Route request: auto_approve | queue_for_review | queue_dual | queue_authorized ``` Step 2, 3, and 4 are unchanged from the existing model. Steps 5โ€“13 are additive. diff --git a/content/docs/architecture/overview.md b/content/docs/architecture/overview.md index 58bf0e8..d1612d7 100644 --- a/content/docs/architecture/overview.md +++ b/content/docs/architecture/overview.md @@ -24,6 +24,29 @@ DCM is **not a provisioning tool**. It is the management plane that sits above p --- + +## Design Priority Order + +Every design decision in DCM is evaluated against this hierarchy. When priorities conflict, higher priorities win. + +| Priority | Principle | What it means | +|----------|-----------|--------------| +| **1. Security** | Industry best practices are the baseline | Security properties present in ALL profiles; profiles control enforcement strictness, not whether security applies | +| **2. Ease of use** | The secure path must be the easy path | Auto-approval for ordinary requests; profile defaults eliminate configuration burden; secure path is easy path | +| **3. Extensibility** | Adaptable through configuration, not code | New requirements as policy additions; new contexts as profile configuration; new providers as contract implementations; custom authority tiers inserted into ordered list without breaking existing references | +| **4. Fit for purpose** | Always required | Everything serves the lifecycle management mission: request โ†’ provision โ†’ operate โ†’ decommission | + +**The `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security."** All security properties are architecturally present in every profile. What varies is automation level, enforcement thresholds, and acceptable manual intervention. + +--- +## Authority Tier Model + +DCM governs decisions through an extensible **authority tier model** โ€” a named, ordered list where each tier expresses a required level of organizational decision gravity. The default tiers are `auto โ†’ reviewed โ†’ verified โ†’ authorized`, but organizations can insert custom tiers between existing ones. Tier weight is derived from list position at evaluation time; existing tier name references always resolve correctly. + +When the tier registry changes, DCM computes a **tier impact diff** identifying any items whose effective authority requirement changed. Security degradations (lower gravity than before) block activation until explicitly accepted by a verified-tier reviewer. See [Authority Tier Model](data-model/authority-tier-model/). + +--- + ## The Problem DCM Solves | Challenge | DCM Response | @@ -158,7 +181,7 @@ DCM uses a **hybrid model**: questions of fact use boolean gates; questions of d **Five scoring signals** aggregate into a request risk score (0โ€“100): operational GateKeeper contributions (45%), actor risk history (20%), completeness warnings (15%), quota pressure (10%), provider accreditation richness (10%). -**Profile-governed thresholds** map the score to approval routing: auto-approve / human_review / dual_approval / committee. Thresholds are tunable per profile without touching individual policies. Profiles can also override enforcement class per policy โ€” escalating operational policies to compliance-class, or demoting non-regulatory compliance policies to operational. +**Profile-governed thresholds** map the score to approval routing: auto / reviewed / verified / authorized (+ custom tiers). Thresholds use a named-tier dynamic list; see [Authority Tier Model](data-model/authority-tier-model/). Thresholds are tunable per profile without touching individual policies. Profiles can also override enforcement class per policy โ€” escalating operational policies to compliance-class, or demoting non-regulatory compliance policies to operational. The Governance Matrix is **always boolean** โ€” scoring never applies to cross-boundary data decisions. diff --git a/content/docs/architecture/specifications/admin-api-spec.md b/content/docs/architecture/specifications/admin-api-spec.md index 894e02f..779cb01 100644 --- a/content/docs/architecture/specifications/admin-api-spec.md +++ b/content/docs/architecture/specifications/admin-api-spec.md @@ -489,6 +489,8 @@ Same as Consumer API. Additional admin-specific codes: ## Scoring Model Administration +> Approval routing thresholds use named-tier dynamic format. See [Authority Tier Model](../data-model/32-authority-tier-model.md) for the complete specification. + ### Get Scoring Thresholds for Profile ``` @@ -499,9 +501,11 @@ Response 200: "profile": "standard", "scoring_thresholds": { "auto_approve_below": 25, - "human_review_above": 25, - "dual_approval_above": 60, - "committee_above": 80 + "approval_routing": [ + { "tier": "reviewed", "max_score": 59 }, + { "tier": "verified", "max_score": 79 }, + { "tier": "authorized", "max_score": 100 } + ] }, "signal_weights": { "operational_gatekeeper": 0.45, @@ -521,9 +525,11 @@ PATCH /admin/api/v1/profiles/{profile_name}/scoring { "scoring_thresholds": { "auto_approve_below": 20, - "human_review_above": 20, - "dual_approval_above": 55, - "committee_above": 75 + "approval_routing": [ + { "tier": "reviewed", "max_score": 59 }, + { "tier": "verified", "max_score": 79 }, + { "tier": "authorized", "max_score": 100 } + ] } } @@ -584,7 +590,7 @@ GET /admin/api/v1/scoring/audit Query parameters: from= to= - routing_decision= + routing_decision= risk_score_above= actor_uuid= resource_type= @@ -596,7 +602,7 @@ Response 200: "score_record_uuid": "", "request_uuid": "", "risk_score": 47, - "routing_decision": "human_review", + "routing_decision": "reviewed", "signal_breakdown": { ... }, "evaluated_at": "" } @@ -604,3 +610,228 @@ Response 200: } ``` + +--- + +## Approval Management + +DCM provides approval gates for requests, policy contributions, provider registrations, and federation contributions. The Admin API is the integration point for recording decisions โ€” it is designed to be called by both human reviewers in the DCM UI and by external systems (ServiceNow, Jira, Slack bots, workflow automation). + +### List Pending Approvals + +``` +GET /admin/api/v1/approvals/pending + +Query parameters: + approval_type= + tier= + reviewer_uuid= # approvals where this actor is an eligible reviewer + +Response 200: +{ + "pending_approvals": [ + { + "approval_uuid": "", + "approval_type": "policy_contribution", + "tier": "authorized", + "subject_uuid": "", + "subject_handle": "tenant/payments/gatekeeper/cost-ceiling", + "required_dcmgroup_uuid": "", # for authorized tier + "quorum_required": 3, + "votes_recorded": 1, + "submitted_at": "", + "window_expires_at": "", + "submitted_by": { "uuid": "", "display_name": "Bob Smith" } + } + ] +} +``` + +### Record an Approval Decision + +``` +POST /admin/api/v1/approvals/{approval_uuid}/vote + +{ + "decision": "approve | reject", + "reason": "", + "recorded_via": "dcm_admin_ui | servicenow | jira | slack_bot | api_direct | other", + "external_reference": "" +} + +Response 200: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 2, + "quorum_required": 3, + "quorum_reached": false, + "pipeline_status": "pending_authorized" +} + +# When quorum is reached or reviewed/verified satisfied: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 3, + "quorum_required": 3, + "quorum_reached": true, + "pipeline_status": "activating" +} + +Response 403: actor is not a member of the required authority group (authorized tier) or not in reviewer role +Response 409: actor has already voted on this approval (verified and authorized tiers enforce distinct voters) +Response 410: approval window has expired +``` + +### Get Approval Detail + +``` +GET /admin/api/v1/approvals/{approval_uuid} + +Response 200: +{ + "approval_uuid": "", + "approval_type": "authorized", + "subject_uuid": "", + "tier": "authorized", + "required_dcmgroup_uuid": "", + "quorum_required": 3, + "window_expires_at": "", + "votes": [ + { + "voter_uuid": "", + "voter_display_name": "Alice Chen", + "decision": "approve", + "recorded_at": "", + "recorded_via": "servicenow", + "external_reference": "CHG0012345" + } + ], + "status": "pending_authorized", + "quorum_reached": false +} +``` + + +--- + +## Authority Tier Registry Management + +> **Implementation note:** The tier registry change impact detection pipeline is specified in [Authority Tier Model](../data-model/32-authority-tier-model.md) Section 7. The endpoints below are the Admin API surface for proposing, reviewing, and activating tier registry changes. The detection mechanism (tier impact diff computation, affected item query, degradation gate) is an implementation responsibility. + +### Propose a Tier Registry Change + +``` +POST /admin/api/v1/tier-registry/changes + +{ + "proposed_tiers": [ + { "name": "auto", "insert_after": null, "decision_gravity": "none" }, + { "name": "reviewed", "insert_after": "auto", "decision_gravity": "routine" }, + { "name": "verified", "insert_after": "reviewed", "decision_gravity": "elevated" }, + { "name": "compliance_reviewed", "insert_after": "verified", "decision_gravity": "elevated" }, + { "name": "authorized", "insert_after": "compliance_reviewed", "decision_gravity": "critical" } + ], + "reason": "Adding compliance_reviewed tier for PCI-DSS regulated actions" +} + +Response 202 Accepted: +{ + "registry_change_uuid": "", + "status": "impact_assessment_pending", + "estimated_ready_at": "" +} +``` + +### Get Tier Registry Impact Report + +``` +GET /admin/api/v1/tier-registry/changes/{change_uuid}/impact + +Response 200: +{ + "registry_change_uuid": "", + "status": "impact_assessed | pending_degradation_review | ready_to_activate | blocked", + "summary": { + "degradations": 0, + "upgrades": 3, + "new_tiers": 1, + "broken_references": 0, + "profile_gaps": 2 + }, + "degradations": [], + "upgrades": [ ... ], + "profile_gaps": [ + { + "profile": "standard", + "missing_tiers": ["compliance_reviewed"], + "gap_effect": "Requests scoring in the compliance_reviewed range will route to verified tier until threshold list is updated" + } + ], + "blocking_items": [] +} +``` + +### Accept a Security Degradation + +``` +POST /admin/api/v1/tier-registry/changes/{change_uuid}/accept-degradation + +{ + "affected_item_uuid": "", + "affected_item_type": "provider_registration_requirement", + "acceptance_reason": "", + "accepted_by": "" +} + +Response 200: +{ + "acceptance_uuid": "", + "degradation_accepted": true, + "remaining_degradations": 0, + "change_status": "ready_to_activate" +} + +Response 403: actor does not hold verified or authorized tier reviewer role +Response 409: degradation already accepted +``` + +### Activate a Tier Registry Change + +``` +POST /admin/api/v1/tier-registry/changes/{change_uuid}/activate + +Response 200: +{ + "registry_change_uuid": "", + "activated_at": "", + "new_registry_version": "1.1.0", + "impact_report_uuid": "" +} + +Response 409: change has unresolved blocking items (broken_references or unaccepted degradations) +``` + +### List Historical Registry Changes + +``` +GET /admin/api/v1/tier-registry/changes?status=activated&limit=20 + +Response 200: +{ + "changes": [ + { + "registry_change_uuid": "", + "status": "activated", + "activated_at": "", + "proposed_by": { "uuid": "", "display_name": "Alice Chen" }, + "summary": { "degradations": 0, "upgrades": 2, "new_tiers": 1 }, + "impact_report_uuid": "" + } + ] +} +``` + diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index bb1de6e..83e2076 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -319,7 +319,7 @@ Response 202 Accepted: "status_url": "/api/v1/requests/{request_uuid}/status", "dry_run_result": null, # null if auto-approve; populated if review required "risk_score": 47, # aggregate request risk score (0โ€“100) - "routing_decision": "human_review", # auto_approved | pending_review | pending_dual_approval | pending_committee + "routing_decision": "reviewed", # auto_approved | pending_review | pending_verified | pending_authorized "score_drivers": [ # top 3 contributing factors (human-readable) "Estimated monthly cost exceeds Tenant ceiling", "Request submitted outside business hours", @@ -1215,7 +1215,7 @@ Response 200: "profile": "standard" }, "advisory_warnings": 1, - "policy_name": "scoring-threshold: standard/human_review" + "policy_name": "scoring-threshold: standard/reviewed" } ], "total": 2 @@ -1224,11 +1224,15 @@ Response 200: ### 6b.3 Approve or Reject a Request +This endpoint is used by reviewers with the appropriate role. It is designed to be callable by external systems (ServiceNow, Jira workflow integrations, Slack bots) that act on behalf of a reviewer โ€” the `Authorization` header identifies which reviewer is recording the decision. DCM provides the gate and audit trail; the review process is the organization's responsibility. See [Design Priorities โ€” Approval Tier Model](../data-model/00-design-priorities.md). + ``` POST /api/v1/approvals/{approval_uuid} { "decision": "approve | reject", - "reason": "" + "reason": "", + "recorded_via": "dcm_ui | servicenow | jira | slack_bot | api_direct | other", + "external_reference": "" } Response 202 Accepted: @@ -1600,7 +1604,7 @@ Response 202 Accepted: "status": "proposed", "shadow_mode": true, "review_required": true, - "review_type": "human_review", + "review_type": "reviewed", "pr_url": "https://git.corp.example.com/dcm-policies/pulls/145", "shadow_results_url": "/flow/api/v1/shadow/" } @@ -1674,6 +1678,77 @@ Response 200: ``` + +--- + +## 9b. Credential Management + +### 9b.1 List Credentials for a Resource + +``` +GET /api/v1/resources/{entity_uuid}/credentials + +Response 200: +{ + "credentials": [ + { + "credential_uuid": "", + "credential_type": "ssh_key", + "status": "active", + "issued_at": "", + "valid_until": "", + "scope": { "operations": ["ssh_access"] }, + "retrieval": { + "endpoint": "/api/v1/credentials//value", + "auth_required": "step_up_mfa", + "retrieval_count": 1, + "last_retrieved_at": "" + }, + "rotation_schedule": { + "next_rotation_at": "", + "rotation_trigger": "scheduled" + } + } + ] +} +``` + +### 9b.2 Retrieve Credential Value + +``` +GET /api/v1/credentials/{credential_uuid}/value +X-DCM-StepUp-Token: # if auth_required: step_up_mfa + +Response 200: +{ + "credential_uuid": "", + "credential_type": "ssh_key", + "value": { "private_key": "...", "public_key": "...", "username": "dcm-provisioned" }, + "valid_until": "", + "retrieval_uuid": "" +} + +Response 410 Gone: { "error": "credential_revoked_or_expired" } +``` + +### 9b.3 Request Credential Rotation + +``` +POST /api/v1/credentials/{credential_uuid}/rotate +{ + "reason": "Scheduled rotation per security policy" +} + +Response 202 Accepted: +{ + "old_credential_uuid": "", + "new_credential_uuid": "", + "transition_window_ends": "", + "new_retrieval_url": "/api/v1/credentials//value" +} +``` + + ## 8. Conformance Levels The Consumer API defines three conformance levels, mirroring the Operator Interface Specification model: diff --git a/content/docs/architecture/specifications/flow-gui-spec.md b/content/docs/architecture/specifications/flow-gui-spec.md index c1a462a..c5653b6 100644 --- a/content/docs/architecture/specifications/flow-gui-spec.md +++ b/content/docs/architecture/specifications/flow-gui-spec.md @@ -834,9 +834,11 @@ Response 200: "active_profile": "standard", "thresholds": { "auto_approve_below": 25, - "human_review_above": 25, - "dual_approval_above": 60, - "committee_above": 80 + "approval_routing": [ + { "tier": "reviewed", "max_score": 59 }, + { "tier": "verified", "max_score": 79 }, + { "tier": "authorized", "max_score": 100 } + ] }, "nodes": [ { @@ -853,7 +855,7 @@ Response 200: The Profile and Governance Management view (Section 7) is extended with a **Scoring Thresholds** panel: -- Visual slider showing auto_approve / human_review / dual_approval / committee bands on a 0โ€“100 scale +- Visual slider showing auto_approve / reviewed / verified / authorized bands on a 0โ€“100 scale - Signal weight configuration (pie chart showing proportional contribution of each signal) - Policy enforcement override management (which policies are promoted/demoted in this profile) - Live preview: "At the current thresholds, X% of last week's requests would have been auto-approved" @@ -863,7 +865,7 @@ The Profile and Governance Management view (Section 7) is extended with a **Scor The Flow Simulation output (Section 5) is extended with a score breakdown panel: ``` -Simulation result: risk_score=47, routing=human_review +Simulation result: risk_score=47, routing=reviewed Score breakdown: Operational GateKeepers: 50 ร— 0.45 = 22.5 @@ -879,7 +881,7 @@ Score breakdown: โ””โ”€โ”€ (richness score: 85/100 โ†’ contribution: 1.5) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Total: 37.8 โ†’ 47 (normalized) - Threshold (human_review): 25 + Threshold (reviewed): 25 Routing decision: HUMAN_REVIEW โœ“ ``` @@ -898,7 +900,7 @@ Request body: Response 200: { "risk_score": 47, - "routing_decision": "human_review", + "routing_decision": "reviewed", "signal_breakdown": { ... }, "threshold_applied": 25, "profile": "standard", diff --git a/content/docs/architecture/specifications/registration-spec.md b/content/docs/architecture/specifications/registration-spec.md index 8adcea6..4c5cd4e 100644 --- a/content/docs/architecture/specifications/registration-spec.md +++ b/content/docs/architecture/specifications/registration-spec.md @@ -60,7 +60,7 @@ provider_type_registry_entry: may_receive_sovereign_data: false # hard limit; never overridden # Approval method defaults (profile may override โ€” see Section 4) - default_approval_method: human_review # auto | human_review | dual_approval | committee + default_approval_method: reviewed # auto | reviewed | verified | authorized # Minimum trust level granted after approval default_trust_level: standard # minimal | standard | elevated | high @@ -82,16 +82,16 @@ provider_type_registry_entry: | # | provider_type_id | Default Approval | Enabled In | |---|-----------------|-----------------|------------| -| 1 | `service_provider` | human_review | all profiles | -| 2 | `information_provider` | human_review | all profiles | -| 3 | `meta_provider` | dual_approval | standard+ | -| 4 | `storage_provider` | dual_approval | all profiles | -| 5 | `message_bus_provider` | human_review | dev+ (external endpoints: standard+) | -| 6 | `policy_provider` (Mode 1-2) | human_review | all profiles | -| 7 | `policy_provider` (Mode 3-4) | dual_approval | standard+ | -| 8 | `credential_provider` | dual_approval | standard+ | -| 9 | `auth_provider` | dual_approval | all profiles | -| 10 | `notification_provider` | human_review | all profiles | +| 1 | `service_provider` | reviewed | all profiles | +| 2 | `information_provider` | reviewed | all profiles | +| 3 | `meta_provider` | verified | standard+ | +| 4 | `storage_provider` | verified | all profiles | +| 5 | `message_bus_provider` | reviewed | dev+ (external endpoints: standard+) | +| 6 | `policy_provider` (Mode 1-2) | reviewed | all profiles | +| 7 | `policy_provider` (Mode 3-4) | verified | standard+ | +| 8 | `credential_provider` | verified | standard+ | +| 9 | `auth_provider` | verified | all profiles | +| 10 | `notification_provider` | reviewed | all profiles | Note: Mode 3-4 Policy Providers are treated as a separate registry entry from Mode 1-2 due to the elevated trust requirements. @@ -155,14 +155,16 @@ Token values are presented exactly once โ€” at creation. They are never retrieva ## 3. Approval Method Configuration +> **Authority Tier Model:** Approval methods (`reviewed`, `verified`, `authorized`) are defined in the [Authority Tier Model](../data-model/32-authority-tier-model.md) as a named, ordered list. Organizations may insert custom tiers. The effective method resolution (Section 3.2) uses tier names; DCM resolves numeric weight from the ordered list at evaluation time (ATM-001). + ### 3.1 The Four Approval Methods | Method | Description | Approval path | |--------|-------------|--------------| | `auto` | DCM validates automatically; activates without human review | All validation checks pass โ†’ active | -| `human_review` | One platform admin must explicitly approve | Submitted โ†’ validated โ†’ pending_approval โ†’ one admin approves โ†’ active | -| `dual_approval` | Two platform admins must independently approve | Submitted โ†’ validated โ†’ pending_approval โ†’ two admins approve โ†’ active | -| `committee` | A declared DCMGroup must reach quorum | Submitted โ†’ validated โ†’ pending_approval โ†’ committee votes โ†’ quorum โ†’ active | +| `reviewed` | One platform admin must explicitly approve | Submitted โ†’ validated โ†’ pending_approval โ†’ one admin approves โ†’ active | +| `verified` | Two platform admins must independently approve | Submitted โ†’ validated โ†’ pending_approval โ†’ two admins approve โ†’ active | +| `authorized` | N members of a declared DCMGroup must record decisions via the Admin API; quorum tracked by DCM; deliberation process is the organization's responsibility | Submitted โ†’ validated โ†’ pending_approval โ†’ DCMGroup members record votes via Admin API (or external systems calling API) โ†’ quorum โ†’ active | ### 3.2 Effective Approval Method Resolution @@ -179,34 +181,34 @@ effective_method = most_restrictive( Resolution rules: - Profile minimum overrides provider type default (always upward; profiles can only tighten) - A valid registration token can relax the effective method to `auto` ONLY if the profile's `allow_token_auto_approval` is true -- `committee` cannot be relaxed by any token +- `authorized` cannot be relaxed by any token ### 3.3 Profile Registration Policy Defaults ```yaml profile_registration_policy: minimal: - min_approval_method: human_review + min_approval_method: reviewed allow_token_auto_approval: true # token can enable auto for any type require_sovereignty_declaration: false require_health_check_before_approval: false dev: - min_approval_method: human_review + min_approval_method: reviewed allow_token_auto_approval: true require_sovereignty_declaration: false require_health_check_before_approval: true standard: - min_approval_method: human_review + min_approval_method: reviewed allow_token_auto_approval: true # tokens can auto-approve non-elevated types token_auto_approval_max_trust: standard # tokens cannot auto-approve elevated types require_sovereignty_declaration: true require_health_check_before_approval: true prod: - min_approval_method: human_review - high_trust_types_require: dual_approval # storage, auth, policy-mode3-4, credential + min_approval_method: reviewed + high_trust_types_require: verified # storage, auth, policy-mode3-4, credential allow_token_auto_approval: false # no auto-approval in prod require_sovereignty_declaration: true require_accreditation_submission: true # must submit at least self_declared @@ -214,7 +216,7 @@ profile_registration_policy: approval_timeout: P7D # auto-reject if not approved within 7 days fsi: - min_approval_method: dual_approval # everything requires dual approval + min_approval_method: verified # everything requires dual approval allow_token_auto_approval: false require_sovereignty_declaration: true require_accreditation_submission: true @@ -224,14 +226,14 @@ profile_registration_policy: approval_timeout: P14D sovereign: - min_approval_method: committee # everything requires committee approval + min_approval_method: authorized # everything requires authorized approval allow_token_auto_approval: false require_sovereignty_declaration: true require_accreditation_submission: true minimum_accreditation_type: regulatory_certification require_hardware_attestation: true require_governance_matrix_check: true - committee_group_handle: "platform/registration-committee" + authorized_group_handle: "platform/registration-authorized" approval_timeout: P30D ``` @@ -341,7 +343,7 @@ Approval flow depends on effective_approval_method: **auto:** Registration immediately advances to ACTIVE after validation passes. -**human_review:** +**reviewed:** ``` Registration enters PENDING_APPROVAL Platform admin notification dispatched (urgency: medium) @@ -354,7 +356,7 @@ On rejection: โ†’ REJECTED with required reason field On timeout (approval_timeout): โ†’ REJECTED with reason "approval_timeout" ``` -**dual_approval:** +**verified:** ``` Registration enters PENDING_APPROVAL Two independent platform admins must approve @@ -364,10 +366,10 @@ Same actor cannot approve twice On timeout: โ†’ REJECTED ``` -**committee:** +**authorized:** ``` Registration enters PENDING_APPROVAL -Committee DCMGroup notified (all members) +Authority group notified (all members) Members vote via Admin API within declared quorum window Quorum reached: โ†’ ACTIVE Quorum not reached within approval_timeout: โ†’ REJECTED @@ -676,10 +678,10 @@ Remote DCM requests federation peering โ–ผ Approval flow (per profile): โ”‚ dev: provisional auto-promoted to verified (if governance matrix permits) - โ”‚ standard: human_review for verified promotion; provisional gets limited scope - โ”‚ prod: dual_approval for verified promotion; no provisional operations - โ”‚ fsi: dual_approval + accreditation check; no provisional - โ”‚ sovereign: committee_approval + hardware attestation; no provisional + โ”‚ standard: reviewed for verified promotion; provisional gets limited scope + โ”‚ prod: verified for verified promotion; no provisional operations + โ”‚ fsi: verified + accreditation check; no provisional + โ”‚ sovereign: authorized_approval + hardware attestation; no provisional โ–ผ Scope assignment per trust posture @@ -704,19 +706,19 @@ profile_federation_policy: standard: permitted_trust_postures: [verified, vouched] - approval_method_for_verified: human_review + approval_method_for_verified: reviewed cross_jurisdiction_permitted: true accreditation_required_for_federation: false prod: permitted_trust_postures: [verified] - approval_method_for_verified: dual_approval + approval_method_for_verified: verified cross_jurisdiction_permitted: true accreditation_required_for_federation: false fsi: permitted_trust_postures: [verified] - approval_method_for_verified: dual_approval + approval_method_for_verified: verified cross_jurisdiction_permitted: false accreditation_required_for_federation: true minimum_peer_accreditation: third_party @@ -724,7 +726,7 @@ profile_federation_policy: sovereign: permitted_trust_postures: [verified] - approval_method_for_verified: committee + approval_method_for_verified: authorized cross_jurisdiction_permitted: false accreditation_required_for_federation: true minimum_peer_accreditation: sovereign_authorization @@ -805,7 +807,7 @@ Provider status โ†’ DEREGISTERED ``` POST /api/v1/admin/providers/{provider_uuid}/force-deregister Role: platform_admin -Requires: dual_approval (fsi/sovereign: committee) +Requires: verified (fsi/sovereign: authorized) Immediate effect: Provider status โ†’ FORCED_DEREGISTERED diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index c0f060b..57a2deb 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -287,7 +287,7 @@ | SMX-003 | Actor Risk History Tracking | View own risk history score and contributing events via Consumer API | โ€” | Monitor actor risk history; reset scores for trusted automation accounts; configure decay parameters | AUD-001, IAM-001 | | SMX-004 | Quota Pressure Scoring | Receive quota_pressure as a score driver when approaching Tenant quota limits | โ€” | Configure per-resource-type quota limits; manage free_threshold parameter | IAM-007, REQ-004 | | SMX-005 | Provider Accreditation Richness Scoring | โ€” | Benefit from lower risk contribution by maintaining rich accreditation portfolio | Configure accreditation richness weights; manage portfolio scoring | ACC-001, PRV-001 | -| SMX-006 | Profile Scoring Threshold Management | โ€” | โ€” | Configure auto_approve/human_review/dual_approval/committee thresholds per profile; manage signal weights; enforce SMX-008 (max auto_approve_below: 50) | POL-005, REQ-004 | +| SMX-006 | Profile Scoring Threshold Management | โ€” | โ€” | Configure approval routing thresholds per profile (auto/reviewed/verified/authorized + custom tiers via named-tier list); manage signal weights; enforce SMX-008 (max auto_approve_below: 50) | POL-005, REQ-004 | | SMX-007 | Policy Enforcement Class Override | โ€” | Contribute policies with declared enforcement_class; receive notification when profile overrides enforcement class | Declare per-profile enforcement class overrides; manage regulatory_mandate flag to protect compliance-class policies from demotion | POL-004, POL-005 | | SMX-008 | Score Audit Trail | Query risk score and routing decision for own requests; view score_drivers and advisory_warnings | โ€” | Query full Score Record detail including signal breakdown and actor risk history; manage score audit retention | AUD-001, REQ-004 | @@ -309,6 +309,36 @@ --- +## 23. Credential Provider Model + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| CPX-001 | Resource Credential Issuance | Receive credential metadata and retrieval URL with realized resource; retrieve credential value via authenticated endpoint | Declare credential requirements in Resource Type Spec; receive credential issuance confirmation | Register Credential Provider; configure credential types and lifetimes per resource type; manage issuance policies | PRV-005, ZTS-002 | +| CPX-002 | DCM Interaction Credential Issuance | โ€” | Validate scoped interaction credential on every DCM dispatch; reject interactions without valid scoped credential (CPX-002) | Configure interaction credential lifetime per profile; manage Credential Provider for DCM-internal use | ZTS-002, PRV-001 | +| CPX-003 | Credential Rotation | Receive rotation notification before old credential expires; retrieve new credential during transition window | Implement rotate endpoint; honor transition window; notify DCM when rotation is complete | Configure rotation schedules and transition windows per credential type; manage pre-expiry rotation warnings | CPX-001, IAM-001 | +| CPX-004 | Emergency Rotation and Security Event Response | Receive immediate notification on emergency rotation; retrieve new credential via fastest channel | Implement immediate revocation with no transition window on security_event trigger | Configure security event triggers; manage emergency rotation audit trail; notify platform admin | CPX-003, OBS-004 | +| CPX-005 | Credential Revocation | Receive revocation notification when credentials are revoked (actor deprovisioned, entity decommissioned); confirm transition to new credential | Implement revoke endpoint with declared SLA; invalidate value immediately on emergency revocation | Manage Credential Revocation Registry; configure revocation cache TTL per profile (PT1M standard, PT30S fsi/sovereign); enforce CPX-007 (decommission blocks on credential revocation) | CPX-001, LCM-007 | +| CPX-006 | Revocation Propagation | โ€” | Refresh revocation cache within profile-governed TTL; validate credential UUID against cache at use time (not only at receipt) | Configure revocation cache TTL; monitor revocation propagation latency; alert on SLA violations | CPX-005, IAM-001 | +| CPX-007 | Audit Trail for Credential Lifecycle | View own credential record history (issue, rotate, revoke events); every value retrieval audited with retrieval_uuid | โ€” | Query full credential audit trail including retrieval count; manage credential audit retention | CPX-001, AUD-001 | + +--- + + +## 24. Authority Tier Model + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ATM-001 | Authority Tier Registry | Reference approval decisions by tier name; tier weight resolved dynamically from ordered list | Implement approval workflows that reference tier names (not hardcoded weights) | Manage the ordered authority tier list; control tier positions and gravity values | POL-005 | +| ATM-002 | Custom Tier Definition | โ€” | โ€” | Contribute custom tiers between existing tiers; declare decision_gravity and dcm_gate semantics; requires verified-tier approval | ATM-001, FCM-001 | +| ATM-003 | Dynamic Threshold Configuration | View which tier an action routes to | โ€” | Configure profile approval_routing as named-tier threshold list; adjust score ranges when new tiers inserted | ATM-001, SMX-001 | +| ATM-004 | Tier Registry Change Impact Detection | Receive notification when tier changes affect owned resources or pending approvals | โ€” | Propose tier registry changes; receive tier impact diff report; review SECURITY_DEGRADATION and BROKEN_REFERENCE items; accept degradations via Admin API | ATM-001, AUD-001 | +| ATM-005 | Degradation Review Gate | โ€” | โ€” | Review and explicitly accept each SECURITY_DEGRADATION item before a registry change activates; provide compensating control rationale; must hold verified or authorized tier reviewer role | ATM-004, IAM-001 | +| ATM-006 | Profile Gap Detection | โ€” | โ€” | Receive PROFILE_GAP warnings when tier registry changes leave profile threshold lists incomplete; update threshold lists or acknowledge gap within approval window | ATM-003, ATM-004 | +| ATM-007 | Tier Registry Audit Trail | Query historical tier registry versions and impact reports | โ€” | Access full audit trail of all tier registry changes: proposal, impact assessment, degradation acceptances, activation | ATM-004, AUD-001 | + +--- + + ## Capability Count Summary | Domain | Capabilities | @@ -335,7 +365,9 @@ | Federated Contribution Model | 7 | | Scoring Model | 8 | | Meta Provider Composability | 7 | -| **Total** | **141** | +| Credential Provider Model | 7 | +| Authority Tier Model | 7 | +| **Total** | **155** | --- diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index e04aff1..8a883dc 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -80,6 +80,23 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture + +### Credential Provider Terms + +| Term | Definition | +|------|-----------| +| **Credential Record** | DCM Data artifact storing credential metadata (UUID, type, scope, expiry, status). Never contains the credential value โ€” values are held by the Credential Provider. | +| **DCM Interaction Credential** | Short-lived (PT15Mโ€“PT1H profile-governed), scoped credential issued before every provider dispatch. Implements ZTS-002. Never stored beyond the interaction window. | +| **Credential Revocation Registry** | Fast-queryable store of revoked credential UUIDs. All components that receive interaction credentials must check this registry at each use. Cache TTL: PT1M standard; PT30S fsi/sovereign. | +| **Transition Window** | Period during rotation when both the old and new credential are valid. Prevents downtime. P1D for consumer credentials; PT5M for dcm_interaction; P7D for x509. | +| **Emergency Rotation** | Rotation triggered by a security event. No transition window โ€” old credential revoked immediately. Fastest-channel notification delivery. | +| **CPX-001โ€“CPX-012** | Credential Provider system policies. Key: CPX-001 (values never in DCM stores โ€” every profile), CPX-002 (every dispatch must present scoped interaction credential), CPX-009 (algorithm and key_usage declared at issuance; validated at use), CPX-012 (CPX-001 applies in ALL profiles โ€” no exceptions). | +| **credential_profile** | Profile-governed credential configuration block controlling: permitted credential types, max lifetime per type, rotation requirements, retrieval auth level (bearer/step-up-mfa/mtls), FIPS level enforcement, approved algorithms, revocation SLA, idle detection threshold, IP binding requirement. | +| **AAL (Authenticator Assurance Level)** | NIST 800-63B vocabulary: AAL1 (minimal/dev โ€” single factor), AAL2 (standard/prod โ€” MFA required for sensitive credentials), AAL2+ (fsi โ€” hardware MFA, FIPS L2), AAL3 (sovereign โ€” hardware-bound, FIPS L3, tamper evidence). | +| **Idle Credential** | A credential issued but not retrieved within the profile-governed threshold. Triggers notification but not automatic revocation. Auto-revocation after 2ร— threshold is profile-configurable. | +| **key_usage** | Declared purpose of a credential: authentication, signing, or encryption. Non-overlapping โ€” a credential issued for authentication cannot be used for signing even if the algorithm supports both. Validated at use time by Credential Provider. | + + ### Meta Provider Composability Terms | Term | Definition | @@ -94,6 +111,45 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture | **MPX-001โ€“MPX-008** | Meta Provider system policies. Key: MPX-001 (compensation required if partial delivery supported), MPX-002 (dependency-reverse decommission), MPX-006 (DEGRADED is a valid terminal state when accepted), MPX-008 (compound payload fully assembled by DCM before dispatch). | + + +### Authority Tier Model Terms + +| Term | Definition | +|------|-----------| +| **Authority Tier Registry** | The ordered list of authority tiers that governs approval routing across all DCM pipelines. Stored as a versioned registry entry. Changes require impact detection before activation. | +| **Tier Impact Diff** | Computed before any tier registry change activates. Compares proposed ordered list to current list; classifies each changed tier as SECURITY_DEGRADATION, BROKEN_REFERENCE, PROFILE_GAP, SECURITY_UPGRADE, or NEW. | +| **SECURITY_DEGRADATION** | Impact classification for a tier whose gravity or position decreased after a registry change. Blocks registry activation until explicitly accepted by a verified-tier or above reviewer (ATM-009). | +| **BROKEN_REFERENCE** | Impact classification when a tier name referenced in active configuration no longer exists in the registry. Blocks activation until resolved (ATM-010). | +| **PROFILE_GAP** | Impact classification when a profile's threshold list is incomplete after new tier insertion. Warning only โ€” does not block activation (ATM-012). | Stored as a versioned registry entry. Custom tiers are inserted into the list by position; existing tier names remain stable. | +| **decision_gravity** | Stable, position-independent severity classification on each tier: `none` (auto), `routine` (reviewed), `elevated` (verified), `critical` (authorized). Used by the scoring model and profile system to reason about tier severity independently of tier names. | +| **Tier Weight** | Numeric value derived from a tier's position in the ordered list. Never hardcoded โ€” resolved at evaluation time. Stored in approval records for point-in-time audit (ATM-008). | +| **Custom Tier** | An organization-defined tier inserted between existing tiers. Must declare `decision_gravity` consistent with position. Requires `verified` tier approval to contribute (ATM-004). | +| **ATM-001โ€“ATM-008** | Authority Tier Model system policies. Key: ATM-001 (tiers identified by name; weight derived from position), ATM-002 (auto tier max_score โ‰ค 50), ATM-003 (custom tier gravity must be consistent with position), ATM-008 (approval records store weight at creation for point-in-time audit). | + + +### Authority Tier Terms + +| Term | Definition | +|------|-----------| +| **Authority Tier** | The required organizational authority level for a decision, expressed as a named position in an ordered list. DCM defines four tiers; organizations define what constitutes sufficient authority at each level. | +| **`auto`** | No human judgment required. System confidence (scoring, validation) is sufficient to proceed. | +| **`reviewed`** | Standard authority required. One qualified reviewer in the relevant domain must record a decision via the DCM Admin API. Who constitutes a qualified reviewer is the organization's definition. | +| **`verified`** | Elevated authority required. Two independent, distinct reviewers must each record a decision. The same actor cannot satisfy both. Enforces separation of duties. | +| **`authorized`** | Highest authority level required. Most consequential decisions โ€” policy governance changes, regulated actions, high-risk provider registrations. N members of a declared DCMGroup must record decisions via the Admin API. The authority group composition (CTO, CISO, security board, one person with delegated authority) is entirely the organization's definition. | +| **DCMGroup (authority context)** | A declared set of actors who constitute the required authority for `authorized` tier decisions. Platform admin declares membership; quorum threshold (N of M) is profile-governed. | +| **`recorded_via`** | Audit field on approval decisions capturing which system submitted the decision (dcm_admin_ui, servicenow, jira, slack_bot, api_direct). Informational for audit; not enforced. | + + +### Design Priority Terms + +| Term | Definition | +|------|-----------| +| **Design Priority Order** | The four-priority hierarchy governing all DCM design decisions: (1) Security โ€” industry best practices are the baseline; (2) Ease of use โ€” secure path must be easy path; (3) Extensibility โ€” adaptable through configuration not code; (4) Fit for purpose โ€” always required. | +| **DPO-001โ€“006** | Design Priority system policies. Key: DPO-001 (security properties present in all profiles), DPO-002 (every security requirement needs ease-of-use mechanism), DPO-005 (`minimal` profile = minimal overhead not minimal security), DPO-006 (when security and ease conflict, redesign ease-of-use not security). | +| **Priority 1 (Security)** | Security properties are architecturally present in ALL profiles. Profiles control enforcement strictness and automation level โ€” never whether the property exists. Non-negotiable across all profiles: CPX-001, SMX-004, SMX-008, CPX-003, CPX-005 first retrieval audit, forbidden algorithm baseline. | +| **Priority 2 (Ease of use)** | The secure path must also be the easy path. Every security requirement must be accompanied by an ease-of-use mechanism. The scoring model auto-approval threshold, profile defaults, and Flow GUI visual condition builder are all Priority 2 implementations. | + ### Scoring Model Terms | Term | Definition | @@ -106,7 +162,7 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture | **actor_risk_history_score** | Decay-weighted (ฮป=0.1, half-life โ‰ˆ7 days) history of an actor's previous request outcomes. Contributes to request risk score. Not exposed to other consumers. | | **quota_pressure_score** | Continuous score representing how close a Tenant is to quota limits for the requested resource type. Zero below 75% utilization; 100 at full quota. | | **accreditation_richness_score** | Weighted sum of a provider's accreditation portfolio normalized to 0โ€“100. Influences placement preference and inversely contributes to provider risk signal. | -| **scoring_threshold** | Profile-governed boundary on the request risk score that maps to an approval routing tier. Four tiers: auto_approve, human_review, dual_approval, committee. `auto_approve_below` may not exceed 50 (SMX-008). | +| **scoring_threshold** | Profile-governed boundary on the request risk score that maps to an approval routing tier. Four tiers: auto_approve, reviewed, verified, authorized. `auto_approve_below` may not exceed 50 (SMX-008). | | **Risk Score Aggregator** | Sub-function of the Policy Engine. Assembles five scoring signals into the request risk score after all compliance-class and Governance Matrix evaluations complete. | | **regulatory_mandate** | Policy metadata flag. When `true`, the policy's `enforcement_class: compliance` cannot be demoted to operational by any profile (SMX-003). Set by platform admins, audited. | | **score_drivers** | Human-readable list of the top contributing factors to a request risk score. Exposed to consumers (top 3 only). Full breakdown in Score Record for platform admins. | @@ -181,6 +237,9 @@ Terms to avoid because they introduce ambiguity. Use the precise alternatives in | FCM | Federated Contribution Model | | SMX | Scoring Model | | MPX | Meta Provider Composability | +| CPX | Credential Provider Model | +| DPO | Design Priority Order | +| ATM | Authority Tier Model | --- From e87c46c5cba326cf5c7a970ed8956af1863ecf76 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sun, 29 Mar 2026 15:29:47 -0500 Subject: [PATCH 43/49] More details on the webhook spec Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 43 +- .../architecture/data-model/event-catalog.md | 749 ++++++++++++++++++ .../data-model/notification-model.md | 3 + .../data-model/webhooks-messaging.md | 3 + content/docs/capabilities-matrix.md | 18 +- content/docs/taxonomy.md | 13 + 6 files changed, 827 insertions(+), 2 deletions(-) create mode 100644 content/docs/architecture/data-model/event-catalog.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 5966f9a..838e117 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4458,7 +4458,47 @@ Peer DCM instances may have different custom tier lists. Resolution strategy: `g --- -## SECTION 65 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 65 โ€” EVENT CATALOG (doc 33 โ€” 33-event-catalog.md) + +> **Full specification:** [33-event-catalog.md](data-model/33-event-catalog.md) โ€” authoritative source for all 82 DCM event types, payload schemas, urgency levels, EVT-001โ€“EVT-007 system policies. + +### Base Envelope (all events share this) +event_uuid (idempotency key โ€” EVT-002: consumers must treat duplicates as already-processed) ยท event_type ยท event_schema_version ยท timestamp (from Commit Log โ€” authoritative) ยท dcm_version ยท dcm_instance_uuid ยท subject (entity_uuid, entity_type, entity_handle, tenant_uuid, actor_uuid) ยท urgency (critical/high/medium/low/info) ยท payload (event-specific) ยท links (self, audit_record) + +### Event Domains (82 total across 20 domains) +request.* (14): submitted โ†’ intent_captured โ†’ layers_assembled โ†’ policies_evaluated โ†’ requires_approval โ†’ approved โ†’ placement_complete โ†’ dispatched โ†’ compound_assembled โ†’ dependencies_resolved โ†’ realized/failed/gatekeeper_rejected/cancelled +entity.* (13): realized, state_changed, modified, ttl_warning, ttl_expired, suspended, resumed, decommissioning, decommissioned, decommission_deferred, ownership_transferred, pending_review, expired +drift.* (4): detected, severity_escalated, resolved, escalated +provider.* (5): registered, deregistered, healthy, unhealthy, degraded +provider_update.* (5): submitted, requires_approval, approved, rejected, auto_approved +rehydration.* (5): started, paused, interrupted, completed, blocked +policy.* (4): activated, deactivated, evaluated, shadow_result +credential.* (4): rotating, revoked, idle, expired +approval.* (4): decision_recorded, quorum_reached, window_expiring, expired +tier_registry.* (4): proposed, impact_assessed, degradation_detected, activated +audit.* (3): chain_integrity_alert, chain_break, forward_failed +dependency.* (2): state_changed + stakeholder.resource_decommissioning +allocation.* (2): pool_capacity_low, released +ingestion.* (3): transitional_created, enriched, promotion_approved +governance.* (3): catalog_item_deprecated, profile_changed, policy_trust_elevated +security.*/sovereignty.*/federation.*/auth.*: unsanctioned_provider_write, sovereignty.violation, sovereignty.migration_required, federation.tunnel_degraded, auth.provider_failover + +### Urgency Levels +critical (push + page if configured) ยท high (push) ยท medium (standard) ยท low (standard) ยท info (batch/webhook only) +critical + audit.* events: NON-SUPPRESSABLE โ€” EVT-005 and EVT-007 + +### Schema Versioning +event_schema_version increments ONLY on breaking changes (removing fields, changing types/semantics). Adding optional fields is NOT breaking. EVT-004. + +### Non-Standard Events +Providers/extensions may publish non-standard events using reverse-DNS prefix (e.g. com.acme.custom_event). EVT-006. + +### EVT-001โ€“EVT-007 System Policies +EVT-001: all events must include base envelope. EVT-002: event_uuid is idempotency key. EVT-003: timestamp from Commit Log. EVT-004: schema version increments on breaking changes only. EVT-005: critical urgency โ†’ push delivery. EVT-006: non-standard events use reverse-DNS prefix. EVT-007: audit.* critical events are non-suppressable. + +--- + +## SECTION 66 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure): @@ -4478,6 +4518,7 @@ When working on this project, apply these instructions in addition to the number 186. **Credential values are NEVER stored in DCM** (CPX-001) โ€” only metadata is stored; values are held by the Credential Provider; retrieved via authenticated endpoint; this applies to ALL credential types including dcm_interaction credentials 187. **Every provider dispatch requires a scoped interaction credential** (CPX-002) โ€” issued before dispatch, scoped to the specific operation+entity+provider, expires PT15M; provider must validate at use time not just receipt; check revocation cache on each use 189. **Security properties are present in ALL profiles โ€” minimal profile is "security with minimal operational overhead" not "minimal security"** โ€” rotation required in all profiles (minimal: P365D max, manual OK); idle detection on in all profiles (minimal: P30D); algorithm baseline in all profiles (minimal: forbidden list); CPX-001 (values never in DCM stores) is absolute โ€” homelab (minimal) uses bearer_token retrieval, no scheduled rotation, no FIPS; sovereign uses mtls+hardware attestation, FIPS Level 3, PT15S revocation cache; same API contract, same data model, same CPX-001 (values never in DCM stores) +195. **33-event-catalog.md is the SINGLE authoritative source for all DCM event types** โ€” 82 events across 20 domains; all events share the base envelope (event_uuid, event_type, event_schema_version, timestamp from Commit Log, urgency, payload, links); consumers implement idempotency using event_uuid; critical urgency events are non-suppressable; non-standard events use reverse-DNS prefix; event_schema_version only increments on breaking changes 194. **Tier registry changes are gated by impact detection** โ€” any change that creates a SECURITY_DEGRADATION (tier gravity or position decreased) blocks activation until each degradation is explicitly accepted by a verified-tier or above reviewer via Admin API; BROKEN_REFERENCE also blocks; PROFILE_GAP is a warning that does not block; all changes produce an impact report in the Audit Store (ATM-009โ€“012) 193. **Authority tiers are named positions in an ordered list โ€” not fixed enum values** โ€” tier weight derived from list position at evaluation time; organizations insert custom tiers between existing ones without breaking existing name references; 'authorized' tier always means 'highest current gravity' regardless of what's been inserted before it; ATM-001: never hardcode tier weights 192. **DCM provides the approval gate and audit trail โ€” the review process is the organization's responsibility** โ€” for authorized tier: DCM tracks quorum of a DCMGroup; the authorized deliberation and vote collection happen outside DCM; external systems (ServiceNow, Jira, Slack bots) can call Admin API to record votes; DCM does NOT build authorized management; for reviewed and verified: same principle โ€” DCM holds the pipeline until the API receives the required decisions diff --git a/content/docs/architecture/data-model/event-catalog.md b/content/docs/architecture/data-model/event-catalog.md new file mode 100644 index 0000000..7849ae8 --- /dev/null +++ b/content/docs/architecture/data-model/event-catalog.md @@ -0,0 +1,749 @@ +--- +title: "DCM Event Catalog" +type: docs +weight: 33 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” Authoritative Event Catalog +**Related Documents:** [Notification Model](23-notification-model.md) | [Webhooks and Messaging](18-webhooks-messaging.md) | [Universal Audit](16-universal-audit.md) | [Credential Provider Model](31-credential-provider-model.md) | [Authority Tier Model](32-authority-tier-model.md) | [Control Plane Components](25-control-plane-components.md) + +> **This is the single authoritative source for all DCM event types.** +> +> The Notification Model (doc 23) defines delivery pipeline, audience resolution, and urgency routing. The Webhooks doc (doc 18) defines the Message Bus integration. This document defines **what events exist, when they fire, and what their payloads contain**. Any document referencing an event type is authoritative only if it agrees with this catalog. Conflicts resolve in favor of this document. + +> **Implementation note:** Consumers (webhook receivers, Notification Providers, Message Bus subscribers, audit tooling) must implement idempotency using `event_uuid`. Events are delivered at-least-once. Per-entity ordering is guaranteed; cross-entity ordering is not. + +--- + +## 1. Base Envelope + +Every DCM event shares a common envelope. Event-specific fields are in the `payload` object. + +```yaml +# DCM Event Envelope โ€” all events +event_uuid: # idempotency key; stable across retries +event_type: # fully qualified: domain.event_name +event_schema_version: "1.0" # increments on breaking payload changes +timestamp: # from Commit Log โ€” authoritative source of truth +dcm_version: # DCM instance version that generated the event +dcm_instance_uuid: # identifies the DCM instance (federation context) + +subject: + entity_uuid: # primary entity this event concerns + entity_type: # entity type FQN (e.g. Compute.VirtualMachine) + entity_handle: # human-readable identifier + tenant_uuid: # tenant scope; null for system-scope events + actor_uuid: # actor who triggered the event; null for system events + +urgency: critical | high | medium | low | info # governs notification routing + +payload: {} # event-specific fields โ€” see Section 3+ + +links: + self: # DCM API URL for the subject entity or record + audit_record: # DCM API URL for the audit record for this event +``` + +### 1.1 Urgency Levels + +| Urgency | Meaning | Delivery expectation | +|---------|---------|---------------------| +| `critical` | Security or compliance event requiring immediate action | Push notification; page if configured | +| `high` | Significant operational event; action likely required | Push notification | +| `medium` | Notable event; review recommended | Standard delivery | +| `low` | Informational; action unlikely required | Standard delivery | +| `info` | Observational; no action expected | Batch or webhook only | + +### 1.2 Schema Versioning + +`event_schema_version` increments when breaking changes occur to the `payload` schema for an event type. Consumers should validate against the declared version. Non-breaking additions (new optional fields) do not increment the version. + +--- + +## 2. Event Domain Index + +| Domain | Events | Description | +|--------|--------|-------------| +| `request.*` | 14 | Request pipeline lifecycle | +| `entity.*` | 13 | Resource entity lifecycle | +| `drift.*` | 4 | Drift detection and resolution | +| `provider.*` | 5 | Provider registration and health | +| `provider_update.*` | 5 | Provider-initiated update lifecycle | +| `rehydration.*` | 5 | Entity rehydration lifecycle | +| `policy.*` | 4 | Policy contribution lifecycle | +| `credential.*` | 4 | Credential lifecycle | +| `approval.*` | 4 | Approval pipeline | +| `tier_registry.*` | 4 | Authority tier registry changes | +| `audit.*` | 3 | Audit chain integrity | +| `dependency.*` | 2 | Entity dependency events | +| `stakeholder.*` | 1 | Stakeholder notifications | +| `allocation.*` | 2 | Resource allocation events | +| `ingestion.*` | 3 | Brownfield ingestion lifecycle | +| `governance.*` | 3 | Catalog and profile governance | +| `security.*` | 2 | Security and sovereignty events | +| `sovereignty.*` | 2 | Sovereignty constraint events | +| `federation.*` | 1 | Federation tunnel events | +| `auth.*` | 1 | Authentication provider events | +| **Total** | **82** | | + +--- + +## 3. Request Events (`request.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `request.submitted` | info | Consumer submitted a request via API or UI | +| `request.intent_captured` | info | Intent State created; entity UUID assigned | +| `request.layers_assembled` | info | Layer assembly complete; compound payload ready for policy evaluation | +| `request.policies_evaluated` | info | Policy evaluation complete; score computed; routing tier determined | +| `request.requires_approval` | medium | Score routed to `reviewed`, `verified`, or `authorized` tier; pipeline holds | +| `request.approved` | info | Required tier approval recorded; pipeline resumes | +| `request.placement_complete` | info | Provider placement complete; Requested State committed | +| `request.dispatched` | info | Payload dispatched to provider(s) | +| `request.compound_assembled` | info | Compound service payload assembled (Meta Provider compound request) | +| `request.dependencies_resolved` | info | Constituent dependencies resolved (Meta Provider) | +| `request.realized` | medium | Provider confirmed realization; Realized State written | +| `request.failed` | high | Request failed at any stage | +| `request.gatekeeper_rejected` | high | GateKeeper policy denied the request | +| `request.cancelled` | low | Consumer cancelled; pipeline terminated | + +### 3.1 Payload Schemas + +#### `request.submitted` / `request.intent_captured` +```yaml +payload: + request_uuid: + catalog_item_uuid: + catalog_item_handle: + resource_type: # FQN e.g. Compute.VirtualMachine + submitted_fields: {} # consumer-declared fields (may be partial) +``` + +#### `request.layers_assembled` / `request.policies_evaluated` +```yaml +payload: + request_uuid: + risk_score: <0-100> # present after policies_evaluated + routing_tier: auto | reviewed | verified | authorized | + score_drivers: # top contributing signals + - signal: operational_gatekeeper + contribution: 12 +``` + +#### `request.requires_approval` +```yaml +payload: + request_uuid: + approval_uuid: + required_tier: reviewed | verified | authorized | + required_tier_gravity: routine | elevated | critical + risk_score: <0-100> + window_expires_at: + dcmgroup_uuid: # non-null for authorized tier + quorum_required: +``` + +#### `request.realized` / `request.failed` +```yaml +payload: + request_uuid: + provider_uuid: + outcome: realized | failed | degraded + failure_reason: + realized_fields: {} # key provider-returned values (IP, VM ID, etc.) + composite_status: # for compound requests +``` + +#### `request.gatekeeper_rejected` +```yaml +payload: + request_uuid: + policy_handle: + enforcement_class: compliance | operational + rejection_reason: + risk_score: <0-100> +``` + +--- + +## 4. Entity Lifecycle Events (`entity.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `entity.realized` | medium | Entity first realized; Realized State written | +| `entity.state_changed` | medium | Entity lifecycle state transition | +| `entity.modified` | info | Entity fields updated (Day-2 operation) | +| `entity.ttl_warning` | medium | TTL expires within declared warning window | +| `entity.ttl_expired` | high | TTL reached; expiry action triggered | +| `entity.suspended` | high | Entity entered SUSPENDED state | +| `entity.resumed` | medium | Entity exited SUSPENDED state | +| `entity.decommissioning` | medium | Decommission pipeline initiated | +| `entity.decommissioned` | low | Entity fully decommissioned; resources released | +| `entity.decommission_deferred` | medium | Decommission blocked by active stakes | +| `entity.ownership_transferred` | medium | Ownership moved to a different Tenant | +| `entity.pending_review` | medium | Entity entered PENDING_REVIEW state | +| `entity.expired` | high | Entity reached terminal expired state | + +### 4.1 Payload Schemas + +#### `entity.realized` +```yaml +payload: + request_uuid: + provider_uuid: + realized_fields: {} # key fields returned by provider + composite_entity: # true for Meta Provider compound services + composite_status: +``` + +#### `entity.state_changed` +```yaml +payload: + previous_state: + new_state: + triggered_by: ttl | decommission | consumer | policy | provider | system + reason: +``` + +#### `entity.ttl_warning` / `entity.ttl_expired` +```yaml +payload: + ttl_expires_at: + expiry_action: decommission | suspend | notify_only + warning_window: # e.g. P7D +``` + +#### `entity.decommissioning` / `entity.decommissioned` +```yaml +payload: + initiated_by: + initiated_at: + reason: + stakes_resolved: + credential_revocation_status: complete | partial | pending +``` + +#### `entity.decommission_deferred` +```yaml +payload: + blocking_stakes: + - stake_uuid: + stake_type: required | management + stakeholder_tenant_uuid: + stakeholder_entity_uuid: + retry_after: +``` + +#### `entity.ownership_transferred` +```yaml +payload: + previous_owner_tenant_uuid: + new_owner_tenant_uuid: + transfer_reason: + transferred_by: +``` + +--- + +## 5. Drift Events (`drift.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `drift.detected` | high | Discovered State differs from Realized State | +| `drift.severity_escalated` | high | Drift severity increased (e.g. minor โ†’ significant) | +| `drift.resolved` | low | Drift resolved via REVERT or UPDATE_DEFINITION | +| `drift.escalated` | high | Drift escalated to human review | + +### 5.1 Payload Schemas + +#### `drift.detected` +```yaml +payload: + drift_record_uuid: + drift_severity: minor | moderate | significant | critical + drifted_fields: + - field: # field path e.g. "cpu_count" + realized_value: + discovered_value: + discovery_run_uuid: + discovered_at: +``` + +#### `drift.severity_escalated` +```yaml +payload: + drift_record_uuid: + previous_severity: minor | moderate | significant | critical + new_severity: minor | moderate | significant | critical + escalation_trigger: time_elapsed | field_count | field_sensitivity +``` + +#### `drift.resolved` +```yaml +payload: + drift_record_uuid: + resolution: REVERT | UPDATE_DEFINITION | MANUAL + resolved_by: + resolved_at: +``` + +--- + +## 6. Provider Events (`provider.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `provider.registered` | info | Provider successfully registered and activated | +| `provider.deregistered` | medium | Provider deregistered; active entities may be affected | +| `provider.healthy` | info | Provider health check returned healthy after unhealthy period | +| `provider.unhealthy` | high | Provider health check failed | +| `provider.degraded` | high | Provider reporting degraded capacity | + +### 6.1 Payload Schemas + +#### `provider.registered` / `provider.deregistered` +```yaml +payload: + provider_uuid: + provider_type: service_provider | meta_provider | credential_provider | auth_provider | ... + provider_handle: + resource_types_affected: [] # on deregistered: types now unserviced + active_entity_count: # on deregistered: entities at risk +``` + +#### `provider.unhealthy` / `provider.degraded` +```yaml +payload: + provider_uuid: + health_check_uuid: + failure_reason: + consecutive_failures: + last_healthy_at: + affected_resource_types: [] +``` + +--- + +## 7. Provider Update Events (`provider_update.*`) + +Provider-initiated update notifications โ€” when a provider reports a change to an entity it manages. + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `provider_update.submitted` | medium | Provider submitted an update notification for a realized entity | +| `provider_update.requires_approval` | medium | Provider update requires consumer approval before applying | +| `provider_update.approved` | info | Consumer approved; Realized State updated | +| `provider_update.rejected` | medium | Consumer rejected; update becomes tracked drift | +| `provider_update.auto_approved` | info | Update auto-approved per policy | + +### 7.1 Payload Schema + +```yaml +payload: + provider_update_uuid: + provider_uuid: + update_type: patch | deprecation | security_advisory | capacity_change + update_summary: + proposed_field_changes: {} # what the provider wants to change + approval_required: + approval_uuid: +``` + +--- + +## 8. Rehydration Events (`rehydration.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `rehydration.started` | info | Rehydration pipeline initiated | +| `rehydration.paused` | medium | Rehydration paused (e.g. waiting on dependent constituent) | +| `rehydration.interrupted` | high | Rehydration interrupted by error or cancellation | +| `rehydration.completed` | medium | All constituents rehydrated; entity OPERATIONAL | +| `rehydration.blocked` | high | Rehydration blocked โ€” provider unavailable or policy prevents | + +### 8.1 Payload Schema + +```yaml +payload: + rehydration_uuid: + trigger: ttl_expiry | manual | drift_recovery | system + constituents_total: + constituents_complete: + block_reason: # for rehydration.blocked +``` + +--- + +## 9. Policy Events (`policy.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `policy.activated` | medium | Policy promoted from shadow to active | +| `policy.deactivated` | medium | Policy deactivated | +| `policy.evaluated` | info | Policy evaluated against a request payload (shadow or active) | +| `policy.shadow_result` | info | Shadow evaluation diverged from expected outcome | + +### 9.1 Payload Schema + +#### `policy.activated` / `policy.deactivated` +```yaml +payload: + policy_uuid: + policy_handle: + policy_type: gatekeeper | validation | transformation | recovery | orchestration_flow + enforcement_class: compliance | operational # for gatekeeper + shadow_period_days: + approved_by: +``` + +#### `policy.shadow_result` +```yaml +payload: + policy_uuid: + request_uuid: + shadow_decision: allow | deny | transform + active_decision: allow | deny | transform # what active policies decided + diverged: + divergence_detail: +``` + +--- + +## 10. Credential Events (`credential.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `credential.rotating` | medium | Rotation initiated; transition window open | +| `credential.revoked` | high | Credential revoked; all holders must stop using | +| `credential.idle` | medium | Credential not retrieved within profile threshold | +| `credential.expired` | medium | Credential reached `valid_until`; no longer valid | + +### 10.1 Payload Schema + +#### `credential.rotating` +```yaml +payload: + credential_uuid: # old credential + new_credential_uuid: + rotation_trigger: pre_expiry | scheduled | security_event | actor_request + transition_window_ends: + retrieval_url: # where to retrieve new value +``` + +#### `credential.revoked` +```yaml +payload: + credential_uuid: + revocation_trigger: actor_deprovisioned | entity_decommissioned | security_event | ... + revocation_reason: + effective_at: + entity_uuid: +``` + +#### `credential.idle` +```yaml +payload: + credential_uuid: + credential_type: + issued_at: + threshold_elapsed: + retrieval_count: 0 +``` + +--- + +## 11. Approval Events (`approval.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `approval.decision_recorded` | info | A reviewer recorded an approve or reject decision | +| `approval.quorum_reached` | medium | Authorized tier quorum satisfied; pipeline resuming | +| `approval.window_expiring` | medium | Approval window approaching expiry (75% elapsed) | +| `approval.expired` | high | Approval window expired without decision | + +### 11.1 Payload Schema + +#### `approval.decision_recorded` +```yaml +payload: + approval_uuid: + subject_type: request | policy_contribution | provider_registration | federation_contribution + subject_uuid: + required_tier: reviewed | verified | authorized | + decision: approve | reject + voter_uuid: + recorded_via: dcm_admin_ui | servicenow | jira | slack_bot | api_direct | other + votes_recorded: + quorum_required: + quorum_reached: +``` + +#### `approval.expired` +```yaml +payload: + approval_uuid: + subject_type: + subject_uuid: + required_tier: + votes_recorded: + quorum_required: + expiry_action: reject | escalate +``` + +--- + +## 12. Tier Registry Events (`tier_registry.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `tier_registry.proposed` | medium | Tier registry change proposed; impact assessment starting | +| `tier_registry.impact_assessed` | medium | Tier impact diff complete; review may be required | +| `tier_registry.degradation_detected` | high | SECURITY_DEGRADATION items found; activation blocked | +| `tier_registry.activated` | medium | Tier registry change activated; new list in effect | + +### 12.1 Payload Schema + +#### `tier_registry.proposed` +```yaml +payload: + registry_change_uuid: + proposed_by: + tiers_added: [] + tiers_removed: [] + tiers_repositioned: [] +``` + +#### `tier_registry.impact_assessed` +```yaml +payload: + registry_change_uuid: + degradations: + broken_references: + profile_gaps: + upgrades: + activation_blocked: +``` + +#### `tier_registry.degradation_detected` +```yaml +payload: + registry_change_uuid: + affected_item_uuid: + affected_item_type: + tier_name: + old_gravity: none | routine | elevated | critical + new_gravity: none | routine | elevated | critical + acceptance_required_by: +``` + +--- + +## 13. Audit Events (`audit.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `audit.chain_integrity_alert` | critical | Hash chain verification failed; audit trail may be compromised | +| `audit.chain_break` | critical | Explicit break detected in audit hash chain | +| `audit.forward_failed` | high | Audit record failed to forward to external audit sink | + +### 13.1 Payload Schema + +#### `audit.chain_integrity_alert` +```yaml +payload: + affected_record_uuid: + expected_hash: + actual_hash: + chain_segment_start: + chain_segment_end: + records_in_segment: +``` + +--- + +## 14. Dependency and Stakeholder Events + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `dependency.state_changed` | medium | A dependency entity changed state; dependents may be affected | +| `stakeholder.resource_decommissioning` | medium | Resource this actor has a stake in is being decommissioned | +| `allocation.pool_capacity_low` | high | Allocation pool approaching capacity limit | +| `allocation.released` | info | Allocation returned to pool | + +### 14.1 Payload Schemas + +#### `dependency.state_changed` +```yaml +payload: + dependency_entity_uuid: + previous_state: + new_state: + dependent_entity_uuids: [] + impact_assessment: degraded | blocked | unaffected +``` + +#### `stakeholder.resource_decommissioning` +```yaml +payload: + resource_entity_uuid: + stake_type: required | management | informational + decommission_at: + action_required: # true for required stakes + action_url: +``` + +--- + +## 15. Ingestion Events (`ingestion.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `ingestion.transitional_created` | info | Brownfield entity created as Transitional entity | +| `ingestion.enriched` | info | Transitional entity enriched with additional data | +| `ingestion.promotion_approved` | medium | Transitional entity approved for promotion to full DCM entity | + +### 15.1 Payload Schema + +```yaml +payload: + ingestion_record_uuid: + source_system: + entity_handle: + confidence_level: high | medium | low + missing_fields: [] # for ingestion.enriched + promoted_entity_uuid: # for ingestion.promotion_approved +``` + +--- + +## 16. Governance Events (`governance.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `governance.catalog_item_deprecated` | medium | Service catalog item marked for deprecation | +| `governance.profile_changed` | high | Active profile configuration changed | +| `governance.policy_trust_elevated` | medium | Policy provider trust level elevated | + +### 16.1 Payload Schema + +#### `governance.profile_changed` +```yaml +payload: + previous_profile: + new_profile: + changed_by: + effective_at: + affected_threshold_tiers: [] +``` + +--- + +## 17. Security and Sovereignty Events + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `security.unsanctioned_provider_write` | critical | Provider wrote to an entity without a corresponding Requested State record | +| `sovereignty.violation` | critical | Data or operation crossed a declared sovereignty boundary | +| `sovereignty.migration_required` | high | Entity must migrate to comply with sovereignty constraints | +| `federation.tunnel_degraded` | high | Federation tunnel to peer DCM degraded or unavailable | +| `auth.provider_failover` | high | Auth Provider failed; failover to secondary | + +### 17.1 Payload Schemas + +#### `security.unsanctioned_provider_write` +```yaml +payload: + provider_uuid: + entity_uuid: + write_detected_at: + changed_fields: [] + discovery_run_uuid: +``` + +#### `sovereignty.violation` +```yaml +payload: + violation_type: data_boundary | operation_boundary | residency_requirement + constraint_uuid: + constraint_handle: + triggering_operation: + remediation_required: +``` + +--- + +## 18. System Policies + +| Policy | Rule | +|--------|------| +| `EVT-001` | Every event must include the base envelope fields (`event_uuid`, `event_type`, `event_schema_version`, `timestamp`, `dcm_version`, `dcm_instance_uuid`, `urgency`). Events omitting required envelope fields are invalid and must not be published. | +| `EVT-002` | `event_uuid` is the idempotency key. Consumers must treat duplicate `event_uuid` values as already-processed. DCM may re-deliver events on failure; this is not a bug. | +| `EVT-003` | `timestamp` is sourced from the Commit Log Stage 1 write. It represents when the event was authoritatively recorded, not when it was delivered. | +| `EVT-004` | `event_schema_version` must increment on any breaking change to a payload schema. Adding optional fields is not a breaking change. Removing fields, changing field types, or changing field semantics are breaking changes. | +| `EVT-005` | Events with `urgency: critical` must be delivered via the push channel if the Notification Provider supports it, regardless of consumer subscription preferences. | +| `EVT-006` | This catalog is the authoritative source for event type names. Any event type not in this catalog is non-standard. Non-standard events may be published by providers or extensions but must use a reverse-DNS prefix (e.g. `com.acme.custom_event`). | +| `EVT-007` | The `audit.*` events with `urgency: critical` are non-suppressable. They are delivered regardless of audience subscription rules and cannot be filtered by consumer preference. | + +--- + +## 19. Event Type Quick Reference + +``` +request.submitted request.intent_captured request.layers_assembled +request.policies_evaluated request.requires_approval request.approved +request.placement_complete request.dispatched request.compound_assembled +request.dependencies_resolved request.realized request.failed +request.gatekeeper_rejected request.cancelled + +entity.realized entity.state_changed entity.modified +entity.ttl_warning entity.ttl_expired entity.suspended +entity.resumed entity.decommissioning entity.decommissioned +entity.decommission_deferred entity.ownership_transferred entity.pending_review +entity.expired + +drift.detected drift.severity_escalated drift.resolved +drift.escalated + +provider.registered provider.deregistered provider.healthy +provider.unhealthy provider.degraded + +provider_update.submitted provider_update.requires_approval provider_update.approved +provider_update.rejected provider_update.auto_approved + +rehydration.started rehydration.paused rehydration.interrupted +rehydration.completed rehydration.blocked + +policy.activated policy.deactivated policy.evaluated +policy.shadow_result + +credential.rotating credential.revoked credential.idle +credential.expired + +approval.decision_recorded approval.quorum_reached approval.window_expiring +approval.expired + +tier_registry.proposed tier_registry.impact_assessed +tier_registry.degradation_detected tier_registry.activated + +audit.chain_integrity_alert audit.chain_break audit.forward_failed + +dependency.state_changed stakeholder.resource_decommissioning +allocation.pool_capacity_low allocation.released + +ingestion.transitional_created ingestion.enriched ingestion.promotion_approved + +governance.catalog_item_deprecated governance.profile_changed +governance.policy_trust_elevated + +security.unsanctioned_provider_write +sovereignty.violation sovereignty.migration_required +federation.tunnel_degraded +auth.provider_failover +``` + +**Total: 82 event types across 20 domains** + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/notification-model.md b/content/docs/architecture/data-model/notification-model.md index d321a0c..fdd28bf 100644 --- a/content/docs/architecture/data-model/notification-model.md +++ b/content/docs/architecture/data-model/notification-model.md @@ -30,6 +30,9 @@ weight: 23 --- + +> **See [Event Catalog](33-event-catalog.md)** โ€” authoritative source for all DCM event types and payload schemas. + ## 1. Purpose The DCM Notification Model defines a **unified, configurable notification pipeline** that routes event notifications to all parties with a stake in a changed resource โ€” not just the original requestor. The audience for any notification is derived from the **entity relationship graph**, not from who submitted the original request. diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md index 0f31362..d25c478 100644 --- a/content/docs/architecture/data-model/webhooks-messaging.md +++ b/content/docs/architecture/data-model/webhooks-messaging.md @@ -30,6 +30,9 @@ weight: 18 --- + +> **See [Event Catalog](33-event-catalog.md)** โ€” authoritative source for all DCM event types and payload schemas. + ## 1. Purpose DCM communicates with the outside world through three complementary mechanisms: diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index 57a2deb..3773869 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -339,6 +339,21 @@ --- +## 25. Event Catalog + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| EVT-001 | Event Subscription | Subscribe to DCM events via Notification Provider or Message Bus; filter by event type, entity type, urgency; idempotency via event_uuid | Publish standard events when provider actions occur; use reverse-DNS prefix for non-standard events | Configure Notification Provider channels and audience routing | IAM-001, OBS-001 | +| EVT-002 | Request Pipeline Events | Receive real-time status of own requests (submitted โ†’ intent_captured โ†’ policies_evaluated โ†’ requires_approval โ†’ approved โ†’ dispatched โ†’ realized/failed) | โ€” | Configure request event delivery per profile; manage urgency routing | REQ-001 | +| EVT-003 | Entity Lifecycle Events | Receive entity lifecycle events (realized, state_changed, ttl_warning, decommissioning, etc.) for owned entities and entities with stakes | โ€” | Configure entity event delivery; manage stakeholder audience routing | LCM-001 | +| EVT-004 | Security and Critical Events | Receive critical security events (audit chain alerts, sovereignty violations, unsanctioned provider writes) regardless of subscription preferences | โ€” | Configure non-suppressable event delivery; manage security team routing | AUD-001, ZTS-001 | +| EVT-005 | Approval Pipeline Events | Receive approval events (requires_approval, decision_recorded, quorum_reached, window_expiring, expired) for own requests and approvals | โ€” | Configure reviewer notification routing; manage approval window alerts | ATM-001, IAM-001 | +| EVT-006 | Provider and Infrastructure Events | โ€” | Publish provider health events (registered, healthy, unhealthy, degraded); publish provider_update events on entity changes | Monitor provider health events; configure provider degradation alerts | PRV-001 | +| EVT-007 | Tier Registry and Governance Events | โ€” | โ€” | Receive tier_registry events (proposed, impact_assessed, degradation_detected, activated); configure governance event routing | ATM-004 | + +--- + + ## Capability Count Summary | Domain | Capabilities | @@ -367,7 +382,8 @@ | Meta Provider Composability | 7 | | Credential Provider Model | 7 | | Authority Tier Model | 7 | -| **Total** | **155** | +| Event Catalog | 7 | +| **Total** | **162** | --- diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index 8a883dc..1af7367 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -113,6 +113,18 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture + +### Event Catalog Terms + +| Term | Definition | +|------|-----------| +| **Event Catalog** | The authoritative source for all DCM event types, their payload schemas, urgency levels, and trigger conditions. See [33-event-catalog.md]. 82 event types across 20 domains. | +| **Event Envelope** | The common wrapper all DCM events share: event_uuid (idempotency key), event_type, event_schema_version, timestamp (from Commit Log), dcm_version, dcm_instance_uuid, subject, urgency, payload, links. | +| **event_uuid** | Stable idempotency key assigned to each event. Consumers must treat duplicate event_uuid values as already-processed โ€” DCM delivers at-least-once. | +| **event_schema_version** | Increments on breaking payload schema changes. Adding optional fields is not breaking. Removing fields, changing types, or changing semantics are breaking. | +| **EVT-001โ€“EVT-007** | Event Catalog system policies. Key: EVT-001 (all events must include base envelope), EVT-002 (event_uuid is idempotency key), EVT-005 (critical urgency events delivered via push regardless of subscriptions), EVT-006 (non-standard events use reverse-DNS prefix), EVT-007 (audit.* critical events are non-suppressable). | + + ### Authority Tier Model Terms | Term | Definition | @@ -240,6 +252,7 @@ Terms to avoid because they introduce ambiguity. Use the precise alternatives in | CPX | Credential Provider Model | | DPO | Design Priority Order | | ATM | Authority Tier Model | +| EVT | Event Catalog | --- From df4da1ac6ed8b4f5057180ddfee33155f17efb96 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sun, 29 Mar 2026 16:10:52 -0500 Subject: [PATCH 44/49] Cleaned up admin api spec. Another consistency run done through all documentation / specs. Signed-off-by: Chris Roadfeldt --- content/docs/architecture/ai-prompt.md | 99 +++-- .../data-model/api-versioning-strategy.md | 390 ++++++++++++++++++ .../data-model/authority-tier-model.md | 2 +- .../data-model/design-priorities.md | 2 +- .../specifications/admin-api-spec.md | 81 ++-- .../specifications/consumer-api-spec.md | 126 +++++- .../specifications/operator-interface-spec.md | 55 ++- content/docs/capabilities-matrix.md | 16 +- content/docs/taxonomy.md | 13 + 9 files changed, 706 insertions(+), 78 deletions(-) create mode 100644 content/docs/architecture/data-model/api-versioning-strategy.md diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 838e117..060c8bb 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4138,30 +4138,25 @@ These items are explicitly unresolved. Do not make assumptions about them โ€” fl --- -## SECTION 59 โ€” CAPABILITIES MATRIX UPDATE (119 capabilities, 19 domains) +## SECTION 59 โ€” CAPABILITIES MATRIX (167 capabilities, 26 domains) -Five new domains added to the capabilities matrix. Total: 126 capabilities across 20 domains. +The DCM Capabilities Matrix contains 167 capabilities across 26 domains. Each capability row specifies what consumers, service providers, and platform admins can do, along with dependencies. -**New domains:** +**Current domain count: 26** +IAM, CAT, REQ, PRV, LCM, DRF, POL, LAY, INF, ING, AUD, OBS, STO, FED, GOV, ACC, ZTS, GMX, DRC, FCM, SMX, MPX, CPX, DPO, ATM, EVT, VER (26 domain prefixes; see taxonomy for full names) -**16. Accreditation Management (ACC-001โ€“006):** Accreditation submission and approval, lifecycle monitoring with P90D renewal warnings, gap response via Recovery Policy, data classification enforcement at interaction boundaries, DCM deployment-level accreditation for federation trust. +**Recent additions (docs 29โ€“34):** +- SMX (Scoring Model, doc 29): risk scoring, approval routing, signal weights, governance matrix +- MPX (Meta Provider, doc 30): compound service definition, constituent orchestration via dependency graph +- CPX (Credential Provider, doc 31): credential lifecycle, rotation, revocation, profile-governed security +- ATM (Authority Tier, doc 32): dynamic ordered tier list, custom tiers, degradation gate, impact detection +- EVT (Event Catalog, doc 33): 82 event types, base envelope, payload schemas, EVT-001โ€“007 +- VER (API Versioning, doc 34): breaking change definition, deprecation lifecycle, version discovery -**17. Zero Trust and Security Posture (ZTS-001โ€“006):** mTLS enforcement, scoped short-lived interaction credentials, certificate rotation management (P14D warning, P7D transition window), zero_trust_posture profile configuration (none/boundary/full/hardware_attested), hardware attestation for sovereign profile, five-check boundary enforcement (identity โ†’ authorization โ†’ accreditation โ†’ matrix โ†’ sovereignty). - -**18. Unified Governance Matrix (GMX-001โ€“007):** Rule authoring in GitOps (four-axis match), boundary enforcement evaluation with rule_uuid in DENY responses, field-level data control (allowlist/blocklist, STRIP_FIELD/REDACT/DENY_REQUEST), sovereignty zone management, compliance domain matrix activation, Tenant/resource-type override rules, rule lifecycle with shadow mode validation. - -**19. Drift Reconciliation (DRC-001โ€“005):** Drift record production with field-level detail, unsanctioned change classification and severity escalation, drift severity classification (field criticality ร— change magnitude), drift resolution tracking with clean-state confirmation, governance matrix integration for expected provider change detection. - -**20. Federated Contribution Model (FCM-001โ€“007):** Consumer policy authoring (policy_author role, PR + shadow mode), provider resource type publication (registry PR for admin review), provider service layer contribution, consumer resource group and definition contribution, federation contribution by peer DCMs (scoped by trust posture), contribution review and lifecycle management (withdraw/status/orphan assignment), contributor scope enforcement (hard DENY via Governance Matrix at contribution time). FCM-001โ€“007 are off the critical path โ€” they extend multi-user capabilities but are not required for the initial end-to-end demonstration. - - -**6 specifications graduated from WIP to Draft:** -dcm-registration-spec.md ยท dcm-opa-integration-spec.md ยท 11-kubernetes-compatibility.md ยท dcm-operator-interface-spec.md ยท dcm-operator-sdk-api.md ยท dcm-admin-api-spec.md - -**2 specifications remain WIP:** -consumer-api-spec.md (missing complete lifecycle endpoint coverage) ยท dcm-flow-gui-spec.md (conceptual only; interaction model not yet specified) - ---- +**SMX-008 hard constraint:** auto_approve_below โ‰ค 50 in ALL profiles +**ATM-002 hard constraint:** auto tier max_score โ‰ค 50 in ALL profiles +**CPX-001 absolute:** credential values NEVER in DCM stores in ANY profile +**EVT-007:** audit.* critical events are non-suppressable ## SECTION 60 โ€” DOCUMENTATION STRUCTURE @@ -4272,14 +4267,14 @@ score_record_uuid, request_uuid, entity_uuid, request_risk_score, routing_decisi ### New API Endpoints Consumer: risk_score + advisory_warnings on POST /api/v1/requests response and GET status -Admin: GET/PATCH /admin/api/v1/profiles/{name}/scoring ยท POST overrides ยท GET/POST /actors/{uuid}/risk-history ยท GET /scoring/audit +Admin: GET/PATCH /api/v1/admin/profiles/{name}/scoring ยท POST overrides ยท GET/POST /actors/{uuid}/risk-history ยท GET /scoring/audit Flow GUI: GET /flow/api/v1/graph/scoring-overlay ยท POST /flow/api/v1/simulate/score ยท Threshold slider in Profile Management view ยท Score breakdown panel in Simulation ### SMX-001โ€“010 System Policies SMX-001: GateKeeper must declare enforcement_class (compliance default). SMX-002: Validation must declare output_class (structural default). SMX-003: regulatory_mandate:true = no profile demotion. SMX-004: Governance Matrix always boolean. SMX-005: signal weights must sum to 1.00. SMX-006: Score Records immutable. SMX-007: actor risk history not exposed to other consumers. SMX-008: auto_approve_below โ‰ค 50. SMX-009: scoring_weight 1โ€“100; aggregate capped at 100 before weighting. SMX-010: Score Record required for every scored request. ### Capabilities -SMX-001 through SMX-008 in Capabilities Matrix Domain 21. Total: 134 capabilities, 21 domains. +SMX-001 through SMX-008 in Capabilities Matrix Domain 21. Total: 167 capabilities, 26 domains. --- @@ -4444,11 +4439,11 @@ When tier registry changes, DCM computes a **tier_impact_diff** โ€” a structured - **PROFILE_GAP**: new tier inserted but profile threshold list not updated โ†’ warning, does not block (ATM-012) - **SECURITY_UPGRADE / STALE_WEIGHT**: informational, does not block -Degradation review gate: each SECURITY_DEGRADATION must be accepted via `POST /admin/api/v1/tier-registry/{change_uuid}/accept-degradation` by a `verified` or `authorized` tier reviewer before activation. +Degradation review gate: each SECURITY_DEGRADATION must be accepted via `POST /api/v1/admin/tier-registry/{change_uuid}/accept-degradation` by a `verified` or `authorized` tier reviewer before activation. Impact report (ATM-011) stored in Audit Store for every registry change, at proposal and at activation. -Admin API: POST /admin/api/v1/tier-registry/changes (propose) ยท GET .../impact (report) ยท POST .../accept-degradation ยท POST .../activate +Admin API: POST /api/v1/admin/tier-registry/changes (propose) ยท GET .../impact (report) ยท POST .../accept-degradation ยท POST .../activate ### ATM-001โ€“ATM-012 System Policies ATM-001: tiers identified by name; weight derived from position. ATM-002: auto tier max_score โ‰ค 50. ATM-003: custom gravity consistent with position. ATM-004: custom tiers require verified-tier approval. ATM-005: custom tiers cannot change existing tier dcm_gate semantics. ATM-006: dcmgroup_required tiers must have DCMGroup declared before use. ATM-007: four gravity values are DCM vocabulary (org cannot add gravity values). ATM-008: approval records store weight at creation time for point-in-time audit. @@ -4465,7 +4460,7 @@ Peer DCM instances may have different custom tier lists. Resolution strategy: `g ### Base Envelope (all events share this) event_uuid (idempotency key โ€” EVT-002: consumers must treat duplicates as already-processed) ยท event_type ยท event_schema_version ยท timestamp (from Commit Log โ€” authoritative) ยท dcm_version ยท dcm_instance_uuid ยท subject (entity_uuid, entity_type, entity_handle, tenant_uuid, actor_uuid) ยท urgency (critical/high/medium/low/info) ยท payload (event-specific) ยท links (self, audit_record) -### Event Domains (82 total across 20 domains) +### Event Domains (82 total across 26 domains) request.* (14): submitted โ†’ intent_captured โ†’ layers_assembled โ†’ policies_evaluated โ†’ requires_approval โ†’ approved โ†’ placement_complete โ†’ dispatched โ†’ compound_assembled โ†’ dependencies_resolved โ†’ realized/failed/gatekeeper_rejected/cancelled entity.* (13): realized, state_changed, modified, ttl_warning, ttl_expired, suspended, resumed, decommissioning, decommissioned, decommission_deferred, ownership_transferred, pending_review, expired drift.* (4): detected, severity_escalated, resolved, escalated @@ -4498,7 +4493,52 @@ EVT-001: all events must include base envelope. EVT-002: event_uuid is idempoten --- -## SECTION 66 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 66 โ€” API VERSIONING STRATEGY (doc 34 โ€” 34-api-versioning-strategy.md) + +> **Full specification:** [34-api-versioning-strategy.md](data-model/34-api-versioning-strategy.md) โ€” breaking change definition, deprecation lifecycle, version discovery, sunset behavior, VER-001โ€“VER-009. + +### Versioning Model +URL path versioning: `/api/v1/`, `/api/v2/`, etc. Version is per-API surface (Consumer, Admin, Provider/OIS, Flow GUI) โ€” NOT per-endpoint. All endpoints in a surface share the same major version. Non-breaking changes do not change the URL. + +### Breaking Change Definition (VER-002) +**Breaking:** removing fields/endpoints, changing field types, changing URL structure, tightening validation, changing HTTP status semantics, removing enum values, changing HTTP method. +**NOT breaking:** adding optional fields, adding endpoints, expanding enums, relaxing validation, adding error codes, performance changes. When in doubt โ†’ treat as breaking. + +### Deprecation Lifecycle +Profile-governed support windows: +- minimal: 90 days notice, 180 days deprecated support +- standard: 180 days / 365 days +- prod: 365 days / 730 days (2 years) +- fsi: 18 months / 3 years +- sovereign: 2 years / 4 years + +Deprecated versions: fully functional until sunset. Bugs fixed; features not backported. `Deprecation`, `Sunset`, `Link` headers on every response (RFC 8594/RFC 9745). VER-003. + +### Version Discovery +`GET /.well-known/dcm-api-versions` โ€” lists all API surfaces, current/supported/deprecated versions, base URLs, changelog URLs. +Per-version changelog: `GET /api/v{N}/changelog` +Machine-readable migration guide (required by VER-008): `GET /api/v{N}/migration-guide` + +### Sunset Behavior +After sunset: `410 Gone` with successor_version, migration_guide_url, sunset_date. +Three events: `governance.api_version_deprecated`, `governance.api_version_sunset_warning` (30 days before), `governance.api_version_sunset`. + +### Version Negotiation +URL path is authoritative. Optional `DCM-API-Version: v1` header for explicit pinning (returns 406 if sunsetted). +`/api/latest/` alias exists but NOT for production โ€” pin to specific version. + +### Preview Endpoints +`/api/v{N}/preview/` โ€” no stability commitment; may change without major version increment; not for production automation. + +### OIS (Provider API) Versioning +Providers declare `ois_version` in capability registration. DCM maintains dispatch compatibility with all supported OIS versions during deprecation window. VER-009. + +### VER-001โ€“VER-009 System Policies +VER-001: URL path versioning only. VER-002: breaking change definition. VER-003: deprecation headers required. VER-004: deprecated versions fully functional until sunset. VER-005: support windows profile-governed. VER-006: latest alias not for production. VER-007: preview endpoints not stable. VER-008: migration guide required per new major version. VER-009: OIS dispatch compatibility during deprecation window. + +--- + +## SECTION 67 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure): @@ -4518,7 +4558,12 @@ When working on this project, apply these instructions in addition to the number 186. **Credential values are NEVER stored in DCM** (CPX-001) โ€” only metadata is stored; values are held by the Credential Provider; retrieved via authenticated endpoint; this applies to ALL credential types including dcm_interaction credentials 187. **Every provider dispatch requires a scoped interaction credential** (CPX-002) โ€” issued before dispatch, scoped to the specific operation+entity+provider, expires PT15M; provider must validate at use time not just receipt; check revocation cache on each use 189. **Security properties are present in ALL profiles โ€” minimal profile is "security with minimal operational overhead" not "minimal security"** โ€” rotation required in all profiles (minimal: P365D max, manual OK); idle detection on in all profiles (minimal: P30D); algorithm baseline in all profiles (minimal: forbidden list); CPX-001 (values never in DCM stores) is absolute โ€” homelab (minimal) uses bearer_token retrieval, no scheduled rotation, no FIPS; sovereign uses mtls+hardware attestation, FIPS Level 3, PT15S revocation cache; same API contract, same data model, same CPX-001 (values never in DCM stores) -195. **33-event-catalog.md is the SINGLE authoritative source for all DCM event types** โ€” 82 events across 20 domains; all events share the base envelope (event_uuid, event_type, event_schema_version, timestamp from Commit Log, urgency, payload, links); consumers implement idempotency using event_uuid; critical urgency events are non-suppressable; non-standard events use reverse-DNS prefix; event_schema_version only increments on breaking changes +196. **API versioning is per-surface not per-endpoint** (VER-001) โ€” Consumer, Admin, Provider/OIS, Flow GUI each have their own major version; all endpoints within a surface share the version; when in doubt whether a change is breaking, it is (VER-002); prod support window is 2 years deprecated after 1 year notice; sovereign is 4 years deprecated after 2 years notice; deprecated versions return Deprecation + Sunset headers (RFC 8594/RFC 9745) +197. **Consumer API has 16 sequential sections (reorganized)** โ€” sections were renumbered 1โ€“16 in logical order: Auth(2), Catalog(3), Requests(4), Resources(5), Drift(6), Groups(7), Approvals(8), Cost(9), Notifications(10), Search(11), Audit(12), Errors(13), Contributions(14), Credentials(15), Conformance(16); old 5b/5c/6b/7b numbering is gone +198. **Admin API base URL is /api/v1/admin/ (version-first)** โ€” NOT /admin/api/v1/; all 40 admin endpoints use /api/v1/admin/; this is the authoritative form used everywhere in the specs and data model docs +199. **Consumer API has idempotency (1.5), rate limiting (1.6), request IDs (1.7), and standard envelope (1.8)** โ€” POST requests support Idempotency-Key header (PT24H retention); rate limits are profile-governed (60/min minimal โ†’ 600/min sovereign) with Retry-After on 429; all list responses use {"items":[...],"total":N,"next_cursor":"..."} envelope; X-DCM-Request-ID and X-DCM-Correlation-ID on all responses +200. **OIS health check response is normative (not optional)** โ€” providers MUST return {status: pass|warn|fail, version, dcm_registration_status}; missing/malformed body = warn; 3 consecutive non-200 = provider.unhealthy event; response format follows RFC 8615 / IANA health+json +195. **33-event-catalog.md is the SINGLE authoritative source for all DCM event types** โ€” 82 events across 26 domains; all events share the base envelope (event_uuid, event_type, event_schema_version, timestamp from Commit Log, urgency, payload, links); consumers implement idempotency using event_uuid; critical urgency events are non-suppressable; non-standard events use reverse-DNS prefix; event_schema_version only increments on breaking changes 194. **Tier registry changes are gated by impact detection** โ€” any change that creates a SECURITY_DEGRADATION (tier gravity or position decreased) blocks activation until each degradation is explicitly accepted by a verified-tier or above reviewer via Admin API; BROKEN_REFERENCE also blocks; PROFILE_GAP is a warning that does not block; all changes produce an impact report in the Audit Store (ATM-009โ€“012) 193. **Authority tiers are named positions in an ordered list โ€” not fixed enum values** โ€” tier weight derived from list position at evaluation time; organizations insert custom tiers between existing ones without breaking existing name references; 'authorized' tier always means 'highest current gravity' regardless of what's been inserted before it; ATM-001: never hardcode tier weights 192. **DCM provides the approval gate and audit trail โ€” the review process is the organization's responsibility** โ€” for authorized tier: DCM tracks quorum of a DCMGroup; the authorized deliberation and vote collection happen outside DCM; external systems (ServiceNow, Jira, Slack bots) can call Admin API to record votes; DCM does NOT build authorized management; for reviewed and verified: same principle โ€” DCM holds the pipeline until the API receives the required decisions diff --git a/content/docs/architecture/data-model/api-versioning-strategy.md b/content/docs/architecture/data-model/api-versioning-strategy.md new file mode 100644 index 0000000..cd3890b --- /dev/null +++ b/content/docs/architecture/data-model/api-versioning-strategy.md @@ -0,0 +1,390 @@ +--- +title: "API Versioning Strategy" +type: docs +weight: 34 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” API Versioning and Lifecycle +**Related Documents:** [Consumer API Specification](../specifications/consumer-api-spec.md) | [Admin API Specification](../specifications/dcm-admin-api-spec.md) | [Operator Interface Specification](../specifications/dcm-operator-interface-spec.md) | [Event Catalog](33-event-catalog.md) | [Registry Governance](20-registry-governance.md) | [Design Priorities](00-design-priorities.md) + +> **This document governs all DCM API surfaces.** Every public API endpoint โ€” Consumer, Admin, Operator Interface (Provider), Flow GUI โ€” follows this versioning strategy. The strategy is designed to make the secure, compatible path the easy path: clients that do nothing get the version they requested; breaking changes are announced with sufficient lead time; the newest version is always the supported version. + +--- + +## 1. Versioning Model + +### 1.1 URL-Based Major Version + +DCM APIs use **URL path versioning** for major versions. The version is the first path segment after the API surface prefix: + +``` +Consumer API: https://{dcm-instance}/api/v1/ +Admin API: https://{dcm-instance}/api/v1/admin/ +Provider API (OIS):https://{dcm-instance}/provider/api/v1/ +Flow GUI API: https://{dcm-instance}/flow/api/v1/ +``` + +The version segment (`v1`, `v2`, etc.) represents a major version. It increments only on breaking changes. Multiple major versions may coexist during a transition window (see Section 4). + +### 1.2 Version Granularity โ€” Per-API, Not Per-Endpoint + +Versioning is **per-API surface**, not per-endpoint. When a breaking change occurs to any endpoint within an API surface, the entire surface increments to the next major version. This means: + +- `v2` of the Consumer API is a complete API surface, not a patchwork of versioned endpoints +- All endpoints within a surface version are internally consistent +- Clients target a single version for all their interactions with that surface + +Individual endpoints are not independently versioned. If a single endpoint needs a breaking change, the API surface version increments and all other endpoints continue unchanged under the new version. + +### 1.3 Minor and Revision Changes + +Non-breaking changes within a major version are documented in the API changelog but do not change the URL. Clients do not need to take any action for non-breaking changes. + +The changelog follows semantic versioning conventions: +- **Minor change**: new optional fields, new endpoints, expanded enum values with backward-compatible defaults +- **Revision**: documentation corrections, clarifications, non-functional specification updates + +--- + +## 2. Breaking Change Definition + +A change is **breaking** if it requires any existing client to modify its code or configuration to continue working correctly. The following changes are always breaking: + +**Request changes:** +- Removing a field that was previously accepted +- Changing a field from optional to required +- Changing a field's type (e.g. string โ†’ integer) +- Removing an accepted enum value +- Changing URL path structure (endpoint rename or restructure) +- Changing HTTP method for an existing operation +- Removing an endpoint + +**Response changes:** +- Removing a field from any response +- Changing a field's type in any response +- Changing a field's name in any response +- Removing a previously returned enum value +- Changing HTTP status code semantics (e.g. 200 โ†’ 202, or changing when 4xx vs 5xx is returned) +- Changing the response envelope structure + +**Behavior changes:** +- Changing default values in ways that alter existing behavior +- Changing idempotency semantics +- Removing a previously supported authentication method +- Tightening validation (rejecting previously accepted inputs) +- Changing pagination behavior in ways that break existing cursor patterns + +**The following are NOT breaking changes:** +- Adding new optional request fields (with sensible defaults) +- Adding new response fields (existing clients safely ignore unknown fields) +- Adding new endpoints +- Expanding an enum with new values (clients must handle unknown enum values gracefully) +- Relaxing validation (accepting previously rejected inputs) +- Adding new error codes (clients that handle errors generically are unaffected) +- Performance improvements, infrastructure changes, security patches +- Documentation improvements + +--- + +## 3. Version Discovery + +Clients can discover available API versions and their status without prior knowledge: + +### 3.1 Well-Known Discovery Endpoint + +``` +GET https://{dcm-instance}/.well-known/dcm-api-versions + +Response 200: +{ + "dcm_version": "1.2.0", + "api_surfaces": { + "consumer": { + "current": "v2", + "supported": ["v1", "v2"], + "versions": { + "v1": { + "status": "deprecated", + "sunset_date": "2027-06-01", + "deprecation_date": "2026-06-01", + "base_url": "/api/v1/", + "changelog_url": "/api/v1/changelog" + }, + "v2": { + "status": "stable", + "released_date": "2026-06-01", + "base_url": "/api/v2/", + "changelog_url": "/api/v2/changelog" + } + } + }, + "admin": { + "current": "v1", + "supported": ["v1"], + "versions": { + "v1": { "status": "stable", "base_url": "/api/v1/admin/" } + } + }, + "provider": { + "current": "v1", + "supported": ["v1"], + "versions": { + "v1": { "status": "stable", "base_url": "/provider/api/v1/" } + } + } + } +} +``` + +### 3.2 Per-Version Changelog + +``` +GET /api/v1/changelog + +Response 200: +{ + "version": "v1", + "changes": [ + { + "date": "2026-01-15", + "type": "minor", + "description": "Added optional `score_drivers` field to request status response", + "affected_endpoints": ["GET /api/v1/requests/{uuid}/status"] + } + ] +} +``` + +--- + +## 4. Deprecation and Sunset Lifecycle + +### 4.1 Deprecation Timeline + +When a new major version is released, the previous version enters a **deprecation period**. The deprecation timeline is profile-governed โ€” production deployments require longer support windows than development environments: + +```yaml +api_version_support_lifecycle: + minimal: + deprecation_notice_period: P90D # 90 days notice before sunset + deprecated_version_support: P180D # old version supported 180 days after deprecation + + dev: + deprecation_notice_period: P60D + deprecated_version_support: P90D + + standard: + deprecation_notice_period: P180D + deprecated_version_support: P365D # 1 year + + prod: + deprecation_notice_period: P365D # 1 year notice + deprecated_version_support: P730D # 2 years support after deprecation + + fsi: + deprecation_notice_period: P548D # 18 months notice + deprecated_version_support: P1095D # 3 years support after deprecation + + sovereign: + deprecation_notice_period: P730D # 2 years notice + deprecated_version_support: P1460D # 4 years support after deprecation +``` + +**Deprecation โ‰  Sunset.** A deprecated version continues to function. Sunset is when it stops working. The deprecation period is the window between "we recommend you migrate" and "you must migrate." + +### 4.2 Deprecation Headers + +When a client calls a deprecated API version, the response includes standard deprecation headers (per [RFC 8594](https://datatracker.ietf.org/doc/html/rfc8594) and [RFC 9745](https://datatracker.ietf.org/doc/html/rfc9745)): + +```http +HTTP/1.1 200 OK +Deprecation: @1749340800 # Unix timestamp when this version was deprecated +Sunset: @1781049600 # Unix timestamp when this version will stop working +Link: ; rel="successor-version" +Link: ; rel="deprecation" +``` + +### 4.3 Deprecation Events + +When a version is deprecated or sunsetted, DCM fires notification events: + +- `governance.api_version_deprecated` โ€” version entered deprecation; Sunset header begins appearing +- `governance.api_version_sunset_warning` โ€” 30 days before sunset; high urgency +- `governance.api_version_sunset` โ€” version has reached sunset date; calls now return 410 Gone + +Platform admins should configure notification routing for these events to ensure API consumers receive timely warning. + +### 4.4 Sunset Behavior + +After the sunset date, calls to the deprecated version return: + +```http +HTTP/1.1 410 Gone +Content-Type: application/json + +{ + "error": "api_version_sunset", + "message": "API version v1 reached its sunset date on 2027-06-01. Migrate to v2.", + "successor_version": "v2", + "migration_guide_url": "/api/v2/migration-guide", + "sunset_date": "2027-06-01" +} +``` + +--- + +## 5. Version Negotiation + +### 5.1 How Clients Specify a Version + +The URL path is the primary versioning mechanism. No headers or query parameters are required โ€” the URL is authoritative: + +``` +GET /api/v1/resources โ†’ Consumer API v1 +GET /api/v2/resources โ†’ Consumer API v2 (when available) +``` + +### 5.2 Version Preference Header (Optional) + +For clients that need to pin to a specific version or test against a new version before migrating, an optional `DCM-API-Version` header is supported: + +```http +GET /api/v1/resources +DCM-API-Version: v1 # explicit pin; returns 406 if v1 is sunsetted +``` + +If the header specifies a sunsetted version, the response is `406 Not Acceptable` with a migration guide reference. + +### 5.3 Latest-Version Alias + +``` +GET /api/latest/resources # always routes to current stable version +``` + +The `latest` alias is provided for development and testing. It is **not recommended for production** โ€” production clients should pin to a specific version to avoid inadvertent breaking changes when a new major version becomes `latest`. + +--- + +## 6. Beta and Preview Endpoints + +New capabilities that are not yet stable may be released as **preview endpoints** within the current major version: + +``` +GET /api/v1/preview/new-feature +``` + +Preview endpoints: +- Are not covered by the stability guarantees of the parent version +- May change or be removed without a major version increment +- Are marked in the API changelog and discovery endpoint as `status: preview` +- Must not be used in production automation without explicit acknowledgment of instability + +```yaml +# Discovery response for a preview endpoint +"new-feature": { + "status": "preview", + "stability_commitment": "none", + "planned_graduation": "v2", + "feedback_url": "https://github.com/dcm-project/discussions" +} +``` + +Preview endpoints graduate to stable when they are included in a new major version release. + +--- + +## 7. Provider API (OIS) Versioning + +The Operator Interface Specification (OIS) governs how DCM calls providers. Provider implementations must support the version of the OIS they declare in their capability registration. + +### 7.1 OIS Version in Capability Registration + +```yaml +provider_registration: + ois_version: "1.0" # which OIS version this provider implements + ois_version_min: "1.0" # minimum OIS version supported + ois_version_max: "1.x" # maximum OIS version supported (x = any minor) +``` + +### 7.2 OIS Compatibility + +DCM maintains backward compatibility with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 during the deprecation window. + +When the OIS version is incremented: +1. DCM announces the new OIS version via the event `governance.ois_version_released` +2. Providers have the deprecation notice period to upgrade their implementation +3. DCM dispatches using the appropriate OIS version per the provider's declared capability +4. After sunset, providers still on deprecated OIS versions receive `410 Gone` on dispatch + +### 7.3 Provider-Initiated API Versioning + +Providers that expose their own management APIs (beyond the standard OIS surface) are responsible for their own versioning. DCM does not version-manage provider-internal APIs. Providers should follow the same breaking-change definition (Section 2) and announce breaking changes via `provider_update.submitted` events. + +--- + +## 8. Client Migration Path + +### 8.1 Migration Guide Structure + +Each new major version publishes a migration guide accessible at: + +``` +GET /api/v{N}/migration-guide +``` + +The migration guide is machine-readable JSON listing all breaking changes from the previous version: + +```json +{ + "from_version": "v1", + "to_version": "v2", + "breaking_changes": [ + { + "change_id": "BC-001", + "type": "field_removed", + "endpoint": "GET /api/v2/resources/{uuid}", + "description": "Field 'legacy_id' removed from response. Use 'entity_uuid' instead.", + "migration": "Replace references to 'legacy_id' with 'entity_uuid'", + "affected_since": "2026-06-01" + } + ], + "non_breaking_additions": [ ... ], + "sunset_date_of_previous_version": "2027-06-01" +} +``` + +### 8.2 Parallel Operation + +During the deprecation window, clients may run v1 and v2 in parallel โ€” for example, migrating one service at a time. Both versions return consistent data from the same underlying DCM data stores. There are no data synchronization concerns between versions. + +--- + +## 9. Internal API Versioning + +DCM internal component APIs (Control Plane components communicating with each other) follow a simpler model: + +- Internal APIs are not exposed externally and not subject to the external versioning lifecycle +- Internal breaking changes require a coordinated deployment of all affected components +- DCM release versions (e.g. `1.2.0`) cover the complete set of internal APIs for that release +- Operators upgrading DCM must upgrade all components together per the release upgrade guide + +--- + +## 10. System Policies + +| Policy | Rule | +|--------|------| +| `VER-001` | All DCM public API surfaces use URL path versioning. The version path segment is the only authoritative version indicator. | +| `VER-002` | A change is breaking if any existing client must modify code or configuration to continue working. When in doubt, treat a change as breaking. | +| `VER-003` | Deprecated API versions must return `Deprecation`, `Sunset`, and `Link` headers on every response during the deprecation period (per RFC 8594 / RFC 9745). | +| `VER-004` | Deprecated versions must remain fully functional until the sunset date. Bugs in deprecated versions are fixed; new features are not backported. | +| `VER-005` | The deprecation notice period and deprecated version support window are profile-governed. Production deployments require longer windows than development. See Section 4.1. | +| `VER-006` | The `latest` version alias is available but must not be recommended for production use. Production clients must pin to a specific version. | +| `VER-007` | Preview endpoints are not stable. They may change or be removed without a major version increment. They are identified by the `/preview/` path segment. | +| `VER-008` | Every new major version must publish a machine-readable migration guide at `/api/v{N}/migration-guide`. | +| `VER-009` | DCM must maintain dispatch compatibility with providers registered on supported OIS versions during the OIS deprecation window. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/authority-tier-model.md b/content/docs/architecture/data-model/authority-tier-model.md index 293fac4..9d8b670 100644 --- a/content/docs/architecture/data-model/authority-tier-model.md +++ b/content/docs/architecture/data-model/authority-tier-model.md @@ -430,7 +430,7 @@ Security degradations block tier registry activation. The blocking gate requires This is the same pattern as the standard approval pipeline โ€” DCM provides the gate; the organization provides the review process. The difference is that the required tier for the degradation review is always at least `verified`, regardless of the profile in use. ``` -POST /admin/api/v1/tier-registry/{change_uuid}/accept-degradation +POST /api/v1/admin/tier-registry/{change_uuid}/accept-degradation { "affected_item_uuid": "", diff --git a/content/docs/architecture/data-model/design-priorities.md b/content/docs/architecture/data-model/design-priorities.md index 71246d0..f1a3add 100644 --- a/content/docs/architecture/data-model/design-priorities.md +++ b/content/docs/architecture/data-model/design-priorities.md @@ -227,7 +227,7 @@ External systems (ServiceNow, Jira, email workflows, Slack bots) connect to DCM The Admin API approval endpoint is designed to be called by external systems, not only by humans in a DCM UI: ``` -POST /admin/api/v1/approvals/{approval_uuid}/vote +POST /api/v1/admin/approvals/{approval_uuid}/vote Authorization: Bearer # any actor who is a member of the required DCMGroup { diff --git a/content/docs/architecture/specifications/admin-api-spec.md b/content/docs/architecture/specifications/admin-api-spec.md index 779cb01..8cae9fc 100644 --- a/content/docs/architecture/specifications/admin-api-spec.md +++ b/content/docs/architecture/specifications/admin-api-spec.md @@ -34,10 +34,32 @@ All Admin API endpoints require Bearer token authentication (same as Consumer AP Base URL: `/api/v1/admin/` +> **Versioning:** See [API Versioning Strategy](../data-model/34-api-versioning-strategy.md). Breaking changes increment the major version. The Admin API follows the same deprecation lifecycle as the Consumer API, with profile-governed support windows. + Step-up MFA is required for destructive operations (Tenant decommission, accreditation revocation, bootstrap credential rotation) regardless of session MFA status. --- +### 1.1 Rate Limiting + +Admin API endpoints have separate rate limits from the Consumer API, applied per authenticated admin actor: + +| Profile | Requests/minute | Burst | +|---------|----------------|-------| +| All profiles | 120 | 40 | + +Rate-limited responses include `Retry-After`, `X-RateLimit-Limit`, `X-RateLimit-Remaining` headers. + +### 1.2 Request and Correlation IDs + +All responses include `X-DCM-Request-ID` and `X-DCM-Correlation-ID` headers (same model as Consumer API). + +### 1.3 Response Envelopes + +List responses use `{"items": [...], "total": N, "next_cursor": "..."}`. Single resources returned directly. Errors use `{"error": "...", "message": "...", "request_id": "..."}`. + +--- + ## 2. Tenant Management ### 2.1 List Tenants @@ -471,21 +493,32 @@ Response 200: ## 11. Error Model -Same as Consumer API. Additional admin-specific codes: +All Admin API errors use the same envelope as the Consumer API: -| HTTP Status | Error Code | Meaning | -|-------------|-----------|---------| -| 403 | `insufficient_role` | Operation requires platform_admin; actor is tenant_admin | -| 403 | `cross_tenant_denied` | tenant_admin attempting operation outside their Tenant | -| 409 | `tenant_has_active_entities` | Tenant decommission blocked; active entities remain | -| 409 | `provider_has_active_entities` | Provider decommission blocked; entities hosted there | +```json +{ + "error": "", // machine-readable snake_case code + "message": "", // human-readable description + "request_id": "", // matches X-DCM-Request-ID header + "details": {} // optional: field-level details +} +``` ---- +**Admin-specific error codes:** -*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* +| Error Code | HTTP Status | When | +|-----------|-------------|------| +| `insufficient_admin_role` | 403 | Actor lacks required admin role | +| `tenant_not_found` | 404 | Tenant UUID not found | +| `provider_not_found` | 404 | Provider UUID not found | +| `approval_already_voted` | 409 | Actor has already voted on this approval | +| `approval_window_expired` | 410 | Approval window has passed | +| `degradation_already_accepted` | 409 | Degradation item already accepted | +| `tier_registry_blocked` | 409 | Registry change has unresolved blocking items | +| `quota_below_current_usage` | 422 | New quota would be below current consumption | +All error responses include `X-DCM-Request-ID` and `X-DCM-Correlation-ID` headers. ---- ## Scoring Model Administration @@ -494,7 +527,7 @@ Same as Consumer API. Additional admin-specific codes: ### Get Scoring Thresholds for Profile ``` -GET /admin/api/v1/profiles/{profile_name}/scoring +GET /api/v1/admin/profiles/{profile_name}/scoring Response 200: { @@ -521,7 +554,7 @@ Response 200: ### Update Scoring Thresholds ``` -PATCH /admin/api/v1/profiles/{profile_name}/scoring +PATCH /api/v1/admin/profiles/{profile_name}/scoring { "scoring_thresholds": { "auto_approve_below": 20, @@ -540,7 +573,7 @@ Response 422: { "error": "threshold_invalid", "reason": "auto_approve_below exce ### Add Policy Enforcement Override ``` -POST /admin/api/v1/profiles/{profile_name}/scoring/overrides +POST /api/v1/admin/profiles/{profile_name}/scoring/overrides { "policy_handle": "platform/gatekeeper/cpu-size-limit", "override_enforcement_class": "compliance", @@ -555,7 +588,7 @@ Response 201 Created: ### Actor Risk History ``` -GET /admin/api/v1/actors/{actor_uuid}/risk-history +GET /api/v1/admin/actors/{actor_uuid}/risk-history Response 200: { @@ -575,7 +608,7 @@ Response 200: "score_half_life_days": 7 } -POST /admin/api/v1/actors/{actor_uuid}/risk-history/reset +POST /api/v1/admin/actors/{actor_uuid}/risk-history/reset { "reason": "Actor confirmed as trusted automation account", "audit_note": "Reviewed and approved by platform admin" @@ -585,7 +618,7 @@ POST /admin/api/v1/actors/{actor_uuid}/risk-history/reset ### Score Audit Trail ``` -GET /admin/api/v1/scoring/audit +GET /api/v1/admin/scoring/audit Query parameters: from= @@ -620,7 +653,7 @@ DCM provides approval gates for requests, policy contributions, provider registr ### List Pending Approvals ``` -GET /admin/api/v1/approvals/pending +GET /api/v1/admin/approvals/pending Query parameters: approval_type= @@ -650,7 +683,7 @@ Response 200: ### Record an Approval Decision ``` -POST /admin/api/v1/approvals/{approval_uuid}/vote +POST /api/v1/admin/approvals/{approval_uuid}/vote { "decision": "approve | reject", @@ -689,7 +722,7 @@ Response 410: approval window has expired ### Get Approval Detail ``` -GET /admin/api/v1/approvals/{approval_uuid} +GET /api/v1/admin/approvals/{approval_uuid} Response 200: { @@ -725,7 +758,7 @@ Response 200: ### Propose a Tier Registry Change ``` -POST /admin/api/v1/tier-registry/changes +POST /api/v1/admin/tier-registry/changes { "proposed_tiers": [ @@ -749,7 +782,7 @@ Response 202 Accepted: ### Get Tier Registry Impact Report ``` -GET /admin/api/v1/tier-registry/changes/{change_uuid}/impact +GET /api/v1/admin/tier-registry/changes/{change_uuid}/impact Response 200: { @@ -778,7 +811,7 @@ Response 200: ### Accept a Security Degradation ``` -POST /admin/api/v1/tier-registry/changes/{change_uuid}/accept-degradation +POST /api/v1/admin/tier-registry/changes/{change_uuid}/accept-degradation { "affected_item_uuid": "", @@ -802,7 +835,7 @@ Response 409: degradation already accepted ### Activate a Tier Registry Change ``` -POST /admin/api/v1/tier-registry/changes/{change_uuid}/activate +POST /api/v1/admin/tier-registry/changes/{change_uuid}/activate Response 200: { @@ -818,7 +851,7 @@ Response 409: change has unresolved blocking items (broken_references or unaccep ### List Historical Registry Changes ``` -GET /admin/api/v1/tier-registry/changes?status=activated&limit=20 +GET /api/v1/admin/tier-registry/changes?status=activated&limit=20 Response 200: { diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index 83e2076..be34e01 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -58,14 +58,114 @@ This specification primarily documents the REST API surface. The Git PR ingress https://{dcm-instance}/api/v1/ ``` -All Consumer API endpoints are versioned. Breaking changes increment the version. Non-breaking additions do not. +All Consumer API endpoints are versioned. Breaking changes increment the major version segment (`v1` โ†’ `v2`). Non-breaking additions do not change the version. + +> **Full versioning strategy:** See [API Versioning Strategy](../data-model/34-api-versioning-strategy.md) for the complete definition of breaking changes, deprecation timeline, version discovery, sunset behavior, deprecation headers, and VER-001โ€“VER-009 system policies. + +**Key rules for Consumer API consumers:** +- Pin to a specific version (`/api/v1/`) in production โ€” do not use the `/api/latest/` alias +- When a version is deprecated, responses include `Deprecation` and `Sunset` headers (RFC 8594/RFC 9745) +- Deprecated versions remain functional until the sunset date โ€” bugs fixed, features not backported +- Version discovery: `GET /.well-known/dcm-api-versions` +- Migration guide: `GET /api/v{N}/migration-guide` + +**What is a breaking change in the Consumer API:** +Removing a field, changing a field type, removing an endpoint, changing HTTP status semantics, tightening validation, changing URL structure. New optional fields, new endpoints, and expanded enums are not breaking. + +**Support windows (profile-governed):** +- `minimal`: 90 days notice, 180 days deprecated support +- `standard`: 180 days notice, 365 days deprecated support +- `prod`: 365 days notice, 730 days (2 years) deprecated support +- `fsi`: 18 months notice, 3 years deprecated support +- `sovereign`: 2 years notice, 4 years deprecated support ### 1.4 Content Type All requests and responses use `application/json`. The DCM Unified Data Model is expressed as JSON throughout the Consumer API. +### 1.5 Idempotency + +DCM's request model provides built-in idempotency for `POST /api/v1/requests`. Each request submission produces an `entity_uuid` at Intent State creation. If a client retries a request submission (e.g. after a network timeout), it may receive a duplicate Intent State โ€” but DCM's deduplication layer detects identical payloads from the same actor within a 5-minute window and returns the existing request record rather than creating a second one. + +For operations where explicit idempotency control is needed, clients may supply an `Idempotency-Key` header: + +```http +POST /api/v1/requests +Idempotency-Key: +``` + +If DCM receives two requests with the same `Idempotency-Key` from the same authenticated actor within PT24H, the second request returns the response from the first. The idempotency key is stored for PT24H then discarded. + +**Which endpoints support `Idempotency-Key`:** +- `POST /api/v1/requests` โ€” resource request submission +- `POST /api/v1/credentials/{uuid}/rotate` โ€” credential rotation request +- `POST /api/v1/resources/{uuid}/rehydrate` โ€” rehydration trigger + +### 1.6 Rate Limiting + +Rate limits are profile-governed and apply per authenticated actor: + +| Profile | Requests/minute | Burst allowance | Rate limit header | +|---------|----------------|-----------------|-------------------| +| `minimal` | 60 | 20 | Yes | +| `standard` | 300 | 100 | Yes | +| `prod` | 600 | 200 | Yes | +| `fsi` | 600 | 200 | Yes | +| `sovereign` | 600 | 200 | Yes | + +When rate limited, DCM returns: + +```http +HTTP/1.1 429 Too Many Requests +Retry-After: 12 +X-RateLimit-Limit: 300 +X-RateLimit-Remaining: 0 +X-RateLimit-Reset: 1749340800 + +{ + "error": "rate_limit_exceeded", + "message": "Request rate limit exceeded. Retry after 12 seconds.", + "retry_after_seconds": 12 +} +``` + +### 1.7 Request and Correlation IDs + +Every DCM API response includes: + +```http +X-DCM-Request-ID: # unique ID for this HTTP request; use for support +X-DCM-Correlation-ID: # links related requests across the pipeline +``` + +Include `X-DCM-Request-ID` when contacting support. Use `X-DCM-Correlation-ID` to trace a request through the audit trail (`GET /api/v1/audit/correlation/{correlation_id}`). + --- +### 1.8 Standard Response Envelopes + +**List responses** always use this envelope: +```json +{ + "items": [...], // always "items" regardless of resource type + "total": 142, // total matching records (before pagination) + "page_size": 25, + "next_cursor": "" // null if no more pages; use as ?cursor= on next request +} +``` + +**Single resource responses** return the resource object directly (no wrapper). + +**Error responses** always use: +```json +{ + "error": "", // machine-readable snake_case code + "message": "", // human-readable description + "request_id": "", // matches X-DCM-Request-ID header + "details": {} // optional: field-level validation errors etc. +} +``` + ## 2. Authentication ### 2.1 Token Acquisition @@ -283,7 +383,7 @@ Same response shape as List Catalog Items. --- -## 4. Request Submission +## 4. Request Submission and Lifecycle ### 4.1 Submit Resource Request @@ -948,7 +1048,7 @@ Response 200: --- -## 5b. Drift Management +## 6. Drift Management ### 5b.1 List Drift Records for a Resource @@ -1045,7 +1145,7 @@ Response 202 Accepted: --- -## 5c. Groups and Relationships +## 7. Groups and Relationships ### 5c.1 List Resource Groups @@ -1153,7 +1253,7 @@ Response 200: --- -## 6b. Requests Management +## 8. Requests and Approvals ### 6b.1 List Requests @@ -1247,7 +1347,7 @@ Response 202 Accepted: --- -## 7b. Cost and Quota +## 9. Cost and Quota ### 7b.1 Get Cost Estimate (Pre-Submission) @@ -1338,7 +1438,7 @@ Response 200: --- -## 7c. Notifications and Webhooks +## 10. Notifications and Webhooks ### 7c.1 List Notifications @@ -1434,7 +1534,7 @@ Response 204 No Content --- -## 8b. Search +## 11. Search ### 8b.1 Cross-Resource Search @@ -1475,7 +1575,7 @@ Response 200: ``` -## 6. Audit Trail +## 12. Audit Trail ### 6.1 Query Audit Records for a Resource @@ -1533,7 +1633,7 @@ Response 200: --- -## 7. Error Model +## 13. Error Model All error responses follow a consistent structure: @@ -1575,7 +1675,7 @@ All error responses follow a consistent structure: --- -## 9. Consumer Contribution Endpoints +## 14. Consumer Contributions Consumers with `policy_author` or `tenant_admin` role can contribute tenant-scoped artifacts directly via the Consumer API. All contributions flow through the GitOps PR model โ€” DCM generates a PR and activates the artifact after the required review period. See [Federated Contribution Model](../data-model/28-federated-contribution-model.md) for the complete contributor permission table. @@ -1681,7 +1781,7 @@ Response 200: --- -## 9b. Credential Management +## 15. Credential Management ### 9b.1 List Credentials for a Resource @@ -1749,7 +1849,7 @@ Response 202 Accepted: ``` -## 8. Conformance Levels +## 16. Conformance Levels The Consumer API defines three conformance levels, mirroring the Operator Interface Specification model: diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md index b4df9f2..c00c6a7 100644 --- a/content/docs/architecture/specifications/operator-interface-spec.md +++ b/content/docs/architecture/specifications/operator-interface-spec.md @@ -46,6 +46,9 @@ Operators conforming to this specification function as Service Providers within ## 1. Introduction +> **OIS Versioning:** Providers declare the OIS version they implement in capability registration (`ois_version`). DCM maintains dispatch compatibility with all supported OIS versions during the deprecation window. See [API Versioning Strategy](../data-model/34-api-versioning-strategy.md) Section 7. + + ### 1.1 Motivation Kubernetes operators are the most mature pattern for managing complex, stateful resources declaratively on Kubernetes. However, operators operate within a single cluster and lack the cross-cluster lifecycle management, multi-tenancy, cost attribution, sovereignty governance, and policy enforcement that enterprise organizations require at scale. @@ -248,20 +251,50 @@ DCM polls the operator's health endpoint every 10 seconds (configurable). A heal **Endpoint:** `GET /health` **Authentication:** Unauthenticated (or internally secured โ€” operator choice) -**Expected response:** HTTP 200 OK for healthy, any non-200 for unhealthy +**Expected response:** HTTP 200 OK for healthy or warn status; any non-200 for unhealthy (fail) -```yaml -# Health response body (optional but recommended) -health_response: - status: pass # pass | warn | fail - version: - uptime_seconds: - kubernetes_connectivity: - dcm_registration_status: - details: - +The health response body is **normative**. DCM uses the `status` field to determine provider health and trigger alerts. Providers that return a non-conforming or absent body are treated as `warn` until three consecutive failures, after which they are treated as `fail`. + +```http +GET /health HTTP/1.1 + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass", // REQUIRED: "pass" | "warn" | "fail" + "version": "", // REQUIRED: provider software version + "dcm_registration_status": "registered", // REQUIRED: "registered" | "unregistered" | "error" + "uptime_seconds": 86423, // RECOMMENDED: seconds since last restart + "checks": { // RECOMMENDED: per-subsystem health + "provider_backend": { + "status": "pass", + "observed_at": "" + }, + "credential_provider_connectivity": { + "status": "pass", + "observed_at": "" + } + }, + "details": {} // OPTIONAL: operator-specific additional detail +} ``` +**Status semantics:** + +| Status | HTTP code | Meaning | DCM behavior | +|--------|-----------|---------|--------------| +| `pass` | 200 | Fully operational | No action | +| `warn` | 200 | Operational but degraded | Fires `provider.degraded` event; alert platform admin | +| `fail` | any non-200 | Not operational | Fires `provider.unhealthy` event; triggers recovery policy | + +The health endpoint format follows [RFC 8615 / IANA health+json](https://www.iana.org/assignments/media-types/application/health+json). + +**DCM polling behavior:** +- Polling interval: declared in provider capability registration (`health_check_interval`, default PT30S) +- Consecutive `fail` threshold before `provider.unhealthy` event: 3 (profile-governed) +- Recovery: first `pass` after `fail` fires `provider.healthy` event + ### 4.3 State Machine - **Ready** โ€” HTTP 200 received. Operator eligible for new requests. diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index 3773869..3f189ee 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -354,6 +354,19 @@ --- +## 26. API Versioning + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| VER-001 | Version Discovery | Discover available API versions and their status via `GET /.well-known/dcm-api-versions`; learn current, supported, and deprecated versions; get changelog and migration guide URLs | Declare supported OIS version in capability registration | Monitor version adoption; manage sunset schedules | โ€” | +| VER-002 | Breaking Change Governance | Receive at least the profile-governed deprecation notice period before a breaking change takes effect; continue using deprecated versions until sunset date | Receive OIS version deprecation notice; migrate to new OIS version before sunset | Declare new major versions; configure deprecation timeline per profile; ensure VER-002 (breaking change definition) is applied | VER-001 | +| VER-003 | Deprecation Headers | Receive `Deprecation`, `Sunset`, and `Link` headers on all responses from deprecated API versions (RFC 8594/RFC 9745); use these to drive migration priority | โ€” | Configure header injection for deprecated versions; ensure headers are accurate | VER-002 | +| VER-004 | Migration Guide | Access machine-readable migration guide at `GET /api/v{N}/migration-guide`; understand all breaking changes from previous version with migration instructions | Access OIS migration guide at `GET /provider/api/v{N}/migration-guide` | Maintain migration guides for all new major versions (required by VER-008) | VER-002 | +| VER-005 | Preview Endpoints | Access preview endpoints at `/api/v{N}/preview/`; understand stability commitment is none; provide feedback before graduation | โ€” | Mark endpoints as preview; graduate to stable in new major version | VER-001 | + +--- + + ## Capability Count Summary | Domain | Capabilities | @@ -383,7 +396,8 @@ | Credential Provider Model | 7 | | Authority Tier Model | 7 | | Event Catalog | 7 | -| **Total** | **162** | +| API Versioning | 5 | +| **Total** | **167** | --- diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index 1af7367..bb4bc86 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -114,6 +114,18 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture + +### API Versioning Terms + +| Term | Definition | +|------|-----------| +| **Breaking Change** | Any change that requires an existing client to modify code or configuration to continue working. Removing fields, changing types, removing endpoints, tightening validation, changing HTTP status semantics. See [34-api-versioning-strategy.md] Section 2 for the complete definition. | +| **Deprecation Period** | The window between when a version is announced as deprecated and when it reaches its sunset date. Deprecated versions continue to function; responses include `Deprecation` and `Sunset` headers. Profile-governed: prod=365 days notice, 2 years support; sovereign=2 years notice, 4 years support. | +| **Sunset Date** | The date after which a deprecated API version returns `410 Gone`. Clients must migrate before this date. | +| **Preview Endpoint** | An endpoint at `/api/v{N}/preview/` path with no stability commitment. May change or be removed without a major version increment. Not for production use. | +| **VER-001โ€“VER-009** | API Versioning system policies. Key: VER-002 (breaking change definition โ€” when in doubt, treat as breaking), VER-003 (deprecation headers required on all deprecated version responses), VER-005 (support windows are profile-governed), VER-008 (machine-readable migration guide required for each new major version). | + + ### Event Catalog Terms | Term | Definition | @@ -253,6 +265,7 @@ Terms to avoid because they introduce ambiguity. Use the precise alternatives in | DPO | Design Priority Order | | ATM | Authority Tier Model | | EVT | Event Catalog | +| VER | API Versioning | --- From d959b60a94a5dbf19da44df7024a2dcd561ed0b3 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Sun, 29 Mar 2026 19:01:08 -0500 Subject: [PATCH 45/49] Updated Auth revocation handling, ssl handling, document standards, handling depedencies better, health endpoints. Signed-off-by: Chris Roadfeldt --- content/docs/README.md | 10 +- content/docs/architecture/ai-prompt.md | 208 +++++++++- .../accreditation-and-authorization-matrix.md | 4 +- .../data-model/api-versioning-strategy.md | 2 +- .../audit-provenance-observability.md | 2 +- .../architecture/data-model/auth-providers.md | 4 +- .../data-model/authority-tier-model.md | 2 +- .../data-model/context-and-purpose.md | 2 +- .../data-model/control-plane-components.md | 7 +- .../data-model/credential-provider-model.md | 70 +++- .../architecture/data-model/dcm-federation.md | 2 +- .../data-model/dcm-self-health.md | 374 ++++++++++++++++++ .../data-model/deployment-redundancy.md | 4 +- .../data-model/entity-relationships.md | 2 +- .../architecture/data-model/entity-types.md | 2 +- .../architecture/data-model/event-catalog.md | 3 +- .../docs/architecture/data-model/examples.md | 2 +- .../federated-contribution-model.md | 2 +- .../architecture/data-model/foundations.md | 2 +- .../architecture/data-model/four-states.md | 2 +- .../data-model/governance-matrix.md | 2 +- .../information-providers-advanced.md | 2 +- .../data-model/information-providers.md | 2 +- .../data-model/ingestion-model.md | 2 +- .../data-model/internal-component-auth.md | 373 +++++++++++++++++ .../data-model/layering-and-versioning.md | 2 +- .../data-model/meta-provider-model.md | 2 +- .../data-model/notification-model.md | 2 +- .../data-model/operational-models.md | 2 +- .../ownership-sharing-allocation.md | 2 +- .../data-model/policy-contract.md | 2 +- .../data-model/policy-profiles.md | 2 +- .../data-model/provider-contract.md | 2 +- .../data-model/registry-governance.md | 2 +- .../data-model/request-dependency-graph.md | 309 +++++++++++++++ .../data-model/resource-grouping.md | 2 +- .../data-model/resource-service-entities.md | 2 +- .../data-model/resource-type-hierarchy.md | 2 +- .../data-model/scheduled-requests.md | 295 ++++++++++++++ .../architecture/data-model/scoring-model.md | 2 +- .../data-model/service-dependencies.md | 2 +- .../data-model/session-revocation.md | 355 +++++++++++++++++ .../data-model/standards-catalog.md | 281 +++++++++++++ .../data-model/storage-providers.md | 2 +- .../data-model/universal-audit.md | 2 +- .../data-model/universal-groups.md | 2 +- .../data-model/webhooks-messaging.md | 2 +- content/docs/architecture/overview.md | 8 + .../specifications/admin-api-spec.md | 107 +++++ .../specifications/consumer-api-spec.md | 174 +++++++- .../architecture/specifications/examples.md | 2 +- .../kubernetes-compatibility.md | 2 +- .../specifications/operator-interface-spec.md | 43 ++ content/docs/capabilities-matrix.md | 66 +++- content/docs/taxonomy.md | 61 +++ 55 files changed, 2773 insertions(+), 53 deletions(-) create mode 100644 content/docs/architecture/data-model/dcm-self-health.md create mode 100644 content/docs/architecture/data-model/internal-component-auth.md create mode 100644 content/docs/architecture/data-model/request-dependency-graph.md create mode 100644 content/docs/architecture/data-model/scheduled-requests.md create mode 100644 content/docs/architecture/data-model/session-revocation.md create mode 100644 content/docs/architecture/data-model/standards-catalog.md diff --git a/content/docs/README.md b/content/docs/README.md index c8e18f4..b7db9f5 100644 --- a/content/docs/README.md +++ b/content/docs/README.md @@ -1,8 +1,4 @@ ---- -title: "DCM Documentation Index" -type: docs -weight: 0 ---- +# DCM Documentation Data Center Management (DCM) is an open-source governing framework for enterprise on-premises and sovereign cloud infrastructure. It provides a hyperscaler-like cloud experience on infrastructure that organizations own and control. @@ -52,8 +48,8 @@ Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI | Policy types | 7 (unified base contract + typed output schemas) | | Control plane components | 9 | | Four lifecycle states | Intent ยท Requested ยท Realized ยท Discovered | -| Capabilities | 155 across 24 domains | -| Data model documents | 38 (32 numbered + 3 foundation + 2 examples + 1 design priorities) | +| Capabilities | 189 across 31 domains | +| Data model documents | 45 (39 numbered + 3 foundation + 2 examples + 1 design priorities) | | Specifications | 10 | | Unresolved questions | 0 | diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index 060c8bb..b0542f8 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4538,7 +4538,196 @@ VER-001: URL path versioning only. VER-002: breaking change definition. VER-003: --- -## SECTION 67 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 67 โ€” SESSION TOKEN REVOCATION (doc 35 โ€” 35-session-revocation.md) + +> **Full specification:** [35-session-revocation.md](data-model/35-session-revocation.md) โ€” session lifecycle, revocation triggers, revocation registry, token introspection, AUTH-016โ€“AUTH-022. + +### Session Record +session_uuid ยท actor_uuid ยท auth_provider_uuid ยท auth_method ยท mfa_verified ยท created_at ยท expires_at ยท status (active/refreshing/revoked/expired) ยท revocation_reason ยท revoked_at ยท revoked_by + +### Session Store +Fast-queryable operational store (not GitOps-backed). Redis/Postgres (standard+) or in-memory (minimal/dev). Profile-governed TTLs: minimal PT8H โ†’ sovereign PT15M. Concurrent session limits: unlimited(minimal) โ†’ 1(sovereign). + +### Revocation Triggers +actor_logout (single session, self) ยท actor_logout_all (all sessions, self) ยท actor_deprovisioned (all sessions โ€” parallel with CPX-006) ยท actor_suspended ยท security_event (emergency, no grace period) ยท concurrent_limit_exceeded (oldest session evicted) ยท auth_provider_deregistered ยท credential_compromised ยท admin_forced_logout + +### Session Revocation Registry +Fast-queryable store of revoked-but-not-yet-expired session UUIDs. ALL components that accept bearer tokens MUST check this on every request. Cache age: PT5M(minimal) โ†’ no cache(sovereign). AUTH-018. + +### Actor Deprovisioning (AUTH-016) +Session revocation and credential revocation (CPX-006) are PARALLEL operations. Deprovisioning not acknowledged until BOTH complete. Neither blocks the other. + +### Emergency Revocation (AUTH-019) +security_event trigger: immediate, no grace period. auth.security_session_revoked event: urgency critical, non-suppressable. SLA: PT30S(standard) โ†’ PT5S(sovereign). + +### Token Introspection +POST /api/v1/auth/introspect (RFC 7662). Returns {active: true/false, session_uuid, actor_uuid, expires_at, mfa_verified, roles}. Requires introspection scope. AUTH-020. + +### Consumer API Session Endpoints +DELETE /api/v1/auth/session (logout single) ยท DELETE /api/v1/auth/sessions (logout all) ยท GET /api/v1/auth/sessions (list active) ยท DELETE /api/v1/auth/sessions/{uuid} (revoke specific) +Admin: POST /api/v1/admin/actors/{uuid}/revoke-sessions (force revoke, requires reason) + +### AUTH-016โ€“AUTH-022 System Policies +AUTH-016: deprovisioning fires session + credential revocation in parallel. AUTH-017: revocation SLA PT5M(minimal) โ†’ PT5S(sovereign). AUTH-018: all components check revocation registry. AUTH-019: emergency revocation = critical urgency, non-suppressable. AUTH-020: introspection endpoint requires introspection scope. AUTH-021: oldest session evicted at concurrent limit. AUTH-022: refresh tokens invalidated when parent session revoked. + +--- + +## SECTION 68 โ€” INTERNAL COMPONENT AUTHENTICATION (doc 36 โ€” 36-internal-component-auth.md) + +> **Full specification:** [36-internal-component-auth.md](data-model/36-internal-component-auth.md) โ€” component identity, Internal CA, bootstrap tokens, communication graph, ICOM-001โ€“ICOM-009. + +### Two-Layer Enforcement +Mesh layer (Istio/mTLS): prevents impersonation at transport. Application layer (DCM): enforces what each component is permitted to do. BOTH required. Network position grants zero trust โ€” internal calls receive same boundary checks as external. + +### Component Identity +Each component has: component_uuid ยท component_type ยท mTLS certificate (from Internal CA) ยท service_account_uuid ยท allowed_operations ยท allowed_targets list. + +### Component Types +api_gateway ยท policy_engine ยท placement_engine ยท request_orchestrator ยท scoring_engine ยท drift_reconciler ยท lifecycle_enforcer ยท notification_router ยท audit_store ยท session_store ยท message_bus ยท credential_provider_proxy + +### Communication Graph (enforced, not advisory) +Consumer/Admin โ†’ API Gateway โ†’ Request Orchestrator โ†’ Policy Engine / Placement Engine / Scoring Engine +All components โ†’ Session Store (revocation check) + Credential Provider Proxy (interaction creds) +ICOM-004: components may ONLY call declared allowed_targets. Unauthorized source โ†’ 403 + ICOM_UNAUTHORIZED_SOURCE audit (urgency: high). ICOM-003. + +### Every Internal Call Requires +1. mTLS certificate from Internal CA (transport identity) +2. ZTS-002 scoped interaction credential (operation authorization) โ€” scoped to specific operation + target component, valid PT5M max +3. Correlation ID + +### Internal CA +Per-deployment CA. Certificates: ECDSA-P-384, P90D lifetime, auto-renew P14D before expiry. CRL + OCSP endpoints. Internal CA root cert installed in ALL component trust stores at deploy time. ICOM-006, ICOM-009. + +### Bootstrap Protocol (ICOM-007) +New component has no cert yet. Platform admin generates one-time bootstrap token (PT1H max). Component uses bootstrap token โ†’ gets first cert from Internal CA โ†’ token invalidated immediately. Kubernetes: token injected as Secret, deleted by component after cert acquisition. Unused tokens auto-expire at PT1H. + +### Certificate Compromise (ICOM-008) +Compromised cert โ†’ added to Internal CA CRL immediately โ†’ all components refresh CRL within profile SLA (PT15S sovereign, PT1M standard) โ†’ ICOM_CERT_COMPROMISED audit (urgency: critical) โ†’ platform admin notified โ†’ new cert issued. + +### ICOM-001โ€“ICOM-009 System Policies +ICOM-001: mTLS required ALL internal calls, no exceptions. ICOM-002: interaction credential required IN ADDITION to mTLS. ICOM-003: unauthorized source โ†’ 403 + high-urgency audit. ICOM-004: components only call declared allowed_targets. ICOM-005: all internal calls audited. ICOM-006: component certs max P90D, Internal CA only. ICOM-007: bootstrap tokens one-time-use PT1H max. ICOM-008: compromised certs โ†’ CRL immediately. ICOM-009: Internal CA root in all trust stores; no external CA certs for internal comms. + +--- + +## SECTION 69 โ€” SCHEDULED AND DEFERRED REQUESTS (doc 37 โ€” 37-scheduled-requests.md) + +> **Full specification:** [37-scheduled-requests.md](data-model/37-scheduled-requests.md) โ€” scheduling model, dual policy evaluation, maintenance windows, Request Scheduler component, SCH-001โ€“SCH-006. + +### Scheduling Model +schedule.dispatch: immediate (default) | at (specific time with not_before/not_after) | window (maintenance window reference) | recurring (cron expression). Added as optional field on POST /api/v1/requests โ€” no new submission endpoint. + +### SCHEDULED Status +Request enters SCHEDULED status in Intent State after passing declaration-time GateKeeper. Visible in GET /api/v1/requests?status=SCHEDULED. Cancellable via DELETE /api/v1/requests/{uuid} before dispatch. request.scheduled event (info urgency). + +### Dual Policy Evaluation (SCH-001) +GateKeeper runs at declaration time (fail fast) AND at dispatch time (validate against current state). Dispatch-time rejection โ†’ FAILED with schedule_policy_rejection (SCH-003). Data, quotas, policies may all change between declaration and dispatch. + +### Deadline Enforcement (SCH-005) +not_after: if passed without dispatch โ†’ FAILED with schedule_deadline_missed. No retry. request.failed event (medium urgency). + +### Maintenance Windows +Reusable named recurrence artifacts. Platform admin creates; consumers reference by window_uuid in schedule. Admin: POST /api/v1/admin/maintenance-windows. Consumer: GET /api/v1/maintenance-windows. + +### New Events (added to doc 33) +request.scheduled ยท request.schedule_cancelled ยท request.schedule_deadline_missed (17 total in request.* domain) + +--- + +## SECTION 70 โ€” REQUEST DEPENDENCY GRAPH (doc 38 โ€” 38-request-dependency-graph.md) + +> **Full specification:** [38-request-dependency-graph.md](data-model/38-request-dependency-graph.md) โ€” consumer-declared cross-request ordering, field injection, PENDING_DEPENDENCY status, RDG-001โ€“RDG-006. + +### What This Is +Consumer-declared ordering of INDEPENDENT requests. Distinct from: type-level deps (doc 07, resolved automatically) and Meta Provider composition (doc 30, platform team defines). Use when no Meta Provider exists for the compound deployment. + +### Request Dependency Group +POST /api/v1/request-groups โ€” submit multiple requests with depends_on declarations using local refs. Response includes group_uuid and per-request entity_uuids. GET /api/v1/request-groups/{uuid} for group status. DELETE to cancel. + +### PENDING_DEPENDENCY Status +Dependent request waits in PENDING_DEPENDENCY until dependency reaches wait_for: acknowledged|approved|dispatched|realized (default: realized). Quota counted at group submission, not at dispatch. RDG-004. + +### Field Injection +inject_fields: pass realized output fields from dependency (e.g. IP address) into dependent request's fields automatically at dispatch time. Subject to Transformation policies. RDG-003. + +### Failure Handling +on_failure: cancel_remaining (dependents โ†’ CANCELLED with dependency_failed) | continue (only directly-dependent requests fail). Group timeout: all non-terminal โ†’ FAILED with group_timeout. RDG-005. + +### Constraints (RDG-001, RDG-002, RDG-006) +Circular deps rejected at submission (422) โ€” must be a DAG. Max 50 requests per group. Request may belong to at most ONE group (409 on second add). + +### New Events (added to doc 33) +request.pending_dependency ยท request.dependency_met ยท request.group_completed ยท request.group_failed + +--- + +## SECTION 71 โ€” DCM SELF-HEALTH ENDPOINTS (doc 39 โ€” 39-dcm-self-health.md) + +> **Full specification:** [39-dcm-self-health.md](data-model/39-dcm-self-health.md) โ€” liveness, readiness, component health, Prometheus metrics, HLT-001โ€“HLT-006. + +### Three Endpoints +GET /livez (liveness โ€” PT5S max, no external calls, unauthenticated, Kubernetes restarts on fail) ยท +GET /readyz (readiness โ€” checks Session Store + Audit Store + Policy Engine + Message Bus + Auth Provider, unauthenticated, Kubernetes removes from LB on fail) ยท +GET /api/v1/admin/health (per-component detail, admin auth required) + +### Liveness (/livez) +{status: pass|fail}. Fail if: deadlocked, Internal CA unreachable. Responds within PT5S. No DB reads, no external calls. HLT-002. + +### Readiness (/readyz) +{status: pass|warn|fail, checks: {session_store, audit_store, policy_engine, message_bus, auth_provider}}. Fail if ANY core dependency unreachable. Warn if optional component degraded. Used for startup probe (failureThreshold 30 ร— 10s = 300s startup allowance). HLT-003, HLT-006. + +### Component Health (/api/v1/admin/health) +Full per-component status: api_gateway, request_orchestrator, policy_engine, placement_engine, scoring_engine, request_scheduler, drift_reconciler, lifecycle_enforcer, discovery_scheduler, notification_router, session_store, audit_store, message_bus, internal_ca. Plus providers{registered/healthy/degraded/unhealthy} and auth_providers summary. + +### Prometheus Metrics (/metrics) +dcm_requests_total ยท dcm_request_duration_seconds ยท dcm_requests_pending_dependency_total ยท dcm_policy_evaluations_total ยท dcm_sessions_active_total ยท dcm_session_revocations_total ยท dcm_drift_open_records_total ยท dcm_providers_healthy_total ยท dcm_internal_ca_certificates_active. HLT-005. + +### Kubernetes Manifest +livenessProbe: /livez PT5S timeout, 10s period, 3 failures. +readinessProbe: /readyz PT10S timeout, 5s period, 6 failures. +startupProbe: /readyz PT10S timeout, 10s period, 30 failures (allows 300s startup). + +--- + +## SECTION 72 โ€” STANDARDS AND COMPLIANCE CATALOG (doc 40 โ€” 40-standards-catalog.md) + +> **Full specification:** [40-standards-catalog.md](data-model/40-standards-catalog.md) โ€” authoritative source for all RFCs, protocols, cryptographic standards, CNCF projects, and compliance frameworks used in DCM. + +### Internet Standards (IETF RFCs) โ€” Normative +Auth/AuthZ: RFC 7519 (JWT) ยท RFC 7517 (JWK) ยท RFC 7662 (Token Introspection) ยท RFC 6749 (OAuth 2.0) ยท RFC 4511 (LDAP) ยท RFC 7643/7644 (SCIM 2.0) +Transport: RFC 8446 (TLS 1.3, preferred) ยท RFC 5246 (TLS 1.2, minimum) ยท RFC 5280 (X.509/CRL) ยท RFC 6960 (OCSP) +Certificate enrollment: RFC 7030 (EST, preferred) ยท RFC 8555 (ACME) ยท RFC 8894 (SCEP, optional) ยท RFC 4210 (CMP, optional) +API lifecycle: RFC 8594 (Sunset header, VER-003) ยท RFC 9745 (Deprecation header, VER-003) +Health/discovery: RFC 8615 (Well-Known URIs, /livez /readyz /.well-known/dcm-api-versions) +Data: ISO 8601 (all timestamps and durations) ยท RFC 8259 (JSON, all API bodies) + +### Cryptographic Standards +Permitted algorithms: ECDSA P-384 (Internal CA, all profiles), AES-256-GCM, SHA-256 minimum, SHA-384/512 for fsi+ +RSA permitted only โ‰ฅ 2048 bits +TLS: 1.3 preferred, 1.2 minimum โ€” TLS 1.0/1.1 strictly prohibited in ALL profiles +FIPS 140-2 Level 1+ (standard/prod), Level 2+ (fsi/fedramp), Level 3 (sovereign/dod_il4) +FORBIDDEN (all profiles, no exceptions): MD5, SHA-1, DES, 3DES, RC4, RSA < 2048 + +### Authentication Assurance Levels (NIST SP 800-63B) +minimal/dev: AAL1 (single factor OK) ยท standard/prod: AAL2 (MFA required) ยท fsi: AAL2+ (phishing-resistant) ยท sovereign: AAL3 (hardware authenticator) + +### Compliance Frameworks and DCM Profile/Overlay Mapping +HIPAA โ†’ fsi profile + hipaa overlay ยท PCI DSS โ†’ pci_dss overlay (P90D max rotation, 12-month audit) ยท FedRAMP Moderate/High โ†’ fedramp_moderate/fedramp_high overlays (NIST 800-53) ยท DoD IL4 โ†’ dod_il4 overlay (FIPS 140-2 Level 2, hardware attestation) ยท GDPR โ†’ sovereignty constraints + data classification ยท ISO 27001 โ†’ all profiles (risk-based approach) ยท SOC 2 โ†’ standard+ (Type II audit trail) ยท NIST SP 800-53 โ†’ FedRAMP profiles + +### CNCF Ecosystem (Graduated Projects) +Kubernetes (deployment, CRD operator, resource model) ยท OPA/Open Policy Agent (policy engine backend, Rego) ยท Prometheus (metrics, /metrics endpoint) ยท OpenTelemetry (tracing, correlation IDs) ยท Istio (internal mTLS service mesh) ยท Argo CD / Flux (GitOps delivery) ยท SPIFFE (workload identity concept โ€” inspiration for ICOM component identity model) + +### Operational Standards (Normative) +W3C SSE / Server-Sent Events (GET /api/v1/requests/{uuid}/stream โ€” live status without polling) ยท OpenAPI 3.1 (REST API spec format โ€” consumer, admin, OIS specs) ยท Unix cron / POSIX (recurring schedule expressions in doc 37) ยท IANA health+json (RFC 8615 โ€” /livez /readyz health response format) ยท GitOps / OpenGitOps v1.0 (all DCM artifacts in Git; PR-based contribution) + +### External CA Credential Providers (Optional) +HashiCorp Vault PKI (native API + EST/ACME; recommended enterprise PKI for fsi/sovereign; operates as subordinate CA) ยท Venafi TLS Protect (ACME/EST/REST) ยท EJBCA (ACME/CMP/SCEP) โ€” all implemented as x509_certificate Credential Providers per doc 31; NOT Auth Providers + +### Policy Family โ†’ Standards Mapping (doc 40 Section 9) +AUTH โ†’ RFC 6749/7519/7662/OIDC/SCIM ยท CPX โ†’ FIPS 140/RFC 5280/8555/7030/8894/4210 ยท ICOM โ†’ RFC 8446/5280/SPIFFE/FIPS 140 ยท VER โ†’ RFC 8594/9745 ยท SES โ†’ RFC 7662/7009 ยท HLT โ†’ RFC 8615/Kubernetes probes ยท ZTS โ†’ NIST SP 800-207/800-63B ยท SCH/RDG โ†’ industry scheduling/DAG patterns ยท SMX/ATM โ†’ organizational risk governance + +--- + +## SECTION 73 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure): @@ -4563,6 +4752,23 @@ When working on this project, apply these instructions in addition to the number 198. **Admin API base URL is /api/v1/admin/ (version-first)** โ€” NOT /admin/api/v1/; all 40 admin endpoints use /api/v1/admin/; this is the authoritative form used everywhere in the specs and data model docs 199. **Consumer API has idempotency (1.5), rate limiting (1.6), request IDs (1.7), and standard envelope (1.8)** โ€” POST requests support Idempotency-Key header (PT24H retention); rate limits are profile-governed (60/min minimal โ†’ 600/min sovereign) with Retry-After on 429; all list responses use {"items":[...],"total":N,"next_cursor":"..."} envelope; X-DCM-Request-ID and X-DCM-Correlation-ID on all responses 200. **OIS health check response is normative (not optional)** โ€” providers MUST return {status: pass|warn|fail, version, dcm_registration_status}; missing/malformed body = warn; 3 consecutive non-200 = provider.unhealthy event; response format follows RFC 8615 / IANA health+json +201. **Doc 35 (35-session-revocation.md) is complete** โ€” session lifecycle: intentโ†’requestedโ†’realized store model; AUTH-016 (deprovisioning revokes sessions AND credentials in parallel); AUTH-017 (revocation SLA: PT5M minimal โ†’ PT5S sovereign); AUTH-018 (ALL components check revocation registry on every bearer token request โ€” no exceptions); AUTH-019 (emergency revocation = critical urgency, non-suppressable); AUTH-020 (introspection endpoint authenticated); AUTH-021 (oldest session revoked on concurrent limit breach); AUTH-022 (refresh tokens invalidated on parent session revocation); session endpoints: DELETE /api/v1/auth/session, DELETE /api/v1/auth/sessions, GET /api/v1/auth/sessions, DELETE /api/v1/auth/sessions/{uuid}; admin: POST /api/v1/admin/actors/{uuid}/revoke-sessions +202. **Doc 36 (36-internal-component-auth.md) is complete** โ€” ICOM-001 (all internal calls mTLS); ICOM-002 (scoped interaction credential required IN ADDITION to mTLS on every call); ICOM-003 (unauthorized source โ†’ 403 + audit); ICOM-004 (components may only call declared allowed_targets); ICOM-005 (all internal calls audited); ICOM-006 (P90D max cert validity); ICOM-007 (bootstrap tokens one-time-use, PT1H max); ICOM-008 (compromised cert โ†’ CRL immediately); ICOM-009 (Internal CA root in all trust stores at deploy); component communication graph is declared and enforced โ€” not implicit; every call: mTLS cert (transport identity) + ZTS-002 interaction credential (operation authorization) +203. **Session revocation and internal component auth complete the zero trust model** โ€” external boundary: providerโ†”DCM uses mTLS + scoped credentials (CPX-001โ€“CPX-012); internal boundary: componentโ†”component uses Internal CA mTLS + ZTS-002 interaction credentials (ICOM-001โ€“ICOM-009); actor sessions: Auth Provider issues tokens; Session Revocation Registry checked on every request (AUTH-018); credentials: Credential Provider manages values that never touch DCM stores (CPX-001); together these four surfaces cover the complete trust boundary + +201. **35-session-revocation.md (AUTH-016โ€“AUTH-022)** โ€” session revocation and credential revocation are PARALLEL on actor deprovisioning (not sequential); revocation registry must be checked on EVERY request by ALL components; sovereign profile: no revocation registry cache; emergency revocation (security_event) is critical urgency + non-suppressable; refresh tokens are invalidated when parent session is revoked (AUTH-022) +202. **36-internal-component-auth.md (ICOM-001โ€“ICOM-009)** โ€” network position grants ZERO trust for internal calls โ€” same five-check boundary model as external; every internal call requires BOTH mTLS cert AND ZTS-002 interaction credential; bootstrap tokens are one-time-use PT1H max; unauthorized source component โ†’ 403 + high-urgency audit; component certs from Internal CA only, max P90D, never external CA +203. **SES and ICOM domains added to capabilities matrix** โ€” matrix is now 177 capabilities across 28 domains; SES-001โ€“SES-005 (session lifecycle, deprovisioning, emergency revocation, introspection, concurrent enforcement); ICOM-001โ€“ICOM-005 (mTLS, bootstrap, call authorization, interaction credentials, cert revocation) +204. **Domain prefix totals now 28** โ€” IAM CAT REQ PRV LCM DRF POL LAY INF ING AUD OBS STO FED GOV ACC ZTS GMX DRC FCM SMX MPX CPX DPO ATM EVT VER SES ICOM; README and taxonomy both updated to 177/28 +205. **Doc 37 (Scheduled Requests): dual policy evaluation** โ€” GateKeeper runs at declaration AND at dispatch; dispatch-time rejection = FAILED not retried; schedule field is optional addition to existing POST /api/v1/requests body; SCHEDULED status is cancellable; not_after deadline miss = terminal FAILED (SCH-005) +206. **Doc 38 (Request Dependency Graph): distinct from type-level and Meta Provider deps** โ€” consumer-declared ad-hoc ordering for independent requests; POST /api/v1/request-groups; PENDING_DEPENDENCY status counts against quota at submission not dispatch; max 50 requests per group; circular deps โ†’ 422 at submission; field injection passes realized outputs into dependent request fields automatically +207. **Doc 39 (DCM Self-Health): three endpoints, different purposes** โ€” /livez (liveness, PT5S max, no external calls, Kubernetes restarts pod on fail) vs /readyz (readiness, checks 5 core dependencies, Kubernetes removes from LB) vs /api/v1/admin/health (per-component detail, admin auth required, Prometheus metrics at /metrics); all follow RFC 8615 / IANA health+json +208. **Capabilities matrix now 189 across 31 domains** โ€” SES(5) ICOM(5) SCH(4) RDG(4) HLT(4) added; domain prefixes: IAM CAT REQ PRV LCM DRF POL LAY INF ING AUD OBS STO FED GOV ACC ZTS GMX DRC FCM SMX MPX CPX DPO ATM EVT VER SES ICOM SCH RDG HLT (31 total) +209. **40-standards-catalog.md is the authoritative source for all DCM standards** โ€” forbidden algorithms (MD5, SHA-1, DES, 3DES, RC4, RSA<2048) are prohibited in ALL profiles with no exceptions; TLS 1.0/1.1 prohibited in ALL profiles; ECDSA P-384 is the mandated algorithm for Internal CA certs; AAL mapping: minimal/dev=AAL1, standard/prod=AAL2, fsi=AAL2+, sovereign=AAL3; doc 40 Section 8 maps every standard to the docs that use it +209. **External CAs belong in the Credential Provider (NOT Auth Provider)** โ€” Auth Provider authenticates identity; Credential Provider manages credential lifecycle; External CAs (Vault PKI, Venafi, EJBCA, AWS ACM PCA) are x509_certificate Credential Providers using ACME(RFC 8555)/EST(RFC 7030)/SCEP/CMP protocols; ICOM-009 updated โ€” trust anchor is any registered CA root, not just built-in Internal CA; doc 36 profile table: cert lifetime P180D(minimal) โ†’ P14D(sovereign); sovereign requires HSM-backed certs if hardware_attested posture +210. **Live updates: SSE stream + OIS interim status** โ€” GET /api/v1/requests/{uuid}/stream (text/event-stream, closes on terminal status) for browser/CLI without polling; events: status_change, progress_updated, approval_required, approval_recorded, heartbeat(30s); OIS providers POST /api/v1/provider/entities/{uuid}/status for interim progress with step_current/step_total/constituent_status; request.progress_updated event added to doc 33; rate-limited: max 1 interim status per 10s per entity +211. **Profile coverage added to docs 36-39** โ€” doc36 cert lifetime table(P180D minimalโ†’P14D sovereign), algorithm min table; doc37 max scheduling horizon(P365D minimalโ†’P7D sovereign), concurrent scheduled limit, maintenance window approval tier; doc38 max group size(100 minimalโ†’5 sovereign), group timeout max, field injection validation strictness, nesting depth; doc39 metrics scraping restrictions(sovereign internal only), /api/v1/admin/health MFA requirements(fsi/sovereign) +212. **40-standards-catalog.md is the authoritative standards reference** โ€” 19 RFCs, 3 cryptographic standards tables (permitted algorithms, forbidden algorithms, FIPS levels), 6 compliance frameworks, 7 CNCF ecosystem projects, W3C SSE, OpenAPI 3.1, SPIFFE (informative), HashiCorp Vault PKI / Venafi / EJBCA as External CA Credential Provider backends (NOT Auth Providers); Section 9 maps all 17 policy families to their standards basis; usage map tracks which standards appear in which documents 195. **33-event-catalog.md is the SINGLE authoritative source for all DCM event types** โ€” 82 events across 26 domains; all events share the base envelope (event_uuid, event_type, event_schema_version, timestamp from Commit Log, urgency, payload, links); consumers implement idempotency using event_uuid; critical urgency events are non-suppressable; non-standard events use reverse-DNS prefix; event_schema_version only increments on breaking changes 194. **Tier registry changes are gated by impact detection** โ€” any change that creates a SECURITY_DEGRADATION (tier gravity or position decreased) blocks activation until each degradation is explicitly accepted by a verified-tier or above reviewer via Admin API; BROKEN_REFERENCE also blocks; PROFILE_GAP is a warning that does not block; all changes produce an impact report in the Audit Store (ATM-009โ€“012) 193. **Authority tiers are named positions in an ordered list โ€” not fixed enum values** โ€” tier weight derived from list position at evaluation time; organizations insert custom tiers between existing ones without breaking existing name references; 'authorized' tier always means 'highest current gravity' regardless of what's been inserted before it; ATM-001: never hardcode tier weights diff --git a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md index b3c1a7f..8c3c3a5 100644 --- a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md +++ b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md @@ -10,7 +10,7 @@ weight: 26 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference > **Foundation Document Reference** @@ -389,6 +389,8 @@ Outbound interaction assembled (DCM โ†’ Provider OR DCM โ†’ DCM) --- +> **Internal component authentication:** See [Internal Component Authentication](36-internal-component-auth.md) for the complete internal auth model including component identity, Internal CA, bootstrap tokens, and ICOM-001โ€“ICOM-009 policies. + ## 5. Zero Trust Interaction Model ### 5.1 Principle diff --git a/content/docs/architecture/data-model/api-versioning-strategy.md b/content/docs/architecture/data-model/api-versioning-strategy.md index cd3890b..22faf0f 100644 --- a/content/docs/architecture/data-model/api-versioning-strategy.md +++ b/content/docs/architecture/data-model/api-versioning-strategy.md @@ -4,7 +4,7 @@ type: docs weight: 34 --- -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference โ€” API Versioning and Lifecycle **Related Documents:** [Consumer API Specification](../specifications/consumer-api-spec.md) | [Admin API Specification](../specifications/dcm-admin-api-spec.md) | [Operator Interface Specification](../specifications/dcm-operator-interface-spec.md) | [Event Catalog](33-event-catalog.md) | [Registry Governance](20-registry-governance.md) | [Design Priorities](00-design-priorities.md) diff --git a/content/docs/architecture/data-model/audit-provenance-observability.md b/content/docs/architecture/data-model/audit-provenance-observability.md index b1f0687..c1c8e01 100644 --- a/content/docs/architecture/data-model/audit-provenance-observability.md +++ b/content/docs/architecture/data-model/audit-provenance-observability.md @@ -11,7 +11,7 @@ weight: 12 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Four States](02-four-states.md) | [Storage Providers](11-storage-providers.md) | [Context and Purpose](00-context-and-purpose.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/auth-providers.md b/content/docs/architecture/data-model/auth-providers.md index ae700d0..e1c1c40 100644 --- a/content/docs/architecture/data-model/auth-providers.md +++ b/content/docs/architecture/data-model/auth-providers.md @@ -10,7 +10,7 @@ weight: 19 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).* -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Webhooks and Messaging](18-webhooks-messaging.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) > **Foundation Document Reference** @@ -491,6 +491,8 @@ dcm auth configure \ --- +> **Session revocation lifecycle:** See [Session Token Revocation](35-session-revocation.md) for the complete session revocation model including AUTH-016โ€“AUTH-022 (actor deprovisioning, revocation registry, token introspection, concurrent session enforcement). + ## 9. System Policies | Policy | Rule | diff --git a/content/docs/architecture/data-model/authority-tier-model.md b/content/docs/architecture/data-model/authority-tier-model.md index 9d8b670..bebf69f 100644 --- a/content/docs/architecture/data-model/authority-tier-model.md +++ b/content/docs/architecture/data-model/authority-tier-model.md @@ -4,7 +4,7 @@ type: docs weight: 32 --- -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference โ€” Authority Tier Specification **Related Documents:** [Design Priorities](00-design-priorities.md) | [Scoring Model](29-scoring-model.md) | [Federated Contribution Model](28-federated-contribution-model.md) | [Policy Profiles](14-policy-profiles.md) | [Registry Governance](20-registry-governance.md) diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md index 45caebd..016e4fc 100644 --- a/content/docs/architecture/data-model/context-and-purpose.md +++ b/content/docs/architecture/data-model/context-and-purpose.md @@ -11,7 +11,7 @@ weight: 0 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md index 71597ea..a33555e 100644 --- a/content/docs/architecture/data-model/control-plane-components.md +++ b/content/docs/architecture/data-model/control-plane-components.md @@ -10,7 +10,7 @@ weight: 25 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference > **Foundation Document Reference** @@ -26,7 +26,7 @@ weight: 25 > Runtime implementations of the three abstractions โ€” not a fourth abstraction -**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Operational Models](24-operational-models.md) | [Policy Profiles](14-policy-profiles.md) +**Related Documents:** [Internal Component Authentication](36-internal-component-auth.md) | [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Operational Models](24-operational-models.md) | [Policy Profiles](14-policy-profiles.md) --- @@ -35,6 +35,9 @@ The DCM Control Plane consists of **nine components** that implement the three f ## 1. Purpose +> **Internal component authentication:** See [Internal Component Authentication](36-internal-component-auth.md) for the mTLS and interaction credential model governing all component-to-component calls within the DCM control plane. + + This document formally defines the DCM control plane components that are referenced throughout the data model documents but not previously specified in detail. Two components are defined here: 1. **The Request Orchestrator** โ€” the event bus and coordinator of the request lifecycle pipeline diff --git a/content/docs/architecture/data-model/credential-provider-model.md b/content/docs/architecture/data-model/credential-provider-model.md index 7fd8f15..625d5d4 100644 --- a/content/docs/architecture/data-model/credential-provider-model.md +++ b/content/docs/architecture/data-model/credential-provider-model.md @@ -10,7 +10,7 @@ weight: 31 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference โ€” Credential Provider Specification **Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Auth Providers](19-auth-providers.md) | [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Scoring Model](29-scoring-model.md) | [Federated Contribution Model](28-federated-contribution-model.md) @@ -916,3 +916,71 @@ When an idle alert fires: --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + +## External CA Integration + +DCM's Credential Provider model natively supports external Certificate Authorities as backends for the `x509_certificate` credential type. This is the correct place for enterprise PKI integration โ€” not the Auth Provider. + +### Supported Protocols + +| Protocol | RFC | Common Implementations | Use case | +|----------|-----|------------------------|----------| +| ACME | RFC 8555 | Let's Encrypt, cert-manager, Venafi, DigiCert | Public and enterprise CAs with ACME support | +| EST | RFC 7030 | Cisco CA, Microsoft NDES, Venafi | Enterprise PKI, IoT, internal use | +| SCEP | RFC 8894 | Microsoft NDES, Cisco iOS CA | Legacy enterprise PKI, network equipment | +| CMP | RFC 4210 | EJBCA, OpenXPKI | High-assurance enterprise PKI | +| Native API | โ€” | HashiCorp Vault PKI, AWS ACM PCA, Azure Key Vault | Cloud-native PKI | + +### External CA Registration + +```yaml +credential_provider_registration: + provider_type: credential_provider + credential_types: [x509_certificate] + + external_ca_config: + ca_protocol: acme | est | scep | cmp | vault_pki | aws_acm_pca | azure_key_vault + ca_endpoint: + + # Protocol-specific + acme_config: + directory_url: + account_key_credential_uuid: + preferred_challenge: dns-01 | http-01 | tls-alpn-01 + + vault_pki_config: + vault_addr: + mount_path: pki + role_name: dcm-internal + vault_token_credential_uuid: + + # Common to all + ca_chain_pem: # for trust store installation + issued_cert_lifetime: P90D # profile-governed; may be overridden by CA + subject_template: "CN={{component_type}}-{{component_uuid}},O=dcm-internal" +``` + +### How DCM Uses External CA Credential Providers + +When an external CA Credential Provider is registered and configured as the trust anchor for internal component auth (doc 36), DCM's component certificate requests flow through the Credential Provider interface instead of the built-in Internal CA: + +``` +Component needs certificate + โ”‚ + โ–ผ Request to Credential Provider Proxy + โ”‚ credential_type: x509_certificate + โ”‚ subject: CN=-,O=dcm-internal + โ”‚ san: [component_uuid, component_name, dns_name] + โ”‚ + โ–ผ Credential Provider Proxy โ†’ External CA Credential Provider + โ”‚ Issues certificate request via configured protocol (ACME/EST/Vault/etc.) + โ”‚ + โ–ผ CA issues certificate (signed by enterprise root) + โ”‚ + โ–ผ Certificate returned to component + โ”‚ Component uses for mTLS โ€” same as built-in CA path + โ”‚ Certificate in enterprise PKI chain โ†’ auditable in enterprise tooling +``` + +This design means DCM's internal mTLS is fully auditable through existing enterprise PKI infrastructure when using an external CA โ€” a key requirement for fsi and sovereign profiles. + diff --git a/content/docs/architecture/data-model/dcm-federation.md b/content/docs/architecture/data-model/dcm-federation.md index 8b11533..ac25d33 100644 --- a/content/docs/architecture/data-model/dcm-federation.md +++ b/content/docs/architecture/data-model/dcm-federation.md @@ -10,7 +10,7 @@ weight: 22 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Federated Contribution Model](28-federated-contribution-model.md) | [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/dcm-self-health.md b/content/docs/architecture/data-model/dcm-self-health.md new file mode 100644 index 0000000..004490a --- /dev/null +++ b/content/docs/architecture/data-model/dcm-self-health.md @@ -0,0 +1,374 @@ +--- +title: "DCM Self-Health Endpoints" +type: docs +weight: 39 +--- + +**Document Status:** โœ… Complete +**Document Type:** Architecture Reference โ€” Operational Health +**Related Documents:** [Deployment and Redundancy](17-deployment-redundancy.md) | [Internal Component Authentication](36-internal-component-auth.md) | [Operator Interface Specification](../specifications/dcm-operator-interface-spec.md) | [Admin API Specification](../specifications/dcm-admin-api-spec.md) + +> **Events:** Health state change events fire as `provider.healthy` / `provider.unhealthy` for external systems, and `governance.profile_changed` when health thresholds are adjusted. See [Event Catalog](33-event-catalog.md). + +> **This document maps to: PROVIDER** +> +> DCM itself must expose the same health contract it requires of Service Providers (doc OIS ยง4). This document specifies DCM's own liveness, readiness, and component health endpoints โ€” required for Kubernetes operator deployment, load balancer health checking, and operational monitoring. + +--- + +## 1. Three Health Endpoints + +DCM exposes three distinct health endpoints, following Kubernetes conventions: + +| Endpoint | Purpose | Failure action | Authentication | +|----------|---------|---------------|---------------| +| `GET /livez` | Is DCM alive? | Kubernetes restarts the pod | None | +| `GET /readyz` | Is DCM ready to serve traffic? | Kubernetes removes from load balancer | None | +| `GET /api/v1/admin/health` | Detailed component status | Informational โ€” no automatic action | Admin auth required | + +Liveness and readiness are unauthenticated because they must work before authentication infrastructure is operational (e.g. during startup). + +--- + +## 2. Liveness โ€” `/livez` + +Liveness answers: **is this DCM process alive?** + +A liveness failure means the process is deadlocked, in an unrecoverable state, or otherwise unable to continue. Kubernetes responds by restarting the pod. + +```http +GET /livez HTTP/1.1 + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass" +} +``` + +**Liveness checks (minimal โ€” fast):** +- Process is responding +- No deadlock detected in core event loop +- Internal CA is reachable (for deployments with component auth) + +**Liveness failure response:** +```http +HTTP/1.1 503 Service Unavailable +Content-Type: application/health+json + +{ + "status": "fail", + "failure_reason": "event_loop_deadlock | internal_ca_unreachable | oom_imminent" +} +``` + +**Liveness SLA:** Must respond within PT5S. No external calls. No database reads. + +--- + +## 3. Readiness โ€” `/readyz` + +Readiness answers: **is this DCM instance ready to serve requests?** + +A readiness failure removes the instance from the load balancer rotation without restarting it. This handles startup, migration, and graceful drain scenarios. + +```http +GET /readyz HTTP/1.1 + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass", + "checks": { + "session_store": "pass", + "audit_store": "pass", + "policy_engine": "pass", + "message_bus": "pass", + "auth_provider": "pass" + } +} +``` + +**Readiness checks:** +- Session Store: can write and read a test record +- Audit Store: reachable and writable +- Policy Engine: responding to internal health ping +- Message Bus: connected and subscribed +- Auth Provider: at least one Auth Provider is responding +- Schema version: database schema matches running code version + +**Readiness failure** (any check fails): +```http +HTTP/1.1 503 Service Unavailable +Content-Type: application/health+json + +{ + "status": "fail", + "checks": { + "session_store": "pass", + "audit_store": "fail", + "policy_engine": "pass", + "message_bus": "pass", + "auth_provider": "pass" + }, + "failing_checks": ["audit_store"] +} +``` + +**Readiness SLA:** Must respond within PT10S. Performs lightweight connectivity checks โ€” no heavy queries. + +### 3.1 Startup vs Operational Readiness + +During startup, DCM goes through a startup sequence before becoming ready: + +``` +Process starts + โ”‚ + โ–ผ /livez โ†’ pass (process alive) + โ”‚ /readyz โ†’ fail (not ready yet) + โ”‚ + โ–ผ Internal CA connects โ†’ component certs verified + โ–ผ Session Store connected โ†’ revocation registry loaded + โ–ผ Audit Store connected โ†’ schema version validated + โ–ผ Policy Engine ready โ†’ policies loaded and shadow mode initialized + โ–ผ Auth Providers connected โ†’ at least one responding + โ–ผ Message Bus connected โ†’ subscriptions established + โ”‚ + โ–ผ /readyz โ†’ pass (ready to serve traffic) +``` + +`startupProbe` in Kubernetes uses `/readyz` with a longer `failureThreshold` to allow startup time before the liveness probe takes over. + +--- + +## 4. Detailed Health โ€” `/api/v1/admin/health` + +The detailed health endpoint provides per-component status for operational monitoring. Requires admin authentication. + +```http +GET /api/v1/admin/health HTTP/1.1 +Authorization: Bearer + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass | warn | fail", + "dcm_version": "1.2.0", + "dcm_instance_uuid": "", + "deployment_profile": "prod", + "uptime_seconds": 864023, + "checked_at": "", + + "components": { + "api_gateway": { + "status": "pass", + "latency_p99_ms": 12, + "requests_per_minute": 340 + }, + "request_orchestrator": { + "status": "pass", + "queue_depth": 3, + "in_flight": 7 + }, + "policy_engine": { + "status": "pass", + "active_policies": 42, + "shadow_policies": 3, + "evaluations_per_minute": 280 + }, + "placement_engine": { + "status": "pass" + }, + "scoring_engine": { + "status": "pass", + "evaluations_per_minute": 280 + }, + "request_scheduler": { + "status": "pass", + "scheduled_requests_queued": 5, + "next_dispatch_at": "" + }, + "drift_reconciler": { + "status": "pass", + "last_cycle_completed_at": "", + "open_drift_records": 2 + }, + "lifecycle_enforcer": { + "status": "pass", + "entities_monitored": 1240, + "ttl_warnings_pending": 3 + }, + "discovery_scheduler": { + "status": "pass", + "pending_jobs": 1, + "last_completed_at": "" + }, + "notification_router": { + "status": "pass", + "providers_active": 2, + "delivery_backlog": 0 + }, + "session_store": { + "status": "pass", + "active_sessions": 47, + "revocation_registry_size": 3 + }, + "audit_store": { + "status": "pass", + "records_last_hour": 1840, + "chain_integrity": "verified" + }, + "message_bus": { + "status": "pass", + "lag_consumer_group_ms": 12 + }, + "internal_ca": { + "status": "pass", + "certificates_active": 12, + "next_expiry_at": "" + } + }, + + "providers": { + "registered": 4, + "healthy": 4, + "degraded": 0, + "unhealthy": 0 + }, + + "auth_providers": { + "registered": 2, + "healthy": 2, + "unhealthy": 0 + } +} +``` + +### 4.1 Status Semantics + +| Status | Meaning | +|--------|---------| +| `pass` | Component fully operational | +| `warn` | Operational but degraded (high latency, reduced capacity, elevated error rate) | +| `fail` | Component not operational; DCM degraded | + +The top-level `status` is the worst status across all components: +- Any `fail` โ†’ top-level `fail` +- Any `warn`, no `fail` โ†’ top-level `warn` +- All `pass` โ†’ top-level `pass` + +--- + +## 5. Kubernetes Manifest + +```yaml +# Standard Kubernetes probe configuration for DCM +livenessProbe: + httpGet: + path: /livez + port: 8443 + scheme: HTTPS + initialDelaySeconds: 10 + periodSeconds: 10 + failureThreshold: 3 + timeoutSeconds: 5 + +readinessProbe: + httpGet: + path: /readyz + port: 8443 + scheme: HTTPS + initialDelaySeconds: 30 + periodSeconds: 5 + failureThreshold: 6 + timeoutSeconds: 10 + +startupProbe: + httpGet: + path: /readyz + port: 8443 + scheme: HTTPS + initialDelaySeconds: 10 + periodSeconds: 10 + failureThreshold: 30 # allow up to 300s for startup + timeoutSeconds: 10 +``` + +--- + +## 6. Prometheus Metrics + +DCM exposes Prometheus-compatible metrics alongside health endpoints: + +``` +GET /metrics # Prometheus scrape endpoint (unauthenticated in cluster; + # configurable for external exposure) +``` + +Key metric families: + +``` +# Request pipeline +dcm_requests_total{status, resource_type, profile} +dcm_request_duration_seconds{quantile, resource_type} +dcm_requests_pending_dependency_total +dcm_requests_scheduled_total + +# Policy engine +dcm_policy_evaluations_total{outcome, enforcement_class} +dcm_policy_shadow_divergences_total + +# Sessions +dcm_sessions_active_total +dcm_session_revocations_total{trigger} + +# Drift +dcm_drift_open_records_total{severity} +dcm_drift_detected_total + +# Providers +dcm_providers_registered_total +dcm_providers_healthy_total +dcm_provider_dispatch_duration_seconds{provider_type, quantile} + +# Internal +dcm_internal_ca_certificates_active +dcm_internal_ca_days_until_next_expiry +``` + +--- + +## 8. Profile-Governed Health Exposure + +| Profile | /livez | /readyz | /api/v1/admin/health | /metrics scraping | +|---------|--------|---------|----------------------|-------------------| +| `minimal` | Unauthenticated | Unauthenticated | Admin auth | Internal network only | +| `dev` | Unauthenticated | Unauthenticated | Admin auth | Internal network only | +| `standard` | Unauthenticated | Unauthenticated | Admin auth | mTLS client cert or auth token | +| `prod` | Unauthenticated | Unauthenticated | Admin auth | mTLS client cert or auth token | +| `fsi` | Unauthenticated | Unauthenticated | Admin auth (MFA required) | mTLS + authorized scraper registration | +| `sovereign` | Unauthenticated within cluster | Unauthenticated within cluster | Admin auth (MFA + step-up) | Disabled externally; internal only | + +**Notes:** +- `/livez` and `/readyz` are always unauthenticated *within the cluster* โ€” Kubernetes probes cannot present auth credentials. However, at the ingress boundary (external load balancer), these paths may be network-restricted. +- For `fsi` and `sovereign` profiles, `/api/v1/admin/health` requires MFA-verified sessions (mfa_verified: true). Step-up MFA is required for sovereign. +- The `sovereign` profile does not expose `/metrics` externally. Prometheus must scrape from within the cluster network only. +- Component-level detail in `/api/v1/admin/health` may be redacted in fsi/sovereign profiles based on the requesting actor's role โ€” SRE sees full detail; read-only admin sees summary only. + +## 7. System Policies + +| Policy | Rule | +|--------|------| +| `HLT-001` | DCM must expose `/livez` and `/readyz` endpoints on the same port as the API, unauthenticated, following RFC 8615 / IANA health+json format. | +| `HLT-002` | `/livez` must respond within PT5S with no external calls or database reads. A non-response within PT5S is treated as liveness failure. | +| `HLT-003` | `/readyz` returns `fail` if the Session Store, Audit Store, Policy Engine, Message Bus, or any Auth Provider is unreachable. It returns `warn` if any optional component is degraded. | +| `HLT-004` | `GET /api/v1/admin/health` requires admin authentication and provides per-component status. It must include the DCM version, instance UUID, and deployment profile. | +| `HLT-005` | DCM must expose Prometheus-compatible metrics at `GET /metrics`. Metrics must include request pipeline, policy engine, session, drift, and provider metrics at minimum. | +| `HLT-006` | The startup sequence must be observable via `/readyz`. DCM must not report `pass` on `/readyz` until the Session Store, Audit Store, Policy Engine, Auth Provider, and Message Bus are all reachable. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/deployment-redundancy.md b/content/docs/architecture/data-model/deployment-redundancy.md index 9f4d088..ec36394 100644 --- a/content/docs/architecture/data-model/deployment-redundancy.md +++ b/content/docs/architecture/data-model/deployment-redundancy.md @@ -10,7 +10,7 @@ weight: 17 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit Model](16-universal-audit.md) | [Policy Organization](14-policy-profiles.md) > **Foundation Document Reference** @@ -455,6 +455,8 @@ Read from any surviving Commit Log replica ## 8. Network Architecture +> **Full internal auth specification:** See [Internal Component Authentication](36-internal-component-auth.md) for component identity model, Internal CA, bootstrap protocol, and ICOM-001โ€“ICOM-009 system policies. + ### 8.1 Service Mesh All DCM component-to-component communication uses a service mesh (Istio or equivalent): diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md index 19473f3..6a1d28f 100644 --- a/content/docs/architecture/data-model/entity-relationships.md +++ b/content/docs/architecture/data-model/entity-relationships.md @@ -11,7 +11,7 @@ weight: 9 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) | [Information Providers](10-information-providers.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/entity-types.md b/content/docs/architecture/data-model/entity-types.md index d13ce03..5a5b420 100644 --- a/content/docs/architecture/data-model/entity-types.md +++ b/content/docs/architecture/data-model/entity-types.md @@ -10,7 +10,7 @@ weight: 1 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/event-catalog.md b/content/docs/architecture/data-model/event-catalog.md index 7849ae8..62310f3 100644 --- a/content/docs/architecture/data-model/event-catalog.md +++ b/content/docs/architecture/data-model/event-catalog.md @@ -4,7 +4,7 @@ type: docs weight: 33 --- -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference โ€” Authoritative Event Catalog **Related Documents:** [Notification Model](23-notification-model.md) | [Webhooks and Messaging](18-webhooks-messaging.md) | [Universal Audit](16-universal-audit.md) | [Credential Provider Model](31-credential-provider-model.md) | [Authority Tier Model](32-authority-tier-model.md) | [Control Plane Components](25-control-plane-components.md) @@ -107,6 +107,7 @@ links: | `request.failed` | high | Request failed at any stage | | `request.gatekeeper_rejected` | high | GateKeeper policy denied the request | | `request.cancelled` | low | Consumer cancelled; pipeline terminated | +| `request.progress_updated` | info | Provider sent interim progress update; constituent_status updated | ### 3.1 Payload Schemas diff --git a/content/docs/architecture/data-model/examples.md b/content/docs/architecture/data-model/examples.md index ab29e93..4ac0888 100644 --- a/content/docs/architecture/data-model/examples.md +++ b/content/docs/architecture/data-model/examples.md @@ -10,7 +10,7 @@ weight: 4 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Reference Examples > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/federated-contribution-model.md b/content/docs/architecture/data-model/federated-contribution-model.md index 53a9dd7..3d8454c 100644 --- a/content/docs/architecture/data-model/federated-contribution-model.md +++ b/content/docs/architecture/data-model/federated-contribution-model.md @@ -10,7 +10,7 @@ weight: 28 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference โ€” Read This First for Multi-User Data Governance **Related Documents:** [Foundational Abstractions](00-foundations.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Policy Profiles](14-policy-profiles.md) | [Registry Governance](20-registry-governance.md) | [DCM Federation](22-dcm-federation.md) | [Governance Matrix](27-governance-matrix.md) | [Consumer API](../specifications/consumer-api-spec.md) diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md index 7d981bf..26da607 100644 --- a/content/docs/architecture/data-model/foundations.md +++ b/content/docs/architecture/data-model/foundations.md @@ -10,7 +10,7 @@ weight: 0 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Foundation โ€” Read This First **Related Documents:** [Data Model Context](00-context-and-purpose.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md index 2bfb861..1ba93be 100644 --- a/content/docs/architecture/data-model/four-states.md +++ b/content/docs/architecture/data-model/four-states.md @@ -11,7 +11,7 @@ weight: 2 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/governance-matrix.md b/content/docs/architecture/data-model/governance-matrix.md index bd88849..d5c7f79 100644 --- a/content/docs/architecture/data-model/governance-matrix.md +++ b/content/docs/architecture/data-model/governance-matrix.md @@ -10,7 +10,7 @@ weight: 27 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/information-providers-advanced.md b/content/docs/architecture/data-model/information-providers-advanced.md index 4bc4699..191eea5 100644 --- a/content/docs/architecture/data-model/information-providers-advanced.md +++ b/content/docs/architecture/data-model/information-providers-advanced.md @@ -10,7 +10,7 @@ weight: 21 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Information Providers](10-information-providers.md) | [Policy Organization](14-policy-profiles.md) | [Universal Audit Model](16-universal-audit.md) | [DCM Federation](22-dcm-federation.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/information-providers.md b/content/docs/architecture/data-model/information-providers.md index 6834808..ca66ee0 100644 --- a/content/docs/architecture/data-model/information-providers.md +++ b/content/docs/architecture/data-model/information-providers.md @@ -11,7 +11,7 @@ weight: 10 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/ingestion-model.md b/content/docs/architecture/data-model/ingestion-model.md index 1215ea6..0f85688 100644 --- a/content/docs/architecture/data-model/ingestion-model.md +++ b/content/docs/architecture/data-model/ingestion-model.md @@ -10,7 +10,7 @@ weight: 13 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/internal-component-auth.md b/content/docs/architecture/data-model/internal-component-auth.md new file mode 100644 index 0000000..cbd1252 --- /dev/null +++ b/content/docs/architecture/data-model/internal-component-auth.md @@ -0,0 +1,373 @@ +--- +title: "Internal Component Authentication" +type: docs +weight: 36 +--- + +**Document Status:** โœ… Complete +**Document Type:** Architecture Reference โ€” Zero Trust Internal Auth +**Related Documents:** [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Credential Provider Model](31-credential-provider-model.md) | [Auth Providers](19-auth-providers.md) | [Session Revocation](35-session-revocation.md) | [Design Priorities](00-design-priorities.md) + +> **This document maps to: DATA + POLICY** +> +> Internal component identities are Data โ€” each component has a UUID, certificate, and service account. Internal auth is Policy โ€” the same five-check boundary model from doc 26 applies at every internal call boundary, with no exceptions for "trusted internal network." This document specifies what was previously only mentioned: how DCM's control plane components authenticate to each other in a distributed deployment. + +--- + +## 1. The Core Principle + +**Network position grants zero trust.** This is stated in doc 26 for external interactions. It applies equally to internal component communication. A call from the Policy Engine to the Placement Engine receives the same boundary checks as a call from an external consumer. The service mesh enforces this at the infrastructure level; DCM enforces it at the application level. + +**Two-layer enforcement:** +1. **Mesh layer (infrastructure):** mTLS mutual authentication (RFC 8446 TLS 1.3), certificate validation (RFC 5280), traffic policies โ€” enforced by the service mesh (Istio or equivalent) +2. **Application layer (DCM):** component identity verification, operation authorization, scoped interaction credentials โ€” enforced by DCM's Ingress and Auth subsystems + +Neither layer alone is sufficient. The mesh layer prevents impersonation at the transport level; the application layer enforces what each component is permitted to do. + +--- + +## 2. Component Identity Model + +Every DCM control plane component has a **component identity** โ€” a stable, verifiable identity used for both mTLS and application-layer authorization. + +```yaml +component_identity: + component_uuid: # stable; assigned at deployment time + component_type: api_gateway | policy_engine | placement_engine | request_orchestrator | + scoring_engine | drift_reconciler | lifecycle_enforcer | notification_router | + audit_store | session_store | message_bus | credential_provider_proxy + component_name: # human-readable; e.g. "policy-engine-eu-west-1" + deployment_uuid: # identifies the DCM deployment instance + + # Certificate identity + mtls_certificate: + subject: "CN=-,O=dcm-internal" + san: [, , ] + issuer_ca: + issued_at: + valid_until: + + # Service account (application layer) + service_account_uuid: # DCM actor of type "component_service_account" + allowed_operations: [] # what this component may call + allowed_targets: [] # which components it may call +``` + +### 2.1 Component Types and Communication Graph + +Not every component may call every other. The allowed communication graph is declared and enforced: + +``` +Consumer/Admin/Provider โ†’ API Gateway +API Gateway โ†’ Request Orchestrator +API Gateway โ†’ Policy Engine (for direct policy evaluation) +API Gateway โ†’ Session Store (token validation) + +Request Orchestrator โ†’ Policy Engine +Request Orchestrator โ†’ Placement Engine +Request Orchestrator โ†’ Scoring Engine +Request Orchestrator โ†’ Audit Store +Request Orchestrator โ†’ Message Bus + +Policy Engine โ†’ Audit Store +Policy Engine โ†’ Message Bus (policy evaluation events) + +Placement Engine โ†’ Audit Store +Placement Engine โ†’ Message Bus + +Scoring Engine โ†’ Audit Store + +Drift Reconciler โ†’ API Gateway (discovery dispatch) +Drift Reconciler โ†’ Audit Store +Drift Reconciler โ†’ Message Bus + +Lifecycle Enforcer โ†’ API Gateway (decommission dispatch) +Lifecycle Enforcer โ†’ Audit Store +Lifecycle Enforcer โ†’ Message Bus + +Notification Router โ†’ Message Bus (subscribe) +Notification Router โ†’ Credential Provider Proxy (notification channel credentials) + +All components โ†’ Session Store (revocation check) +All components โ†’ Credential Provider Proxy (interaction credential requests) +``` + +**ICOM-004:** Components may only call components declared in their `allowed_targets` list. A call from an unexpected source component is rejected with `403 Forbidden` and an audit record. + +--- + +## 3. Certificate Issuance and Internal CA + +### 3.1 Certificate Authority for Internal Components + +Each DCM deployment uses a **registered Certificate Authority (CA)** for issuing component mTLS certificates. This may be: + +**Option A โ€” Built-in Internal CA (default):** DCM operates its own CA per deployment. Simple to configure; no external dependencies; suitable for minimal through standard profiles. + +**Option B โ€” External CA via Credential Provider:** An enterprise CA registered as a Credential Provider (HashiCorp Vault PKI, Venafi TLS Protect, EJBCA, AWS ACM Private CA, Azure Key Vault). The external CA issues component certificates using the standard Credential Provider interface โ€” DCM requests certificates via the provider's API (ACME/EST/SCEP/CMP). See [Credential Provider Model](31-credential-provider-model.md) for registration. Recommended for fsi and sovereign profiles where the enterprise PKI chain must be maintained. + +Both options satisfy ICOM-001 (mTLS required). The distinction is who issues the certificates, not whether mTLS is used. + +**The registered CA's root certificate is installed in all component trust stores at deployment time.** For Option B, the Credential Provider's CA root (which may itself be a subordinate of an enterprise root) is the trust anchor. + +```yaml +internal_ca: + ca_uuid: + deployment_uuid: + ca_type: built_in | external_credential_provider + credential_provider_uuid: # if ca_type: external + external_ca_protocol: acme | est | scep | cmp | null # if external + root_cert_fingerprint: + certificate_lifetime: P90D # profile-governed โ€” see table below + renewal_trigger: P14D + algorithm: ECDSA-P-384 # FIPS-compliant; all profiles + crl_endpoint: + ocsp_endpoint: +``` + +```yaml +internal_ca: + ca_uuid: + deployment_uuid: + root_cert_fingerprint: + certificate_lifetime: P90D # all component certs valid 90 days + renewal_trigger: P14D # renew 14 days before expiry + algorithm: ECDSA-P-384 # FIPS-compliant for all profiles + crl_endpoint: # revocation list for component certs + ocsp_endpoint: # online status check +``` + +### 3.2 Profile-Governed Certificate Configuration + +| Profile | Cert lifetime | Renewal trigger | Bootstrap token TTL | Min key algorithm | +|---------|--------------|-----------------|--------------------|--------------------| +| `minimal` | P180D | P30D | PT4H | RSA-2048 (min) | +| `dev` | P90D | P14D | PT1H | RSA-2048 (min) | +| `standard` | P90D | P14D | PT1H | ECDSA-P-256 (min) | +| `prod` | P90D | P14D | PT1H | ECDSA-P-384 | +| `fsi` | P30D | P7D | PT30M | ECDSA-P-384 | +| `sovereign` | P14D | P3D | PT15M | ECDSA-P-384 (HSM-backed if hardware_attested) | + +> **sovereign profile:** Certificates must be HSM-backed if the deployment posture is `hardware_attested`. The external CA option (Option B) using an HSM-backed Vault PKI backend satisfies this requirement. + +### 3.3 Certificate Lifecycle + +``` +Component starts + โ”‚ + โ–ผ Does component have a valid certificate? + โ”‚ YES โ†’ Use existing certificate + โ”‚ NO (first start or expired) โ†’ Request certificate from Internal CA + โ”‚ + โ–ผ Certificate request to Internal CA: + โ”‚ component_uuid, component_type, deployment_uuid + โ”‚ CSR signed with bootstrap key (see Section 5) + โ”‚ + โ–ผ Internal CA issues certificate + โ”‚ Subject: CN=-,O=dcm-internal + โ”‚ SAN: component_uuid, component_name, internal DNS name + โ”‚ Valid for: P90D (profile-governed) + โ”‚ + โ–ผ Component stores certificate; begins accepting mTLS connections + โ”‚ + โ–ผ 14 days before expiry: auto-renewal + Background thread requests new certificate + Transition: both old and new cert valid for PT1H + Old cert retired after transition +``` + +--- + +## 4. Application-Layer Authorization + +mTLS verifies **who** is calling. Application-layer authorization verifies **what** the caller is permitted to do. + +### 4.1 Interaction Credential for Internal Calls + +Every internal component call follows the same ZTS-002 scoped interaction credential model used for external provider dispatch: + +``` +Component A prepares to call Component B + โ”‚ + โ–ผ Request interaction credential from Credential Provider Proxy: + โ”‚ credential_type: dcm_interaction + โ”‚ issued_to.component_uuid: + โ”‚ scope.operations: [] + โ”‚ scope.target_component: + โ”‚ valid_until: + โ”‚ + โ–ผ Call Component B with: + โ”‚ mTLS certificate (transport identity) + โ”‚ Interaction credential in Authorization header (operation authorization) + โ”‚ Correlation ID (tracing) + โ”‚ + โ–ผ Component B validates: + โ”‚ 1. mTLS cert from Component A's known CA โœ“ + โ”‚ 2. Interaction credential: not revoked, not expired, scoped to this operation โœ“ + โ”‚ 3. Component A is in allowed_sources for this endpoint โœ“ + โ”‚ 4. Operation matches declared scope โœ“ + โ”‚ โ†’ All pass: process request + โ”‚ โ†’ Any fail: 403 + audit record ICOM_AUTH_FAILURE +``` + +### 4.2 Internal Endpoint Authorization + +Each internal component endpoint declares which source components are permitted to call it: + +```yaml +internal_endpoint: + component: policy_engine + endpoint: POST /internal/evaluate + allowed_sources: + - api_gateway + - request_orchestrator + required_scope: policy.evaluate + audit_every_call: true # all internal calls are audited +``` + +**ICOM-003:** Internal endpoints that receive calls from unauthorized source components return 403 and write an `ICOM_UNAUTHORIZED_SOURCE` audit record. This audit record has urgency: high โ€” unexpected internal call patterns are security signals. + +--- + +## 5. Bootstrap โ€” First Certificate + +The bootstrap problem: a new component needs a certificate, but it has no certificate yet to authenticate its request. DCM solves this with a **bootstrap token** mechanism. + +### 5.1 Bootstrap Token + +At deployment time, the platform admin generates a one-time bootstrap token for each component: + +``` +POST /api/v1/admin/components/bootstrap-tokens + +{ + "component_type": "policy_engine", + "component_uuid": "", + "deployment_uuid": "", + "expires_at": "" // short-lived: PT1H maximum +} + +Response 201: +{ + "bootstrap_token": "", // one-time use; stored as env var or secret + "component_uuid": "", + "expires_at": "" +} +``` + +### 5.2 First Certificate Acquisition + +``` +New component starts with bootstrap_token in environment + โ”‚ + โ–ผ POST /internal/ca/issue-certificate + โ”‚ Authorization: Bootstrap + โ”‚ Body: { component_uuid, component_type, deployment_uuid, csr_pem } + โ”‚ + โ–ผ Internal CA validates: + โ”‚ Bootstrap token not expired + โ”‚ Bootstrap token not previously used (one-time) + โ”‚ component_uuid matches token's declared component_uuid + โ”‚ + โ–ผ Certificate issued + โ”‚ Bootstrap token invalidated immediately after use + โ”‚ + โ–ผ Component uses certificate for all subsequent communication + No further need for bootstrap token +``` + +**ICOM-007:** Bootstrap tokens are one-time-use and must expire within PT1H of creation. A bootstrap token that is not used within PT1H is automatically invalidated. Platform admins must generate new tokens if a component fails to start within the window. + +### 5.3 Kubernetes Deployment Integration + +In Kubernetes deployments, bootstrap tokens are injected as Kubernetes Secrets and mounted as environment variables. The component reads the bootstrap token on startup, acquires its certificate, then deletes the Kubernetes Secret. This ensures the bootstrap credential is not persisted beyond initial use. + +```yaml +# Kubernetes Secret (deleted by component after first cert acquisition) +apiVersion: v1 +kind: Secret +metadata: + name: dcm-policy-engine-bootstrap +type: Opaque +stringData: + DCM_BOOTSTRAP_TOKEN: "" + DCM_COMPONENT_UUID: "" + DCM_INTERNAL_CA_ENDPOINT: "https://dcm-internal-ca.dcm-system.svc.cluster.local" +``` + +--- + +## 6. Certificate Compromise Response + +If a component certificate is compromised, the response follows the same emergency pattern as credential compromise: + +``` +Certificate compromise detected + โ”‚ + โ–ผ Compromised cert added to Internal CA CRL + โ”‚ CRL update propagated to all components within SLA: + โ”‚ standard/prod: PT1M + โ”‚ fsi/sovereign: PT15S + โ”‚ + โ–ผ Component identity suspended in DCM + โ”‚ All active interaction credentials for this component โ†’ revoked + โ”‚ ICOM_CERT_COMPROMISED audit record written + โ”‚ + โ–ผ Platform admin notified (urgency: critical) + โ”‚ + โ–ผ New certificate issued for legitimate component instance + โ”‚ Previous certificate remains in CRL permanently + โ”‚ + โ–ผ Component resumes with new certificate +``` + +**ICOM-008:** Compromised internal component certificates are added to the Internal CA CRL immediately. All other components refresh their CRL cache within the profile-governed SLA and reject connections presenting the revoked certificate. + +--- + +## 7. Deployment Architecture Summary + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ DCM Control Plane โ”‚ +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” mTLS+cred โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚API Gatewayโ”‚โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ†’โ”‚Request Orchestratorโ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ mTLS+cred (each call) โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ†“ โ†“ โ†“ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚Policy Engineโ”‚ โ”‚Placement โ”‚ โ”‚Scoring Engineโ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚Engine โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚Credential Providerโ”‚ โ”‚Session Storeโ”‚ โ”‚Internal CA โ”‚ โ”‚ +โ”‚ โ”‚Proxy โ”‚ โ”‚ โ”‚ โ”‚(cert authority) โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ +โ”‚ Service Mesh (Istio): mTLS enforcement at transport layer โ”‚ +โ”‚ All calls: authenticated + authorized + audited โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +--- + +## 8. System Policies + +| Policy | Rule | +|--------|------| +| `ICOM-001` | All internal component-to-component communication must use mTLS with certificates issued by the deployment's Internal CA. Plaintext internal communication is prohibited in all profiles. | +| `ICOM-002` | Every internal call must present a scoped interaction credential (ZTS-002) in addition to the mTLS certificate. The mTLS certificate proves identity; the interaction credential proves authorization for the specific operation. | +| `ICOM-003` | Internal endpoints reject calls from components not in their `allowed_sources` list with 403 and an `ICOM_UNAUTHORIZED_SOURCE` audit record (urgency: high). | +| `ICOM-004` | Components may only call components declared in their `allowed_targets` list. Attempts to call unauthorized components are rejected at the mesh layer (traffic policy) and, if they reach the application layer, at the application layer. | +| `ICOM-005` | All internal component calls are audited: source component, target component, operation, interaction credential UUID, outcome. Internal audit records are written to the same Audit Store as external interactions. | +| `ICOM-006` | Component certificates are issued by the Internal CA with a maximum validity of P90D and renewed automatically P14D before expiry. Component certificates may not be issued by external CAs. | +| `ICOM-007` | Bootstrap tokens are one-time-use and expire within PT1H. A bootstrap token that has been used is immediately invalidated. Unused tokens are invalidated at expiry. | +| `ICOM-008` | Compromised internal component certificates are added to the Internal CA CRL immediately. All components refresh their CRL cache within the profile-governed SLA. | +| `ICOM-009` | The trust anchor for internal component mTLS is a registered root or intermediate CA whose certificate is installed in all component trust stores at deployment time. The trust anchor may be the built-in Internal CA or an external CA registered as a Certificate Provider (e.g. HashiCorp Vault PKI, Venafi, EJBCA) โ€” see [Credential Provider Model](31-credential-provider-model.md) Section on External CAs. Components do not accept certificates from unregistered trust anchors. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md index b8ab663..ac85bc2 100644 --- a/content/docs/architecture/data-model/layering-and-versioning.md +++ b/content/docs/architecture/data-model/layering-and-versioning.md @@ -11,7 +11,7 @@ weight: 3 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/meta-provider-model.md b/content/docs/architecture/data-model/meta-provider-model.md index 75488fa..c74f8ac 100644 --- a/content/docs/architecture/data-model/meta-provider-model.md +++ b/content/docs/architecture/data-model/meta-provider-model.md @@ -10,7 +10,7 @@ weight: 30 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference โ€” Meta Provider Specification **Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Service Dependencies](07-service-dependencies.md) | [Four States](02-four-states.md) | [Operational Models](24-operational-models.md) | [Scoring Model](29-scoring-model.md) | [Control Plane Components](25-control-plane-components.md) diff --git a/content/docs/architecture/data-model/notification-model.md b/content/docs/architecture/data-model/notification-model.md index fdd28bf..89d8bf2 100644 --- a/content/docs/architecture/data-model/notification-model.md +++ b/content/docs/architecture/data-model/notification-model.md @@ -10,7 +10,7 @@ weight: 23 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/operational-models.md b/content/docs/architecture/data-model/operational-models.md index 3c091f5..33250e8 100644 --- a/content/docs/architecture/data-model/operational-models.md +++ b/content/docs/architecture/data-model/operational-models.md @@ -10,7 +10,7 @@ weight: 24 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/ownership-sharing-allocation.md b/content/docs/architecture/data-model/ownership-sharing-allocation.md index ceb75f1..e2534dc 100644 --- a/content/docs/architecture/data-model/ownership-sharing-allocation.md +++ b/content/docs/architecture/data-model/ownership-sharing-allocation.md @@ -10,7 +10,7 @@ weight: 5 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/policy-contract.md b/content/docs/architecture/data-model/policy-contract.md index 9529d5b..a358fbb 100644 --- a/content/docs/architecture/data-model/policy-contract.md +++ b/content/docs/architecture/data-model/policy-contract.md @@ -10,7 +10,7 @@ weight: -8 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Foundation **Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Profiles](14-policy-profiles.md) | [Governance Matrix](27-governance-matrix.md) | [OPA Integration](../specifications/dcm-opa-integration-spec.md) diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md index 8aff0e1..d872844 100644 --- a/content/docs/architecture/data-model/policy-profiles.md +++ b/content/docs/architecture/data-model/policy-profiles.md @@ -12,7 +12,7 @@ weight: 14 > **Universal Group Model:** Policy Groups (`group_class: policy_collection`) and Policy Profiles (`group_class: policy_profile`) are expressions of the [Universal Group Model](15-universal-groups.md). The structures defined in this document remain authoritative for policy-specific behavior; the universal model adds composability, cross-type membership, and the ability to include policy groups within composite groups. -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Scoring Model](29-scoring-model.md) | [Context and Purpose](00-context-and-purpose.md) | [Data Layers and Assembly](03-layering-and-versioning.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/provider-contract.md b/content/docs/architecture/data-model/provider-contract.md index 6219b60..6586f6a 100644 --- a/content/docs/architecture/data-model/provider-contract.md +++ b/content/docs/architecture/data-model/provider-contract.md @@ -10,7 +10,7 @@ weight: -9 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Foundation **Related Documents:** [Foundational Abstractions](00-foundations.md) | [Policy Contract](B-policy-contract.md) | [Registration Specification](../specifications/dcm-registration-spec.md) | [Governance Matrix](27-governance-matrix.md) | [Accreditation](26-accreditation-and-authorization-matrix.md) diff --git a/content/docs/architecture/data-model/registry-governance.md b/content/docs/architecture/data-model/registry-governance.md index 97828c9..ea7c7cb 100644 --- a/content/docs/architecture/data-model/registry-governance.md +++ b/content/docs/architecture/data-model/registry-governance.md @@ -10,7 +10,7 @@ weight: 20 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Auth Providers](19-auth-providers.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/request-dependency-graph.md b/content/docs/architecture/data-model/request-dependency-graph.md new file mode 100644 index 0000000..9077fb8 --- /dev/null +++ b/content/docs/architecture/data-model/request-dependency-graph.md @@ -0,0 +1,309 @@ +--- +title: "Consumer Request Dependency Graph" +type: docs +weight: 38 +--- + +**Document Status:** โœ… Complete +**Document Type:** Architecture Reference โ€” Cross-Request Ordering +**Related Documents:** [Service Dependencies](07-service-dependencies.md) | [Scheduled Requests](37-scheduled-requests.md) | [Meta Provider Composability](30-meta-provider-model.md) | [Operational Models](24-operational-models.md) | [Consumer API Specification](../specifications/consumer-api-spec.md) + +> **Events:** Dependency resolution events (`request.dependencies_resolved`, `dependency.state_changed`) are defined in the [Event Catalog](33-event-catalog.md). + +> **This document maps to: DATA + PROVIDER** +> +> **Distinction from existing dependency models:** +> - Doc 07 (Service Dependencies): *type-level* dependencies โ€” DCM knows that a VM *type* requires an IP type. Resolved automatically during layer assembly. +> - Doc 30 (Meta Provider): *compound service* dependencies โ€” a Meta Provider declares its own constituents and DCM sequences them. Consumer does not manage this. +> - **This document**: *consumer-declared cross-request ordering* โ€” a consumer submitting multiple independent requests says "Request B may not dispatch until Request A is realized." These are requests for different resource types that have no type-level dependency; the consumer is expressing an ordering constraint for their specific deployment. + +--- + +## 1. The Problem + +A consumer deploying a three-tier application submits three requests: a database VM, an application VM, and a load balancer. Without ordering, all three dispatch simultaneously. But the application VM's startup configuration needs the database's IP address, which only exists after the database is realized. + +This is not a type-level dependency (the VM type does not require a VM type). It is a *deployment-time ordering constraint* declared by the consumer for this specific deployment. + +Meta Provider composition handles this when a platform team has pre-defined the compound service. But consumers also need to express ad-hoc ordering for their own deployments without requiring a Meta Provider to exist. + +--- + +## 2. The Request Dependency Graph + +A Request Dependency Group is a consumer-declared set of requests with ordering constraints between them. + +```yaml +request_dependency_group: + group_uuid: + group_handle: "three-tier-app-deploy" # optional, consumer-defined + + requests: + - request_uuid: # database VM + depends_on: [] # no dependencies โ€” dispatches immediately + + - request_uuid: # application VM + depends_on: + - request_uuid: + wait_for: realized # dispatch only after db is REALIZED + inject_fields: # optional: inject realized fields into this request + - from_field: "realized_fields.primary_ip" + to_field: "fields.db_host" + + - request_uuid: # load balancer + depends_on: + - request_uuid: + wait_for: realized + inject_fields: + - from_field: "realized_fields.primary_ip" + to_field: "fields.backend_hosts[0]" + + # Group-level options + on_failure: cancel_remaining | continue # what to do if a request fails + timeout: PT2H # group-level deadline +``` + +### 2.1 wait_for Values + +| Value | Meaning | +|-------|---------| +| `acknowledged` | Dispatch as soon as dependency has an entity_uuid | +| `approved` | Dispatch when dependency has passed approval | +| `dispatched` | Dispatch when dependency has been sent to its provider | +| `realized` | Dispatch only when dependency is fully realized (default, most common) | + +### 2.2 Field Injection + +The `inject_fields` mechanism passes realized output fields from a dependency directly into a dependent request's fields โ€” without the consumer having to poll and re-submit. The injection happens at dispatch time, after the dependency is realized. + +``` +Dependency realized โ†’ Realized State written + โ”‚ + โ–ผ DCM reads inject_fields declarations for dependent requests + โ”‚ For each injection: extract from_field from Realized State + โ”‚ Inject into dependent request's field at to_field path + โ”‚ + โ–ผ Dependent request proceeds to layer assembly with injected fields +``` + +Field injection is subject to the same transformation policies as any other field โ€” if a policy transforms `db_host`, the injection result passes through it. + +--- + +## 3. Submitting a Dependency Group + +### 3.1 Declare and Submit in One Call + +``` +POST /api/v1/request-groups + +{ + "group_handle": "three-tier-app-deploy", + "on_failure": "cancel_remaining", + "timeout": "PT2H", + "requests": [ + { + "ref": "db", # local reference within this submission + "catalog_item_uuid": "", + "fields": { "cpu_count": 8, "memory_gb": 32, "role": "database" } + }, + { + "ref": "app", + "catalog_item_uuid": "", + "fields": { "cpu_count": 4, "memory_gb": 16, "role": "application" }, + "depends_on": [ + { + "ref": "db", + "wait_for": "realized", + "inject_fields": [ + { "from_field": "realized_fields.primary_ip", "to_field": "fields.db_host" } + ] + } + ] + }, + { + "ref": "lb", + "catalog_item_uuid": "", + "fields": { "backend_port": 8080 }, + "depends_on": [ + { "ref": "app", "wait_for": "realized", + "inject_fields": [ + { "from_field": "realized_fields.primary_ip", "to_field": "fields.backend_hosts[0]" } + ] + } + ] + } + ] +} + +Response 202: +{ + "group_uuid": "", + "group_handle": "three-tier-app-deploy", + "requests": [ + { "ref": "db", "request_uuid": "", "entity_uuid": "", "status": "ACKNOWLEDGED" }, + { "ref": "app", "request_uuid": "", "entity_uuid": "", "status": "PENDING_DEPENDENCY" }, + { "ref": "lb", "request_uuid": "", "entity_uuid": "", "status": "PENDING_DEPENDENCY" } + ], + "estimated_completion": "" +} +``` + +### 3.2 Add an Existing Request to a Group + +``` +POST /api/v1/request-groups/{group_uuid}/members + +{ + "request_uuid": "", + "depends_on": [ ... ] +} +``` + +### 3.3 Query Group Status + +``` +GET /api/v1/request-groups/{group_uuid} + +Response 200: +{ + "group_uuid": "", + "group_handle": "three-tier-app-deploy", + "status": "in_progress | completed | failed | cancelled", + "requests": [ + { "request_uuid": "", "ref": "db", "status": "REALIZED" }, + { "request_uuid": "", "ref": "app", "status": "DISPATCHED" }, + { "request_uuid": "", "ref": "lb", "status": "PENDING_DEPENDENCY" } + ], + "created_at": "", + "timeout_at": "" +} +``` + +### 3.4 Cancel a Group + +``` +DELETE /api/v1/request-groups/{group_uuid} + +# Cancels all PENDING_DEPENDENCY and ACKNOWLEDGED requests in the group +# Already-dispatched requests follow standard cancellation model +Response 204 +``` + +--- + +## 4. PENDING_DEPENDENCY Status + +A request in a dependency group that is waiting for its dependency to reach `wait_for` state has status `PENDING_DEPENDENCY`. This is a new status in the Intent State lifecycle: + +``` +ACKNOWLEDGED โ†’ PENDING_DEPENDENCY โ†’ [dependency met] โ†’ LAYERS_ASSEMBLED โ†’ ... โ†’ REALIZED +``` + +`PENDING_DEPENDENCY` requests: +- Are visible in `GET /api/v1/requests` with `status=PENDING_DEPENDENCY` +- Can be cancelled: `DELETE /api/v1/requests/{uuid}` +- Receive the `request.pending_dependency` event (new, info urgency) +- Do not time out independently โ€” the group-level `timeout` governs + +--- + +## 5. Failure Handling + +### 5.1 `on_failure: cancel_remaining` + +When a request in the group fails and `on_failure: cancel_remaining` is set: + +``` +Request fails + โ”‚ + โ–ผ All PENDING_DEPENDENCY and ACKNOWLEDGED requests in group โ†’ CANCELLED + โ”‚ failure_reason: dependency_failed + โ”‚ + โ–ผ request.failed event for the failing request + โ”‚ request.cancelled events for each cancelled dependent + โ”‚ + โ–ผ Group status โ†’ failed +``` + +### 5.2 `on_failure: continue` + +Failed request is marked FAILED; dependents that depended on it are also marked FAILED with `dependency_failed`. Independent requests in the group continue unaffected. + +### 5.3 Group Timeout + +If the group `timeout` duration elapses without all requests reaching a terminal state: + +``` +Group timeout reached + โ”‚ + โ–ผ All non-terminal requests โ†’ FAILED + โ”‚ failure_reason: group_timeout + โ”‚ + โ–ผ request.failed events for each + โ”‚ Group status โ†’ failed +``` + +--- + +## 6. Relationship to Meta Providers + +Request dependency groups and Meta Providers solve overlapping but distinct problems: + +| | Request Dependency Group | Meta Provider | +|--|---|---| +| **Who declares** | Consumer at request time | Platform team at catalog time | +| **Reusable** | No โ€” ad hoc | Yes โ€” catalog item | +| **Type constraints** | None โ€” any resources | Defined by Meta Provider spec | +| **Policy governance** | Standard consumer request policies | Meta Provider policies (MPX-*) | +| **Field injection** | Consumer-declared inject_fields | Meta Provider handles internally | +| **Use case** | Ad-hoc deployment ordering | Standard compound service | + +When a standard compound service exists as a Meta Provider, consumers should use it. Request dependency groups are for deployments that don't fit a predefined compound service pattern. + +--- + +## 7. New Events + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `request.pending_dependency` | info | Request entered PENDING_DEPENDENCY state | +| `request.dependency_met` | info | Dependency reached wait_for state; request proceeding | +| `request.group_completed` | medium | All requests in group reached terminal state | +| `request.group_failed` | high | Group failed or timed out | + +--- + +## 9. Profile-Governed Dependency Group Configuration + +| Profile | Max group size | Max group timeout | Field injection validation | Max nesting depth | +|---------|---------------|-------------------|---------------------------|-------------------| +| `minimal` | 100 | P30D | advisory (warn only) | 3 | +| `dev` | 100 | P7D | advisory | 3 | +| `standard` | 50 | P3D | enforced | 3 | +| `prod` | 25 | P1D | enforced + audited | 3 | +| `fsi` | 10 | PT8H | enforced + audited + policy gated | 2 | +| `sovereign` | 5 | PT4H | enforced + audited + policy gated | 2 | + +**Max group size:** Maximum number of requests in a single dependency group. RDG-002 sets the absolute upper bound at 100; profiles may set lower limits. + +**Max group timeout:** Maximum value of the `timeout` field. Groups declaring a timeout beyond the profile limit are rejected (422). + +**Field injection validation:** `advisory` โ€” warns if injected fields fail schema validation but proceeds; `enforced` โ€” rejects dispatch if injected fields fail validation; `policy gated` โ€” field injection also passes through GateKeeper policy evaluation. + +**Max nesting depth:** Maximum depth of `depends_on` chains. Aโ†’Bโ†’C is depth 2. Exceeding this is rejected at submission (422). + +## 8. System Policies + +| Policy | Rule | +|--------|------| +| `RDG-001` | Circular dependencies within a request group are rejected at submission time (422 Unprocessable Entity). DCM validates the dependency graph is a DAG before acknowledging the group. | +| `RDG-002` | Maximum group size is 50 requests. Groups exceeding this must use Meta Provider composition or be split into multiple groups. | +| `RDG-003` | Field injection (`inject_fields`) is subject to all active Transformation policies. Injected values are not exempt from policy evaluation. | +| `RDG-004` | `PENDING_DEPENDENCY` requests count against the consumer's quota. Resources are reserved at group submission, not at dispatch time. | +| `RDG-005` | Group-level `timeout` is measured from group submission. Individual requests do not have independent timeouts while in PENDING_DEPENDENCY status. | +| `RDG-006` | A request may belong to at most one dependency group. Attempts to add a request to a second group return 409 Conflict. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md index 53da258..c10e4ee 100644 --- a/content/docs/architecture/data-model/resource-grouping.md +++ b/content/docs/architecture/data-model/resource-grouping.md @@ -13,7 +13,7 @@ weight: 8 > **Universal Group Model:** The constructs defined in this document (DCM Tenant, Resource Groups) are superseded by the [Universal Group Model](15-universal-groups.md) for new implementations. Existing constructs map 1:1 to `group_class` values in the universal model โ€” UUIDs, handles, and API references are preserved. This document remains valid as a reference for pre-universal implementations and for understanding the migration path. -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md index 8a629b8..51c7eae 100644 --- a/content/docs/architecture/data-model/resource-service-entities.md +++ b/content/docs/architecture/data-model/resource-service-entities.md @@ -11,7 +11,7 @@ weight: 6 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Operational Models](24-operational-models.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/resource-type-hierarchy.md b/content/docs/architecture/data-model/resource-type-hierarchy.md index b7d3b3f..2f7c731 100644 --- a/content/docs/architecture/data-model/resource-type-hierarchy.md +++ b/content/docs/architecture/data-model/resource-type-hierarchy.md @@ -11,7 +11,7 @@ weight: 5 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/scheduled-requests.md b/content/docs/architecture/data-model/scheduled-requests.md new file mode 100644 index 0000000..6a69ecd --- /dev/null +++ b/content/docs/architecture/data-model/scheduled-requests.md @@ -0,0 +1,295 @@ +--- +title: "Scheduled and Deferred Requests" +type: docs +weight: 37 +--- + +**Document Status:** โœ… Complete +**Document Type:** Architecture Reference โ€” Request Scheduling +**Related Documents:** [Resource and Service Entities](06-resource-service-entities.md) | [Operational Models](24-operational-models.md) | [Request Dependency Graph](38-request-dependency-graph.md) | [Event Catalog](33-event-catalog.md) | [Consumer API Specification](../specifications/consumer-api-spec.md) + +> **This document maps to: DATA + PROVIDER** +> +> A scheduled request is still a request โ€” it goes through the same Intent โ†’ Requested โ†’ Realized pipeline. The only difference is when the pipeline's dispatch step fires. Scheduling is a field on the request, not a separate object type. The Request Orchestrator handles dispatch timing; the Policy Engine evaluates at declaration time (gatekeeping) and again at dispatch time (policy correctness at the moment of execution). + +--- + +## 1. The Scheduling Model + +### 1.1 Core Concept + +Every DCM request has an implicit `schedule: immediate`. Scheduled requests make this explicit: + +```yaml +# Standard immediate request (implicit) +schedule: + dispatch: immediate + +# Deferred โ€” dispatch at a specific time +schedule: + dispatch: at + not_before: "2026-04-01T02:00:00Z" # UTC; dispatch begins at or after this time + not_after: "2026-04-01T04:00:00Z" # optional deadline; cancel if missed + +# Maintenance window โ€” dispatch during the next matching window +schedule: + dispatch: window + window_id: # references a declared Maintenance Window + not_after: "2026-04-30T00:00:00Z" # optional: cancel if no window occurs before this + +# Recurring โ€” for decommission, TTL extension, or rehydration operations +schedule: + dispatch: recurring + cron: "0 2 * * 0" # cron expression (UTC) + max_occurrences: 4 # optional limit + not_after: "2026-12-31T00:00:00Z" # optional end date +``` + +### 1.2 What Can Be Scheduled + +Scheduling applies to any request operation that results in a dispatch to a provider. This includes: + +| Operation | Scheduling supported | Notes | +|-----------|---------------------|-------| +| Resource creation | โœ… | Full scheduling model | +| Resource update (PATCH) | โœ… | Full scheduling model | +| Suspend | โœ… | Full scheduling model | +| Resume | โœ… | Full scheduling model | +| Decommission | โœ… | Full scheduling model; `not_after` recommended | +| Rehydration | โœ… | Full scheduling model | +| TTL extension | โœ… | Full scheduling model | +| Discovery trigger | โŒ | Handled by Discovery Scheduling Model (doc 24 ยง4) | + +--- + +## 2. Request State During Deferral + +A scheduled request moves through the four states with one additional intermediate status: + +``` +Submit request with schedule.dispatch: at + โ”‚ + โ–ผ ACKNOWLEDGED (Intent State created) + โ”‚ entity_uuid assigned + โ”‚ schedule stored in Intent State + โ”‚ + โ–ผ Policy evaluation at declaration time + โ”‚ GateKeeper policies run immediately + โ”‚ If rejected: request fails before entering queue + โ”‚ If approved: request enters scheduled queue + โ”‚ + โ–ผ SCHEDULED (new status within Intent State) + โ”‚ stored in Request Scheduler queue + โ”‚ visible via GET /api/v1/requests/{uuid}/status + โ”‚ cancellable: DELETE /api/v1/requests/{uuid} + โ”‚ + โ–ผ [at not_before time] โ†’ Policy re-evaluation at dispatch + โ”‚ Transformation policies re-run (data may have changed) + โ”‚ GateKeeper re-evaluation with current data + โ”‚ If still approved: proceed to LAYERS_ASSEMBLED โ†’ dispatch + โ”‚ If rejected at dispatch time: FAILED with reason schedule_policy_rejection + โ”‚ + โ–ผ DISPATCHED โ†’ REALIZED (normal pipeline) +``` + +### 2.1 Why Policy Runs Twice + +Policies are evaluated at declaration time to catch obvious rejections early (fail fast). They run again at dispatch time because data may have changed โ€” quota may be exhausted, a compliance policy may have been activated, the actor's role may have changed. The dispatch-time evaluation uses the current policy set, not the one in effect at declaration. + +**SCH-003:** Scheduled requests that fail dispatch-time policy re-evaluation enter FAILED state with `failure_reason: schedule_policy_rejection`. The consumer receives a `request.failed` event with the policy rejection detail. + +--- + +## 3. Maintenance Windows + +A Maintenance Window is a reusable schedule artifact โ€” a named recurrence that scheduled requests can reference. This allows operations teams to declare approved change windows once and have requests automatically slot into them. + +```yaml +maintenance_window: + window_uuid: + window_handle: "weekly-sunday-0200-utc" + description: "Weekly maintenance window โ€” low traffic period" + + # Recurrence + cron: "0 2 * * 0" # every Sunday at 02:00 UTC + duration: PT2H # window is 2 hours long + + # Scope + tenant_uuid: # null = platform-wide window + resource_types: [] # empty = all resource types + + # Approval + status: active | suspended + approved_by: + effective_from: + + # Metadata + created_at: + created_by: +``` + +### 3.1 Maintenance Window API + +``` +# Platform admin operations +POST /api/v1/admin/maintenance-windows +GET /api/v1/admin/maintenance-windows +GET /api/v1/admin/maintenance-windows/{window_uuid} +PATCH /api/v1/admin/maintenance-windows/{window_uuid} +DELETE /api/v1/admin/maintenance-windows/{window_uuid} + +# Consumer operations +GET /api/v1/maintenance-windows # list windows visible to consumer +GET /api/v1/maintenance-windows/{uuid} # describe a specific window +``` + +--- + +## 4. The Request Scheduler Component + +The Request Scheduler is a DCM control plane component responsible for managing the scheduled request queue and triggering dispatch at the appropriate time. + +``` +Request Scheduler responsibilities: + - Maintain a priority queue of SCHEDULED requests ordered by not_before + - Poll queue; dispatch requests when not_before is reached + - Check not_after deadlines; cancel expired requests with reason: schedule_deadline_missed + - Listen for maintenance_window events to trigger window-scheduled requests + - On dispatch: hand off to Request Orchestrator (same path as immediate requests) + - Write SCHEDULED status updates to Intent State + - Publish request.scheduled and request.schedule_cancelled events +``` + +### 4.1 Deadline Enforcement + +If a request has `not_after` set and the deadline passes before dispatch: + +``` +not_after reached without dispatch + โ”‚ + โ–ผ Request status โ†’ FAILED + โ”‚ failure_reason: schedule_deadline_missed + โ”‚ + โ–ผ request.failed event published (urgency: medium) + โ”‚ consumer notified + โ”‚ + โ–ผ Intent State marked terminal โ€” no further retries +``` + +--- + +## 5. Consumer API Additions + +### 5.1 Submit Scheduled Request + +Scheduling is an optional `schedule` field on the existing request submission body: + +``` +POST /api/v1/requests + +{ + "catalog_item_uuid": "", + "fields": { ... }, + "schedule": { + "dispatch": "at", + "not_before": "2026-04-01T02:00:00Z", + "not_after": "2026-04-01T06:00:00Z" + } +} + +Response 202: +{ + "request_uuid": "", + "entity_uuid": "", + "status": "SCHEDULED", + "scheduled_dispatch_at": "2026-04-01T02:00:00Z", + "schedule_deadline": "2026-04-01T06:00:00Z" +} +``` + +### 5.2 List Scheduled Requests + +``` +GET /api/v1/requests?status=SCHEDULED + +Response 200: +{ + "items": [ + { + "request_uuid": "", + "entity_uuid": "", + "status": "SCHEDULED", + "catalog_item_handle": "compute.vm.standard", + "scheduled_dispatch_at": "2026-04-01T02:00:00Z", + "schedule_deadline": "2026-04-01T06:00:00Z", + "created_at": "" + } + ], + "total": 3 +} +``` + +### 5.3 Cancel Scheduled Request + +Cancellation uses the existing endpoint โ€” no new endpoint needed: + +``` +DELETE /api/v1/requests/{request_uuid} + +# Works on SCHEDULED requests; moves status to CANCELLED +# Returns 409 if request is already dispatched (past SCHEDULED) + +Response 204 No Content +``` + +--- + +## 6. New Events + +Two new event types for the Event Catalog (doc 33): + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `request.scheduled` | info | Request entered SCHEDULED queue | +| `request.schedule_cancelled` | low | Scheduled request cancelled before dispatch | +| `request.schedule_deadline_missed` | medium | not_after passed without dispatch | + +These add to the `request.*` domain. Updated domain total: 17 request events. + +--- + +## 8. Profile-Governed Scheduling Configuration + +Scheduling constraints are profile-governed to reflect the operational risk tolerance of each deployment context: + +| Profile | Max scheduling horizon | Max concurrent scheduled/actor | Recurring max frequency | Maintenance window approval tier | +|---------|----------------------|-------------------------------|------------------------|----------------------------------| +| `minimal` | P365D | unlimited | PT1H | auto | +| `dev` | P365D | 50 | PT1H | auto | +| `standard` | P90D | 20 | PT4H | reviewed | +| `prod` | P30D | 10 | PT12H | reviewed | +| `fsi` | P14D | 5 | PT24H | verified | +| `sovereign` | P7D | 3 | PT24H | authorized | + +**Max scheduling horizon:** How far in the future a `not_before` may be set. Requests with `not_before` beyond the profile limit are rejected (422) at submission. + +**Max concurrent scheduled/actor:** How many SCHEDULED (not yet dispatched) requests a single actor may have at one time. Exceeding this limit returns 429. + +**Recurring max frequency:** The minimum interval between recurring dispatches. A cron expression that would dispatch more frequently than this is rejected. + +**Maintenance window approval tier:** The authority tier required to create or modify a Maintenance Window (see ATM-001, doc 32). + +## 7. System Policies + +| Policy | Rule | +|--------|------| +| `SCH-001` | Scheduled requests undergo GateKeeper policy evaluation at declaration time (to catch rejections early) and again at dispatch time (to validate against current state). Both evaluations must pass. | +| `SCH-002` | The `not_before` field must be a future timestamp at submission time. DCM rejects scheduled requests with a past `not_before` (returns 422). | +| `SCH-003` | Requests that fail dispatch-time policy re-evaluation enter FAILED state with `failure_reason: schedule_policy_rejection`. Consumers receive a `request.failed` event with the rejection detail. | +| `SCH-004` | Scheduled requests are cancellable (DELETE /api/v1/requests/{uuid}) at any time before dispatch. Once the Request Orchestrator has accepted the handoff (status moves beyond SCHEDULED), cancellation follows the standard cancellation model. | +| `SCH-005` | If `not_after` is set and passes without dispatch, the request enters FAILED state with `failure_reason: schedule_deadline_missed`. No retry is attempted. | +| `SCH-006` | Maintenance Windows are platform-level or tenant-scoped artifacts requiring platform admin approval. Window schedules are versioned artifacts subject to standard DCM lifecycle. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/scoring-model.md b/content/docs/architecture/data-model/scoring-model.md index fc47190..ce72412 100644 --- a/content/docs/architecture/data-model/scoring-model.md +++ b/content/docs/architecture/data-model/scoring-model.md @@ -10,7 +10,7 @@ weight: 29 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference โ€” Scoring Model Specification **Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Policy Profiles](14-policy-profiles.md) | [Control Plane Components](25-control-plane-components.md) | [Governance Matrix](27-governance-matrix.md) | [Federated Contribution Model](28-federated-contribution-model.md) diff --git a/content/docs/architecture/data-model/service-dependencies.md b/content/docs/architecture/data-model/service-dependencies.md index e038e02..e3d38aa 100644 --- a/content/docs/architecture/data-model/service-dependencies.md +++ b/content/docs/architecture/data-model/service-dependencies.md @@ -11,7 +11,7 @@ weight: 7 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Entity Relationships](09-entity-relationships.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/session-revocation.md b/content/docs/architecture/data-model/session-revocation.md new file mode 100644 index 0000000..e1e78b9 --- /dev/null +++ b/content/docs/architecture/data-model/session-revocation.md @@ -0,0 +1,355 @@ +--- +title: "Session Token Revocation" +type: docs +weight: 35 +--- + +**Document Status:** โœ… Complete +**Document Type:** Architecture Reference โ€” Session Lifecycle and Revocation +**Related Documents:** [Auth Providers](19-auth-providers.md) | [Credential Provider Model](31-credential-provider-model.md) | [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Event Catalog](33-event-catalog.md) | [Design Priorities](00-design-priorities.md) + +> **This document maps to: DATA + POLICY** +> +> A session is a Data artifact with a UUID, lifecycle state, and audit trail. Session revocation is a Policy concern โ€” it fires on triggers defined here and enforced by the Auth Provider and Ingress layer. This document extends the Auth Provider model (doc 19) with the explicit revocation lifecycle that was previously unspecified. +> +> **Relationship to credential revocation:** CPX-006 (doc 31) governs credential revocation โ€” when an actor is deprovisioned, all credentials issued to that actor are revoked. This document governs the complementary concern: active *session tokens* must also be invalidated on the same trigger. Credential revocation and session revocation are parallel processes that both fire on actor deprovisioning. + +--- + +## 1. What a Session Is + +A DCM session represents an authenticated actor's active interaction context. It is created when an actor successfully authenticates through an Auth Provider and is destroyed (or expires) when the session ends. + +```yaml +session_record: + session_uuid: + actor_uuid: + auth_provider_uuid: # which provider issued the session + auth_method: oidc | ldap | api_key | mtls | built_in + mfa_verified: # whether per-session MFA was completed + step_up_verified_at: # last step-up MFA completion + + created_at: + last_active_at: + expires_at: # absolute expiry (from token_ttl) + + refresh_token_uuid: # if refresh_enabled: true + refresh_expires_at: + + status: active | refreshing | revoked | expired + revocation_reason: + revoked_at: + revoked_by: + + # Provenance + client_ip: + user_agent: + tenant_uuid: + + # Concurrent session position + session_sequence: # 1 = oldest active session for this actor +``` + +### 1.1 Session Store + +Active sessions are maintained in a **Session Store** โ€” a fast-queryable, low-latency store separate from the Realized State Store. The Session Store is not GitOps-backed; it is operational state that does not need version history. + +```yaml +session_store: + implementation: redis | postgres | in_memory # profile-governed + ttl_enforcement: hard # sessions expire at expires_at regardless + revocation_index: true # fast lookup by session_uuid for revocation + actor_index: true # fast lookup by actor_uuid for bulk revocation +``` + +**Profile-governed defaults:** + +| Profile | Store | Session TTL | Refresh TTL | Max concurrent | +|---------|-------|-------------|-------------|---------------| +| `minimal` | in_memory or sqlite | PT8H | P7D | unlimited | +| `dev` | redis or postgres | PT4H | P3D | 10 | +| `standard` | redis or postgres | PT1H | P1D | 5 | +| `prod` | redis or postgres | PT30M | PT8H | 3 | +| `fsi` | redis or postgres | PT15M | PT1H | 2 | +| `sovereign` | redis or postgres (HSM-backed) | PT15M | PT30M | 1 | + +--- + +## 2. Revocation Triggers + +Session revocation invalidates a session immediately โ€” regardless of its remaining TTL. The following triggers cause revocation: + +| Trigger | Scope | Who initiates | Behavior | +|---------|-------|--------------|---------| +| `actor_logout` | Single session | Actor (self) | Immediate; that session only | +| `actor_logout_all` | All sessions for actor | Actor (self) | Immediate; all active sessions for this actor | +| `actor_deprovisioned` | All sessions for actor | SCIM / Platform admin | Immediate; fires before deprovisioning acknowledged | +| `actor_suspended` | All sessions for actor | Platform admin | Immediate | +| `security_event` | Specified sessions or all | Platform admin / security automation | Immediate; emergency channel notification | +| `concurrent_limit_exceeded` | Oldest session(s) | System | Oldest session revoked when new session created beyond limit | +| `auth_provider_deregistered` | All sessions from that provider | Platform admin | Immediate; actors must re-authenticate via another provider | +| `credential_compromised` | All sessions for actor | Security automation | Immediate; correlates with CPX emergency rotation | +| `admin_forced_logout` | Specified session(s) | Platform admin | Immediate | + +--- + +## 3. Revocation Lifecycle + +### 3.1 Standard Revocation + +``` +Revocation trigger fires + โ”‚ + โ–ผ Session record status โ†’ revoked + โ”‚ revoked_at, revocation_reason, revoked_by written + โ”‚ + โ–ผ Refresh token invalidated (if exists) + โ”‚ Cannot be exchanged; refresh endpoint returns 401 + โ”‚ + โ–ผ Session UUID added to Session Revocation Registry + โ”‚ (fast-queryable; all DCM components check this on every request) + โ”‚ + โ–ผ Revocation event published to Message Bus + โ”‚ event_type: auth.session_revoked + โ”‚ session_uuid, actor_uuid, revocation_trigger, revoked_at + โ”‚ + โ–ผ Audit record written + session_uuid, actor_uuid, revocation_trigger, revoked_by, revoked_at +``` + +### 3.2 Actor Deprovisioning Revocation (parallel with CPX-006) + +Actor deprovisioning fires both credential revocation (CPX-006) and session revocation simultaneously. Neither blocks the other; both must complete before the deprovisioning is acknowledged. + +``` +Actor deprovisioning initiated + โ”‚ + โ”œโ”€โ”€โ†’ Credential revocation (CPX-006) + โ”‚ All credentials issued to actor_uuid โ†’ revoked + โ”‚ Credential Revocation Registry updated + โ”‚ + โ””โ”€โ”€โ†’ Session revocation (this document) + All active sessions for actor_uuid โ†’ revoked + Session Revocation Registry updated + auth.session_revoked events published per session + โ”‚ + โ–ผ Both complete โ†’ deprovisioning acknowledged + actor_deprovisioned event published + Audit record for deprovisioning written +``` + +### 3.3 Emergency Revocation (Security Event) + +Security events bypass the standard pipeline. Revocation is immediate with no grace period. + +``` +Security event detected + โ”‚ + โ–ผ Target sessions determined + โ”‚ (single session, all sessions for actor, or all sessions from a provider) + โ”‚ + โ–ผ Sessions โ†’ revoked immediately + โ”‚ Session Revocation Registry updated within SLA: + โ”‚ standard/prod: PT30S + โ”‚ fsi: PT10S + โ”‚ sovereign: PT5S + โ”‚ + โ–ผ auth.security_session_revoked event published (critical urgency) + โ”‚ Routed to security team via configured Notification Provider + โ”‚ + โ–ผ Platform admin notified regardless of profile + โ”‚ + โ–ผ All in-flight requests from these sessions โ†’ 401 Unauthorized +``` + +--- + +## 4. Session Revocation Registry + +The Session Revocation Registry is the authoritative list of revoked-but-not-yet-expired session UUIDs. Every DCM component that accepts bearer tokens must check this registry on each request. + +```yaml +session_revocation_registry: + # Session UUID โ†’ revocation record + # Fast in-memory cache with TTL equal to original session TTL + # After the original session TTL would have expired, the entry is + # removed (the session would have been invalid anyway) + + entry: + session_uuid: + revoked_at: + original_expires_at: # entry removed after this time + revocation_trigger: +``` + +**Cache refresh behavior by profile:** + +| Profile | Max cache age | Behavior on cache miss | +|---------|--------------|----------------------| +| `minimal` | PT5M | Check authoritative store; cache result | +| `standard` | PT1M | Check authoritative store; cache result | +| `prod` | PT30S | Check authoritative store; cache result | +| `fsi` | PT10S | Check authoritative store; cache result | +| `sovereign` | PT5S | No cache โ€” always check authoritative store | + +--- + +## 5. Token Introspection + +DCM's Ingress layer exposes a token introspection endpoint for internal components and external systems that need to validate a token without maintaining their own cache: + +``` +POST /api/v1/auth/introspect + +Authorization: Bearer +Content-Type: application/json + +{ + "token": "" +} + +Response 200 (active session): +{ + "active": true, + "session_uuid": "", + "actor_uuid": "", + "expires_at": "", + "mfa_verified": true, + "tenant_uuid": "", + "roles": ["consumer"], + "scopes": ["read", "write"] +} + +Response 200 (revoked or expired): +{ + "active": false, + "reason": "revoked | expired | not_found" +} +``` + +This follows [RFC 7662 (OAuth 2.0 Token Introspection)](https://datatracker.ietf.org/doc/html/rfc7662). + +--- + +## 6. Consumer API โ€” Session Management Endpoints + +### 6.1 Logout (Single Session) + +``` +DELETE /api/v1/auth/session + +Response 204 No Content +``` + +Revokes the session corresponding to the bearer token in the `Authorization` header. No body required. + +### 6.2 Logout All Sessions + +``` +DELETE /api/v1/auth/sessions + +Response 204 No Content +``` + +Revokes all active sessions for the authenticated actor. + +### 6.3 List Active Sessions + +``` +GET /api/v1/auth/sessions + +Response 200: +{ + "items": [ + { + "session_uuid": "", + "created_at": "", + "last_active_at": "", + "expires_at": "", + "auth_method": "oidc", + "client_ip": "", + "current": true // true for the session making this request + } + ], + "total": 2 +} +``` + +### 6.4 Revoke Specific Session + +``` +DELETE /api/v1/auth/sessions/{session_uuid} + +Response 204 No Content +Response 404: session not found or does not belong to this actor +``` + +### 6.5 Admin: Force Revoke Session(s) + +``` +POST /api/v1/admin/actors/{actor_uuid}/revoke-sessions + +{ + "scope": "all | session", + "session_uuid": "", // required if scope: session + "reason": "" // required for audit trail +} + +Response 204 No Content +Response 404: actor not found +``` + +--- + +## 7. Concurrent Session Enforcement + +When `concurrent_sessions: N` is declared and a new session would exceed the limit, the oldest active session is revoked automatically: + +``` +New authentication succeeds + โ”‚ + โ–ผ Count active sessions for actor_uuid + โ”‚ If count >= concurrent_sessions limit: + โ”‚ Revoke oldest session (by created_at) + โ”‚ Trigger: concurrent_limit_exceeded + โ”‚ + โ–ผ New session created +``` + +The evicted actor receives an `auth.session_revoked` notification if a Notification Provider is configured with the actor's notification preferences. The event does not block the new session creation. + +--- + +## 8. Relationship to the Credential Revocation Model + +Session revocation (this document) and credential revocation (doc 31, CPX-001โ€“CPX-012) are parallel but distinct: + +| | Session Revocation | Credential Revocation | +|--|---|---| +| **What** | Bearer token / session cookie validity | API key, x509, SSH key, service account token | +| **Store** | Session Revocation Registry | Credential Revocation Registry | +| **Propagation** | Auth layer cache refresh | Message Bus โ†’ all components | +| **Actor deprovision** | All sessions revoked | All credentials revoked | +| **TTL** | Session TTL (minutes to hours) | Credential TTL (hours to years) | +| **Emergency SLA** | PT5Sโ€“PT30S | PT30Sโ€“PT5M | +| **Event** | `auth.session_revoked` | `credential.revoked` | + +**AUTH-016:** On actor deprovisioning, session revocation and credential revocation are parallel operations. The deprovisioning is not acknowledged until both are confirmed complete. + +--- + +## 9. System Policies + +| Policy | Rule | +|--------|------| +| `AUTH-016` | On actor deprovisioning, session revocation and credential revocation (CPX-006) are parallel operations. Deprovisioning is not acknowledged until both complete. | +| `AUTH-017` | Session revocation must propagate to the Session Revocation Registry within the profile-governed SLA: minimal PT5M, standard PT1M, prod PT30S, fsi PT10S, sovereign PT5S. | +| `AUTH-018` | All DCM components that accept bearer tokens must check the Session Revocation Registry on each request. Cache age must not exceed the profile-governed maximum (sovereign: no cache). | +| `AUTH-019` | Emergency session revocation (security_event trigger) fires immediately with no grace period. The `auth.security_session_revoked` event has `urgency: critical` and is non-suppressable. | +| `AUTH-020` | The token introspection endpoint (`POST /api/v1/auth/introspect`) must be authenticated. Access requires an actor or service account with the `introspection` scope. | +| `AUTH-021` | When concurrent session limits are enforced, the oldest session is revoked before the new session is created. The evicted actor is notified via Notification Provider if configured. | +| `AUTH-022` | Refresh tokens are invalidated when their parent session is revoked. A revoked refresh token returns 401 on exchange; it cannot be used to create a new session. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/standards-catalog.md b/content/docs/architecture/data-model/standards-catalog.md new file mode 100644 index 0000000..bfd6c87 --- /dev/null +++ b/content/docs/architecture/data-model/standards-catalog.md @@ -0,0 +1,281 @@ +--- +title: "Standards and Compliance Catalog" +type: docs +weight: 40 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” Normative Standards +**Purpose:** Single authoritative source for all RFCs, protocols, specifications, and compliance frameworks referenced by the DCM architecture. For each standard: what it is, where DCM uses it, and what obligation it places on implementations. + +> **How to read this document:** +> - **Normative** โ€” DCM implementations MUST comply with this standard in the specified context +> - **Informative** โ€” DCM draws on this standard as guidance or reference without strict compliance +> - **Optional** โ€” DCM supports this standard in applicable profiles or configurations + +--- + +## 1. Internet Standards (IETF RFCs) + +### 1.1 Authentication and Authorization + +| RFC | Title | Use in DCM | Obligation | +|-----|-------|-----------|-----------| +| **RFC 7519** | JSON Web Token (JWT) | Bearer token format for session tokens and API key tokens; claims carry actor_uuid, roles, tenant_uuid, exp | Normative | +| **RFC 7517** | JSON Web Key (JWK) | Public key format for Auth Provider OIDC verification keys; JWKS endpoint for key discovery | Normative | +| **RFC 7662** | OAuth 2.0 Token Introspection | `POST /api/v1/auth/introspect` โ€” validates bearer tokens; response format `{active, session_uuid, actor_uuid, exp, roles}` | Normative | +| **RFC 6749** | OAuth 2.0 Authorization Framework | Authorization flow for OIDC Auth Providers; client credentials flow for service account API keys | Informative | +| **RFC 4511** | Lightweight Directory Access Protocol (LDAP) | LDAP/FreeIPA/Active Directory Auth Provider integration; bind operations, search filters for group membership | Normative | +| **RFC 7643** | SCIM 2.0 Core Schema | Actor and group provisioning schema for enterprise IdP integration; SCIM deprovision triggers session + credential revocation | Normative | +| **RFC 7644** | SCIM 2.0 Protocol | SCIM REST API for actor provisioning; DELETE triggers AUTH-016 (session) and CPX-006 (credential) revocation in parallel | Normative | + +### 1.2 Transport Security + +| RFC | Title | Use in DCM | Obligation | +|-----|-------|-----------|-----------| +| **RFC 8446** | TLS 1.3 | All external API communication; preferred TLS version; mandatory cipher suite compliance | Normative | +| **RFC 5246** | TLS 1.2 | Permitted TLS version for compatibility; minimum acceptable version; TLS 1.0/1.1 prohibited | Normative | +| **RFC 5280** | X.509 PKI Certificate and CRL Profile | All DCM certificates (component mTLS, Internal CA, Credential Provider certs); CRL format for revocation; certificate chain validation | Normative | +| **RFC 6960** | Online Certificate Status Protocol (OCSP) | Internal CA OCSP endpoint for real-time certificate status; Internal CA CRL supplement | Normative | + +### 1.3 Certificate Enrollment + +| RFC | Title | Use in DCM | Obligation | +|-----|-------|-----------|-----------| +| **RFC 7030** | Enrollment over Secure Transport (EST) | Certificate enrollment for Internal CA (alternative to bootstrap token); preferred for automated cert lifecycle | Informative | +| **RFC 8555** | ACME โ€” Automatic Certificate Management Environment | Automated certificate lifecycle for external-facing TLS certificates; provider certificates | Informative | +| **RFC 8894** | Simple Certificate Enrolment Protocol (SCEP) | Legacy certificate enrollment for environments without EST/ACME support | Optional | +| **RFC 4210** | Certificate Management Protocol (CMP) | X.509 PKI certificate management in enterprise PKI environments | Optional | + +### 1.4 API Lifecycle + +| RFC | Title | Use in DCM | Obligation | +|-----|-------|-----------|-----------| +| **RFC 8594** | The Sunset HTTP Header Field | Deprecated API version responses include `Sunset: ` header (VER-003); also `Deprecation` header | Normative | +| **RFC 9745** | The Deprecation HTTP Header Field | Deprecated API version responses include `Deprecation: ` header paired with RFC 8594 Sunset | Normative | + +### 1.5 Service Discovery and Health + +| RFC | Title | Use in DCM | Obligation | +|-----|-------|-----------|-----------| +| **RFC 8615** | Well-Known Uniform Resource Identifiers | `GET /.well-known/dcm-api-versions` (version discovery); `/livez` and `/readyz` path conventions; IANA health+json media type | Normative | + +### 1.6 Data Formats + +| RFC | Title | Use in DCM | Obligation | +|-----|-------|-----------|-----------| +| **ISO 8601** | Date and Time Format | All timestamps in DCM: `created_at`, `expires_at`, `not_before`, `not_after`, event timestamps; durations as ISO 8601 periods (P90D, PT8H) | Normative | +| **RFC 8259** | The JavaScript Object Notation (JSON) Data Interchange Format | All DCM API request/response bodies; all entity definitions in stores | Normative | + +--- + +## 2. Identity and Access Protocols + +| Protocol | Specification | Use in DCM | Obligation | +|----------|--------------|-----------|-----------| +| **OIDC / OpenID Connect** | OpenID Foundation Core 1.0 | Primary enterprise Auth Provider type; ID token format; JWKS endpoint for key verification; userinfo endpoint for actor enrichment | Normative | +| **SAML 2.0** | OASIS SAML 2.0 | Auth Provider type for organizations without OIDC; assertion format for role mapping | Optional | +| **mTLS** | RFC 8446 + RFC 5280 | All internal component-to-component communication (ICOM-001); provider-to-DCM authentication in ZTS model | Normative | +| **LDAP v3** | RFC 4511 | FreeIPA, Active Directory, OpenLDAP Auth Provider types; group membership queries for RBAC | Normative | +| **SCIM 2.0** | RFC 7643 + RFC 7644 | Optional enterprise provisioning; actor creation, update, deprovision; deprovision triggers parallel session + credential revocation | Optional | + +--- + +## 3. Cryptographic Standards + +| Standard | Use in DCM | Profiles | Obligation | +|----------|-----------|---------|-----------| +| **ECDSA P-384** | Internal CA certificates; component mTLS certs; preferred curve for all DCM-issued certificates | All profiles | Normative for Internal CA | +| **ECDSA P-256** | Permitted for performance-constrained contexts where P-384 is not available | minimal, dev | Optional | +| **RSA โ‰ฅ 2048** | Permitted for compatibility with legacy systems; RSA < 2048 prohibited | All profiles | Conditional | +| **AES-256-GCM** | Credential encryption at rest; audit record encryption (sovereign); data classification-driven | standard+ | Normative | +| **AES-128-GCM** | Permitted for minimal/dev profiles where performance matters | minimal, dev | Conditional | +| **SHA-256** | Hash function for audit chain integrity; entity handle generation; minimum acceptable | All profiles | Normative | +| **SHA-384 / SHA-512** | Preferred hash function for fsi/sovereign profiles | fsi, sovereign | Normative for fsi+ | +| **FIPS 140-2 Level 1** | Minimum cryptographic module requirement for standard/prod | standard, prod | Normative | +| **FIPS 140-2 Level 2** | Cryptographic module requirement for regulated environments | fsi, fedramp_moderate | Normative | +| **FIPS 140-3 Level 3** | Cryptographic module requirement for sovereign deployments | sovereign, dod_il4 | Normative | +| **TLS 1.3** | Preferred; mandatory cipher suites; forward secrecy required | All profiles | Normative (preferred) | +| **TLS 1.2** | Minimum acceptable; TLS 1.0/1.1 strictly prohibited | All profiles | Normative (minimum) | + +### 3.1 Forbidden Algorithms + +DCM prohibits the following algorithms in all profiles: + +| Algorithm | Reason | +|-----------|--------| +| MD5 | Cryptographically broken | +| SHA-1 | Deprecated; collision attacks demonstrated | +| DES | 56-bit key; insecure | +| 3DES / Triple-DES | Deprecated; Sweet32 attack | +| RC4 | Cryptographically broken | +| RSA < 2048 | Insufficient key length | +| ECDSA curves weaker than P-256 | Insufficient security level | + +--- + +## 4. Operational Standards and Protocols + +| Standard | Specification | Use in DCM | Obligation | +|----------|--------------|-----------|-----------| +| **Prometheus / OpenMetrics** | Prometheus exposition format; OpenMetrics spec | `GET /metrics` scrape endpoint; all DCM metric families (HLT-005); provider health metrics | Normative | +| **OpenTelemetry (OTel)** | CNCF OpenTelemetry specification | Distributed tracing for request pipeline; X-DCM-Correlation-ID propagation; span context for audit provenance | Informative | +| **Kubernetes API** | kubernetes.io API conventions | Resource type spec format mirrors k8s YAML; CRD-based DCM Operator integration; probe endpoints (/livez, /readyz) | Normative (k8s deployments) | +| **GitOps / OpenGitOps** | OpenGitOps principles (v1.0) | All DCM data model artifacts stored in Git; PR-based contribution model; Git as source of truth for policy and layer definitions | Normative | +| **Unix cron** | POSIX cron expression format | Recurring schedule expressions in scheduled requests (doc 37) and maintenance window definitions | Normative | +| **IANA health+json** | RFC 8615 + IANA media type registry | Health response format for `/livez`, `/readyz`, `/api/v1/admin/health`, and OIS health endpoint | Normative | +| **W3C Server-Sent Events (SSE)** | W3C Living Standard | `GET /api/v1/requests/{uuid}/stream` live request status stream; events: status_change, progress_updated, approval_required, approval_recorded, heartbeat; stream closes on terminal status; alternative to polling for browser/CLI consumers | Normative | +| **OpenAPI 3.1** | OpenAPI Initiative 3.1 | REST API specification format for Consumer API, Admin API, and Operator Interface Specification; schema definitions for request/response bodies | Normative | +| **SPIFFE** | CNCF SPIFFE Specification v1.0 | Workload identity framework that inspired DCM's internal component identity model (ICOM); each DCM component has a stable UUID and certificate analogous to a SPIFFE ID; Istio/Envoy enforce SPIFFE-compatible workload identity | Informative | +| **Istio / Service Mesh** | Istio service mesh specification | Internal component mTLS enforcement; traffic policies; circuit breaking; observability; service-to-service authorization | Normative (distributed deployments) | + +| **HashiCorp Vault PKI** | HashiCorp Vault PKI Secrets Engine | External CA Credential Provider backend (optional); issues x509 component certificates via native API or EST/ACME; recommended for fsi/sovereign profiles where enterprise PKI chain is required; typically operates as a subordinate CA of the organization root | Optional | +| **Venafi TLS Protect** | Venafi Platform | External CA Credential Provider backend (optional); enterprise certificate lifecycle management; ACME/EST/REST API integration | Optional | +| **EJBCA** | Enterprise JavaBeans Certificate Authority | External CA Credential Provider backend (optional); ACME/CMP/SCEP integration | Optional | + +--- + +## 5. Compliance Frameworks + +These frameworks drive specific DCM profiles, overlays, and policy constraints. DCM does not certify compliance โ€” it provides the architectural primitives that enable compliant implementations. + +### 5.1 US Federal and Defense + +| Framework | Full Name | DCM Profile/Overlay | Key DCM Requirements | +|-----------|-----------|-------------------|---------------------| +| **NIST SP 800-53** | Security and Privacy Controls for Information Systems | `fedramp_moderate`, `fedramp_high` | Policy control families mapped to DCM policy domains; access control, audit, configuration management | +| **NIST SP 800-63B** | Digital Identity Guidelines | All profiles (AAL mapping) | AAL1 (minimal/dev), AAL2 (standard/prod), AAL2+ (fsi), AAL3 (sovereign); MFA requirements per level | +| **FedRAMP Moderate** | Federal Risk and Authorization Management Program โ€” Moderate | `fedramp_moderate` overlay | NIST 800-53 Moderate baseline; FIPS 140-2 Level 1+; Federal data handling requirements | +| **FedRAMP High** | Federal Risk and Authorization Management Program โ€” High | `fedramp_high` overlay | NIST 800-53 High baseline; FIPS 140-2 Level 2+; enhanced audit retention | +| **DoD IL4** | Department of Defense Impact Level 4 | `dod_il4` overlay | Controlled Unclassified Information; FIPS 140-2 Level 2; hardware attestation; enhanced logging | +| **FIPS 140-2/140-3** | Federal Information Processing Standard โ€” Cryptographic Modules | fsi+ profiles | Cryptographic module validation; forbidden algorithm enforcement; key management requirements | + +### 5.2 Industry Compliance + +| Framework | Full Name | DCM Profile/Overlay | Key DCM Requirements | +|-----------|-----------|-------------------|---------------------| +| **PCI DSS** | Payment Card Industry Data Security Standard | `pci_dss` overlay | Req 8.3.9: P90D maximum credential rotation; network segmentation via sovereignty constraints; cardholder data access logging; 12-month audit retention | +| **HIPAA** | Health Insurance Portability and Accountability Act | `fsi` profile; `hipaa` overlay | PHI access logging; minimum necessary access (RBAC); audit controls; transmission security (TLS 1.2+); workforce authentication (MFA) | +| **SOC 2** | Service Organization Control 2 | `standard`+ profiles | Type II audit trail requirements; availability, security, confidentiality trust service criteria; change management via GitOps | +| **ISO 27001** | Information Security Management Systems | All profiles | Risk-based approach; asset management; access control; cryptography; operations security; incident management | + +### 5.3 Data Protection / Sovereignty + +| Framework | Full Name | DCM Feature | Key DCM Requirements | +|-----------|-----------|------------|---------------------| +| **GDPR** | General Data Protection Regulation (EU) | Sovereignty constraints; data classification | Data residency enforcement; right to erasure model (entity decommission + audit retention policy); data minimization via field-level classification; consent/purpose tracking via Governance Matrix | +| **Schrems II** | CJEU ruling on EU-US data transfers | Sovereignty constraints; federation boundaries | Data transfer restrictions between DCM federation peers; sovereign profile enforcement | + +--- + +## 6. CNCF Ecosystem + +DCM is designed for CNCF ecosystem compatibility. The following CNCF projects are referenced: + +| Project | CNCF Status | DCM Use | +|---------|------------|---------| +| **Kubernetes** | Graduated | Deployment target; CRD-based DCM Operator; resource model inspiration | +| **Open Policy Agent (OPA)** | Graduated | Policy engine backend option; Rego policies for DCM GateKeeper and Validation policy types | +| **Prometheus** | Graduated | Metrics exposition format; DCM scrape endpoint | +| **OpenTelemetry** | Graduated | Distributed tracing; correlation ID propagation | +| **Istio** | Graduated | Service mesh for internal mTLS; traffic policies | +| **Argo CD / Flux** | Graduated | GitOps delivery for DCM layer definitions and policy artifacts | + +--- + +## 7. Authentication Assurance Levels (NIST SP 800-63B) + +DCM maps profile security postures to NIST Authentication Assurance Levels: + +| Profile | AAL | Requirements | +|---------|-----|-------------| +| `minimal` | AAL1 | Single-factor authentication acceptable; password or API key | +| `dev` | AAL1 | Single-factor authentication acceptable | +| `standard` | AAL2 | MFA required for all actor sessions; phishing-resistant preferred | +| `prod` | AAL2 | MFA required; TOTP, FIDO2, or hardware token | +| `fsi` | AAL2+ | MFA required; phishing-resistant authenticator (FIDO2/hardware token) | +| `sovereign` | AAL3 | Hardware-based authenticator required; verifier impersonation resistance; physical authenticator possession | + +--- + +## 8. Standard Usage Map โ€” Where Each Standard Appears + +| Standard | Documents | +|----------|----------| +| RFC 7519 (JWT) | 19-auth-providers, 35-session-revocation, consumer-api-spec | +| RFC 7517 (JWK) | 19-auth-providers | +| RFC 7662 (Token Introspection) | 35-session-revocation, consumer-api-spec | +| RFC 7643/7644 (SCIM 2.0) | 19-auth-providers | +| RFC 8446 (TLS 1.3) | 14-policy-profiles, 26-accreditation, 31-credential-provider, 36-internal-component-auth | +| RFC 5280 (X.509/CRL) | 31-credential-provider, 36-internal-component-auth | +| RFC 6960 (OCSP) | 36-internal-component-auth | +| RFC 7030 (EST) | 31-credential-provider | +| RFC 8555 (ACME) | 31-credential-provider | +| RFC 8894 (SCEP) | 31-credential-provider | +| RFC 4210 (CMP) | 31-credential-provider | +| RFC 8594 (Sunset) | 34-api-versioning-strategy, consumer-api-spec | +| RFC 9745 (Deprecation) | 34-api-versioning-strategy, consumer-api-spec | +| RFC 8615 (Well-Known URIs) | 34-api-versioning-strategy, 39-dcm-self-health, dcm-operator-interface-spec | +| RFC 8259 (JSON) | All specifications | +| ISO 8601 (timestamps) | All documents | +| OIDC / OpenID Connect | 19-auth-providers, consumer-api-spec | +| LDAP (RFC 4511) | 19-auth-providers | +| FIPS 140 | 14-policy-profiles, 31-credential-provider, 36-internal-component-auth | +| NIST SP 800-63B (AAL) | 31-credential-provider | +| NIST SP 800-53 | 14-policy-profiles | +| HIPAA | 14-policy-profiles, 31-credential-provider | +| PCI DSS | 14-policy-profiles, 31-credential-provider | +| FedRAMP | 14-policy-profiles | +| GDPR | 08-resource-grouping, 22-dcm-federation | +| ISO 27001 | 14-policy-profiles, 26-accreditation | +| Kubernetes | 11-kubernetes-compatibility, dcm-operator-sdk-api | +| OPA | dcm-opa-integration-spec | +| Prometheus | 12-audit-provenance, 39-dcm-self-health | +| OpenTelemetry | 12-audit-provenance | +| Istio | 17-deployment-redundancy, 36-internal-component-auth | +| GitOps/OpenGitOps | 00-context-and-purpose, 20-registry-governance, 28-federated-contribution | +| W3C SSE (Server-Sent Events) | consumer-api-spec, 39-dcm-self-health | +| OpenAPI 3.1 | dcm-operator-interface-spec, consumer-api-spec, dcm-admin-api-spec | +| SPIFFE (conceptual) | 36-internal-component-auth | +| HashiCorp Vault PKI | 31-credential-provider-model, 36-internal-component-auth | +| RFC 7009 (Token Revocation) | 35-session-revocation | +| NIST SP 800-63B (AAL) | 31-credential-provider-model, 14-policy-profiles | +| SCH policies (scheduling) | 37-scheduled-requests | +| RDG policies (dependency graph) | 38-request-dependency-graph | +| HLT policies (self-health) | 39-dcm-self-health | +| SES policies (session revocation) | 35-session-revocation | +| ICOM policies (internal component auth) | 36-internal-component-auth | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + +--- + +## 9. Policy Family to Standard Mapping + +Each DCM system policy family maps to one or more industry standards. This table supports compliance traceability. + +| Policy Family | Standards Basis | Key Policies | +|--------------|----------------|-------------| +| **AUTH-001โ€“015** | RFC 6749, RFC 7519, OIDC Core, NIST SP 800-63B, RFC 7643/7644 | Auth Provider lifecycle; session TTL; MFA enforcement; SCIM provisioning | +| **AUTH-016โ€“022** | RFC 7662, RFC 6749 spirit, OAuth 2.0 best practices | Session revocation; token introspection; refresh token invalidation | +| **CPX-001โ€“012** | FIPS 140-2/3, RFC 5280, RFC 8555/7030/8894/4210, NIST SP 800-57 | Credential never stored; rotation; revocation; algorithm baseline | +| **ATM-001โ€“012** | ISO 27001 change management; organizational governance practices | Authority tier ordering; security degradation gate; profile gap detection | +| **EVT-001โ€“007** | OpenTelemetry, CNCF event-driven best practices | Event envelope; idempotency; non-suppressable audit events | +| **VER-001โ€“009** | RFC 8594, RFC 9745, industry API lifecycle practices | Breaking change definition; deprecation headers; migration guides | +| **SES-001โ€“005** | RFC 7662, RFC 7009, OAuth 2.0 security best practices | Session lifecycle; concurrent limits; emergency revocation | +| **ICOM-001โ€“009** | RFC 8446, RFC 5280, SPIFFE conceptual model, FIPS 140 | mTLS; component identity; Internal CA; bootstrap; certificate revocation | +| **SCH-001โ€“006** | Industry job scheduling practices; dual-evaluation pattern | Scheduled request dual policy evaluation; deadline enforcement | +| **RDG-001โ€“006** | DAG-based workflow ordering; dependency injection patterns | Circular dependency rejection; quota at group submission; field injection | +| **HLT-001โ€“006** | RFC 8615, Kubernetes probe conventions, Prometheus OpenMetrics | Liveness/readiness; unauthenticated probes; profile-governed metrics exposure | +| **DPO-001โ€“006** | Design-by-contract; security-first architecture principles | Design priority order; security as Priority 1 | +| **ZTS-001โ€“005** | Zero Trust Architecture (NIST SP 800-207); NIST SP 800-63B | Five-check boundary model; mTLS; scoped interaction credentials | +| **MPX-001โ€“008** | Service mesh composition patterns; dependency graph execution | Meta Provider constituent orchestration; compensation | +| **SMX-001โ€“010** | Risk scoring; NIST RMF; organizational risk tolerance | Hybrid scoring; approval routing; enforcement class | +| **FCM-001โ€“008** | GitOps contribution model; CNCF governance practices | Federated policy contribution; shadow validation; trust levels | +| **GMX-001โ€“006** | Governance Matrix; policy-as-code; organizational controls | Cross-domain policy enforcement; data classification | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md index a71b5ba..ccd5ed0 100644 --- a/content/docs/architecture/data-model/storage-providers.md +++ b/content/docs/architecture/data-model/storage-providers.md @@ -11,7 +11,7 @@ weight: 11 > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Four States](02-four-states.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Information Providers](10-information-providers.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/universal-audit.md b/content/docs/architecture/data-model/universal-audit.md index bb15ce6..3c21575 100644 --- a/content/docs/architecture/data-model/universal-audit.md +++ b/content/docs/architecture/data-model/universal-audit.md @@ -10,7 +10,7 @@ weight: 16 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Storage Providers](11-storage-providers.md) | [Universal Groups](15-universal-groups.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md index 2715e3f..9b52cde 100644 --- a/content/docs/architecture/data-model/universal-groups.md +++ b/content/docs/architecture/data-model/universal-groups.md @@ -10,7 +10,7 @@ weight: 15 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Grouping](08-resource-grouping.md) | [Entity Relationships](09-entity-relationships.md) | [Policy Organization](14-policy-profiles.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md index d25c478..72d309c 100644 --- a/content/docs/architecture/data-model/webhooks-messaging.md +++ b/content/docs/architecture/data-model/webhooks-messaging.md @@ -10,7 +10,7 @@ weight: 18 > > Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Related Documents (updated):** [Notification Model](23-notification-model.md) | [Entity Relationships](09-entity-relationships.md) > **Foundation Document Reference** diff --git a/content/docs/architecture/overview.md b/content/docs/architecture/overview.md index d1612d7..f6c084e 100644 --- a/content/docs/architecture/overview.md +++ b/content/docs/architecture/overview.md @@ -39,6 +39,14 @@ Every design decision in DCM is evaluated against this hierarchy. When prioritie **The `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security."** All security properties are architecturally present in every profile. What varies is automation level, enforcement thresholds, and acceptable manual intervention. --- +## Session Token Revocation and Internal Component Authentication + +**Session token revocation** defines the complete lifecycle for actor sessions โ€” how tokens are created, refreshed, and revoked. Revocation is profile-governed: `minimal` allows PT5M propagation SLA; `sovereign` requires PT5S. The Session Revocation Registry is checked on every authenticated request โ€” no component may skip this check (AUTH-018). Emergency revocation (security event) fires a `critical` urgency event that is non-suppressable. See [Session Token Revocation](data-model/session-revocation/). + +**Internal component authentication** closes the zero trust model at the internal boundary. Every call between DCM control plane components โ€” API Gateway to Request Orchestrator, Policy Engine to Storage Provider, etc. โ€” requires both a mTLS certificate from the Internal CA and a scoped interaction credential (ZTS-002). Components may only call targets declared in their `allowed_targets` list (ICOM-004). Bootstrap tokens are one-time-use and expire within PT1H (ICOM-007). See [Internal Component Authentication](data-model/internal-component-auth/). + +--- + ## Authority Tier Model DCM governs decisions through an extensible **authority tier model** โ€” a named, ordered list where each tier expresses a required level of organizational decision gravity. The default tiers are `auto โ†’ reviewed โ†’ verified โ†’ authorized`, but organizations can insert custom tiers between existing ones. Tier weight is derived from list position at evaluation time; existing tier name references always resolve correctly. diff --git a/content/docs/architecture/specifications/admin-api-spec.md b/content/docs/architecture/specifications/admin-api-spec.md index 8cae9fc..8344b6d 100644 --- a/content/docs/architecture/specifications/admin-api-spec.md +++ b/content/docs/architecture/specifications/admin-api-spec.md @@ -491,6 +491,113 @@ Response 200: --- +## 13. DCM Self-Health Endpoints + +DCM exposes three health endpoints, each with a distinct purpose: + +```http +# Liveness โ€” is the process alive? (Kubernetes liveness probe) +GET /livez +# No auth required. Max response time: PT5S. +# Returns 200 OK with {"status":"ok"} if alive. +# Returns 503 if process is deadlocked or unresponsive. + +# Readiness โ€” is DCM ready to serve traffic? (Kubernetes readiness probe) +GET /readyz +# No auth required. Max response time: PT10S. +# Returns 200 OK with {"status":"ready"} if all required stores are reachable. +# Returns 503 with {"status":"not_ready","reasons":["store_unreachable"]} otherwise. + +# Operational health โ€” rich health for operators and monitoring systems +GET /api/v1/admin/health +Authorization: Bearer + +Response 200: +{ + "dcm_version": "", + "profile": "prod", + "status": "healthy", // healthy | degraded | critical + "components": { + "request_orchestrator": { "status": "healthy" }, + "policy_engine": { "status": "healthy" }, + "placement_engine": { "status": "healthy" }, + "credential_provider": { "status": "degraded", "reason": "rotation_pending" } + }, + "stores": { + "intent_store": { "status": "healthy", "latency_p99_ms": 12 }, + "requested_store": { "status": "healthy", "latency_p99_ms": 8 }, + "realized_store": { "status": "healthy", "latency_p99_ms": 9 } + }, + "providers": { + "total": 4, + "healthy": 3, + "degraded": 1, + "unhealthy": 0 + } +} + +# Prometheus metrics +GET /metrics +# Unauthenticated (secured by network policy in production). +# Returns Prometheus text format metrics. +``` + +> **Full model:** See [DCM Self-Health](../data-model/39-dcm-self-health.md) โ€” HLT-001โ€“HLT-006. + + +## 12. Session Management (Admin) + +Platform admins can force-revoke sessions for any actor โ€” used on actor compromise, policy violation, or deprovisioning. + +```http +# Force-revoke all sessions for an actor +POST /api/v1/admin/actors/{actor_uuid}/revoke-sessions +Authorization: Bearer + +{ + "reason": "security_event", // REQUIRED + "notify_actor": true // send notification event +} + +Response 202 Accepted: +{ + "sessions_revoked": 3, + "actor_uuid": "", + "revocation_propagated_at": "" +} +``` + +```http +# List active sessions for any actor (admin view) +GET /api/v1/admin/actors/{actor_uuid}/sessions +Authorization: Bearer + +Response 200: +{ + "items": [ + { + "session_uuid": "", + "created_at": "", + "expires_at": "", + "auth_method": "ldap", + "mfa_verified": true, + "status": "active" + } + ], + "total": 1 +} +``` + +**Error codes specific to session management:** + +| Error Code | HTTP | When | +|-----------|------|------| +| `actor_not_found` | 404 | Actor UUID not found | +| `no_active_sessions` | 404 | Actor has no active sessions | + +> **Full model:** See [Session Token Revocation](../data-model/35-session-revocation.md) โ€” AUTH-016โ€“AUTH-022. + + ## 11. Error Model All Admin API errors use the same envelope as the Consumer API: diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index be34e01..c4a9ab2 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -238,6 +238,78 @@ X-DCM-StepUp-Token: --- +### 2.5 Session Management + +DCM issues a session token on successful authentication. Sessions can be managed and revoked by the authenticated actor. + +```http +# Log out current session +DELETE /api/v1/auth/session +Authorization: Bearer + +Response 204 No Content +``` + +```http +# Log out all sessions for this actor +DELETE /api/v1/auth/sessions +Authorization: Bearer + +Response 204 No Content +``` + +```http +# List active sessions for this actor +GET /api/v1/auth/sessions +Authorization: Bearer + +Response 200: +{ + "items": [ + { + "session_uuid": "", + "created_at": "", + "expires_at": "", + "auth_method": "oidc", + "mfa_verified": true, + "last_active_at": "", + "is_current": true + } + ], + "total": 2 +} +``` + +```http +# Revoke a specific session +DELETE /api/v1/auth/sessions/{session_uuid} +Authorization: Bearer + +Response 204 No Content +``` + +```http +# Token introspection (RFC 7662) โ€” for internal components and trusted integrations +POST /api/v1/auth/introspect +Authorization: Bearer + +{ "token": "" } + +Response 200: +{ + "active": true, + "session_uuid": "", + "actor_uuid": "", + "tenant_uuid": "", + "expires_at": "", + "mfa_verified": true, + "auth_method": "oidc" +} +``` + +> **Session revocation model:** See [Session Token Revocation](../data-model/35-session-revocation.md) for the complete session lifecycle, revocation triggers, revocation registry, profile-governed TTLs, and AUTH-016โ€“AUTH-022 system policies. + + ## 3. Service Catalog ### 3.1 List Catalog Items @@ -488,7 +560,58 @@ Response 200: } ``` -### 4.3 Consumer Request Status Lifecycle +### 4.3 Live Request Status Stream (Server-Sent Events) + +For consumers that want live status updates without polling, DCM exposes a Server-Sent Events (SSE) stream per request: + +``` +GET /api/v1/requests/{request_uuid}/stream +Accept: text/event-stream +Authorization: Bearer + +# Response: HTTP 200, Content-Type: text/event-stream +# Connection stays open; events pushed as state changes + +event: status_change +data: {"status":"PROVISIONING","at":"2026-04-01T02:00:05Z","current_step":"Configuring network interfaces"} + +event: progress_updated +data: {"step_current":3,"step_total":7,"step_label":"Configuring network interfaces","constituent_status":[{"ref":"vm","status":"REALIZED"},{"ref":"dns","status":"PROVISIONING"}]} + +event: status_change +data: {"status":"COMPLETED","at":"2026-04-01T02:03:12Z"} + +# Stream closes on terminal status (COMPLETED, FAILED, CANCELLED) +``` + +**SSE events on this stream:** + +| Event name | When | Data fields | +|------------|------|-------------| +| `status_change` | Request status changes | status, at, current_step | +| `progress_updated` | Provider sends interim progress | step_current, step_total, step_label, constituent_status | +| `approval_required` | Request routed to approval tier | approval_uuid, required_tier, window_expires_at | +| `approval_recorded` | A reviewer votes | votes_recorded, quorum_required, quorum_reached | +| `heartbeat` | Every 30s (keep-alive) | ts | + +**Constituent status** (for compound/Meta Provider requests): +```json +{ + "constituent_status": [ + { "ref": "vm", "status": "REALIZED", "entity_uuid": "" }, + { "ref": "ip", "status": "REALIZED", "entity_uuid": "" }, + { "ref": "dns", "status": "PROVISIONING", "entity_uuid": null }, + { "ref": "storage", "status": "PENDING", "entity_uuid": null } + ] +} +``` + +**Fallback:** Consumers that cannot use SSE (e.g. some proxy configurations) should use polling via `GET /api/v1/requests/{uuid}/status` with an appropriate interval. + +**Connection limits:** One SSE stream per request_uuid per actor. Opening a second stream closes the first. + + +### 4.4 Consumer Request Status Lifecycle ``` ACKNOWLEDGED โ†’ request received; intent created @@ -509,7 +632,7 @@ COMPENSATION_FAILED โ†’ rollback failed; platform admin notified; orphan det PENDING_REVIEW โ†’ conflict detected requiring human resolution ``` -### 4.4 Cancel Request +### 4.5 Cancel Request Cancellation is only available before the PROVISIONING state. Once a provider is executing, cancellation moves to CANCELLING and depends on provider support. @@ -533,6 +656,53 @@ Response 409 Conflict (if cancellation not possible): --- +### 4.5 Request Groups (Dependency Graph) + +Consumers can declare an ordered dependency graph across independent requests using request groups. DCM dispatches constituent requests in dependency order. + +```http +# Create a request group +POST /api/v1/request-groups +Authorization: Bearer + +{ + "label": "provision-web-stack", + "requests": [ + { "request_uuid": "", "depends_on": [] }, + { "request_uuid": "", "depends_on": [""] }, + { "request_uuid": "", "depends_on": [""] } + ] +} + +Response 201: +{ + "request_group_uuid": "", + "label": "provision-web-stack", + "status": "pending", + "requests": [...] +} +``` + +```http +# Get request group status +GET /api/v1/request-groups/{group_uuid} +Authorization: Bearer + +Response 200: +{ + "request_group_uuid": "", + "status": "in_progress", + "requests": [ + { "request_uuid": "", "status": "realized", "dispatched_at": "..." }, + { "request_uuid": "", "status": "dispatched", "dispatched_at": "..." }, + { "request_uuid": "", "status": "pending_dependency", "blocked_by": [""] } + ] +} +``` + +> **Request dependency graph model:** See [Request Dependency Graph](../data-model/38-request-dependency-graph.md) for cycle detection, partial failure handling, and RDG-001โ€“RDG-006 system policies. + + ## 5. Resource Management ### 5.1 List Owned Resources diff --git a/content/docs/architecture/specifications/examples.md b/content/docs/architecture/specifications/examples.md index 40227b1..b530e7b 100644 --- a/content/docs/architecture/specifications/examples.md +++ b/content/docs/architecture/specifications/examples.md @@ -4,7 +4,7 @@ type: docs weight: 7 --- -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Status:** Draft โ€” Examples document - no WIP status needed; always current with architecture. **Document Type:** Reference Examples **Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Provider Contract](../data-model/A-provider-contract.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) | [Registration](dcm-registration-spec.md) | [OPA Integration](dcm-opa-integration-spec.md) diff --git a/content/docs/architecture/specifications/kubernetes-compatibility.md b/content/docs/architecture/specifications/kubernetes-compatibility.md index d0f3e82..25e33e4 100644 --- a/content/docs/architecture/specifications/kubernetes-compatibility.md +++ b/content/docs/architecture/specifications/kubernetes-compatibility.md @@ -25,7 +25,7 @@ weight: 6 -**Document Status:** ๐Ÿ”„ In Progress +**Document Status:** โœ… Complete **Document Type:** Architecture Reference **Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) diff --git a/content/docs/architecture/specifications/operator-interface-spec.md b/content/docs/architecture/specifications/operator-interface-spec.md index c00c6a7..184905a 100644 --- a/content/docs/architecture/specifications/operator-interface-spec.md +++ b/content/docs/architecture/specifications/operator-interface-spec.md @@ -645,6 +645,49 @@ If the provider returns `status: too_late`, it must still send the completed rea ``` +### 6.4 Interim Status Reporting + +For long-running operations (provisioning complex resources, compound service constituents), providers may send interim progress updates to DCM without waiting for terminal status. This gives DCM โ€” and therefore consumers โ€” live visibility into multi-step operations. + +**DCM endpoint for interim status:** + +``` +POST /api/v1/provider/entities/{entity_uuid}/status + +Authorization: Bearer +Content-Type: application/json + +{ + "request_id": "", + "lifecycle_state": "PROVISIONING", // current state โ€” not yet terminal + "progress": { + "step_current": 3, + "step_total": 7, + "step_label": "Configuring network interfaces", + "step_started_at": "", + "estimated_completion": "" + }, + "constituent_status": [ // for compound/Meta Provider operations + { "ref": "vm", "status": "REALIZED", "completed_at": "" }, + { "ref": "ip", "status": "REALIZED", "completed_at": "" }, + { "ref": "dns", "status": "PROVISIONING", "started_at": "" }, + { "ref": "storage", "status": "PENDING", "started_at": null } + ], + "notes": "" +} + +Response 202 Accepted +``` + +DCM uses interim status to: +1. Update `current_step` and progress fields in the request status response +2. Publish `request.progress_updated` event (info urgency) to the Message Bus +3. Deliver live status updates to consumers via SSE stream (see Consumer API Section 4.3) + +**Frequency:** Providers should not send interim status more frequently than once per 10 seconds. DCM rate-limits interim status calls per entity_uuid. + +**Terminal status** is still reported via the existing create/update response callback โ€” interim status supplements, not replaces it. + ## 7. Field Mapping Specification *Required for Level 2 conformance.* diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index 3f189ee..0a1ad28 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -367,6 +367,65 @@ --- +## 27. Session Revocation + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| SES-001 | Session Lifecycle Management | View own active sessions; logout single session (DELETE /api/v1/auth/session); logout all sessions; revoke specific session by UUID | โ€” | Force revoke sessions for any actor; view session store health | IAM-001, AUTH-001 | +| SES-002 | Actor Deprovisioning Session Revocation | โ€” | โ€” | Parallel session + credential revocation on actor deprovisioning; deprovisioning not acknowledged until both complete (AUTH-016) | SES-001, CPX-005 | +| SES-003 | Emergency Session Revocation | Receive critical notification on security-event session revocation | โ€” | Trigger emergency revocation (security_event); revocation propagates within profile SLA (PT5S sovereign to PT30S standard) | SES-001, EVT-001 | +| SES-004 | Token Introspection | โ€” | Call POST /api/v1/auth/introspect to validate bearer tokens without maintaining own revocation cache | Configure introspection endpoint access; manage introspection scope grants | SES-001, IAM-001 | +| SES-005 | Concurrent Session Enforcement | Oldest session auto-revoked when new session exceeds concurrent limit; receive notification via Notification Provider | โ€” | Configure concurrent_sessions limit per profile; monitor session counts | SES-001 | + +--- + +## 28. Internal Component Authentication + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ICOM-001 | Component Identity and mTLS | โ€” | โ€” | Manage Internal CA; issue and rotate component certificates; all inter-component calls use mTLS (ICOM-001) | ZTS-001, CPX-001 | +| ICOM-002 | Component Bootstrap | โ€” | โ€” | Generate one-time bootstrap tokens (PT1H max lifetime); components acquire first certificate via bootstrap token; token invalidated after single use (ICOM-007) | ICOM-001 | +| ICOM-003 | Internal Call Authorization | โ€” | โ€” | Declare allowed_sources per internal endpoint; declare allowed_targets per component; unauthorized source calls rejected with ICOM_UNAUTHORIZED_SOURCE audit record (urgency: high) | ICOM-001, AUD-001 | +| ICOM-004 | Internal Interaction Credentials | โ€” | โ€” | Every internal call presents a scoped ZTS-002 interaction credential in addition to mTLS; credential scoped to specific operation and target component | ICOM-001, CPX-002, ZTS-002 | +| ICOM-005 | Component Certificate Revocation | โ€” | โ€” | Compromised component certificates added to Internal CA CRL immediately; CRL cache refresh within profile SLA (PT15S sovereign to PT1M standard); ICOM_CERT_COMPROMISED audit record (urgency: critical) | ICOM-001, AUD-001 | + +--- + + +## 29. Scheduled and Deferred Requests + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| SCH-001 | Request Scheduling | Submit requests with schedule.dispatch: at/window/recurring; SCHEDULED requests visible in GET /api/v1/requests; cancellable before dispatch; receive request.scheduled event | โ€” | Manage Maintenance Windows; configure Request Scheduler; monitor scheduled queue depth | REQ-001 | +| SCH-002 | Maintenance Windows | Reference maintenance windows in scheduled requests; view available windows at GET /api/v1/maintenance-windows | โ€” | Create/manage/suspend maintenance windows; approve window schedules; configure platform-wide windows | SCH-001, GOV-001 | +| SCH-003 | Dual Policy Evaluation | โ€” | โ€” | Understand that scheduled requests run GateKeeper at declaration AND at dispatch; dispatch-time failure โ†’ FAILED with schedule_policy_rejection (SCH-003) | SCH-001, POL-001 | +| SCH-004 | Deadline Enforcement | Set not_after on scheduled requests; receive request.failed(schedule_deadline_missed) if deadline passes without dispatch | โ€” | Monitor deadline miss rates; configure alerting on deadline misses | SCH-001, EVT-001 | + +--- + +## 30. Request Dependency Graph + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| RDG-001 | Dependency Group Submission | Submit POST /api/v1/request-groups with requests and depends_on declarations; local refs within submission; receive group_uuid and per-request entity_uuids | โ€” | Monitor group queue depth; configure max group size | REQ-001 | +| RDG-002 | Field Injection | Declare inject_fields to pass realized output fields (e.g. IP address) from dependency into dependent request fields automatically at dispatch time | โ€” | โ€” | RDG-001, REQ-001 | +| RDG-003 | PENDING_DEPENDENCY Status | Track dependent requests in PENDING_DEPENDENCY status; cancel pending requests individually or cancel whole group; receive request.pending_dependency and request.dependency_met events | โ€” | Monitor PENDING_DEPENDENCY queue depth; detect stalled groups | RDG-001, EVT-001 | +| RDG-004 | Group Failure Handling | Configure on_failure: cancel_remaining or continue; group-level timeout; group status via GET /api/v1/request-groups/{uuid} | โ€” | Monitor group failure rates | RDG-001 | + +--- + +## 31. DCM Self-Health + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| HLT-001 | Liveness Probe | โ€” | โ€” | GET /livez: fast liveness check (PT5S max, no external calls); Kubernetes restarts pod on failure; unauthenticated | โ€” | +| HLT-002 | Readiness Probe | โ€” | โ€” | GET /readyz: checks Session Store, Audit Store, Policy Engine, Message Bus, Auth Provider connectivity; Kubernetes removes from LB on failure; startup sequence observable via readyz | โ€” | +| HLT-003 | Component Health Detail | โ€” | โ€” | GET /api/v1/admin/health: per-component status (pass/warn/fail), metrics, queue depths, provider/auth summary; admin auth required | IAM-001 | +| HLT-004 | Prometheus Metrics | โ€” | โ€” | GET /metrics: Prometheus scrape endpoint; request pipeline, policy, session, drift, provider, internal CA metrics | OBS-001 | + +--- + + ## Capability Count Summary | Domain | Capabilities | @@ -397,7 +456,12 @@ | Authority Tier Model | 7 | | Event Catalog | 7 | | API Versioning | 5 | -| **Total** | **167** | +| Session Revocation | 5 | +| Internal Component Auth | 5 | +| Scheduled Requests | 4 | +| Request Dependency Graph | 4 | +| DCM Self-Health | 4 | +| **Total** | **189** | --- diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md index bb4bc86..12bc7cf 100644 --- a/content/docs/taxonomy.md +++ b/content/docs/taxonomy.md @@ -115,6 +115,62 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture + + + +### External CA and Live Update Terms + +| Term | Definition | +|------|-----------| +| **External CA Credential Provider** | A Credential Provider backend that issues x509 certificates using standard protocols (ACME/RFC 8555, EST/RFC 7030, SCEP, CMP, or native API like HashiCorp Vault PKI). Recommended for fsi and sovereign profiles to maintain enterprise PKI chain. Registered trust anchor root cert must be installed in all component trust stores. | +| **Trust Anchor** | The root or intermediate CA certificate installed in all DCM component trust stores. May be the built-in Internal CA or an external CA registered as a Credential Provider. ICOM-009: components only accept certificates from registered trust anchors. | +| **Server-Sent Events (SSE)** | W3C standard HTTP/1.1 unidirectional event stream. DCM exposes `GET /api/v1/requests/{uuid}/stream` as an SSE endpoint for live request status updates without polling. Stream closes on terminal status. | +| **Interim Status** | Provider-sent progress update during a long-running operation, via `POST /api/v1/provider/entities/{uuid}/status`. Includes step_current/step_total, step_label, and constituent_status array for compound operations. Triggers `request.progress_updated` event. | +| **constituent_status** | Array of named component statuses in a compound/Meta Provider request (e.g. `[{ref: "vm", status: "REALIZED"}, {ref: "dns", status: "PROVISIONING"}]`). Surfaced in SSE stream and polling response so consumers can track multi-part operations. | + + +### Scheduling and Dependency Terms + +| Term | Definition | +|------|-----------| +| **Scheduled Request** | A DCM request with an explicit dispatch schedule (at a specific time, during a maintenance window, or recurring). Goes through the same pipeline as immediate requests; policy evaluates at declaration AND at dispatch time. | +| **PENDING_DEPENDENCY** | Intent State status for a request in a dependency group waiting for its declared dependency to reach the required wait_for state before dispatch. | +| **Request Dependency Group** | A consumer-declared set of requests with ordering constraints (depends_on) between them. Distinct from type-level dependencies (doc 07) and Meta Provider composition (doc 30). | +| **Field Injection** | Mechanism for passing realized output fields from a dependency automatically into a dependent request's fields at dispatch time. Subject to Transformation policies. | +| **Maintenance Window** | A reusable, named recurrence artifact declaring approved change windows. Consumers reference window_uuid in scheduled requests to slot into the next matching window. | +| **SCH-001โ€“SCH-006** | Scheduled requests system policies. Key: SCH-001 (dual policy evaluation: declaration + dispatch), SCH-003 (dispatch-time policy rejection โ†’ FAILED), SCH-005 (not_after deadline miss โ†’ FAILED, no retry). | +| **RDG-001โ€“RDG-006** | Request dependency graph policies. Key: RDG-001 (circular deps rejected at submission), RDG-002 (max 50 requests per group), RDG-004 (PENDING_DEPENDENCY requests count against quota), RDG-006 (request may belong to one group only). | + +### Self-Health Terms + +| Term | Definition | +|------|-----------| +| **Liveness (/livez)** | Fast DCM health check (PT5S max, no external calls). Failure โ†’ Kubernetes restarts the pod. Unauthenticated. | +| **Readiness (/readyz)** | DCM readiness check โ€” validates Session Store, Audit Store, Policy Engine, Message Bus, Auth Provider. Failure โ†’ removed from load balancer. Used for startup probes. | +| **HLT-001โ€“HLT-006** | Self-health system policies. Key: HLT-001 (livez and readyz required, unauthenticated), HLT-002 (livez PT5S max, no external calls), HLT-003 (readyz fails if core dependencies unreachable), HLT-005 (Prometheus metrics required). | + + +### Session Revocation Terms + +| Term | Definition | +|------|-----------| +| **Session Record** | DCM Data artifact tracking an active actor session: session_uuid, actor_uuid, auth_provider_uuid, created_at, expires_at, status (active/refreshing/revoked/expired), revocation metadata. | +| **Session Revocation Registry** | Fast-queryable store of revoked-but-not-yet-expired session UUIDs. All components that accept bearer tokens must check this on every request. Cache age is profile-governed (PT5M minimal โ†’ no cache sovereign). | +| **Session Store** | Operational store for active sessions (not GitOps-backed). Separate from Realized State Store. Backed by Redis or Postgres (standard+) or in-memory (minimal/dev). | +| **Token Introspection** | RFC 7662 endpoint (`POST /api/v1/auth/introspect`) for validating bearer tokens. Returns active/inactive plus session metadata. Requires `introspection` scope. | +| **AUTH-016โ€“AUTH-022** | Session revocation system policies. Key: AUTH-016 (deprovisioning fires session + credential revocation in parallel), AUTH-017 (revocation SLA), AUTH-018 (all components check revocation registry), AUTH-019 (emergency revocation: critical urgency, non-suppressable). | + +### Internal Component Auth Terms + +| Term | Definition | +|------|-----------| +| **Internal CA** | The Certificate Authority operated by each DCM deployment for issuing mTLS certificates to internal components. Not exposed externally. Root cert installed in all component trust stores at deployment time. | +| **Component Identity** | Each DCM control plane component has a stable UUID, an mTLS certificate from the Internal CA, and a service account with declared allowed_sources and allowed_targets. | +| **Bootstrap Token** | A one-time-use credential (max PT1H lifetime) that enables a new component to acquire its first mTLS certificate from the Internal CA. Invalidated immediately after use. | +| **Component Communication Graph** | The declared graph of which components may call which others. Components may only call `allowed_targets`; endpoints only accept calls from `allowed_sources`. Violations are rejected and audited (ICOM-003, ICOM-004). | +| **ICOM-001โ€“ICOM-009** | Internal Component Auth system policies. Key: ICOM-001 (mTLS required for all internal calls), ICOM-002 (interaction credential required in addition to mTLS), ICOM-007 (bootstrap tokens one-time-use, PT1H max), ICOM-008 (compromised certs โ†’ CRL immediately). | + + ### API Versioning Terms | Term | Definition | @@ -266,6 +322,11 @@ Terms to avoid because they introduce ambiguity. Use the precise alternatives in | ATM | Authority Tier Model | | EVT | Event Catalog | | VER | API Versioning | +| SES | Session Revocation | +| ICOM | Internal Component Auth | +| SCH | Scheduled Requests | +| RDG | Request Dependency Graph | +| HLT | DCM Self-Health | --- From b319b7ddd47113b245de8dff748daee1bfcff715 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Mon, 30 Mar 2026 08:28:44 -0400 Subject: [PATCH 46/49] Massive update with AEP applied, API specs coming together, Consistency run applied. Signed-off-by: Chris Roadfeldt --- content/docs/DISCUSSION-TOPICS.md | 543 +++++ content/docs/architecture/ai-prompt.md | 204 +- .../architecture/data-model/event-catalog.md | 38 +- .../architecture/data-model/foundations.md | 1 + .../data-model/itsm-integration.md | 670 ++++++ .../data-model/operational-reference.md | 525 +++++ .../data-model/policy-contract.md | 33 +- .../data-model/provider-contract.md | 30 + .../data-model/standards-catalog.md | 18 +- .../11-kubernetes-compatibility.md | 422 ++++ .../specifications/cncf-strategy.md | 19 +- .../specifications/consumer-api-spec.md | 6 +- .../specifications/dcm-admin-api-spec.md | 973 ++++++++ .../specifications/dcm-admin-gui-spec.md | 250 ++ .../specifications/dcm-consumer-gui-spec.md | 919 ++++++++ .../specifications/dcm-examples.md | 874 +++++++ .../specifications/dcm-flow-gui-spec.md | 906 ++++++++ .../dcm-opa-integration-spec.md | 575 +++++ .../dcm-operator-interface-spec.md | 1137 ++++++++++ .../specifications/dcm-operator-sdk-api.md | 669 ++++++ .../specifications/dcm-provider-gui-spec.md | 324 +++ .../specifications/dcm-registration-spec.md | 835 +++++++ .../dcm-rhdh-integration-spec.md | 637 ++++++ content/docs/capabilities-matrix.md | 149 +- .../docs/data-model/00-context-and-purpose.md | 384 ++++ .../docs/data-model/00-design-priorities.md | 312 +++ content/docs/data-model/00-foundations.md | 305 +++ content/docs/data-model/01-entity-types.md | 351 +++ content/docs/data-model/02-four-states.md | 862 +++++++ .../data-model/03-layering-and-versioning.md | 2019 +++++++++++++++++ content/docs/data-model/04-examples.md | 490 ++++ .../04b-ownership-sharing-allocation.md | 312 +++ .../data-model/05-resource-type-hierarchy.md | 504 ++++ .../06-resource-service-entities.md | 894 ++++++++ .../data-model/07-service-dependencies.md | 553 +++++ .../docs/data-model/08-resource-grouping.md | 393 ++++ .../data-model/09-entity-relationships.md | 1076 +++++++++ .../data-model/10-information-providers.md | 451 ++++ .../docs/data-model/11-storage-providers.md | 796 +++++++ .../12-audit-provenance-observability.md | 466 ++++ content/docs/data-model/13-ingestion-model.md | 481 ++++ content/docs/data-model/14-policy-profiles.md | 1653 ++++++++++++++ .../docs/data-model/15-universal-groups.md | 682 ++++++ content/docs/data-model/16-universal-audit.md | 618 +++++ .../data-model/17-deployment-redundancy.md | 791 +++++++ .../docs/data-model/18-webhooks-messaging.md | 831 +++++++ content/docs/data-model/19-auth-providers.md | 693 ++++++ .../docs/data-model/20-registry-governance.md | 643 ++++++ .../21-information-providers-advanced.md | 704 ++++++ content/docs/data-model/22-dcm-federation.md | 597 +++++ .../docs/data-model/23-notification-model.md | 606 +++++ .../docs/data-model/24-operational-models.md | 864 +++++++ .../data-model/25-control-plane-components.md | 826 +++++++ ...-accreditation-and-authorization-matrix.md | 634 ++++++ .../docs/data-model/27-governance-matrix.md | 840 +++++++ .../28-federated-contribution-model.md | 592 +++++ content/docs/data-model/29-scoring-model.md | 516 +++++ .../docs/data-model/30-meta-provider-model.md | 502 ++++ .../31-credential-provider-model.md | 984 ++++++++ .../data-model/32-authority-tier-model.md | 527 +++++ content/docs/data-model/33-event-catalog.md | 782 +++++++ .../data-model/34-api-versioning-strategy.md | 386 ++++ .../docs/data-model/35-session-revocation.md | 351 +++ .../data-model/36-internal-component-auth.md | 369 +++ .../docs/data-model/37-scheduled-requests.md | 291 +++ .../data-model/38-request-dependency-graph.md | 305 +++ content/docs/data-model/39-dcm-self-health.md | 370 +++ .../docs/data-model/40-standards-catalog.md | 293 +++ .../data-model/41-operational-reference.md | 521 +++++ .../docs/data-model/42-itsm-integration.md | 666 ++++++ .../docs/data-model/A-provider-contract.md | 560 +++++ content/docs/data-model/B-policy-contract.md | 430 ++++ content/docs/schemas/README.md | 176 ++ content/docs/taxonomy.md | 71 + 74 files changed, 41064 insertions(+), 16 deletions(-) create mode 100644 content/docs/DISCUSSION-TOPICS.md create mode 100644 content/docs/architecture/data-model/itsm-integration.md create mode 100644 content/docs/architecture/data-model/operational-reference.md create mode 100644 content/docs/architecture/specifications/11-kubernetes-compatibility.md create mode 100644 content/docs/architecture/specifications/dcm-admin-api-spec.md create mode 100644 content/docs/architecture/specifications/dcm-admin-gui-spec.md create mode 100644 content/docs/architecture/specifications/dcm-consumer-gui-spec.md create mode 100644 content/docs/architecture/specifications/dcm-examples.md create mode 100644 content/docs/architecture/specifications/dcm-flow-gui-spec.md create mode 100644 content/docs/architecture/specifications/dcm-opa-integration-spec.md create mode 100644 content/docs/architecture/specifications/dcm-operator-interface-spec.md create mode 100644 content/docs/architecture/specifications/dcm-operator-sdk-api.md create mode 100644 content/docs/architecture/specifications/dcm-provider-gui-spec.md create mode 100644 content/docs/architecture/specifications/dcm-registration-spec.md create mode 100644 content/docs/architecture/specifications/dcm-rhdh-integration-spec.md create mode 100644 content/docs/data-model/00-context-and-purpose.md create mode 100644 content/docs/data-model/00-design-priorities.md create mode 100644 content/docs/data-model/00-foundations.md create mode 100644 content/docs/data-model/01-entity-types.md create mode 100644 content/docs/data-model/02-four-states.md create mode 100644 content/docs/data-model/03-layering-and-versioning.md create mode 100644 content/docs/data-model/04-examples.md create mode 100644 content/docs/data-model/04b-ownership-sharing-allocation.md create mode 100644 content/docs/data-model/05-resource-type-hierarchy.md create mode 100644 content/docs/data-model/06-resource-service-entities.md create mode 100644 content/docs/data-model/07-service-dependencies.md create mode 100644 content/docs/data-model/08-resource-grouping.md create mode 100644 content/docs/data-model/09-entity-relationships.md create mode 100644 content/docs/data-model/10-information-providers.md create mode 100644 content/docs/data-model/11-storage-providers.md create mode 100644 content/docs/data-model/12-audit-provenance-observability.md create mode 100644 content/docs/data-model/13-ingestion-model.md create mode 100644 content/docs/data-model/14-policy-profiles.md create mode 100644 content/docs/data-model/15-universal-groups.md create mode 100644 content/docs/data-model/16-universal-audit.md create mode 100644 content/docs/data-model/17-deployment-redundancy.md create mode 100644 content/docs/data-model/18-webhooks-messaging.md create mode 100644 content/docs/data-model/19-auth-providers.md create mode 100644 content/docs/data-model/20-registry-governance.md create mode 100644 content/docs/data-model/21-information-providers-advanced.md create mode 100644 content/docs/data-model/22-dcm-federation.md create mode 100644 content/docs/data-model/23-notification-model.md create mode 100644 content/docs/data-model/24-operational-models.md create mode 100644 content/docs/data-model/25-control-plane-components.md create mode 100644 content/docs/data-model/26-accreditation-and-authorization-matrix.md create mode 100644 content/docs/data-model/27-governance-matrix.md create mode 100644 content/docs/data-model/28-federated-contribution-model.md create mode 100644 content/docs/data-model/29-scoring-model.md create mode 100644 content/docs/data-model/30-meta-provider-model.md create mode 100644 content/docs/data-model/31-credential-provider-model.md create mode 100644 content/docs/data-model/32-authority-tier-model.md create mode 100644 content/docs/data-model/33-event-catalog.md create mode 100644 content/docs/data-model/34-api-versioning-strategy.md create mode 100644 content/docs/data-model/35-session-revocation.md create mode 100644 content/docs/data-model/36-internal-component-auth.md create mode 100644 content/docs/data-model/37-scheduled-requests.md create mode 100644 content/docs/data-model/38-request-dependency-graph.md create mode 100644 content/docs/data-model/39-dcm-self-health.md create mode 100644 content/docs/data-model/40-standards-catalog.md create mode 100644 content/docs/data-model/41-operational-reference.md create mode 100644 content/docs/data-model/42-itsm-integration.md create mode 100644 content/docs/data-model/A-provider-contract.md create mode 100644 content/docs/data-model/B-policy-contract.md create mode 100644 content/docs/schemas/README.md diff --git a/content/docs/DISCUSSION-TOPICS.md b/content/docs/DISCUSSION-TOPICS.md new file mode 100644 index 0000000..d9901e4 --- /dev/null +++ b/content/docs/DISCUSSION-TOPICS.md @@ -0,0 +1,543 @@ +# DCM โ€” Discussion Topics + +> **Status Update (2026-03):** The DCM architecture has been substantially completed. All original 19 architecture review items, 10 session-added items, and 21 community questions have been resolved. The items below represent topics for ongoing community discussion and future evolution. +> +> **Current state:** 0 unresolved architectural questions ยท 126 capabilities across 20 domains ยท 33 data model documents ยท 10 specifications +> +> **Foundation:** Three abstractions (Data ยท Provider ยท Policy) ยท Unified Governance Matrix ยท Federated Contribution Model ยท Full OPA/Rego validation complete + +## Open Community Discussion Topics + +### 1. Kubernetes / CNCF Strategy (community decisions) +- CNCF submission scope: Operator Interface Spec as specification project first; DCM project after Level 2 reference implementation +- SIG App Delivery and SIG Cluster Lifecycle engagement before Sandbox submission +- Named adopters and TOC sponsor: project team action items +- Level 2 conformance scope now formally defined + +### 2. Implementation Decisions (engineering decisions) +- KubeVirt reference implementation timeline โ†’ team estimates against defined Level 2 scope +- SDK language support beyond Go โ†’ community SDKs encouraged; Go SDK is reference implementation +- Non-Kubernetes container runtime support โ†’ implementation detail + +### 3. Future Evolution Topics +- Normative data specifications (JSON Schema / OpenAPI) โ€” the code-generation layer +- AI/ML Provider type โ€” as DCM becomes AI-ready, a dedicated ML workload provider type +- Billing Provider type โ€” deeper integration with enterprise billing and showback systems +- CMDB Provider type โ€” dedicated contract for CMDB integration +- Multi-cloud federation model โ€” extending DCM federation to public cloud providers +- GitOps PR UX improvements โ€” better tooling for policy review workflow + +### 4. Governance Questions +- Community governance model โ€” how will the DCM project make decisions once public? +- Certified Profile Program โ€” self-certified vs project-reviewed for compliance profiles +- Registry tier promotion โ€” Tier 3 (Organization) to Tier 2 (Verified Community) pathway + +--- + +*See [00-foundations.md](data-model/00-foundations.md) for the three-abstraction model that resolved the major architectural questions.* + + +--- + +**Original discussion topics (archived):** + +# DCM โ€” Topics for Discussion and Modification + +**Document Status:** ๐Ÿ”„ Active +**Purpose:** A living document capturing topics that require further discussion, design decisions that need revisiting, and new capabilities to be incorporated into the DCM architecture and data model. +**Process:** When a topic is resolved, move it to the appropriate architecture or data model document and mark it resolved here with a reference to where it was documented. + +--- + +## How to Use This Document + +- **Add** any topic that surfaces during design, review, or implementation that needs a decision or deeper discussion +- **Tag** each item with its area, priority, and status +- **Resolve** items by documenting the decision in the appropriate document and updating the status here +- **Never delete** resolved items โ€” keep the full history for audit and traceability + +--- + +## Status Key + +| Status | Meaning | +|--------|---------| +| ๐Ÿ”ด Blocking | Must be resolved before dependent work can proceed | +| ๐ŸŸก Active | Under active discussion | +| ๐ŸŸข Resolved | Decision made โ€” documented in referenced document | +| โšช Parked | Acknowledged but deferred โ€” revisit later | + +--- + +## Priority Key + +| Priority | Meaning | +|----------|---------| +| P1 | Critical โ€” affects foundational architecture | +| P2 | High โ€” affects multiple components or documents | +| P3 | Medium โ€” affects a specific component or document | +| P4 | Low โ€” enhancement or refinement | + +--- + +## Open Topics + +--- + +### TOPIC-001 โ€” Webhook Integration + +**Area:** Control Plane, Provider Contract, Egress +**Priority:** P2 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +DCM needs a webhook integration model that allows external systems to be notified of DCM events and state changes in real time. Webhooks are a standard integration pattern that complements the existing API-first model and Message Bus โ€” they enable a push-based notification model for consumers, providers, and external systems that cannot or do not poll DCM. + +#### Use Cases + +**Consumer Notifications** +- Notify a consumer's CI/CD pipeline when a resource request transitions to REALIZED state +- Notify an application team when their Resource/Service Entity enters DEGRADED state +- Notify a Tenant owner when an ownership transfer is initiated or completed +- Notify consumers when a dependency graph node fails during realization + +**Provider Notifications** +- Notify a provider when a new request payload is dispatched to them +- Notify a provider when DCM initiates a discovery request +- Notify a provider when a decommission is requested for one of their entities + +**External System Integration** +- Notify an external ITSM system (ServiceNow, Jira) when a request is created, updated, or completed +- Notify a monitoring system when Entity lifecycle state changes +- Notify a FinOps platform when new Resource/Service Entities are realized or decommissioned +- Notify a compliance system when GateKeeper policies fire or sovereignty constraints are applied + +**Operational Notifications** +- Notify SRE teams when provider capacity falls below threshold +- Notify security teams when unsanctioned changes are detected +- Notify auditors when specific policy types are triggered + +#### Design Questions to Resolve + +1. **Webhook registration model** โ€” how do consumers, providers, and external systems register webhooks with DCM? Is registration via the Consumer API, Provider Registration, or a dedicated Webhook API? + +2. **Event taxonomy** โ€” what is the full list of events DCM can emit via webhook? Should this be an extensible registry similar to the Resource Type Registry? + +3. **Payload format** โ€” should webhook payloads use the DCM unified data model format, or a simplified event notification format? Should the full state payload be included or just a reference + event type? + +4. **Authentication and security** โ€” how does DCM authenticate outbound webhook calls? Options include: HMAC signatures, OAuth tokens, mTLS, API keys. How does the receiving system verify the webhook is genuinely from DCM? + +5. **Retry and reliability** โ€” what is DCM's obligation if a webhook delivery fails? Should DCM retry? How many times? With what backoff strategy? What happens if a webhook endpoint is consistently unavailable? + +6. **Ordering guarantees** โ€” are webhook events delivered in order? What happens if events arrive out of order at the receiving end? + +7. **Filtering** โ€” can webhook registrations declare filters โ€” only receive events of specific types, for specific Resource Types, for specific Tenants, or for specific Resource Groups? + +8. **Policy Engine integration** โ€” should GateKeeper and other policy types be able to trigger webhook notifications as a policy action? This would make webhooks a first-class policy response alongside ALERT, REVERT, etc. + +9. **Provider webhook obligations** โ€” should providers be required to support webhook endpoints as part of their Provider Contract? Or is webhook support optional for providers? + +10. **Tenant scoping** โ€” should webhook registrations be scoped to a Tenant, meaning a webhook can only receive events for resources owned by the registering Tenant? Or should there be platform-level webhooks that span Tenants (for SRE/Audit personas)? + +11. **Webhook versioning** โ€” as DCM evolves, webhook payload schemas will change. How are webhook payload versions managed? Should webhook registrations declare which payload schema version they expect? + +12. **Relationship to Message Bus** โ€” DCM already has a Message Bus component. What is the distinction between webhook integration and Message Bus integration? Are webhooks the outbound consumer-facing layer on top of the Message Bus? + +#### Initial Design Thoughts + +Webhooks fit naturally as an **Egress capability** โ€” they are outbound notifications from DCM to external systems, which is consistent with the existing Egress zone in the architecture (Messaging Protocol, Interoperability API). + +The webhook registration model should likely be part of the **Consumer API** for consumer-facing webhooks and part of the **Provider Registration** for provider-facing webhooks. + +Webhook events should be **typed and versioned** โ€” consistent with DCM's universal versioning model. An event type like `entity.state.changed` should have a version, and webhook registrations should declare which version they support. + +Webhook payloads should carry **provenance information** โ€” the event payload should include enough context to trace back to the originating request, entity, and policy that caused the event. This is consistent with DCM's auditability requirements. + +**Policy Engine integration** is particularly interesting โ€” if the Policy Engine can fire webhooks as a response action, it enables real-time governance notifications without requiring consumers to poll DCM. This aligns with the DCM goal of getting actionable information to the right people as fast as possible. + +#### References +- [Resource/Service Entities](data-model/06-resource-service-entities.md) โ€” provider lifecycle events +- [Service Dependencies](data-model/07-service-dependencies.md) โ€” dependency failure notifications +- Architecture: Egress zone, Message Bus, API Gateway + +--- + +### TOPIC-002 โ€” Intent Store and Intent Payload Structure + +**Area:** Data Model +**Priority:** P1 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The Intent State is captured when a consumer submits a request but the exact structure of the Intent payload has not been formally specified. The Intent payload is the consumer's raw declared desire โ€” what they asked for before any processing, enrichment, or policy application. It needs a formal definition that is consistent with the four-state model and the layering model. + +#### Questions to Resolve + +1. What fields are required in an Intent payload vs optional? +2. How does the Intent payload reference a Resource Type โ€” by UUID, by fully qualified name, or both? +3. How does the Intent payload declare its Tenant membership? +4. How does the Intent payload declare dependency requirements at the intent level? +5. How does the Intent payload declare group memberships? +6. How is the Intent payload versioned โ€” does it carry a version or is it always a snapshot? + +#### References +- [Context and Purpose](data-model/00-context-and-purpose.md) โ€” four states +- [Layering and Versioning](data-model/03-layering-and-versioning.md) โ€” Request Layer +- [Resource Grouping](data-model/08-resource-grouping.md) โ€” Tenant and group membership + +--- + +### TOPIC-003 โ€” GateKeeper vs Validation Policy Distinction + +**Area:** Policy Engine +**Priority:** P2 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The distinction between GateKeeper and Validation policy categories needs better examples and a clearer formal definition. Both involve checking data against rules, but GateKeeper has override authority while Validation is pass/fail only. The boundary between them needs to be unambiguous. + +#### Questions to Resolve + +1. What is the precise trigger condition that makes a policy a GateKeeper vs a Validation policy? +2. Can a GateKeeper policy both block AND modify in the same execution? +3. Are there cases where Validation and GateKeeper would produce different outcomes for the same rule? +4. Should GateKeeper policies require explicit authorization (e.g., only CISO-owned policies can be GateKeeper)? + +#### References +- [Layering and Versioning](data-model/03-layering-and-versioning.md) โ€” Policy Layer section + +--- + +### TOPIC-004 โ€” Audit vs Observability Component Separation + +**Area:** Control Plane +**Priority:** P3 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The original architecture documents noted that Audit and Observability were separated for a reason that was later forgotten. This needs to be formally resolved โ€” are they truly separate components with distinct responsibilities, or should they be merged? + +#### Initial Thinking + +**Audit** โ€” focused on compliance evidence and transaction traceability. Reads provenance data intrinsic to data objects. Produces compliance reports, audit trails, and interrogation capability for Auditors, Security teams, and SRE personas. Historical record oriented. + +**Observability** โ€” focused on operational visibility โ€” metrics, health, performance, real-time monitoring. Consumes provider lifecycle events, Entity state changes, and system health data. Operational present-state oriented. + +These are likely genuinely separate concerns. Audit is about what happened and why. Observability is about what is happening now. + +#### Questions to Resolve + +1. Are Audit and Observability separate Atomic Components with separate APIs? +2. Do they share a data store or maintain separate stores? +3. How do provider lifecycle events flow to both components? + +--- + +### TOPIC-005 โ€” Message Bus Consumer Ingress Question + +**Area:** Control Plane, Consumer Ingress +**Priority:** P3 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The original architecture documents raised an unresolved question: should the Message Bus be offered as consumer ingress (inbound) in addition to egress (outbound)? The API-first principle suggests it should be egress only, but there are valid integration scenarios where external systems need to push data into DCM asynchronously. + +#### Questions to Resolve + +1. Should consumers be able to submit requests via the Message Bus, or only via the Consumer API? +2. If Message Bus ingress is supported, how are requests authenticated and authorized? +3. How does Message Bus ingress interact with the Intent State capture โ€” is the message treated as an Intent payload? + +--- + +### TOPIC-006 โ€” Cache Architecture + +**Area:** Data Model, Control Plane +**Priority:** P2 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +Several unresolved questions exist about where data caches live, how they are synchronized, and which cache is authoritative when caches diverge. This has implications for distributed DCM deployments and sovereignty scenarios. + +#### Questions to Resolve + +1. Where should data caches live? Hub DCM? Regional DCM? Sovereign DCM? All locations? +2. Should cache synchronization be push, pull, or both? +3. Which cache is authoritative when caches diverge? +4. What mechanism maintains consistency across distributed caches? +5. How do cache architecture decisions interact with sovereignty constraints โ€” can cached data cross sovereignty boundaries? + +--- + +### TOPIC-007 โ€” Cross-Tenant Dependencies + +**Area:** Data Model, Multi-Tenancy +**Priority:** P2 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The dependency model currently assumes dependencies are resolved within a single Tenant. However, real-world deployments will have cross-tenant dependencies โ€” a Payments Tenant application that depends on a shared DNS service owned by a Platform Tenant. The data model needs to formally address how cross-tenant dependencies are declared, resolved, and governed. + +#### Questions to Resolve + +1. How is a cross-tenant dependency declared โ€” does it reference the Tenant UUID of the dependency owner? +2. What authorization is required for a cross-tenant dependency? Does the owning Tenant need to approve? +3. How does cost attribution work for cross-tenant service consumption? +4. How does drift detection work when a dependency is in another Tenant? +5. Can a GateKeeper policy block cross-tenant dependencies? + +--- + +### TOPIC-008 โ€” Provider Trust Validation Mechanism + +**Area:** Service Providers, Provider Contract +**Priority:** P1 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The Provider Contract includes a Trust Contract โ€” providers must be validated and certified to participate in the DCM ecosystem. The mechanism for establishing, maintaining, and revoking trust has not been designed. + +#### Questions to Resolve + +1. What is the certification process for a new provider? +2. What technical mechanism validates trust at request time? +3. How is trust revoked if a provider violates their contract? +4. Should trust be per-provider or per-catalog-item? +5. How does the trust chain interact with sovereignty requirements? + +--- + +### TOPIC-009 โ€” Physical Shared Infrastructure Tenancy + +**Area:** Data Model, Tenancy +**Priority:** P2 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The current model assigns every Resource/Service Entity to exactly one DCM Tenant. This works cleanly for most cases but edge cases exist for truly shared physical infrastructure โ€” a rack, a network switch, a power circuit โ€” where the ownership model may be genuinely joint or ambiguous. + +#### Questions to Resolve + +1. Is jointly-owned physical infrastructure a valid DCM use case, or is it always owned by a Platform/Infrastructure Tenant? +2. If joint ownership is valid, how is it modeled without breaking the single-Tenant rule? +3. Does the Whole Allocation model cover all physical infrastructure scenarios? + +--- + +### TOPIC-010 โ€” Embedded Technology-Specific Data Bundles + +**Area:** Data Model +**Priority:** P3 +**Status:** ๐ŸŸก Active +**Raised:** 2026-03 + +#### Description + +The original data model discussion raised the question of whether the data model should allow embedded target-technology-specific data bundles โ€” for example, a Terraform HCL block or an Ansible vars file embedded directly in an entity definition. The authors noted "not convinced this is something we want." + +#### Questions to Resolve + +1. Should technology-specific data bundles be allowed in entity definitions? +2. If allowed, must they be in clear text and appropriate for Git storage? +3. How would embedded bundles interact with the portability model โ€” they would clearly be portability-breaking? +4. Is there a better mechanism โ€” such as a provider-specific extension field with an appropriate portability classification? + +--- + +## Resolved Topics + +--- + +### TOPIC-R001 โ€” Field Override Control Mechanism + +**Area:** Data Model, Policy Engine, Layering +**Priority:** P1 +**Status:** ๐ŸŸข Resolved +**Raised:** 2026-03 +**Resolved:** 2026-03 + +#### Decision + +Field override control is implemented as a **standard Policy Engine mechanism** using a graduated three-level model: Level 1 (no declaration โ€” fully overridable), Level 2 (simple `override: allow|constrained|immutable`), Level 3 (full actor matrix with per-actor permissions, trusted grants, and expansion rules). The Policy Engine is the sole authority for setting override control. The Request Payload Processor enforces structural layer rules only. + +#### Documented In +- [Data Layers and Assembly Process](data-model/03-layering-and-versioning.md) โ€” Section 5a +- [Context and Purpose](data-model/00-context-and-purpose.md) โ€” Section 4.4 + +--- + +### TOPIC-R002 โ€” Storage/Networking Bundling vs Dependency Model (Q53) + +**Area:** Data Model, Entity Relationships, Service Dependencies +**Priority:** P1 +**Status:** ๐ŸŸข Resolved +**Raised:** 2026-03 +**Resolved:** 2026-03 + +#### Decision + +The conflict between the enhancement documents (storage bundled in VM schema) and the data model (separate first-class entities) is resolved through a **universal Entity Relationship model**: + +- Bundled consumer declarations are expanded by the Request Payload Processor into first-class Resource/Service Entities with their own UUIDs +- The relationship between parent and child entities is expressed using the universal relationship model โ€” bidirectional, UUID-keyed, with lifecycle policies +- The same relationship model is used for ALL entity relationships โ€” compute to storage, application to web server, resource to business unit โ€” minimizing variance +- Lifecycle policies (destroy|retain|detach|notify on parent destroy/suspend/modify) replace the ephemeral/persistent classification +- Expansion rules live in the Resource Type Specification โ€” portable and declarative +- The dependency graph concept is unified into the Entity Relationship Graph + +#### Documented In +- [Entity Relationships](data-model/09-entity-relationships.md) โ€” complete relationship model +- [Information Providers](data-model/10-information-providers.md) โ€” external data relationships +- [Service Dependencies](data-model/07-service-dependencies.md) โ€” updated to reference entity relationships + +--- + +### TOPIC-R003 โ€” Information Provider Model + +**Area:** Data Model, Provider Contract +**Priority:** P1 +**Status:** ๐ŸŸข Resolved +**Raised:** 2026-03 +**Resolved:** 2026-03 + +#### Decision + +Information Providers are a first-class provider type in DCM. They follow the same registration, health check, trust, and contract model as Service Providers where applicable. Key decisions: + +- Information types live in the same DCM registry as Resource Types โ€” distinguished by category prefix (Business.*, Identity.*, Compliance.*, Operations.*) +- Standard vs extended data โ€” DCM only relies on standard fields for operational decisions; organizations can extend with domain-specific fields +- Stable external key model โ€” DCM UUID wraps external UUID; if external system changes its UUID, only the reference record changes +- Three-mode verification โ€” scheduled (Mode 1), provider push (Mode 2, contractual obligation), on-demand (Mode 3, fallback) +- Internal business data follows the standard resource entity model when DCM owns it; external references use the Information Provider model + +#### Documented In +- [Information Providers](data-model/10-information-providers.md) +- [Entity Relationships](data-model/09-entity-relationships.md) โ€” external relationship structure +- [Resource Type Hierarchy](data-model/05-resource-type-hierarchy.md) โ€” information type categories added + +--- + +--- + +### TOPIC-011 โ€” Enhancement Document Compatibility Findings + +**Area:** Data Model, Provider Contract, Policy Engine, Catalog +**Priority:** P1 +**Status:** ๐Ÿ”ด Blocking +**Raised:** 2026-03 + +#### Description + +A compatibility review of six DCM enhancement documents against the data model identified areas of strong alignment and cross-cutting gaps that need resolution before the enhancements can be considered fully aligned with the data model. + +#### Cross-Cutting Gaps (affect all or most enhancements) + +1. **Field-Level Provenance** โ€” absent in all enhancement documents. The Policy Engine spec mutates request payloads without recording provenance. The KubeVirt SP returns minimal status rather than full DCM-format realized payloads. + +2. **UUIDs as Primary Identifiers** โ€” most specs use `name` as the natural key rather than UUID as the primary key. The data model requires UUIDs as primary identifiers on all entities. + +3. **Portability Classification** โ€” absent from all provider and catalog specs. Every field must declare `universal|conditional|provider-specific|exclusive` classification. + +4. **Sovereignty Declarations** โ€” entirely absent from SP Registration and KubeVirt SP. Provider registration must include sovereignty capability declarations as a contractual obligation. + +5. **Tenant Support** โ€” explicitly deferred in V1 in the Policy Engine spec. The data model treats Tenant as mandatory and non-overridable. A clear migration path from V1 to Tenant support is needed. + +6. **Universal Versioning** โ€” catalog items use `apiVersion: v1alpha1` rather than Major.Minor.Revision. All definitions require universal versioning. + +7. **Deprecation Model** โ€” absent from all enhancement documents. All definitions require the `active โ†’ deprecated โ†’ retired` lifecycle. + +#### Document-Specific Gaps + +**SP Registration Flow:** +- Registration payload missing: sovereignty declarations, ownership model declaration, dependency declarations, capacity model mode declaration +- Uses `name` as natural key rather than UUID as primary key +- Capacity data (totalCpu, totalMemory) treated as static rather than dynamic โ€” needs reconciliation with three-mode capacity model + +**Service Type Definitions:** +- `providerHints` not formally marked as portability-breaking โ€” silently bypasses portability enforcement +- Storage and networking bundled with compute โ€” conflicts with the service dependency model which treats these as separate dependent services with their own lifecycle +- No UUIDs on field definitions, no provenance metadata, no versioning beyond v1alpha1 + +**Service Provider Health Check:** +- Covers liveness only โ€” insufficient for full Provider Lifecycle Events contract +- Does not address DEGRADATION, MAINTENANCE, UNSANCTIONED_CHANGE, CAPACITY_CHANGE events +- Binary 200/non-200 model needs to coexist with structured event payload model + +**Policy Engine:** +- `selected_provider` as a direct policy output conflicts with the data model's specificity narrowing model for provider selection +- No policy versioning or GitOps integration โ€” data model requires policies maintained via GitOps +- No provenance recording on field mutations โ€” data model requires provenance on every modification +- Constraint immutability model is compatible and can be mapped to `override_preference` field metadata + +**Catalog Item Schema:** +- No UUIDs, no Tenant scoping, no portability classification, no deprecation model, no provenance +- `editable` field concept is valuable and not explicitly covered in data model โ€” worth incorporating into the Resource Type Specification field definition +- `dependsOn` conditional field pattern maps to `conditional` portability classification but needs formal alignment + +**KubeVirt Service Provider:** +- Realized payload returns minimal status rather than complete DCM-format payload โ€” Denaturalization requirement not met +- No unsanctioned change detection โ€” only VMI phase changes reported +- Registration gaps same as SP Registration Flow spec +- `namespace` in response payload is provider-native concept with no DCM equivalent + +#### Questions to Resolve + +1. Should the enhancement documents be updated to align with the data model, or should the data model be adjusted where the enhancements reveal practical implementation constraints? +2. For the storage/networking bundling vs. dependency model tension โ€” is this a V1 simplification that gets resolved in V2, or does the data model need a "monolithic service" concept? +3. For `selected_provider` in the Policy Engine โ€” should provider selection remain a policy output, or should it be moved to a dedicated placement component that consumes narrowed field sets from the Policy Engine? +4. What is the migration path from V1 (no Tenant support) to Tenant-mandatory? +5. Should the `editable` field concept from the Catalog Item Schema be formally incorporated into the Resource Type Specification? + +#### References +- SP Registration: https://github.com/dcm-project/enhancements/blob/main/enhancements/sp-registration-flow/sp-registration-flow.md +- Service Types: https://github.com/dcm-project/enhancements/blob/main/enhancements/service-type-definitions/service-type-definitions.md +- Health Check: https://github.com/dcm-project/enhancements/blob/main/enhancements/service-provider-health-check/service-provider-health-check.md +- Policy Engine: https://github.com/dcm-project/enhancements/blob/main/enhancements/policy-engine/policy-engine.md +- Catalog Item: https://github.com/dcm-project/enhancements/blob/main/enhancements/catalog-item-schema/catalog-item-schema.md +- KubeVirt SP: https://github.com/dcm-project/enhancements/blob/main/enhancements/kubevirt-sp/kubevirt-sp.md + +--- + +Copy the following template and fill in the fields: + +```markdown +### TOPIC-NNN โ€” Title + +**Area:** +**Priority:** +**Status:** ๐ŸŸก Active +**Raised:** + +#### Description + + + +#### Questions to Resolve + +1. + +#### References +- +``` + +--- + +*This document is maintained by the DCM Project team. Add topics freely โ€” no topic is too small if it needs a decision.* diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md index b0542f8..e7f59c9 100644 --- a/content/docs/architecture/ai-prompt.md +++ b/content/docs/architecture/ai-prompt.md @@ -4727,7 +4727,204 @@ AUTH โ†’ RFC 6749/7519/7662/OIDC/SCIM ยท CPX โ†’ FIPS 140/RFC 5280/8555/7030/889 --- -## SECTION 73 โ€” WORKING INSTRUCTIONS FOR AI MODELS +## SECTION 73 โ€” OPERATIONAL REFERENCE (doc 41 โ€” 41-operational-reference.md) + +> **Full specification:** [41-operational-reference.md](data-model/41-operational-reference.md) โ€” GitOps store partitioning, store migration playbook, disaster recovery scenarios, OPS-001โ€“007. + +### GitOps Store Partitioning (Section 1) +Three strategies: tenant-shard (hash(tenant_uuid) % N shards โ€” recommended), per-tenant (one repo per tenant โ€” MSP/strict isolation), time-based archiving (active vs cold archive repos). Triggers: >50k entities, clone time >PT30S, >500 tenants, repo >10GB. Layer Store partitioned by resource domain (Compute.*, Network.*, etc.). Shallow clones for read-only consumers; read mirrors for audit/drift. + +### Store Migration Playbook (Section 2) +5-phase pattern: Prepare โ†’ Backfill โ†’ Validate โ†’ Cutover โ†’ Decommission (after burn-in). Dual-write mode during migration. Audit chain must be unbroken across cutover (OPS-002). Source stays read-only for burn-in period โ€” DO NOT decommission early (OPS-003). Profile-governed burn-in: P7D(minimal) โ†’ P90D(fsi/sovereign). Rollback available during burn-in by re-pointing to source. Common paths: SQLiteโ†’PostgreSQL (evaluationโ†’standard), PostgreSQLโ†’CockroachDB (HA requirement), GitOps repo restructuring (monorepoโ†’shards via git filter-repo). + +### Disaster Recovery Scenarios (Section 3) +5 scenarios with RTO/RPO per profile: +- S1 Component failure: pod restart; RTO PT1M(sovereign)โ€“PT15M(minimal); RPO 0 (stateless) +- S2 Store failure: failover/restore; RTO PT5M(fsi)โ€“PT2H(minimal); RPO 0(GitOps)โ€“PT15M(Audit) +- S3 Full control plane loss: redeploy + store reconnect; RTO PT5M(fsi)โ€“PT30M(minimal); RPO 0 +- S4 Partial region loss: federation reroutes; sovereignty-scoped may block until region recovers +- S5 Repave (complete loss, Git intact): bootstrap from Git, restore operational stores from backup, rehydrate managed resources; RTO PT2H(fsi)โ€“PT24H(minimal) + +Post-recovery validation checklist (Section 3.6): /livez pass, /readyz pass, audit chain verify, store validate, drift scan, cert status, write post-incident audit record. OPS-005. + +### OPS-001โ€“007 System Policies +OPS-001: partitioning declared in deployment manifest. OPS-002: audit chain continuity across migration. OPS-003: source read-only during burn-in; do not decommission early. OPS-004: RTO must be met per profile. OPS-005: post-recovery checklist written to audit store. OPS-006: Audit Store minimum P365D retention all profiles. OPS-007: Git remotes must have push access from 2+ geographically separated locations. + +--- + +## SECTION 74 โ€” WEB INTERFACE SPECIFICATIONS (3 specs) + +> **Consumer GUI:** [dcm-consumer-gui-spec.md](specifications/dcm-consumer-gui-spec.md) โ€” Consumer Portal wrapping all 16 Consumer API sections, bounded by tenancy +> **Admin GUI:** [dcm-admin-gui-spec.md](specifications/dcm-admin-gui-spec.md) โ€” Admin Panel wrapping all Admin API sections, role-gated +> **Provider GUI:** [dcm-provider-gui-spec.md](specifications/dcm-provider-gui-spec.md) โ€” Provider management shell + 11 type-specific extension sets + +### Unified Shell Architecture (GUI-010) +ONE application, THREE role-gated surfaces: Consumer Portal (all actors) + Admin Panel (platform_admin/sre/auditor/security/policy_owner/finops) + Provider Management (provider_owner role). One login, one session token โ€” navigation adapts to highest privilege level. Flow GUI (policy authoring) is linked/embeddable within Admin Panel. + +### Consumer Portal (dcm-consumer-gui-spec.md โ€” 20 sections) +Tenancy: X-DCM-Tenant header; ContextSelector in masthead (hidden for single-tenant actors). +Navigation (PatternFly grouped left nav): Service Catalog | MY WORK (Requests, Resources, Dependency Groups) | Approvals [badge] | GOVERNANCE (Cost & Quota, Audit Trail, Contributions) | SETTINGS (Notifications, Sessions). **Hide, not disable** โ€” unavailable items hidden entirely. +Key capability โ€” Live Status (GUI-002): SSE stream (GET /api/v1/requests/{uuid}/stream); events: status_change, progress_updated, approval_required, approval_recorded, heartbeat; constituent_status array for compound/Meta Provider requests; fallback to polling. +Key capability โ€” Request form: rendered from catalog item schema; live cost estimate; scheduling section (at/window/recurring); dependency group linking with field injection declaration. +Key capability โ€” ITSM Bridge (GUI-014, section 8): ITSM references (ServiceNow/Jira change records, CMDB CIs) displayed on entity Overview tab; ITSM Notification Provider translates DCM events โ†’ ITSM records; ITSM systems call Admin API to record approval votes (DCM records decision, ITSM runs CAB process); CMDB sync is one-way DCMโ†’CMDB via Notification Provider subscription. +Key capability โ€” Consumer Audit Trail (section 11): own resource audit trail with Correlation ID trace through full pipeline; filterable by operation/type/date; export CSV. +Key capability โ€” Drift Report (section 6.3): cross-resource drift view with severity sorting; Revert All Critical bulk action; export drift report. +Security: MFA step-up inline (no page leave); strict CSP (connect-src self + DCM API origin only; no inline scripts). + +### Admin Panel (dcm-admin-gui-spec.md) +Platform health dashboard: component health grid, provider health summary, pending approvals count โ€” all from GET /api/v1/admin/health. +Tier registry editor: drag-and-drop reorder; impact report shows SECURITY_DEGRADATION(red)/BROKEN_REFERENCE(orange)/PROFILE_GAP(yellow)/SECURITY_UPGRADE(green); activate blocked until all blocking items resolved. +Scoring editor: visual slider with hard-stop โ€” auto_approve_below slider maximum is 50 (SMX-008/ATM-002); signal weights must sum to 100%. +Flow GUI accessible via link or embedded iframe for policy_owner/sre roles. + +### Provider Management (dcm-provider-gui-spec.md) +Common shell for ALL 11 provider types: overview, config, health history, audit trail, notifications. +Provider ownership: declared at registration (owner_team_uuid); provider_owner role scoped per provider UUID. +Service Provider extensions: capacity management (manual override for emergencies), managed entities with drift indicators, naturalization mapping viewer, test naturalization tool, interim status config. +Credential Provider extensions: inventory (metadata only โ€” NEVER values), rotation management, revocation registry search, external CA config (protocol, chain, CRL/OCSP status), algorithm compliance view (forbidden algorithms must be zero). +Auth Provider extensions: connection status with failover chain, SCIM sync status, session statistics by auth method. + +### Security Model (all GUI surfaces) +Navigation: hide not disable. Roles from session token; re-checked on token refresh. Tenancy: X-DCM-Tenant enforced UI + API (defense in depth). Step-up MFA: inline prompt without page navigation; cached PT10M. CSP: connect-src self + DCM API; no inline scripts; no eval(). + +### RHDH Integration (dcm-rhdh-integration-spec.md) +PRIMARY deployment model. 6 plugin packages as Dynamic Plugins (no RHDH rebuild): @dcm/backstage-plugin (frontend) ยท @dcm/backstage-plugin-backend (proxy + SSE relay + auth) ยท @dcm/backstage-plugin-catalog-backend (entity provider: DCMService + DCMResource) ยท @dcm/backstage-plugin-scaffolder-backend (dcm:request:submit, dcm:request:wait with live log, dcm:request:group, dcm:catalog:refresh) ยท @dcm/backstage-permission-policy (DCM roles โ†’ Backstage permissions) ยท @dcm/backstage-plugin-auth-backend (RHDH as DCM OIDC provider). +Auth: RHDH/Keycloak โ†’ OIDC token exchange โ†’ DCM session token (cached in RHDH backend by backstage user ref). Tenancy: RHDH Group context โ†’ X-DCM-Tenant header (via dcm-tenant-{uuid} group naming convention). +Entity model: DCMService kind (catalog items, namespace=dcm-catalog) auto-generated from DCM API every PT5M. DCMResource kind (realized resources, namespace=dcm-tenant-uuid) auto-synced from Realized State. Both search-indexed in RHDH. +Scaffolder = request form: each DCM catalog item generates one Backstage Software Template from its JSON Schema. Template wizard โ†’ dcm:request:submit โ†’ dcm:request:wait (live log: step N/M, constituent status) โ†’ dcm:catalog:refresh โ†’ entity appears in catalog. NO separate request form UI needed. +PatternFly nav: NavGroup(MY WORK > Requests/Resources/Groups) ยท NavItem with NotificationBadge (Approvals [count]) ยท NavGroup(GOVERNANCE > Cost&Quota/Contributions) ยท NavGroup(SETTINGS > Notifications/Sessions). Tenant via RHDH ContextSelector in masthead. +Pre-built RHDH value: RHSSO/Keycloak auth (no auth code) ยท RBAC plugin (no-code role management) ยท TechDocs (DCM docs in-portal) ยท ArgoCD plugin (layer store visibility) ยท AAP plugin (Ansible provider job status) ยท OCM plugin (cluster management alongside DCM). +Migration: Standalone SPA โ†’ RHDH via Dynamic Plugin loading only; no data migration, no API changes; 5-phase migration ending in standalone SPA decommission. + +--- + +## SECTION 75 โ€” ITSM INTEGRATION (doc 42 โ€” 42-itsm-integration.md) + +> **Full specification:** [42-itsm-integration.md](data-model/42-itsm-integration.md) โ€” 12th provider type, 8th policy type, 6 example policies, ITSM-001โ€“007, ITSM-POL-001โ€“004. + +### Design Principle +DCM replaces the infrastructure ticket as the provisioning mechanism. ITSM integration is ADDITIVE โ€” never required for DCM to function. Non-blocking by default. Organizations opt into blocking gates explicitly. + +### ITSM Provider (12th provider type) +Bidirectional: outbound (DCM events โ†’ ITSM records) + inbound (ITSM approvals โ†’ DCM votes). Implements full base Provider contract (PRV-001). Supported systems: ServiceNow, Jira Service Management, BMC Remedy/Helix, Freshservice, PagerDuty, Opsgenie, ManageEngine, Cherwell, TOPdesk, generic_rest. +Key capabilities: create/update/close change_request, create/update/close incident, create/update/retire cmdb_ci, create service_request, inbound_approval, inbound_request_initiation. +Credentials: auth via Credential Provider (ITSM-001). Inbound webhooks: HMAC-SHA256 verified (ITSM-003). + +### ITSM Action Policy (8th policy type) +Side-effect policy โ€” fires on DCM events, triggers ITSM action, does NOT block pipeline by default. +Output schema: itsm_provider_uuid, action, action_payload (template expressions: {{ field }}), store_reference_on_entity, block_until_created, block_timeout, on_failure. +ITSM-005: block_until_created REQUIRES block_timeout โ€” pipeline never permanently stalled. +ITSM-POL-002: NOT a GateKeeper substitute except via explicit block_until_created mechanism. +ITSM-POL-004: Multiple ITSM Policies on same event fire INDEPENDENTLY. + +### 6 Policy Examples +1. create_change_request on request.dispatched (ServiceNow) โ€” log and continue +2. block_until_created for PCI-scope tenants (compliance gate with PT30M timeout) +3. create_cmdb_ci on entity.realized โ€” with IP address from realized_fields +4. create_incident on drift.detected (significant/critical severity, Jira) +5. retire_cmdb_ci on entity.decommissioned +6. close_change_request on request.realized + +### ITSM Entity References +itsm_references[] on entity business data: system, record_type, record_id, record_url, status, last_synced_at. Preserved through entity lifecycle. In audit records. + +### recorded_via Field +Already on approval vote API: dcm_admin_ui | servicenow | jira | slack_bot | api_direct | other. ITSM Provider populates this on inbound approvals. + +### Event Catalog Additions +itsm.record_created, itsm.record_updated, itsm.record_failed โ€” new itsm.* domain (21st domain; 85 total events). + +### ITSM-001โ€“007 + ITSM-POL-001โ€“004 +ITSM-001: base Provider contract applies. ITSM-002: non-blocking by default. ITSM-003: inbound webhook HMAC auth. ITSM-004: references persist through lifecycle. ITSM-005: block_until_created mandatory timeout. ITSM-006: unmapped resource types silently skipped for CMDB sync. ITSM-007: missing template fields โ†’ warning + empty string (no block). + +--- + +## SECTION 76 โ€” PROVIDER CALLBACK AUTHENTICATION (doc 43 โ€” 43-provider-callback-auth.md) + +**Purpose:** Specifies how Service Providers authenticate inbound calls to the DCM control plane (the Provider Callback API). Resolves the gap between the outbound credential model (DCM โ†’ Provider) and the inbound model (Provider โ†’ DCM). + +**Two-layer model:** +- **Layer 1 โ€” mTLS:** Provider presents its registered certificate on every TLS connection. DCM validates the chain against the registered CA and the stored certificate fingerprint for that provider_uuid. Proves transport-level identity. +- **Layer 2 โ€” Provider Callback Credential:** A `dcm_interaction` credential issued by the Credential Provider at provider activation time. Scoped to `provider_uuid` + `allowed_operations`. Short-lived (PT15M fsi/sovereign; PT1H standard). Presented as `Authorization: Bearer` on every callback call. Proves operation-level authorization. +- Both layers are required. mTLS alone does not prove authorization. The credential alone cannot establish the connection. + +**Entity-level authorization (per call, independent of credential):** +- `realized_state_push`: DCM verifies `credential.provider_uuid` matches the `provider_uuid` in the Requested State record for that `resource_id`. Prevents a provider from pushing state for entities it was not dispatched to. +- `update_notification`: DCM verifies the calling provider is the current Realized State provider for the entity AND the `notification_type` was declared in the provider's registration. +- `lifecycle_event`: DCM verifies the calling provider is the provider on record for the resource. + +**Credential lifecycle:** +- Issued at provider activation; delivered via the activation response (retrieved via Credential Provider) +- Rotated automatically by DCM before expiry (DCM initiates rotation; provider must implement refresh) +- Transition window (50% of credential lifetime) during which both old and new credentials are accepted +- Revoked immediately on: provider deregistration, 5+ scope violations in PT1H, admin explicit revocation, certificate expiry without rotation + +**Registration special case:** +- Initial registration (`POST /api/v1/providers`) uses a registration token (single-use, admin-issued) not the callback credential โ€” no callback credential exists until activation +- Re-registration (same name, updating version/capabilities) uses the active callback credential +- Re-registration that changes sovereignty declaration requires a new registration token and triggers a new approval pipeline + +**System policies:** PCA-001 through PCA-010. Key: PCA-003 (entity-level ownership check is independent of credential), PCA-004 (5 scope violations โ†’ auto-suspend), PCA-010 (all inbound calls produce audit records including rejected ones โ€” no silent failures). + + +## SECTION 77 โ€” AEP API ALIGNMENT + +DCM's four OpenAPI specifications follow AEP (API Enhancement Proposals โ€” aep.dev) conventions in three specific areas: + +**1. Custom methods (AEP-136):** Actions on resources use colon syntax. `POST /resources/{id}:suspend` not `POST /resources/{id}/suspend`. Applies to all state-transition and action operations in Consumer and Admin APIs (30 path conversions total: :suspend, :resume, :rehydrate, :rotate, :extend-ttl, :transfer, :bulk-decommission, :acknowledge, :revert, :approve, :reject, :reinstate, :trigger, :rebuild, :activate, :vote, :revoke-sessions, :accept-degradation, :rotate-credential, :read-all, and others). + +**2. Long-Running Operations (AEP-151):** Async operations that produce a trackable result return an `Operation` resource (not `202 Accepted` with no body). The `Operation` has: `name` (stable poll URL), `done` (boolean), `metadata` (stage/progress/resource_uuid), `response` (present when done=true, success), `error` (present when done=true, failure). 14 consumer-facing operations return Operation: submitRequest, createRequestGroup, updateResource, decommissionResource, suspendResource, resumeResource, rehydrateResource, initiateOwnershipTransfer, bulkDecommission, revertDrift, rotateCredential, contributePolicy, contributeResourceGroup (consumer API) + decommissionTenant (admin API). Fire-and-forget operations (capacity reports, interim status, lifecycle events, discovery triggers) retain `202 Accepted` โ€” no Operation body. + +**3. Pagination (AEP-158):** `page_size` and `page_token` query parameters (not `limit`/`cursor`). Responses include `next_page_token`. + +**Deliberately NOT aligned:** Resource names (DCM retains bare UUIDs โ€” immutability across ownership transfers is more important than AEP naming), timestamp field names (`created_at` not `create_time` โ€” would require data model change with no functional benefit). + +**Operator API note:** The operator-facing Services API (dcm-operator-api.yaml) uses the callback pattern โ€” operators are NOT consumer-facing LRO callers. `updateResource` in the operator spec uses `202 Accepted` (async with callback) + `200 OK` (sync with RealizedStatePayload), not an Operation resource. + +## SECTION 78 โ€” KESSEL INTEGRATION EVALUATION (doc 44 โ€” 44-kessel-integration-evaluation.md) + +**Status:** Pre-implementation evaluation. Discussion with Kessel team required before any implementation. No DCM architecture changes should be made based on this document until alignment is confirmed. + +**What Kessel is:** +- **Kessel Relations:** Authorization service built on SpiceDB (Google Zanzibar / ReBAC). `CheckPermission(subject, permission, resource)` traverses relationship graph. Zookie consistency tokens for "read your own writes" guarantee. +- **Kessel Asset Inventory:** Hybrid cloud resource state tracking (current-state snapshot store, gRPC streaming API). Integrates with Kessel Relations for auth-filtered inventory queries. + +**Integration Option A โ€” Kessel Relations as DCM Auth Provider:** +- Handles checks 1 and 2 of DCM's five-check boundary model (identity + authorization). Checks 3-5 (accreditation, data matrix, sovereignty) remain in DCM's Policy Engine โ€” cannot be delegated. +- DCM's entity relationship graph (operational: requires/constituent/shareable) must NOT go in Kessel Relations โ€” only access-control relationships. +- Approval gate quorum stays in DCM (Kessel answers "is actor authorized to vote?"; DCM tracks "how many have voted"). +- Integration path: new `auth_mode: kessel_rebac` Auth Provider registration. +- Zookie tokens must be threaded through DCM request context for consistency. + +**Integration Option B โ€” Kessel Inventory as Discovered State Storage Provider:** +- Discovered State only (most ephemeral store, current-state snapshot). Intent, Requested, Realized stores remain in DCM โ€” not replaceable by Kessel Inventory. +- Drift detection logic stays in DCM's DRC component regardless โ€” Kessel Inventory is a data source, not a drift engine. +- Field-level provenance, lifecycle state machine, audit chain โ€” all stay in DCM. +- Integration path: Storage Provider with `storage_sub_type: snapshot_store` backed by Kessel Inventory. No data model changes. + +**Pros (Relations):** SpiceDB production-grade, Zanzibar consistency, scalable graph traversal, shared source of truth across Red Hat products, single CheckPermission call replaces multi-step group-lookup + policy-evaluation. + +**Pros (Inventory):** Feeds same discovered state to ACM/Insights/HCC, reduces DCM's operational burden for ephemeral store, gRPC streaming fits provider push pattern. + +**Cons (Relations):** Hard runtime dependency (unavailability = safe-deny mode), schema coupling requires coordinated evolution, quorum model doesn't fit, entity relationship graph stays in DCM anyway. + +**Cons (Inventory):** Four-state model mismatch (Kessel is upsert/current-state only), no field-level provenance, no drift detection, schema extensibility for DCM-specific types unconfirmed, project maturity risk. + +**18 questions for Kessel team documented in Section 6. 10 blocking items in Section 10.** + +**KESSEL-001 through KESSEL-007** are the proposed system policies โ€” not active until Kessel alignment is complete. +## SECTION 79 โ€” CONSISTENCY REVIEW (doc 45 โ€” 45-consistency-review.md) + +Full consistency review completed 2026-03. Key findings and fixes: + +**Fixed:** AEP colon paths applied to admin-api-spec.md (13 paths) and consumer-api-spec.md (4 remaining); stale entity_type values corrected (`allocated_resource`โ†’`infrastructure_resource`, `resource_entity`โ†’`infrastructure_resource`); stale scoring threshold keys replaced with named-tier format comments; `provider_id`โ†’`provider_uuid` in DCM API endpoint paths; Operation (LRO) polling section added to consumer spec. + +**Canonical rules:** entity_type has exactly three values (infrastructure_resource, composite_resource, process_resource). Custom method paths use colon syntax in ALL docs. `provider_uuid` in DCM APIs; `resource_id` (operator-assigned) in callback APIs โ€” these are intentionally distinct. Lifecycle state UPPERCASE in YAML, lowercase in prose โ€” by design. + +**Three implementation decisions still needed:** (1) resource_type field: accept FQN string or require UUID at dispatch? (2) Operation polling: same endpoint as request status or separate? (3) API Gateway must map resource_id โ†’ entity_uuid at callback boundary. + + +## SECTION 80 โ€” WORKING INSTRUCTIONS FOR AI MODELS When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure): @@ -4769,6 +4966,11 @@ When working on this project, apply these instructions in addition to the number 210. **Live updates: SSE stream + OIS interim status** โ€” GET /api/v1/requests/{uuid}/stream (text/event-stream, closes on terminal status) for browser/CLI without polling; events: status_change, progress_updated, approval_required, approval_recorded, heartbeat(30s); OIS providers POST /api/v1/provider/entities/{uuid}/status for interim progress with step_current/step_total/constituent_status; request.progress_updated event added to doc 33; rate-limited: max 1 interim status per 10s per entity 211. **Profile coverage added to docs 36-39** โ€” doc36 cert lifetime table(P180D minimalโ†’P14D sovereign), algorithm min table; doc37 max scheduling horizon(P365D minimalโ†’P7D sovereign), concurrent scheduled limit, maintenance window approval tier; doc38 max group size(100 minimalโ†’5 sovereign), group timeout max, field injection validation strictness, nesting depth; doc39 metrics scraping restrictions(sovereign internal only), /api/v1/admin/health MFA requirements(fsi/sovereign) 212. **40-standards-catalog.md is the authoritative standards reference** โ€” 19 RFCs, 3 cryptographic standards tables (permitted algorithms, forbidden algorithms, FIPS levels), 6 compliance frameworks, 7 CNCF ecosystem projects, W3C SSE, OpenAPI 3.1, SPIFFE (informative), HashiCorp Vault PKI / Venafi / EJBCA as External CA Credential Provider backends (NOT Auth Providers); Section 9 maps all 17 policy families to their standards basis; usage map tracks which standards appear in which documents +213. **41-operational-reference.md covers the three operational gaps** โ€” GitOps partitioning (3 strategies; tenant-shard recommended; trigger thresholds table), store migration (5-phase dual-write playbook; never decommission source before burn-in; audit chain continuity required OPS-002), DR (5 scenarios; post-recovery validation checklist mandatory OPS-005; Audit Store minimum P365D retention all profiles OPS-006); RTO: PT1M sovereign component โ†’ PT24H minimal repave +214. **Three GUI specs define the unified DCM web application** โ€” one application with role-gated surfaces: Consumer Portal (all actors โ€” catalog, SSE live status with constituent tracking, resources, approvals, cost, sessions), Admin Panel (platform roles โ€” health dashboard, tier registry drag-drop editor with hard-stops, scoring slider max 50), Provider Management (provider_owner role โ€” 11 provider types with common shell + type-specific tabs; Credential Provider shows metadata never values; algorithm compliance must show zero forbidden algorithm violations) +215. **Doc 42 adds 12th provider type (ITSM Provider) and 8th policy type (ITSM Action Policy)** โ€” ITSM is ADDITIVE (DCM never requires it); ITSM Action Policy is side-effect only (non-blocking default); block_until_created requires block_timeout (never permanently stalls pipeline, ITSM-005); recorded_via field already existed on approval vote โ€” ITSM inbound approvals use it; 6 policy examples covering ServiceNow + Jira for provisioning, CMDB sync, incident on drift, decommission; itsm.* event domain adds 3 events (85 total, 21 domains) +215. **Consumer GUI ITSM bridge (GUI-011, section 8 of dcm-consumer-gui-spec.md)** โ€” ITSM is a CONSUMER of DCM events, not a source of truth; DCM is the system of record; CMDB sync is one-way DCMโ†’CMDB via Notification Provider subscription to entity.* events; ITSM approval votes call POST /api/v1/admin/approvals/{uuid}/vote โ€” the CAB process happens in ITSM, DCM just records the outcome; ITSM references stored as business data fields on entities; Audit Trail (section 11) is consumer-scoped own-resource view; cross-tenant audit is Admin Panel only +215. **RHDH/Backstage is the PRIMARY consumer GUI deployment model** โ€” 6 Dynamic Plugin packages (no RHDH rebuild); Scaffolder IS the request form (auto-generated templates from catalog item JSON Schema); DCMService + DCMResource entity kinds sync to RHDH catalog every PT5M; tenancy via RHDH Group context โ†’ X-DCM-Tenant; OIDC token exchange for auth delegation; PatternFly NavGroup/NavItem/NotificationBadge for sidebar; Approvals shows live pending count badge; all compliance enforced by DCM control plane โ€” RHDH is a client 195. **33-event-catalog.md is the SINGLE authoritative source for all DCM event types** โ€” 82 events across 26 domains; all events share the base envelope (event_uuid, event_type, event_schema_version, timestamp from Commit Log, urgency, payload, links); consumers implement idempotency using event_uuid; critical urgency events are non-suppressable; non-standard events use reverse-DNS prefix; event_schema_version only increments on breaking changes 194. **Tier registry changes are gated by impact detection** โ€” any change that creates a SECURITY_DEGRADATION (tier gravity or position decreased) blocks activation until each degradation is explicitly accepted by a verified-tier or above reviewer via Admin API; BROKEN_REFERENCE also blocks; PROFILE_GAP is a warning that does not block; all changes produce an impact report in the Audit Store (ATM-009โ€“012) 193. **Authority tiers are named positions in an ordered list โ€” not fixed enum values** โ€” tier weight derived from list position at evaluation time; organizations insert custom tiers between existing ones without breaking existing name references; 'authorized' tier always means 'highest current gravity' regardless of what's been inserted before it; ATM-001: never hardcode tier weights diff --git a/content/docs/architecture/data-model/event-catalog.md b/content/docs/architecture/data-model/event-catalog.md index 62310f3..7283dd7 100644 --- a/content/docs/architecture/data-model/event-catalog.md +++ b/content/docs/architecture/data-model/event-catalog.md @@ -688,6 +688,42 @@ payload: --- +## 21. ITSM Events (`itsm.*`) + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `itsm.record_created` | info | ITSM Provider successfully created a record in the external ITSM system | +| `itsm.record_updated` | info | ITSM Provider successfully updated an existing ITSM record | +| `itsm.record_failed` | medium | ITSM Provider failed to create/update a record; or `block_until_created` timeout reached | + +### 21.1 Payload Schema + +#### `itsm.record_created` / `itsm.record_updated` +```yaml +payload: + itsm_provider_uuid: + itsm_system: servicenow | jira_service_management | ... + action: create_change_request | create_incident | ... + record_type: change_request | incident | cmdb_ci | service_request + record_id: "" # e.g. CHG0012345, INC-4821 + record_url: "" # deep link to record in ITSM system + policy_handle: "" # which ITSM Policy triggered this + stored_on_entity: + +``` + +#### `itsm.record_failed` +```yaml +payload: + itsm_provider_uuid: + action: + failure_reason: + timeout_expired: # true if block_until_created timeout hit + policy_handle: +``` + +--- + ## 19. Event Type Quick Reference ``` @@ -743,7 +779,7 @@ federation.tunnel_degraded auth.provider_failover ``` -**Total: 82 event types across 20 domains** +**Total: 85 event types across 21 domains** --- diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md index 26da607..d543339 100644 --- a/content/docs/architecture/data-model/foundations.md +++ b/content/docs/architecture/data-model/foundations.md @@ -153,6 +153,7 @@ Data fields are assembled from multiple contributing layers in a deterministic p | **Message Bus Provider** | Async event streaming | DCM publishes/subscribes โ†” Provider | | **Registry Provider** | Serves the resource type registry | DCM pulls โ†’ Provider serves | | **Peer DCM** | Another DCM instance (federation) | DCM โ†” DCM via federation tunnel | +| **ITSM Provider** | Bidirectional integration with ITSM systems (ServiceNow, Jira, Remedy, etc.); creates/updates ITSM records from DCM events; routes ITSM approvals back to DCM | DCM โ†’ ITSM (outbound) / ITSM โ†’ DCM (inbound) | **The unified Provider base contract** is defined in [A-provider-contract.md](A-provider-contract.md). All eleven Provider types implement this base contract. What varies is the capability declaration โ€” what operations the Provider exposes and what data flows in which direction. diff --git a/content/docs/architecture/data-model/itsm-integration.md b/content/docs/architecture/data-model/itsm-integration.md new file mode 100644 index 0000000..d7f1419 --- /dev/null +++ b/content/docs/architecture/data-model/itsm-integration.md @@ -0,0 +1,670 @@ +--- +title: "ITSM Integration" +type: docs +weight: 42 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Architecture Reference โ€” ITSM Provider Type and ITSM Policy Type +**Related Documents:** [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Notification Model](23-notification-model.md) | [Event Catalog](33-event-catalog.md) | [Authority Tier Model](32-authority-tier-model.md) | [Consumer API Specification](../specifications/consumer-api-spec.md) + +> **Design principle:** DCM is built to *replace* the infrastructure ticket as the primary provisioning mechanism. ITSM integration is additive โ€” it enriches DCM entities with ITSM metadata, enables ITSM-initiated requests, and provides bidirectional lifecycle traceability for organizations that need it for compliance. **DCM never requires an ITSM system to function.** +> +> Two new additions to the DCM architecture: +> 1. **ITSM Provider** โ€” a new Provider type (12th) that speaks ITSM system APIs bidirectionally +> 2. **ITSM Policy** โ€” a new Policy output type (8th) that triggers ITSM actions as a side-effect of DCM pipeline events + +--- + +## 1. ITSM Provider + +### 1.1 What an ITSM Provider Is + +An ITSM Provider is a DCM Provider that connects DCM to an external IT Service Management system. It handles: + +- **Outbound**: DCM lifecycle events โ†’ ITSM records (create change requests, update CMDB CIs, close incidents, link tickets to entities) +- **Inbound**: ITSM approvals and decisions โ†’ DCM (change approval recorded via approval vote API, request initiation from ITSM workflow) +- **Sync**: ITSM record references stored on DCM entities as business data (bidirectional link) + +The ITSM Provider is **not** a Service Provider (it doesn't realize resources), **not** a Notification Provider (though it may create notification-like records), and **not** a Policy Provider (though ITSM approval status may inform DCM policies). It is its own type because it has a bidirectional contract, manages external record lifecycle, and requires specific capability declarations around ITSM system connectivity. + +### 1.2 Data Flow + +``` +DCM lifecycle event fires (e.g. request.dispatched) + โ”‚ + โ–ผ ITSM Policy evaluates (see Section 3) + โ”‚ Determines: should an ITSM action fire? Which action? + โ”‚ + โ–ผ ITSM Provider receives action request + โ”‚ Translates to target system's API format + โ”‚ Calls ITSM system (ServiceNow, Jira, etc.) + โ”‚ + โ–ผ ITSM system creates/updates record + โ”‚ Returns record ID (CHG0012345, INC-4821, etc.) + โ”‚ + โ–ผ ITSM Provider stores reference on DCM entity + โ”‚ entity.business_data.itsm_references[] updated + โ”‚ + โ–ผ ITSM Provider reports back to DCM + itsm_reference_created event published + External record ID in audit record + +โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + +ITSM system approves a change record + โ”‚ + โ–ผ ITSM system calls DCM API (via webhook or polling) + โ”‚ POST /api/v1/admin/approvals/{uuid}/vote + โ”‚ { decision: "approve", recorded_via: "servicenow", + โ”‚ external_reference: "CHG0012345" } + โ”‚ + โ–ผ DCM records approval vote + โ”‚ approval.decision_recorded event + โ”‚ + โ–ผ Pipeline resumes if quorum/tier satisfied +``` + +### 1.3 Capability Declaration + +```yaml +itsm_provider_capabilities: + itsm_system: servicenow | jira_service_management | bmc_remedy | bmc_helix | + freshservice | zendesk | pagerduty | opsgenie | manageengine | + cherwell | topdesk | generic_rest + + # What this provider can do + supported_actions: + - create_change_request # create a change record for DCM provisioning events + - update_change_request # update change record on state transitions + - close_change_request # close change record on realization/failure + - create_incident # create incident for failures, drift, security events + - update_incident # update incident on resolution + - close_incident # close incident on recovery + - update_cmdb_ci # update CMDB configuration item record + - create_cmdb_ci # create new CMDB CI for realized entities + - retire_cmdb_ci # retire CMDB CI on decommission + - create_service_request # create service request record + - link_parent_record # link DCM entity to existing ITSM record + - inbound_approval # accept approval decisions from ITSM system + - inbound_request_initiation # allow ITSM workflows to submit DCM requests + + # System connectivity + endpoint_url: # ITSM system API base URL + api_version: # system-specific API version + auth_credential_uuid: # references Credential Provider + + # Bidirectional webhook (for inbound) + inbound_webhook: + enabled: + secret_credential_uuid: # HMAC secret for webhook verification + + # Field mappings (system-specific) + field_mapping_ref: # path to field mapping YAML in Layer Store + + # CMDB CI type mapping + cmdb_ci_type_map: + - dcm_resource_type: Compute.VirtualMachine + itsm_ci_type: cmdb_ci_server # ServiceNow CI class + - dcm_resource_type: Network.VLAN + itsm_ci_type: cmdb_ci_network_gear + - dcm_resource_type: Storage.Volume + itsm_ci_type: cmdb_ci_storage_device +``` + +### 1.4 Required API Endpoints (ITSM Provider implements) + +``` +POST {provider_base}/actions # DCM submits action requests +GET {provider_base}/actions/{action_id} # DCM checks action status +GET {provider_base}/records/{record_id} # DCM retrieves record status +POST {provider_base}/inbound # ITSM system sends inbound events +GET /health # standard OIS health check +``` + +### 1.5 DCM Entity ITSM References + +Realized entities gain an `itsm_references` block in business data: + +```yaml +itsm_references: + - system: servicenow + provider_uuid: + record_type: change_request + record_id: "CHG0012345" + record_url: "https://corp.service-now.com/nav_to.do?uri=change_request.do?sys_id=..." + created_at: + status: approved # DCM's view of the record status + last_synced_at: + + - system: jira_service_management + provider_uuid: + record_type: incident + record_id: "INC-4821" + record_url: "https://corp.atlassian.net/browse/INC-4821" + created_at: + status: open + last_synced_at: +``` + +--- + +## 2. Supported ITSM Systems + +### 2.1 ServiceNow + +**API:** REST Table API (`/api/now/table/`), Business Rule webhooks, Flow Designer + +```yaml +# ServiceNow ITSM Provider registration +itsm_provider_registration: + provider_handle: "servicenow-prod" + itsm_system: servicenow + endpoint_url: "https://corp.service-now.com" + api_version: "v2" + auth_credential_uuid: # api_key or oauth2 credential + + supported_actions: + - create_change_request # โ†’ change_request table + - update_change_request + - close_change_request + - create_incident # โ†’ incident table + - update_cmdb_ci # โ†’ cmdb_ci_server (or mapped class) + - create_cmdb_ci + - retire_cmdb_ci + - inbound_approval # Change Advisory Board approval โ†’ DCM vote + + # ServiceNow-specific field mapping + change_request_template: + assignment_group: "Infrastructure Automation" + category: "Software" + risk: "2" # Low + impact: "3" # Low + # DCM fields injected at runtime: + short_description: "DCM: Provision {resource_type} '{entity_handle}'" + description: "Requested by: {actor_handle}\nTenant: {tenant_handle}\nDCM Request: {request_uuid}" + + # CAB approval โ†’ DCM vote mapping + inbound_approval: + webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote" + trigger_on: "change_request.state โ†’ 'Approved'" + decision_field: "state" + decision_map: + "Approved": "approve" + "Rejected": "reject" + "Cancelled": "reject" + external_reference_field: "number" # โ†’ CHG0012345 + + cmdb_ci_type_map: + - dcm_resource_type: Compute.VirtualMachine + itsm_ci_type: cmdb_ci_server + - dcm_resource_type: Network.VLAN + itsm_ci_type: cmdb_ci_netgear + - dcm_resource_type: Storage.Volume + itsm_ci_type: cmdb_ci_disk + - dcm_resource_type: Kubernetes.Cluster + itsm_ci_type: cmdb_ci_kubernetes_cluster +``` + +**Inbound: CAB approval flow** + +``` +ServiceNow Change Advisory Board approves CHG0012345 + โ”‚ + โ–ผ ServiceNow Business Rule fires on state change โ†’ "Approved" + โ”‚ Calls DCM webhook: POST /api/v1/admin/approvals/{uuid}/vote + โ”‚ Headers: X-ServiceNow-Signature: + โ”‚ Body: { decision: "approve", recorded_via: "servicenow", + โ”‚ external_reference: "CHG0012345" } + โ”‚ + โ–ผ DCM verifies HMAC signature against secret_credential_uuid + โ”‚ Records approval vote + โ”‚ Pipeline resumes if tier satisfied +``` + +### 2.2 Jira Service Management (Atlassian) + +**API:** REST API v3, Atlassian Connect webhooks, Automation rules + +```yaml +itsm_provider_registration: + provider_handle: "jira-service-mgmt-prod" + itsm_system: jira_service_management + endpoint_url: "https://corp.atlassian.net" + api_version: "3" + auth_credential_uuid: # API token or OAuth2 + + supported_actions: + - create_change_request # โ†’ Jira issue (Change type) + - update_change_request + - close_change_request + - create_incident # โ†’ Jira issue (Incident type) + - create_service_request # โ†’ Jira issue (Service Request type) + - inbound_approval # Jira Change approval โ†’ DCM vote + + change_request_template: + project_key: "OPS" + issue_type: "Change" + summary: "DCM: {resource_type} '{entity_handle}'" + description: | + *Requested by:* {actor_handle} + *Tenant:* {tenant_handle} + *DCM Request UUID:* {request_uuid} + *Catalog Item:* {catalog_item_handle} + priority: "Medium" + labels: ["dcm-automated", "{tenant_handle}"] + + inbound_approval: + webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote" + trigger_on: "issue.status โ†’ 'Approved'" + decision_map: + "Approved": "approve" + "Declined": "reject" + external_reference_field: "key" # โ†’ OPS-4821 +``` + +### 2.3 BMC Remedy / Helix ITSM + +**API:** REST API (Remedy AR System REST), webhook callbacks + +```yaml +itsm_provider_registration: + provider_handle: "bmc-helix-prod" + itsm_system: bmc_helix + endpoint_url: "https://remedy.corp.example.com/api/arsys/v1" + api_version: "v1" + + supported_actions: + - create_change_request # โ†’ CHG:Infrastructure Change + - update_change_request + - close_change_request + - create_incident # โ†’ HPD:Help Desk + - update_cmdb_ci # โ†’ AST:Config Item + - inbound_approval + + change_request_template: + form: "CHG:Infrastructure Change" + Location_Company: "{tenant_handle}" + Summary: "DCM: Provision {resource_type} '{entity_handle}'" + Categorization_Tier_1: "Infrastructure" + Categorization_Tier_2: "Provisioning" + Change_Type: "Normal" +``` + +### 2.4 Freshservice + +```yaml +itsm_provider_registration: + provider_handle: "freshservice-prod" + itsm_system: freshservice + endpoint_url: "https://corp.freshservice.com/api/v2" + + supported_actions: + - create_change_request + - update_change_request + - close_change_request + - create_incident + - create_service_request + + change_request_template: + type: "Normal" + risk: "Low" + impact: "Low" + subject: "DCM: {resource_type} '{entity_handle}'" + description: "Tenant: {tenant_handle} | Actor: {actor_handle} | Request: {request_uuid}" + group_id: +``` + +### 2.5 PagerDuty (Incident Management) + +```yaml +itsm_provider_registration: + provider_handle: "pagerduty-prod" + itsm_system: pagerduty + endpoint_url: "https://api.pagerduty.com" + + supported_actions: + - create_incident # for DCM failures, drift, security events + - update_incident + - close_incident + + # PagerDuty Events API v2 + incident_template: + service_id: + escalation_policy_id: + payload: + summary: "DCM {event_type}: {entity_handle}" + severity: "{{ drift_severity | map: criticalโ†’critical, significantโ†’error, moderateโ†’warning, minorโ†’info }}" + source: "dcm" + custom_details: + entity_uuid: "{entity_uuid}" + tenant: "{tenant_handle}" + dcm_event: "{event_type}" +``` + +### 2.6 Generic REST (Custom ITSM) + +For ITSM systems not natively supported, the `generic_rest` type allows template-based HTTP calls: + +```yaml +itsm_provider_registration: + provider_handle: "custom-itsm-prod" + itsm_system: generic_rest + endpoint_url: "https://itsm.corp.example.com/api" + + action_templates: + - action: create_change_request + method: POST + path: "/changes" + headers: + Content-Type: "application/json" + X-API-Key: "{{ credential_value }}" + body_template: | + { + "title": "DCM: {{ resource_type }} '{{ entity_handle }}'", + "requested_by": "{{ actor_handle }}", + "category": "Infrastructure", + "external_id": "{{ request_uuid }}" + } + response_id_path: "$.id" # JSONPath to extract record ID from response + + - action: inbound_approval + inbound_field: "status" + decision_map: + "approved": "approve" + "rejected": "reject" +``` + +--- + +## 3. ITSM Policy Type + +### 3.1 What an ITSM Policy Is + +An **ITSM Policy** is a new DCM Policy output type (8th, alongside GateKeeper, Validation, Transformation, Recovery, Orchestration Flow, Governance Matrix Rule, and Lifecycle Policy). + +It fires as a **side-effect policy** โ€” it does not block pipeline execution (it is not a GateKeeper) and does not transform the payload. It fires on a DCM event and triggers an ITSM action via a registered ITSM Provider. The pipeline continues whether or not the ITSM action succeeds; ITSM failures are logged and alerted but do not block DCM operations. + +**Key distinction:** An ITSM Policy is about *record-keeping and integration* with external governance systems. A GateKeeper Policy is about *allowing or blocking* operations. These are complementary, not competing. + +### 3.2 Output Schema + +```yaml +# ITSM Policy output schema +itsm_policy_output: + type: itsm_action # new output type identifier + + # Required + itsm_provider_uuid: # which ITSM Provider to call + action: create_change_request | update_change_request | close_change_request | + create_incident | update_incident | close_incident | + update_cmdb_ci | create_cmdb_ci | retire_cmdb_ci | + create_service_request | link_parent_record + + # Payload โ€” fields to pass to ITSM Provider + # Supports template variables from the triggering event payload + action_payload: + : + + # How to handle ITSM failure + on_failure: log_and_continue | alert_and_continue | alert_only + + # Store the ITSM record reference on the DCM entity (optional) + store_reference_on_entity: + reference_label: # human-readable label for the reference + + # Require ITSM record creation before dispatch (optional โ€” see note) + block_until_created: # default: false + block_timeout: # max wait if block_until_created: true +``` + +> **`block_until_created`:** When `true`, the ITSM Policy behaves like a pre-dispatch gate โ€” DCM waits for the ITSM record to be created before dispatching to the Service Provider. This is used when organizational policy requires a change record to exist before any provisioning begins. When `false` (default), the ITSM record is created in parallel with or after dispatch โ€” suitable for notification-only use cases. + +### 3.3 Example Policies + +#### Policy 1: Create Change Request on Dispatch (ServiceNow) + +```yaml +policy_handle: "create-change-on-dispatch" +policy_type: itsm_action +enforcement_level: soft +status: active + +match: + payload_type: request.dispatched + conditions: + - field: resource_type + operator: in + value: [Compute.VirtualMachine, Storage.Volume, Network.VLAN] + +output: + type: itsm_action + itsm_provider_uuid: + action: create_change_request + action_payload: + short_description: "DCM: Provision {{ resource_type }} '{{ entity_handle }}'" + description: | + Automated provisioning via DCM. + Request UUID: {{ request_uuid }} + Actor: {{ actor_handle }} + Tenant: {{ tenant_handle }} + Catalog Item: {{ catalog_item_handle }} + risk: "{{ risk_score | map: <25โ†’'Low', <60โ†’'Medium', elseโ†’'High' }}" + store_reference_on_entity: true + reference_label: "Change Request" + on_failure: alert_and_continue +``` + +#### Policy 2: Block Dispatch Until Change Record Exists (Compliance Gate) + +```yaml +policy_handle: "require-change-record-before-dispatch" +policy_type: itsm_action +enforcement_level: hard +status: active + +match: + payload_type: request.layers_assembled + conditions: + - field: tenant_handle + operator: in + value: [payments-team, pci-scope-team] + +output: + type: itsm_action + itsm_provider_uuid: + action: create_change_request + action_payload: + short_description: "DCM: {{ resource_type }} provision โ€” {{ tenant_handle }}" + change_type: "Normal" + assignment_group: "Change Advisory Board" + store_reference_on_entity: true + reference_label: "Change Request (PCI Scope)" + block_until_created: true + block_timeout: PT30M + on_failure: alert_and_continue +``` + +#### Policy 3: Update CMDB on Realization + +```yaml +policy_handle: "sync-cmdb-on-realization" +policy_type: itsm_action +status: active + +match: + payload_type: entity.realized + conditions: + - field: resource_type + operator: in + value: [Compute.VirtualMachine, Compute.BareMetalServer] + +output: + type: itsm_action + itsm_provider_uuid: + action: create_cmdb_ci + action_payload: + name: "{{ entity_handle }}" + ip_address: "{{ realized_fields.primary_ip }}" + os: "{{ realized_fields.os_family }}" + managed_by: "DCM" + environment: "{{ tenant_handle }}" + correlation_id: "{{ entity_uuid }}" + store_reference_on_entity: true + reference_label: "CMDB CI" + on_failure: alert_and_continue +``` + +#### Policy 4: Create Incident on Drift (Jira) + +```yaml +policy_handle: "create-incident-on-critical-drift" +policy_type: itsm_action +status: active + +match: + payload_type: drift.detected + conditions: + - field: drift_severity + operator: in + value: [significant, critical] + +output: + type: itsm_action + itsm_provider_uuid: + action: create_incident + action_payload: + summary: "DCM Drift: {{ entity_handle }} โ€” {{ drift_severity }}" + description: | + DCM has detected significant configuration drift. + Entity: {{ entity_handle }} ({{ entity_uuid }}) + Severity: {{ drift_severity }} + Drifted fields: {{ drifted_fields | count }} fields + Detected at: {{ discovered_at }} + View in DCM: https://dcm.corp/resources/{{ entity_uuid }}/drift + priority: "{{ drift_severity | map: criticalโ†’'Highest', significantโ†’'High' }}" + labels: ["dcm-drift", "{{ resource_type | slugify }}"] + store_reference_on_entity: true + reference_label: "Drift Incident" + on_failure: log_and_continue +``` + +#### Policy 5: Retire CMDB CI on Decommission + +```yaml +policy_handle: "retire-cmdb-on-decommission" +policy_type: itsm_action +status: active + +match: + payload_type: entity.decommissioned + +output: + type: itsm_action + itsm_provider_uuid: + action: retire_cmdb_ci + action_payload: + correlation_id: "{{ entity_uuid }}" # find CI by DCM entity UUID + install_status: "7" # ServiceNow: Retired + retired_at: "{{ event_timestamp }}" + decommission_reason: "DCM decommission โ€” {{ actor_handle }}" + on_failure: alert_and_continue +``` + +#### Policy 6: Close Change Record on Completion + +```yaml +policy_handle: "close-change-on-completion" +policy_type: itsm_action +status: active + +match: + payload_type: request.realized + conditions: + - field: entity.itsm_references[?(@.record_type=='change_request')].record_id + operator: exists + +output: + type: itsm_action + itsm_provider_uuid: + action: close_change_request + action_payload: + state: "3" # ServiceNow: Closed + close_code: "Successful" + close_notes: "Provisioning completed successfully by DCM. Entity: {{ entity_uuid }}" + on_failure: log_and_continue +``` + +--- + +## 4. ITSM Provider System Policies + +| Policy | Rule | +|--------|------| +| `ITSM-001` | ITSM Providers implement the base Provider contract (PRV-001) including registration, health check, sovereignty declaration, and zero trust authentication. ITSM system connectivity credentials must reference a registered Credential Provider โ€” no plaintext credentials in provider registration. | +| `ITSM-002` | DCM does not require ITSM integration to function. ITSM Policies with `on_failure: alert_and_continue` (the default) never block DCM pipeline execution. Organizations must explicitly set `block_until_created: true` to gate pipeline on ITSM record creation. | +| `ITSM-003` | Inbound events from ITSM systems must be authenticated. ITSM Providers must verify HMAC signatures or OAuth tokens on all inbound webhooks before forwarding to DCM. Unauthenticated inbound events are rejected and logged. | +| `ITSM-004` | ITSM record references stored on DCM entities follow entity lifecycle โ€” they are included in the Realized State record, preserved through updates, and retained in the decommissioned entity record for audit purposes. | +| `ITSM-005` | ITSM Policies that use `block_until_created: true` must declare a `block_timeout`. If the ITSM system does not confirm record creation within the timeout, the policy fires `on_failure` behavior and the block is released โ€” the pipeline continues. A blocked pipeline is never permanently stalled by ITSM unavailability. | +| `ITSM-006` | Field mappings between DCM entities and ITSM CI types must be declared in the ITSM Provider capability registration. Unmapped resource types are silently skipped for CMDB sync actions. | +| `ITSM-007` | Template expressions in ITSM Policy `action_payload` fields must resolve using values from the triggering event payload. Template expressions that reference unavailable fields produce a warning in the audit record and substitute an empty string. They do not block ITSM action execution. | + +--- + +## 5. ITSM Policy System Policies + +| Policy | Rule | +|--------|------| +| `ITSM-POL-001` | ITSM Policies follow the full Policy base contract (B-policy-contract.md): lifecycle (developing โ†’ proposed โ†’ active), shadow mode validation, audit obligation on every evaluation, domain precedence. | +| `ITSM-POL-002` | ITSM Policies are side-effect policies โ€” they do not produce pipeline decisions (allow/deny/transform). They may not be used as GateKeeper substitutes except through the explicit `block_until_created: true` mechanism, which has its own timeout guarantee (ITSM-005). | +| `ITSM-POL-003` | ITSM Policy evaluation is recorded in the audit trail. The audit record includes: policy handle, matched event, ITSM provider UUID, action requested, ITSM record ID returned, and outcome (success/failure/timeout). | +| `ITSM-POL-004` | Multiple ITSM Policies may fire on the same event. All fire independently โ€” one policy's failure does not prevent other ITSM Policies from executing. | + +--- + +## 6. Additions to the Foundations Document + +The foundations document provider type table gains a 12th row: + +| Provider Type | Capability | Data direction | +|--------------|-----------|----------------| +| **ITSM Provider** | Bidirectional integration with ITSM systems; creates/updates ITSM records from DCM events; routes ITSM approvals back to DCM | DCM โ†’ ITSM (outbound) / ITSM โ†’ DCM (inbound) | + +The foundations document policy type table gains an 8th entry: + +| Policy Type | Output | Pipeline role | +|------------|--------|---------------| +| **ITSM Action** | Triggers action in connected ITSM system; optionally stores record reference on entity; optionally gates pipeline on record creation | Side-effect (non-blocking by default) | + +--- + +## 7. Event Catalog Additions + +Two new events for the Event Catalog (doc 33): + +| Event Type | Urgency | Trigger | +|-----------|---------|---------| +| `itsm.record_created` | info | ITSM Provider successfully created a record in external system | +| `itsm.record_failed` | medium | ITSM Provider failed to create/update record; `block_until_created` timeout reached | + +These extend the existing event catalog with a new `itsm.*` domain prefix. + +--- + +## 8. Standards Catalog Addition + +ITSM integration standards and protocols used: + +| Standard | Use in DCM ITSM | +|----------|----------------| +| ServiceNow REST Table API | Primary integration for ServiceNow create/update/query | +| Jira REST API v3 | Primary integration for Atlassian Jira Service Management | +| BMC AR REST API v1 | Primary integration for BMC Remedy/Helix | +| PagerDuty Events API v2 | Incident creation for alert-type ITSM integrations | +| ITIL v4 Change Management | Conceptual framework for DCM change record lifecycle mapping | +| JSON:API | Standard used by several ITSM REST APIs | +| HMAC-SHA256 | Inbound webhook signature verification for all ITSM systems | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/operational-reference.md b/content/docs/architecture/data-model/operational-reference.md new file mode 100644 index 0000000..41ec0d4 --- /dev/null +++ b/content/docs/architecture/data-model/operational-reference.md @@ -0,0 +1,525 @@ +--- +title: "Operational Reference: GitOps Scale, Store Migration, Disaster Recovery" +type: docs +weight: 41 +--- + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** SRE Reference โ€” GitOps Scale, Store Migration, Disaster Recovery +**Related Documents:** [Storage Providers](11-storage-providers.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Four States Model](02-four-states.md) | [Internal Component Authentication](36-internal-component-auth.md) | [DCM Self-Health Endpoints](39-dcm-self-health.md) + +> **Audience:** Platform engineers and SREs operating DCM in production. This document covers three operational concerns that require guidance beyond the architectural specifications: GitOps store partitioning at large scale, migrating between store implementations, and recovering from failure scenarios. + +--- + +## 1. GitOps Store Scale and Partitioning + +### 1.1 When a Repo Becomes Too Large + +The default DCM GitOps layout uses one repository per store type (Intent, Requested, Layer, Policy). For most deployments this is correct. At large scale โ€” tens of thousands of active entities or hundreds of active tenants โ€” a single repository can exhibit: + +- Git operation latency (clone, fetch, log) growing beyond SLA +- CI/CD pipeline fan-out delays as every write triggers the full repository +- Access control granularity limits (all tenants share one repo) +- Search index sync lag from large diffs + +**Thresholds that suggest partitioning:** + +| Signal | Threshold | Recommended action | +|--------|-----------|-------------------| +| Entities in Intent/Requested store | > 50,000 active | Consider tenant-shard partitioning | +| Git clone time | > PT30S | Add shallow-clone depth; consider partitioning | +| PR merge latency | > PT5M | Partition or add write buffer | +| Tenant count | > 500 | Consider per-tenant repositories | +| Repository size on disk | > 10 GB | Partition | + +These are guidelines, not hard limits. Profile, hardware, and Git host performance all affect the actual inflection point. + +### 1.2 Partitioning Strategies + +DCM supports three partitioning strategies. All are compatible with the storage contract โ€” partitioning changes how stores are organized, not what the store contract requires. + +#### Strategy A โ€” Tenant Shard Partitioning (recommended for most) + +Split each store type into N shard repositories, with tenants assigned to shards by a deterministic hash of `tenant_uuid`: + +``` +shard = hash(tenant_uuid) % N + +dcm-intent-shard-0/ โ† tenants whose hash(uuid) % N == 0 +dcm-intent-shard-1/ โ† tenants whose hash(uuid) % N == 1 + tenants/ + {tenant-uuid}/ + requests/ + {request-uuid}/ + intent.yaml +``` + +**DCM configuration:** +```yaml +gitops_store: + intent_store: + partitioning: tenant_shard + shard_count: 8 + shard_routing: hash_mod # deterministic; no routing table needed + repositories: + - shard: 0 + url: https://git.corp/dcm/dcm-intent-shard-0 + - shard: 1 + url: https://git.corp/dcm/dcm-intent-shard-1 + # ... +``` + +**Operational implications:** Adding shards requires re-hashing. Plan shard counts for 3โ€“5 years of expected growth; use a power of 2 to simplify future doubling. + +#### Strategy B โ€” Per-Tenant Repositories (for strict isolation) + +Each tenant has its own set of store repositories. Used when: +- Tenants are separate organizations (MSP model) +- Compliance requires complete data isolation per tenant +- Different retention policies per tenant + +``` +dcm-intent-{tenant-uuid}/ โ† one repository per tenant + requests/ + {request-uuid}/ + intent.yaml +``` + +**Operational implications:** Repository count scales with tenant count. Requires automation for tenant onboarding (repository creation, access provisioning). Git host must support large numbers of repositories. + +#### Strategy C โ€” Time-Based Archiving (for retention management) + +Active entities stay in the primary repository. Entities past a declared age threshold are archived to read-only archive repositories: + +``` +dcm-intent-active/ โ† current entities (hot) +dcm-intent-archive-2025/ โ† entities from 2025 (cold, read-only) +dcm-intent-archive-2024/ โ† entities from 2024 (cold, read-only) +``` + +DCM's audit and search components are configured with both active and archive repository lists. Strategy C is typically combined with Strategy A or B. + +### 1.3 Large-Scale Layer Store Partitioning + +The Layer Store grows more slowly than the Intent/Requested stores (layers are reused across requests). Layer Store partitioning is by domain rather than by tenant: + +``` +dcm-layers-compute/ โ† Compute.* resource type layers +dcm-layers-network/ โ† Network.* resource type layers +dcm-layers-storage/ โ† Storage.* resource type layers +dcm-layers-platform/ โ† Platform.* and cross-cutting layers +dcm-policies-core/ โ† System and core policies +dcm-policies-tenant/ โ† Tenant-contributed policies (per tenant or sharded) +``` + +Domain-partitioned Layer stores are configured in DCM's layer assembly engine: + +```yaml +layer_store: + repositories: + - domain: Compute.* + url: https://git.corp/dcm/dcm-layers-compute + priority: provider_contribution # provider contributions go here + - domain: Network.* + url: https://git.corp/dcm/dcm-layers-network + - domain: "*" # catch-all for uncategorized + url: https://git.corp/dcm/dcm-layers-platform +``` + +### 1.4 Shallow Clones and Read-Only Mirrors + +For read-heavy operations (audit, search index rebuild, drift reconciliation) that do not need full Git history: + +```yaml +gitops_store: + read_operations: + clone_depth: 1 # shallow clone for read-only consumers + use_mirror: true # read from read-only mirror; writes go to primary + mirror_url: https://git-mirror.corp/dcm/ + mirror_sync_lag_max: PT5M # alert if mirror is more than 5 minutes behind +``` + +--- + +## 2. Store Migration + +### 2.1 Migration Principles + +DCM store migrations follow three invariants: + +1. **No data loss** โ€” every record in the source store must exist in the target store after migration +2. **Audit chain continuity** โ€” the audit hash chain must be unbroken across the migration; audit records written before and after must chain correctly +3. **Read availability during migration** โ€” DCM continues serving read requests throughout; write availability may be briefly paused during cutover + +### 2.2 Migration Playbook Structure + +Every store migration follows this pattern regardless of source or target implementation: + +``` +Phase 1 โ€” Prepare + โ”‚ Provision target store alongside source + โ”‚ Validate target store meets storage contract (health check, write test, read test) + โ”‚ Configure DCM to write to BOTH source and target (dual-write mode) + โ”‚ +Phase 2 โ€” Backfill + โ”‚ Export all existing records from source + โ”‚ Import records to target in chronological order (preserving provenance timestamps) + โ”‚ Verify record counts match; spot-check content hashes + โ”‚ +Phase 3 โ€” Validate + โ”‚ Run DCM's store validation suite against target + โ”‚ Verify audit chain integrity on target store + โ”‚ Verify search index can be rebuilt from target store + โ”‚ +Phase 4 โ€” Cutover + โ”‚ Brief write pause (PT30Sโ€“PT5M depending on profile) + โ”‚ Disable dual-write; switch DCM to target as primary + โ”‚ Verify /readyz returns healthy + โ”‚ Resume writes to target only + โ”‚ +Phase 5 โ€” Decommission source (after burn-in period) + Default burn-in: P30D (standard), P90D (fsi/sovereign) + Keep source in read-only mode during burn-in for rollback +``` + +### 2.3 Common Migration Paths + +#### SQLite โ†’ PostgreSQL (minimal/dev โ†’ standard) + +Typical trigger: scaling beyond single-node evaluation environment. + +```bash +# Step 1: Export from SQLite +dcm-admin store export \ + --store realized \ + --format jsonl \ + --output realized-export.jsonl + +# Step 2: Import to PostgreSQL +dcm-admin store import \ + --store realized \ + --source realized-export.jsonl \ + --target postgres://pg-host:5432/dcm_realized \ + --validate-chain + +# Step 3: Enable dual-write +dcm-admin store dual-write enable \ + --store realized \ + --primary sqlite://dcm-realized.db \ + --secondary postgres://pg-host:5432/dcm_realized + +# Step 4: Validate +dcm-admin store validate \ + --store realized \ + --target postgres://pg-host:5432/dcm_realized \ + --check-count --check-chain --check-spot-sample 0.05 + +# Step 5: Cutover +dcm-admin store cutover \ + --store realized \ + --target postgres://pg-host:5432/dcm_realized +``` + +#### PostgreSQL single-instance โ†’ CockroachDB / PostgreSQL HA + +Typical trigger: HA requirement for production; multi-region deployment. + +**Key difference from SQLite โ†’ PostgreSQL:** CockroachDB uses serializable isolation and distributed transactions. Test write throughput under realistic load before cutover โ€” CockroachDB's latency profile differs from single-node PostgreSQL. + +```yaml +# Pre-migration checklist +migration_checklist: + - Load test target under realistic DCM write volume (PT4H minimum) + - Verify CockroachDB schema compatibility (DCM uses standard PostgreSQL wire protocol) + - Configure connection pooler (PgBouncer or similar) โ€” CockroachDB default connection count + - Verify time synchronization (CockroachDB requires NTP within PT500MS across nodes) + - Test audit chain write under partition scenario +``` + +#### GitOps Store โ€” Repo Restructuring + +Restructuring a GitOps repository (e.g. monorepo to sharded) requires special handling because Git history must be preserved. + +``` +Step 1: Enable write buffer โ€” all new writes queue while migration proceeds +Step 2: git filter-repo or git subtree to extract tenant directories to shard repos +Step 3: Validate file counts and content hashes in each shard +Step 4: Update DCM gitops_store configuration to point to shards +Step 5: Drain write buffer โ€” queued writes replay to new shard repos +Step 6: Verify search index rebuild from shards +Step 7: Archive or delete monorepo after burn-in period +``` + +### 2.4 Rollback Procedure + +If migration fails before cutover: disable dual-write, discard target, no impact to production. + +If migration fails after cutover (during burn-in): + +``` +1. Alert: /readyz reports degraded or source store discrepancy detected +2. dcm-admin store rollback --store --to source + (requires source still in read-only mode โ€” NOT decommissioned) +3. DCM restarts reads/writes from source +4. Export any writes that reached target but not source (if any, during dual-write gap) +5. Import gap records to source +6. Re-enable source as primary +``` + +**This is why burn-in period exists.** Do not decommission source stores until burn-in completes. + +### 2.5 Profile-Governed Migration Constraints + +| Profile | Min dual-write duration | Max cutover pause | Burn-in period | +|---------|------------------------|-------------------|----------------| +| `minimal` | P1D | PT5M | P7D | +| `dev` | P3D | PT5M | P14D | +| `standard` | P7D | PT2M | P30D | +| `prod` | P14D | PT1M | P30D | +| `fsi` | P30D | PT30S | P90D | +| `sovereign` | P60D | PT30S | P90D | + +--- + +## 3. Disaster Recovery Runbook + +### 3.1 DCM Recovery Architecture + +DCM's recovery model is built on a key property: **all durable state is in the stores, not in the control plane.** Control plane components (Policy Engine, Request Orchestrator, etc.) are stateless and can be restarted without data loss. Recovery from most failures is component restart, not data restoration. + +The five DCM stores and their recovery characteristics: + +| Store | Implementation | Data durability | Recovery method | +|-------|---------------|----------------|-----------------| +| Intent Store | GitOps (Git) | Git replication + remote | Re-clone from remote | +| Requested Store | GitOps or write-once | Git replication / DB replication | Re-clone or DB restore | +| Layer Store | GitOps (Git) | Git replication + remote | Re-clone from remote | +| Realized Store | Write-once (PostgreSQL/CockroachDB) | DB replication / WAL | DB failover or restore | +| Audit Store | Append-only (Kafka/PostgreSQL) | Replication / WAL | Kafka failover or restore | + +### 3.2 Recovery Scenarios and Procedures + +#### Scenario 1: Single Component Failure (Most Common) + +**Symptoms:** One DCM component (e.g. Policy Engine) is unhealthy. `/readyz` shows degraded. Requests may be delayed but not lost. + +**RTO:** PT5M +**RPO:** 0 (no data loss โ€” components are stateless) + +``` +1. Identify failing component via GET /api/v1/admin/health +2. Check component logs for panic/OOM/deadlock +3. Kubernetes: pod restart is automatic (liveness probe) + Manual: kubectl rollout restart deployment/dcm-policy-engine +4. Monitor /readyz โ€” should recover within PT2M of pod restart +5. If component repeatedly fails: check Internal CA cert expiry (ICOM-006) + dcm-admin component cert-status --component policy-engine +6. Write post-incident note to DCM audit store +``` + +#### Scenario 2: Store Failure (Database / Kafka) + +**Symptoms:** `/readyz` fails specific store check. Requests queue or fail depending on which store. + +**Realized Store failure (highest severity โ€” blocks realization):** + +``` +RTO target: PT30M (standard), PT15M (prod), PT5M (fsi/sovereign) +RPO: 0 for PostgreSQL HA (synchronous replication); near-zero for async + +1. Confirm store failure: GET /api/v1/admin/health โ†’ realized_store: fail +2. If HA: check if automatic failover occurred + kubectl get pods -n dcm-stores | grep postgres + Check PostgreSQL replication lag / CockroachDB node status +3. Manual failover if automatic did not trigger: + dcm-admin store failover --store realized --target replica-2 +4. Verify replication caught up: dcm-admin store lag --store realized +5. Verify /readyz recovers +6. Root cause analysis: WAL lag, disk full, network partition +``` + +**Audit Store failure:** + +``` +RTO: PT1H acceptable (audit trail can tolerate temporary buffering) +RPO: profile-governed โ€” see Audit Store write buffer policy + +1. DCM buffers audit records locally (Commit Log) during store outage + Write buffer capacity: profile-governed (PT1H standard, PT15M sovereign) +2. Restore Kafka cluster from replica or snapshot +3. DCM drains buffer to restored store automatically on reconnection +4. Verify chain integrity: dcm-admin audit chain-verify --since +``` + +**GitOps Store failure (Intent/Requested/Layer):** + +``` +RTO: PT30M (stores are remountable from Git remote) +RPO: 0 (all writes go to Git remote; loss only if remote is also lost) + +1. Git remote unreachable: check network connectivity +2. If Git host is down: DCM switches to cached/buffered mode + New requests queue in write buffer; reads served from local clone +3. Write buffer capacity: PT4H (standard) โ€” configure per deployment +4. When Git host recovers: buffer drains automatically +5. Force drain: dcm-admin store drain-buffer --store intent +``` + +#### Scenario 3: Full Control Plane Loss + +**Symptoms:** All DCM pods down. Stores intact. Users cannot submit requests. + +**RTO:** PT15M (kubernetes deployment restart) +**RPO:** 0 (stores are external โ€” no data in pods) + +``` +1. Verify stores are healthy (connect directly): + dcm-admin store health-check --all --direct + +2. Verify Internal CA is available: + curl -k https://dcm-internal-ca.dcm-system.svc.cluster.local/health + +3. Restart DCM deployment (Kubernetes): + kubectl rollout restart deployment -n dcm-system + +4. Monitor /readyz โ€” startup sequence should complete within PT3M: + watch -n 5 kubectl get pods -n dcm-system + +5. Verify session store recovers: + GET /api/v1/admin/health โ†’ session_store: pass + +6. Alert consumers: any in-flight requests at time of failure + are in ACKNOWLEDGED/DISPATCHED state and may need status check + dcm-admin requests find --status in-flight --since +``` + +#### Scenario 4: Partial Region Loss (Multi-Region Deployments) + +**Symptoms:** One region's DCM instance degraded. Other regions operational. + +``` +1. DCM federation routes requests away from degraded region (automatic) + Verify: GET /api/v1/admin/health โ†’ federation peer status + +2. If region is sovereign-scoped (data must not leave): + Alert: sovereignty.migration_required event fires + Consumers in that region may be blocked until region recovers + +3. For non-sovereign regions: traffic reroutes automatically + Monitor: dcm_requests_total{region} for traffic shift + +4. Region recovery: standard Scenario 3 procedure + After recovery: drift detection validates recovered state +``` + +#### Scenario 5: Complete Loss (Repave) + +The nuclear scenario: entire DCM installation destroyed. Git remote intact. + +**RTO:** PT4Hโ€“PT24H (depends on infrastructure provisioning speed) +**RPO:** 0 for GitOps stores; near-zero for Realized/Audit stores + +``` +1. Provision new Kubernetes cluster (or equivalent) + +2. Deploy DCM bootstrap installer: + helm install dcm-bootstrap dcm/dcm-bootstrap \ + --set gitops.manifest_url=https://git.corp/dcm/dcm-deployment \ + --set gitops.manifest_ref= + +3. DCM bootstrap reads dcm_deployment manifest from Git + Provisions itself: control plane, Internal CA, stores + +4. Restore Realized Store from backup: + dcm-admin store restore --store realized \ + --from s3://dcm-backups/realized/latest \ + --validate-chain + +5. Restore Audit Store from backup or Kafka snapshot: + dcm-admin store restore --store audit \ + --from s3://dcm-backups/audit/latest \ + --chain-verify + +6. Intent/Requested/Layer stores: re-clone from Git remote (already current) + +7. DCM rehydrates managed resources in dependency order: + dcm-admin rehydrate --all-tenants --dry-run # verify plan first + dcm-admin rehydrate --all-tenants + +8. Drift detection validates recovered state matches declared state: + dcm-admin drift scan --all --post-recovery + +9. Re-issue Internal CA certificates for all components: + (handled automatically by bootstrap โ€” components acquire new certs) + +10. Notify consumers: recovery complete; request status available +``` + +### 3.3 Recovery Time Objectives by Profile + +| Profile | Scenario 1 (component) | Scenario 2 (store) | Scenario 3 (full CP) | Scenario 5 (repave) | +|---------|------------------------|--------------------|-----------------------|---------------------| +| `minimal` | PT15M | PT2H | PT30M | PT24H | +| `standard` | PT5M | PT30M | PT15M | PT8H | +| `prod` | PT2M | PT15M | PT10M | PT4H | +| `fsi` | PT2M | PT5M | PT5M | PT2H | +| `sovereign` | PT1M | PT5M | PT5M | PT2H | + +### 3.4 Recovery Point Objectives + +| Store | Standard RPO | fsi/sovereign RPO | Notes | +|-------|-------------|------------------|-------| +| Intent Store | 0 | 0 | Git remote is source of truth | +| Requested Store | 0 | 0 | Write-once; replicated | +| Layer Store | 0 | 0 | Git remote is source of truth | +| Realized Store | PT5M | PT1M | Async replication lag | +| Audit Store | PT15M | PT1M | Kafka replication + write buffer | + +### 3.5 Backup Schedule + +DCM does not manage backups of infrastructure stores directly โ€” that responsibility belongs to the storage platform. Recommended schedules by store: + +| Store | Backup method | Frequency | Retention | +|-------|--------------|-----------|-----------| +| Intent / Requested / Layer | Git push to offsite remote | Continuous | Per Git host policy | +| Realized Store | PostgreSQL PITR + daily snapshot | Continuous WAL + P1D snapshot | P90D (standard), P365D (fsi/sovereign) | +| Audit Store | Kafka topic snapshot | P4H | P365D (all profiles โ€” regulatory minimum) | +| Internal CA | Key material backup to HSM/Vault | On change | P7Y (key material outlives certs) | + +### 3.6 Post-Recovery Validation Checklist + +Run after any Scenario 3+ recovery: + +``` +โ–ก /livez returns pass on all control plane pods +โ–ก /readyz returns pass (all 5 core dependencies green) +โ–ก GET /api/v1/admin/health shows all components pass +โ–ก dcm-admin audit chain-verify --full returns no broken links +โ–ก dcm-admin store validate --all returns no discrepancies +โ–ก dcm-admin drift scan --all returns no unexpected drift +โ–ก Internal CA certificates valid for all components (ICOM-006) +โ–ก At least one Auth Provider healthy (GET /api/v1/admin/health โ†’ auth_providers) +โ–ก Search index rebuild complete (if search index store was affected) +โ–ก Session Store empty (expected โ€” all sessions expired during outage; users re-authenticate) +โ–ก Write post-incident note to audit store with recovery timeline +โ–ก Notify consumers of recovery completion +``` + +--- + +## 4. System Policies + +| Policy | Rule | +|--------|------| +| `OPS-001` | GitOps store partitioning strategy must be declared in the DCM deployment manifest. Changes to partitioning strategy require dual-write migration procedure (Section 2). | +| `OPS-002` | Store migrations must maintain audit chain continuity across cutover. Audit records written to the source store before cutover and to the target store after cutover must form an unbroken chain. | +| `OPS-003` | Source stores must remain accessible in read-only mode for the profile-governed burn-in period after cutover. Source stores must not be decommissioned until the burn-in period completes and rollback is confirmed unnecessary. | +| `OPS-004` | Recovery from Scenario 3 (full control plane loss) must complete within the profile-governed RTO. If RTO cannot be met, the incident must be escalated and root cause must address the recovery path. | +| `OPS-005` | The post-recovery validation checklist (Section 3.6) must be completed and its results written to the audit store before declaring an incident resolved. | +| `OPS-006` | Audit Store backups must be retained for a minimum of P365D in all profiles, regardless of other data retention policies, to satisfy regulatory audit trail requirements. | +| `OPS-007` | Git remote repositories serving as GitOps stores must be configured with push access from at least two geographically separated locations to prevent single-point-of-failure data loss. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/data-model/policy-contract.md b/content/docs/architecture/data-model/policy-contract.md index a358fbb..aada912 100644 --- a/content/docs/architecture/data-model/policy-contract.md +++ b/content/docs/architecture/data-model/policy-contract.md @@ -371,7 +371,36 @@ lifecycle_policy_output: --- -## 15. Policy Composition +## 15. Output Schema โ€” ITSM Action + +The ITSM Action policy type triggers actions in connected ITSM systems as a side-effect of DCM pipeline events. + +```yaml +itsm_action_output: + type: itsm_action + itsm_provider_uuid: # registered ITSM Provider UUID + action: create_change_request | update_change_request | close_change_request | + create_incident | update_incident | close_incident | + update_cmdb_ci | create_cmdb_ci | retire_cmdb_ci | + create_service_request | link_parent_record + action_payload: + : + store_reference_on_entity: # default: false + reference_label: + block_until_created: # default: false โ€” see ITSM-005 + block_timeout: # required if block_until_created: true + on_failure: log_and_continue | alert_and_continue | alert_only +``` + +> **See [ITSM Integration](42-itsm-integration.md)** for full ITSM Provider registration, capability declarations, supported ITSM systems (ServiceNow, Jira, Remedy, Freshservice, PagerDuty, generic REST), policy examples, and system policies (ITSM-001โ€“007, ITSM-POL-001โ€“004). + +**Key constraints:** +- ITSM Action policies are side-effect only โ€” they do not produce allow/deny decisions +- `block_until_created: true` creates a pipeline gate with mandatory timeout (ITSM-005) +- Multiple ITSM Action policies on the same event fire independently (ITSM-POL-004) +- Full audit record produced on every evaluation (ITSM-POL-003) + +## 16. Policy Composition Policies compose naturally through the domain precedence model: @@ -388,7 +417,7 @@ For a single request, all active matching policies at all domain levels evaluate --- -## 16. Related Policies +## 17. Related Policies | Policy | Rule | |--------|------| diff --git a/content/docs/architecture/data-model/provider-contract.md b/content/docs/architecture/data-model/provider-contract.md index 6586f6a..b43bb88 100644 --- a/content/docs/architecture/data-model/provider-contract.md +++ b/content/docs/architecture/data-model/provider-contract.md @@ -532,3 +532,33 @@ Profile-governed approval methods override provider type defaults. See [Registra --- *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + +## ITSM Provider + +**What it does:** Provides bidirectional integration with external ITSM systems. Outbound: receives DCM lifecycle event data and creates/updates records in the ITSM system (change requests, incidents, CMDB CIs). Inbound: routes ITSM approval decisions back to DCM as approval votes, and accepts ITSM-initiated request submissions. + +**Additional endpoints (ITSM Provider implements):** +``` +POST {provider_base}/actions # receive action request from DCM +GET {provider_base}/actions/{action_id} # DCM checks action completion status +GET {provider_base}/records/{record_id} # DCM retrieves record status from ITSM +POST {provider_base}/inbound # ITSM system sends inbound approval/event +``` + +**Capability declaration extension:** +```yaml +itsm_provider_capabilities: + itsm_system: servicenow | jira_service_management | bmc_remedy | bmc_helix | + freshservice | zendesk | pagerduty | opsgenie | manageengine | + cherwell | topdesk | generic_rest + supported_actions: [] + endpoint_url: + auth_credential_uuid: + inbound_webhook: + enabled: + secret_credential_uuid: + field_mapping_ref: + cmdb_ci_type_map: [] +``` + +> **See [ITSM Integration](42-itsm-integration.md)** for complete capability declaration schemas, supported systems, and system policies ITSM-001โ€“007. diff --git a/content/docs/architecture/data-model/standards-catalog.md b/content/docs/architecture/data-model/standards-catalog.md index bfd6c87..20f6cf6 100644 --- a/content/docs/architecture/data-model/standards-catalog.md +++ b/content/docs/architecture/data-model/standards-catalog.md @@ -252,6 +252,21 @@ DCM maps profile security postures to NIST Authentication Assurance Levels: --- +## 10. ITSM Integration Standards + +These standards and protocols are used by ITSM Provider implementations: + +| Standard / Protocol | Use in DCM ITSM Integration | Obligation | +|--------------------|-----------------------------|------------| +| **ServiceNow REST Table API** | Primary integration for ServiceNow; create/read/update change_request, incident, cmdb_ci tables | Normative for ServiceNow provider | +| **Jira REST API v3** | Primary integration for Jira Service Management; issue create/update/transition | Normative for Jira provider | +| **BMC AR REST API v1** | Integration for BMC Remedy/Helix ITSM; form-based create/update | Normative for BMC provider | +| **PagerDuty Events API v2** | Incident creation and update for alert-type integrations | Normative for PagerDuty provider | +| **HMAC-SHA256** | Inbound webhook signature verification for all ITSM systems; shared secret via Credential Provider | Normative | +| **ITIL v4 Change Management** | Conceptual framework for DCM change record lifecycle mapping (Normal, Standard, Emergency change types) | Informative | +| **JSON:API** | Used by several ITSM REST APIs for response formatting | Informative | +| **JSONPath** | Template expression resolution for `generic_rest` action payloads; response field extraction | Normative for generic_rest | + ## 9. Policy Family to Standard Mapping Each DCM system policy family maps to one or more industry standards. This table supports compliance traceability. @@ -275,7 +290,8 @@ Each DCM system policy family maps to one or more industry standards. This table | **SMX-001โ€“010** | Risk scoring; NIST RMF; organizational risk tolerance | Hybrid scoring; approval routing; enforcement class | | **FCM-001โ€“008** | GitOps contribution model; CNCF governance practices | Federated policy contribution; shadow validation; trust levels | | **GMX-001โ€“006** | Governance Matrix; policy-as-code; organizational controls | Cross-domain policy enforcement; data classification | +| **ITSM-001โ€“007 + ITSM-POL-001โ€“004** | ITIL v4, ServiceNow/Jira/Remedy REST APIs, HMAC-SHA256, ITIL change management | ITSM Provider registration; inbound webhook auth; ITSM Policy evaluation; blocking gate with timeout guarantee | --- -*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* \ No newline at end of file diff --git a/content/docs/architecture/specifications/11-kubernetes-compatibility.md b/content/docs/architecture/specifications/11-kubernetes-compatibility.md new file mode 100644 index 0000000..614e22c --- /dev/null +++ b/content/docs/architecture/specifications/11-kubernetes-compatibility.md @@ -0,0 +1,422 @@ +# DCM โ€” Kubernetes Compatibility and Concept Mappings + + +> ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress +> +> All questions resolved. Cluster-as-a-Service model defined. Namespace-to-Tenant mapping, admission webhook model, and managed K8s integration all specified. +> +> **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.** +> +> The Kubernetes operator integration layer โ€” including the Operator Interface Specification, Operator SDK API, and Kubernetes compatibility mappings โ€” represents design intent that has not yet been validated against implementation. Specific interface contracts, API signatures, SDK method names, and CRD structures **will change** as implementation work begins. +> +> **Do not build against these specifications yet.** They are published to share design direction and invite feedback, not as stable contracts. +> +> Known gaps and open items for this section: +> - Operator Interface Specification: reconciliation hook signatures are provisional +> - Operator SDK API: Go module structure and dependency model not yet finalized +> - Kubernetes Compatibility Mappings: some concept mappings remain under discussion +> - SDK code examples are illustrative only โ€” not yet tested against a real implementation +> +> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). + + + +**Document Status:** โœ… Complete +**Document Type:** Architecture Reference +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) + +--- + +## 1. Purpose + +DCM is designed as a **superset of Kubernetes** โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries that Kubernetes alone cannot address. + +This document serves three purposes: + +1. **Defines the formal mapping** between Kubernetes concepts and DCM concepts โ€” enabling implementors to understand how the two models relate and where DCM extends beyond Kubernetes +2. **Establishes DCM Resource Types** for standard Kubernetes resources โ€” so that Kubernetes-managed resources participate in the DCM registry alongside non-Kubernetes resources +3. **Documents the boundary** between what Kubernetes governs and what DCM governs โ€” making clear that DCM extends Kubernetes rather than replacing it + +--- + +## 2. The Superset Relationship + +### 2.1 What Kubernetes Provides + +Kubernetes is a container orchestration platform that provides: +- Declarative desired-state management within a single cluster +- A controller/operator pattern for extending resource management +- Namespace-based isolation within a cluster +- RBAC for access control within a cluster +- A rich ecosystem of operators for managing complex stateful resources + +### 2.2 What DCM Adds + +DCM extends Kubernetes upward by providing: + +| Capability | Kubernetes | DCM | +|------------|-----------|-----| +| Scope | Single cluster | Multi-cluster, multi-infrastructure | +| Tenancy | Namespace isolation | First-class Tenant model with ownership | +| Policy | RBAC + admission webhooks | Full Policy Engine with Validation/Transformation/GateKeeper | +| Data lineage | Not provided | Field-level provenance on all data | +| Cost attribution | Not provided | Full lifecycle cost analysis | +| Drift detection | Basic โ€” controller reconciles | Full four-state model with Intent/Requested/Realized/Discovered | +| Service catalog | Not provided | Full self-service catalog with RBAC-governed presentation | +| Sovereignty | Not provided | Sovereignty declarations, placement constraints, compliance evidence | +| Information context | Labels/annotations | First-class Information Provider relationships | +| Non-Kubernetes resources | Not provided | VMware, bare metal, OpenStack, etc. all managed through same model | + +### 2.3 What DCM Does Not Replace + +DCM does not replace Kubernetes at the runtime level. Kubernetes continues to: +- Schedule and run containers +- Manage Pod lifecycle within a cluster +- Enforce network policies within a cluster +- Provide the Kubernetes API for cluster-native tooling +- Run operators that manage complex stateful resources + +DCM manages the management plane โ€” the lifecycle of what gets requested, provisioned, owned, governed, and decommissioned. Kubernetes manages the execution plane โ€” the runtime behavior of what is running. + +--- + +## 3. Core Concept Mappings + +### 3.1 Resource Model + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Custom Resource Definition (CRD) | Resource Type Specification | CRD schema โ†’ DCM Resource Type fields | DCM Resource Type is the portable, provider-agnostic equivalent. CRD is the Kubernetes-specific implementation schema. | +| Custom Resource (CR) | Requested State payload โ†’ Realized State entity | CR is the naturalized form of the DCM payload | The operator translates DCM Requested State into a CR (Naturalization) and translates CR status back to DCM Realized State (Denaturalization). | +| Built-in resource (Pod, Service, PV) | DCM Resource Type in Compute.*, Network.*, Storage.* | Kubernetes built-ins are valid DCM Resource Types | See Section 5 for standard Kubernetes resource type mappings. | +| Kubernetes object | Resource/Service Entity | Every Kubernetes object managed by DCM has a corresponding DCM entity with UUID and provenance | | + +### 3.2 Control Loop + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Operator reconciliation loop | Realization + Drift Detection combined | Reconciliation IS the realization process โ€” the operator drives actual state toward desired state | DCM's Drift Detection compares Discovered State against Realized State. The operator's reconciliation loop is the mechanism that corrects drift. | +| Desired state (CR spec) | Requested State | CR spec is the naturalized form of the DCM Requested State | DCM stores the Requested State in DCM format. The operator translates it to CR spec format. | +| Actual state (CR status) | Realized State | CR status is the Kubernetes-native form of the DCM Realized State | The operator must denaturalize CR status back to DCM Realized State format and report it to DCM. | +| Watch/Inform pattern | DCM Discovered State polling | Kubernetes watch events are the mechanism for keeping DCM Discovered State current | | + +### 3.3 Isolation and Multi-tenancy + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Namespace | DCM Tenant boundary | One namespace per DCM Tenant (per_tenant strategy) | Kubernetes namespace provides the physical isolation enforcement. DCM Tenant provides the ownership and governance model. A single DCM Tenant maps to exactly one namespace per cluster. | +| Namespace | DCM Resource Group | In shared namespace strategies, Resource Group labels replace namespace isolation | When multiple Tenants share a namespace, DCM Resource Group labels provide logical separation. | +| Kubernetes RBAC | DCM IDM/IAM + Policy Engine | Kubernetes RBAC is the runtime enforcement mechanism. DCM Policy Engine governs who can request what via the service catalog. | DCM policies determine what a user can request. Kubernetes RBAC determines what a running workload can do. These are complementary, not duplicative. | +| ServiceAccount | DCM Identity.ServiceAccount Information Type | Kubernetes ServiceAccounts that DCM provisions or references are modeled as DCM Information Type entities | | + +### 3.4 Relationships and Dependencies + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| ownerReference | Entity Relationship (`contains`/`contained_by`) | Kubernetes ownerReferences are a subset of DCM entity relationships โ€” ownership only | DCM relationships are richer โ€” supporting `requires`, `depends_on`, `references`, `peer`, `manages` in addition to ownership. During Denaturalization, ownerReferences are translated to DCM `contains` relationships. | +| Finalizers | Lifecycle policy (`retain`, `detach`) | Kubernetes finalizers implement DCM lifecycle policies at the Kubernetes level | When DCM declares `on_parent_destroy: retain` for a storage entity, the operator implements this using Kubernetes finalizers to prevent deletion until DCM confirms the lifecycle policy has been applied. | +| Label selectors | Resource Group membership | Kubernetes label selectors used for DCM Resource Group filtering | DCM mandatory labels (`dcm-tenant-id`, `dcm-entity-id`) are used as label selectors for Resource Group queries. | + +### 3.5 Data Model + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Labels | DCM entity metadata + relationships | DCM-mandatory labels (`dcm-managed`, `dcm-tenant-id`, `dcm-entity-id`, etc.) carry core DCM identity data. Custom labels may map to DCM Information Type relationships. | | +| Annotations | DCM field-level provenance + metadata | Annotations used by DCM to carry request correlation data during the request lifecycle | `dcm-request-id` annotation on a CR identifies the DCM request that created or last modified it โ€” enabling unsanctioned change detection. | +| Resource version | Entity version (Revision component) | Kubernetes resource versions map to DCM entity Revision increments | Major and Minor versions are managed by DCM based on breaking/non-breaking changes. Kubernetes resource version increments map to DCM Revision increments. | +| Generation | Requested State version | CR generation increments correspond to new DCM Requested State records | Each new generation of a CR corresponds to a new intent/request cycle in DCM. | + +### 3.6 Lifecycle + +| Kubernetes Concept | DCM Concept | Relationship | Notes | +|-------------------|-------------|--------------|-------| +| Pod phases (Pending, Running, Succeeded, Failed, Unknown) | DCM lifecycle states | Pod phases map to DCM lifecycle states via condition_mappings declaration | | +| CRD conditions | DCM lifecycle states and events | Standard conditions (Ready, Degraded, Progressing) map to DCM states and events via the field mapping specification | | +| Kubernetes events | DCM lifecycle events | Kubernetes watch events trigger DCM lifecycle event reports | The operator translates Kubernetes events into DCM lifecycle event types (ENTITY_HEALTH_CHANGE, DEGRADATION, UNSANCTIONED_CHANGE, etc.) | +| Cluster deletion | DCM decommission workflow | Cluster deletion triggers DCM's full decommission lifecycle โ€” lifecycle policies applied to all related entities | | + +--- + + +## 3a. Cluster as a Service โ€” The Primary Model + +A Kubernetes cluster is a first-class catalog item in DCM. Any authorized Tenant can request and own a cluster through the service catalog, the same way they request a VM or a network. This is not a special case โ€” it is the expected primary consumption model for Kubernetes infrastructure in DCM. + +**How it works:** + +```yaml +# Consumer requests a cluster via the catalog +catalog_item: Platform.KubernetesCluster +provider: CAPI-based Service Provider (or managed K8s Service Provider) +tenant_uuid: + +# The resulting entity: +entity: + resource_type: Platform.KubernetesCluster + tenant_uuid: # Tenant owns the cluster + lifecycle_state: OPERATIONAL + fields: + kubernetes_version: "1.29" + node_count: 3 + api_endpoint: "https://cluster-01.eu-west.example.com" + kubeconfig_ref: # via Credential Provider +``` + +**Ownership scope:** When a Tenant owns a `Platform.KubernetesCluster` entity, that Tenant owns everything within the cluster boundary โ€” including cluster-scoped resources (ClusterRoles, StorageClasses, PersistentVolumes, CRDs registered for that cluster). The cluster entity is the ownership boundary. DCM treats the cluster as an opaque resource from a Tenant ownership perspective โ€” the Tenant gets the cluster; what's inside it belongs to them. + +**The Meta Provider pattern:** A Cluster-as-a-Service catalog item typically composes multiple constituent resources: +```yaml +Platform.KubernetesCluster โ†’ constituent providers: + - Compute resources (control plane + worker nodes) + - Network resources (load balancer, ingress) + - Storage resources (CSI driver + storage class) + - DNS records (cluster API endpoint) + - Credential issuance (kubeconfig via Credential Provider) +``` + +This is a Meta Provider โ€” the cluster catalog item orchestrates all constituents and presents a single entity to the Tenant. + +**Sovereignty and accreditation:** Cluster placement follows the standard Placement Engine model. Sovereignty constraints declared by the Tenant apply to cluster placement โ€” a GDPR-scoped Tenant requesting a cluster gets a cluster placed in an EU sovereignty zone. The CAPI provider (or managed K8s Service Provider) must hold appropriate accreditations. + +**Post-provision:** Once the cluster is OPERATIONAL, it can optionally register with DCM as a nested Service Provider for workload resources. The Tenant can then request workload resources (Deployments, Services, PersistentVolumes) against their cluster through the same DCM service catalog. This creates the superset model: DCM provisions the cluster โ†’ cluster becomes a workload Service Provider โ†’ Tenant uses DCM to manage workloads on their cluster. + + +## 4. Where DCM Extends Beyond Kubernetes + +These are concepts that exist in DCM but have no Kubernetes equivalent. They are the capabilities DCM adds that justify the superset positioning. + +### 4.1 Intent State + +Kubernetes has no concept of a consumer's original intent separate from the desired state. Once you apply a manifest, Kubernetes only knows the current desired state โ€” not what the consumer originally asked for or why. + +DCM's Intent State is the immutable record of what the consumer asked for, stored before any policy processing or layer enrichment. This enables: +- Rehydration โ€” replaying the original intent through current policies to produce a new request +- Intent portability โ€” the same intent applied to a different provider +- Audit โ€” answering "what did the consumer originally ask for?" independently of what was realized + +### 4.2 Field-Level Provenance + +Kubernetes has no concept of where a field value came from or why it was set. A field in a CR spec is a field โ€” there is no lineage. + +DCM's field-level provenance carries the full lineage of every field value through the entire lifecycle โ€” which layer set it, which policy modified it, which provider realized it, and why each change was made. This enables complete audit trails and sovereignty evidence. + +### 4.3 Data Layers and Assembly + +Kubernetes has no equivalent to DCM's layering model. A Kubernetes manifest is a flat declaration โ€” there is no concept of organizational standards, site-specific configuration, and service-specific configuration being separate layers that compose into a final manifest. + +DCM's layering model enables 36 layer definitions to govern 40,000 VMs without duplication โ€” impossible in the Kubernetes model. + +### 4.4 Policy Engine + +Kubernetes admission webhooks provide some policy capability (validation, mutation) but are cluster-scoped, apply at admission time only, and have no concept of hierarchy (Global โ†’ Tenant โ†’ User policy levels) or field-level override control. + +DCM's Policy Engine operates at the management plane level, applies across all clusters and providers, enforces a three-level hierarchy with field-level override control (allow/constrained/immutable), and carries policy decisions as provenance metadata in the payload. + +### 4.5 Cost Analysis + +Kubernetes has no native cost attribution model. Tools like Kubecost exist but are add-ons with no integration into the request lifecycle. + +DCM's cost analysis is built into the lifecycle model โ€” cost attribution is tracked from request time through realization, operation, and decommission for every entity. + +### 4.6 Information Providers + +Kubernetes has no concept of structured relationships to external organizational data (Business Units, Cost Centers, Product Owners). Labels and annotations are unstructured key-value pairs with no type safety, no external system integration, and no verification model. + +DCM's Information Provider model gives every entity structured, verified, versioned relationships to external organizational data with a stable external key model. + +### 4.7 Cross-Cluster Lifecycle + +Kubernetes manages resources within a single cluster. Multi-cluster management requires additional tools (ACM, Argo CD, Fleet) that are not part of the core Kubernetes model. + +DCM manages the lifecycle of resources across multiple clusters as a first-class capability โ€” the same Resource Type can be instantiated on any cluster that has a conformant Service Provider registered. + +--- + +## 5. Standard Kubernetes Resource Type Mappings + +These are the DCM Resource Type registry entries for standard Kubernetes resource types. Operators implementing these types should use these registry UUIDs and field definitions. + +### 5.1 Compute + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Compute.Pod` | Pod | Lowest-level compute unit | +| `Compute.Container` | Container (within a Pod) | Sub-entity of Pod โ€” expanded via bundled declaration | +| `Compute.Deployment` | Deployment | Managed set of Pods | +| `Compute.StatefulSet` | StatefulSet | Stateful managed set of Pods | +| `Compute.Job` | Job | One-time execution workload | +| `Compute.CronJob` | CronJob | Scheduled execution workload | + +### 5.2 Network + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Network.Service` | Service | In-cluster service discovery and load balancing | +| `Network.Ingress` | Ingress | External HTTP/HTTPS routing | +| `Network.NetworkPolicy` | NetworkPolicy | In-cluster network isolation | + +### 5.3 Storage + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Storage.PersistentVolume` | PersistentVolume | Cluster-level storage resource | +| `Storage.PersistentVolumeClaim` | PersistentVolumeClaim | Consumer's storage declaration โ€” expanded into Storage.PersistentVolume relationship | +| `Storage.StorageClass` | StorageClass | Storage type definition โ€” maps to DCM Provider Catalog Item | +| `Storage.ConfigMap` | ConfigMap | Configuration data storage | +| `Storage.Secret` | Secret | Sensitive data storage | + +### 5.4 Platform + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Platform.KubernetesCluster` | Kubernetes Cluster (via CAPI or managed service) | The cluster itself is a DCM-managed resource | +| `Platform.Namespace` | Namespace | Maps to DCM Tenant boundary in per_tenant strategy | +| `Platform.CustomResourceDefinition` | CRD | CRD registration maps to DCM Resource Type registration | + +### 5.5 Identity + +| DCM Resource Type | Kubernetes Equivalent | Notes | +|------------------|----------------------|-------| +| `Security.ServiceAccount` | ServiceAccount | Kubernetes identity for workloads | +| `Security.Role` | Role / ClusterRole | Kubernetes RBAC role | +| `Security.RoleBinding` | RoleBinding / ClusterRoleBinding | Kubernetes RBAC binding | + +--- + +## 6. The Kubernetes Information Provider + +Kubernetes clusters function as both Service Providers (for provisioning resources) and Information Providers (for querying existing state). As an Information Provider, a Kubernetes cluster exposes its current resource state to DCM for: + +- **Brownfield ingestion** โ€” discovering existing resources and bringing them under DCM lifecycle management +- **Discovered State** โ€” DCM's Discovered State for Kubernetes resources comes from querying the Kubernetes API +- **Drift detection** โ€” comparing DCM Realized State against what Kubernetes actually has + +### 6.1 Kubernetes as Information Provider Registration + +```yaml +information_provider_registration: + name: kubernetes-cluster-01 + implements: + - information_type: Platform.KubernetesCluster + - information_type: Compute.Pod + - information_type: Storage.PersistentVolume + # ... all resource types the cluster contains + endpoint: + kubernetes_credentials: + auth_method: + discovery_capabilities: + label_selector: "dcm-managed=true" + # Only returns DCM-managed resources by default + full_discovery: true + # Can also return all resources for brownfield ingestion +``` + +### 6.2 Discovered State from Kubernetes + +DCM queries the Kubernetes API using the Kubernetes Information Provider to populate Discovered State: + +``` +DCM Drift Detection + โ”‚ + โ–ผ +Kubernetes Information Provider + โ”‚ GET /apis/{group}/{version}/namespaces/{ns}/{kind} + โ”‚ Filter: label dcm-entity-id = {entity_uuid} + โ–ผ +Discovered State payload (DCM format) + โ”‚ Kubernetes object denaturalized to DCM format + โ–ผ +Compare against Realized State + โ”‚ Field-by-field comparison + โ–ผ +UNSANCTIONED_CHANGE if differences found + โ”‚ Reported to Policy Engine for response determination +``` + +--- + +## 7. Kubernetes-Native Patterns and DCM Equivalents + +### 7.1 GitOps + +Kubernetes GitOps (Argo CD, Flux) manages Kubernetes manifests in Git and synchronizes them to clusters. DCM's data model is also Git-based โ€” all layers, Resource Type definitions, and policy definitions are stored in Git. + +The relationship: DCM manages the **request lifecycle** (what gets asked for, approved, and provisioned). GitOps manages the **deployment lifecycle** (what gets deployed to a cluster from a Git repository). These are complementary: + +- DCM governs the provisioning request โ€” "is this consumer allowed to provision this resource?" +- GitOps deploys application code to the provisioned resource +- DCM and GitOps together form a complete lifecycle: DCM provisions the cluster, GitOps deploys applications to it + +### 7.2 Helm + +Helm charts are packages of Kubernetes manifests that can be parameterized. In DCM terms, a Helm chart is a form of Catalog Item โ€” a curated, parameterized offering of a set of Kubernetes resources. + +DCM does not replace Helm โ€” it can use Helm as a delivery mechanism inside a Service Provider. The Service Provider receives the DCM Requested State, translates it to Helm values, and uses Helm to deploy the resources. The operator pattern is preferred for Day 2 management (Helm has limited reconciliation), but Helm remains valid for initial provisioning. + +### 7.3 Cluster API (CAPI) + +CAPI is the Kubernetes sub-project for managing Kubernetes clusters themselves using the Kubernetes API and operator pattern. CAPI clusters are a natural fit for DCM's `Platform.KubernetesCluster` Resource Type โ€” a CAPI-based operator would be the Service Provider for provisioning new Kubernetes clusters as DCM-managed resources. + +This is particularly significant: DCM managing the lifecycle of Kubernetes clusters through CAPI means DCM can provision the very infrastructure that operators run on. The superset relationship becomes concrete โ€” DCM provisions the cluster, the cluster runs the operators, the operators provision the resources that DCM manages. + +--- + +## 8. Migration Path โ€” Kubernetes-Native to DCM-Managed + +Organizations running Kubernetes today can adopt DCM incrementally: + +### Phase 1 โ€” Observation (no operator changes) +Deploy DCM with the Kubernetes Information Provider. DCM observes existing resources via the Kubernetes API and builds a Discovered State inventory. No changes to existing operators or workloads. + +### Phase 2 โ€” Brownfield Ingestion (no operator changes) +DCM promotes Discovered State records to Realized State โ€” assuming lifecycle management of existing resources. Resources get DCM UUIDs, Tenant assignments, and provenance records. Existing resources are now DCM-managed without any operator changes. + +### Phase 3 โ€” Level 1 Conformance (minimal operator changes) +Operators implement Level 1 of this specification via the DCM Operator SDK. New resources are provisioned through DCM's service catalog. Existing resources managed via brownfield ingestion continue as-is. + +### Phase 4 โ€” Level 2 Conformance (moderate operator changes) +Operators implement Level 2 โ€” full field mappings, capacity reporting, lifecycle events. DCM gains placement intelligence, drift detection, and cross-cluster management capabilities. + +### Phase 5 โ€” Level 3 Conformance (complete integration) +Operators implement Level 3 โ€” sovereignty declarations, provenance, discovery endpoint. Full DCM capabilities available. + +--- + +## 9. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | How does the Namespace-to-Tenant mapping work when a cluster has existing namespaces that predate DCM adoption? | Brownfield migration | โœ… Resolved | +| 2 | Should `Platform.KubernetesCluster` be the boundary for a DCM deployment, or can DCM manage resources across clusters without treating the cluster as a DCM entity? | Architecture scope | โœ… Resolved | +| 3 | How does DCM interact with Kubernetes admission webhooks โ€” do they duplicate Policy Engine functions or complement them? | Policy model | โœ… Resolved | +| 4 | Should the Kubernetes Information Provider be a built-in DCM component or a separately deployed provider? | Deployment architecture | โœ… Resolved | +| 5 | How does the DCM superset model interact with managed Kubernetes services (EKS, GKE, AKS) where cluster management is outside the user's control? | Cloud provider integration | โœ… Resolved | + +--- + +## 10. Related Concepts + +- **DCM Operator Interface Specification** โ€” the technical contract for operators integrating with DCM +- **DCM Operator SDK** โ€” Go library implementing this specification for operator developers +- **Entity Relationships** โ€” DCM's universal relationship model, of which Kubernetes ownerReferences are a subset +- **Resource Type Hierarchy** โ€” the DCM registry where Kubernetes Resource Types are registered +- **Information Providers** โ€” the DCM model for the Kubernetes API as a discoverable information source +- **Four States** โ€” DCM's Intent/Requested/Realized/Discovered model, which extends Kubernetes' desired/actual model + +--- + + + +## Resolution Notes + +**Q1:** Pre-existing namespaces are handled by the brownfield ingestion model. Each namespace maps to one DCM Tenant. Resources without clear ownership land in the `__transitional__` Tenant and are promoted by a platform admin. Same flow as brownfield VM ingestion โ€” no special handling required. + +**Q2:** DCM manages resources across multiple clusters simultaneously. `Platform.KubernetesCluster` is a DCM-managed resource type โ€” both something DCM provisions as a catalog item (Cluster as a Service) and something DCM tracks when externally provisioned. A Tenant can own a full cluster as a catalog item; the cluster is not the boundary of a DCM deployment. DCM's organizational boundary is the Tenant. A single DCM deployment routes requests to Service Providers across many clusters, and can provision new clusters as service catalog items. + +**Q3:** Admission webhooks and the DCM Policy Engine are complementary layers, not duplicates. Admission webhooks enforce cluster-native policy (security contexts, image policies, resource quotas). The DCM Policy Engine enforces DCM request policy (business rules, data governance, sovereignty). A DCM-managed workload resource is validated by both โ€” DCM Policy Engine before dispatch, admission webhook at the cluster. This is defense in depth. + +**Q4:** The Kubernetes Information Provider is a separately deployed provider that registers with DCM as a standard Information Provider. It serves cluster state, namespace inventory, and workload status. There are no built-in Information Providers in DCM's architecture โ€” all Information Providers follow the unified base contract and are independently deployable. + +**Q5:** Managed Kubernetes services (EKS, GKE, AKS) register as Service Providers of resource type `Platform.ManagedKubernetesCluster`. DCM manages workload resources within the cluster (Deployments, Services, PersistentVolumes) but explicitly does not manage the cluster control plane. Sovereignty enforcement applies at cluster selection โ€” DCM places workloads on clusters satisfying sovereignty constraints. The cloud provider manages cluster infrastructure. + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/cncf-strategy.md b/content/docs/architecture/specifications/cncf-strategy.md index 88ad0ce..9c1558e 100644 --- a/content/docs/architecture/specifications/cncf-strategy.md +++ b/content/docs/architecture/specifications/cncf-strategy.md @@ -1,8 +1,4 @@ ---- -title: "CNCF Strategy" -type: docs -weight: 7 ---- +# DCM โ€” CNCF Strategy and Community Engagement Plan > **โš ๏ธ Active Development Notice** > @@ -246,3 +242,16 @@ The following artifacts must be ready before a CNCF Sandbox submission is credib **Q5:** Level 2 conformance requires: full dispatch/cancel/discover cycle, full realized state reporting, governance matrix enforcement at the provider boundary, and health check compliance. These requirements are now formally defined in the Operator Interface Specification. The project team estimates timeline based on available engineering resources against this defined scope. *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + +## Red Hat Developer Hub / Backstage Integration + +DCM implements a Backstage plugin suite (`@dcm/backstage-plugin-*`) for deployment as RHDH Dynamic Plugins. This is the primary consumer-facing deployment model. See [RHDH Integration Specification](dcm-rhdh-integration-spec.md) for the complete architecture. + +**CNCF alignment:** Backstage is a CNCF incubating project. DCM's RHDH integration follows CNCF best practices for developer portals and internal developer platforms (IDPs). + +| Component | CNCF Status | DCM Use | +|-----------|------------|---------| +| Backstage | Incubating | Primary consumer GUI platform | +| Backstage Software Templates | Backstage feature | Auto-generated from DCM catalog items | +| Backstage Catalog | Backstage feature | DCMService and DCMResource entity kinds | +| Backstage Permission Framework | Backstage feature | DCM role โ†’ Backstage permission bridge | diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md index c4a9ab2..94f8074 100644 --- a/content/docs/architecture/specifications/consumer-api-spec.md +++ b/content/docs/architecture/specifications/consumer-api-spec.md @@ -1,8 +1,4 @@ ---- -title: "DCM Consumer API Specification" -type: docs -weight: 0 ---- +# DCM Consumer API Specification > **๐Ÿ“‹ Draft** > diff --git a/content/docs/architecture/specifications/dcm-admin-api-spec.md b/content/docs/architecture/specifications/dcm-admin-api-spec.md new file mode 100644 index 0000000..7e684c7 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-admin-api-spec.md @@ -0,0 +1,973 @@ +# DCM Admin API Specification + +> **๐Ÿ“‹ Draft** +> +> This specification has been promoted from Work in Progress to Draft status. Complete Admin API covering all platform admin operations with request/response examples. It is ready for implementation feedback but has not yet been formally reviewed for final release. +> +> This specification defines the DCM Admin API โ€” the platform administration interface. Published to share design direction and invite feedback. Do not build production integrations against this specification until it reaches draft status. + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Ready for implementation feedback +**Document Type:** Technical Specification +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Consumer API Specification](consumer-api-spec.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) + +--- + +## Abstract + +The Admin API is the platform administration interface for DCM. It is served through the same Ingress API as the Consumer API and Provider API but is restricted to actors with `platform_admin` or `tenant_admin` roles. It covers operations that consumers cannot perform โ€” Tenant lifecycle management, provider registration review, accreditation approval, quota administration, discovery management, orphan resolution, recovery decision escalation, and bootstrap operations. + +--- + +## 1. Authentication and Authorization + +All Admin API endpoints require Bearer token authentication (same as Consumer API). Role requirements are declared per endpoint: + +| Role | Scope | +|------|-------| +| `platform_admin` | All Admin API operations across all Tenants | +| `tenant_admin` | Tenant-scoped Admin API operations for their own Tenant only | + +Base URL: `/api/v1/admin/` + +> **Versioning:** See [API Versioning Strategy](../data-model/34-api-versioning-strategy.md). Breaking changes increment the major version. The Admin API follows the same deprecation lifecycle as the Consumer API, with profile-governed support windows. + +Step-up MFA is required for destructive operations (Tenant decommission, accreditation revocation, bootstrap credential rotation) regardless of session MFA status. + +--- + +### 1.1 Rate Limiting + +Admin API endpoints have separate rate limits from the Consumer API, applied per authenticated admin actor: + +| Profile | Requests/minute | Burst | +|---------|----------------|-------| +| All profiles | 120 | 40 | + +Rate-limited responses include `Retry-After`, `X-RateLimit-Limit`, `X-RateLimit-Remaining` headers. + +### 1.2 Request and Correlation IDs + +All responses include `X-DCM-Request-ID` and `X-DCM-Correlation-ID` headers (same model as Consumer API). + +### 1.3 Response Envelopes + +List responses use `{"items": [...], "total": N, "next_cursor": "..."}`. Single resources returned directly. Errors use `{"error": "...", "message": "...", "request_id": "..."}`. + +--- + +## 2. Tenant Management + +### 2.1 List Tenants + +``` +GET /api/v1/admin/tenants +Role: platform_admin + +Query params: status=, page, page_size + +Response 200: +{ + "tenants": [ + { + "tenant_uuid": "", + "handle": "payments-team", + "display_name": "Payments Platform", + "status": "active", + "deployment_posture": "prod", + "compliance_domains": ["hipaa"], + "recovery_profile": "notify-and-wait", + "entity_count": 142, + "created_at": "" + } + ], + "total": 12 +} +``` + +### 2.2 Create Tenant + +``` +POST /api/v1/admin/tenants +Role: platform_admin + +{ + "handle": "new-team", + "display_name": "New Team", + "deployment_posture": "standard", + "compliance_domains": [], + "recovery_profile_override": null, + "initial_admin_actor_uuid": "" +} + +Response 201 Created: +{ + "tenant_uuid": "", + "status": "active" +} +``` + +### 2.3 Suspend / Reinstate Tenant + +``` +POST /api/v1/admin/tenants/{tenant_uuid}/suspend +POST /api/v1/admin/tenants/{tenant_uuid}/reinstate +Role: platform_admin + +{ + "reason": "", + "notify_tenant_admin": true +} +``` + +### 2.4 Decommission Tenant + +``` +DELETE /api/v1/admin/tenants/{tenant_uuid} +Role: platform_admin +Requires: step-up MFA + +{ + "reason": "", + "force": false, # true: decommission even if active entities remain + "notify_tenant_admin": true +} + +Response 409 Conflict (if active entities and force=false): +{ + "error": "tenant_has_active_entities", + "active_entity_count": 47, + "resolution": "Decommission all entities first, or use force=true" +} +``` + +--- + +## 3. Provider Management + +### 3.1 List Registered Providers + +``` +GET /api/v1/admin/providers +Role: platform_admin + +Query params: type=, status= + +Response 200: +{ + "providers": [ + { + "provider_uuid": "", + "handle": "eu-west-prod-1", + "provider_type": "service", + "status": "active", + "health": "healthy", + "accreditation_count": 2, + "max_data_classification": "phi" + } + ] +} +``` + +### 3.2 Review Provider Registration + +New provider registrations in `proposed` status require platform admin review: + +``` +GET /api/v1/admin/providers/pending +Role: platform_admin + +POST /api/v1/admin/providers/{provider_uuid}/approve +POST /api/v1/admin/providers/{provider_uuid}/reject +{ + "reason": "" +} +``` + +### 3.3 Suspend Provider + +``` +POST /api/v1/admin/providers/{provider_uuid}/suspend +Role: platform_admin + +{ + "reason": "", + "affect_existing_entities": "notify_only | block_new_requests | migrate" +} +``` + +--- + +## 4. Accreditation Management + +### 4.1 List Accreditations + +``` +GET /api/v1/admin/accreditations +Role: platform_admin + +Query params: subject_type, framework, status= + +Response 200: +{ + "accreditations": [ + { + "accreditation_uuid": "", + "subject_uuid": "", + "subject_type": "service_provider", + "framework": "hipaa", + "accreditation_type": "baa", + "status": "active", + "valid_until": "", + "days_until_expiry": 89 + } + ] +} +``` + +### 4.2 Approve Accreditation + +``` +POST /api/v1/admin/accreditations/{accreditation_uuid}/approve +Role: platform_admin +Requires: step-up MFA + +{ + "review_notes": "", + "certificate_verified": true +} +``` + +### 4.3 Revoke Accreditation + +``` +DELETE /api/v1/admin/accreditations/{accreditation_uuid} +Role: platform_admin +Requires: step-up MFA + +{ + "revocation_reason": "", + "affected_entity_action": "notify_only | block_new_requests | migrate_entities" +} +``` + +--- + +## 5. Discovery Management + +### 5.1 Trigger Discovery + +``` +POST /api/v1/admin/discovery/trigger +Role: platform_admin | tenant_admin + +{ + "scope": "entity | resource_type | provider | tenant", + "entity_uuid": "", + "resource_type": "Compute.VirtualMachine", + "provider_uuid": "", + "tenant_uuid": "", + "reason": "incident investigation", + "priority": "high | standard | background" +} + +Response 202 Accepted: +{ + "discovery_job_uuid": "", + "status": "queued", + "priority": "high", + "estimated_start": "" +} +``` + +### 5.2 Discovery Job Status + +``` +GET /api/v1/admin/discovery/jobs/{discovery_job_uuid} + +Response 200: +{ + "discovery_job_uuid": "", + "status": "running | completed | failed", + "entities_discovered": 47, + "new_entities_found": 2, + "started_at": "", + "completed_at": "", + "orphan_candidates_found": 1 +} +``` + +--- + +## 6. Orphan Management + +### 6.1 List Orphan Candidates + +``` +GET /api/v1/admin/orphans +Role: platform_admin + +Query params: provider_uuid, status= + +Response 200: +{ + "orphan_candidates": [ + { + "orphan_candidate_uuid": "", + "provider_uuid": "", + "provider_entity_id": "vm-0a1b2c3d", + "suspected_request_uuid": "", + "resource_type": "Compute.VirtualMachine", + "discovered_at": "", + "status": "under_review" + } + ] +} +``` + +### 6.2 Resolve Orphan Candidate + +``` +POST /api/v1/admin/orphans/{orphan_candidate_uuid}/resolve +Role: platform_admin + +{ + "resolution": "manual_decommission | adopt_into_dcm | mark_false_positive", + "reason": "", + "target_tenant_uuid": "" # required if resolution=adopt_into_dcm +} +``` + +--- + +## 7. Recovery Decision Management + +Platform admins can resolve pending recovery decisions for any entity: + +``` +GET /api/v1/admin/recovery-decisions/pending +Role: platform_admin + +Response 200: +{ + "pending_decisions": [ + { + "recovery_decision_uuid": "", + "entity_uuid": "", + "trigger": "DISPATCH_TIMEOUT", + "entity_state": "TIMEOUT_PENDING", + "deadline": "", + "tenant_uuid": "" + } + ] +} + +POST /api/v1/admin/recovery-decisions/{recovery_decision_uuid} +Role: platform_admin + +{ + "action": "DRIFT_RECONCILE | DISCARD_AND_REQUEUE | DISCARD_NO_REQUEUE", + "reason": "" +} +``` + +--- + +## 8. Quota Management + +### 8.1 View Tenant Quotas + +``` +GET /api/v1/admin/tenants/{tenant_uuid}/quotas +Role: platform_admin | tenant_admin + +Response 200: +{ + "quotas": [ + { + "resource_type": "Compute.VirtualMachine", + "limit": 100, + "current_usage": 47, + "policy_uuid": "" + } + ] +} +``` + +### 8.2 Update Quota + +``` +PUT /api/v1/admin/tenants/{tenant_uuid}/quotas/{resource_type} +Role: platform_admin + +{ + "new_limit": 150, + "reason": "Q2 capacity increase approved by FinOps" +} +``` + +--- + +## 9. Search Index Management + +``` +POST /api/v1/admin/search-index/rebuild +Role: platform_admin + +{ + "scope": "full | tenant | resource_type", + "tenant_uuid": "", + "reason": "Recovery after index corruption" +} + +Response 202 Accepted: +{ + "rebuild_job_uuid": "", + "estimated_duration": "PT2H", + "degraded_during_rebuild": true +} + +GET /api/v1/admin/search-index/status + +Response 200: +{ + "status": "healthy | degraded | rebuilding | unavailable", + "staleness_seconds": 42, + "last_full_rebuild": "", + "entity_count": 8421 +} +``` + +--- + +## 10. Bootstrap Operations + +### 10.1 Rotate Bootstrap Admin Credential + +``` +POST /api/v1/admin/bootstrap/rotate-credential +Role: platform_admin +Requires: step-up MFA (hardware_token_mfa for fsi/sovereign) + +{ + "new_credential_ref": "", + "reason": "Initial bootstrap credential rotation" +} +``` + +### 10.2 Deployment Health + +``` +GET /api/v1/admin/health + +Response 200: +{ + "overall": "healthy | degraded | critical", + "components": [ + { "component": "request_orchestrator", "status": "healthy" }, + { "component": "policy_engine", "status": "healthy" }, + { "component": "placement_engine", "status": "healthy" }, + { "component": "lifecycle_constraint_enforcer", "status": "healthy" }, + { "component": "discovery_scheduler", "status": "healthy" }, + { "component": "notification_router", "status": "healthy" }, + { "component": "cost_analysis", "status": "healthy" }, + { "component": "search_index", "status": "degraded", "staleness_seconds": 180 }, + { "component": "intent_store", "status": "healthy" }, + { "component": "requested_store", "status": "healthy" }, + { "component": "realized_store", "status": "healthy" } + ], + "active_profile": { + "deployment_posture": "prod", + "compliance_domains": ["hipaa"], + "recovery_posture": "notify-and-wait", + "zero_trust_posture": "full" + } +} +``` + +--- + +## 13. DCM Self-Health Endpoints + +DCM exposes three health endpoints, each with a distinct purpose: + +```http +# Liveness โ€” is the process alive? (Kubernetes liveness probe) +GET /livez +# No auth required. Max response time: PT5S. +# Returns 200 OK with {"status":"ok"} if alive. +# Returns 503 if process is deadlocked or unresponsive. + +# Readiness โ€” is DCM ready to serve traffic? (Kubernetes readiness probe) +GET /readyz +# No auth required. Max response time: PT10S. +# Returns 200 OK with {"status":"ready"} if all required stores are reachable. +# Returns 503 with {"status":"not_ready","reasons":["store_unreachable"]} otherwise. + +# Operational health โ€” rich health for operators and monitoring systems +GET /api/v1/admin/health +Authorization: Bearer + +Response 200: +{ + "dcm_version": "", + "profile": "prod", + "status": "healthy", // healthy | degraded | critical + "components": { + "request_orchestrator": { "status": "healthy" }, + "policy_engine": { "status": "healthy" }, + "placement_engine": { "status": "healthy" }, + "credential_provider": { "status": "degraded", "reason": "rotation_pending" } + }, + "stores": { + "intent_store": { "status": "healthy", "latency_p99_ms": 12 }, + "requested_store": { "status": "healthy", "latency_p99_ms": 8 }, + "realized_store": { "status": "healthy", "latency_p99_ms": 9 } + }, + "providers": { + "total": 4, + "healthy": 3, + "degraded": 1, + "unhealthy": 0 + } +} + +# Prometheus metrics +GET /metrics +# Unauthenticated (secured by network policy in production). +# Returns Prometheus text format metrics. +``` + +> **Full model:** See [DCM Self-Health](../data-model/39-dcm-self-health.md) โ€” HLT-001โ€“HLT-006. + + +## 12. Session Management (Admin) + +Platform admins can force-revoke sessions for any actor โ€” used on actor compromise, policy violation, or deprovisioning. + +```http +# Force-revoke all sessions for an actor +POST /api/v1/admin/actors/{actor_uuid}/revoke-sessions +Authorization: Bearer + +{ + "reason": "security_event", // REQUIRED + "notify_actor": true // send notification event +} + +Response 202 Accepted: +{ + "sessions_revoked": 3, + "actor_uuid": "", + "revocation_propagated_at": "" +} +``` + +```http +# List active sessions for any actor (admin view) +GET /api/v1/admin/actors/{actor_uuid}/sessions +Authorization: Bearer + +Response 200: +{ + "items": [ + { + "session_uuid": "", + "created_at": "", + "expires_at": "", + "auth_method": "ldap", + "mfa_verified": true, + "status": "active" + } + ], + "total": 1 +} +``` + +**Error codes specific to session management:** + +| Error Code | HTTP | When | +|-----------|------|------| +| `actor_not_found` | 404 | Actor UUID not found | +| `no_active_sessions` | 404 | Actor has no active sessions | + +> **Full model:** See [Session Token Revocation](../data-model/35-session-revocation.md) โ€” AUTH-016โ€“AUTH-022. + + +## 11. Error Model + +All Admin API errors use the same envelope as the Consumer API: + +```json +{ + "error": "", // machine-readable snake_case code + "message": "", // human-readable description + "request_id": "", // matches X-DCM-Request-ID header + "details": {} // optional: field-level details +} +``` + +**Admin-specific error codes:** + +| Error Code | HTTP Status | When | +|-----------|-------------|------| +| `insufficient_admin_role` | 403 | Actor lacks required admin role | +| `tenant_not_found` | 404 | Tenant UUID not found | +| `provider_not_found` | 404 | Provider UUID not found | +| `approval_already_voted` | 409 | Actor has already voted on this approval | +| `approval_window_expired` | 410 | Approval window has passed | +| `degradation_already_accepted` | 409 | Degradation item already accepted | +| `tier_registry_blocked` | 409 | Registry change has unresolved blocking items | +| `quota_below_current_usage` | 422 | New quota would be below current consumption | + +All error responses include `X-DCM-Request-ID` and `X-DCM-Correlation-ID` headers. + + +## Scoring Model Administration + +> Approval routing thresholds use named-tier dynamic format. See [Authority Tier Model](../data-model/32-authority-tier-model.md) for the complete specification. + +### Get Scoring Thresholds for Profile + +``` +GET /api/v1/admin/profiles/{profile_name}/scoring + +Response 200: +{ + "profile": "standard", + "scoring_thresholds": { + "auto_approve_below": 25, + "approval_routing": [ + { "tier": "reviewed", "max_score": 59 }, + { "tier": "verified", "max_score": 79 }, + { "tier": "authorized", "max_score": 100 } + ] + }, + "signal_weights": { + "operational_gatekeeper": 0.45, + "completeness": 0.15, + "actor_risk_history": 0.20, + "quota_pressure": 0.10, + "provider_risk": 0.10 + }, + "policy_enforcement_overrides": [] +} +``` + +### Update Scoring Thresholds + +``` +PATCH /api/v1/admin/profiles/{profile_name}/scoring +{ + "scoring_thresholds": { + "auto_approve_below": 20, + "approval_routing": [ + { "tier": "reviewed", "max_score": 59 }, + { "tier": "verified", "max_score": 79 }, + { "tier": "authorized", "max_score": 100 } + ] + } +} + +Response 200: { "profile": "standard", "updated_at": "", "effective_immediately": true } +Response 422: { "error": "threshold_invalid", "reason": "auto_approve_below exceeds maximum of 50 (SMX-008)" } +``` + +### Add Policy Enforcement Override + +``` +POST /api/v1/admin/profiles/{profile_name}/scoring/overrides +{ + "policy_handle": "platform/gatekeeper/cpu-size-limit", + "override_enforcement_class": "compliance", + "rationale": "Prod profile: CPU limit is a hard constraint", + "applies_to_resource_types": ["Compute.VirtualMachine"] +} + +Response 201 Created: +{ "override_uuid": "", "policy_handle": "...", "effective_immediately": true } +``` + +### Actor Risk History + +``` +GET /api/v1/admin/actors/{actor_uuid}/risk-history + +Response 200: +{ + "actor_uuid": "", + "current_score": 30, + "events": [ + { + "event_type": "validation_failure", + "occurred_at": "", + "request_uuid": "", + "base_contribution": 5, + "decayed_contribution": 3.2, + "days_ago": 4 + } + ], + "decay_lambda": 0.1, + "score_half_life_days": 7 +} + +POST /api/v1/admin/actors/{actor_uuid}/risk-history/reset +{ + "reason": "Actor confirmed as trusted automation account", + "audit_note": "Reviewed and approved by platform admin" +} +``` + +### Score Audit Trail + +``` +GET /api/v1/admin/scoring/audit + +Query parameters: + from= + to= + routing_decision= + risk_score_above= + actor_uuid= + resource_type= + +Response 200: +{ + "score_records": [ + { + "score_record_uuid": "", + "request_uuid": "", + "risk_score": 47, + "routing_decision": "reviewed", + "signal_breakdown": { ... }, + "evaluated_at": "" + } + ] +} +``` + + +--- + +## Approval Management + +DCM provides approval gates for requests, policy contributions, provider registrations, and federation contributions. The Admin API is the integration point for recording decisions โ€” it is designed to be called by both human reviewers in the DCM UI and by external systems (ServiceNow, Jira, Slack bots, workflow automation). + +### List Pending Approvals + +``` +GET /api/v1/admin/approvals/pending + +Query parameters: + approval_type= + tier= + reviewer_uuid= # approvals where this actor is an eligible reviewer + +Response 200: +{ + "pending_approvals": [ + { + "approval_uuid": "", + "approval_type": "policy_contribution", + "tier": "authorized", + "subject_uuid": "", + "subject_handle": "tenant/payments/gatekeeper/cost-ceiling", + "required_dcmgroup_uuid": "", # for authorized tier + "quorum_required": 3, + "votes_recorded": 1, + "submitted_at": "", + "window_expires_at": "", + "submitted_by": { "uuid": "", "display_name": "Bob Smith" } + } + ] +} +``` + +### Record an Approval Decision + +``` +POST /api/v1/admin/approvals/{approval_uuid}/vote + +{ + "decision": "approve | reject", + "reason": "", + "recorded_via": "dcm_admin_ui | servicenow | jira | slack_bot | api_direct | other", + "external_reference": "" +} + +Response 200: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 2, + "quorum_required": 3, + "quorum_reached": false, + "pipeline_status": "pending_authorized" +} + +# When quorum is reached or reviewed/verified satisfied: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 3, + "quorum_required": 3, + "quorum_reached": true, + "pipeline_status": "activating" +} + +Response 403: actor is not a member of the required authority group (authorized tier) or not in reviewer role +Response 409: actor has already voted on this approval (verified and authorized tiers enforce distinct voters) +Response 410: approval window has expired +``` + +### Get Approval Detail + +``` +GET /api/v1/admin/approvals/{approval_uuid} + +Response 200: +{ + "approval_uuid": "", + "approval_type": "authorized", + "subject_uuid": "", + "tier": "authorized", + "required_dcmgroup_uuid": "", + "quorum_required": 3, + "window_expires_at": "", + "votes": [ + { + "voter_uuid": "", + "voter_display_name": "Alice Chen", + "decision": "approve", + "recorded_at": "", + "recorded_via": "servicenow", + "external_reference": "CHG0012345" + } + ], + "status": "pending_authorized", + "quorum_reached": false +} +``` + + +--- + +## Authority Tier Registry Management + +> **Implementation note:** The tier registry change impact detection pipeline is specified in [Authority Tier Model](../data-model/32-authority-tier-model.md) Section 7. The endpoints below are the Admin API surface for proposing, reviewing, and activating tier registry changes. The detection mechanism (tier impact diff computation, affected item query, degradation gate) is an implementation responsibility. + +### Propose a Tier Registry Change + +``` +POST /api/v1/admin/tier-registry/changes + +{ + "proposed_tiers": [ + { "name": "auto", "insert_after": null, "decision_gravity": "none" }, + { "name": "reviewed", "insert_after": "auto", "decision_gravity": "routine" }, + { "name": "verified", "insert_after": "reviewed", "decision_gravity": "elevated" }, + { "name": "compliance_reviewed", "insert_after": "verified", "decision_gravity": "elevated" }, + { "name": "authorized", "insert_after": "compliance_reviewed", "decision_gravity": "critical" } + ], + "reason": "Adding compliance_reviewed tier for PCI-DSS regulated actions" +} + +Response 202 Accepted: +{ + "registry_change_uuid": "", + "status": "impact_assessment_pending", + "estimated_ready_at": "" +} +``` + +### Get Tier Registry Impact Report + +``` +GET /api/v1/admin/tier-registry/changes/{change_uuid}/impact + +Response 200: +{ + "registry_change_uuid": "", + "status": "impact_assessed | pending_degradation_review | ready_to_activate | blocked", + "summary": { + "degradations": 0, + "upgrades": 3, + "new_tiers": 1, + "broken_references": 0, + "profile_gaps": 2 + }, + "degradations": [], + "upgrades": [ ... ], + "profile_gaps": [ + { + "profile": "standard", + "missing_tiers": ["compliance_reviewed"], + "gap_effect": "Requests scoring in the compliance_reviewed range will route to verified tier until threshold list is updated" + } + ], + "blocking_items": [] +} +``` + +### Accept a Security Degradation + +``` +POST /api/v1/admin/tier-registry/changes/{change_uuid}/accept-degradation + +{ + "affected_item_uuid": "", + "affected_item_type": "provider_registration_requirement", + "acceptance_reason": "", + "accepted_by": "" +} + +Response 200: +{ + "acceptance_uuid": "", + "degradation_accepted": true, + "remaining_degradations": 0, + "change_status": "ready_to_activate" +} + +Response 403: actor does not hold verified or authorized tier reviewer role +Response 409: degradation already accepted +``` + +### Activate a Tier Registry Change + +``` +POST /api/v1/admin/tier-registry/changes/{change_uuid}/activate + +Response 200: +{ + "registry_change_uuid": "", + "activated_at": "", + "new_registry_version": "1.1.0", + "impact_report_uuid": "" +} + +Response 409: change has unresolved blocking items (broken_references or unaccepted degradations) +``` + +### List Historical Registry Changes + +``` +GET /api/v1/admin/tier-registry/changes?status=activated&limit=20 + +Response 200: +{ + "changes": [ + { + "registry_change_uuid": "", + "status": "activated", + "activated_at": "", + "proposed_by": { "uuid": "", "display_name": "Alice Chen" }, + "summary": { "degradations": 0, "upgrades": 2, "new_tiers": 1 }, + "impact_report_uuid": "" + } + ] +} +``` + diff --git a/content/docs/architecture/specifications/dcm-admin-gui-spec.md b/content/docs/architecture/specifications/dcm-admin-gui-spec.md new file mode 100644 index 0000000..f21c7a4 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-admin-gui-spec.md @@ -0,0 +1,250 @@ +# DCM Admin Web GUI Specification + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Specification โ€” Platform Administration Interface +**Related Documents:** [Admin API Specification](dcm-admin-api-spec.md) | [Consumer GUI Specification](dcm-consumer-gui-spec.md) | [Provider GUI Specification](dcm-provider-gui-spec.md) | [Flow GUI Specification](dcm-flow-gui-spec.md) + +> **Status:** Draft โ€” Ready for implementation feedback +> +> The Admin GUI is the platform operations console for Platform Admins, SREs, Policy Owners, Security teams, and Auditors. It wraps the Admin API and exposes all platform management capabilities. Like the Consumer GUI, it participates in the same session model โ€” same login, role-gated access to the admin panel. + +--- + +## 1. Architecture + +### 1.1 Unified Shell Model + +The Admin GUI is not a separate application. It is an **additional surface within the DCM web application**, revealed when the authenticated actor holds a platform-level role (`platform_admin`, `sre`, `auditor`, `security`, `policy_owner`). + +``` +DCM Web Application +โ”œโ”€โ”€ Consumer Portal (visible to all authenticated actors) +โ”‚ โ””โ”€โ”€ [section 2โ€“11 of Consumer GUI spec] +โ”‚ +โ”œโ”€โ”€ Admin Panel (visible to platform-level role holders) +โ”‚ โ””โ”€โ”€ [this spec โ€” section 2โ€“11] +โ”‚ +โ”œโ”€โ”€ Provider Management (visible to provider owner roles) +โ”‚ โ””โ”€โ”€ [Provider GUI spec โ€” dcm-provider-gui-spec.md] +โ”‚ +โ””โ”€โ”€ Flow GUI (linked / embedded for policy_owner and sre) + โ””โ”€โ”€ [dcm-flow-gui-spec.md] +``` + +Navigation adapts to the actor's highest privilege level. A Platform Admin sees all three surfaces. A consumer-only actor sees only the Consumer Portal. + +### 1.2 Authentication and Role Mapping + +The Admin GUI requires the session to carry a platform-level role. The role check is performed client-side on every page load and server-enforced by the Admin API on every request. + +| Role | Admin sections available | +|------|--------------------------| +| `platform_admin` | All Admin sections | +| `sre` | Health, Discovery, Orphan Management, Scoring, Session Management | +| `security` | Audit, Session Management (force revoke), Accreditation | +| `policy_owner` | Policy management, Approval management, Tier Registry | +| `auditor` | Audit (read-only), Scoring audit trail | +| `finops` | Quota management, Cost aggregation (if FinOps module enabled) | + +--- + +## 2. Platform Overview Dashboard + +Landing page for all platform-level roles. Data aggregated from `GET /api/v1/admin/health` and related endpoints. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ DCM Platform Health โ— All Systems โœ…โ”‚ +โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚ +โ”‚ Control Plane Providers โ”‚ +โ”‚ API Gateway โœ… pass Registered: 12 โ”‚ +โ”‚ Policy Engine โœ… pass Healthy: 11 โ”‚ +โ”‚ Scoring Engine โœ… pass Degraded: 1 โš ๏ธ โ”‚ +โ”‚ Request Sched. โœ… pass Unhealthy: 0 โ”‚ +โ”‚ Drift Reconciler โœ… pass โ”‚ +โ”‚ Auth Providers โ”‚ +โ”‚ Stores Registered: 2 โ”‚ +โ”‚ Session Store โœ… pass Healthy: 2 โ”‚ +โ”‚ Audit Store โœ… pass โ”‚ +โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โ”‚ +โ”‚ Pending Approvals: 3 ๐Ÿ”” | Open Drift Records: 7 | Orphans: 0โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +Dashboard widgets (configurable per role): +- Control plane component health grid +- Provider health summary with degraded/unhealthy callouts +- Pending approvals count (with link to approval queue) +- Open drift records by severity +- Active sessions count +- Request pipeline throughput (requests/minute, last 1 hour) +- Scheduled requests queue depth + +--- + +## 3. Tenant Management + +**API:** `GET /api/v1/admin/tenants`, `POST /api/v1/admin/tenants`, `POST /api/v1/admin/tenants/{uuid}/suspend`, `POST /api/v1/admin/tenants/{uuid}/reinstate`, `DELETE /api/v1/admin/tenants/{uuid}` + +- Tenant list with status, member count, resource count, quota utilization +- Create tenant form: name, description, initial quota set, initial admin member +- Tenant detail: members, resource count by type, quota view, active sessions count +- Suspend / reinstate / decommission with confirmation dialog requiring typed tenant name +- Tenant audit trail: all admin actions taken on this tenant + +--- + +## 4. Provider Management + +**API:** `GET /api/v1/admin/providers`, `GET /api/v1/admin/providers/pending`, `POST /api/v1/admin/providers/{uuid}/approve`, `POST /api/v1/admin/providers/{uuid}/reject`, `POST /api/v1/admin/providers/{uuid}/suspend` + +> **Full provider management** (configuration, capacity, entity lists, type-specific management) is in the **[Provider GUI](dcm-provider-gui-spec.md)**. This section covers the admin-level registration approval workflow. + +- Pending registrations list: provider type, submitter, submission time, capability declaration summary +- Registration review: full capability declaration YAML, validation result, automated checks passed/failed +- Approve / reject with comment (recorded in audit trail) +- Active providers list: health status, entity count, last health check time +- Suspend provider: warns if active entities will be affected and shows count + +--- + +## 5. Accreditation Management + +**API:** `GET /api/v1/admin/accreditations`, `POST /api/v1/admin/accreditations/{uuid}/approve`, `DELETE /api/v1/admin/accreditations/{uuid}` + +- Pending accreditations queue with submission detail +- Approve / revoke with required comment +- Accreditation expiry calendar: upcoming renewals in P90D window highlighted +- Accreditation gap detection: data classifications that require accreditations not currently held + +--- + +## 6. Discovery and Orphan Management + +**API:** `POST /api/v1/admin/discovery/trigger`, `GET /api/v1/admin/discovery/jobs/{uuid}`, `GET /api/v1/admin/orphans`, `POST /api/v1/admin/orphans/{uuid}/resolve` + +### 6.1 Discovery Console + +- Trigger on-demand discovery by resource type and provider +- Discovery job status with progress (resources scanned, new discoveries, changes detected) +- Discovery history: recent jobs with outcome summary + +### 6.2 Orphan Resolution Queue + +- List orphan candidates: resources discovered in provider that have no DCM Realized State record +- Per-orphan action: Ingest (create Realized State record), Decommission (instruct provider to delete), Ignore (mark as known-unmanaged) +- Bulk actions for same-type orphans + +--- + +## 7. Quota Management + +**API:** `GET /api/v1/admin/tenants/{uuid}/quotas`, `PUT /api/v1/admin/tenants/{uuid}/quotas/{resource_type}` + +- Per-tenant quota view: current limits vs current usage vs projected usage +- Inline edit quota values; save triggers `PUT /api/v1/admin/tenants/{uuid}/quotas/{resource_type}` +- Quota utilization heatmap across all tenants (who is using the most of what) +- Quota alert configuration: threshold for "approaching limit" notifications + +--- + +## 8. Scoring Model Administration + +**API:** `GET /api/v1/admin/profiles/{name}/scoring`, `PATCH /api/v1/admin/profiles/{name}/scoring`, `POST /api/v1/admin/profiles/{name}/scoring/overrides`, `GET /api/v1/admin/scoring/audit`, `GET /api/v1/admin/actors/{uuid}/risk-history` + +### 8.1 Profile Scoring Configuration + +- Score threshold table per profile: approval routing tiers vs score ranges +- Visual slider interface for threshold adjustment (guardrail: auto_approve_below โ‰ค 50 enforced โ€” slider hard-stops at 50) +- Signal weight editor: operational_gatekeeper (45%), completeness (15%), actor_risk_history (20%), quota_pressure (10%), provider_accreditation (10%) โ€” weights must sum to 100% +- Preview: submit a sample request to see the score it would receive under current config + +### 8.2 Policy Enforcement Override + +- Per-policy enforcement class override: escalate operational โ†’ compliance, or demote compliance โ†’ operational +- Required justification field; change recorded in audit trail +- Shadow policies table with divergence rates โ€” promotes action for high-divergence policies + +### 8.3 Actor Risk History + +- Search by actor UUID or handle +- Risk signal history timeline: what events contributed to elevated risk score +- Manual reset capability (requires `platform_admin` role + comment) + +--- + +## 9. Approval Management + +**API:** `GET /api/v1/admin/approvals/pending`, `POST /api/v1/admin/approvals/{uuid}/vote`, `GET /api/v1/admin/approvals/{uuid}` + +- All pending approvals across all tenants (Platform Admin view) vs own queue (approver view) +- Filter by tier (reviewed / verified / authorized), resource type, tenant, age +- Approval detail: request payload, risk score breakdown, policy evaluation results, existing votes +- Vote with comment; authorized-tier quorum tracker +- Expired approvals: review and optionally reopen + +--- + +## 10. Authority Tier Registry + +**API:** `POST /api/v1/admin/tier-registry/changes`, `GET /api/v1/admin/tier-registry/changes/{uuid}/impact`, `POST /api/v1/admin/tier-registry/changes/{uuid}/accept-degradation`, `POST /api/v1/admin/tier-registry/changes/{uuid}/activate` + +- Current tier registry: ordered list display (auto โ†’ reviewed โ†’ verified โ†’ authorized โ†’ [custom tiers]) +- Propose change: drag-and-drop reordering with add/remove custom tier +- Impact report: automatically fetched after proposal; displays SECURITY_DEGRADATION (red), BROKEN_REFERENCE (orange), PROFILE_GAP (yellow), SECURITY_UPGRADE (green) +- Degradation acceptance: per-item accept flow with required compensating control rationale +- Activate button disabled until all blocking items are resolved + +--- + +## 11. Audit and Compliance + +**API:** `GET /api/v1/audit/...` (admin-scoped, cross-tenant) + +Visible to `auditor` and `platform_admin` roles. + +- **Platform-wide audit trail**: all DCM actions across all tenants; filterable by actor, tenant, resource type, operation, date range +- **Compliance reports**: pre-built reports for common frameworks (SOC 2 Type II, FedRAMP, HIPAA) โ€” export to PDF/CSV +- **Audit chain integrity status**: last verification timestamp; trigger re-verification; alert on chain break +- **Cross-tenant correlation**: enter correlation ID to trace a request end-to-end across tenants and providers + +--- + +## 12. Session and Security Management + +**API:** `GET /api/v1/admin/actors/{uuid}/...`, `POST /api/v1/admin/actors/{uuid}/revoke-sessions` + +Visible to `security` and `platform_admin` roles. + +- **Active sessions**: all active sessions across all actors; filterable by auth provider, role, tenant +- **Force revoke**: select one or all sessions for an actor; requires reason (logged to audit) +- **Security events**: real-time feed of auth.security_session_revoked and ICOM_UNAUTHORIZED_SOURCE events +- **Internal component certificates**: table of component cert expiry dates; alert on certs expiring within P14D; ICOM-006 compliance view + +--- + +## 13. Health and Operations + +**API:** `GET /api/v1/admin/health`, `GET /api/v1/admin/discovery/trigger`, `POST /api/v1/admin/search-index/rebuild` + +Visible to `sre` and `platform_admin` roles. + +- Full component health detail (Section 2 dashboard expanded view) +- Prometheus metrics viewer (embedded Grafana or linked) +- Manual operations: trigger discovery, rebuild search index, rotate bootstrap credential +- Deployment info: DCM version, instance UUID, profile, uptime +- **Runbook links**: direct links to doc 41 (Operational Reference) scenarios from health page + +--- + +## 14. Flow GUI Integration + +The Flow GUI (policy authoring tool โ€” separate spec) is accessible to actors with `policy_owner` or `sre` roles: + +- **Link** from the Admin Panel navigation to the Flow GUI +- **Embedded iframe** option for deployments that want a unified navigation experience +- Active profile and policy summary widgets from Flow GUI embeddable on the Admin dashboard + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/dcm-consumer-gui-spec.md b/content/docs/architecture/specifications/dcm-consumer-gui-spec.md new file mode 100644 index 0000000..1ce75b7 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-consumer-gui-spec.md @@ -0,0 +1,919 @@ +# DCM Consumer Web GUI Specification + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Specification โ€” Consumer Web Interface +**Related Documents:** [RHDH Integration Specification](dcm-rhdh-integration-spec.md) | [Consumer API Specification](consumer-api-spec.md) | [Admin GUI Specification](dcm-admin-gui-spec.md) | [Provider GUI Specification](dcm-provider-gui-spec.md) | [Flow GUI Specification](dcm-flow-gui-spec.md) | [Auth Providers](../data-model/19-auth-providers.md) | [Session Revocation](../data-model/35-session-revocation.md) + +> **Status:** Draft โ€” Ready for implementation feedback +> +> **Primary deployment target:** Red Hat Developer Hub (RHDH) or upstream Backstage. The DCM Consumer Portal is implemented as a Backstage plugin suite. The standalone SPA mode is an alternative for deployments that do not run RHDH. See [RHDH Integration Specification](dcm-rhdh-integration-spec.md) for the complete Backstage plugin architecture. +> +> **Design goals (in priority order):** +> 1. **Low time to market** โ€” RHDH brings auth, search, TechDocs, RBAC, GitOps integration, and a full component library pre-built. DCM plugins extend rather than rebuild. +> 2. **Ease of use** โ€” familiar mental models (PatternFly/OpenShift language); task-oriented navigation; zero-ticket provisioning. +> 3. **Extensible** โ€” plugin architecture means new resource types surface automatically; no GUI code changes for new catalog items. +> 4. **Security and governance by design** โ€” tenancy, RBAC, audit, and policy constraints are architectural, not add-ons. + +--- + +## 1. Deployment Models + +### 1.1 RHDH / Backstage Mode (Primary) + +DCM is implemented as a Backstage plugin suite loaded into an RHDH or Backstage instance. DCM capabilities appear as a first-class section within the existing RHDH navigation. + +``` +RHDH Instance +โ”œโ”€โ”€ Pre-built RHDH capabilities (Catalog, Create, TechDocs, Search, ...) +โ”‚ โ””โ”€โ”€ These work unchanged โ€” DCM augments them +โ”‚ +โ””โ”€โ”€ DCM Plugin Suite (loaded as Dynamic Plugins) + โ”œโ”€โ”€ @dcm/plugin frontend plugin โ€” nav + pages + entity tabs + โ”œโ”€โ”€ @dcm/plugin-backend backend plugin โ€” API proxy, SSE relay, auth + โ”œโ”€โ”€ @dcm/plugin-catalog-backend catalog processor + entity provider + โ”œโ”€โ”€ @dcm/plugin-scaffolder-backend custom scaffolder actions + โ””โ”€โ”€ @dcm/permission-policy DCM โ†’ Backstage permission bridge +``` + +**Why RHDH first:** +- Auth (RHSSO/Keycloak/OIDC) is already configured โ€” no auth plumbing +- RBAC plugin already provides no-code permission management +- Search indexes DCM entities alongside existing catalog entities +- TechDocs renders DCM data model docs and runbooks in-portal +- Dynamic Plugins โ€” DCM plugins load without rebuilding the RHDH image (OCI or npm) +- Existing integrations: ArgoCD/GitOps (layer store visibility), Tekton (scaffolding pipelines), AAP (DCM providers that use Ansible), OCM (cluster management alongside DCM service catalog) + +### 1.2 Standalone SPA Mode (Alternative) + +For deployments without RHDH, DCM ships a standalone React SPA using PatternFly components. The same plugin modules are used; the host application is a lightweight Backstage-compatible shell rather than full RHDH. + +``` +Standalone DCM App +โ”œโ”€โ”€ PatternFly Page shell (Header, Sidebar, Content) +โ”œโ”€โ”€ Auth: DCM Auth Provider (OIDC/LDAP/built-in) +โ””โ”€โ”€ DCM plugin modules (same packages as RHDH mode) +``` + +The standalone mode provides feature parity. RHDH mode is recommended because it provides broader platform capabilities without additional investment. + +--- + +## 2. Navigation Architecture + +### 2.1 Design Principles + +**Organized by what users want to accomplish, not by API object.** A user who wants to "check on my database VM" thinks "My Resources" โ€” not "Realized State Entity Management." The navigation labels match the user's mental model. + +**PatternFly grouped hierarchical left nav.** Following OpenShift console and PatternFly's recommended pattern for administrative interfaces with multiple entity types. Consistent with what Red Hat users already know. + +**Stable groups, role-gated items.** Group headings are always visible; items within them hide (not disable) based on the actor's roles. A user who gains the `approver` role sees Approvals appear without any change to the application. + +**Tenant context in the header, not in the nav.** Following RHDH's namespace/group context selector pattern โ€” the active tenant is ambient context shown in the masthead, switchable without navigating away. + +### 2.2 Navigation Structure + +``` +[RHDH Masthead] +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ โฌก Red Hat Developer Hub [Search] ๐Ÿ‘ค User ๐Ÿข Tenant โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + +[Sidebar โ€” DCM section within RHDH nav] +โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” + DCM โ† Section header in RHDH sidebar +โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” + ๐Ÿช Service Catalog + + MY WORK โ† NavGroup (non-clickable group label) + ๐Ÿ“‹ Requests + ๐Ÿ–ฅ Resources + ๐Ÿ”— Dependency Groups + + ๐Ÿ”” Approvals [3] โ† NavItem with NotificationBadge (orange, count) + + GOVERNANCE โ† NavGroup + ๐Ÿ’ฐ Cost & Quota + ๐Ÿ“ค Contributions โ† hidden if not contributor role + + SETTINGS โ† NavGroup + ๐Ÿ”” Notifications + ๐Ÿ” Sessions +โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” +``` + +**PatternFly components used:** +- `Nav` with `variant="default"` โ€” left sidebar navigation +- `NavGroup` โ€” non-clickable group labels (MY WORK, GOVERNANCE, SETTINGS) +- `NavItem` โ€” clickable navigation items with optional `` +- `NavItemSeparator` โ€” visual divider between major areas +- `PageHeader` with `Masthead` โ€” tenant context selector (ContextSelector component) + +### 2.3 Tenant Context Selector + +The active tenant is displayed in the masthead as a `ContextSelector` (PatternFly): + +``` +[Masthead right side] + ๐Ÿ‘ค alice@corp.com ๐Ÿข Payments Team โ–พ + โ”œโ”€โ”€ Payments Team โ† current + โ”œโ”€โ”€ Platform Team + โ””โ”€โ”€ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + All My Tenants +``` + +- Single-tenant actors: selector hidden; tenant name displayed static +- Multi-tenant actors: dropdown triggers `X-DCM-Tenant` header change; page data refreshes +- Tenant switch does not navigate; current page re-fetches with new tenant context + +### 2.4 Role-Gating Rules + +| NavItem | Visible when actor has | +|---------|----------------------| +| Service Catalog | Any role (always visible) | +| Requests | `consumer` or any role | +| Resources | `consumer` or any role | +| Dependency Groups | `consumer` or any role | +| Approvals + badge | `approver` role | +| Cost & Quota | `consumer` or `tenant_admin` | +| Contributions | `contributor` role | +| Notifications | Any role | +| Sessions | Any role | + +Items not shown for a role are **hidden entirely** โ€” no disabled states in the nav. + +--- + +## 3. Service Catalog + +**Route:** `/dcm/catalog` +**API:** `GET /api/v1/catalog`, `GET /api/v1/catalog/{uuid}`, `GET /api/v1/catalog/search` + +### 3.1 In RHDH Mode โ€” Software Templates Integration + +In RHDH, DCM catalog items are exposed as **Backstage Software Templates**. The standard RHDH "Create" page becomes the DCM service catalog. + +``` +RHDH "Create" page +โ””โ”€โ”€ DCM Templates category + โ”œโ”€โ”€ ๐Ÿ–ฅ Standard VM (t-shirt sizes) + โ”œโ”€โ”€ ๐Ÿ—„ Database โ€” PostgreSQL + โ”œโ”€โ”€ ๐ŸŒ Virtual Network + โ”œโ”€โ”€ ๐Ÿ”’ TLS Certificate + โ””โ”€โ”€ ๐Ÿ“ฆ [More DCM catalog items...] +``` + +Templates are auto-generated by `@dcm/plugin-catalog-backend` โ€” it reads `GET /api/v1/catalog` and emits one Template entity per DCM catalog item. The template's input schema is derived directly from the DCM catalog item's field schema. No manual template authoring needed when a new resource type appears. + +DCM-specific catalog view at `/dcm/catalog` provides additional DCM context: +- Cost estimate per item (calls `POST /api/v1/cost/estimate` with default values) +- Availability indicator (quota headroom) +- Dependency graph preview +- Provider badge and SLA indicators + +### 3.2 Catalog Browser + +- **Card grid** โ€” PatternFly `Gallery` with `GalleryItem` cards +- **Filter toolbar** โ€” PatternFly `Toolbar` with category chips, tag filter, search input +- Card shows: name, description, provider type badge, cost estimate, availability +- Quick-request button opens the scaffolder template directly from the card + +### 3.3 Catalog Item Detail + +- Full description, field schema viewer, dependency declaration, TechDocs link +- Live cost estimate โ€” updates as user adjusts quantity/size fields (debounced) +- Quota check: remaining quota for this resource type in active tenant +- "Request This Service" โ†’ opens Scaffolder wizard + +--- + +## 4. Request Submission โ€” Scaffolder Integration + +**In RHDH mode:** Request submission uses the Backstage Scaffolder. DCM does not build a separate request form. The Scaffolder is the request form. + +### 4.1 Template Structure + +Each DCM catalog item generates a Backstage Software Template: + +```yaml +apiVersion: scaffolder.backstage.io/v1beta3 +kind: Template +metadata: + name: dcm-compute-vm-standard + title: "Standard VM" + description: "Provision a standard virtual machine" + annotations: + dcm.io/catalog-item-uuid: "" + dcm.io/resource-type: "Compute.VirtualMachine" + tags: [dcm, compute, infrastructure] +spec: + type: dcm-resource + parameters: + # Auto-generated from DCM catalog item field schema + - title: "Configure Your VM" + required: [cpu_count, memory_gb, os_family, name] + properties: + cpu_count: + title: CPU Cores + type: integer + enum: [2, 4, 8, 16, 32] + default: 4 + ui:widget: select + memory_gb: + title: Memory (GB) + type: integer + enum: [8, 16, 32, 64, 128] + default: 16 + name: + title: Resource Name + type: string + pattern: "^[a-z0-9-]{3,63}$" + ui:help: "Lowercase letters, numbers, hyphens. 3-63 characters." + + - title: "Scheduling (Optional)" + properties: + dispatch: + title: When to provision + type: string + enum: [immediate, at, window] + default: immediate + not_before: + title: Not before (UTC) + type: string + format: date-time + ui:widget: datetime + ui:if: "dispatch === 'at'" + window_id: + title: Maintenance Window + type: string + ui:field: dcm:MaintenanceWindowPicker + ui:if: "dispatch === 'window'" + + - title: "Review" + # Auto-populated review step showing cost estimate and policy pre-check + + steps: + - id: dcm-cost-estimate + name: "Estimate cost" + action: dcm:request:estimate + input: + catalogItemUuid: "${{ parameters.catalog_item_uuid }}" + fields: "${{ parameters }}" + + - id: dcm-submit + name: "Submit request" + action: dcm:request:submit + input: + catalogItemUuid: "${{ parameters.catalog_item_uuid }}" + fields: "${{ parameters }}" + schedule: + dispatch: "${{ parameters.dispatch }}" + notBefore: "${{ parameters.not_before }}" + + - id: dcm-wait + name: "Waiting for provisioning..." + action: dcm:request:wait + input: + requestUuid: "${{ steps.dcm-submit.output.requestUuid }}" + timeoutMinutes: 30 + + - id: register + name: "Register in catalog" + action: dcm:catalog:refresh + input: + entityUuid: "${{ steps.dcm-submit.output.entityUuid }}" + + output: + links: + - title: "View Resource" + url: "${{ steps.dcm-wait.output.entityUrl }}" + - title: "View Request" + url: "${{ steps.dcm-submit.output.requestUrl }}" +``` + +### 4.2 Custom Scaffolder Actions + +Provided by `@dcm/plugin-scaffolder-backend`: + +| Action | Description | +|--------|-------------| +| `dcm:request:estimate` | Call `POST /api/v1/cost/estimate`; output cost breakdown for review step | +| `dcm:request:submit` | `POST /api/v1/requests`; output requestUuid, entityUuid | +| `dcm:request:wait` | Poll `GET /api/v1/requests/{uuid}/status` (or SSE); surface live status in Scaffolder log; resolve on terminal state | +| `dcm:request:group` | `POST /api/v1/request-groups`; submit multiple requests with dependency graph | +| `dcm:catalog:refresh` | Trigger entity provider refresh for new entity; output entityUrl for output links | +| `dcm:approval:notify` | Fire notification to approver group when approval required | + +### 4.3 Live Status in Scaffolder + +The `dcm:request:wait` action streams progress to the Scaffolder log panel using the Scaffolder's built-in log streaming UI: + +``` +โณ Waiting for provisioning... + 09:00:05 Status: ACKNOWLEDGED + 09:00:07 Status: ASSEMBLING layers + 09:00:12 Status: DISPATCHED to provider + 09:01:05 Status: PROVISIONING + Step 3/7: Configuring network interfaces + โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 3 of 7 complete + 09:03:12 โœ… Status: REALIZED + Resource: payments-api-server-01 + IP: 10.42.0.105 +``` + +This reuses the Scaffolder's existing log streaming rather than building a custom progress component. + +--- + +## 5. My Work โ€” Requests + +**Route:** `/dcm/requests` +**API:** `GET /api/v1/requests`, `GET /api/v1/requests/{uuid}/status`, `GET /api/v1/requests/{uuid}/stream`, `DELETE /api/v1/requests/{uuid}` + +### 5.1 Requests List + +PatternFly `Table` with Toolbar filter: + +``` +[MY REQUESTS] +Filter: [Status โ–พ] [Resource Type โ–พ] [Date Range โ–พ] [Search name...] + + Name Type Status Submitted + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + payments-api-server-01 Compute.VM โ— REALIZED 2h ago + dev-db-postgres-02 Database โณ PROVISIONING 15m ago [Live โ–ถ] + uat-lb-internal LoadBalancer โš  REQUIRES APPR. 1h ago [Review โ†’] + batch-runner-scheduled Compute.VM ๐Ÿ• SCHEDULED Tomorrow +``` + +Status badges use PatternFly `Label` component: +- `โ— REALIZED` โ€” green Label +- `โณ PROVISIONING` โ€” blue Label + spinner +- `โš  REQUIRES APPROVAL` โ€” orange Label +- `๐Ÿ• SCHEDULED` โ€” grey Label +- `โœ— FAILED` โ€” red Label + +"Live โ–ถ" button opens an inline `Drawer` (PatternFly) showing the SSE status stream โ€” no page navigation required. + +### 5.2 Live Status Drawer + +Clicking "Live โ–ถ" opens a right-side Drawer anchored to the page: + +``` +โ”Œโ”€โ”€โ”€ Request Status โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ โœ• โ”€โ” +โ”‚ dev-db-postgres-02 โณ PROVISIONING โ”‚ +โ”‚ โ”‚ +โ”‚ Timeline: โ”‚ +โ”‚ โœ… 09:00:00 ACKNOWLEDGED โ”‚ +โ”‚ โœ… 09:00:02 ASSEMBLING โ”‚ +โ”‚ โœ… 09:00:47 DISPATCHED โ†’ provider-postgres-prod โ”‚ +โ”‚ โณ 09:01:05 PROVISIONING โ”‚ +โ”‚ โ”‚ +โ”‚ Progress: Step 3 of 7 โ”‚ +โ”‚ Configuring network interfaces โ”‚ +โ”‚ โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 43% โ”‚ +โ”‚ โ”‚ +โ”‚ Est. completion: ~4 minutes โ”‚ +โ”‚ [Cancel Request] โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +Powered by `GET /api/v1/requests/{uuid}/stream` (SSE). Drawer closes automatically on terminal status, replaces status badge in the table row. + +--- + +## 6. My Work โ€” Resources + +**Route:** `/dcm/resources` +**API:** `GET /api/v1/resources`, resource sub-resource endpoints + +### 6.1 Resource List + +``` +[MY RESOURCES] +Filter: [State โ–พ] [Type โ–พ] [Group โ–พ] [Tag โ–พ] [Search...] [Table/Card โ–พ] + + โ— 3 resources with drift detected [View Drift Report โ†’] + + Name Type State Provider TTL + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + payments-api-01 โšก Compute.VM โœ… OPER. k8s-prod 87d [ยทยทยท] + dev-db-001 Database โœ… OPER. db-prod โ€” [ยทยทยท] + uat-loadbalancer ๐Ÿ”ด Network.LB โš  DRIFT net-prod โ€” [ยทยทยท] +``` + +Icons: +- `โšก` โ€” TTL warning (< 14 days) +- `๐Ÿ”ด` dot โ€” open drift record +- `ยทยทยท` โ€” action menu (Suspend, Update, Extend TTL, Transfer, Decommission) + +State badges use PatternFly `Label`: +- `โœ… OPERATIONAL` โ€” green +- `๐ŸŸก SUSPENDED` โ€” yellow +- `๐Ÿ”ต MAINTENANCE` โ€” blue +- `โš  DRIFT` โ€” orange (custom โ€” shows worst drift severity) +- `โœ— FAILED` โ€” red + +### 6.2 Resource Entity Page + +Clicking a resource opens its **Backstage entity page** (standard RHDH pattern). DCM contributes tabs via entity page tab extensions: + +``` +[Entity Header] +payments-api-server-01 โœ… OPERATIONAL +Compute.VirtualMachine | provider: k8s-prod | owner: payments-team + +[Tabs] + Overview โ”‚ Drift ๐Ÿ”ด โ”‚ Audit โ”‚ Cost โ”‚ Credentials โ”‚ Relations โ”‚ Docs +``` + +**Overview tab:** IP address, hostname, CPU, memory, OS, provider, realized date, TTL, all realized fields from provider. + +**Drift tab ๐Ÿ”ด** (badge shows drift severity): +``` +Open Drift Records (1 significant, 1 minor) + + Field Realized Value Discovered Value Severity + cpu_count 4 8 โš  significant + memory_gb 16 16 โœ… no drift + tags [app:api] [app:api, env:prod] โ„น minor + + Actions: [Revert All] [Accept Changes (Update Definition)] [Acknowledge Minor] +``` + +**Cost tab:** Actual cost from `GET /api/v1/resources/{uuid}/cost`. PatternFly `ChartBar` for cost by billing dimension. + +**Credentials tab:** +``` + Credential Type Expires Last Retrieved + ssh-key-payments-01 ssh_key 2027-06-01 2 days ago + api-key-svc-acct api_key never โ€” + + [Retrieve Value โ†’] โ† triggers inline step-up MFA prompt + [Request Rotation โ†’] +``` + +**Relations tab:** PatternFly `TopologyView` or a simple dependency graph showing what this resource depends on and what depends on it. + +### 6.3 Drift Report โ€” Cross-Resource View + +**Route:** `/dcm/resources/drift` +Accessible via "View Drift Report โ†’" banner on resource list when drift exists. + +``` +[DRIFT REPORT โ€” Payments Team] 3 resources with open drift + + Severity Resource Field Realized Discovered Since + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + โ— Critical prod-db-primary replication enabled disabled 4h + โ— Signif. uat-lb-internal cpu_count 4 8 2d + โ— Minor dev-app-01 tag:env staging (missing) 1w + + [Revert All Critical] [Export Drift Report] [Configure Drift Alerts] + + Auto-remediation status: + prod-db-primary: REVERT_PENDING โ€” waiting for maintenance window +``` + +Severity filter chips at top (PatternFly `ChipGroup`). Clicking a row opens the resource entity Drift tab. + + +--- + +## 7. My Work โ€” Dependency Groups + +**Route:** `/dcm/dependency-groups` +**API:** `GET /api/v1/request-groups`, `GET /api/v1/request-groups/{uuid}`, `DELETE /api/v1/request-groups/{uuid}` + +``` +[MY DEPENDENCY GROUPS] + + three-tier-app-deploy in_progress Started 30m ago + โ”œโ”€โ”€ db โœ… REALIZED payments-db-01 09:01 + โ”œโ”€โ”€ app โณ DISPATCHED (pending db IP โ†’ db_host injected) + โ””โ”€โ”€ lb โ—‹ PENDING (waiting for app) + + [Cancel Group] [View All Requests] +``` + +Progress bar shows N of M constituents realized. + +--- + +## 8. ITSM Integration + +DCM is designed to eliminate the infrastructure ticket as the primary provisioning mechanism. However, organizations that operate ITSM systems (ServiceNow, Jira Service Management, Remedy) still require bidirectional traceability โ€” change records for compliance, incident linkage for troubleshooting, and CMDB accuracy. + +DCM's ITSM integration is a **bridge, not a dependency.** DCM does not require an ITSM system to function. ITSM integration is additive โ€” it enriches DCM entities with ITSM metadata and notifies ITSM of DCM lifecycle events. + +### 8.1 ITSM Reference Linking + +Every DCM resource entity supports an optional `itsm_references` metadata block in its business data: + +``` +[Resource Entity Header] +payments-api-server-01 โœ… OPERATIONAL + +[ITSM References โ€” visible as a metadata card on Overview tab] + Change Record CHG0012345 Approved 2026-03-15 [View in ServiceNow โ†—] + Incident INC0048291 (linked on creation) [View in Jira โ†—] + CMDB Item CI-VM-08821 Auto-synced [View in CMDB โ†—] +``` + +References are stored as business data fields on the entity โ€” they follow the entity through its full lifecycle and appear in audit records. They are **not** required for DCM to function. + +### 8.2 ITSM Event Webhook Integration + +DCM fires lifecycle events to the Message Bus. An ITSM Notification Provider subscribes to these events and creates/updates ITSM records accordingly: + +| DCM Event | ITSM Action (configurable) | +|-----------|---------------------------| +| `request.requires_approval` | Create Change Request draft in ServiceNow / Jira | +| `request.realized` | Close Change Request; update CMDB CI | +| `request.failed` | Create Incident; link to Change Request | +| `entity.state_changed` | Update CMDB CI state | +| `drift.detected` (significant/critical) | Create Incident in ITSM | +| `entity.decommissioned` | Retire CMDB CI; close related Change Requests | + +This is implemented as a **Notification Provider** registered in DCM โ€” a webhook consumer that translates DCM events to ITSM API calls. No changes to DCM core are needed. + +### 8.3 ITSM Ticket as Approval Mechanism + +For organizations that require ITSM change board approval, DCM's `authorized` tier approval mechanism accepts votes recorded via the ITSM system: + +``` +[Request reaches authorized tier โ†’ approval required] + โ”‚ + โ–ผ DCM fires request.requires_approval + โ”‚ Notification Provider creates Change Request in ServiceNow + โ”‚ + โ–ผ Change Board reviews in ServiceNow (existing process unchanged) + โ”‚ Approval decision โ†’ ServiceNow calls DCM Admin API: + โ”‚ POST /api/v1/admin/approvals/{uuid}/vote + โ”‚ { "decision": "approve", "recorded_via": "servicenow", "voter_uuid": "..." } + โ”‚ + โ–ผ DCM records vote; quorum tracked by DCM + โ”‚ Audit trail includes: who voted, via which system, at what time +``` + +The approval decision is made in ServiceNow using the organization's existing CAB process. DCM records the outcome. Neither system depends on the other's internal workflow model. + +### 8.4 ITSM Reference UI + +In the Consumer GUI, ITSM references surface in two places: + +**On resource entity Overview tab** โ€” ITSM References card (shown only when references exist): +``` +ITSM References + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ CHG0012345 Change Request Approved [Open โ†—] โ”‚ + โ”‚ INC0048291 Incident Resolved [Open โ†—] โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + [Add Reference] (opens modal: type, ID, system URL) +``` + +**On request status page** โ€” when a request is at `requires_approval` and an ITSM reference is attached: +``` +โณ REQUIRES APPROVAL โ€” verified tier + Approval being tracked via ServiceNow Change Board + CHG0012345 [View in ServiceNow โ†—] + Quorum: 0 / 2 votes recorded +``` + +### 8.5 CMDB Sync + +DCM entities are the **system of record** for realized state. The CMDB is a **consumer** of DCM data, not a producer. CMDB sync flows one way: DCM โ†’ CMDB. + +The sync is implemented via the Notification Provider subscription to `entity.*` events. A CMDB sync Notification Provider maps DCM entity fields to CMDB CI attributes and calls the CMDB API on every state change. + +**CMDB field mapping** is declared in the provider registration โ€” it is not hardcoded. Different CMDB systems (ServiceNow CMDB, iTop, Device42) use the same event subscription pattern with different field mapping configurations. + +--- + +## 9. Approvals + +**Route:** `/dcm/approvals` +**API:** `GET /api/v1/approvals/pending`, `POST /api/v1/approvals/{uuid}` + +Visible only to actors with `approver` role. NavItem shows orange badge with pending count. + +``` +[PENDING APPROVALS โ€” 3] + + Request Tenant Tier Risk Expires + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + Large GPU VM (8x A100) AI Team verified 72/100 2h 14m + Prod DB 32-core Data Team reviewed 41/100 23h + Bulk decommission (12) Platform authorized 88/100 45m โ† quorum 1/3 + + [Review โ†’] +``` + +Clicking "Review โ†’" opens the Approval Detail page: +- Full request payload summary +- Risk score breakdown (signal chart) +- Policy evaluation results (which policies flagged this) +- Existing votes (for authorized tier quorum tracking) +- [Approve] [Reject] buttons with required comment field +- Approver cannot vote on own requests (blocked UI-side + server-side) + +--- + +## 10. Governance โ€” Cost and Quota + +**Route:** `/dcm/cost` + +``` +[COST & QUOTA โ€” Payments Team] This month: $12,450 vs budget: $15,000 + + Quota Utilization + Compute.VM โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 80% (16/20 allocated) โš  + Storage.Block โ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 22% (11/50 TB) + Network.LB โ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 8% (2/25 units) + + Cost by Resource Type (last 30 days) + [PatternFly ChartDonut or ChartBar] + + Top Cost Resources + payments-api-01 Compute.VM $3,200/mo + payments-db-01 Database $2,800/mo + ... + + [Export CSV] [View All Resources] +``` + +--- + +## 11. Governance โ€” Audit Trail + +**Route:** `/dcm/audit` +**API:** `GET /api/v1/resources/{uuid}/audit`, `GET /api/v1/audit/correlation/{correlation_id}` + +Visible to all actors for their own resources. Cross-tenant audit is in the Admin Panel. + +``` +[MY AUDIT TRAIL] +Filter: [Resource Type โ–พ] [Operation โ–พ] [Date Range โ–พ] [Correlation ID search...] + + Time Resource Operation Actor + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + 5m ago payments-api-01 PATCH alice@corp.com + 2h ago dev-db-001 DRIFT_REVERT system + Yesterday uat-lb-internal REALIZED alice@corp.com + 2 days ago batch-runner-01 DECOMMISSIONED bob@corp.com + + [View Full Record โ†’] [Export CSV] +``` + +**Record detail drawer:** Operation, Resource, Actor (with auth method), Time, Correlation ID trace, fields changed, policy evaluations, risk score. + +**Correlation ID trace:** links to full request pipeline view โ€” Intent โ†’ Requested โ†’ Dispatch โ†’ Realized โ†’ Provider chain โ€” for the actor's own requests. + +--- + +## 12. Governance โ€” Contributions + +**Route:** `/dcm/contributions` +**API:** `GET /api/v1/contribute`, `POST /api/v1/contribute/policy`, `DELETE /api/v1/contribute/{uuid}` + +Visible only to actors with `contributor` role. + +``` +[MY CONTRIBUTIONS] + + Handle Type Status Submitted + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + payments-network-policy policy โ— active 3 months ago + gpu-quota-validator policy ๐Ÿ”ต shadow 1 week ago [Divergence: 2.3%] + ml-resource-group resource_group โณ reviewing 2 days ago + + [Submit New Policy] [Submit Resource Group] +``` + +Shadow divergence percentage shown for shadow-mode contributions โ€” clicking opens divergence case viewer. + +--- + +## 13. Settings โ€” Notifications + +**Route:** `/dcm/notifications` +**API:** `GET /api/v1/notifications`, `GET /api/v1/webhooks`, `POST /api/v1/webhooks` + +``` +[NOTIFICATIONS] [Mark All Read] + + ๐Ÿ”” Request REALIZED: payments-api-server-01 2m ago [View โ†’] + โš  Drift detected: uat-loadbalancer (significant) 1h ago [View โ†’] + โฐ TTL Warning: dev-db-001 expires in 7 days 3h ago [Extend โ†’] + + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + [WEBHOOK SUBSCRIPTIONS] + + URL Events Status + https://my-system/dcm-webhook request.*, drift.* โœ… active [Test] [Delete] + + [Add Webhook] +``` + +--- + +## 14. Settings โ€” Sessions + +**Route:** `/dcm/sessions` +**API:** `GET /api/v1/auth/sessions`, `DELETE /api/v1/auth/sessions/{uuid}` + +``` +[ACTIVE SESSIONS] + + Device Auth Method Created Last Active + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + This session โ— OIDC Today 09:00 Active now + Chrome / Mac (10.0.1.5) OIDC Yesterday 2h ago [Revoke] + API Client (svc-acct) api_key 3 days ago 1h ago [Revoke] + + [Sign Out All Other Sessions] +``` + +--- + +## 15. In RHDH โ€” TechDocs Integration + +DCM publishes TechDocs for: +- Service catalog item documentation (from DCM layer store) +- Resource type specifications +- DCM data model documentation (this spec set) + +TechDocs are indexed by RHDH search โ€” users can search "how do I provision a GPU cluster" and find both the catalog item and the documentation. + +DCM entity pages link to TechDocs automatically via `backstage.io/techdocs-ref` annotation on generated catalog entities. + +--- + +## 16. Search Integration + +In RHDH mode, DCM entities are indexed in Backstage search: + +- DCM catalog items indexed as `DCMService` entities โ€” searchable by name, description, tags +- DCM realized resources indexed as `DCMResource` entities โ€” searchable by handle, IP, type +- TechDocs content indexed โ€” searchable documentation +- Search collator provided by `@dcm/plugin-catalog-backend` + +Search result rendering: +``` + ๐Ÿ–ฅ payments-api-server-01 DCMResource Compute.VM โ— OPERATIONAL + 10.42.0.105 ยท k8s-prod ยท payments-team +``` + +--- + + +### 16.1 Global Search Bar + +PatternFly `SearchInput` in masthead โ€” keyboard shortcut `/` to focus. Searches across catalog items, resources, requests, and TechDocs (RHDH mode). + +### 16.2 Search Results Page + +**Route:** `/dcm/search?q=` + +``` +[SEARCH RESULTS: "payments db"] 12 results + + [All โ–พ] [Catalog Items] [Resources] [Requests] [Docs] โ† filter chips + + CATALOG ITEMS (2) + โ— Database.PostgreSQL Managed PostgreSQL instance [Request โ†’] + โ— Database.MySQL Managed MySQL instance [Request โ†’] + + MY RESOURCES (3) + โ— payments-db-primary Database.PostgreSQL โœ… OPERATIONAL [View โ†’] + โ— payments-db-replica Database.PostgreSQL โœ… OPERATIONAL [View โ†’] + + MY REQUESTS (1) + โ— payments-db-archive SCHEDULED Tomorrow 02:00 [View โ†’] +``` + +Results render incrementally per category with PatternFly `Spinner` while loading. Keyboard navigation through results supported. + +## 17. Path to Production โ€” Effort Reduction Summary + +| Traditional Workflow | DCM + RHDH | +|---------------------|------------| +| Submit infrastructure ticket | Browse catalog โ†’ click "Request" โ†’ Scaffolder wizard | +| Wait for human review (hours/days) | Policy engine auto-approves in seconds; escalates if needed | +| Infrastructure team provisions manually | DCM dispatches to provider automatically | +| Receive confirmation email | SSE live status in Scaffolder; notification in portal | +| Manually update CMDB | Entity appears in RHDH catalog automatically after REALIZED | +| Track cost in spreadsheet | Cost tab on every resource entity page | +| Periodic compliance audits | Drift detection continuous; audit trail always current | +| Decommission via ticket | Self-service Delete with stake resolution | +| Document runbooks separately | TechDocs in same portal, linked from entity pages | +| Separate access management | RHDH RBAC + DCM permission policy โ€” one place | + +**Time to first production resource (new user):** +1. Log in (SSO โ€” already have corporate credentials) +2. Browse catalog or search for service +3. Fill in Scaffolder form (5โ€“10 fields, schema-driven, cost estimate live) +4. Submit โ†’ watch live provisioning log +5. Resource appears in catalog and "My Resources" + +**Target: < 10 minutes from login to operational resource.** + +--- + +## 18. Security Model + +### 16.1 Tenancy Enforcement + +- Active tenant: RHDH Group context โ†’ `X-DCM-Tenant` header on all DCM API calls +- All data filtered server-side by DCM Consumer API (defense in depth) +- Entity ownership in RHDH catalog: `spec.owner = group:` โ€” users see only own-tenant entities + +### 16.2 RBAC + +- RHDH RBAC plugin maps Backstage permissions to DCM roles +- DCM roles in session token drive nav visibility (client-side, defense-in-depth) +- DCM Consumer API enforces roles server-side on every request +- New roles propagate from IdP via SCIM or OIDC claims โ€” no manual assignment + +### 16.3 Step-Up MFA + +Credential retrieval, ownership transfer, and bulk decommission trigger inline MFA: +- PatternFly `Modal` overlay โ€” no page navigation +- User completes second factor in RHDH auth provider +- Step-up token cached PT10M (per session model) + +### 16.4 Content Security Policy (Standalone Mode) + +- `default-src 'self'` +- `connect-src 'self' ` +- No inline scripts; no eval(); SRI on external assets + +### 16.5 Governance and Regulatory Compliance + +All compliance constraints are enforced by the DCM control plane โ€” the GUI is a client. The GUI surfaces compliance state: +- Data classification badges on entity detail pages (from `data_classification` fields) +- Sovereignty constraint indicators on entities with cross-boundary restrictions +- Accreditation status visible in entity overview +- Audit trail always accessible; chain integrity status shown + +The GUI cannot be used to bypass policy โ€” every action goes through the Consumer API which applies the full five-check boundary model (doc 26), scoring, GateKeeper, and policy evaluation. + +--- + +## 19. Extensibility + +### 17.1 New Resource Types โ€” Zero GUI Code + +When a new resource type is registered in DCM: +1. `@dcm/plugin-catalog-backend` detects it via the resource type registry +2. A new Software Template is auto-generated from the field schema +3. A new `DCMService` catalog entity appears in RHDH +4. The template's input form renders from JSON Schema โ€” no UI code needed +5. The resource entity page uses the same Overview/Drift/Audit/Cost/Credentials/Relations tabs โ€” no new tabs needed for standard resource types + +Resource type-specific UI extensions are possible via entity page tab contributions if a resource type requires specialized visualization (e.g. a topology view for network types). + +### 17.2 Custom Plugin Extensions + +Teams can contribute additional entity page tabs, catalog cards, or nav items via standard Backstage plugin extension points. DCM plugin APIs for contributing extensions: + +```typescript +// Register a custom entity tab for a specific resource type +dcmPlugin.registerEntityTab({ + resourceType: 'Network.VirtualNetwork', + component: NetworkTopologyTab, + title: 'Topology', + icon: NetworkIcon, +}); + +// Register a custom catalog card +dcmPlugin.registerCatalogCard({ + component: GPUAvailabilityCard, + position: 'right', +}); +``` + +### 17.3 Multi-Portal Deployments + +Large organizations with multiple RHDH instances (one per business unit) can: +- Point multiple RHDH instances at the same DCM control plane +- Each RHDH instance filters DCM entities to its tenant scope +- DCM federation handles cross-instance resource visibility where permitted + +--- + +## 20. Conformance + +A conforming Consumer GUI implementation must: + +1. Implement all DCM nav groups: Service Catalog, My Work (Requests/Resources/Groups), Approvals, Governance (Cost & Quota/Contributions), Settings +2. Implement SSE-based live status with constituent tracking (with polling fallback) +3. Enforce tenancy context (`X-DCM-Tenant`) on all API calls +4. Hide (not disable) role-gated navigation items +5. Implement step-up MFA inline for gated operations +6. Surface drift, audit, cost, credentials, and relationships as entity page tabs +7. In RHDH mode: implement all six plugin packages with the specified capabilities +8. Auto-generate Software Templates from DCM catalog item schemas (RHDH mode) + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/dcm-examples.md b/content/docs/architecture/specifications/dcm-examples.md new file mode 100644 index 0000000..4a13734 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-examples.md @@ -0,0 +1,874 @@ +# DCM โ€” Examples and Use Cases + +**Document Status:** โœ… Complete +**Status:** Draft โ€” Examples document - no WIP status needed; always current with architecture. +**Document Type:** Reference Examples +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Provider Contract](../data-model/A-provider-contract.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) | [Registration](dcm-registration-spec.md) | [OPA Integration](dcm-opa-integration-spec.md) + +--- + +## Overview + +This document provides end-to-end worked examples for the most important DCM use cases. Each example shows the complete interaction โ€” payloads, state transitions, API calls, and Rego policies where applicable โ€” so implementors can trace exactly what happens at each step. + +Examples are organized by the three foundational abstractions: +- **Section 1** โ€” Orchestration examples (Policy) +- **Section 2** โ€” Provider interaction examples (Provider) +- **Section 3** โ€” API interaction examples (Consumer API, Admin API) +- **Section 4** โ€” Registration flow examples + +--- + +# Section 1 โ€” Orchestration Examples + +## 1.1 Basic Request Lifecycle (End-to-End) + +The complete path for a consumer requesting a VM. Shows all payload type events, which policies fire at each step, and the state transitions. + +### Setup: Active artifacts + +```yaml +# Named workflow (Level 1 orchestration) +Orchestration Flow Policy: system/workflows/request-lifecycle + ordered: true + steps: [request.initiated, request.intent_captured, + request.layers_assembled, request.placement_complete, + request.dispatched] + +# Dynamic policies (Level 2 orchestration) +GateKeeper: org/gatekeeper/vm-size-limits (fires on request.layers_assembled) +Transformation: org/transformation/inject-monitoring (fires on request.layers_assembled) +GateKeeper: system/gatekeeper/sovereignty-check (fires on request.placement_complete) +``` + +### Step-by-step + +**Step 1 โ€” Consumer submits request:** +``` +POST /api/v1/requests +{ "catalog_item_uuid": "vm-standard-uuid", + "fields": { "cpu_count": 4, "memory_gb": 8, "os_family": "rhel" } } + +โ†’ Response 202: { "request_uuid": "req-001", "entity_uuid": "ent-001", + "status": "ACKNOWLEDGED" } +โ†’ Event published: { "type": "request.initiated", "entity_uuid": "ent-001", + "payload": { "fields": {...} } } +โ†’ Intent State written to Intent Store +``` + +**Step 2 โ€” Layer assembly:** +``` +Event: request.initiated +โ†’ Named workflow step 1 fires: capture-intent policy acknowledges +โ†’ New event: request.intent_captured + +Event: request.intent_captured +โ†’ Named workflow step 2 fires: assemble-layers policy runs +โ†’ Base layer applied: data_center = "EU-WEST-DC1" +โ†’ Org layer applied: monitoring_agent = "datadog-agent:7.42" +โ†’ Policy layer applied: backup_policy = "daily-30d-eu-west" +โ†’ New event: request.layers_assembled + payload now includes all merged fields with provenance +``` + +**Step 3 โ€” Dynamic policies fire on request.layers_assembled:** +``` +Event: request.layers_assembled +โ†’ [PARALLEL] All policies matching this payload type evaluate simultaneously: + + GateKeeper vm-size-limits evaluates: + input.payload.fields.cpu_count.value = 4 + 4 <= 32 โ†’ allow: true + + Transformation inject-monitoring evaluates: + monitoring_endpoint not in payload โ†’ mutation: + { field: "fields.monitoring_endpoint", + operation: "set", + value: "https://metrics.internal.prod.example.com" } + +โ†’ All GateKeepers: allow +โ†’ Transformations applied to payload +โ†’ New event: request.policies_evaluated +``` + +**Step 4 โ€” Placement:** +``` +Event: request.policies_evaluated +โ†’ Named workflow step 3: run-placement policy + +โ†’ Placement Engine: + Step 1: Sovereignty filter โ€” EU-WEST-DC1 requirement โ†’ 3 providers eligible + Step 2: Accreditation filter โ€” no PHI in payload โ†’ all 3 pass + Step 3: Capability filter โ€” all support VirtualMachine โ†’ all 3 pass + Step 4: Reserve query โ†’ parallel queries to EU-WEST-Prod-1,2,3 + EU-WEST-Prod-1: confirmed, utilization 42%, cost $0.32/hr + EU-WEST-Prod-2: confirmed, utilization 61%, cost $0.32/hr + EU-WEST-Prod-3: insufficient capacity + Step 5: Tie-break โ†’ Step 4 cost equal โ†’ Step 5 least loaded โ†’ Prod-1 wins + Step 6: Confirm Prod-1; release holds on Prod-2 + +โ†’ Requested State written to Requested Store + (requested.yaml + assembly-provenance.yaml + placement.yaml + dependencies.yaml) +โ†’ New event: request.placement_complete +``` + +**Step 5 โ€” Dispatch and realization:** +``` +Event: request.placement_complete +โ†’ Governance Matrix evaluated: payload data_classification = internal/public โ†’ ALLOW +โ†’ Named workflow step 4: dispatch policy +โ†’ Provider EU-WEST-Prod-1 receives dispatch payload +โ†’ Provider naturalizes to OpenStack Nova format +โ†’ OpenStack provisions VM +โ†’ Provider denaturalizes result โ†’ DCM unified format +โ†’ Realized State written (with provider_entity_id: "vm-0a1b2c3d") +โ†’ Status callback: COMPLETED + +Consumer polls: GET /api/v1/requests/req-001/status +โ†’ { "status": "COMPLETED", "entity_uuid": "ent-001" } +``` + +--- + +## 1.2 Human Approval Gate (Conditional Step Insertion) + +A production VM request that requires manager approval before dispatch. Shows how a GateKeeper policy inserts a waiting step without modifying the named workflow. + +### Setup: Additional active policy + +```rego +# GateKeeper fires on request.policies_evaluated for prod VMs over $100/month +package dcm.gatekeeper.prod_vm_approval_gate + +deny contains reason if { + input.payload.type == "request.policies_evaluated" + input.deployment.deployment_posture == "prod" + input.payload.cost_estimate.per_month > 100 + not input.payload.approvals["manager_approval"] + reason := "Production VMs over $100/month require manager approval" +} + +# Signal that approval is the resolution path (not a permanent reject) +requires_approval := true if count(deny) > 0 +approval_type := "manager_approval" if count(deny) > 0 +``` + +### Step-by-step + +``` +After Step 3 (dynamic policies evaluate): +โ†’ GateKeeper prod_vm_approval_gate fires +โ†’ deny: ["Production VMs over $100/month require manager approval"] +โ†’ requires_approval: true, approval_type: "manager_approval" + +โ†’ Policy Engine sees GateKeeper deny WITH requires_approval flag +โ†’ Entity enters AWAITING_APPROVAL state (not FAILED) +โ†’ Notification dispatched: + audience: manager (from actor's group membership via relationship graph) + event_type: request.requires_approval + action_url: /api/v1/requests/req-001/approve + action_deadline: PT24H + +Manager approves: +POST /api/v1/requests/req-001/approve +{ "approval_type": "manager_approval", "approver_uuid": "mgr-001" } + +โ†’ payload.approvals["manager_approval"] = { approved: true, by: "mgr-001" } +โ†’ GateKeeper re-evaluates: approval present โ†’ allow +โ†’ Pipeline resumes from request.policies_evaluated +โ†’ Placement โ†’ Dispatch โ†’ Realization (same as 1.1 Steps 4-5) +``` + +--- + +## 1.3 Policy-Gated Request โ€” Hard Block with Clear Error + +Shows a request blocked by a hard GateKeeper with a consumer-visible error message. + +```rego +package dcm.gatekeeper.approved_os_images + +deny contains reason if { + input.payload.type == "request.layers_assembled" + not input.payload.fields.os_family.value in {"rhel", "ubuntu-lts", "coreos"} + reason := sprintf( + "OS '%s' is not in the approved image list. Approved: rhel, ubuntu-lts, coreos", + [input.payload.fields.os_family.value] + ) +} +``` + +``` +Consumer submits: { "os_family": "windows-server" } + +โ†’ request.layers_assembled fires +โ†’ GateKeeper approved_os_images: deny +โ†’ Entity โ†’ FAILED (no requires_approval flag โ†’ hard block) + +Consumer response: +{ "status": "FAILED", + "failure_reason": "OS 'windows-server' is not in the approved image list.", + "retry_eligible": true, + "policy_uuid": "gatekeeper-approved-os-uuid", + "suggestion": "Resubmit with os_family: rhel, ubuntu-lts, or coreos" } +``` + +--- + +## 1.4 Compound Service โ€” Meta Provider with Dependency Ordering + +A web application stack provisioned as a single catalog item: VM + IP + DNS + LoadBalancer. + +### Named workflow for compound service + +```rego +package dcm.orchestration.webapp_stack + +steps := [ + {"step": 1, "payload_type": "request.initiated", + "policy_handle": "system/orchestration/capture-intent", "on_fail": "halt"}, + {"step": 2, "payload_type": "request.intent_captured", + "policy_handle": "system/orchestration/assemble-compound", "on_fail": "halt"}, + {"step": 3, "payload_type": "request.compound_assembled", + "policy_handle": "system/orchestration/resolve-dependencies", "on_fail": "halt"}, + {"step": 4, "payload_type": "request.dependencies_resolved", + "policy_handle": "system/orchestration/dispatch-constituents", "on_fail": "compensate"} +] + +ordered := true +``` + +### Execution + +``` +Meta Provider receives compound dispatch payload: + component.ip: { resource_type: Network.IPAddress, depends_on: [] } + component.vm: { resource_type: Compute.VirtualMachine, depends_on: [] } + component.dns: { resource_type: DNS.Record, depends_on: [ip, vm], required: partial } + component.lb: { resource_type: Network.LoadBalancer, depends_on: [vm, ip], required: partial } + +Dependency-ordered execution: + Round 1 (no dependencies): ip, vm โ†’ provisioned in parallel + ip โ†’ REALIZED: 10.1.45.23/32 + vm โ†’ REALIZED: vm-0a1b2c3d + + Round 2 (depend on ip+vm): dns, lb โ†’ provisioned in parallel + dns โ†’ FAILED (DNS service degraded) + lb โ†’ REALIZED: lb-7f8e9d + + Compound evaluation: + dns: required_for_delivery = partial โ†’ DEGRADED, not FAILED + lb: required_for_delivery = partial โ†’ REALIZED + +Compound entity state: DEGRADED (dns failed; vm+ip+lb realized) +Notification: owner notified "WebApp Stack provisioned in degraded state โ€” DNS unavailable" + +Recovery policy fires (PARTIAL_REALIZATION trigger): + profile=prod โ†’ NOTIFY_AND_WAIT + Consumer sees notification with options: accept degraded | trigger dns retry +``` + +--- + +## 1.5 Drift Detection and Automated Remediation + +Discovery finds VM memory has changed without a DCM request. Shows the full drift โ†’ policy โ†’ revert flow. + +``` +Scheduled discovery (PT15M interval): +โ†’ Provider queried for vm-0a1b2c3d +โ†’ Discovered: memory_gb = 16 +โ†’ Realized State: memory_gb = 8 +โ†’ No Requested State record explains the change + +Drift Reconciliation Component: + field: memory_gb + realized_value: 8, discovered_value: 16 + change_magnitude: 100% increase โ†’ "significant" (standard profile: 10-50% threshold) + field_criticality: medium (from Resource Type Spec) + unsanctioned: true โ†’ elevate one level โ†’ "critical" + +Drift record created: + overall_severity: critical + unsanctioned: true + +Policy Engine evaluates drift record: + Active drift response policy (standard profile, critical severity, unsanctioned): + action: ESCALATE โ†’ notify platform admin + SRE + owner + +Notifications dispatched: + Owner: "Critical unsanctioned change on vm-0a1b2c3d: memory_gb 8โ†’16" + Platform Admin: same (urgency: critical) + SRE on-call: same (via PagerDuty Notification Provider) + +If consumer submits: REVERT +โ†’ New request submitted from Realized State (memory_gb: 8) +โ†’ Full governance pipeline โ†’ new Requested State โ†’ dispatch โ†’ revert +โ†’ Next discovery: memory_gb = 8 โ†’ drift.resolved event +``` + +--- + +## 1.6 Recovery Flow โ€” Dispatch Timeout with NOTIFY_AND_WAIT + +Provider does not respond within PT30M. Profile is `prod` โ†’ `recovery-notify-and-wait`. + +``` +T+0: Request dispatched to EU-WEST-Prod-1 +T+30M: Dispatch timeout fires + Entity โ†’ TIMEOUT_PENDING + Recovery trigger: DISPATCH_TIMEOUT + +Recovery Policy (prod profile โ†’ recovery-notify-and-wait): + action: NOTIFY_AND_WAIT + deadline: PT4H + on_deadline_exceeded: ESCALATE + +Notifications dispatched: + Owner: "Request req-001 timed out. Choose how to proceed by T+4H." + action_url: /api/v1/resources/ent-001/recovery-decisions + +Consumer queries: +GET /api/v1/resources/ent-001/recovery-decisions +โ†’ { "trigger": "DISPATCH_TIMEOUT", + "deadline": "...", + "available_actions": [ + { "action": "DRIFT_RECONCILE", + "description": "Let discovery determine actual state" }, + { "action": "DISCARD_AND_REQUEUE", + "description": "Clean up and retry" } + ] } + +T+45M: Provider responds (late response) with realized payload + Entity in TIMEOUT_PENDING โ†’ LATE_RESPONSE_RECEIVED fires + +Recovery policy for LATE_RESPONSE_RECEIVED (prod โ†’ notify-and-wait): + action: NOTIFY_AND_WAIT (same โ€” human decides whether to accept late work) + notification updated: "Provider completed after timeout. Accept or discard?" + +Consumer POSTs: { "action": "DISCARD_AND_REQUEUE" } +โ†’ Best-effort cleanup sent to provider +โ†’ Entity โ†’ FAILED +โ†’ New request cycle created (same entity_uuid) +โ†’ Orphan detection triggered for EU-WEST-Prod-1 +``` + +--- + +## 1.7 Federation-Routed Request + +Consumer in Regional DCM A requests a resource that gets placed on a provider registered with Regional DCM B via Hub DCM. + +``` +Consumer โ†’ Regional DCM A: + POST /api/v1/requests { resource_type: Compute.VirtualMachine, ... } + +Regional DCM A Placement Engine: + Step 1: Sovereignty filter โ†’ local providers all at capacity + Step 2: Query Hub DCM (Peer DCM provider) for available regional capacity + โ†’ Hub responds: Regional DCM B has EU-WEST-Prod-2 with capacity + +Governance Matrix check (Regional DCM A โ†’ Hub DCM): + subject: dcm_peer (Regional DCM A) + data.classification: internal (assembled payload fields) + target: dcm_peer (Hub DCM), trust_posture: verified + โ†’ Decision: ALLOW (internal data, verified peer) + +Hub DCM routes to Regional DCM B: + Governance Matrix check (Hub โ†’ Regional DCM B): + same: ALLOW + Regional DCM B forwards to EU-WEST-Prod-2 + +Realized State flows back: + Provider โ†’ Regional DCM B โ†’ Hub DCM โ†’ Regional DCM A + Each hop: Governance Matrix evaluated + Final Realized State written to Regional DCM A's Realized Store + entity_uuid preserved throughout + provider_entity_id: "vm-eu-west-b-0012" +``` + +--- + +## 1.8 Brownfield Ingestion Workflow + +An existing VM discovered by a provider that DCM did not provision. + +```rego +# Orchestration Flow Policy for brownfield ingestion +package dcm.orchestration.brownfield_ingestion + +steps := [ + {"step": 1, "payload_type": "discovery.new_entity_found", + "policy_handle": "system/ingestion/create-transitional-record"}, + {"step": 2, "payload_type": "ingestion.transitional_created", + "policy_handle": "system/ingestion/enrich-from-information-providers"}, + {"step": 3, "payload_type": "ingestion.enriched", + "policy_handle": "system/ingestion/await-operator-promotion"}, + {"step": 4, "payload_type": "ingestion.promotion_approved", + "policy_handle": "system/ingestion/promote-to-tenant"} +] +ordered := true +``` + +``` +Discovery cycle finds vm-legacy-0001 (no matching Realized State UUID): + +Step 1: Event: discovery.new_entity_found +โ†’ INGEST: create Transitional entity in __transitional__ Tenant + entity_uuid assigned + lifecycle_state: INGESTION_PENDING + data_classification: internal (default) + +Step 2: Event: ingestion.transitional_created +โ†’ ENRICH: Information Providers queried: + CMDB (authority: primary): + business_unit: "Payments Platform" + cost_center: "PAYM-4421" + product_owner: "Jane Smith" + compliance_scope: "PCI-DSS" + HR System (authority: secondary): + team: "payments-platform-eng" + +Step 3: Event: ingestion.enriched +โ†’ Notification to Platform Admin: + "Brownfield entity discovered. Review and assign to Tenant." + action_url: /api/v1/admin/ingestion/ing-001/promote + +Step 4: Operator approves: +POST /api/v1/admin/ingestion/ing-001/promote +{ "target_tenant_uuid": "payments-tenant-uuid", + "compliance_overlay": "pci-dss" } + +โ†’ Entity moved from __transitional__ to payments-tenant +โ†’ Intent State created from discovered configuration +โ†’ drift detection activated +โ†’ lifecycle_state: OPERATIONAL +``` + +--- + +# Section 2 โ€” Provider Interaction Examples + +## 2.1 Service Provider โ€” Full Dispatch Cycle + +``` +DCM sends dispatch payload to Service Provider endpoint: + +POST https://provider.example.com/dispatch +Authorization: mTLS + scoped credential (scope: dispatch, entity: ent-001, ttl: PT15M) +Content-Type: application/json + +{ + "dispatch_uuid": "disp-001", + "entity_uuid": "ent-001", + "requested_state_uuid": "req-state-001", + "payload": { + "resource_type": "Compute.VirtualMachine", + "fields": { + "cpu_count": { "value": 4, "provenance": {...} }, + "memory_gb": { "value": 8, "provenance": {...} }, + "os_family": { "value": "rhel", "provenance": {...} }, + "monitoring_endpoint": { + "value": "https://metrics.internal.prod.example.com", + "provenance": { "origin": { "source_type": "policy", + "source_uuid": "transform-inject-monitoring" } } + } + } + } +} + +Provider naturalizes (DCM โ†’ OpenStack Nova): +{ + "server": { + "name": "ent-001", + "flavorRef": "m1.xlarge", # 4 vCPU, 8GB + "imageRef": "rhel-9.2-latest", + "metadata": { "dcm_entity_uuid": "ent-001", + "dcm_requested_state": "req-state-001", + "monitoring_endpoint": "https://metrics..." } + } +} + +OpenStack provisions โ†’ returns server object. + +Provider denaturalizes (OpenStack โ†’ DCM unified): +{ + "realized_state_uuid": "real-001", + "entity_uuid": "ent-001", + "corresponding_requested_state_uuid": "req-state-001", + "source_type": "initial_realization", + "fields": { + "cpu_count": { "value": 4, ... }, + "memory_gb": { "value": 8, ... }, + "provider_entity_id": { "value": "vm-0a1b2c3d" }, + "assigned_ip_address": { "value": "10.1.45.23" }, + "hypervisor_host": { "value": "compute-07.eu-west" } + } +} + +DCM receives โ†’ writes to Realized Store. +``` + +## 2.2 Information Provider โ€” Assembly Enrichment + +``` +During layer assembly Step 2 (layer resolution), DCM queries CMDB Information Provider: + +POST https://cmdb.corp.example.com/query +Authorization: mTLS +{ + "query_uuid": "qry-001", + "data_type": "business_data", + "lookup_key": { "type": "actor_uuid", "value": "actor-payments-001" } +} + +Response: +{ + "data": { + "business_unit": { "value": "Payments Platform", + "confidence": { "band": "very_high", "score": 97 }, + "authority_level": "primary" }, + "cost_center": { "value": "PAYM-4421", + "confidence": { "band": "very_high", "score": 97 } }, + "product_owner": { "value": "Jane Smith", + "confidence": { "band": "high", "score": 85 } } + }, + "data_freshness": "2026-03-15T08:00:00Z" +} + +DCM injects into assembled payload as a data layer: + business_unit.provenance.origin.source_type = "information_provider" + business_unit.provenance.origin.source_uuid = "cmdb-provider-uuid" +``` + +## 2.3 Policy Provider Mode 3 โ€” OPA Sidecar Evaluation + +``` +Assembly reaches Step 5 (pre-placement policy processing): + +DCM sends payload to OPA sidecar: +POST http://opa-sidecar:8181/v1/data/dcm/gatekeeper/vm_size_limits +{ + "input": { + "payload": { + "type": "request.layers_assembled", + "fields": { "cpu_count": { "value": 4 }, ... } + }, + "actor": { "uuid": "actor-001", "roles": ["developer"], + "tenant_uuid": "payments-uuid" }, + "deployment": { "deployment_posture": "prod", + "compliance_domains": ["hipaa"] }, + "entity": null, + "provider": null + } +} + +OPA response: +{ + "result": { + "allow": true, + "deny": [], + "field_locks": [], + "warnings": [] + } +} + +DCM Policy Engine reads result โ†’ allow โ†’ pipeline continues. +``` + +## 2.4 Notification Provider โ€” Relationship Graph Audience + +``` +Event: entity.decommissioning (VLAN-100 entering DECOMMISSIONING state) + +Notification Router: + 1. Load relationship graph for VLAN-100: + VM-A (AppTeam, attached_to, stake_strength: required) + VM-B (DevTeam, attached_to, stake_strength: required) + VM-C (OpsTeam, attached_to, stake_strength: optional) + + 2. Resolve audiences: + VLAN-100 owner (NetworkOps): audience_role = owner + VM-A owner (AppTeam admin): audience_role = stakeholder + stakeholder_reason: { via_entity: "VM-A", via_relationship: "attached_to" } + VM-B owner (DevTeam admin): audience_role = stakeholder + VM-C owner (OpsTeam admin): audience_role = observer (optional stake) + + 3. Per-actor notification envelopes generated (4 total) + +POST https://slack-notif.corp.example.com/deliver +{ + "notification_uuid": "notif-001", + "event_type": "entity.decommissioning", + "urgency": "high", + "entity": { "uuid": "vlan-100-uuid", "display_name": "VLAN-100" }, + "audience": { + "actor_uuid": "appteam-admin-uuid", + "audience_role": "stakeholder", + "stakeholder_reason": { + "via_entity_uuid": "vm-a-uuid", + "via_entity_display_name": "VM-A (payments-api-server-01)", + "via_relationship_type": "attached_to" + } + }, + "context": { "change_summary": "VLAN-100 decommission initiated" }, + "requires_action": false +} + +Slack provider delivers: + "#payments-platform: โš ๏ธ VLAN-100 is being decommissioned. + Your VM 'payments-api-server-01' is attached to it. + Action required: migrate VM network attachment before decommission completes." +``` + +--- + +# Section 3 โ€” Consumer API Examples + +## 3.1 Complete Request Lifecycle (API Perspective) + +``` +# 1. Browse catalog +GET /api/v1/catalog?category=Compute +X-DCM-Tenant: payments-tenant-uuid +Authorization: Bearer + +Response: { "catalog_items": [ + { "catalog_item_uuid": "vm-standard-uuid", + "resource_type": "Compute.VirtualMachine", + "display_name": "Standard Linux VM", + "estimated_cost": { "per_hour": 0.32, "currency": "USD" }, + "accreditations": [{ "framework": "hipaa", "status": "active" }] + } +] } + +# 2. Describe catalog item (see schema + constraints) +GET /api/v1/catalog/vm-standard-uuid + +Response includes: + "schema.fields[cpu_count].constraint": { "type": "range", "min": 1, "max": 32 } + "schema.fields[monitoring_agent].constraint.visibility": "hidden" # injected by policy + +# 3. Submit request +POST /api/v1/requests +{ "catalog_item_uuid": "vm-standard-uuid", + "fields": { "cpu_count": 4, "memory_gb": 8, "os_family": "rhel", + "name": "payments-api-server-01" } } + +Response 202: { "request_uuid": "req-001", "entity_uuid": "ent-001", + "status": "ACKNOWLEDGED", + "status_url": "/api/v1/requests/req-001/status" } + +# 4. Poll status (or use webhook) +GET /api/v1/requests/req-001/status + +Sequence of responses: + { "status": "ASSEMBLING" } # layer assembly running + { "status": "DISPATCHED" } # sent to provider + { "status": "PROVISIONING" } # provider executing + { "status": "COMPLETED", + "resource_url": "/api/v1/resources/ent-001" } + +# 5. Get realized resource +GET /api/v1/resources/ent-001 + +Response: +{ "entity_uuid": "ent-001", + "lifecycle_state": "OPERATIONAL", + "drift_status": "clean", + "fields": { + "cpu_count": { "value": 4, "confidence": { "band": "very_high" } }, + "assigned_ip_address": { "value": "10.1.45.23", + "confidence": { "band": "very_high" } } + }, + "estimated_cost_per_hour": 0.32 } +``` + +## 3.2 Provider Update Notification โ€” Consumer Approval Flow + +``` +# Provider submits auto-scale notification (memory doubled) +POST /api/v1/provider/entities/ent-001/update-notification +Authorization: mTLS (provider cert) +{ "provider_uuid": "eu-west-prod-1-uuid", + "notification_uuid": "notif-001", + "notification_type": "auto_scale", + "changed_fields": { + "memory_gb": { "previous_value": 8, "new_value": 16, + "change_reason": "Auto-scale at 85% utilization" } + } } + +โ†’ DCM evaluates: no pre-authorization policy for this tenant โ†’ REQUIRES_CONSUMER_APPROVAL +โ†’ Entity โ†’ PENDING_REVIEW +โ†’ Notification to owner: "Provider requests to update memory_gb: 8โ†’16. Approve?" + +# Consumer reviews pending notification +GET /api/v1/resources/ent-001/provider-notifications + +Response: { "notifications": [{ + "notification_uuid": "notif-001", + "notification_type": "auto_scale", + "status": "pending_approval", + "change_summary": "memory_gb: 8 โ†’ 16", + "change_reason": "Auto-scale at 85% utilization" +}] } + +# Consumer approves +POST /api/v1/resources/ent-001/provider-notifications/notif-001/approve +{ "decision": "approve", "reason": "Legitimate auto-scale event" } + +Response 202: { "decision": "approve", "realized_state_uuid": "real-002" } + +โ†’ New Requested State created (source_type: provider_update) +โ†’ New Realized State snapshot written (memory_gb: 16) +โ†’ Audit: PROVIDER_UPDATE_APPLIED +``` + +--- + +# Section 4 โ€” Admin API Examples + +## 4.1 Review and Approve Provider Registration + +``` +# New provider submitted registration +# Platform admin receives notification (urgency: medium) +# "New provider registration pending review: eu-west-prod-1" + +# List pending registrations +GET /api/v1/admin/registrations/pending +Authorization: Bearer + +Response: { "registrations": [{ + "registration_uuid": "reg-001", + "provider_type_id": "service_provider", + "handle": "org/compute/eu-west-prod-1", + "submitted_at": "2026-03-15T09:00:00Z", + "validation_status": "passed", # all 8 automated checks passed + "sovereignty_zone": "eu-west-sovereign", + "accreditations": [{ "framework": "hipaa", "type": "baa" }], + "health_check_status": "healthy", + "governance_matrix_pre_check": "ALLOW" +}] } + +# Admin reviews and approves +POST /api/v1/admin/registrations/reg-001/approve +{ "review_notes": "Certificate verified against corp CA. BAA reviewed and valid." } + +Response: { "registration_uuid": "reg-001", "status": "ACTIVE" } + +โ†’ Provider enters active registry +โ†’ Governance Matrix re-evaluated with this provider active +โ†’ Notification to provider operator: "Registration approved. Provider UUID: eu-west-prod-1-uuid" +``` + +## 4.2 Resolve Orphan Candidate + +``` +# Discovery found vm-legacy-0001 after a timeout-cancelled request + +GET /api/v1/admin/orphans +Response: { "orphan_candidates": [{ + "orphan_candidate_uuid": "orp-001", + "provider_uuid": "eu-west-prod-1-uuid", + "provider_entity_id": "vm-legacy-0001", + "suspected_request_uuid": "req-failed-001", + "resource_type": "Compute.VirtualMachine", + "discovered_at": "2026-03-15T10:30:00Z", + "status": "under_review" +}] } + +# Admin investigates: vm-legacy-0001 matches the timed-out request +# Decision: adopt into DCM lifecycle under the original requesting tenant + +POST /api/v1/admin/orphans/orp-001/resolve +{ "resolution": "adopt_into_dcm", + "reason": "Confirmed match for timed-out request req-failed-001", + "target_tenant_uuid": "payments-tenant-uuid" } + +Response: { "resolution": "adopt_into_dcm", + "new_entity_uuid": "ent-001", # original entity UUID preserved + "status": "OPERATIONAL" } + +โ†’ Entity promoted from orphan candidate to full DCM lifecycle +โ†’ Realized State written +โ†’ drift detection activated +โ†’ original request_uuid marked COMPLETED (late completion) +``` + +--- + +# Section 5 โ€” Registration Flow Example + +## 5.1 Complete Provider Onboarding โ€” Service Provider + +``` +# Step 1: Platform admin issues registration token +POST /api/v1/admin/registration-tokens +{ "provider_type_id": "service_provider", + "expires_in": "PT72H", + "scope": { + "provider_handle_pattern": "org/compute/eu-west-*", + "sovereignty_zone": "eu-west-sovereign", + "grants_auto_approval": false # human review still required + }, + "purpose": "EU-WEST production compute provider onboarding" } + +Response: { "token_uuid": "tok-001", + "token_value": "DCM_REG_abc123...", # shown once only + "expires_at": "2026-03-18T09:00:00Z" } + +# Step 2: Provider operator submits registration +POST /api/v1/provider/register +X-DCM-Registration-Token: DCM_REG_abc123... +Content-Type: application/json +# (mTLS certificate presented at TLS layer) +{ + "provider_type_id": "service_provider", + "handle": "org/compute/eu-west-prod-1", + "display_name": "EU West Production Compute", + "version": "2.1.0", + "sovereignty_declaration": { + "operating_jurisdictions": ["DE", "FR", "NL"], + "data_residency_zones": ["eu-west-sovereign"] + }, + "accreditations": [ + { "accreditation_uuid": "acc-hipaa-001", "framework": "hipaa", + "accreditation_type": "baa", "status": "active" } + ], + "capabilities": { + "resource_types": [ + { "fqn": "Compute.VirtualMachine", "spec_version": "2.1.0", + "catalog_item_uuid": "vm-standard-uuid" } + ], + "cancellation": { "supports_cancellation": true, + "cancellation_supported_during": ["DISPATCHED", "PROVISIONING"] }, + "discovery": { "supports_discovery": true, "discovery_method": "api_query" }, + "cost_metadata": { "opex_per_unit_per_hour": 0.28, "currency": "USD" } + }, + "health_endpoint": "https://eu-west-prod-1.corp.example.com/health", + "delivery_endpoint": "https://eu-west-prod-1.corp.example.com/dispatch" +} + +Response 202: { "registration_uuid": "reg-001", "status": "VALIDATING", + "token_recognized": true, "auto_approval_eligible": false } + +# Step 3: Automated validation runs (8 checks) +# V1: service_provider enabled in prod profile โœ“ +# V2: Governance Matrix pre-check: ALLOW โœ“ +# V3: Token valid, matches handle pattern โœ“ +# V4: mTLS certificate valid, corp CA chain โœ“ +# V5: Sovereignty declaration complete โœ“ +# V6: Capability declaration internally consistent โœ“ +# V7: Health endpoint reachable, returns { "status": "healthy" } โœ“ +# V8: BAA accreditation present (prod requires accreditation submission) โœ“ +โ†’ Status โ†’ PENDING_APPROVAL + +# Step 4: Platform admin notified, reviews, approves (see Section 4.1) +# Step 5: Status โ†’ ACTIVE +# Provider enters registry, capacity monitoring begins +``` + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/dcm-flow-gui-spec.md b/content/docs/architecture/specifications/dcm-flow-gui-spec.md new file mode 100644 index 0000000..364c790 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-flow-gui-spec.md @@ -0,0 +1,906 @@ +# DCM Flow GUI Specification + +> **๐Ÿ“‹ Draft** +> +> This specification defines the DCM Flow GUI โ€” the visual interface for platform engineers to compose, test, simulate, and manage DCM's policy-driven orchestration. All views, data contracts, API endpoints, and component structure are specified. Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Ready for implementation feedback +**Document Type:** Technical Specification +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [OPA Integration Specification](dcm-opa-integration-spec.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [Policy Contract](../data-model/B-policy-contract.md) | [Consumer API](consumer-api-spec.md) | [Admin API](dcm-admin-api-spec.md) + +--- + +## Abstract + +The DCM Flow GUI is the visual interface for platform engineers to compose, test, and manage DCM's data-driven orchestration. Because policies ARE the orchestration in DCM, the Flow GUI is fundamentally a **visual policy composer** โ€” it makes the active policy graph visible and editable without requiring direct YAML or Rego authoring. + +The Flow GUI is a **platform engineer tool**, not a consumer tool. It operates with platform admin or policy author role permissions. Consumers interact with DCM through the Consumer API and Web UI, not through the Flow GUI. + +--- + +## 1. Architecture and Component Structure + +### 1.1 Component Diagram + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Browser (SPA) โ”‚ +โ”‚ Flow GUI Application โ€” React single-page application โ”‚ +โ”‚ Authentication: Bearer token (same session as Consumer API) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ HTTPS REST + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Flow GUI Service โ”‚ +โ”‚ Purpose: aggregate data for Flow GUI views โ”‚ +โ”‚ Deployed alongside DCM control plane โ”‚ +โ”‚ Authentication: validates Bearer token; requires policy_author โ”‚ +โ”‚ or platform_admin role โ”‚ +โ”‚ โ”‚ +โ”‚ Reads from: โ”‚ +โ”‚ Policy Engine โ€” live graph, firing frequency โ”‚ +โ”‚ GitOps stores โ€” policy artifacts, PR status โ”‚ +โ”‚ Observability โ€” event volumes, error rates โ”‚ +โ”‚ OPA sidecar โ€” test harness, shadow results โ”‚ +โ”‚ Writes via: โ”‚ +โ”‚ Git API โ€” create PRs for policy changes โ”‚ +โ”‚ Admin API โ€” shadow mode promotion, profile changes โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +### 1.2 Authentication and Authorization + +The Flow GUI uses the same session token as the Consumer API. Required roles: + +| Role | Access | +|------|--------| +| `platform_admin` | Full read/write โ€” all views, all authoring, profile management | +| `policy_author` | Read all views; author policies in assigned domains; cannot manage profiles or promote shadow policies | +| `platform_observer` | Read-only โ€” all views; no authoring; no simulation write | + +### 1.3 Base URL + +``` +https://{dcm-instance}/flow/api/v1/ +``` + +Distinct from the Consumer API base URL to make routing and access control clear. + +--- + +## 2. The Execution Graph View + +### 2.1 What It Shows + +The primary view shows the live execution graph: which policies are active, which payload types they match, how they compose with each other, and their firing frequency. This is the "live map" of DCM's orchestration state. + +``` +[request.initiated] โ”€โ”€โ†’ [IntentCapturePolicy] โ”€โ”€โ†’ [request.intent_captured] + โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ โ–ผ + [LayerAssembly] [CostCheck] [AuthzCheck] + (system/blue) (tenant/yellow)(system/blue) + โ”‚ + โ–ผ + [request.layers_assembled] + โ”‚ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ โ–ผ + [GateKeeper: [Transform: [GovMatrix: + vm-size-limits] inject-mon.] phi-boundary] + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ–ผ + [request.policies_evaluated] +``` + +**Visual conventions:** +- **Node color by domain:** system=blue, platform=green, tenant=yellow, resource_type=purple +- **Node shape by policy type:** GateKeeper=shield, Transformation=gear, Recovery=arrow, Governance Matrix=lock, Orchestration Flow=rectangle +- **Edge thickness:** proportional to firing frequency (last 1h) +- **Edge color:** green=allow path, red=deny path, amber=conditional +- **Node badge:** shadow mode indicator (S), deprecated indicator (D) + +### 2.2 API โ€” Fetch Execution Graph + +``` +GET /flow/api/v1/graph + +Query parameters: + payload_type= filter to policies matching this payload type + resource_type= filter to policies applicable to this resource type + domain= filter by policy domain + policy_type= filter by policy type + tenant_uuid= include tenant-domain policies for this Tenant + +Response 200: +{ + "graph": { + "nodes": [ + { + "node_id": "", # policy_uuid + "label": "vm-size-limits", + "policy_type": "gatekeeper", + "domain": "tenant", + "tenant_uuid": "", + "handle": "tenant/payments/gatekeeper/vm-size-limits", + "version": "1.2.0", + "status": "active", + "shadow_mode": false, + "match_payload_types": ["request.layers_assembled"], + "match_conditions_summary": "cpu_count > 32 OR memory_gb > 256", + "firing_frequency": { + "last_1h": 3, + "last_24h": 47, + "last_7d": 312 + }, + "deny_rate_24h": 0.06 # 6% of evaluations resulted in deny + } + ], + "edges": [ + { + "from_payload_type": "request.layers_assembled", + "to_node_id": "", + "edge_type": "policy_fires_on", + "volume_24h": 47 + }, + { + "from_node_id": "", + "to_payload_type": "request.policies_evaluated", + "edge_type": "produces", + "condition": "on_allow" + } + ] + }, + "payload_types": [ + { + "payload_type": "request.layers_assembled", + "volume_24h": 789, + "active_policy_count": 4 + } + ], + "last_updated": "" +} +``` + +### 2.3 API โ€” Get Policy Node Detail + +``` +GET /flow/api/v1/graph/nodes/{policy_uuid} + +Response 200: +{ + "policy_uuid": "", + "handle": "tenant/payments/gatekeeper/vm-size-limits", + "version": "1.2.0", + "policy_type": "gatekeeper", + "domain": "tenant", + "concern_type": "security", + "enforcement": "soft", + "status": "active", + + "match_conditions": { + "payload_type": "request.layers_assembled", + "conditions": [ + { "field": "payload.fields.cpu_count.value", "operator": "gt", "value": 32 } + ] + }, + + "output_schema": { + "decision": "deny", + "reason_template": "cpu_count {value} exceeds maximum 32" + }, + + "firing_history": [ + { "timestamp": "", "result": "deny", "request_uuid": "" }, + { "timestamp": "", "result": "allow", "request_uuid": "" } + ], + + "git_path": "policy-store/tenant/payments/gatekeeper/vm-size-limits/v1.2.0.yaml", + "pr_url": null, # null if no pending PR; URL if change in review + + "compliance_basis": null, + "review_required_before": null, + + "test_suite": { + "test_count": 3, + "last_run": "", + "result": "pass" + } +} +``` + +--- + +## 3. Policy Canvas โ€” Static Flow Builder + +### 3.1 Interaction Model + +The Policy Canvas is a drag-and-drop interface for building named Orchestration Flow Policies (Level 1 orchestration โ€” named workflow artifacts). The output is a valid DCM Orchestration Flow Policy YAML committed via a Git PR. + +**Key constraint:** The canvas never writes directly to the Policy Store. All saves generate a Git PR. The PR goes through the standard review process. Shadow mode activates automatically when the PR is created โ€” the proposed workflow evaluates against real traffic in shadow mode until merged. + +### 3.2 Canvas Operations + +| Operation | Description | Backend action | +|-----------|-------------|----------------| +| Drag payload type node | Add a workflow step | Canvas state update (local) | +| Connect nodes | Declare step sequence | Canvas state update (local) | +| Set step conditions | Add conditions to a step | Canvas state update (local) | +| Set failure behavior | halt / skip / escalate | Canvas state update (local) | +| Preview YAML | Show generated policy YAML | `GET /flow/api/v1/canvas/preview` | +| Save as PR | Create Git PR with policy YAML | `POST /flow/api/v1/canvas/save` | +| Load existing | Load an existing flow policy | `GET /flow/api/v1/policies/{policy_uuid}/canvas` | + +### 3.3 API โ€” Preview Canvas as YAML + +``` +POST /flow/api/v1/canvas/preview + +Request body: +{ + "handle": "org/orchestration/vm-provisioning-flow", + "concern_type": "orchestration_flow", + "ordered": true, + "steps": [ + { + "step": 1, + "payload_type": "request.initiated", + "policy_handle": "system/orchestration/capture-intent", + "on_fail": "halt" + }, + { + "step": 2, + "payload_type": "request.intent_captured", + "policy_handle": "system/orchestration/assemble-layers", + "on_fail": "halt" + }, + { + "step": 3, + "payload_type": "request.layers_assembled", + "policy_handle": "system/orchestration/run-placement", + "on_fail": "halt", + "condition": "not payload.placement_complete" + } + ], + "applicable_resource_types": ["Compute.VirtualMachine"] +} + +Response 200: +{ + "yaml": "# Generated by DCM Flow GUI\n# Handle: org/orchestration/vm-provisioning-flow\n...", + "rego": "package dcm.orchestration.vm_provisioning_flow\n...", + "validation": { + "valid": true, + "warnings": ["Step 3 condition references 'payload.placement_complete' which is not in the standard payload vocabulary"] + } +} +``` + +### 3.4 API โ€” Save Canvas as Git PR + +``` +POST /flow/api/v1/canvas/save + +Request body: +{ + "canvas_definition": { ... }, # same as preview request + "commit_message": "Add VM provisioning orchestration flow", + "pr_title": "feat(orchestration): VM provisioning named workflow", + "pr_description": "Defines explicit sequence for VM provisioning requests", + "target_branch": "main", + "shadow_mode": true # proposed status โ€” shadow evaluates before merge +} + +Response 201 Created: +{ + "pr_uuid": "", + "pr_url": "https://git.corp.example.com/dcm-policies/pulls/142", + "pr_status": "open", + "shadow_mode_activated": true, + "policy_handle": "org/orchestration/vm-provisioning-flow", + "policy_status": "proposed" # active in shadow mode; not yet enforced +} +``` + +### 3.5 API โ€” Load Existing Flow Policy into Canvas + +``` +GET /flow/api/v1/policies/{policy_uuid}/canvas + +Response 200: +{ + "canvas_definition": { + "handle": "...", + "ordered": true, + "steps": [...] + }, + "yaml": "...", + "policy_uuid": "", + "version": "1.2.0", + "git_path": "..." +} +``` + +--- + +## 4. Policy Authoring Interface + +### 4.1 Visual Condition Builder + +For simple policies (field comparisons, role checks, quota checks), a visual condition builder generates valid Rego without requiring Rego knowledge. + +**Supported condition types:** + +| Field type | Operators | Example | +|-----------|-----------|---------| +| Numeric field | equals, not_equals, gt, gte, lt, lte, in_range | `cpu_count > 32` | +| String field | equals, not_equals, in_list, matches_regex | `os_family in [rhel, ubuntu-lts]` | +| List field | contains, does_not_contain | `actor.roles contains platform_admin` | +| Boolean field | is_true, is_false | `payload.fields.production_workload = true` | +| Existence | exists, does_not_exist | `payload.fields.cost_center exists` | + +### 4.2 API โ€” Generate Policy from Visual Conditions + +``` +POST /flow/api/v1/policies/generate + +Request body: +{ + "policy_type": "gatekeeper", + "handle": "tenant/payments/gatekeeper/vm-size-limits", + "concern_type": "security", + "domain": "tenant", + "tenant_uuid": "", + "enforcement": "soft", + "match": { + "payload_type": "request.layers_assembled", + "resource_type": "Compute.VirtualMachine", + "conditions": [ + { "field": "payload.fields.cpu_count.value", "operator": "gt", "value": 32 } + ], + "condition_logic": "any" + }, + "output": { + "decision": "deny", + "reason_template": "cpu_count {payload.fields.cpu_count.value} exceeds maximum 32 for this Tenant" + }, + "audit_on": ["DENY"], + "notification_on": ["DENY"] +} + +Response 200: +{ + "yaml": "# DCM GateKeeper Policy\n...", + "rego": "package dcm.gatekeeper.vm_size_limits\n\ndeny contains reason if {\n input.payload.type == \"request.layers_assembled\"\n input.payload.fields.cpu_count.value > 32\n reason := sprintf(\"cpu_count %d exceeds maximum 32\", [input.payload.fields.cpu_count.value])\n}\n", + "validation": { + "valid": true, + "warnings": [] + } +} +``` + +### 4.3 Rego Editor + +For complex policies requiring full Rego expressiveness, the GUI includes an embedded Rego editor with: + +- **Input schema autocomplete:** all valid `input.*` paths from the DCM input document schema +- **DCM built-in reference:** sidebar showing available built-in functions and constants +- **Real-time syntax validation:** calls OPA `/v1/compile` to validate without evaluation +- **Test case runner:** executes the policy against saved test cases + +### 4.4 API โ€” Validate Rego + +``` +POST /flow/api/v1/policies/validate-rego + +Request body: +{ + "rego": "package dcm.gatekeeper.example\n\ndeny contains reason if {\n input.payload.fields.cpu_count.value > 32\n reason := \"too many CPUs\"\n}\n", + "policy_type": "gatekeeper" +} + +Response 200: +{ + "valid": true, + "warnings": [], + "errors": [], + "output_schema_match": true, # output matches declared policy_type schema + "input_paths_used": [ + "input.payload.fields.cpu_count.value" + ], + "input_paths_unknown": [] # paths that don't exist in the input document schema +} + +Response 200 (with errors): +{ + "valid": false, + "errors": [ + { "line": 4, "column": 5, "message": "undefined variable: reason_text" } + ] +} +``` + +### 4.5 Test Case Management + +``` +# List test cases for a policy +GET /flow/api/v1/policies/{policy_uuid}/tests + +Response 200: +{ + "test_cases": [ + { + "test_uuid": "", + "name": "Reject oversized VM", + "input_payload": { "payload": { "type": "request.layers_assembled", "fields": { "cpu_count": { "value": 64 } } } }, + "expected_output": { "deny": ["cpu_count 64 exceeds maximum 32"] }, + "last_result": "pass", + "last_run": "" + } + ] +} + +# Create test case from a real recent request +POST /flow/api/v1/policies/{policy_uuid}/tests/from-request +{ + "request_uuid": "", # saves that request's payload as a test case + "expected_output": { "deny": [] }, + "test_name": "Normal VM request โ€” should allow" +} + +# Run all test cases +POST /flow/api/v1/policies/{policy_uuid}/tests/run + +Response 200: +{ + "run_uuid": "", + "result": "pass", # pass | fail | error + "test_results": [ + { + "test_uuid": "", + "name": "Reject oversized VM", + "result": "pass", + "actual_output": { "deny": ["cpu_count 64 exceeds maximum 32"] }, + "expected_output": { "deny": ["cpu_count 64 exceeds maximum 32"] } + } + ], + "duration_ms": 42 +} +``` + +--- + +## 5. Flow Simulation + +### 5.1 Simulation Model + +Platform engineers simulate a synthetic request through the active policy engine without creating real state. The simulation runs against the live Policy Engine with a caller-constructed payload. No audit records are written. No Requested State is created. + +### 5.2 API โ€” Simulate Request + +``` +POST /flow/api/v1/simulate + +Request body: +{ + "catalog_item_uuid": "", # optional; used to seed field schema + "resource_type": "Compute.VirtualMachine", + "tenant_uuid": "", + "synthetic_fields": { + "cpu_count": 64, + "memory_gb": 128, + "os_family": "rhel" + }, + "synthetic_actor": { + "roles": ["developer"], + "group_memberships": ["payments-team"] + }, + "include_policy_types": ["gatekeeper", "transformation", "governance_matrix"] +} + +Response 200: +{ + "simulation_uuid": "", + "result": "rejected", # allowed | rejected | degraded + "terminal_reason": "GateKeeper policy rejected at step request.layers_assembled", + + "execution_trace": [ + { + "step": 1, + "payload_type": "request.initiated", + "policies_evaluated": [], + "result": "pass", + "duration_ms": 2 + }, + { + "step": 2, + "payload_type": "request.intent_captured", + "policies_evaluated": [], + "result": "pass", + "duration_ms": 1 + }, + { + "step": 3, + "payload_type": "request.layers_assembled", + "policies_evaluated": [ + { + "policy_uuid": "", + "policy_handle": "tenant/payments/gatekeeper/vm-size-limits", + "policy_type": "gatekeeper", + "result": "deny", + "reason": "cpu_count 64 exceeds maximum 32", + "duration_ms": 8 + }, + { + "policy_uuid": "", + "policy_handle": "org/transformation/inject-monitoring", + "policy_type": "transformation", + "result": "applied", + "mutations": [ + { "field": "fields.monitoring_endpoint", "operation": "set", "value": "https://metrics..." } + ], + "duration_ms": 3 + } + ], + "result": "rejected", + "terminal": true + } + ], + + "assembled_payload_snapshot": { + "fields": { + "cpu_count": { "value": 64, "provenance": { "origin": { "source_type": "consumer_request" } } }, + "monitoring_endpoint": { "value": "https://metrics...", "provenance": { "origin": { "source_type": "policy" } } } + } + }, + + "cost_estimate": { + "total_per_hour": 1.28, + "currency": "USD", + "note": "Estimated assuming request would have been allowed" + } +} +``` + +### 5.3 Simulation vs Shadow Mode + +| | Simulation | Shadow Mode | +|-|-----------|------------| +| Trigger | Manual, synthetic payload | Automatic on real traffic | +| Audit record | Never written | Written to Validation Store | +| Policy status | Evaluates active policies | Evaluates proposed policies | +| Use case | "What if?" exploration | Pre-activation validation | +| Real data | No | Yes | + +--- + +## 6. Shadow Mode Dashboard + +### 6.1 What It Shows + +Shows all proposed policies currently in shadow mode and their evaluation results against real traffic. + +### 6.2 API โ€” List Shadow Policies + +``` +GET /flow/api/v1/shadow + +Response 200: +{ + "shadow_policies": [ + { + "policy_uuid": "", + "handle": "tenant/payments/gatekeeper/new-cost-check", + "policy_type": "gatekeeper", + "status": "proposed", + "shadow_since": "", + "pr_url": "https://git.corp.example.com/dcm-policies/pulls/143", + "pr_status": "open", + + "shadow_results_24h": { + "total_evaluations": 156, + "would_have_denied": 4, + "would_have_allowed": 152, + "divergence_from_active": 4, + "divergence_rate": 0.026 + } + } + ] +} +``` + +### 6.3 API โ€” Shadow Policy Detail with Divergence Cases + +``` +GET /flow/api/v1/shadow/{policy_uuid} + +Response 200: +{ + "policy_uuid": "", + "shadow_results_24h": { + "total_evaluations": 156, + "divergence_cases": [ + { + "request_uuid": "", + "timestamp": "", + "active_result": "allow", + "shadow_result": "deny", + "shadow_reason": "Estimated cost $480/month exceeds budget ceiling $300/month", + "requester": "Bob Smith", + "resource_type": "Compute.VirtualMachine" + } + ] + } +} +``` + +### 6.4 API โ€” Promote Shadow Policy to Active + +``` +POST /flow/api/v1/shadow/{policy_uuid}/promote +{ + "reason": "Shadow results reviewed โ€” divergence rate acceptable; promoting to active" +} + +Response 202 Accepted: +{ + "policy_uuid": "", + "status": "active", + "pr_action": "approved_and_merged", + "promoted_at": "" +} + +Response 403 Forbidden: +{ + "error": "insufficient_role", + "reason": "Policy promotion requires platform_admin role" +} +``` + +--- + +## 7. Profile and Governance Management + +### 7.1 Active Profile View + +``` +GET /flow/api/v1/profile + +Response 200: +{ + "deployment_posture": { + "name": "prod", + "description": "Production โ€” full zero trust, dual approval for high-trust providers, human review for all registrations", + "active_policy_groups": 12, + "hard_constraints": [ + "sovereign/classified data never crosses any boundary", + "All providers require at least self_declared accreditation" + ] + }, + "compliance_domains": [ + { + "domain": "hipaa", + "description": "HIPAA/HITECH compliance โ€” PHI classification, BAA requirements, minimum necessary principle", + "active_policy_groups": 4, + "key_requirements": ["PHI requires BAA accreditation", "All PHI interactions audited", "No PHI export without regulatory cert"] + } + ], + "recovery_posture": "notify-and-wait", + "zero_trust_posture": "full", + "total_active_policies": 47 +} +``` + +### 7.2 Payload Type Browser + +``` +GET /flow/api/v1/payload-types + +Response 200: +{ + "payload_types": [ + { + "payload_type": "request.layers_assembled", + "description": "Layer assembly complete โ€” payload enriched with all layer fields", + "volume_24h": 789, + "active_policy_count": 4, + "sample_payload": { + "type": "request.layers_assembled", + "fields": { + "cpu_count": { "value": 4 }, + "memory_gb": { "value": 8 } + } + }, + "downstream_payload_types": ["request.policies_evaluated", "recovery.gatekeeper_denied"] + } + ] +} +``` + +--- + +## 8. Notification Flow View + +### 8.1 API โ€” Notification Flow for an Entity + +``` +GET /flow/api/v1/notifications/flow/{entity_uuid} + +Response 200: +{ + "entity_uuid": "", + "entity_display_name": "VLAN-100", + "relationship_graph_depth": 2, + + "notification_audiences": [ + { + "actor_uuid": "", + "display_name": "NetworkOps Team", + "audience_role": "owner", + "stakeholder_reason": null, + "notification_providers": ["slack-corp", "pagerduty-prod"] + }, + { + "actor_uuid": "", + "display_name": "AppTeam Admin", + "audience_role": "stakeholder", + "stakeholder_reason": { + "via_entity": "VM-A", + "via_relationship": "attached_to", + "stake_strength": "required" + }, + "notification_providers": ["slack-corp"] + } + ], + + "active_notification_providers": [ + { + "provider_uuid": "", + "display_name": "slack-corp", + "status": "healthy", + "delivery_success_rate_24h": 0.998 + } + ] +} +``` + +--- + +## 9. Error Model + +All Flow GUI API errors follow the standard DCM error format: + +```json +{ + "error": "", + "message": "", + "request_id": "", + "timestamp": "" +} +``` + +| HTTP Status | Error Code | Meaning | +|-------------|-----------|---------| +| 403 | `insufficient_role` | Operation requires platform_admin or policy_author role | +| 404 | `policy_not_found` | Policy UUID not found in active policy store | +| 409 | `pr_already_open` | A PR already exists for this policy handle | +| 422 | `invalid_canvas` | Canvas definition is invalid (disconnected steps, unknown payload types) | +| 422 | `rego_invalid` | Rego syntax error or output schema mismatch | +| 422 | `simulation_failed` | Simulation could not be executed (missing fields, invalid tenant) | +| 503 | `policy_engine_unavailable` | Policy Engine unreachable โ€” graph data may be stale | +| 503 | `git_unavailable` | GitOps store unreachable โ€” PR creation unavailable | + +--- + +## 10. Conformance Levels + +**Level 1 โ€” Read-Only:** Execution Graph View (read), Profile View, Payload Type Browser, Notification Flow View. Suitable for dashboards and observability integrations. + +**Level 2 โ€” Standard:** All Level 1 plus Flow Simulation, Shadow Mode Dashboard (view only), Policy Node Detail. Required for platform engineer tooling. + +**Level 3 โ€” Full:** All Level 2 plus Policy Canvas (save as PR), Policy Authoring Interface, Test Case Management, Shadow Mode Promotion. Required for full policy lifecycle management. + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + + +--- + +## 11. Scoring Model Views + +### 11.1 Risk Score Overlay on Execution Graph + +The Execution Graph View has a **Score Mode** toggle that overlays risk scoring information: + +- Each operational-class GateKeeper node displays its `scoring_weight` +- Node background color shifts from green (weight 1โ€“20) through amber (21โ€“50) to red (51โ€“100) +- A running score accumulator shows the current aggregate as the user traces a path through the graph +- Compliance-class GateKeeper nodes display a lock icon โ€” they are always boolean + +### 11.2 API โ€” Get Score Configuration for Graph Overlay + +``` +GET /flow/api/v1/graph/scoring-overlay + +Response 200: +{ + "active_profile": "standard", + "thresholds": { + "auto_approve_below": 25, + "approval_routing": [ + { "tier": "reviewed", "max_score": 59 }, + { "tier": "verified", "max_score": 79 }, + { "tier": "authorized", "max_score": 100 } + ] + }, + "nodes": [ + { + "node_id": "", + "enforcement_class": "operational", + "scoring_weight": 35, + "avg_contribution_24h": 28.5 + } + ] +} +``` + +### 11.3 Threshold Configuration UI (Profile Management) + +The Profile and Governance Management view (Section 7) is extended with a **Scoring Thresholds** panel: + +- Visual slider showing auto_approve / reviewed / verified / authorized bands on a 0โ€“100 scale +- Signal weight configuration (pie chart showing proportional contribution of each signal) +- Policy enforcement override management (which policies are promoted/demoted in this profile) +- Live preview: "At the current thresholds, X% of last week's requests would have been auto-approved" + +### 11.4 Score Breakdown in Simulation + +The Flow Simulation output (Section 5) is extended with a score breakdown panel: + +``` +Simulation result: risk_score=47, routing=reviewed + +Score breakdown: + Operational GateKeepers: 50 ร— 0.45 = 22.5 + โ”œโ”€โ”€ cost-ceiling: +35 ("Cost $620/month exceeds $500") + โ””โ”€โ”€ off-hours: +15 ("Request outside business hours") + Completeness: 20 ร— 0.15 = 3.0 + โ””โ”€โ”€ cost_center_absent: +10 + Actor risk history: 30 ร— 0.20 = 6.0 + โ””โ”€โ”€ (2 recent events) + Quota pressure: 48 ร— 0.10 = 4.8 + โ””โ”€โ”€ (87% utilized) + Provider risk: 15 ร— 0.10 = 1.5 + โ””โ”€โ”€ (richness score: 85/100 โ†’ contribution: 1.5) + โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + Total: 37.8 โ†’ 47 (normalized) + Threshold (reviewed): 25 + Routing decision: HUMAN_REVIEW โœ“ +``` + +### 11.5 API โ€” Get Score Simulation + +``` +POST /flow/api/v1/simulate/score + +Request body: +{ + "synthetic_fields": { ... }, + "synthetic_actor": { "roles": ["developer"], "risk_history_score_override": 30 }, + "profile_override": "prod" # optional โ€” simulate with different profile thresholds +} + +Response 200: +{ + "risk_score": 47, + "routing_decision": "reviewed", + "signal_breakdown": { ... }, + "threshold_applied": 25, + "profile": "standard", + "advisory_warnings": [...] +} +``` + diff --git a/content/docs/architecture/specifications/dcm-opa-integration-spec.md b/content/docs/architecture/specifications/dcm-opa-integration-spec.md new file mode 100644 index 0000000..4a09ae9 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-opa-integration-spec.md @@ -0,0 +1,575 @@ +# DCM OPA Integration Specification + +> **๐Ÿ“‹ Draft** +> +> This specification has been promoted from Work in Progress to Draft status. All questions resolved. All 7 policy types validated with working Rego examples. OPA/Rego confirmed as complete reference implementation. It is ready for implementation feedback but has not yet been formally reviewed for final release. +> +> This specification defines the OPA integration contract for DCM Policy Providers. It is published to share design direction and invite feedback. Do not build production integrations against this specification until it reaches draft status. + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Ready for implementation feedback +**Document Type:** Technical Specification +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) + +--- + +## Abstract + +This specification defines how Open Policy Agent (OPA) integrates with the DCM Policy Engine as the reference implementation for Mode 3 Policy Providers. It defines the DCM payload schema as an OPA input document, the expected decision schema as OPA output, the built-in functions DCM provides to Rego policies, and the test harness contract for validating policies before activation. + +OPA is not required to implement DCM โ€” any Mode 3 Policy Provider can implement DCM's policy contract. However, OPA with Rego is the recommended reference implementation, and this specification enables implementors and integrators to build standards-compliant DCM policy engines. + +--- + +## 1. Introduction + +### 1.1 The Policy Engine Contract + +DCM's Policy Engine evaluates policies at multiple points in the request lifecycle. The engine receives a payload, evaluates all active matching policies, and accumulates mutations. The OPA integration maps this contract to Rego evaluation. + +DCM policy types: +- **GateKeeper** โ€” approve or reject; output is a decision (allow/deny + reason) +- **Validation** โ€” verify correctness; output is a validation result (pass/fail + details) +- **Transformation** โ€” enrich or modify; output is a set of field mutations +- **Recovery** โ€” respond to failure/ambiguity; output is a recovery action +- **Orchestration Flow** โ€” coordinate pipeline steps; output is a flow directive + +All five types share the same OPA input schema. The output schema differs per type. + +### 1.2 Mode 3 Policy Provider + +A Mode 3 Policy Provider executes OPA Rego bundles. DCM dispatches the policy input document to the OPA instance and receives the decision document. The OPA instance may be: +- Embedded within DCM (the reference implementation) +- A sidecar OPA instance (co-located with DCM) +- A remote OPA instance (requires network call; latency considerations apply) + +--- + +## 2. Input Schema โ€” DCM Payload as OPA Document + +Every OPA policy evaluation receives the following input document: + +```rego +# input document structure +input := { + # The current payload being evaluated + "payload": { + "type": "request.initiated", # payload type from the vocabulary + "entity_uuid": "...", + "resource_type": "Compute.VirtualMachine", + "version": "2.1.0", + "fields": { + "cpu_count": { + "value": 4, + "provenance": { "origin": {...}, "modifications": [...] } + } + # ... all assembled fields with provenance + } + }, + + # The requesting actor context + "actor": { + "uuid": "...", + "type": "human", # human | service_account | system + "tenant_uuid": "...", + "roles": ["developer"], + "groups": ["payments-team", "eu-west-users"], + "mfa_verified": true, + "auth_level": "oidc_mfa" + }, + + # The active deployment governance + "deployment": { + "posture": "prod", + "compliance_domains": ["hipaa", "gdpr"], + "recovery_posture": "notify-and-wait", + "profile_uuid": "..." + }, + + # Entity context (null for new requests) + "entity": { + "uuid": "...", + "lifecycle_state": "OPERATIONAL", + "ownership_model": "whole_allocation", + "owned_by_tenant_uuid": "...", + "relationship_count": 3, + "drift_status": "clean" + }, + + # Provider context (null before placement) + "provider": { + "uuid": "...", + "sovereignty_declaration": {...}, + "trust_score": 94, + "capacity_confidence": "high" + }, + + # DCM built-in data (resolved by DCM before OPA evaluation) + "dcm": { + "tenant": { + "uuid": "...", + "display_name": "Payments Platform", + "active_entity_count": { "Compute.VirtualMachine": 47 }, + "compliance_overlays": ["hipaa"] + }, + "cost_estimate": { + "per_hour": 0.32, + "confidence": "high" + } + } +} +``` + +--- + +## 3. Output Schema โ€” OPA Decision Documents + +### 3.1 GateKeeper Output + +```rego +package dcm.gatekeeper.vm_size_limits + +import future.keywords + +# Main decision +allow if { + input.payload.fields.cpu_count.value <= max_cpu +} + +deny contains reason if { + input.payload.fields.cpu_count.value > max_cpu + reason := sprintf("cpu_count %d exceeds maximum %d for tenant %s", + [input.payload.fields.cpu_count.value, max_cpu, input.actor.tenant_uuid]) +} + +# DCM reads the deny set; empty = allow +max_cpu := 32 +``` + +DCM output contract: +```json +{ + "allow": true, + "deny": [], + "warnings": [], + "policy_uuid": "...", + "evaluated_at": "..." +} +``` + +### 3.2 Transformation Output + +```rego +package dcm.transformation.inject_monitoring + +mutations contains mutation if { + input.payload.type == "request.layers_assembled" + not input.payload.fields.monitoring_endpoint + mutation := { + "field": "monitoring_endpoint", + "value": concat(".", ["https://metrics.internal", input.deployment.posture, "example.com"]), + "source_type": "policy", + "operation_type": "enrichment", + "reason": "Standard monitoring endpoint injection" + } +} +``` + +DCM output contract: +```json +{ + "mutations": [ + { + "field": "monitoring_endpoint", + "value": "https://metrics.internal.prod.example.com", + "source_type": "policy", + "operation_type": "enrichment", + "reason": "Standard monitoring endpoint injection" + } + ], + "policy_uuid": "..." +} +``` + +### 3.3 Recovery Policy Output + +```rego +package dcm.recovery.discard_on_timeout + +action := "DISCARD_AND_REQUEUE" if { + input.payload.type == "recovery.timeout_fired" + input.entity.lifecycle_state == "TIMEOUT_PENDING" +} +``` + +DCM output contract: +```json +{ + "action": "DISCARD_AND_REQUEUE", + "action_parameters": { "requeue_delay": "PT0S" }, + "policy_uuid": "..." +} +``` + +--- + +## 4. DCM Built-in Functions for Rego + +DCM provides built-in functions callable from Rego policies: + +```rego +# Entity relationship graph queries +dcm.entity.relationships(entity_uuid) + # Returns: array of relationship records for the entity + +dcm.entity.has_relationship(entity_uuid, relationship_type) + # Returns: bool + +dcm.entity.stakeholder_count(entity_uuid, min_stake_strength) + # Returns: int + +# Information Provider data +dcm.entity.field_confidence(entity_uuid, field_path) + # Returns: { band, score, authority_level } + +# Sovereignty checks +dcm.sovereignty.compatible(entity_uuid, provider_uuid) + # Returns: bool + +dcm.sovereignty.violates(entity_uuid, data_residency_requirement) + # Returns: bool + +# Cost queries +dcm.cost.estimate(catalog_item_uuid, fields) + # Returns: { per_hour, currency, confidence } + +# Tenant quota queries +dcm.tenant.active_count(tenant_uuid, resource_type) + # Returns: int + +dcm.tenant.has_authorization(granting_tenant_uuid, consuming_tenant_uuid, resource_type) + # Returns: bool +``` + +--- + +## 5. Policy Bundle Structure + +OPA policies for DCM are packaged as bundles: + +``` +dcm-policy-bundle/ +โ”œโ”€โ”€ .manifest +โ”‚ { +โ”‚ "roots": ["dcm"], +โ”‚ "metadata": { +โ”‚ "dcm_policy_type": "gatekeeper", +โ”‚ "resource_types": ["Compute.VirtualMachine"], +โ”‚ "domain": "tenant", +โ”‚ "handle": "org/policies/vm-size-limits", +โ”‚ "version": "1.0.0" +โ”‚ } +โ”‚ } +โ”œโ”€โ”€ dcm/ +โ”‚ โ””โ”€โ”€ gatekeeper/ +โ”‚ โ””โ”€โ”€ vm_size_limits/ +โ”‚ โ””โ”€โ”€ policy.rego +โ””โ”€โ”€ tests/ + โ””โ”€โ”€ vm_size_limits_test.rego +``` + +--- + +## 6. Test Harness + +DCM provides a test harness that policy authors use to validate policies against sample payloads before activation: + +``` +POST /api/v1/admin/policies/test + +{ + "policy_bundle": "", + "test_cases": [ + { + "description": "VM within CPU limit should be allowed", + "input": { + "payload": { "type": "request.initiated", "fields": { "cpu_count": { "value": 4 } } }, + "actor": { "roles": ["developer"] }, + "deployment": { "posture": "prod" } + }, + "expected_output": { "allow": true, "deny": [] } + } + ] +} +``` + +The test harness is also used during shadow mode โ€” DCM runs the policy against real traffic and compares actual output to expected output before the policy activates. + +--- + +## 7. Policy Shadow Mode with OPA + +When a policy is in `proposed` status, DCM evaluates it in shadow mode: + +1. Policy bundle loaded into a shadow OPA instance +2. Every real request payload is evaluated by both active policies AND shadow policies +3. Shadow outputs recorded in the Validation Store (not applied to requests) +4. Policy authors review shadow results via the Admin API or Flow GUI +5. On approval (no adverse results): policy status โ†’ `active` + + +--- + +## 8. Policy Model Validation โ€” All Seven Types + +This section validates that OPA/Rego can express all seven DCM policy types and both levels of the orchestration model. Each type is shown with a working Rego example and an assessment. + +### 8.1 GateKeeper + +```rego +package dcm.gatekeeper.vm_size_limits + +import future.keywords + +allow if { + input.payload.type == "request.layers_assembled" + input.payload.fields.cpu_count.value <= 32 +} + +deny contains reason if { + input.payload.type == "request.layers_assembled" + input.payload.fields.cpu_count.value > 32 + reason := sprintf("cpu_count %d exceeds maximum 32", + [input.payload.fields.cpu_count.value]) +} + +field_locks contains lock if { + input.deployment.compliance_domains[_] == "hipaa" + lock := {"field": "fields.patient_id", "lock_type": "immutable"} +} +``` +**Assessment:** Clean. Set-based deny with reasons, allow rules, field locks as set output. + +### 8.2 Validation + +```rego +package dcm.validation.memory_alignment + +field_results contains result if { + input.payload.fields.memory_gb.value % 2 != 0 + result := { + "field": "fields.memory_gb", + "result": "invalid", + "message": "memory_gb must be a power of 2" + } +} + +result := "pass" if count(field_results) == 0 +result := "fail" if count(field_results) > 0 +``` +**Assessment:** Clean. Set comprehension for field results. + +### 8.3 Transformation + +```rego +package dcm.transformation.inject_monitoring + +import future.keywords + +mutations contains mutation if { + input.payload.type == "request.layers_assembled" + not input.payload.fields.monitoring_endpoint + mutation := { + "field": "fields.monitoring_endpoint", + "operation": "set", + "value": concat(".", ["https://metrics.internal", + input.deployment.deployment_posture, "example.com"]), + "reason": "Standard monitoring endpoint injection", + "source_type": "enrichment" + } +} +``` +**Assessment:** Clean. Multiple mutations as independent set members. + +### 8.4 Recovery + +```rego +package dcm.recovery.timeout_response + +action := "NOTIFY_AND_WAIT" if { + input.payload.type == "recovery.timeout_fired" + input.deployment.deployment_posture in ["prod", "fsi", "sovereign"] +} + +action := "DRIFT_RECONCILE" if { + input.payload.type == "recovery.timeout_fired" + input.deployment.deployment_posture in ["minimal", "dev", "standard"] +} + +action_parameters := {"deadline": "PT4H", "on_deadline_exceeded": "ESCALATE"} + if action == "NOTIFY_AND_WAIT" +``` +**Assessment:** Clean. Conditional action based on trigger + context. + +### 8.5 Orchestration Flow (Named Workflow) + +```rego +package dcm.orchestration.request_lifecycle + +steps := [ + {"step": 1, "payload_type": "request.initiated", + "policy_handle": "system/orchestration/capture-intent", "on_fail": "halt"}, + {"step": 2, "payload_type": "request.intent_captured", + "policy_handle": "system/orchestration/assemble-layers", "on_fail": "halt"}, + {"step": 3, "payload_type": "request.layers_assembled", + "policy_handle": "system/orchestration/run-placement", "on_fail": "halt"}, + {"step": 4, "payload_type": "request.placement_complete", + "policy_handle": "system/orchestration/dispatch", "on_fail": "halt"} +] + +ordered := true +``` +**Assessment:** Clean. Step sequence as an array with `ordered: true` flag. GateKeeper and Transformation policies declared in separate packages fire on the same payload types independently โ€” the Policy Engine coordinates both. + +### 8.6 Governance Matrix Rule + +```rego +package dcm.governance_matrix.phi_federation + +import future.keywords + +decision := "DENY" if { + input.data.classification == "phi" + input.target.type == "dcm_peer" + not "hipaa" in input.target.accreditation_held +} + +decision := "ALLOW_WITH_CONDITIONS" if { + input.data.classification == "phi" + input.target.type == "dcm_peer" + "hipaa" in input.target.accreditation_held + input.target.trust_posture == "verified" +} + +field_permissions := { + "mode": "allowlist", + "paths": ["fields.resource_type", "fields.lifecycle_state"], + "on_blocked_field": "STRIP_FIELD" +} if decision == "ALLOW_WITH_CONDITIONS" + +enforcement := "hard" if decision == "DENY" +enforcement := "soft" if decision != "DENY" +``` +**Assessment:** Clean. Four-axis input maps directly to OPA's input document. Decision + field permissions + enforcement as structured output. + +### 8.7 Lifecycle Policy + +```rego +package dcm.lifecycle.required_dependency + +import future.keywords + +on_related_destroy := "cascade" if { + input.payload.type == "relationship.related_entity_destroying" + input.relationship.stake_strength == "required" +} + +on_related_destroy := "notify" if { + input.payload.type == "relationship.related_entity_destroying" + input.relationship.stake_strength == "preferred" +} + +propagation_depth := 1 +action_delay := "PT0S" +``` +**Assessment:** Clean. Relationship event conditions; action output. + +--- + +## 9. Three Things the Policy Engine Does That OPA Does Not + +OPA evaluates each package independently and returns results. The Policy Engine provides three coordination functions that OPA alone cannot: + +**1. Cross-policy ordered enforcement:** OPA produces the Orchestration Flow step sequence; the Policy Engine tracks which steps have fired and enforces ordering. Clean separation โ€” OPA declares; Policy Engine enforces. + +**2. Hard enforcement composition:** OPA returns `enforcement: "hard"` as output metadata; the Policy Engine ensures hard DENY wins over all soft decisions. Clean โ€” OPA produces the flag; Policy Engine applies the composition algorithm. + +**3. Domain precedence sequencing:** Multiple packages match the same payload type. The Policy Engine evaluates them in domain precedence order (system โ†’ platform โ†’ tenant โ†’ resource_type โ†’ entity) and composes results. Clean โ€” each OPA package is stateless and independently evaluable; Policy Engine manages composition. + +**Conclusion:** OPA/Rego is a complete reference implementation for all seven DCM policy types and both levels of the orchestration model. No model gaps exist. + + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + + +--- + +## Scoring Model โ€” OPA/Rego Patterns + +### Operational GateKeeper Output Schema + +```rego +package dcm.gatekeeper.operational.cost_ceiling + +# Operational-class GateKeeper produces risk_score_contribution, not deny +# enforcement_class: operational is declared in policy YAML metadata + +risk_score_contribution[result] { + input.payload.cost_estimate.per_month > 500 + result := { + "contribution": 35, + "label": "cost_ceiling_exceeded", + "reason": sprintf( + "Estimated monthly cost $%v exceeds Tenant ceiling $500", + [input.payload.cost_estimate.per_month] + ) + } +} + +# Operational GateKeepers can also produce hard deny for extreme values +deny contains reason { + input.payload.cost_estimate.per_month > 10000 + reason := "Cost exceeds absolute maximum โ€” manual review required before submission" +} +``` + +### Advisory Validation Output Schema + +```rego +package dcm.validation.advisory.cost_center + +# Advisory-class Validation produces completeness_contribution + warning +# output_class: advisory is declared in policy YAML metadata + +completeness_warnings[warning] { + not input.payload.fields.cost_center + warning := { + "contribution": 10, + "warning_code": "recommended_field_absent", + "warning_message": "cost_center not provided โ€” cost attribution will use Tenant default", + "field": "fields.cost_center" + } +} +``` + +### Validation โ€” Structural vs Advisory in Same Package + +```rego +package dcm.validation.vm_fields + +# Structural validation (output_class: structural) +fail contains reason { + not input.payload.fields.cpu_count + reason := { + "field": "fields.cpu_count", + "code": "required_field_absent", + "message": "cpu_count is required" + } +} + +# Advisory validation (output_class: advisory โ€” separate policy) +# Never mix structural and advisory in the same policy artifact +``` + diff --git a/content/docs/architecture/specifications/dcm-operator-interface-spec.md b/content/docs/architecture/specifications/dcm-operator-interface-spec.md new file mode 100644 index 0000000..3715a76 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-operator-interface-spec.md @@ -0,0 +1,1137 @@ +# DCM Operator Interface Specification + + +> ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress +> +> All questions resolved. Level 0โ€“4 conformance levels defined. Cluster-scoped resource ownership clarified. CAPI integration specified. +> +> **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.** +> +> The Kubernetes operator integration layer โ€” including the Operator Interface Specification, Operator SDK API, and Kubernetes compatibility mappings โ€” represents design intent that has not yet been validated against implementation. Specific interface contracts, API signatures, SDK method names, and CRD structures **will change** as implementation work begins. +> +> **Do not build against these specifications yet.** They are published to share design direction and invite feedback, not as stable contracts. +> +> Known gaps and open items for this section: +> - Operator Interface Specification: reconciliation hook signatures are provisional +> - Operator SDK API: Go module structure and dependency model not yet finalized +> - Kubernetes Compatibility Mappings: some concept mappings remain under discussion +> - SDK code examples are illustrative only โ€” not yet tested against a real implementation +> +> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). + + + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Ready for implementation feedback +**Document Type:** Technical Specification +**Maintainers:** Red Hat FlightPath Team +**GitHub:** https://github.com/dcm-project +**Last Updated:** 2026-03 + +--- + +## Abstract + +This specification defines the interface by which Kubernetes operators integrate with the DCM (Data Center Management) control plane as first-class Service Providers. An operator that conforms to this specification becomes a DCM Service Provider, enabling its managed resources to participate in DCM's unified lifecycle management, multi-tenancy, policy governance, cost analysis, drift detection, and service catalog. + +DCM is designed as a superset of Kubernetes โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries. This specification is the technical contract that enables that extension without requiring operators to abandon their existing Kubernetes-native design. + +Operators conforming to this specification function as Service Providers within a single DCM instance. In federated deployments (Hub-Spoke or Peer topology), the operator registers with the appropriate Regional or local DCM instance โ€” federation routing is handled by DCM, not by the operator. + +--- + +## 1. Introduction + +> **OIS Versioning:** Providers declare the OIS version they implement in capability registration (`ois_version`). DCM maintains dispatch compatibility with all supported OIS versions during the deprecation window. See [API Versioning Strategy](../data-model/34-api-versioning-strategy.md) Section 7. + + +### 1.1 Motivation + +Kubernetes operators are the most mature pattern for managing complex, stateful resources declaratively on Kubernetes. However, operators operate within a single cluster and lack the cross-cluster lifecycle management, multi-tenancy, cost attribution, sovereignty governance, and policy enforcement that enterprise organizations require at scale. + +DCM provides these capabilities at the management plane level โ€” above individual clusters. By conforming to this specification, an operator's managed resources become: + +- **Multi-tenant** โ€” DCM Tenant ownership and isolation applied automatically +- **Cost-attributed** โ€” resource costs tracked and attributed across the full lifecycle +- **Policy-governed** โ€” organizational policies applied at request time via DCM's Policy Engine +- **Cross-cluster** โ€” the same resource type managed across multiple clusters through DCM +- **Self-service** โ€” automatically available in the DCM Service Catalog for consumer request +- **Sovereignty-compliant** โ€” placement and operational constraints enforced by DCM's GateKeeper policies +- **Audit-complete** โ€” full provenance chain from intent through realization + +### 1.2 Scope + +This specification defines: +- The HTTP API an operator must expose to participate in DCM +- The data format for all API payloads (DCM Unified Data Model) +- The registration, health, capacity, status, and lifecycle event contracts +- The field mapping specification for translating between DCM format and CRD format +- Conformance levels and what each level unlocks in DCM + +This specification does not define: +- How operators implement their internal reconciliation logic +- Which specific Kubernetes distributions operators must support +- The internal architecture of the DCM control plane +- Provider-specific business logic or domain knowledge + +### 1.3 Relationship to the DCM Service Provider Contract + +This specification is a Kubernetes-specific instantiation of the DCM Service Provider Contract. All general Service Provider Contract requirements apply. This specification adds Kubernetes-specific requirements and guidance. Where this specification and the general Service Provider Contract conflict, this specification takes precedence for Kubernetes operator implementations. + +### 1.4 Terminology + +- **Operator** โ€” a Kubernetes controller that manages custom resources via a Custom Resource Definition (CRD) +- **DCM Control Plane** โ€” the DCM management system that routes requests and manages lifecycle +- **Adapter** โ€” a component that sits between DCM and an operator, implementing this specification on the operator's behalf (used when the operator cannot be modified directly) +- **Native implementation** โ€” an operator that implements this specification directly, without an adapter +- **CR** โ€” Custom Resource โ€” an instance of a CRD managed by the operator +- **CRD** โ€” Custom Resource Definition โ€” the Kubernetes schema definition for a CR +- **Reconciliation loop** โ€” the operator's control loop that drives actual state toward desired state + +--- + +## 2. Conformance Levels + +This specification defines three conformance levels. Higher levels unlock additional DCM capabilities. An operator may implement any level โ€” DCM accepts operators at all levels, with capabilities gated by the declared conformance level. + +**Design principle:** Level 1 must be achievable in a single day of work for an existing operator. Level 3 is the target for operators that want full DCM integration. The SDK (see Section 9) handles all protocol concerns โ€” operator developers only implement business logic. + +### 2.1 Level 1 โ€” Basic + +**What it requires:** +- Operator registration with DCM on startup +- Health check endpoint (`GET /health`) +- Basic status reporting to DCM when resource state changes + +**What it unlocks:** +- Operator resources appear in the DCM Service Catalog +- Basic lifecycle state tracking (PROVISIONING, OPERATIONAL, FAILED, DECOMMISSIONED) +- Health monitoring via DCM Observability +- Basic cost tracking (resource exists/does not exist) + +**Estimated implementation effort:** 1 day using the DCM Operator SDK + +### 2.2 Level 2 โ€” Standard + +Level 2 conformance is required for providers that support auto-scaling, auto-healing, or provider-side maintenance operations. Level 2 includes all Level 1 requirements plus the Provider Update Notification API (Section 7a). + + +**What it requires:** All Level 1 requirements, plus: +- Capacity reporting to DCM (scheduled registration) +- Full lifecycle event reporting (DEGRADED, MAINTENANCE, UNSANCTIONED_CHANGE, etc.) +- Complete realized state payloads in DCM Unified Data Model format +- Field mapping declaration (CRD fields mapped to DCM Resource Type fields) + +**What it unlocks:** All Level 1 capabilities, plus: +- Intelligent placement โ€” DCM can route requests based on real capacity data +- Drift detection โ€” DCM compares discovered state against realized state +- Full cost attribution โ€” granular resource cost tracking throughout lifecycle +- Cross-cluster management โ€” DCM can route the same resource type to multiple clusters +- Dependency graph participation โ€” operator resources participate in DCM entity relationships + +**Estimated implementation effort:** 2-3 days using the DCM Operator SDK + +### 2.3 Level 3 โ€” Full + +**What it requires:** All Level 2 requirements, plus: +- Sovereignty capability declaration +- Field-level provenance in realized state payloads +- Override control metadata support +- Discovery endpoint (`POST /discover`) โ€” operator can discover existing resources for brownfield ingestion +- Decommission confirmation callback + +**What it unlocks:** All Level 2 capabilities, plus: +- Sovereignty enforcement โ€” DCM can enforce placement and operational constraints per regulatory requirements +- Full audit chain โ€” complete provenance from intent through realization +- Brownfield ingestion โ€” existing resources can be imported into DCM lifecycle management +- Override control enforcement โ€” policy-set field locks honored in operator requests + +**Estimated implementation effort:** 3-5 days using the DCM Operator SDK + +--- + +## 3. Registration API + +### 3.1 Overview + +Operators register with DCM on startup. Registration informs DCM of the operator's endpoint, the resource types it manages, its capabilities, and its conformance level. Registration is idempotent โ€” re-registering with the same name updates the existing registration rather than creating a duplicate. + +### 3.2 Registration Endpoint + +**DCM endpoint:** `POST /api/v1/providers` + +**Timing:** Called by the operator (or adapter) during startup, after the HTTP server is ready. Retried with exponential backoff on failure. Registration failure does not block operator startup โ€” the operator functions normally for Kubernetes consumers even if DCM registration fails. + +### 3.3 Registration Payload + +```yaml +# Registration request payload +provider_registration: + name: + display_name: + conformance_level: <1|2|3> + endpoint: + version: + + service_types: + - service_type: + service_type_uuid: + crd_reference: + group: + version: + kind: + operations_supported: [CREATE, READ, UPDATE, DELETE, DISCOVER] + # DISCOVER only required for Level 3 + field_mapping_ref: + + kubernetes: + cluster_id: + cluster_endpoint: + namespace_strategy: + # per_tenant: one namespace per DCM Tenant + # shared: all DCM resources in one namespace, isolated by labels + # per_resource: one namespace per resource instance + + metadata: + region: + zone: + cluster_type: + cluster_version: + + # Level 2+ required + capacity: + update_mode: + update_frequency_seconds: + + # Level 3 required + sovereignty_capabilities: + data_residency_regions: [] + operational_sovereignty: + hard_tenancy_supported: + air_gapped_capable: + compliance_frameworks: [] +``` + +### 3.4 Registration Response + +```yaml +# Success response +provider_registration_response: + provider_id: + name: + status: + conformance_level_accepted: <1|2|3> + capabilities_enabled: + - service_catalog + - health_monitoring + - cost_tracking + # Level 2+ + - placement + - drift_detection + - cross_cluster_management + # Level 3 + - sovereignty_enforcement + - brownfield_ingestion + - full_audit_chain +``` + +--- + +## 4. Health Check API + +### 4.1 Overview + +DCM polls the operator's health endpoint every 10 seconds (configurable). A healthy operator is eligible to receive new resource requests. An unhealthy operator is excluded from placement decisions. + +### 4.2 Health Endpoint + +**Endpoint:** `GET /health` +**Authentication:** Unauthenticated (or internally secured โ€” operator choice) +**Expected response:** HTTP 200 OK for healthy or warn status; any non-200 for unhealthy (fail) + +The health response body is **normative**. DCM uses the `status` field to determine provider health and trigger alerts. Providers that return a non-conforming or absent body are treated as `warn` until three consecutive failures, after which they are treated as `fail`. + +```http +GET /health HTTP/1.1 + +HTTP/1.1 200 OK +Content-Type: application/health+json + +{ + "status": "pass", // REQUIRED: "pass" | "warn" | "fail" + "version": "", // REQUIRED: provider software version + "dcm_registration_status": "registered", // REQUIRED: "registered" | "unregistered" | "error" + "uptime_seconds": 86423, // RECOMMENDED: seconds since last restart + "checks": { // RECOMMENDED: per-subsystem health + "provider_backend": { + "status": "pass", + "observed_at": "" + }, + "credential_provider_connectivity": { + "status": "pass", + "observed_at": "" + } + }, + "details": {} // OPTIONAL: operator-specific additional detail +} +``` + +**Status semantics:** + +| Status | HTTP code | Meaning | DCM behavior | +|--------|-----------|---------|--------------| +| `pass` | 200 | Fully operational | No action | +| `warn` | 200 | Operational but degraded | Fires `provider.degraded` event; alert platform admin | +| `fail` | any non-200 | Not operational | Fires `provider.unhealthy` event; triggers recovery policy | + +The health endpoint format follows [RFC 8615 / IANA health+json](https://www.iana.org/assignments/media-types/application/health+json). + +**DCM polling behavior:** +- Polling interval: declared in provider capability registration (`health_check_interval`, default PT30S) +- Consecutive `fail` threshold before `provider.unhealthy` event: 3 (profile-governed) +- Recovery: first `pass` after `fail` fires `provider.healthy` event + +### 4.3 State Machine + +- **Ready** โ€” HTTP 200 received. Operator eligible for new requests. +- **NotReady** โ€” Non-200 or timeout received 3 consecutive times (configurable threshold). Operator excluded from placement. Existing resources not affected. +- **Recovery** โ€” Single HTTP 200 transitions NotReady back to Ready immediately. + +--- + +## 5. Capacity Reporting API + +*Required for Level 2 conformance.* + +### 5.1 Overview + +DCM maintains an internal capacity rating per operator, per service type, per location. Operators report capacity on a configurable schedule. DCM uses capacity data for intelligent placement decisions. + +### 5.2 Capacity Registration + +**DCM endpoint:** `POST /api/v1/providers/{provider_id}/capacity` + +```yaml +capacity_report: + provider_id: + report_timestamp: + next_report_at: + capacity_by_service_type: + - service_type_uuid: + available_units: + reserved_units: + committed_units: + unit_definition: + kubernetes_resources: + available_cpu: + available_memory: + available_storage: + node_count: +``` + +### 5.3 Capacity Denial + +When DCM dispatches a request the operator cannot fulfill, the operator **must** reject it with `INSUFFICIENT_RESOURCES`. DCM receives the denial and retries with an alternative provider. + +```yaml +# Denial response to a resource creation request +denial_response: + request_id: + denial_reason: INSUFFICIENT_RESOURCES + denial_timestamp: + service_type_uuid: + estimated_available_at: + details: +``` + +DCM updates its internal capacity rating for this operator immediately upon receiving a denial. + +--- + +## 6. Resource Lifecycle API + +### 6.1 Overview + +DCM dispatches resource lifecycle operations to the operator via standard REST endpoints. The operator translates these into Kubernetes CR operations (Naturalization) and reports results back to DCM in DCM Unified Data Model format (Denaturalization). + +### 6.2 Standard Endpoints + +| Method | Endpoint | Description | Required Level | +|--------|----------|-------------|---------------| +| `POST` | `/api/v1/{service_type}` | Create a new resource | Level 1 | +| `GET` | `/api/v1/{service_type}` | List all resources | Level 1 | +| `GET` | `/api/v1/{service_type}/{resource_id}` | Get a specific resource | Level 1 | +| `PUT` | `/api/v1/{service_type}/{resource_id}` | Update a resource | Level 2 | +| `DELETE` | `/api/v1/{service_type}/{resource_id}` | Delete a resource | Level 1 | +| `POST` | `/api/v1/{service_type}/discover` | Discover existing resources | Level 3 | + +### 6.3 Create Request + +DCM sends the Requested State payload to the operator. The operator naturalizes it to a Kubernetes CR and submits it. The operator responds immediately with a PROVISIONING status โ€” not waiting for reconciliation to complete. + +```yaml +# Create request from DCM โ€” Requested State payload in DCM format +create_request: + request_id: + tenant_uuid: + resource_type_uuid: + resource_type_name: Storage.Database + spec: + + relationships: + + metadata: + override_control: + +``` + +```yaml +# Create response โ€” immediate acknowledgment +create_response: + resource_id: + dcm_request_id: + lifecycle_state: PROVISIONING + kubernetes_reference: + namespace: + name: + uid: +``` + +### 6.4 Realized State Payload + +When the operator's reconciliation loop completes provisioning, it pushes the realized state to DCM. This is the critical Denaturalization step โ€” translating Kubernetes-native status into DCM Unified Data Model format. + +**DCM endpoint:** `PUT /api/v1/instances/{resource_id}/status` + +```yaml +# Realized state payload โ€” DCM Unified Data Model format +realized_state: + resource_id: + dcm_entity_uuid: + lifecycle_state: + realized_timestamp: + + spec: + + + + # Level 3 โ€” provenance for each field + field_provenance: + : + source_type: provider + source_uuid: + timestamp: + + kubernetes_reference: + namespace: + name: + uid: + resource_version: + + relationships: + +``` + +### 6.5 Delete and Decommission + +When DCM requests deletion, the operator deletes the CR and confirms decommission via the realized state endpoint with `lifecycle_state: DECOMMISSIONED`. + +For **Level 3**, the operator must wait for DCM confirmation before deleting โ€” this allows DCM to apply lifecycle policies (retain, detach) before the operator acts. + +```yaml +# Decommission confirmation callback (Level 3) +# DCM calls this before the operator deletes +decommission_confirmation: + resource_id: + lifecycle_policies_applied: + - entity_uuid: + policy_applied: retain + # storage was retained, not deleted with the parent + - entity_uuid: + policy_applied: destroy + proceed_with_deletion: +``` + +--- + + +--- + +## 7a. Provider Update Notification API + +This section defines the Provider Update Notification endpoint โ€” the formal mechanism by which Service Providers report authorized state changes to DCM. This is a **Level 2** conformance requirement for providers that support auto-scaling, auto-healing, or provider-side maintenance operations. + +### 7a.1 Overview + +The Provider Update Notification API enables providers to report authorized state changes so DCM can update its Realized State with a traceable Requested State record. This is distinct from drift โ€” a provider submitting an update notification is asserting that the change was authorized (by a pre-existing policy or operational agreement). DCM evaluates the assertion and decides whether to accept or reject it. + +**Key principle:** Providers never write directly to DCM's Realized State. They submit a notification; DCM processes it through its governance pipeline; DCM writes the Realized State if approved. + +### 7a.2 Conformance Requirements + +| Conformance Level | Requirement | +|------------------|-------------| +| Level 1 โ€” Basic | Not required. Providers at Level 1 report all state changes as lifecycle events; DCM handles them as drift. | +| Level 2 โ€” Standard | Required for providers that implement auto-scaling, auto-healing, or provider-side maintenance. | +| Level 3 โ€” Full | Required. All authorized provider-side state changes must use this API. | + +### 7a.3 Endpoint + +``` +POST /api/v1/provider/entities/{entity_uuid}/update-notification +Host: {dcm-instance} +Authorization: mTLS (provider certificate) +Content-Type: application/json +``` + +**Note:** This endpoint is on the DCM API Gateway, not on the provider. Providers call DCM; DCM does not poll providers for updates. + +### 7a.4 Request Payload + +```json +{ + "provider_uuid": "", + "notification_uuid": "", + "notification_type": "authorized_change | maintenance_change | auto_scale | auto_heal", + "changed_fields": { + "": { + "previous_value": "", + "new_value": "", + "change_reason": "", + "authorizing_policy_ref": "" + } + }, + "effective_at": "", + "provider_evidence_ref": "" +} +``` + +**`notification_uuid`** is an idempotency key. If DCM receives the same `notification_uuid` twice, it acknowledges the second request without reprocessing. + +**`authorizing_policy_ref`** is the UUID of the DCM policy that pre-authorized this type of change. If null, DCM will evaluate whether a policy covers this change. If no policy covers it, the notification is rejected. + +### 7a.5 Response Codes + +| Response | Meaning | +|----------|---------| +| `202 Accepted` | Notification accepted. DCM is processing. Use `notification_status_url` to poll. | +| `200 OK` (with `status: approved`) | Notification accepted and Realized State updated. | +| `200 OK` (with `status: pending_approval`) | Notification queued pending consumer approval. Entity in PENDING_REVIEW. | +| `200 OK` (with `status: rejected`) | Notification rejected. Realized State not updated. Discrepancy is now drift. | +| `409 Conflict` | A notification for this entity is already being processed. Retry after the `retry_after` interval. | +| `422 Unprocessable` | Notification payload malformed or entity UUID not found in this provider's scope. | + +```json +{ + "notification_uuid": "", + "status": "approved | pending_approval | rejected", + "realized_state_uuid": "", + "rejection_reason": "", + "retry_after": "", + "notification_status_url": "/api/v1/provider/notifications/{notification_uuid}" +} +``` + +### 7a.6 Notification Status Polling + +``` +GET /api/v1/provider/notifications/{notification_uuid} + +Response: +{ + "notification_uuid": "", + "status": "processing | approved | pending_approval | rejected", + "entity_uuid": "", + "realized_state_uuid": "", + "consumer_approval_required": true | false, + "consumer_notified_at": "", + "resolved_at": "" +} +``` + +### 7a.7 Idempotency + +Provider Update Notifications are idempotent by `notification_uuid`. If DCM crashes between receiving a notification and writing the Realized State, the provider can safely resend the same notification. DCM will not create duplicate Realized State records. + +### 7a.8 Pre-Authorization Declarations + +Providers may declare categories of updates they routinely make โ€” enabling organizations to pre-authorize them in policy rather than reviewing each one: + +```json +{ + "provider_uuid": "", + "update_capabilities": [ + { + "notification_type": "auto_scale", + "affected_fields": ["cpu_count", "memory_gb"], + "max_change_magnitude": "2x", + "typical_trigger": "Resource utilization threshold" + }, + { + "notification_type": "auto_heal", + "affected_fields": ["storage_device_id", "network_interface_id"], + "max_change_magnitude": "replacement", + "typical_trigger": "Hardware failure" + } + ] +} +``` + +This declaration is part of provider registration (Section 3.3) and is surfaced in the Service Catalog to help consumers understand what provider-side changes they can expect. + + + +--- + +## 7b. Cancellation API + +This section defines the cancellation endpoint that Service Providers implement for Level 2+ conformance. Providers that declare `supports_cancellation: true` in their registration must implement this endpoint. + +### 7b.1 Cancellation Endpoint + +``` +POST /cancel (on the provider, called by DCM) +Authorization: DCM mTLS certificate + +Body: +{ + "cancellation_uuid": "", + "entity_uuid": "", + "requested_state_uuid": "", + "reason": "consumer_requested | timeout | policy_triggered", + "requested_at": "", + "best_effort": true +} +``` + +### 7b.2 Response + +| Code | Meaning | +|------|---------| +| `200 OK` (status: cancelled) | Cancellation clean; no resources provisioned | +| `200 OK` (status: partial_rollback) | Cancellation attempted; some resources may remain | +| `200 OK` (status: too_late) | Provider completed before cancellation arrived; late response forthcoming | +| `409 Conflict` | Already cancelled or already completed | + +```json +{ + "cancellation_uuid": "", + "status": "cancelled | partial_rollback | too_late", + "resources_remaining": [], + "late_response_expected": false, + "notes": "" +} +``` + +### 7b.3 Late Response After Cancellation + +If the provider returns `status: too_late`, it must still send the completed realization response via the standard realized-state callback. DCM's Late Response Pipeline handles this โ€” the provider does not need to do anything different. The `LATE_RESPONSE_RECEIVED` Recovery Policy fires on the DCM side. + +### 7b.4 Capability Declaration + +```json +{ + "cancellation_capabilities": { + "supports_cancellation": true, + "cancellation_supported_during": ["DISPATCHED", "PROVISIONING"], + "partial_rollback_possible": true, + "cancellation_response_time_seconds": 30 + } +} +``` + + +### 6.4 Interim Status Reporting + +For long-running operations (provisioning complex resources, compound service constituents), providers may send interim progress updates to DCM without waiting for terminal status. This gives DCM โ€” and therefore consumers โ€” live visibility into multi-step operations. + +**DCM endpoint for interim status:** + +``` +POST /api/v1/provider/entities/{entity_uuid}/status + +Authorization: Bearer +Content-Type: application/json + +{ + "request_id": "", + "lifecycle_state": "PROVISIONING", // current state โ€” not yet terminal + "progress": { + "step_current": 3, + "step_total": 7, + "step_label": "Configuring network interfaces", + "step_started_at": "", + "estimated_completion": "" + }, + "constituent_status": [ // for compound/Meta Provider operations + { "ref": "vm", "status": "REALIZED", "completed_at": "" }, + { "ref": "ip", "status": "REALIZED", "completed_at": "" }, + { "ref": "dns", "status": "PROVISIONING", "started_at": "" }, + { "ref": "storage", "status": "PENDING", "started_at": null } + ], + "notes": "" +} + +Response 202 Accepted +``` + +DCM uses interim status to: +1. Update `current_step` and progress fields in the request status response +2. Publish `request.progress_updated` event (info urgency) to the Message Bus +3. Deliver live status updates to consumers via SSE stream (see Consumer API Section 4.3) + +**Frequency:** Providers should not send interim status more frequently than once per 10 seconds. DCM rate-limits interim status calls per entity_uuid. + +**Terminal status** is still reported via the existing create/update response callback โ€” interim status supplements, not replaces it. + +## 7. Field Mapping Specification + +*Required for Level 2 conformance.* + +### 7.1 Overview + +The field mapping declaration tells DCM how to translate between DCM Unified Data Model fields and the operator's CRD fields. This mapping enables DCM to: +- Generate CRs from DCM Requested State payloads (Naturalization) +- Extract DCM Realized State from CR status (Denaturalization) +- Understand which DCM fields correspond to which CRD fields for drift detection + +### 7.2 Field Mapping Declaration Format + +```yaml +field_mapping: + service_type: Storage.Database + service_type_uuid: + crd_reference: + group: postgresql.cnpg.io + version: v1 + kind: Cluster + + # DCM Requested State โ†’ Kubernetes CR (Naturalization) + dcm_to_cr: + - dcm_path: resources.cpu + cr_path: spec.instances[0].resources.requests.cpu + transform: + required: true + + - dcm_path: resources.memory + cr_path: spec.instances[0].resources.requests.memory + transform: gigabytes_to_kubernetes_memory + required: true + + - dcm_path: engine + cr_path: spec.imageName + transform: engine_version_to_image + # engine: postgresql, version: 15 โ†’ imageName: ghcr.io/cloudnative-pg/postgresql:15 + required: true + + - dcm_path: metadata.name + cr_path: metadata.name + required: true + + - dcm_path: tenant_uuid + cr_path: metadata.labels.dcm-tenant-id + required: true + + - dcm_path: dcm_entity_uuid + cr_path: metadata.labels.dcm-entity-id + required: true + # All DCM-managed CRs must be labeled with their DCM entity UUID + # This enables discovery and drift detection + + # Kubernetes CR status โ†’ DCM Realized State (Denaturalization) + cr_status_to_dcm: + - cr_path: status.phase + dcm_path: lifecycle_state + transform: cr_phase_to_dcm_state + # Mapping defined in condition_mappings below + + - cr_path: status.readyInstances + dcm_path: realized_data.ready_instances + transform: none + + - cr_path: status.instancesStatus[0].ip + dcm_path: realized_data.connection.host + transform: none + + - cr_path: status.certificates.serverCASecret + dcm_path: realized_data.tls.ca_secret_ref + transform: none + + # Kubernetes conditions โ†’ DCM lifecycle states + condition_mappings: + - kubernetes_condition: "Ready=True" + dcm_lifecycle_state: OPERATIONAL + + - kubernetes_condition: "Ready=False,Progressing=True" + dcm_lifecycle_state: PROVISIONING + + - kubernetes_condition: "Ready=False,Progressing=False" + dcm_lifecycle_state: FAILED + + - kubernetes_condition: "Degraded=True" + dcm_lifecycle_state: DEGRADED + + # Kubernetes events โ†’ DCM lifecycle events + lifecycle_event_mappings: + - kubernetes_event: condition_change + condition: "Ready=False" + dcm_event: ENTITY_HEALTH_CHANGE + severity: WARNING + + - kubernetes_event: condition_change + condition: "Degraded=True" + dcm_event: DEGRADATION + severity: CRITICAL + + - kubernetes_event: spec_change_without_dcm_request + dcm_event: UNSANCTIONED_CHANGE + severity: WARNING + # Detected when CR spec changes without a corresponding DCM request ID + # Indicates drift โ€” someone modified the CR directly in Kubernetes + + # Namespace strategy implementation + namespace_strategy: + type: per_tenant + namespace_name_pattern: "dcm-{tenant_uuid_short}" + # {tenant_uuid_short} = first 8 chars of tenant UUID + labels_required: + dcm-managed: "true" + dcm-tenant-id: "{tenant_uuid}" + dcm-entity-id: "{entity_uuid}" +``` + +### 7.3 Mandatory CR Labels + +All CRs created by a DCM-conformant operator must carry these labels. These labels enable DCM's discovery and drift detection capabilities: + +| Label | Value | Purpose | +|-------|-------|---------| +| `dcm-managed` | `"true"` | Identifies this CR as DCM-managed | +| `dcm-tenant-id` | DCM Tenant UUID | Tenant ownership | +| `dcm-entity-id` | DCM Entity UUID | Links CR to DCM entity record | +| `dcm-provider-id` | DCM Provider UUID | Which provider created this | +| `dcm-request-id` | DCM Request UUID | Which request created this | + +Any CR change that does not have a corresponding DCM request ID in its update metadata is flagged as an UNSANCTIONED_CHANGE and reported to DCM. + +--- + +## 8. Lifecycle Event API + +*Required for Level 2 conformance.* + +### 8.1 Overview + +Operators must notify DCM of any event that affects the operational status of a managed resource. DCM acts as the Tenant advocate โ€” it receives events, evaluates them through the Policy Engine, and determines the appropriate response. + +### 8.2 Event Endpoint + +**DCM endpoint:** `POST /api/v1/instances/{resource_id}/events` + +### 8.3 Standard Event Types + +| Event Type | Trigger | Severity | Required Level | +|------------|---------|----------|---------------| +| `ENTITY_HEALTH_CHANGE` | CR condition changes | INFO/WARNING | Level 2 | +| `DEGRADATION` | Resource is degraded but operational | WARNING | Level 2 | +| `MAINTENANCE_SCHEDULED` | Planned maintenance window | INFO | Level 2 | +| `MAINTENANCE_STARTED` | Maintenance has begun | INFO | Level 2 | +| `MAINTENANCE_COMPLETED` | Maintenance completed | INFO | Level 2 | +| `UNSANCTIONED_CHANGE` | CR modified without DCM request | WARNING | Level 2 | +| `CAPACITY_CHANGE` | Available capacity changed significantly | INFO | Level 2 | +| `DECOMMISSION_NOTICE` | Operator is shutting down | CRITICAL | Level 2 | +| `PROVIDER_DEGRADATION` | Operator itself is degraded | CRITICAL | Level 2 | + +```yaml +# Event payload +lifecycle_event: + event_uuid: + event_type: UNSANCTIONED_CHANGE + provider_id: + resource_id: + dcm_entity_uuid: + event_timestamp: + severity: WARNING + requires_immediate_action: true + + details: + changed_fields: + - field_path: spec.instances[0].resources.requests.cpu + previous_value: "2000m" + current_value: "4000m" + changed_by: + changed_at: + + kubernetes_reference: + namespace: + name: + resource_version: +``` + +--- + +## 9. DCM Operator SDK + +### 9.1 Overview + +The DCM Operator SDK is an open source Go library that handles all DCM protocol concerns for operator developers. Using the SDK, an operator developer only needs to: + +1. Import the SDK +2. Configure field mappings (declarative YAML) +3. Add SDK hooks at key points in the reconciliation loop + +The SDK handles registration, health check endpoint exposure, capacity reporting, status translation, lifecycle event emission, provenance generation, and label management. + +### 9.2 SDK Initialization + +```go +import dcmsdk "github.com/dcm-project/operator-sdk" + +func main() { + // Load field mapping configuration + mappings, err := dcmsdk.LoadFieldMappings("dcm-mappings.yaml") + + // Initialize DCM SDK + dcm, err := dcmsdk.New(dcmsdk.Config{ + ProviderName: "cloudnativepg-provider", + DisplayName: "CloudNativePG Service Provider", + ConformanceLevel: dcmsdk.Level2, + DCMEndpoint: os.Getenv("DCM_ENDPOINT"), + OperatorEndpoint: os.Getenv("OPERATOR_ENDPOINT"), + FieldMappings: mappings, + CapacityReporter: &PostgresCapacityReporter{}, + }) + + // Start HTTP server with DCM endpoints automatically registered + dcm.StartServer(":8080") + + // Register with DCM on startup + dcm.Register(context.Background()) + + // Start operator manager + mgr.Start(ctrl.SetupSignalHandler()) +} +``` + +### 9.3 Reconciliation Loop Integration + +```go +func (r *ClusterReconciler) Reconcile( + ctx context.Context, + req ctrl.Request, +) (ctrl.Result, error) { + + cluster := &cnpgv1.Cluster{} + if err := r.Get(ctx, req.NamespacedName, cluster); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + + // Check if this CR is DCM-managed + if !r.DCM.IsManagedResource(cluster) { + return ctrl.Result{}, nil + // Not a DCM resource โ€” normal operator behavior + } + + // Detect unsanctioned changes + if r.DCM.IsUnsanctionedChange(cluster) { + r.DCM.ReportEvent(ctx, cluster, dcmsdk.UnsanctionedChange{ + ChangedFields: r.DCM.DetectChangedFields(cluster), + }) + } + + // ... existing reconciliation logic ... + + // Report current state to DCM + realizedState, err := r.DCM.TranslateStatus(cluster) + if err != nil { + return ctrl.Result{}, err + } + r.DCM.ReportStatus(ctx, cluster, realizedState) + + return ctrl.Result{}, nil +} +``` + +### 9.4 SDK Responsibilities + +The SDK automatically handles: +- Self-registration on startup with retry and exponential backoff +- Health check HTTP endpoint (`GET /health`) +- Capacity reporting on configurable schedule +- CR label injection on creation (`dcm-managed`, `dcm-tenant-id`, etc.) +- Unsanctioned change detection (spec change without DCM request ID) +- Status translation using field mapping configuration +- Lifecycle event formatting and delivery to DCM +- Provenance metadata generation for realized state payloads (Level 3) + +--- + +## 10. Kubernetes-to-DCM Concept Mappings + +Understanding how Kubernetes concepts map to DCM concepts is essential for implementing this specification correctly. + +| Kubernetes Concept | DCM Concept | Notes | +|-------------------|-------------|-------| +| Custom Resource Definition (CRD) | Resource Type Specification | CRD schema maps to DCM Resource Type fields | +| Custom Resource (CR) | Requested State โ†’ Realized State | CR is the naturalized form of the DCM payload | +| Operator reconciliation loop | Realization + Drift Detection | Reconciliation IS the realization process | +| CR status subresource | Realized State payload | Status must be denaturalized to DCM format | +| Kubernetes Namespace | DCM Tenant boundary | One namespace per Tenant (per_tenant strategy) | +| ownerReference | Entity Relationship | ownerReferences map to `contains`/`contained_by` relationships | +| Labels/Annotations | DCM Entity metadata | DCM-specific labels declared as mandatory | +| Finalizers | Lifecycle policy enforcement | Finalizers implement `retain` lifecycle policies | +| Kubernetes conditions | DCM lifecycle states | Mapped via condition_mappings declaration | +| Watch events | DCM lifecycle events | Kubernetes watch โ†’ DCM event translation | +| Kubernetes RBAC | DCM IDM/IAM + Policy Engine | Kubernetes RBAC is the runtime enforcement; DCM Policy Engine governs the request | +| Kubernetes cluster | DCM Resource Type: Platform.KubernetesCluster | The cluster itself is a DCM-managed resource | + +--- + +## 11. Conformance Testing + +### 11.1 Overview + +The DCM project provides a conformance test suite that validates an operator's implementation against this specification. Operators that pass the conformance test suite at their declared level can claim DCM conformance. + +### 11.2 Test Suite Structure + +``` +dcm-operator-conformance/ +โ”œโ”€โ”€ level1/ +โ”‚ โ”œโ”€โ”€ registration_test.go +โ”‚ โ”œโ”€โ”€ health_check_test.go +โ”‚ โ””โ”€โ”€ basic_status_test.go +โ”œโ”€โ”€ level2/ +โ”‚ โ”œโ”€โ”€ capacity_test.go +โ”‚ โ”œโ”€โ”€ lifecycle_events_test.go +โ”‚ โ”œโ”€โ”€ realized_state_test.go +โ”‚ โ””โ”€โ”€ field_mapping_test.go +โ””โ”€โ”€ level3/ + โ”œโ”€โ”€ sovereignty_test.go + โ”œโ”€โ”€ provenance_test.go + โ”œโ”€โ”€ discovery_test.go + โ””โ”€โ”€ decommission_confirmation_test.go +``` + +### 11.3 Running the Conformance Tests + +```bash +# Run Level 1 conformance tests against a running operator +dcm-conformance test \ + --level 1 \ + --operator-endpoint https://my-operator:8080 \ + --dcm-endpoint https://dcm-control-plane:8080 \ + --service-type Storage.Database + +# Run all levels +dcm-conformance test --level 3 --operator-endpoint ... +``` + +### 11.4 Conformance Certification + +Operators that pass the conformance test suite may: +- Use the "DCM Compatible โ€” Level N" badge in their documentation +- Be listed in the DCM Operator Registry +- Receive inclusion in the DCM default Service Catalog for participating organizations + +--- + +## 12. Security Considerations + +### 12.1 Authentication + +DCM authenticates outbound requests to operators using the trust model established during registration. Operators must validate that incoming requests originate from the DCM control plane. The specific authentication mechanism is declared in the provider registration: + +```yaml +trust_declaration: + auth_method: + auth_config: +``` + +### 12.2 Namespace Isolation + +When using the `per_tenant` namespace strategy, operators must enforce that resources in one namespace cannot access resources in another namespace. This is the physical enforcement of DCM's hard tenancy model at the Kubernetes level. + +### 12.3 Unsanctioned Change Detection + +Operators must monitor for changes to DCM-managed CRs that did not originate from a DCM request. Any such change is an UNSANCTIONED_CHANGE event and must be reported to DCM immediately. DCM's Policy Engine determines the appropriate response (REVERT, UPDATE_DEFINITION, ALERT, etc.). + +--- + +## 13. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the specification be submitted to CNCF as a sandbox project or proposed as a Kubernetes SIG? | Community adoption strategy | โœ… Resolved | +| 2 | Should conformance certification be self-certified (test suite passes) or require DCM project review? | Community trust | โœ… Resolved | +| 3 | How should the specification handle operators that manage cluster-scoped (non-namespaced) resources? | Namespace strategy | โœ… Resolved โ€” Two models: (A) Cluster-as-a-Service: Tenant owns the entire cluster entity including all cluster-scoped resources within it; (B) Shared cluster: cluster-scoped governance resources belong to __platform__ Tenant. Cluster-as-a-Service is the primary model. | +| 4 | Should the SDK support non-Go operator frameworks (Java Operator SDK, Python kopf)? | Ecosystem breadth | โœ… Resolved | +| 5 | How does the specification interact with Kubernetes Cluster API โ€” can CAPI clusters be DCM-managed resources? | Scope | โœ… Resolved | +| 6 | Should there be a Level 0 โ€” a pure label-based passive mode requiring no operator changes? | Adoption friction | โœ… Resolved | + +--- + +## Appendix A โ€” Example Implementation Checklist + +### Level 1 Checklist +- [ ] Operator registers with DCM on startup via `POST /api/v1/providers` +- [ ] Registration retried with exponential backoff on failure +- [ ] `GET /health` endpoint returns HTTP 200 when healthy +- [ ] `GET /health` returns non-200 when operator cannot fulfill requests +- [ ] Status reported to DCM when resource transitions to OPERATIONAL, FAILED, or DECOMMISSIONED +- [ ] All DCM-managed CRs labeled with mandatory DCM labels +- [ ] Create response returns PROVISIONING state immediately + +### Level 2 Checklist +- [ ] All Level 1 items complete +- [ ] Capacity reported to DCM on configurable schedule +- [ ] Capacity denial returns `INSUFFICIENT_RESOURCES` with proper payload +- [ ] Full realized state payload in DCM Unified Data Model format +- [ ] Field mapping declaration complete and validated +- [ ] All standard lifecycle event types implemented +- [ ] Unsanctioned change detection active +- [ ] CR condition changes translated to DCM lifecycle events + +### Level 3 Checklist +- [ ] All Level 2 items complete +- [ ] Sovereignty capabilities declared in registration +- [ ] Field-level provenance included in realized state payloads +- [ ] `POST /discover` endpoint implemented +- [ ] Decommission confirmation callback handled +- [ ] Override control metadata honored in CR creation + +--- + +## Appendix B โ€” Relationship to Other Specifications + +- **DCM Data Model** โ€” defines the Unified Data Model format used in all API payloads +- **DCM Service Provider Contract** โ€” the general provider contract this specification extends +- **DCM Resource Type Registry** โ€” where DCM Resource Types are registered; operators must reference registry UUIDs +- **AEP (API Enhancement Proposals)** โ€” the DCM API follows AEP standards for REST API design +- **OpenAPI 3.1.0** โ€” all API schemas are defined in OpenAPI 3.1.0 + +--- + +*This specification is maintained by the DCM Project. For questions, contributions, or conformance certification see [GitHub](https://github.com/dcm-project).* + + +## Resolution Notes + +**Q1:** Submit the Operator Interface Specification as a CNCF specification project (not a Sandbox project requiring a working implementation). SIG App Delivery and SIG Cluster Lifecycle engagement happens before submission. See cncf-strategy.md for the full submission strategy. + +**Q2:** Self-certified via automated test suite is the conformance gate โ€” this is the low-friction path that enables broad adoption. An optional 'DCM Verified' badge is available via DCM project review for organizations wanting a higher-trust production claim. This mirrors Kubernetes conformance: automated test suite gates access; CNCF certification provides the badge. + +**Q3:** Two distinct models apply, and it is important to not conflate them: + +**Model A โ€” Cluster as a catalog item (example Service Provider implementation):** A Kubernetes cluster can be offered as a catalog item that any authorized Tenant requests and owns โ€” this is a natural use of DCM's Service Provider model, not a special architectural feature. From DCM's perspective, `Platform.KubernetesCluster` is simply a resource type whose Service Provider happens to provision Kubernetes clusters (e.g., via CAPI). The Tenant owns the resulting cluster entity, including all cluster-scoped resources within it, because the cluster is the resource boundary. This is an example of how DCM's architecture enables complex resources as services โ€” DCM has no special knowledge of Kubernetes; it treats the cluster as any other resource entity. + +**Model B โ€” Shared cluster infrastructure (the exception):** When multiple Tenants share a single cluster (the multi-tenant cluster model), cluster-scoped resources that govern the shared infrastructure itself (admission webhook configurations, cluster-level network policies, CRD registrations) cannot be owned by any single Tenant โ€” they belong to the `__platform__` system Tenant. These are resources that, if modified by a Tenant, would affect all other Tenants on the cluster. The distinction: resources *inside* a Tenant-owned cluster are always Tenant-owned; resources that *govern shared cluster infrastructure* belong to `__platform__`. + +**The rule:** Cluster-scoped resources are owned by the Tenant that owns the cluster. If no single Tenant owns the cluster (shared infrastructure), cluster-scoped governance resources belong to `__platform__`. Operators managing cluster-scoped resources implement the standard base contract. The catalog item scope (`scope: cluster` vs `scope: namespaced`) determines which ownership model applies and what role is required to request it. + +**Q4:** The Operator Interface Specification is a REST/HTTP API specification and is language-agnostic by definition. The Go SDK is the reference implementation. Operators in any language implement the specification directly via HTTP โ€” no language-specific adapter is required. Community SDKs for Java and Python are encouraged as community projects under the DCM umbrella; the DCM project does not maintain them in v1. + +**Q5:** CAPI clusters are `Platform.KubernetesCluster` resources in DCM. The CAPI operator registers as a Service Provider for this resource type. Once provisioned, a CAPI cluster can optionally register with DCM as a nested DCM deployment or as a Service Provider for workload resources (the Meta Provider pattern). Sovereignty constraints are enforced at the CAPI provider selection level. + +**Q6:** Level 0 exists as a label-based passive discovery mode. Organizations apply DCM labels to existing operator-managed resources. DCM discovers and tracks these resources (they appear in inventory, drift detection runs against them) but DCM does not dispatch to or control them. No operator code changes are required for Level 0. This is the brownfield ingestion model applied to operators โ€” the lowest possible adoption friction. + diff --git a/content/docs/architecture/specifications/dcm-operator-sdk-api.md b/content/docs/architecture/specifications/dcm-operator-sdk-api.md new file mode 100644 index 0000000..2535771 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-operator-sdk-api.md @@ -0,0 +1,669 @@ +# DCM Operator SDK โ€” API Design + + +> ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress +> +> All questions resolved. Local durable queue, mock test harness, Prometheus metrics, and dynamic field resolution all specified. +> +> **This section is explicitly a work in progress and is less mature than the core DCM data model and architecture documentation.** +> +> The Kubernetes operator integration layer โ€” including the Operator Interface Specification, Operator SDK API, and Kubernetes compatibility mappings โ€” represents design intent that has not yet been validated against implementation. Specific interface contracts, API signatures, SDK method names, and CRD structures **will change** as implementation work begins. +> +> **Do not build against these specifications yet.** They are published to share design direction and invite feedback, not as stable contracts. +> +> Known gaps and open items for this section: +> - Operator Interface Specification: reconciliation hook signatures are provisional +> - Operator SDK API: Go module structure and dependency model not yet finalized +> - Kubernetes Compatibility Mappings: some concept mappings remain under discussion +> - SDK code examples are illustrative only โ€” not yet tested against a real implementation +> +> Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues). + + + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Ready for implementation feedback +**Document Type:** Technical Design +**Language:** Go +**Repository:** https://github.com/dcm-project/operator-sdk +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) | [Kubernetes Compatibility](11-kubernetes-compatibility.md) + +--- + +## 1. Purpose + +This document defines the public API of the DCM Operator SDK โ€” the Go library that enables Kubernetes operators to implement the DCM Operator Interface Specification with minimal code changes. The SDK handles all DCM protocol concerns so that operator developers only need to implement business logic โ€” field mappings and reconciliation hooks. + +**Design principle:** The SDK must be adoptable in a single day. If implementing Level 1 takes more than a day, the API is too complex. + +--- + +## 2. Package Structure + +``` +github.com/dcm-project/operator-sdk/ +โ”œโ”€โ”€ pkg/ +โ”‚ โ”œโ”€โ”€ client/ # DCM control plane client +โ”‚ โ”œโ”€โ”€ config/ # SDK configuration +โ”‚ โ”œโ”€โ”€ mapping/ # Field mapping engine +โ”‚ โ”œโ”€โ”€ reconciler/ # Reconciliation loop helpers +โ”‚ โ”œโ”€โ”€ registration/ # Provider registration +โ”‚ โ”œโ”€โ”€ server/ # HTTP server with DCM endpoints +โ”‚ โ”œโ”€โ”€ status/ # Status translation and reporting +โ”‚ โ”œโ”€โ”€ events/ # Lifecycle event types and emission +โ”‚ โ”œโ”€โ”€ discovery/ # Brownfield discovery helpers (Level 3) +โ”‚ โ””โ”€โ”€ provenance/ # Provenance metadata generation (Level 3) +โ”œโ”€โ”€ api/ +โ”‚ โ””โ”€โ”€ v1/ # DCM API type definitions +โ””โ”€โ”€ examples/ + โ”œโ”€โ”€ level1/ # Minimal Level 1 implementation example + โ”œโ”€โ”€ level2/ # Full Level 2 implementation example + โ””โ”€โ”€ level3/ # Complete Level 3 implementation example +``` + +--- + +## 3. Core Types + +### 3.1 Config + +```go +// Config is the primary SDK configuration structure. +// All fields have sensible defaults โ€” only DCMEndpoint, +// OperatorEndpoint, and ProviderName are required. +type Config struct { + // Required + ProviderName string + DCMEndpoint string + OperatorEndpoint string + + // Required โ€” at least one ServiceType must be declared + ServiceTypes []ServiceTypeConfig + + // Optional โ€” defaults to Level1 if not specified + ConformanceLevel ConformanceLevel + + // Optional โ€” defaults to "unknown" if not specified + DisplayName string + Version string + + // Optional โ€” field mappings loaded from file if not inline + FieldMappings []FieldMapping + FieldMappingFiles []string + + // Level 2+ โ€” capacity reporter + // If nil and ConformanceLevel >= Level2, SDK returns error on init + CapacityReporter CapacityReporter + + // Level 3 โ€” sovereignty and provenance + SovereigntyCapabilities *SovereigntyCapabilities + + // Optional โ€” HTTP server configuration + ServerConfig ServerConfig + + // Optional โ€” registration retry configuration + RegistrationConfig RegistrationConfig + + // Optional โ€” health check configuration + HealthConfig HealthConfig + + // Optional โ€” logger (defaults to zap logger) + Logger logr.Logger +} + +// ConformanceLevel declares the operator's DCM conformance level +type ConformanceLevel int + +const ( + Level1 ConformanceLevel = 1 + Level2 ConformanceLevel = 2 + Level3 ConformanceLevel = 3 +) + +// ServiceTypeConfig declares a DCM Resource Type this operator implements +type ServiceTypeConfig struct { + // DCM Resource Type name โ€” e.g., "Storage.Database" + ServiceTypeName string + // DCM Resource Type UUID from the registry + ServiceTypeUUID string + // Kubernetes CRD this service type maps to + CRDReference CRDReference + // Operations this operator supports for this type + OperationsSupported []Operation +} + +type CRDReference struct { + Group string + Version string + Kind string +} + +type Operation string + +const ( + OperationCreate Operation = "CREATE" + OperationRead Operation = "READ" + OperationUpdate Operation = "UPDATE" + OperationDelete Operation = "DELETE" + OperationDiscover Operation = "DISCOVER" // Level 3 only +) +``` + +### 3.2 Client โ€” DCM Control Plane Interface + +```go +// Client is the interface for communicating with the DCM control plane. +// The SDK creates and manages this internally โ€” operator developers +// use it only through the higher-level SDK methods. +type Client interface { + // Register sends the provider registration to DCM. + // Returns the DCM-assigned provider UUID on success. + Register(ctx context.Context, reg ProviderRegistration) (string, error) + + // ReportStatus sends a realized state payload to DCM. + ReportStatus(ctx context.Context, resourceID string, status RealizedState) error + + // ReportEvent sends a lifecycle event to DCM. + ReportEvent(ctx context.Context, resourceID string, event LifecycleEvent) error + + // ReportCapacity sends a capacity update to DCM. + // Required for Level 2+. + ReportCapacity(ctx context.Context, capacity CapacityReport) error + + // ConfirmDecommission acknowledges a decommission request from DCM. + // Required for Level 3. + ConfirmDecommission(ctx context.Context, resourceID string, confirmation DecommissionConfirmation) error +} +``` + +### 3.3 SDK โ€” Primary Interface + +```go +// SDK is the primary interface for the DCM Operator SDK. +// Operator developers interact with DCM through this interface. +type SDK interface { + // --- Lifecycle --- + + // Register sends the provider registration to DCM. + // Called during operator startup. Retries with exponential backoff. + // Does not block โ€” runs in background goroutine. + Register(ctx context.Context) + + // Shutdown gracefully deregisters the operator from DCM and + // stops background goroutines. + Shutdown(ctx context.Context) error + + // --- HTTP Server --- + + // StartServer starts the HTTP server with all DCM-required endpoints. + // Blocks until context is cancelled. + StartServer(ctx context.Context, addr string) error + + // Handler returns an http.Handler for use with an existing HTTP server. + // Alternative to StartServer when the operator already has an HTTP server. + Handler() http.Handler + + // --- Reconciliation Helpers --- + + // IsManagedResource returns true if the Kubernetes object + // carries DCM management labels. + IsManagedResource(obj client.Object) bool + + // IsUnsanctionedChange returns true if the object's spec has changed + // without a corresponding DCM request annotation. + // Used in reconciliation loops to detect drift. + IsUnsanctionedChange(obj client.Object) bool + + // DetectChangedFields returns the list of fields that changed + // relative to the last known DCM request state. + DetectChangedFields(obj client.Object) []FieldChange + + // InjectLabels adds DCM-required labels to a Kubernetes object + // before creation. Called before submitting a CR to Kubernetes. + InjectLabels(obj client.Object, req CreateRequest) client.Object + + // AnnotateRequest adds the DCM request ID annotation to a + // Kubernetes object. Used to mark changes as DCM-sanctioned. + AnnotateRequest(obj client.Object, requestID string) client.Object + + // --- Status Translation --- + + // TranslateStatus translates a Kubernetes object's status + // to a DCM RealizedState using the configured field mappings. + TranslateStatus(obj client.Object) (RealizedState, error) + + // ReportStatus translates and reports status to DCM in one call. + // Convenience wrapper for TranslateStatus + Client.ReportStatus. + ReportStatus(ctx context.Context, obj client.Object) error + + // --- Event Emission --- + + // ReportEvent sends a lifecycle event to DCM. + ReportEvent(ctx context.Context, obj client.Object, event LifecycleEventType, details EventDetails) error + + // ReportUnsanctionedChange is a convenience method for reporting + // an unsanctioned change event with the detected changed fields. + ReportUnsanctionedChange(ctx context.Context, obj client.Object, changes []FieldChange) error + + // ReportDegradation reports a DEGRADATION event to DCM. + ReportDegradation(ctx context.Context, obj client.Object, reason string) error + + // ReportHealthChange reports an ENTITY_HEALTH_CHANGE event. + ReportHealthChange(ctx context.Context, obj client.Object, healthy bool, reason string) error + + // --- Capacity --- + + // StartCapacityReporting starts the background capacity reporting + // goroutine. Required for Level 2+. Called automatically by StartServer. + StartCapacityReporting(ctx context.Context) + + // --- Discovery (Level 3) --- + + // BuildDiscoveryResponse queries Kubernetes for existing resources + // and returns them in DCM Realized State format. + // Used to implement the POST /discover endpoint. + BuildDiscoveryResponse(ctx context.Context, k8sClient client.Client, opts DiscoveryOptions) ([]RealizedState, error) +} +``` + +--- + +## 4. Field Mapping API + +```go +// FieldMapping declares how a DCM Resource Type maps to a Kubernetes CRD. +// Can be loaded from a YAML file or declared inline in Go. +type FieldMapping struct { + ServiceTypeName string + ServiceTypeUUID string + CRDReference CRDReference + + // DCM Requested State โ†’ Kubernetes CR spec (Naturalization) + DCMToCR []FieldMap + + // Kubernetes CR status โ†’ DCM Realized State (Denaturalization) + CRStatusToDCM []FieldMap + + // Kubernetes conditions โ†’ DCM lifecycle states + ConditionMappings []ConditionMapping + + // Kubernetes events โ†’ DCM lifecycle event types + LifecycleEventMappings []LifecycleEventMapping + + // Namespace strategy for this resource type + NamespaceStrategy NamespaceStrategy +} + +// FieldMap declares a single field translation +type FieldMap struct { + // Source field path โ€” dot-notation, supports array indexing + // e.g., "resources.cpu" or "nodes.controlPlane[0].cpu" + SourcePath string + + // Destination field path + DestPath string + + // Transform function name โ€” registered in the transform registry + // "none" for direct copy, or a named transform + Transform string + + // Required โ€” if true and source field is absent, returns error + Required bool + + // Default โ€” used when source field is absent and Required is false + Default interface{} +} + +// ConditionMapping maps a Kubernetes condition to a DCM lifecycle state +type ConditionMapping struct { + // Kubernetes condition expression โ€” e.g., "Ready=True" + // Supports AND: "Ready=False,Progressing=True" + KubernetesCondition string + + // DCM lifecycle state + DCMLifecycleState LifecycleState +} + +// LifecycleEventMapping maps a Kubernetes event to a DCM event type +type LifecycleEventMapping struct { + // "condition_change" | "spec_change_without_dcm_request" | "deletion" + KubernetesEvent string + + // Condition that triggers this mapping (for condition_change events) + Condition string + + // DCM event type + DCMEventType LifecycleEventType + + // Severity + Severity EventSeverity +} + +// Transform registry โ€” operator developers register custom transforms +type TransformRegistry interface { + // Register adds a named transform function + Register(name string, fn TransformFunc) error + + // Get retrieves a transform function by name + Get(name string) (TransformFunc, error) +} + +// TransformFunc transforms a value from source to destination format +type TransformFunc func(value interface{}) (interface{}, error) +``` + +--- + +## 5. Status and State Types + +```go +// LifecycleState represents the DCM lifecycle state of a resource +type LifecycleState string + +const ( + LifecycleStateProvisioning LifecycleState = "PROVISIONING" + LifecycleStateOperational LifecycleState = "OPERATIONAL" + LifecycleStateDegraded LifecycleState = "DEGRADED" + LifecycleStateSuspended LifecycleState = "SUSPENDED" + LifecycleStateFailed LifecycleState = "FAILED" + LifecycleStateDecommissioned LifecycleState = "DECOMMISSIONED" +) + +// RealizedState is the DCM Unified Data Model representation of +// a resource's realized state. This is what the operator sends +// to DCM after successful provisioning or status change. +type RealizedState struct { + // DCM resource ID (returned by DCM in the create request) + ResourceID string + + // DCM entity UUID + DCMEntityUUID string + + // Current lifecycle state + LifecycleState LifecycleState + + // Timestamp of this realization + RealizedTimestamp time.Time + + // All realized fields in DCM Unified Data Model format + Spec map[string]interface{} + + // Level 3 โ€” field-level provenance + FieldProvenance map[string]FieldProvenance + + // Kubernetes reference for correlation + KubernetesReference KubernetesReference + + // Relationships created during realization + Relationships []RelationshipRecord +} + +// KubernetesReference carries Kubernetes-specific identity for correlation +type KubernetesReference struct { + Namespace string + Name string + UID types.UID + ResourceVersion string + Generation int64 +} + +// FieldProvenance carries lineage for a single field (Level 3) +type FieldProvenance struct { + SourceType string // "provider" + SourceUUID string // operator provider UUID + Timestamp time.Time + Reason string +} +``` + +--- + +## 6. Event Types + +```go +// LifecycleEventType represents a DCM lifecycle event type +type LifecycleEventType string + +const ( + EventEntityHealthChange LifecycleEventType = "ENTITY_HEALTH_CHANGE" + EventDegradation LifecycleEventType = "DEGRADATION" + EventMaintenanceScheduled LifecycleEventType = "MAINTENANCE_SCHEDULED" + EventMaintenanceStarted LifecycleEventType = "MAINTENANCE_STARTED" + EventMaintenanceCompleted LifecycleEventType = "MAINTENANCE_COMPLETED" + EventUnsanctionedChange LifecycleEventType = "UNSANCTIONED_CHANGE" + EventCapacityChange LifecycleEventType = "CAPACITY_CHANGE" + EventDecommissionNotice LifecycleEventType = "DECOMMISSION_NOTICE" + EventProviderDegradation LifecycleEventType = "PROVIDER_DEGRADATION" +) + +// EventSeverity represents the severity of a lifecycle event +type EventSeverity string + +const ( + SeverityInfo EventSeverity = "INFO" + SeverityWarning EventSeverity = "WARNING" + SeverityCritical EventSeverity = "CRITICAL" +) + +// LifecycleEvent is the payload sent to DCM for a lifecycle event +type LifecycleEvent struct { + EventUUID string + EventType LifecycleEventType + ProviderID string + ResourceID string + DCMEntityUUID string + EventTimestamp time.Time + Severity EventSeverity + RequiresImmediateAction bool + Details EventDetails + KubernetesReference KubernetesReference +} + +// EventDetails carries event-specific detail data +type EventDetails struct { + // For UNSANCTIONED_CHANGE events + ChangedFields []FieldChange + + // For DEGRADATION events + DegradationReason string + AffectedComponents []string + + // For MAINTENANCE events + MaintenanceWindow *MaintenanceWindow + MaintenanceReason string + + // For CAPACITY_CHANGE events + PreviousCapacity *CapacityReport + CurrentCapacity *CapacityReport + + // Human-readable message for any event type + Message string +} + +// FieldChange describes a single field change in an unsanctioned change event +type FieldChange struct { + FieldPath string + PreviousValue interface{} + CurrentValue interface{} + ChangedBy string // Kubernetes user or service account + ChangedAt time.Time +} +``` + +--- + +## 7. Capacity Types + +```go +// CapacityReporter is the interface operator developers implement +// to report capacity data to DCM. The SDK calls this on schedule. +type CapacityReporter interface { + // GetCapacity returns the current capacity for all service types. + // Called by the SDK on the configured reporting schedule. + GetCapacity(ctx context.Context) (CapacityReport, error) +} + +// CapacityReport contains capacity data for all service types +type CapacityReport struct { + ProviderID string + ReportTimestamp time.Time + NextReportAt time.Time + CapacityByServiceType []ServiceTypeCapacity +} + +// ServiceTypeCapacity contains capacity for a single service type +type ServiceTypeCapacity struct { + ServiceTypeUUID string + AvailableUnits int + ReservedUnits int + CommittedUnits int + UnitDefinition string + KubernetesResources KubernetesResourceCapacity +} + +// KubernetesResourceCapacity contains raw Kubernetes resource availability +type KubernetesResourceCapacity struct { + AvailableCPUMillicores int64 + AvailableMemoryBytes int64 + AvailableStorageBytes int64 + NodeCount int +} +``` + +--- + +## 8. Constructor and Initialization + +```go +// New creates and initializes a new DCM SDK instance. +// Returns an error if the configuration is invalid or +// if required components for the declared conformance level +// are missing. +func New(config Config) (SDK, error) + +// NewWithClient creates a new SDK instance with a pre-configured +// DCM client. Used primarily for testing. +func NewWithClient(config Config, client Client) (SDK, error) + +// LoadFieldMappings loads field mapping declarations from YAML files. +// Accepts one or more file paths or glob patterns. +func LoadFieldMappings(paths ...string) ([]FieldMapping, error) + +// MustNew creates a new SDK instance and panics if initialization fails. +// Convenience function for use in main() where error handling via +// panic is acceptable. +func MustNew(config Config) SDK +``` + +--- + +## 9. Minimal Level 1 Example + +```go +package main + +import ( + "context" + "os" + + dcmsdk "github.com/dcm-project/operator-sdk" + ctrl "sigs.k8s.io/controller-runtime" +) + +func main() { + // Minimal Level 1 configuration + dcm, err := dcmsdk.New(dcmsdk.Config{ + ProviderName: "my-operator", + DisplayName: "My Operator DCM Provider", + DCMEndpoint: os.Getenv("DCM_ENDPOINT"), + OperatorEndpoint: os.Getenv("OPERATOR_ENDPOINT"), + ConformanceLevel: dcmsdk.Level1, + ServiceTypes: []dcmsdk.ServiceTypeConfig{ + { + ServiceTypeName: "Storage.Database", + ServiceTypeUUID: "dcm-registry-uuid-for-storage-database", + CRDReference: dcmsdk.CRDReference{ + Group: "postgresql.cnpg.io", + Version: "v1", + Kind: "Cluster", + }, + OperationsSupported: []dcmsdk.Operation{ + dcmsdk.OperationCreate, + dcmsdk.OperationRead, + dcmsdk.OperationDelete, + }, + }, + }, + FieldMappingFiles: []string{"dcm-mappings.yaml"}, + }) + if err != nil { + panic(err) + } + + ctx := ctrl.SetupSignalHandler() + + // Register with DCM in background โ€” does not block startup + dcm.Register(ctx) + + // Start HTTP server with health + DCM endpoints + go dcm.StartServer(ctx, ":8080") + + // Start operator manager (existing code unchanged) + mgr, _ := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{}) + mgr.Start(ctx) +} + +// In reconciliation loop โ€” minimal Level 1 additions +func (r *ClusterReconciler) Reconcile( + ctx context.Context, + req ctrl.Request, +) (ctrl.Result, error) { + + cluster := &cnpgv1.Cluster{} + if err := r.Get(ctx, req.NamespacedName, cluster); err != nil { + return ctrl.Result{}, client.IgnoreNotFound(err) + } + + // Only process DCM-managed resources + if !r.DCM.IsManagedResource(cluster) { + return ctrl.Result{}, nil + } + + // Existing reconciliation logic here... + + // Report status to DCM (SDK handles translation via field mappings) + r.DCM.ReportStatus(ctx, cluster) + + return ctrl.Result{}, nil +} +``` + +--- + +## 10. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Should the SDK support non-Go operator frameworks via a language-agnostic REST adapter? | Ecosystem breadth | โœ… Resolved | +| 2 | How should the SDK handle DCM endpoint unavailability โ€” queue events locally or drop? | Reliability | โœ… Resolved | +| 3 | Should field mappings support dynamic resolution โ€” a transform that queries external data? | Flexibility | โœ… Resolved | +| 4 | Should the SDK provide a testing framework for unit testing operator-DCM integration? | Developer experience | โœ… Resolved | +| 5 | Should the SDK expose metrics (Prometheus) for DCM registration status, event delivery success, etc.? | Observability | โœ… Resolved | + +--- + + + +## Resolution Notes + +**Q1:** No language-agnostic REST adapter is needed in the Go SDK โ€” the Operator Interface Specification is itself language-agnostic. Operators in any language implement the specification directly via HTTP. Community SDKs for Java/Python are encouraged as community projects. The Go SDK is the reference implementation only. + +**Q2:** Queue locally, always. The SDK maintains a local durable queue (SQLite โ€” simple, no external dependencies) with configurable capacity and TTL. On DCM reconnection, queued events are replayed in order. If the local queue reaches capacity (DCM unavailable for an extended period), the SDK enters DEGRADED mode: new events are still accepted up to the hard capacity limit, then dropped with a QUEUE_OVERFLOW audit record and an alert via the operator's configured alerting channel. Dropping events silently is never acceptable โ€” the system is designed to be the authoritative source of truth. + +**Q3:** Dynamic field resolution is implemented as an Information Provider reference in the field mapping declaration. The SDK declares 'this field resolves from Information Provider X with lookup key Y'. DCM resolves the value during layer assembly via the standard Information Provider query. This keeps transformation logic in DCM's Policy Engine where it belongs and is auditable via standard field provenance. + +**Q4:** A mock DCM test harness ships as a first-class component of the SDK. The harness implements the registration, dispatch, cancel, and discover endpoints with configurable behaviors: inject failures, inject delays, return specific payloads, simulate timeout scenarios. Operators use the test harness for unit and integration testing without a live DCM deployment. This is essential for adoption โ€” operators must be able to test DCM integration in CI without a full environment. + +**Q5:** Prometheus metrics are mandatory, not optional. The SDK exposes: registration_status (gauge), event_delivery_total (counter, labels: status=success|failure), event_delivery_duration_seconds (histogram), local_queue_depth (gauge, only when local queuing active), dispatch_duration_seconds (histogram), discovery_cycle_duration_seconds (histogram). Metrics endpoint follows the standard DCM observability model and is required for Level 2 conformance. + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/dcm-provider-gui-spec.md b/content/docs/architecture/specifications/dcm-provider-gui-spec.md new file mode 100644 index 0000000..aa29b51 --- /dev/null +++ b/content/docs/architecture/specifications/dcm-provider-gui-spec.md @@ -0,0 +1,324 @@ +# DCM Provider Management GUI Specification + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Specification โ€” Provider Management Interface +**Related Documents:** [Unified Provider Contract](../data-model/A-provider-contract.md) | [OIS Specification](dcm-operator-interface-spec.md) | [Registration Specification](dcm-registration-spec.md) | [Admin GUI Specification](dcm-admin-gui-spec.md) | [Credential Provider Model](../data-model/31-credential-provider-model.md) + +> **Status:** Draft โ€” Ready for implementation feedback +> +> The Provider Management GUI is the interface for teams responsible for operating and maintaining DCM providers. It surfaces in the DCM web application for actors holding provider owner roles. Each of the eleven DCM provider types has a common management shell plus type-specific extension panels. + +--- + +## 1. Architecture + +### 1.1 Surface in Unified Shell + +Provider management is a **third surface in the unified DCM web application**. An actor who owns a provider (registered in a provider's `owner_team_uuid` or holds the `provider_owner` role scoped to a provider) sees a "Providers" section in their navigation alongside the Consumer Portal. + +``` +DCM Web Application +โ”œโ”€โ”€ Consumer Portal +โ”œโ”€โ”€ Admin Panel โ† platform_admin role +โ””โ”€โ”€ Provider Management โ† provider_owner role + โ”œโ”€โ”€ My Providers (list) + โ””โ”€โ”€ [Provider Type] โ†’ [type-specific management] +``` + +### 1.2 Provider Owner Identity + +Provider ownership is declared at registration time and may include: +- A tenant UUID (the team that owns this provider) +- A group UUID (the specific group within that tenant) +- Named contacts (from Business Data) + +The `provider_owner` role is scoped to specific provider UUIDs โ€” a team may own multiple providers and sees all of them. Platform Admins see all registered providers across all owners. + +### 1.3 Common Management Shell + +Every provider type (all eleven) renders within the same management shell. The shell provides: +- Provider name, type badge, health status indicator +- Registration status (pending / active / suspended / deregistered) +- Navigation tabs that vary by provider type +- Health check response viewer +- Audit trail for all admin actions on this provider + +--- + +## 2. Common Provider Tabs (All Types) + +### 2.1 Overview + +- Provider UUID, handle, type, registration date, owner team/group +- Health status: current `pass | warn | fail` with last check timestamp +- Health response detail (from OIS `/health` endpoint โ€” live refresh every 30s) +- Connection details: endpoint URL, OIS version declared +- Registration token status (active / expiry date) + +### 2.2 Configuration + +- Provider capability declaration YAML viewer (read-only; changes require re-registration or update submission) +- Editable fields: display name, description, owner contact, notification preferences +- Profile compatibility indicator: which DCM profiles this provider is certified to operate under + +### 2.3 Health History + +- 30-day health check history: pass / warn / fail timeline +- Degraded periods highlighted; duration of each incident +- Correlation with entity realization failures during degraded periods + +### 2.4 Audit Trail + +- All admin actions on this provider: approval, suspension, config changes, capacity updates +- Actor who performed each action, timestamp, comment +- Filterable by action type + +### 2.5 Notifications + +- Notification endpoint configuration for this provider +- Which DCM events this provider subscribes to (webhook subscriptions for provider.* events) +- Test webhook delivery + +--- + +## 3. Service Provider โ€” Extended Tabs + +Service Providers (the most common type โ€” realize infrastructure resources) have the richest management surface. + +### 3.1 Capacity Management + +**API:** `POST /api/v1/providers/{uuid}/capacity` + +- Current capacity report: available units per resource type per location +- Historical capacity charts: capacity utilization over time +- **Manual capacity update form**: override reported capacity for emergency situations +- Capacity denial history: requests denied due to insufficient capacity +- Alert configuration: notify when capacity below threshold + +### 3.2 Managed Entities + +- All DCM entities realized by this provider: type, tenant, state, TTL, drift status +- Filterable by tenant (for Platform Admin), resource type, state +- Entity count by state (pie chart) +- Entities with open drift records: grouped by drift severity +- Entities approaching TTL expiry in next P7D +- Click-through to entity detail (read-only view for provider owner; editable for Platform Admin) + +### 3.3 Naturalization Mapping + +- Resource type โ†’ provider-native mapping declarations +- View the Naturalization configuration for each supported resource type +- Denaturalization mapping: what provider-native fields map back to DCM fields +- **Test naturalization**: submit a DCM payload and see the naturalized version without dispatching + +### 3.4 Interim Status Configuration + +- Enable / disable interim status reporting per resource type +- Reporting frequency configuration (minimum interval, max steps) +- View recent interim status payloads for debugging + +### 3.5 Realization History + +- Recent realization requests: accepted / failed / in-progress +- Mean realization time by resource type (last 30 days) +- Failure analysis: top failure reasons with counts +- In-flight realizations with live status + +--- + +## 4. Credential Provider โ€” Extended Tabs + +### 4.1 Credential Inventory + +- All credentials managed by this provider: type, entity scope, issued date, expiry, last retrieved +- Never shows credential values โ€” only metadata +- Filter by credential type (api_key, x509_certificate, ssh_key, etc.) +- Credentials approaching expiry: highlighted red within renewal trigger window + +### 4.2 Rotation Management + +- Credentials currently in rotation (transition window open): old UUID โ†’ new UUID pairs +- Manual rotation trigger for specific credentials +- Rotation history with trigger reason + +### 4.3 Revocation Registry + +- Summary: total revoked, still-in-TTL (in registry), post-TTL (pruned from registry) +- Search by credential UUID to check revocation status +- Emergency revocation form: revoke by credential UUID or entity UUID with required reason + +### 4.4 External CA Configuration (if ca_type: external) + +- CA protocol in use (ACME / EST / SCEP / CMP / Vault PKI / etc.) +- CA endpoint connectivity status +- Certificate chain view: root CA โ†’ intermediate โ†’ issued certs +- Pending certificate requests +- CRL / OCSP endpoint status + +### 4.5 Algorithm Compliance View + +- Algorithms in use across managed credentials +- Forbidden algorithm violations (should be zero โ€” highlighted red if any found) +- Algorithm distribution chart: ECDSA P-384 vs P-256 vs RSA vs other +- Upcoming algorithm deprecations from doc 40 forbidden list + +--- + +## 5. Auth Provider โ€” Extended Tabs + +### 5.1 Connection Status + +- Auth Provider endpoint connectivity: pass / warn / fail +- Failover chain: current primary, configured fallbacks, activation status +- Token validation latency (p50, p99) โ€” last 1 hour + +### 5.2 Actor and Group Sync + +- SCIM sync status (if SCIM 2.0 enabled): last sync timestamp, records synced, errors +- Group โ†’ DCM role mapping table (read-only; changes via configuration update) +- Actor provisioning audit: recent SCIM-triggered creates, updates, deactivations + +### 5.3 Session Statistics + +- Active session count from this Auth Provider +- Session distribution by auth method (OIDC / LDAP / API key / mTLS) +- MFA verification rates: mfa_verified: true vs false breakdown +- Step-up MFA events in last 24h + +### 5.4 Configuration + +- Auth Provider YAML viewer +- Editable: display name, session TTL, concurrent session limit, role mapping (changes go through standard artifact lifecycle โ€” proposed โ†’ reviewed โ†’ active) +- Shadow mode toggle for configuration changes + +--- + +## 6. Policy Provider โ€” Extended Tabs + +### 6.1 Policy Inventory + +- Policies managed by this provider: handle, type, enforcement class, status (active / shadow / deprecated) +- Policy evaluation counts and outcomes (last 24h) +- Shadow divergence alerts: policies with >5% divergence rate from expected + +### 6.2 Trust Level Management + +- Current trust level: local / community / verified / authoritative +- Trust elevation request form (requires Platform Admin approval) +- Trust history + +### 6.3 Policy Contribution Pipeline + +- Policies submitted for contribution to the DCM Policy Registry +- Lifecycle status: shadow validation period, community review, approval +- Withdraw pending contributions + +--- + +## 7. Information Provider โ€” Extended Tabs + +### 7.1 Data Source Status + +- Connection status to upstream data source +- Last successful sync and record count +- Data freshness indicator (stale threshold from provider registration) + +### 7.2 Confidence Score Management + +- Declared confidence scores per data field +- Confidence override history (when DCM overrode provider confidence based on corroboration) + +### 7.3 Query Performance + +- Response time distribution for DCM queries to this provider +- Cache hit rate (if DCM caches this provider's data) +- Top queried fields + +--- + +## 8. Storage Provider โ€” Extended Tabs + +### 8.1 Store Health + +- Store-specific health: write latency, read latency, replication lag (if replicated), disk utilization +- Consistency guarantee compliance: declared vs observed consistency level + +### 8.2 Capacity and Retention + +- Storage utilization by store type (Intent, Requested, Realized, Audit) +- Retention policy status: records approaching retention deadline +- Partition / shard status (for GitOps stores using partitioning strategies from doc 41) + +--- + +## 9. Notification and Message Bus Providers โ€” Extended Tabs + +### 9.1 Notification Provider + +- Delivery channel status: email, Slack, PagerDuty, etc. +- Delivery success rate (last 24h) +- Failed deliveries with retry status +- Audience routing test: send a test notification to a specific audience + +### 9.2 Message Bus Provider + +- Broker connectivity status +- Topic / stream inventory: DCM topics and consumer group lag +- Message throughput (messages/minute by topic) +- Dead letter queue: messages that failed delivery after retry + +--- + +## 10. Meta Provider โ€” Extended Tabs + +### 10.1 Compound Service Status + +- Active compound service instances: all constituent statuses +- Constituents in PENDING_DEPENDENCY state across all active instances +- Failed compensation attempts (COMPENSATION_FAILED state) + +### 10.2 Constituent Provider Health + +- Health of each provider this Meta Provider depends on +- Impact analysis: if Provider X degrades, which compound services are affected + +--- + +## 11. Registry Provider and Peer DCM โ€” Extended Tabs + +### 11.1 Registry Provider + +- Resource type registry sync status: last sync, version, record count +- Type registration submissions pending review +- Registry health: response time, availability + +### 11.2 Peer DCM (Federation) + +- Federation tunnel status: connected / degraded / disconnected +- Peer DCM version and deployment profile +- Cross-instance request routing: requests forwarded to / from this peer (last 24h) +- Sovereignty boundary status: which data classifications are permitted across this tunnel + +--- + +## 12. Provider-Scoped Security + +### 12.1 Provider Interaction Credentials + +- Interaction credentials issued to this provider: count, last issued, rotation status +- Emergency credential revocation capability +- Audit of all credential retrievals by this provider (CPX-005: first retrieval always audited) + +### 12.2 mTLS Certificate Status + +- Provider's mTLS certificate: issuer, expiry, OCSP status +- Certificate renewal tracking (for external CA managed certs) + +### 12.3 Provider Audit Trail Contribution + +- Provenance records emitted by this provider: count, last emitted +- Audit forwarding status: is the provider successfully forwarding audit events? + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/dcm-registration-spec.md b/content/docs/architecture/specifications/dcm-registration-spec.md new file mode 100644 index 0000000..3a9a44c --- /dev/null +++ b/content/docs/architecture/specifications/dcm-registration-spec.md @@ -0,0 +1,835 @@ +# DCM Registration Specification + +> **๐Ÿ“‹ Draft** +> +> This specification has been promoted from Work in Progress to Draft status. All questions resolved. Complete registration pipeline for all 11 provider types with full capability declaration schemas and federation trust model. It is ready for implementation feedback but has not yet been formally reviewed for final release. +> +> This specification defines the unified registration flow for all DCM provider types. Published to share design direction and invite feedback. + +**Version:** 0.1.0-draft +**Status:** Draft โ€” Ready for implementation feedback +**Document Type:** Technical Specification +**Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Control Plane Components](../data-model/25-control-plane-components.md) | [Governance Matrix](../data-model/27-governance-matrix.md) | [Accreditation and Authorization Matrix](../data-model/26-accreditation-and-authorization-matrix.md) | [Policy Profiles](../data-model/14-policy-profiles.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md) + +--- + +## Abstract + +This specification defines the unified registration flow by which all DCM provider types establish a trusted, governed relationship with a DCM deployment. It covers: the Provider Type Registry, the registration token model, the approval method configuration, the step-by-step registration pipeline, trust establishment, the per-type capability declaration schemas, the ongoing lifecycle after activation, federated trust configuration, and profile-bound registration policy defaults. + +--- + +## 1. Provider Type Registry + +The Provider Type Registry is the authoritative list of provider types that a DCM deployment will accept registrations for. It follows the same three-tier registry model as the Resource Type Registry. + +### 1.1 Registry Tiers + +| Tier | Maintained By | Examples | +|------|--------------|---------| +| **Core** | DCM Project | The eleven built-in provider types | +| **Verified Community** | Named community maintainers | Domain-specific provider types | +| **Organization** | Deploying organization | Custom/proprietary integrations | + +### 1.2 Provider Type Registry Entry + +```yaml +provider_type_registry_entry: + artifact_metadata: + uuid: + handle: "provider-types/service-provider" + version: "1.0.0" + status: active + tier: core + + provider_type_id: service_provider + display_name: "Service Provider" + description: "Realizes infrastructure resources for DCM" + + # What this provider type is permitted to do + permissions: + may_receive_assembled_payload: true + may_write_realized_state: true + may_write_discovered_state: true + may_receive_scoped_credentials: true + may_receive_phi_by_default: false # requires HIPAA accreditation + may_receive_sovereign_data: false # hard limit; never overridden + + # Approval method defaults (profile may override โ€” see Section 4) + default_approval_method: reviewed # auto | reviewed | verified | authorized + + # Minimum trust level granted after approval + default_trust_level: standard # minimal | standard | elevated | high + + # Which deployment profiles permit this provider type + enabled_in_profiles: [minimal, dev, standard, prod, fsi, sovereign] + + # Capability declaration schema reference + capability_schema_ref: "schemas/service-provider-capabilities-v1.0.0" + + # Health check requirements + health_check: + endpoint_required: true + minimum_check_interval: PT1M + failure_threshold: 3 # failures before degraded status +``` + +### 1.3 The Eleven Core Provider Types + +| # | provider_type_id | Default Approval | Enabled In | +|---|-----------------|-----------------|------------| +| 1 | `service_provider` | reviewed | all profiles | +| 2 | `information_provider` | reviewed | all profiles | +| 3 | `meta_provider` | verified | standard+ | +| 4 | `storage_provider` | verified | all profiles | +| 5 | `message_bus_provider` | reviewed | dev+ (external endpoints: standard+) | +| 6 | `policy_provider` (Mode 1-2) | reviewed | all profiles | +| 7 | `policy_provider` (Mode 3-4) | verified | standard+ | +| 8 | `credential_provider` | verified | standard+ | +| 9 | `auth_provider` | verified | all profiles | +| 10 | `notification_provider` | reviewed | all profiles | + +Note: Mode 3-4 Policy Providers are treated as a separate registry entry from Mode 1-2 due to the elevated trust requirements. + +--- + +## 2. Registration Token Model + +Registration tokens are pre-issued by platform admins to authorize specific registrations without requiring full manual review at submission time. + +### 2.1 Token Structure + +```yaml +registration_token: + token_uuid: + token_value: + issued_by: + issued_at: + expires_at: # short-lived; default PT72H + single_use: true # token invalidated after first use + + scope: + provider_type_id: service_provider # which provider type this authorizes + provider_handle_pattern: "eu-west-*" # optional: restrict to matching handles + sovereignty_zone: eu-west-sovereign # optional: restrict to this zone + grants_auto_approval: true # whether token enables auto-approval + # grants_auto_approval: false = token still required but human review still needed + # (useful for tracking/auditing expected registrations without bypassing review) + + max_trust_level_granted: standard # token cannot grant higher than this +``` + +### 2.2 Token Issuance + +``` +POST /api/v1/admin/registration-tokens +Role: platform_admin + +{ + "provider_type_id": "service_provider", + "expires_in": "PT72H", + "scope": { + "provider_handle_pattern": "eu-west-*", + "sovereignty_zone": "eu-west-sovereign", + "grants_auto_approval": true + }, + "purpose": "EU-WEST production compute provider onboarding" +} + +Response 201 Created: +{ + "token_uuid": "", + "token_value": "", + "expires_at": "", + "scope": { ... } +} +``` + +Token values are presented exactly once โ€” at creation. They are never retrievable again (stored as a hash). Platform admins must transmit the token securely to the provider operator. + +--- + +## 3. Approval Method Configuration + +> **Authority Tier Model:** Approval methods (`reviewed`, `verified`, `authorized`) are defined in the [Authority Tier Model](../data-model/32-authority-tier-model.md) as a named, ordered list. Organizations may insert custom tiers. The effective method resolution (Section 3.2) uses tier names; DCM resolves numeric weight from the ordered list at evaluation time (ATM-001). + +### 3.1 The Four Approval Methods + +| Method | Description | Approval path | +|--------|-------------|--------------| +| `auto` | DCM validates automatically; activates without human review | All validation checks pass โ†’ active | +| `reviewed` | One platform admin must explicitly approve | Submitted โ†’ validated โ†’ pending_approval โ†’ one admin approves โ†’ active | +| `verified` | Two platform admins must independently approve | Submitted โ†’ validated โ†’ pending_approval โ†’ two admins approve โ†’ active | +| `authorized` | N members of a declared DCMGroup must record decisions via the Admin API; quorum tracked by DCM; deliberation process is the organization's responsibility | Submitted โ†’ validated โ†’ pending_approval โ†’ DCMGroup members record votes via Admin API (or external systems calling API) โ†’ quorum โ†’ active | + +### 3.2 Effective Approval Method Resolution + +The effective approval method for a specific registration is the most restrictive result of: + +``` +effective_method = most_restrictive( + provider_type_registry.default_approval_method, + active_profile.registration_policy.min_approval_method, + registration_token.grants_auto_approval ? relax_to_auto : no_change +) +``` + +Resolution rules: +- Profile minimum overrides provider type default (always upward; profiles can only tighten) +- A valid registration token can relax the effective method to `auto` ONLY if the profile's `allow_token_auto_approval` is true +- `authorized` cannot be relaxed by any token + +### 3.3 Profile Registration Policy Defaults + +```yaml +profile_registration_policy: + minimal: + min_approval_method: reviewed + allow_token_auto_approval: true # token can enable auto for any type + require_sovereignty_declaration: false + require_health_check_before_approval: false + + dev: + min_approval_method: reviewed + allow_token_auto_approval: true + require_sovereignty_declaration: false + require_health_check_before_approval: true + + standard: + min_approval_method: reviewed + allow_token_auto_approval: true # tokens can auto-approve non-elevated types + token_auto_approval_max_trust: standard # tokens cannot auto-approve elevated types + require_sovereignty_declaration: true + require_health_check_before_approval: true + + prod: + min_approval_method: reviewed + high_trust_types_require: verified # storage, auth, policy-mode3-4, credential + allow_token_auto_approval: false # no auto-approval in prod + require_sovereignty_declaration: true + require_accreditation_submission: true # must submit at least self_declared + require_health_check_before_approval: true + approval_timeout: P7D # auto-reject if not approved within 7 days + + fsi: + min_approval_method: verified # everything requires dual approval + allow_token_auto_approval: false + require_sovereignty_declaration: true + require_accreditation_submission: true + minimum_accreditation_type: third_party # self_declared not accepted + require_health_check_before_approval: true + require_governance_matrix_check: true # governance matrix evaluated at registration + approval_timeout: P14D + + sovereign: + min_approval_method: authorized # everything requires authorized approval + allow_token_auto_approval: false + require_sovereignty_declaration: true + require_accreditation_submission: true + minimum_accreditation_type: regulatory_certification + require_hardware_attestation: true + require_governance_matrix_check: true + authorized_group_handle: "platform/registration-authorized" + approval_timeout: P30D +``` + +--- + +## 4. Registration Pipeline + +### 4.1 Lifecycle States + +``` +SUBMITTED โ†’ VALIDATING โ†’ PENDING_APPROVAL โ†’ ACTIVE + โ†˜ REJECTED (validation failure) + โ†˜ REJECTED (approval denied) + +Additional states: +ACTIVE โ†’ SUSPENDED (platform admin action or health failure) +ACTIVE โ†’ DEREGISTERING โ†’ DEREGISTERED (graceful removal) +ACTIVE โ†’ FORCED_DEREGISTERED (immediate removal) +``` + +### 4.2 Step 1 โ€” Submission + +Provider submits registration payload to DCM: + +``` +POST /api/v1/provider/register +Content-Type: application/json +X-DCM-Registration-Token: # optional; enables auto-approval if valid + +{ + "provider_type_id": "service_provider", + "handle": "eu-west-prod-1", + "display_name": "EU West Production Compute Provider", + "version": "2.1.0", + + # Mutual TLS certificate presented at connection level + # DCM extracts the certificate fingerprint from the TLS handshake + + "sovereignty_declaration": { ... }, + "accreditations": [ ... ], + "capabilities": { ... }, # per-type capability declaration + "health_endpoint": "https://provider.example.com/health", + "delivery_endpoint": "https://provider.example.com/dispatch" +} + +Response 202 Accepted: +{ + "registration_uuid": "", + "status": "VALIDATING", + "token_recognized": true, + "auto_approval_eligible": true, + "estimated_activation": "" +} +``` + +### 4.3 Step 2 โ€” Validation (automated) + +DCM runs automated validation checks. All must pass before advancing to PENDING_APPROVAL: + +``` +Validation checks: + V1: Provider type permitted in active profile + โ†’ Check Provider Type Registry: enabled_in_profiles includes active posture + โ†’ FAIL: REJECTED with reason "provider_type_not_enabled_in_profile" + + V2: Governance Matrix pre-check + โ†’ Evaluate matrix: is a provider of this type, in this zone, with these + accreditations, permitted to register? + โ†’ FAIL: REJECTED with reason "governance_matrix_denied" + rule_uuid + + V3: Registration token validation (if provided) + โ†’ Token exists and not expired + โ†’ Token matches provider_type_id and handle pattern + โ†’ Token not already used + โ†’ FAIL: Token invalid; fall back to non-token approval method + + V4: Certificate validation + โ†’ mTLS certificate presented and valid + โ†’ Certificate chain acceptable (registered CA or pinned self-signed) + โ†’ Certificate not in revocation list + โ†’ FAIL: REJECTED with reason "certificate_invalid" + + V5: Sovereignty declaration completeness + โ†’ Required fields present (if profile requires declaration) + โ†’ Jurisdiction codes valid + โ†’ FAIL: REJECTED with reason "sovereignty_declaration_incomplete" + + V6: Capability declaration consistency + โ†’ Declared capabilities consistent with provider type + โ†’ No contradictory declarations + โ†’ FAIL: REJECTED with reason "capability_declaration_invalid" + + V7: Health endpoint reachability + โ†’ DCM contacts health_endpoint + โ†’ Provider responds with valid health payload + โ†’ FAIL: status โ†’ PENDING_APPROVAL with warning (profile may require passing) + + V8: Accreditation submission check + โ†’ If profile requires accreditation submission: at least one accreditation present + โ†’ Accreditation type meets profile minimum + โ†’ FAIL: REJECTED with reason "accreditation_insufficient" +``` + +### 4.4 Step 3 โ€” Approval + +Approval flow depends on effective_approval_method: + +**auto:** Registration immediately advances to ACTIVE after validation passes. + +**reviewed:** +``` +Registration enters PENDING_APPROVAL +Platform admin notification dispatched (urgency: medium) +Platform admin reviews in Admin API or Flow GUI: + GET /api/v1/admin/registrations/pending + POST /api/v1/admin/registrations/{registration_uuid}/approve + POST /api/v1/admin/registrations/{registration_uuid}/reject +On approval: โ†’ ACTIVE +On rejection: โ†’ REJECTED with required reason field +On timeout (approval_timeout): โ†’ REJECTED with reason "approval_timeout" +``` + +**verified:** +``` +Registration enters PENDING_APPROVAL +Two independent platform admins must approve +First approval: recorded; notification sent to other admins for second approval +Second approval by different actor: โ†’ ACTIVE +Same actor cannot approve twice +On timeout: โ†’ REJECTED +``` + +**authorized:** +``` +Registration enters PENDING_APPROVAL +Authority group notified (all members) +Members vote via Admin API within declared quorum window +Quorum reached: โ†’ ACTIVE +Quorum not reached within approval_timeout: โ†’ REJECTED +``` + +### 4.5 Step 4 โ€” Activation + +On ACTIVE status: +- Provider enters the DCM provider registry +- Governance matrix rules are re-evaluated with this provider now active +- Capacity monitoring begins (if Service or Information Provider) +- Health check polling begins +- Certificate rotation schedule established +- Activation audit record written: PROVIDER_ACTIVATED +- Notification: platform admin + Tenant admins (if Tenant-scoped provider) + +--- + +## 5. Per-Type Capability Declaration Schemas + +### 5.1 Service Provider Capabilities + +```yaml +service_provider_capabilities: + resource_types: + - resource_type_fqn: Compute.VirtualMachine + resource_type_spec_version: "2.1.0" + catalog_item_uuid: + availability_zones: [eu-west-1a, eu-west-1b] + max_instances: 1000 + + capacity_model: + reporting_method: reserve_query | static_declaration | both + reserve_query_endpoint: /reserve + reserve_query_timeout: PT10S + static_capacity: + Compute.VirtualMachine: 500 + + cancellation: + supports_cancellation: true + cancellation_supported_during: [DISPATCHED, PROVISIONING] + partial_rollback_possible: true + + discovery: + supports_discovery: true + discovery_endpoint: /discover + discovery_method: api_query | passive_event | hybrid + supports_incremental_discovery: true + + naturalization: + target_format: openstack_nova | vmware_vsphere | custom + custom_schema_ref: + + cost_metadata: + capex_allocation_per_unit: 12.50 + opex_per_unit_per_hour: 0.28 + currency: USD + cost_data_dynamic_source: null | + + data_handling: + max_data_classification_accepted: restricted + phi_capable: false # true requires HIPAA BAA accreditation + pci_capable: false +``` + +### 5.2 Information Provider Capabilities + +```yaml +information_provider_capabilities: + data_domains: + - domain: business_data + data_types: [business_unit, cost_center, product_owner] + authority_level: primary | secondary | supplementary + schema_version: "1.0.0" + query_endpoint: /query + write_back_supported: false + + query_capacity: + max_queries_per_second: 100 + rate_limit_window: 60s + burst_capacity: 200 + + confidence_model: + data_freshness_sla: PT1H + corroboration_sources: [cmdb, hr_system] + + caching: + cacheable: true + cache_ttl: PT15M + cache_invalidation_webhook: /invalidate +``` + +### 5.3 Storage Provider Capabilities + +```yaml +storage_provider_capabilities: + store_types_supported: + - store_type: gitops + branch_per_request: true + pr_semantics: true + search_index_companion: true + - store_type: write_once_snapshot + entity_uuid_keyed: true + hash_chain_integrity: true + point_in_time_query: true + + consistency: + guarantee: strong | eventual | bounded_staleness + bounded_staleness_max: PT5M + + replication: + geo_replicated: true + replication_regions: [eu-west, eu-north] + synchronous_replication: true + + encryption: + at_rest: AES-256 + hsm_backed: false + key_management: provider_managed | customer_managed | hsm + + retention: + supports_retention_policy: true + minimum_retention: P1Y + maximum_retention: P10Y + tamper_evident: true +``` + +### 5.4 Policy Provider Capabilities + +```yaml +policy_provider_capabilities: + mode: 1 | 2 | 3 | 4 + policy_types_supported: + - gatekeeper + - validation + - transformation + - recovery + - orchestration_flow + + framework: opa | cedar | custom + rego_version: "1.0" # for OPA providers + + # Mode 3/4 specific + remote_endpoint: https://policy.example.com/evaluate + endpoint_sovereignty_zone: eu-west-sovereign + evaluation_latency_p95: PT200MS + supports_bundle_push: true + supports_bundle_pull: true + + shadow_mode_supported: true + test_harness_endpoint: /test +``` + +### 5.5 Auth Provider Capabilities + +```yaml +auth_provider_capabilities: + authentication_modes: + - api_key + - ldap + - oidc + - oidc_mfa + - saml + - mtls + - hardware_token + - hardware_token_mfa + + mfa_methods: + - totp + - push_notification + - hardware_token + + rbac_model: flat | hierarchical | attribute_based + external_idp_integration: true + idp_protocols: [oidc, saml, ldap] + + token_lifetime_config: + default_lifetime: PT1H + min_lifetime: PT5M + max_lifetime: PT8H + step_up_supported: true + + builtin: false # true for DCM's built-in auth provider +``` + +### 5.6 Notification Provider Capabilities + +```yaml +notification_provider_capabilities: + delivery_channels: + - channel_type: slack + supports_threading: true + supports_urgency_routing: true + config_schema_ref: + - channel_type: pagerduty + supports_escalation: true + config_schema_ref: + - channel_type: webhook + protocols: [https] + auth_modes: [hmac_sha256, mtls, bearer] + config_schema_ref: + - channel_type: email + html_supported: true + + delivery_guarantees: + at_least_once: true + idempotency_key: notification_uuid + max_delivery_latency_seconds: 30 + retry_policy: + max_attempts: 7 + backoff: exponential + on_exhaustion: dead_letter + + sovereignty_aware_delivery: true # checks endpoint jurisdiction before delivery +``` + +### 5.7 Credential Provider Capabilities + +```yaml +credential_provider_capabilities: + credential_types: + - api_key + - x509_certificate + - ssh_key + - service_account_token + - database_password + - hsm_backed_key + + secret_engines: + - vault + - aws_secrets_manager + - azure_key_vault + - gcp_secret_manager + + rotation_support: true + hsm_backed: false + fips_140_2_level: 1 | 2 | 3 # for sovereign deployments + dynamic_secrets: true # generate credentials on demand +``` + +### 5.8 Message Bus Provider Capabilities + +```yaml +message_bus_provider_capabilities: + protocols: [kafka, amqp, mqtt, grpc] + persistence: true + durability: at_least_once | exactly_once + max_throughput_msg_per_sec: 100000 + retention: + message_retention: P7D + retention_configurable: true + external_endpoints: false # true if messages can leave sovereignty boundary + encryption_in_transit: TLS-1.3 + encryption_at_rest: AES-256 +``` + +### 5.9 Meta Provider Capabilities + +```yaml +meta_provider_capabilities: + constituent_provider_types: + - service_provider + - information_provider + + composition_model: sequential | parallel | conditional + partial_delivery_supported: true + compensation_supported: true + + resource_types_composed: + - resource_type_fqn: ApplicationStack.WebApp + constituent_resource_types: + - Compute.VirtualMachine + - Network.IPAddress + - DNS.Record + - Network.LoadBalancer +``` + +--- + +## 6. Federated Trust Configuration + +### 6.1 Federation Trust Postures + +| Posture | Description | Operations permitted | +|---------|-------------|---------------------| +| `verified` | Manually verified and approved by local platform admin | Full declared scope per tunnel authorization | +| `vouched` | Introduced through a trusted Hub DCM | Vouching authority's declared scope; cannot exceed voucher's scope | +| `provisional` | Cryptographically verified but not yet manually approved | catalog_query only (if profile permits) | + +### 6.2 Federation Trust Registration Flow + +``` +Remote DCM requests federation peering + โ”‚ + โ–ผ Cryptographic verification (always): + โ”‚ mTLS certificate validation + โ”‚ Certificate not in revocation list + โ”‚ Certificate signed by acceptable CA + + โ–ผ Governance matrix pre-check: + โ”‚ Is federation with this peer's jurisdiction/accreditation permitted? + + โ–ผ Trust posture determination: + โ”‚ Prior record of this remote UUID? โ†’ verified or vouched (per prior record) + โ”‚ No prior record โ†’ provisional + + โ–ผ Approval flow (per profile): + โ”‚ dev: provisional auto-promoted to verified (if governance matrix permits) + โ”‚ standard: reviewed for verified promotion; provisional gets limited scope + โ”‚ prod: verified for verified promotion; no provisional operations + โ”‚ fsi: verified + accreditation check; no provisional + โ”‚ sovereign: authorized_approval + hardware attestation; no provisional + + โ–ผ Scope assignment per trust posture + + โ–ผ Tunnel established with governance matrix enforcement +``` + +### 6.3 Profile Federation Trust Policy + +```yaml +profile_federation_policy: + minimal: + permitted_trust_postures: [verified, vouched, provisional] + auto_promote_provisional: true + cross_jurisdiction_permitted: true + accreditation_required_for_federation: false + + dev: + permitted_trust_postures: [verified, vouched, provisional] + auto_promote_provisional: true + provisional_permitted_operations: [catalog_query, resource_query] + cross_jurisdiction_permitted: true + + standard: + permitted_trust_postures: [verified, vouched] + approval_method_for_verified: reviewed + cross_jurisdiction_permitted: true + accreditation_required_for_federation: false + + prod: + permitted_trust_postures: [verified] + approval_method_for_verified: verified + cross_jurisdiction_permitted: true + accreditation_required_for_federation: false + + fsi: + permitted_trust_postures: [verified] + approval_method_for_verified: verified + cross_jurisdiction_permitted: false + accreditation_required_for_federation: true + minimum_peer_accreditation: third_party + re_verification_interval: PT8H + + sovereign: + permitted_trust_postures: [verified] + approval_method_for_verified: authorized + cross_jurisdiction_permitted: false + accreditation_required_for_federation: true + minimum_peer_accreditation: sovereign_authorization + hardware_attestation_required: true + data_classification_boundary: internal + re_verification_interval: PT4H +``` + +--- + +## 7. Ongoing Lifecycle After Activation + +### 7.1 Health Monitoring + +``` +DCM polls provider health endpoint every health_check_interval + โ”‚ + โ”œโ”€โ”€ Response: healthy โ†’ no action; next poll scheduled + โ”œโ”€โ”€ Response: degraded โ†’ DCM updates capacity rating; reduces routing preference + โ”œโ”€โ”€ No response (1 failure) โ†’ warning; retry at shorter interval + โ”œโ”€โ”€ No response (failure_threshold reached) โ†’ provider status โ†’ DEGRADED + โ”‚ Notification: platform admin (urgency: high) + โ”‚ New requests no longer routed to this provider + โ””โ”€โ”€ No response (2ร— failure_threshold) โ†’ provider status โ†’ UNAVAILABLE + Active entities checked; drift detection triggered + Platform admin notification (urgency: critical) +``` + +### 7.2 Certificate Rotation + +```yaml +certificate_rotation: + rotation_interval: P90D # profile-governed default + transition_window: P7D # old cert valid during transition + pre_rotation_warning: P14D # warn provider P14D before expiry + +# Rotation flow: +POST /api/v1/provider/certificates/rotate +{ + "new_certificate_pem": "", + "transition_window": "P7D" +} +# DCM accepts both old and new certificates during transition window +# After transition window: old certificate rejected +``` + +### 7.3 Capability Updates + +Providers may update their capability declarations (new resource types, updated capacity models, new accreditations). Capability updates go through a simplified registration amendment flow: + +``` +POST /api/v1/provider/capabilities/update +{ + "amendment_type": "add_resource_type | remove_resource_type | update_capacity | add_accreditation", + "changes": { ... } +} + +โ†’ VALIDATING (automated checks only) +โ†’ PENDING_APPROVAL (if amendment_type is add_resource_type or sovereignty change) +โ†’ ACTIVE (capability declarations updated) +``` + +### 7.4 Deregistration + +**Graceful deregistration:** +``` +Provider submits deregistration intent +DCM checks: active entities hosted at this provider +If active entities > 0: + Decision required: migrate_entities | decommission_entities | reject_deregistration +Platform admin approves deregistration plan +Provider enters DEREGISTERING state +Entity migration or decommission completes +Provider status โ†’ DEREGISTERED +``` + +**Forced deregistration:** +``` +POST /api/v1/admin/providers/{provider_uuid}/force-deregister +Role: platform_admin +Requires: verified (fsi/sovereign: authorized) + +Immediate effect: + Provider status โ†’ FORCED_DEREGISTERED + All active entities โ†’ INDETERMINATE_REALIZATION + Governance matrix re-evaluated for all affected entities + Recovery policy fires: DRIFT_RECONCILE or NOTIFY_AND_WAIT per profile +``` + +--- + +## 8. Error Model + +| Error Code | Meaning | +|-----------|---------| +| `provider_type_not_enabled` | Provider type not permitted in active profile | +| `governance_matrix_denied` | Governance matrix pre-check denied registration | +| `certificate_invalid` | mTLS certificate invalid or not from acceptable CA | +| `token_invalid` | Registration token expired, used, or type mismatch | +| `token_insufficient_scope` | Token present but does not grant required approval level | +| `sovereignty_declaration_incomplete` | Required sovereignty fields missing | +| `accreditation_insufficient` | Active profile requires higher accreditation type | +| `capability_declaration_invalid` | Capability declarations internally inconsistent | +| `approval_timeout` | Registration not approved within approval_timeout period | +| `health_check_failed` | Provider health endpoint unreachable during validation | +| `duplicate_handle` | A provider with this handle already exists in active status | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/architecture/specifications/dcm-rhdh-integration-spec.md b/content/docs/architecture/specifications/dcm-rhdh-integration-spec.md new file mode 100644 index 0000000..49fe9fa --- /dev/null +++ b/content/docs/architecture/specifications/dcm-rhdh-integration-spec.md @@ -0,0 +1,637 @@ +# DCM Red Hat Developer Hub Integration Specification + +**Document Status:** ๐Ÿ”„ In Progress +**Document Type:** Specification โ€” RHDH / Backstage Integration Architecture +**Related Documents:** [Consumer GUI Specification](dcm-consumer-gui-spec.md) | [Admin GUI Specification](dcm-admin-gui-spec.md) | [Provider GUI Specification](dcm-provider-gui-spec.md) | [Consumer API Specification](consumer-api-spec.md) | [Auth Providers](../data-model/19-auth-providers.md) | [Standards Catalog](../data-model/40-standards-catalog.md) + +> **Status:** Draft โ€” Ready for implementation feedback +> +> This specification defines the complete integration between DCM and Red Hat Developer Hub (RHDH) or upstream Backstage. It covers plugin architecture, entity model, auth delegation, permission mapping, Software Template auto-generation, and deployment. + +--- + +## 1. Integration Architecture Overview + +### 1.1 Layering Model + +DCM and RHDH are separate systems that integrate at well-defined boundaries. DCM remains authoritative for all infrastructure state; RHDH provides the developer experience layer. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ RHDH / Backstage โ”‚ +โ”‚ Software Catalog โ”‚ Scaffolder โ”‚ TechDocs โ”‚ Search โ”‚ +โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”‚ +โ”‚ @dcm/plugin suite (Dynamic Plugins) โ”‚ +โ”‚ โ”œโ”€โ”€ Entity Provider โ† pulls from DCM API โ”‚ +โ”‚ โ”œโ”€โ”€ Scaffolder Actions โ†’ pushes to DCM API โ”‚ +โ”‚ โ”œโ”€โ”€ Frontend Plugin โ† reads DCM API via proxy โ”‚ +โ”‚ โ”œโ”€โ”€ Permission Policy โ†” DCM roles โ”‚ +โ”‚ โ””โ”€โ”€ Auth Bridge โ†” DCM Auth Provider (OIDC token exchange) โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ DCM Consumer API (HTTPS) + โ”‚ X-DCM-Tenant from RHDH group context + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ DCM Control Plane โ”‚ +โ”‚ Consumer API โ”‚ Policy Engine โ”‚ Scoring โ”‚ Providers โ”‚ +โ”‚ Stores: Intent, Requested, Realized, Discovered, Audit โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +**DCM is authoritative for:** resource state, policy decisions, audit trail, realized data, cost, drift detection. + +**RHDH is authoritative for:** developer experience, documentation, search index, Software Templates, organization/group model. + +### 1.2 Plugin Packages + +The DCM RHDH integration is delivered as six npm packages, all loadable as RHDH Dynamic Plugins: + +| Package | Type | Purpose | +|---------|------|---------| +| `@dcm/backstage-plugin` | Frontend | Nav, pages, entity tabs, drawers | +| `@dcm/backstage-plugin-backend` | Backend | API proxy, SSE relay, auth middleware | +| `@dcm/backstage-plugin-catalog-backend` | Backend | Entity provider, catalog processor | +| `@dcm/backstage-plugin-scaffolder-backend` | Backend | Custom scaffolder actions | +| `@dcm/backstage-permission-policy` | Backend | DCM โ†’ Backstage permission bridge | +| `@dcm/backstage-plugin-auth-backend` | Backend | RHDH as DCM Auth Provider (optional) | + +--- + +## 2. Authentication and Token Flow + +### 2.1 RHDH as DCM Auth Provider + +The recommended pattern: configure RHDH (Keycloak/RHSSO) as the Auth Provider for both RHDH and DCM. DCM trusts OIDC tokens issued by the same IdP that RHDH uses. + +``` +User authenticates โ†’ RHDH (via Keycloak/RHSSO OIDC) + โ”‚ + RHDH issues Backstage session token + OIDC access token + โ”‚ + DCM plugin backend receives OIDC access token + โ”‚ + DCM plugin backend presents OIDC token to DCM Consumer API + (/api/v1/auth/token with grant_type: oidc_token_exchange) + โ”‚ + DCM issues its own session token (JWT with actor_uuid, roles, tenant_scope) + โ”‚ + DCM session token cached in RHDH backend (keyed by Backstage user entity ref) + โ”‚ + All subsequent DCM API calls use DCM session token +``` + +DCM is registered as an OIDC Auth Provider with the same issuer as RHDH's Keycloak: + +```yaml +# DCM Auth Provider registration +auth_provider_registration: + provider_type: auth_provider + auth_method: oidc + oidc_config: + issuer: https://keycloak.corp.com/realms/corporate + client_id: dcm-api + trust_level: authoritative + role_mapping: + group_role_map: + - external_group: dcm-consumers + dcm_role: consumer + - external_group: dcm-approvers + dcm_role: approver + - external_group: dcm-platform-admins + dcm_role: platform_admin +``` + +### 2.2 Token Lifetime and Refresh + +- RHDH session: governed by Keycloak session settings (typically PT8H) +- DCM session token: PT30M (prod profile) โ€” refreshed transparently by RHDH backend plugin +- DCM plugin backend maintains a token cache: `backstage_user_ref โ†’ dcm_session_token` +- Token refresh: triggered when DCM session token is within PT5M of expiry + +### 2.3 Service Account Token for Entity Provider + +The `@dcm/backstage-plugin-catalog-backend` entity provider runs as a background service, not on behalf of a user. It uses a DCM service account: + +```yaml +# DCM service account for RHDH catalog entity provider +service_account: + handle: rhdh-catalog-provider + roles: [catalog_reader] # read-only: catalog items + realized entities + credential_type: api_key + rotation: P30D +``` + +The service account API key is stored as a Kubernetes Secret and mounted into the RHDH backend pod. + +### 2.4 Tenancy from RHDH Group Context + +The active RHDH namespace/group context maps to `X-DCM-Tenant`: + +```typescript +// In @dcm/backstage-plugin-backend โ€” DCM API proxy middleware +const groupContext = request.headers['x-backstage-namespace'] || + userEntity.spec?.memberOf?.[0]; +const tenantUuid = await dcmTenantCache.resolveFromGroup(groupContext); +proxyRequest.headers['X-DCM-Tenant'] = tenantUuid; +``` + +Tenant UUID resolution: `@dcm/backstage-plugin-catalog-backend` maintains a `RHDH Group ref โ†’ DCM Tenant UUID` mapping, populated during entity sync. + +--- + +## 3. Entity Model + +### 3.1 Custom Entity Kinds + +DCM introduces two custom Backstage entity kinds: + +#### DCMService (catalog item) + +Represents a DCM service catalog item โ€” something a user can request. + +```yaml +apiVersion: dcm.io/v1alpha1 +kind: DCMService +metadata: + name: compute-vm-standard + namespace: dcm-catalog # shared namespace for all DCM catalog items + annotations: + dcm.io/catalog-item-uuid: "" + dcm.io/resource-type-fqn: "Compute.VirtualMachine" + backstage.io/techdocs-ref: url:/docs/compute-vm + tags: [compute, infrastructure, self-service] +spec: + type: dcm-service + lifecycle: production + owner: group:platform-team + providedBy: + providerHandle: k8s-operator-prod + providerType: service_provider + fieldSchema: + # JSON Schema for the request form โ€” auto-populated from DCM catalog item + $ref: "dcm-api://catalog//schema" + costEstimate: + currency: USD + estimatedMonthly: 240 + billingDimensions: [cpu_count, memory_gb] + availability: + quotaRemaining: 4 # computed at sync time + sla: "99.9%" +``` + +#### DCMResource (realized entity) + +Represents a realized DCM resource โ€” something that exists. + +```yaml +apiVersion: dcm.io/v1alpha1 +kind: DCMResource +metadata: + name: payments-api-server-01 + namespace: payments-team # namespace = DCM tenant + annotations: + dcm.io/entity-uuid: "" + dcm.io/resource-type-fqn: "Compute.VirtualMachine" + dcm.io/provider-uuid: "" + dcm.io/request-uuid: "" # the request that created this + backstage.io/techdocs-ref: url: +spec: + type: Compute.VirtualMachine + lifecycle: production + owner: group:payments-team + system: payments-platform + realizedFields: # key fields from Realized State + primary_ip: "10.42.0.105" + hostname: "payments-api-server-01.corp.internal" + cpu_count: 4 + memory_gb: 16 + os_family: rhel + lifecycleState: OPERATIONAL # DCM lifecycle state + ttlExpiresAt: "2026-09-15" + driftStatus: none # none | minor | moderate | significant | critical + providerHandle: k8s-operator-prod + dependsOn: + - dcmresource:payments-team/payments-db-01 +``` + +### 3.2 Entity Provider + +`@dcm/backstage-plugin-catalog-backend` implements a `EntityProvider` that: + +1. On startup: fetches all DCM catalog items โ†’ emits `DCMService` entities +2. On startup: fetches all realized entities for each tenant โ†’ emits `DCMResource` entities +3. On schedule (default: every PT5M): polls for changes, emits delta mutations +4. On `dcm:catalog:refresh` scaffolder action: triggers immediate refresh for specific entity + +```typescript +class DcmEntityProvider implements EntityProvider { + async refresh(logger: Logger): Promise { + // Fetch catalog items + const catalogItems = await this.dcmApi.getCatalogItems(); + const serviceEntities = catalogItems.map(toDCMServiceEntity); + + // Fetch realized resources per tenant + const tenants = await this.dcmApi.getTenants(); // admin service account + const resourceEntities = (await Promise.all( + tenants.map(t => this.dcmApi.getResources(t.uuid)) + )).flat().map(toDCMResourceEntity); + + await this.connection.applyMutation({ + type: 'full', + entities: [...serviceEntities, ...resourceEntities], + }); + } +} +``` + +### 3.3 Catalog Processor + +Handles entity validation, relationship resolution, and annotation enrichment for `DCMService` and `DCMResource` kinds. + +--- + +## 4. Software Template Auto-Generation + +### 4.1 Generation Model + +`@dcm/backstage-plugin-catalog-backend` automatically generates Backstage Software Templates from DCM catalog items. No manual template authoring is needed when new resource types appear. + +Generation pipeline: +``` +GET /api/v1/catalog โ†’ DCM catalog items + โ”‚ + For each catalog item: + โ”‚ GET /api/v1/catalog/{uuid} โ†’ field schema + โ”‚ + Transform: + โ”‚ field schema โ†’ Backstage template parameters (JSON Schema compatible) + โ”‚ catalog item metadata โ†’ template metadata + โ”‚ provider info โ†’ template tags + โ”‚ + Emit as Backstage Template entity +``` + +### 4.2 Schema Transformation Rules + +| DCM field type | Backstage ui:widget | Notes | +|---------------|---------------------|-------| +| `enum` list | `select` | Options from DCM enum | +| `string` with pattern | `text` + pattern validation | Pattern in JSON Schema | +| `integer` range | `number` or `select` | Select if < 10 options | +| `boolean` | `checkbox` | โ€” | +| `uuid` reference | `dcm:EntityPicker` | Custom picker component | +| `duration` (ISO 8601) | `dcm:DurationPicker` | Custom picker | +| `datetime` | `datetime` | Standard Backstage widget | +| Injected field (read-only) | `readonly` | Shows source layer in tooltip | + +### 4.3 Multi-Step Template Structure + +All generated templates follow a consistent multi-step structure: + +``` +Step 1: "Configure [Service Name]" โ† DCM required fields +Step 2: "Options" โ† Optional DCM fields + scheduling +Step 3: "Scheduling (Optional)" โ† dispatch: immediate/at/window/recurring +Step 4: "Review" โ† cost estimate + pre-flight check + โ† Submit โ† +Step 5: "Provisioning..." โ† dcm:request:submit + dcm:request:wait (live log) +Step 6: "Complete" โ† link to entity in catalog + resource URL +``` + +--- + +## 5. Scaffolder Actions Reference + +All actions in `@dcm/backstage-plugin-scaffolder-backend`: + +### `dcm:request:estimate` + +```typescript +input: + catalogItemUuid: string // DCM catalog item UUID + fields: object // field values from template parameters + tenantUuid?: string // defaults to RHDH group context + +output: + estimatedMonthlyCost: number + currency: string + breakdown: Array<{dimension: string, cost: number}> + quotaCheck: {passes: boolean, remaining: number} + policyPreCheck: {passes: boolean, warnings: string[]} +``` + +**Purpose:** Called during the Review step. Provides cost estimate, quota check, and policy pre-flight. Does not submit the request. + +### `dcm:request:submit` + +```typescript +input: + catalogItemUuid: string + fields: object + schedule?: {dispatch: 'immediate'|'at'|'window'|'recurring', notBefore?: string, notAfter?: string, windowId?: string} + dependsOn?: Array<{requestUuid: string, waitFor: string, injectFields?: ...}> + +output: + requestUuid: string + entityUuid: string // UUID the resource will have when realized + status: string // typically ACKNOWLEDGED + requestUrl: string // link to request in DCM consumer portal +``` + +### `dcm:request:wait` + +```typescript +input: + requestUuid: string + timeoutMinutes?: number // default: 30 + pollIntervalSeconds?: number // default: 5; uses SSE if available + +output: + status: 'REALIZED'|'FAILED'|'CANCELLED' + entityUuid: string + entityUrl: string // link to entity in RHDH catalog + realizedFields: object // key fields from provider (IP, hostname, etc.) + failureReason?: string +``` + +Streams status updates to the Scaffolder log panel: +``` +[LOG] 09:01:05 Status: PROVISIONING โ€” Step 3/7: Configuring network interfaces +[LOG] 09:03:12 โœ… REALIZED โ€” IP: 10.42.0.105, Hostname: payments-api-server-01.corp.internal +``` + +### `dcm:request:group` + +```typescript +input: + groupHandle?: string + onFailure?: 'cancel_remaining'|'continue' + timeout?: string // ISO 8601 duration + requests: Array<{ + ref: string, // local reference within this submission + catalogItemUuid: string, + fields: object, + dependsOn?: Array<{ref: string, waitFor: string, injectFields?: ...}> + }> + +output: + groupUuid: string + requests: Array<{ref: string, requestUuid: string, entityUuid: string}> + groupUrl: string +``` + +### `dcm:catalog:refresh` + +```typescript +input: + entityUuid: string // DCM entity UUID to refresh in RHDH catalog + +output: + entityRef: string // Backstage entity ref: dcmresource:/ + entityUrl: string // URL to entity page in RHDH +``` + +Triggers immediate re-poll of the entity provider for the specified entity. The entity appears in RHDH catalog within PT30S of REALIZED status. + +--- + +## 6. Permission Framework Integration + +### 6.1 DCM Permissions in Backstage + +`@dcm/backstage-permission-policy` defines DCM permissions in Backstage permission framework terms: + +```typescript +// DCM permission definitions +export const dcmPermissions = { + // Resource permissions + resourceRead: createPermission({name: 'dcm.resource.read', attributes: {action: 'read'}}), + resourceUpdate: createPermission({name: 'dcm.resource.update', attributes: {action: 'update'}}), + resourceDelete: createPermission({name: 'dcm.resource.delete', attributes: {action: 'delete'}}), + + // Catalog permissions + catalogRequest: createPermission({name: 'dcm.catalog.request', attributes: {action: 'create'}}), + + // Approval permissions + approvalVote: createPermission({name: 'dcm.approval.vote', attributes: {action: 'update'}}), + + // Admin permissions + tenantManage: createPermission({name: 'dcm.tenant.manage', attributes: {action: 'update'}}), + providerManage: createPermission({name: 'dcm.provider.manage', attributes: {action: 'update'}}), +}; +``` + +### 6.2 Role Mapping + +The permission policy maps Backstage group membership to DCM permission grants: + +```typescript +class DcmPermissionPolicy implements PermissionPolicy { + async handle(request: PolicyQuery, user?: BackstageIdentityResponse) { + const groups = user?.identity.ownershipEntityRefs ?? []; + + // Basic consumer permissions โ€” all authenticated users + if (isAuthenticated(user)) { + if (DCM_READ_PERMISSIONS.includes(request.permission.name)) { + return { result: AuthorizeResult.ALLOW }; + } + } + + // Role-based grants + if (groups.includes('group:dcm-approvers')) { + if (request.permission.name === 'dcm.approval.vote') { + return { result: AuthorizeResult.ALLOW }; + } + } + + if (groups.includes('group:dcm-platform-admins')) { + return { result: AuthorizeResult.ALLOW }; // all permissions + } + + return { result: AuthorizeResult.DENY }; + } +} +``` + +### 6.3 RHDH RBAC Plugin Integration + +The RHDH RBAC plugin provides a no-code UI for managing role assignments. DCM roles are represented as RHDH group memberships: + +``` +RHDH RBAC UI: + Role: dcm-consumers โ†’ Group: all-authenticated-users + Role: dcm-approvers โ†’ Groups: [payments-leads, platform-approvers] + Role: dcm-platform-admins โ†’ Groups: [platform-team] + Role: dcm-contributors โ†’ Groups: [policy-authors, power-users] +``` + +Changes to group membership propagate to DCM via SCIM 2.0 (if configured) or OIDC group claims on next login. + +--- + +## 7. Deployment + +### 7.1 Dynamic Plugin Loading + +All DCM plugins are deployed as RHDH Dynamic Plugins โ€” no RHDH image rebuild required: + +```yaml +# RHDH app-config.yaml additions +dynamicPlugins: + frontend: + dcm.backstage-plugin: + disabled: false + backend: + dcm.backstage-plugin-backend: + disabled: false + dcm.backstage-plugin-catalog-backend: + disabled: false + dcm.backstage-plugin-scaffolder-backend: + disabled: false + dcm.backstage-permission-policy: + disabled: false +``` + +Plugins loaded from OCI registry or npm. New plugin versions deployed by updating the tag โ€” no RHDH pod rebuild. + +### 7.2 RHDH Configuration + +```yaml +# app-config.yaml โ€” DCM integration configuration +dcm: + baseUrl: https://dcm.corp.internal + apiPath: /api/v1 + + # Service account for catalog entity provider + serviceAccount: + apiKey: + $env: DCM_SERVICE_ACCOUNT_API_KEY + + # Catalog entity sync configuration + catalog: + syncIntervalSeconds: 300 # poll DCM API every 5 minutes + refreshOnScaffolderComplete: true + entityNamespace: dcm-catalog # for DCMService entities + + # Tenant resolution + tenancy: + groupNamespacePrefix: "dcm-tenant-" # RHDH group dcm-tenant-{uuid} โ†’ tenant uuid + fallbackTenantUuid: null # null = require explicit group context + + # Auth delegation + auth: + oidcIssuer: https://keycloak.corp.com/realms/corporate + clientId: dcm-rhdh-bridge + clientSecret: + $env: DCM_OIDC_CLIENT_SECRET + + # Feature flags + features: + liveStatusSse: true # use SSE for request status (fallback to polling if false) + costEstimateInCatalog: true # show cost estimate on catalog cards + quotaCheckOnBrowse: true # show quota availability in catalog + autoGenerateTemplates: true # auto-generate Scaffolder templates from catalog items +``` + +### 7.3 Kubernetes Deployment Pattern + +```yaml +# RHDH configuration in OpenShift/Kubernetes +apiVersion: v1 +kind: ConfigMap +metadata: + name: rhdh-app-config + namespace: rhdh +data: + app-config.dcm.yaml: | + dcm: + baseUrl: https://dcm-api.dcm-system.svc.cluster.local + # ... (internal cluster DNS for in-cluster communication) + +--- +apiVersion: v1 +kind: Secret +metadata: + name: dcm-integration-secrets + namespace: rhdh +stringData: + DCM_SERVICE_ACCOUNT_API_KEY: "" + DCM_OIDC_CLIENT_SECRET: "" +``` + +### 7.4 Zero-Trust in Cluster + +RHDH backend โ†’ DCM Consumer API communication: +- Both running in same Kubernetes cluster (typically) +- mTLS enforced by Istio service mesh (ICOM model applies to RHDH as a client) +- RHDH is not a DCM internal component โ€” it is an external client that uses the Consumer API +- Auth: OIDC token exchange (Section 2.1) โ€” RHDH backend presents OIDC access token; DCM issues session token + +--- + +## 8. RHDH Pre-Built Capabilities Leveraged + +### 8.1 No-Build Integrations (Immediate Value) + +These work before writing any DCM-specific code: + +| RHDH Feature | DCM Benefit | Config needed | +|-------------|-------------|---------------| +| Keycloak/RHSSO auth | SSO into DCM portal โ€” same login as everything else | Configure OIDC provider | +| RBAC Plugin | No-code role management | Define DCM groups | +| TechDocs | DCM docs rendered in-portal | Add `techdocs-ref` annotations | +| Search | DCM entities searchable | Provided by catalog backend plugin | +| Kubernetes plugin | See DCM pods alongside resources | Standard RHDH Kubernetes plugin config | +| ArgoCD plugin | Layer store GitOps visibility | Standard RHDH ArgoCD plugin config | +| Tekton plugin | DCM scaffolding pipeline visibility | Standard RHDH Tekton plugin config | + +### 8.2 Ansible Automation Platform Plugin + +RHDH ships an existing AAP (Ansible Automation Platform) plugin. DCM Service Providers that use Ansible Automation Platform can surface AAP job status directly in RHDH: + +``` +DCMResource entity page +โ””โ”€โ”€ Additional tab contributed by AAP plugin: + "Automation" โ† shows AAP job runs for this resource's provisioning +``` + +This requires no DCM code โ€” it emerges from RHDH's existing AAP plugin + `dcm.io/aap-job-id` annotation on DCMResource entities. + +### 8.3 OCM (Open Cluster Management) Plugin + +Organizations using OCM for cluster lifecycle management get cluster management alongside DCM service catalog in the same portal โ€” genuinely one pane of glass for sovereign cloud operations. + +--- + +## 9. Migration Path โ€” Standalone SPA โ†’ RHDH + +Organizations starting with the standalone SPA can migrate to RHDH mode without data migration: + +``` +Phase 1 โ€” Standalone SPA + DCM deployed; standalone React app as consumer portal + All DCM functionality operational + +Phase 2 โ€” Install RHDH + RHDH deployed in same cluster + Configure Keycloak/RHSSO (shared IdP) + +Phase 3 โ€” Load DCM Dynamic Plugins + Add DCM plugins to RHDH app-config + No RHDH rebuild required (Dynamic Plugins) + DCM entities appear in RHDH catalog + +Phase 4 โ€” Auto-generate Templates + DCM catalog items become Backstage Software Templates + Developers start using RHDH "Create" for DCM requests + +Phase 5 โ€” Decommission standalone SPA + Users have migrated to RHDH + Standalone SPA can be retired +``` + +Total migration effort: primarily configuration. No data migration, no API changes, no DCM control plane changes. + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md index 0a1ad28..d8f3934 100644 --- a/content/docs/capabilities-matrix.md +++ b/content/docs/capabilities-matrix.md @@ -23,6 +23,20 @@ | IAM-005 | Multi-Factor Authentication | Satisfy per-session and step-up MFA challenges | โ€” | Configure MFA methods; declare step-up operations | IAM-001 | | IAM-006 | SCIM Automated Provisioning | Actor created/updated/deprovisioned from IdP automatically | โ€” | Configure SCIM endpoint and attribute mappings | IAM-001 | | IAM-007 | Tenant Scope Enforcement | Access restricted to authorized Tenants | โ€” | Declare Tenant membership; configure cross-tenant policies | IAM-003, IAM-004 | +| AUTH-002 | Multi-Auth-Provider Routing | โ€” | โ€” | Register multiple Auth Providers simultaneously; ingress routes by authentication signal | IAM-001 | +| AUTH-003 | Auth Provider Trust Level Enforcement | Requests evaluated per provider trust level (authoritative / verified / advisory) | โ€” | Configure trust level per registered Auth Provider | IAM-001 | +| AUTH-004 | Auth Provider Artifact Versioning | โ€” | โ€” | Manage role/tenant mapping versioning through standard DCM artifact lifecycle; activate/deprecate mappings | IAM-003 | +| AUTH-005 | Auth Provider Failover | Existing sessions remain valid on provider failure; new auth routes to failover chain | โ€” | Configure failover chain; monitor provider health; manage session cache TTL | IAM-001 | +| AUTH-006 | Auth Context in Audit Trail | โ€” | โ€” | Auth Provider identity and ingress context automatically recorded in all audit records | IAM-001 | +| AUTH-007 | Auth Provider Credential Security | โ€” | โ€” | Enforce Auth Provider config credentials reference Credential Provider; no plaintext credentials | IAM-001 | +| AUTH-008 | No Anonymous Access | โ€” | โ€” | Enforce authenticated access at all ingress surfaces across all profiles | IAM-001 | +| AUTH-009 | Webhook and Message Bus Authentication | Authenticate webhook registrations | โ€” | Enforce authentication on all inbound surfaces regardless of profile | IAM-001 | +| AUTH-010 | Per-Actor Rate Limiting | Receive 429 responses when rate limit exceeded | โ€” | Configure rate limits per actor; manage burst allowances | IAM-001 | +| AUTH-011 | Git PR Identity Resolution | Git PR submissions resolve to same actor identity as API/UI login | โ€” | Configure Auth Provider to trust Git server's identity assertion | IAM-001 | +| AUTH-012 | SCIM Automated Provisioning | Actor created/updated/deprovisioned from IdP automatically via SCIM | โ€” | Configure SCIM 2.0 endpoint; manage suspension-on-deprovision policy | IAM-001 | +| AUTH-013 | In-Flight Request Continuity on Auth Failure | In-flight requests before auth failure are not interrupted | โ€” | Configure session cache TTL; manage graceful degradation | IAM-001 | +| AUTH-014 | Two-Tier MFA Enforcement | Satisfy per-session MFA at login and step-up MFA for high-risk operations | โ€” | Configure per-session and step-up MFA; declare step-up trigger operations | IAM-005 | +| AUTH-015 | Built-In Auth Provider Storage Backend | โ€” | โ€” | Configure built-in Auth Provider storage backend (SQLite for minimal/dev; PostgreSQL/MySQL for standard+) | IAM-001 | --- @@ -290,6 +304,8 @@ | SMX-006 | Profile Scoring Threshold Management | โ€” | โ€” | Configure approval routing thresholds per profile (auto/reviewed/verified/authorized + custom tiers via named-tier list); manage signal weights; enforce SMX-008 (max auto_approve_below: 50) | POL-005, REQ-004 | | SMX-007 | Policy Enforcement Class Override | โ€” | Contribute policies with declared enforcement_class; receive notification when profile overrides enforcement class | Declare per-profile enforcement class overrides; manage regulatory_mandate flag to protect compliance-class policies from demotion | POL-004, POL-005 | | SMX-008 | Score Audit Trail | Query risk score and routing decision for own requests; view score_drivers and advisory_warnings | โ€” | Query full Score Record detail including signal breakdown and actor risk history; manage score audit retention | AUD-001, REQ-004 | +| SMX-009 | Scoring Weight Range Enforcement | โ€” | Declare operational GateKeeper scoring_weight between 1 and 100 | Enforce weight range at policy activation; reject out-of-range weights | SMX-001 | +| SMX-010 | Score Breakdown Audit Inclusion | View score breakdown in request audit record | โ€” | Configure score breakdown storage in Audit Store for all scored requests | SMX-001, AUD-001 | --- @@ -305,6 +321,7 @@ | MPX-005 | Transparent Constituent Visibility | Query and manage DCM-visible constituent entities independently (when transparency mode); receive constituent-level drift alerts | Declare composition_visibility mode; register transparent constituents with deterministic UUIDs | Configure visibility mode per compound resource type; manage constituent entity lifecycle policies | MPX-001, DRF-001 | | MPX-006 | Compound Execution Status Tracking | Monitor compound execution round progress via request status; see component-level status during long-running compositions | Send intermediate status events to DCM during execution; declare status_reporting.interval | Monitor compound execution health; configure execution timeout alerts | MPX-001, REQ-008 | | MPX-007 | Nested Meta Provider Composition | Request high-order compound services composed of other compound services (max depth 3) | Implement as a Meta Provider that calls other Meta Providers as constituents; declare max_nesting_depth | Configure nesting depth limits; manage nested compensation chains | MPX-001, PRV-009 | +| MPX-008 | Meta Provider Nesting Depth Enforcement | โ€” | Declare nesting depth in Meta Provider registration | Enforce maximum nesting depth of 3 at placement time; reject deeper compositions | MPX-001 | --- @@ -320,6 +337,11 @@ | CPX-005 | Credential Revocation | Receive revocation notification when credentials are revoked (actor deprovisioned, entity decommissioned); confirm transition to new credential | Implement revoke endpoint with declared SLA; invalidate value immediately on emergency revocation | Manage Credential Revocation Registry; configure revocation cache TTL per profile (PT1M standard, PT30S fsi/sovereign); enforce CPX-007 (decommission blocks on credential revocation) | CPX-001, LCM-007 | | CPX-006 | Revocation Propagation | โ€” | Refresh revocation cache within profile-governed TTL; validate credential UUID against cache at use time (not only at receipt) | Configure revocation cache TTL; monitor revocation propagation latency; alert on SLA violations | CPX-005, IAM-001 | | CPX-007 | Audit Trail for Credential Lifecycle | View own credential record history (issue, rotate, revoke events); every value retrieval audited with retrieval_uuid | โ€” | Query full credential audit trail including retrieval count; manage credential audit retention | CPX-001, AUD-001 | +| CPX-008 | IP-Bound Credentials for fsi/sovereign | โ€” | โ€” | Enforce IP binding (bound_to_ip) on all credentials issued for fsi and sovereign profiles | CPX-001, ZTS-002 | +| CPX-009 | Algorithm and Key Usage Declaration | โ€” | Declare algorithm and key_usage on credential records at issuance | Enforce declaration at issuance; reject credentials without declared algorithm | CPX-001 | +| CPX-010 | Idle Credential Detection | Receive notification when credential reaches idle threshold | โ€” | Configure idle_detection_threshold per profile; enforce alert-only action | CPX-001 | +| CPX-011 | Compliance Domain Additive Credential Requirements | โ€” | โ€” | Enforce additive credential requirements when compliance domains are active on a profile | CPX-001, POL-005 | +| CPX-012 | Credential Value Store Isolation (All Profiles) | โ€” | โ€” | Enforce credential values never stored in DCM stores in ALL profiles including minimal | CPX-001 | --- @@ -335,6 +357,11 @@ | ATM-005 | Degradation Review Gate | โ€” | โ€” | Review and explicitly accept each SECURITY_DEGRADATION item before a registry change activates; provide compensating control rationale; must hold verified or authorized tier reviewer role | ATM-004, IAM-001 | | ATM-006 | Profile Gap Detection | โ€” | โ€” | Receive PROFILE_GAP warnings when tier registry changes leave profile threshold lists incomplete; update threshold lists or acknowledge gap within approval window | ATM-003, ATM-004 | | ATM-007 | Tier Registry Audit Trail | Query historical tier registry versions and impact reports | โ€” | Access full audit trail of all tier registry changes: proposal, impact assessment, degradation acceptances, activation | ATM-004, AUD-001 | +| ATM-008 | Approval Record Tier Weight Snapshot | โ€” | โ€” | Store tier name and resolved weight at approval record creation; historical records retain weight for audit comparison across regime changes | ATM-001 | +| ATM-009 | Tier Registry Degradation Gate | Receive notification when tier change produces degradation affecting owned resources | โ€” | Block tier registry activation on SECURITY_DEGRADATION items; require verified-tier acceptance per item | ATM-001 | +| ATM-010 | Broken Reference Gate | โ€” | โ€” | Block tier registry activation when BROKEN_REFERENCE items exist (removed tier still referenced in active config) | ATM-001 | +| ATM-011 | Tier Change Impact Report | โ€” | โ€” | Generate and store tier impact report in Audit Store at proposal and activation time | ATM-001, AUD-001 | +| ATM-012 | Profile Gap Warning on Tier Insertion | โ€” | โ€” | Detect PROFILE_GAP when new tier inserted but profile threshold list not updated; emit non-blocking warning | ATM-001, POL-005 | --- @@ -363,6 +390,9 @@ | VER-003 | Deprecation Headers | Receive `Deprecation`, `Sunset`, and `Link` headers on all responses from deprecated API versions (RFC 8594/RFC 9745); use these to drive migration priority | โ€” | Configure header injection for deprecated versions; ensure headers are accurate | VER-002 | | VER-004 | Migration Guide | Access machine-readable migration guide at `GET /api/v{N}/migration-guide`; understand all breaking changes from previous version with migration instructions | Access OIS migration guide at `GET /provider/api/v{N}/migration-guide` | Maintain migration guides for all new major versions (required by VER-008) | VER-002 | | VER-005 | Preview Endpoints | Access preview endpoints at `/api/v{N}/preview/`; understand stability commitment is none; provide feedback before graduation | โ€” | Mark endpoints as preview; graduate to stable in new major version | VER-001 | +| VER-006 | Latest Alias Production Warning | โ€” | โ€” | Support `latest` version alias; emit response header discouraging production use | VER-001 | +| VER-007 | Preview Endpoint Instability Declaration | โ€” | โ€” | Mark preview endpoints explicitly; may change or be removed without major version bump | VER-001 | +| VER-009 | Provider Dispatch Compatibility | โ€” | Maintain backward compatibility with DCM dispatch payloads from supported prior versions | Manage provider dispatch versioning; maintain supported version matrix | VER-001 | --- @@ -376,6 +406,12 @@ | SES-003 | Emergency Session Revocation | Receive critical notification on security-event session revocation | โ€” | Trigger emergency revocation (security_event); revocation propagates within profile SLA (PT5S sovereign to PT30S standard) | SES-001, EVT-001 | | SES-004 | Token Introspection | โ€” | Call POST /api/v1/auth/introspect to validate bearer tokens without maintaining own revocation cache | Configure introspection endpoint access; manage introspection scope grants | SES-001, IAM-001 | | SES-005 | Concurrent Session Enforcement | Oldest session auto-revoked when new session exceeds concurrent limit; receive notification via Notification Provider | โ€” | Configure concurrent_sessions limit per profile; monitor session counts | SES-001 | +| AUTH-017 | Session Revocation Propagation SLA | โ€” | โ€” | Propagate revocation to Session Revocation Registry within profile SLA (PT5S sovereign โ†’ PT30S standard) | SES-001 | +| AUTH-018 | Per-Request Revocation Registry Check | โ€” | Check Session Revocation Registry on each request bearing a bearer token | Configure revocation registry query path; manage registry availability | SES-001 | +| AUTH-019 | Emergency Revocation No-Grace Period | Receive critical notification on emergency revocation | โ€” | Emergency session revocation fires immediately with no grace period | SES-003 | +| AUTH-020 | Introspection Endpoint Authentication | โ€” | Authenticate introspection calls using provider interaction credential | Configure introspection scope grants; manage endpoint access | SES-004 | +| AUTH-021 | Oldest Session Revocation on Limit | Receive notification when oldest session is auto-revoked | โ€” | Configure concurrent_sessions limit; enforce oldest-first revocation order | SES-005 | +| AUTH-022 | Refresh Token Invalidation on Session Revoke | โ€” | โ€” | Invalidate refresh token when parent session is revoked; return REVOKED_SESSION error on use | SES-001 | --- @@ -388,6 +424,9 @@ | ICOM-003 | Internal Call Authorization | โ€” | โ€” | Declare allowed_sources per internal endpoint; declare allowed_targets per component; unauthorized source calls rejected with ICOM_UNAUTHORIZED_SOURCE audit record (urgency: high) | ICOM-001, AUD-001 | | ICOM-004 | Internal Interaction Credentials | โ€” | โ€” | Every internal call presents a scoped ZTS-002 interaction credential in addition to mTLS; credential scoped to specific operation and target component | ICOM-001, CPX-002, ZTS-002 | | ICOM-005 | Component Certificate Revocation | โ€” | โ€” | Compromised component certificates added to Internal CA CRL immediately; CRL cache refresh within profile SLA (PT15S sovereign to PT1M standard); ICOM_CERT_COMPROMISED audit record (urgency: critical) | ICOM-001, AUD-001 | +| ICOM-006 | Component Certificate Maximum Validity | โ€” | โ€” | Issue internal component certificates with maximum validity P90D; enforce expiry and rotation | ICOM-001 | +| ICOM-008 | Compromised Certificate Immediate CRL | โ€” | โ€” | Add compromised internal component certificates to Internal CA CRL immediately; propagate within PT60S | ICOM-001 | +| ICOM-009 | Trust Anchor Registration | โ€” | โ€” | Register root or intermediate CA as trust anchor for internal mTLS; reject certificates not chaining to registered trust anchor | ICOM-001 | --- @@ -400,6 +439,8 @@ | SCH-002 | Maintenance Windows | Reference maintenance windows in scheduled requests; view available windows at GET /api/v1/maintenance-windows | โ€” | Create/manage/suspend maintenance windows; approve window schedules; configure platform-wide windows | SCH-001, GOV-001 | | SCH-003 | Dual Policy Evaluation | โ€” | โ€” | Understand that scheduled requests run GateKeeper at declaration AND at dispatch; dispatch-time failure โ†’ FAILED with schedule_policy_rejection (SCH-003) | SCH-001, POL-001 | | SCH-004 | Deadline Enforcement | Set not_after on scheduled requests; receive request.failed(schedule_deadline_missed) if deadline passes without dispatch | โ€” | Monitor deadline miss rates; configure alerting on deadline misses | SCH-001, EVT-001 | +| SCH-005 | Not-After Expiry Failure | Receive FAILED status when scheduled request expires before dispatch | โ€” | Configure not_after enforcement; manage SCHEDULE_EXPIRED recovery policy | SCH-001 | +| SCH-006 | Maintenance Window Platform Authorization | โ€” | โ€” | Require platform_admin or tenant_admin authority to create/modify Maintenance Windows | SCH-004 | --- @@ -411,6 +452,8 @@ | RDG-002 | Field Injection | Declare inject_fields to pass realized output fields (e.g. IP address) from dependency into dependent request fields automatically at dispatch time | โ€” | โ€” | RDG-001, REQ-001 | | RDG-003 | PENDING_DEPENDENCY Status | Track dependent requests in PENDING_DEPENDENCY status; cancel pending requests individually or cancel whole group; receive request.pending_dependency and request.dependency_met events | โ€” | Monitor PENDING_DEPENDENCY queue depth; detect stalled groups | RDG-001, EVT-001 | | RDG-004 | Group Failure Handling | Configure on_failure: cancel_remaining or continue; group-level timeout; group status via GET /api/v1/request-groups/{uuid} | โ€” | Monitor group failure rates | RDG-001 | +| RDG-005 | Group-Level Timeout Enforcement | Receive TIMEOUT failure when group-level timeout elapses | โ€” | Configure group_timeout independent of individual request timeouts | RDG-001 | +| RDG-006 | Single Group Membership Enforcement | โ€” | โ€” | Reject attempts to add a request to more than one dependency group | RDG-001 | --- @@ -422,6 +465,75 @@ | HLT-002 | Readiness Probe | โ€” | โ€” | GET /readyz: checks Session Store, Audit Store, Policy Engine, Message Bus, Auth Provider connectivity; Kubernetes removes from LB on failure; startup sequence observable via readyz | โ€” | | HLT-003 | Component Health Detail | โ€” | โ€” | GET /api/v1/admin/health: per-component status (pass/warn/fail), metrics, queue depths, provider/auth summary; admin auth required | IAM-001 | | HLT-004 | Prometheus Metrics | โ€” | โ€” | GET /metrics: Prometheus scrape endpoint; request pipeline, policy, session, drift, provider, internal CA metrics | OBS-001 | +| HLT-005 | Prometheus Metrics Endpoint | โ€” | โ€” | Expose Prometheus-compatible metrics at GET /metrics including request throughput, store latency, policy eval time, provider health counters | HLT-001 | +| HLT-006 | Startup Readiness via /readyz | โ€” | โ€” | Report PASSING on /readyz only after all required stores available and bootstrap complete | HLT-001 | + +--- + + +## 32. Operational Reference + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| OPS-001 | GitOps Store Partitioning | โ€” | โ€” | Declare partitioning strategy in deployment manifest; execute tenant-shard, per-tenant, or time-based archiving migration; configure shard routing and mirror lag monitoring | STO-001 | +| OPS-002 | Store Migration | โ€” | โ€” | Execute dual-write migration between store implementations; maintain audit chain continuity across cutover; enforce profile-governed burn-in before source decommission | STO-001, AUD-001 | +| OPS-003 | Disaster Recovery | โ€” | โ€” | Execute scenario-specific recovery procedures (component/store/full-CP/repave); meet profile-governed RTOs (PT1Mโ€“PT15M component, PT5Mโ€“PT2H store, PT5Mโ€“PT30M full-CP); complete post-recovery validation checklist | HLT-001, AUD-001 | +| OPS-004 | Backup Management | โ€” | โ€” | Configure store-appropriate backup schedules (Git push for GitOps stores, PITR for Realized Store, Kafka snapshots for Audit Store); enforce P365D minimum Audit Store retention | STO-001 | + +--- + + +## 33. Web Interfaces + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| GUI-001 | Consumer Portal โ€” Catalog, Requests, Resources | Browse catalog; live cost estimate; submit requests with scheduling and dependency groups; live SSE status with constituent tracking; cross-resource drift report; consumer-scoped audit trail with correlation ID trace | โ€” | โ€” | CAT-001, REQ-001, SCH-001, RDG-001, EVT-002 | +| GUI-002 | Consumer Portal โ€” Live Request Status | Real-time status via SSE stream (status_change, progress_updated, approval events, heartbeat); constituent status for compound requests; approval flow inline; fallback to polling | โ€” | โ€” | REQ-001, EVT-002 | +| GUI-003 | Consumer Portal โ€” Resource Management | View/filter owned resources by state and type; lifecycle state badges; drift indicator; resource detail with Overview/Drift/Audit/Cost/Credentials/Relationships/Groups tabs; state-sensitive action buttons; bulk operations | โ€” | โ€” | LCM-001, DRF-001, AUD-001 | +| GUI-004 | Consumer Portal โ€” Session and Security | View active sessions; revoke individual or all other sessions; step-up MFA prompt for gated operations; tenant context selector; role-gated navigation (hide not disable) | โ€” | โ€” | SES-001, IAM-001 | +| GUI-005 | Admin Panel โ€” Platform Dashboard | Control plane component health grid; provider health summary; pending approvals count; open drift records by severity; request throughput; all driven by GET /api/v1/admin/health | โ€” | Platform Admins, SREs configure dashboard widgets; role-gated sections | HLT-003 | +| GUI-006 | Admin Panel โ€” Governance and Approvals | Approval queue (all tenants); approval detail with risk score breakdown; authority tier registry editor (drag-and-drop reordering, impact report visualization, degradation acceptance flow); scoring threshold editor (auto_approve_below โ‰ค 50 hard-stop) | โ€” | Policy Owners and Platform Admins | ATM-004, SMX-001 | +| GUI-007 | Admin Panel โ€” Audit and Compliance | Platform-wide cross-tenant audit trail; pre-built compliance reports (SOC 2, FedRAMP, HIPAA); audit chain integrity status; correlation ID trace; session and security event feed | โ€” | Auditors, Security team, Platform Admins | AUD-001, SES-003 | +| GUI-008 | Provider Management โ€” Common Shell | Overview, configuration, health history, audit trail, and notification tabs for all 11 provider types; provider owner role gates access; Platform Admins see all providers | โ€” | Provider owners manage own providers; Platform Admins manage all | PRV-001, IAM-001 | +| GUI-009 | Provider Management โ€” Type Extensions | Service Provider: capacity, managed entities, naturalization mapping, realization history; Credential Provider: inventory, rotation, revocation, external CA config, algorithm compliance; Auth Provider: session stats, SCIM sync, connection status; Policy Provider: trust level, contribution pipeline | โ€” | Provider owners access type-specific tabs for their provider type | GUI-008, PRV-001 | +| GUI-011 | RHDH Plugin Suite | Use DCM capabilities within Red Hat Developer Hub (RHDH) or Backstage via Dynamic Plugins (@dcm/backstage-plugin-*); no RHDH rebuild required for updates | โ€” | Configure RHDH app-config.yaml with DCM connection; configure Dynamic Plugin loading | +| GUI-012 | Scaffolder Template Auto-Generation | DCM catalog items automatically generate Backstage Software Templates; new resource types appear as templates without UI code; field schema โ†’ JSON Schema โ†’ Scaffolder form | โ€” | Configure @dcm/backstage-plugin-catalog-backend; template generation is automatic | +| GUI-013 | DCM Entity Provider | DCMService (catalog items) and DCMResource (realized entities) appear in RHDH Software Catalog; entities sync every PT5M; search-indexed; tenancy enforced via namespace | โ€” | Service account credential configuration; sync interval configuration | +| GUI-014 | ITSM Integration Bridge | View ITSM references (ServiceNow, Jira) on resource entity Overview tab; link change records to DCM requests; see ITSM-sourced approval votes in request status; CMDB reference on entity pages | ITSM systems receive DCM lifecycle events via Notification Provider and call Admin API to record approval votes; CMDB sync via webhook subscription | Configure ITSM Notification Provider; map DCM event โ†’ ITSM action; configure CMDB field mapping | EVT-001, GUI-002 | +| GUI-010 | Unified Shell | Single DCM web application with role-gated surfaces: Consumer Portal (all actors), Admin Panel (platform roles), Provider Management (provider_owner role), Flow GUI link (policy_owner/sre); one login, one session; no separate applications | โ€” | Platform Admins configure which surfaces are available | IAM-001, SES-001 | + +--- + + +## 34. ITSM Integration + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ITSM-001 | ITSM Provider Registration | โ€” | Register as ITSM Provider with declared capabilities (supported_actions, itsm_system, field_mapping_ref, cmdb_ci_type_map); implement standard OIS health check | Register ITSM Providers; review and approve ITSM Provider registrations; configure inbound webhook authentication | PRV-001, CPX-001 | +| ITSM-002 | Outbound ITSM Record Creation | View ITSM references on resource entities (change request, incident, CMDB CI links with deep links to ITSM system) | Receive action requests from DCM; create/update records in ITSM system; return record ID for storage on entity | Configure ITSM Policies (create_change_request, create_incident, update_cmdb_ci); configure block_until_created for compliance gates | ITSM-001, POL-001 | +| ITSM-003 | Inbound ITSM Approval Routing | โ€” | Verify HMAC signature on inbound webhook; forward ITSM approval decisions to DCM Admin API approval vote endpoint | Configure inbound webhook secret (Credential Provider); monitor approval routing from ITSM systems (ServiceNow CAB, Jira workflow) | ITSM-001, CPX-001, ATM-001 | +| ITSM-004 | ITSM Policy Authoring | โ€” | โ€” | Author ITSM Action policies with template expressions; configure shadow validation; configure on_failure behavior; use block_until_created for pipeline gates (with mandatory timeout per ITSM-005) | ITSM-001, POL-001 | +| ITSM-005 | CMDB Synchronization | View CMDB CI reference on resource entities; CI auto-created on realization, auto-retired on decommission | Receive create_cmdb_ci and retire_cmdb_ci actions; maintain dcm_entity_uuid correlation on CMDB CI records | Configure CMDB CI type mapping per resource type; monitor CMDB sync failures | ITSM-001, ITSM-002 | +| ITSM-006 | ITSM Field Mapping Declaration | โ€” | โ€” | Declare field mappings between DCM entity fields and ITSM CI types in ITSM Provider config; validate against Resource Type Specs | ITSM-001 | +| ITSM-007 | ITSM Policy Template Expression Validation | โ€” | โ€” | Validate template expressions in ITSM Policy action_payload at policy activation; reject unresolvable expressions | ITSM-001, POL-003 | + +--- + + +## 35. Provider Callback Authentication + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| PCA-001 | Two-Layer Provider Callback Authentication | โ€” | Present valid mTLS certificate (Layer 1) and provider callback credential (Layer 2) on every call to DCM callback endpoints | Configure DCM CA trust anchor; issue provider callback credentials at activation; enforce both layers | PRV-001, ZTS-001 | +| PCA-002 | Provider Callback Credential Scope Enforcement | โ€” | Use callback credential scoped to own provider_uuid only; cannot act on other providers | Enforce credential scope at validation; reject cross-provider credential use | PRV-001 | +| PCA-003 | Entity-Level Callback Authorization | โ€” | Receive 403 ENTITY_NOT_OWNED_BY_PROVIDER when pushing state for entities not dispatched to this provider | Enforce per-call entity ownership check independent of credential validity | PRV-001, REQ-007 | +| PCA-004 | Scope Violation Auto-Suspension | Receive critical notification when owned provider is suspended due to scope violations | โ€” | Auto-suspend provider and notify platform admin after 5 consecutive scope violations within PT1H | PRV-001, ZTS-001 | +| PCA-005 | Callback Credential Issued by Credential Provider | โ€” | Retrieve callback credential via Credential Provider at activation; not directly from API Gateway | Issue callback credentials exclusively through Credential Provider; reject direct credential issuance requests | PRV-001, CPX-001 | +| PCA-006 | Registration Token Single-Use Enforcement | โ€” | Use registration token for initial registration only; obtain callback credential after activation | Invalidate registration token after first successful use regardless of expiry timestamp | PRV-001 | +| PCA-007 | Sovereignty Change Re-Registration | โ€” | Submit new registration with new registration token when sovereignty declaration changes | Require new registration token and approval pipeline for sovereignty declaration changes | PRV-001, GMX-004 | +| PCA-008 | Callback Credential Pre-Expiry Rotation | โ€” | Implement credential refresh; receive new credential before old expires during transition window | Initiate rotation before expiry; maintain transition window (50% of credential lifetime) | PRV-001, CPX-001 | +| PCA-009 | IP-Bound Callback Credentials for fsi/sovereign | โ€” | Present callback calls from declared bound_to_ip address for fsi/sovereign profiles | Enforce IP binding on callback credentials for fsi and sovereign profiles | PCA-001, CPX-008 | +| PCA-010 | All Inbound Provider Calls Produce Audit Records | โ€” | โ€” | Write audit record for every inbound provider call including rejected calls; no silent failures | PCA-001, AUD-001 | --- @@ -461,7 +573,42 @@ | Scheduled Requests | 4 | | Request Dependency Graph | 4 | | DCM Self-Health | 4 | -| **Total** | **189** | +| Identity and Access Management | 21 | +| Service Catalog | 7 | +| Request Lifecycle Management | 10 | +| Provider Contract and Realization | 9 | +| Resource Lifecycle Management | 7 | +| Drift Detection and Remediation | 5 | +| Policy Management | 7 | +| Data Layer Management | 5 | +| Information and Data Integration | 6 | +| Ingestion and Brownfield Management | 4 | +| Audit and Compliance | 5 | +| Observability and Operations | 5 | +| Storage and State Management | 6 | +| DCM Federation and Multi-Instance | 5 | +| Platform Governance and Administration | 7 | +| Accreditation Management | 6 | +| Zero Trust and Security Posture | 6 | +| Unified Governance Matrix | 7 | +| Drift Reconciliation | 5 | +| Federated Contribution Model | 7 | +| Scoring Model | 10 | +| Meta Provider Composability | 8 | +| Credential Provider Model | 12 | +| Authority Tier Model | 12 | +| Event Catalog | 7 | +| API Versioning | 8 | +| Session Revocation | 11 | +| Internal Component Authentication | 8 | +| Scheduled and Deferred Requests | 6 | +| Request Dependency Graph | 6 | +| DCM Self-Health | 6 | +| Operational Reference | 4 | +| Web Interfaces | 14 | +| ITSM Integration | 7 | +| Provider Callback Authentication | 10 | +| **Total** | **269** | --- diff --git a/content/docs/data-model/00-context-and-purpose.md b/content/docs/data-model/00-context-and-purpose.md new file mode 100644 index 0000000..8f0e641 --- /dev/null +++ b/content/docs/data-model/00-context-and-purpose.md @@ -0,0 +1,384 @@ +# DCM Data Model โ€” Context and Purpose + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** โœ… Complete +**Related Documents:** [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md) + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” foundational data model, provenance, four lifecycle states + + + +--- + +## 1. Purpose + +The DCM Data Model is the foundational layer upon which the entire DCM framework operates. It is not a storage mechanism or a database schema โ€” it is the **lingua franca of DCM**. Every component in the DCM architecture communicates through the data model in some form, whether reading, writing, validating, enriching, transforming, or comparing data. + +The data model exists to solve a problem that is endemic to enterprise IT: **there is no single, trustworthy, consistent representation of infrastructure state**. Tools proliferate, CMDBs diverge, and the result is that no one knows with confidence what exists, what was requested, what was provisioned, or whether the current state matches the intended state. + +The DCM Data Model establishes a **unified, versioned, declarative single source of truth** for all infrastructure state across the full lifecycle of every resource DCM manages. + +--- + +## 2. Role in the DCM Architecture + +The data model is not owned by any single component โ€” it is the contract between all components. Every major DCM capability acts on data in a specific and well-defined way: + +| Component | Relationship to Data | +|-----------|---------------------| +| Request Payload Processor | Assembles and enriches data into a complete request payload | +| Policy Engine | Reads, validates, transforms, and gates data based on policy definitions | +| Service Provider | Consumes data (via Naturalization) and returns data (via Denaturalization) | +| Orchestration | Coordinates component interactions based on data state and dependencies | +| Audit | Records data at every state transition for compliance evidence | +| Drift Reconciliation | Compares versions of data across states to detect and remediate drift | +| Cost Analysis | Derives cost information from data throughout the resource lifecycle | +| Resource Discovery | Produces data representing the current discovered state of resources | +| IDM / IAM | Gates access to data and operations based on identity and role | +| Service Catalog | Exposes available services based on data definitions and RBAC policy | + +This means the data model is effectively the **API between all DCM components** โ€” even components that do not communicate directly are coupled through the data model. A well-designed data model makes every component easier to build, test, and evolve independently. + +--- + +## 3. Universal Identity Requirement + +Every data object in DCM must have a **UUID (Universally Unique Identifier)**. This is not optional โ€” it is a foundational requirement that applies to every entity in the data model without exception. + +UUIDs serve several critical functions: + +- **Unambiguous reference** โ€” any component, policy, layer, catalog item, or process that touches a data object can reference it precisely and without ambiguity +- **Provenance anchoring** โ€” every change recorded in a data object's lineage references the UUID of the entity that caused the change +- **Dependency mapping** โ€” relationships between resources, services, and components are expressed as UUID references, never by name alone +- **Audit fidelity** โ€” audit records reference UUIDs, ensuring that even if names or labels change, the audit trail remains accurate and traceable +- **Cross-state correlation** โ€” the same resource across Intent, Requested, Realized, and Discovered states can be correlated via UUID chains + +This applies to all entities including but not limited to: resource definitions, catalog items, data layers, policies, policy sets, components, service providers, consumers, and requests. + +--- + +## 4. Field-Level Provenance and Data Lineage + +One of the most critical requirements of the DCM Data Model is the ability to trace the complete lineage of any piece of data at any stage of the pipeline. This is not a logging concern โ€” it is a **structural requirement of the data model itself**. + +### 4.1 The Requirement + +At any point in the DCM pipeline, for any field in any data object, it must be possible to answer: + +- What is the current value of this field? +- Where did this value originate? (catalog item, base layer, intermediate layer, policy, consumer input, discovery) +- Has this value been modified since origination? +- If modified: + - What is the complete history of modifications? + - Which entity caused each modification? (identified by UUID) + - What type of entity caused it? (policy, layer, component, provider) + - When did each modification occur? + - What was the value before each modification? + - Why was the modification made? (enrichment, validation, transformation, gatekeeping) +- What is the complete chain of custody of this field from origin to current value? + +### 4.2 Why Field-Level Lineage Matters + +Document-level versioning alone is insufficient for DCM's requirements. Consider a resource request flowing through the pipeline: + +1. Consumer selects a catalog item โ€” catalog item UUID recorded +2. Base resource definition layer applied โ€” base layer UUID recorded, fields established +3. Intermediate layers applied โ€” each layer UUID recorded, field overrides recorded +4. Policy Engine validates โ€” policy UUID recorded, validation outcome recorded +5. Policy Engine enriches โ€” policy UUID recorded, enriched field values and their source recorded +6. Policy Engine transforms โ€” policy UUID recorded, transformation recorded with before/after values +7. Request payload submitted โ€” complete provenance chain intact across all fields + +Without field-level provenance, it is impossible to determine after the fact whether a specific field value came from a consumer request, a data layer, a security policy, or a business rule. This ambiguity is unacceptable in a governed, auditable system. + +### 4.3 Provenance as a Structural Element + +Field-level provenance must be carried within the data object itself โ€” not in an external log. This ensures that: + +- The data and its lineage are always co-located and cannot be separated +- Any consumer of the data can inspect its lineage without querying an external system +- Provenance survives data export, migration, and portability scenarios +- The audit capability reads provenance that is intrinsic to the data, not reconstructed from logs + +### 4.4 Provenance Metadata Structure + +Every field that can be created or modified by any DCM process carries provenance metadata alongside its value. The conceptual structure is: + +```yaml +field_name: + value: + metadata: + # Simple override declaration (Level 2) โ€” most fields only need this + override: + # OR matrix declaration (Level 3) โ€” for actor-specific governance + override_matrix: + default: + actors: + trusted_grants: + + # Always present regardless of level + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + constraint_schema: + + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_type: + source_uuid: + operation_type: + actor: + timestamp: + reason: +``` + +**Note:** The `metadata` block is set exclusively by the Policy Engine. Data layers and the Request Payload Processor never set override control. `operation_type: lock` is used when a GateKeeper policy sets `override: immutable`. `operation_type: grant` is used when a trusted_grant is issued. The three levels of override control are: Level 1 (no declaration โ€” fully overridable), Level 2 (simple `override:` property), Level 3 (full `override_matrix:` with per-actor permissions). See the Layering and Assembly document Section 5a for the complete model. + +### 4.5 Provenance Obligations + +Every DCM component that reads and modifies data carries a provenance obligation: + +| Component | Provenance Obligation | +|-----------|----------------------| +| Request Payload Processor | Record source UUID and type for every field assembled from layers and catalog items | +| Policy Engine | Record policy UUID, operation type, and reason for every field it enriches, transforms, or validates | +| Service Provider (Denaturalization) | Record provider UUID and timestamp for every field returned in the realized payload | +| Resource Discovery | Record provider UUID, discovery timestamp, and interrogation method for every field in the discovered payload | +| Data Layers | Each layer must declare its UUID so downstream provenance records can reference it | +| Catalog Items | Each catalog item must declare its UUID so downstream provenance records can reference it | + +Provenance recording is **not optional** for any component that modifies data. A component that modifies data without recording provenance violates the data model contract. + +### 4.6 Relationship to Audit + +The Audit capability in DCM reads provenance data that is intrinsic to every data object. This means: + +- Audit does not reconstruct history from logs โ€” it reads lineage that was recorded at the point of change +- Any data object can be audited at any time, in any state, by any authorized persona +- The audit trail is as durable and immutable as the data itself +- Compliance evidence is produced from the data, not from a separate audit system that could diverge from the data + +--- + +## 5. Foundational Constraints + +The DCM Data Model is governed by three foundational constraints that apply universally and without exception: + +### 5.1 Declarative + +Data in DCM describes **what something is or should be**, not how to achieve it. Every entity in the data model is a complete, self-describing statement of state. The procedures required to achieve that state are the concern of the Service Provider, not the data model. + +This means: +- A resource definition declares its desired configuration, not the steps to configure it +- A policy declares its conditions and outcomes, not its execution logic +- A layer declares its overrides, not the merge algorithm used to apply them + +### 5.2 Idempotent in Operation + +Applying the same data to the same system multiple times must always produce the same result. No operation on DCM data should have different outcomes based on how many times it has been applied. + +This is critical for: +- **Drift reconciliation** โ€” reapplying desired state to a drifted resource must produce correct results +- **DC rehydration** โ€” replaying the full set of declared states must reconstruct the environment correctly +- **Retry scenarios** โ€” failed operations can be safely retried without risk of inconsistent state +- **Audit and compliance** โ€” the same data, applied by anyone at any time, produces the same verifiable outcome + +### 5.3 Immutable if Versioned + +Once a version of any entity is published, it cannot be modified. If a change is required, a new version must be created. The previous version remains intact and accessible. + +This constraint is what makes the following capabilities trustworthy: +- **Audit trails** โ€” every state at every point in time is preserved and verifiable +- **Drift detection** โ€” comparison between states is meaningful because neither state can change retroactively +- **Intent portability** โ€” a previously declared intent can be replayed against current policies with confidence that the original intent is unchanged +- **Rollback** โ€” reverting to a previous version is always possible because previous versions are never destroyed +- **Chain of trust** โ€” the provenance of any configuration can be traced through an unbroken chain of immutable versions + +--- + +## 5a. Artifact Metadata Standard + +Every DCM artifact โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions, and all other defined or stored objects โ€” carries a universal **Artifact Metadata** block. This is a structural requirement that applies to all artifacts without exception. + +### The Purpose + +Artifact metadata answers: **who created this, when, who owns it, what changed, and how do we contact them?** It is the identity and accountability record for the artifact itself โ€” distinct from field-level provenance which tracks data value lineage. + +### The Five Artifact Statuses + +All DCM artifacts follow a five-status lifecycle: + +| Status | Meaning | Key Behavior | +|--------|---------|-------------| +| `developing` | In active development | Dev mode / dev pipeline only. Not applied in production. | +| `proposed` | Submitted for validation | Shadow execution for policies โ€” output captured, not applied. In PR review for data artifacts. | +| `active` | Live and governing | Applied to all relevant requests. | +| `deprecated` | Being phased out | Still works, replacement available, warning on use. | +| `retired` | End of life | Cannot be used. Terminal status. | + +### Key Design Decisions + +**created_by vs owned_by:** Deliberately separate. The creator is the audit record โ€” who physically submitted the artifact. The owner is the accountability record โ€” who is responsible and receives notifications for conflicts, deprecation warnings, and policy violations. + +**Contact info โ€” two modes:** When an Identity Provider is registered, the `uuid` field links to the IdP record and `display_name` is a non-authoritative display cache. In standalone/air-gapped mode, `uuid` is absent and `display_name` + `email` are the primary identity fields. Both modes are fully supported. + +**created_via:** Declares the ingestion path โ€” `pr` (full GitOps review history), `api` (direct submission), `migration` (imported, limited provenance), `system` (DCM-created). Makes audit quality transparent. + +**Proposed shadow execution:** Policy artifacts in `proposed` status execute in shadow mode against real traffic โ€” output is captured and reported but never applied. Enables safe validation before activation. + +See [Data Layers and Assembly โ€” Section 4b](03-layering-and-versioning.md) for the complete artifact metadata structure and all field definitions. + +--- + +## 6. The Four States + +DCM tracks the lifecycle of every resource through four distinct states. Together, these four states provide complete visibility into the gap between what was wanted, what was asked for, what was built, and what actually exists. + +### 6.1 Intent State + +The **Intent State** represents what a consumer wants to happen. It is the declared desire, captured at the moment a consumer initiates a request, before any processing, validation, or enrichment has occurred. + +- **When it is created:** When a consumer submits a request via the Web UI or Consumer API +- **Where it is stored:** Intent Store +- **Key characteristic:** Captures the consumer's raw intent โ€” what they asked for in their own terms +- **Primary use:** Source for Intent Portability โ€” replaying an intent through current policies to produce a new request for a different environment or provider + +### 6.2 Requested State + +The **Requested State** represents the fully processed, policy-validated, and enriched payload that has been submitted to a Service Provider for execution. It is the output of the Request Payload Processor after all policies have been applied and all data layers have been merged. + +- **When it is created:** When the Request Payload Processor completes processing and submits to the API Gateway +- **Where it is stored:** Request Store +- **Key characteristic:** Represents a complete, validated, provider-ready declaration of desired state +- **Primary use:** Record of what was formally requested; input to audit and drift processes + +### 6.3 Realized State + +The **Realized State** represents what was actually provisioned or executed by a Service Provider, returned to DCM in unified data model format via Denaturalization. It is the ground truth of what was built. + +- **When it is created:** When a Service Provider completes execution and returns the realized payload to the API Gateway +- **Where it is stored:** Realized Store +- **Key characteristic:** Must be a complete representation of the provisioned resource in DCM unified format โ€” not a status code, but a full state description +- **Primary use:** Baseline for drift detection; source of truth for audit and reporting; input to cost analysis + +### 6.4 Discovered State + +The **Discovered State** represents what actually exists in the environment as interrogated by a Service Provider during a discovery operation. It is an independent observation of reality, not derived from any previous DCM state. + +- **When it is created:** When a Service Provider completes a discovery cycle and returns the discovered payload to DCM +- **Where it is stored:** Discovered Store +- **Key characteristic:** Produced independently of the Realized State โ€” it is what is actually there, regardless of what was supposed to be there +- **Primary use:** Drift detection (compared against Realized State); brownfield ingestion (pathway to lifecycle ownership of unmanaged resources) + +### 6.5 State Relationships and Lifecycle Flow + +The four states relate to each other as follows: + +``` +Consumer Request + โ”‚ + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ INTENT โ”‚ โ—„โ”€โ”€ What the consumer wants +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Policy Engine processes, enriches, validates + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REQUESTED โ”‚ โ—„โ”€โ”€ What was formally submitted to the provider +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Service Provider executes + โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ REALIZED โ”‚ โ—„โ”€โ”€ What was actually built +โ”‚ STATE โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ + โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ Compare โ—„โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”‚ DISCOVERED โ”‚ โ—„โ”€โ”€ What actually exists now + โ”‚ โ”‚ STATE โ”‚ + โ–ผ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + Drift Detection +``` + +**Key operations across states:** +- **Drift Detection:** Discovered State vs. Realized State +- **Request Validation:** Requested State vs. Policy definitions +- **Intent Portability:** Intent State โ†’ re-process through current policies โ†’ new Requested State +- **Brownfield Ingestion:** Discovered State โ†’ enrichment โ†’ Realized State (lifecycle ownership) + +--- + +## 7. Data as the Provider Contract Boundary + +The data model defines the boundary between DCM and its Service Providers. DCM is explicitly **not concerned with how a provider accomplishes its work** โ€” only with the data that crosses the boundary in both directions. + +This means: +- Providers are interchangeable as long as they honor the data contract +- New providers can be added without changing DCM's core data model +- Provider implementation can evolve independently of DCM +- The contract is enforced at the data level โ€” conformant data in, conformant data out + +The two mechanisms that enforce this boundary are: +- **Naturalization** โ€” the provider's responsibility to transform DCM unified data into its own tool-specific format for execution +- **Denaturalization** โ€” the provider's responsibility to transform its tool-specific result data back into DCM unified format for return to the control plane + +This separation of concerns is what makes DCM technology-agnostic while maintaining a consistent and trustworthy data model across all providers. + +--- + +## 8. Open Questions + +The following questions remain unresolved and require decisions before the data model specification can be considered complete: + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Where should data caches live? | Cache architecture, latency, sovereignty | โœ… Resolved โ€” caches at Regional DCM level; Info Provider caches co-located with source; Sovereign DCM uses local static from signed bundles (CACHE-001) | +| 2 | Should cache synchronization be push, pull, or both? | Consistency model | โœ… Resolved โ€” hybrid push-pull; pull on schedule; push for time-sensitive events via Message Bus (CACHE-002) | +| 3 | Which cache is authoritative when caches diverge? | Conflict resolution | โœ… Resolved โ€” GitOps stores always authoritative; caches are projections; divergence triggers rebuild from authoritative store (CACHE-003) | +| 4 | What mechanism maintains consistency across distributed caches? | Data integrity | โœ… Resolved โ€” hash-based heartbeat divergence detection + push invalidation; PT2H staleness alert; signed bundles for Sovereign DCM (CACHE-004) | +| 5 | Should the data model allow embedded target-technology-specific data bundles? | Portability | โœ… Resolved โ€” native_passthrough field sanctioned; always audit-logged; opaque mode blocked in fsi/sovereign (DATA-001) | +| 6 | How are the four states represented physically? | Physical model | โœ… Resolved โ€” Intent/Requested in Git; Realized in Event Stream; Discovered in Discovered Store (STO-005) | +| 7 | What is the performance impact of field-level provenance at scale? What optimization strategies are acceptable? | Scalability, storage cost | โœ… Resolved โ€” three configurable provenance models: full_inline, deduplicated (Model B recommended), tiered; profile-appropriate Policy Groups; see docs 03 and 06 (OPS-001) | +| 8 | Should provenance metadata be stored inline with field data or in a linked provenance document? | Data model structure, query performance | โœ… Resolved โ€” three-level structure: implicit chain ref, inline delta, linked history document; all reconstructable from stored facts; see doc 03 (OPS-002) | + +--- + +## 9. Related Concepts + +- **Sovereign Execution Posture** โ€” the target end state the data model enables by providing a verified, auditable chain of custody through the full resource lifecycle +- **CMDB Replacement** โ€” DCM's four-state model is intended to replace the fragmented multi-CMDB problem by becoming the singular resource domain +- **GitOps** โ€” all entities in the data model are stored in Git, enabling version control, change tracking, and standard software lifecycle practices +- **Data Lineage** โ€” the complete chain of custody of any field value from its origin through every modification, recorded within the data object itself +- **Field-Level Provenance** โ€” the structural mechanism by which data lineage is captured, carried, and made available for audit and compliance purposes +- **UUID** โ€” the universal identity mechanism that makes provenance references, dependency mapping, and cross-state correlation unambiguous and durable + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/00-design-priorities.md b/content/docs/data-model/00-design-priorities.md new file mode 100644 index 0000000..a5f5bed --- /dev/null +++ b/content/docs/data-model/00-design-priorities.md @@ -0,0 +1,312 @@ +# DCM Design Priorities + +**Document Status:** โœ… Stable โ€” Foundational reference +**Document Type:** Architecture Reference โ€” Design Philosophy +**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Policy Profiles](14-policy-profiles.md) | [Scoring Model](29-scoring-model.md) | [Credential Provider Model](31-credential-provider-model.md) + +> **This document maps to: DATA + PROVIDER + POLICY** +> +> Design priorities govern every decision across all three abstractions. They are not guidelines โ€” they are the decision framework used when priorities conflict. Every contributor, implementer, and reviewer should apply this framework. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) + +--- + +## The Four Priorities + +Every design decision in DCM is evaluated against this hierarchy. When priorities conflict, higher priorities win. When there is no conflict, all four apply simultaneously. + +--- + +### Priority 1 โ€” Industry Best Practices for Security + +Security is not a feature, a profile option, or a compliance checkbox. It is the baseline that every other design decision must respect. + +**What this means:** + +Security properties โ€” value separation, rotation, audit trails, idle detection, algorithm baselines, scoped credentials, revocation propagation, shadow mode evaluation โ€” are **architecturally present in every profile**. What profiles control is enforcement strictness, threshold values, and automation level โ€” not whether the security property applies. + +**The `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security."** + +A `minimal` profile deployment: +- Rotates credentials (at longer intervals with manual triggers acceptable โ€” not never) +- Detects idle credentials (at a generous P30D threshold โ€” not never) +- Requires algorithm baselines (via forbidden list โ€” not null) +- Runs shadow mode on contributed policies (always โ€” not optionally) +- Audits first credential retrieval (always โ€” not sometimes) +- Maintains revocation registry (at PT5M cache TTL โ€” not disabled) + +The security model is present and correct. The enforcement strictness and automation burden are reduced. + +**When security and convenience conflict, security wins** โ€” but the design must find a way to make the secure option easy. A security model that is routinely bypassed because it is too burdensome has failed at both security and usability. The profile system is the mechanism: the right profile makes secure behavior automatic, not effortful. + +**Security properties that are non-negotiable in all profiles:** + +| Property | Rule | Reference | +|----------|------|-----------| +| Credential values never in DCM stores | CPX-001 โ€” absolute, no profile exception | doc 31 | +| Governance Matrix always boolean | SMX-004 โ€” scoring never applies to boundary decisions | doc 29 | +| Every provider dispatch requires scoped interaction credential | CPX-002 | doc 31 | +| Shadow mode on all contributed policies | FCM-004 | doc 28 | +| auto_approve_below โ‰ค 50 in all profiles | SMX-008 | doc 29 | +| First credential retrieval always audited | CPX-005 | doc 31 | +| Forbidden algorithm baseline always enforced | CPX-009 (no null approved_algorithms) | doc 31 | +| Revocation registry always maintained | CPX-003 | doc 31 | + +--- + +### Priority 2 โ€” Ease of Use + +DCM exists to enable self-service for application teams. If the right path is also the hard path, teams will find other paths โ€” and those other paths are ungoverned. + +**What this means:** + +The secure path must also be the easy path. Profile defaults should work for most deployments without customization. Ordinary requests should auto-approve without human intervention. Policy authoring should not require Rego expertise for common patterns. The Flow GUI, scoring model, and consumer contribution endpoints all exist to make governed behavior less operationally burdensome. + +**Ease of use serves security.** An organization that finds DCM too cumbersome and routes requests outside DCM has eliminated all of DCM's security benefits. A homelab team that circumvents credential management because it's too complex has no credential management. + +**The design principle:** When implementing a security requirement, simultaneously design the ease-of-use mechanism that makes it effortless to comply with. The scoring model's auto-approval threshold (not making every request require human review) is ease of use in service of security. + +**Things that should be easy in all profiles:** +- Requesting a standard resource (auto-approve for clean requests) +- Authoring a common policy without Rego expertise (visual condition builder) +- Retrieving a credential after resource provisioning (direct API call) +- Understanding why a request was scored a certain way (score_drivers field) +- Contributing a policy (API endpoint, not manual GitOps PR) + +--- + +### Priority 3 โ€” Extensibility and Capability Grouping + +The profile system, compliance domain overlays, policy groups, capability extensions, and registry governance make DCM adaptable to arbitrary organizational requirements without code changes. + +**What this means:** + +New compliance requirements should be expressible as policy additions within the existing framework. New provider types should fit the existing Provider base contract. New deployment contexts should be addressable through profile configuration. A platform that requires modifying source code for each new deployment context is not a platform โ€” it is a template. + +**Grouping is the mechanism for extensibility.** Compliance domain overlays compose with base profiles. Policy groups compose with profile policies. Capability extensions compose with base provider contracts. The three-abstraction model (Data, Provider, Policy) is the foundation that makes all of this compositional. + +**Extensibility must not compromise security or usability.** An extension mechanism that allows downstream users to disable security properties (rather than scale them) fails priority 1. An extension mechanism that requires expertise to configure fails priority 2. The profile system's hard constraints (SMX-008: auto_approve โ‰ค 50; CPX-001: no values in DCM stores) are precisely the boundaries that prevent extensibility from undermining security. + +--- + +### Priority 4 โ€” Fit for Purpose (Always Required) + +DCM must manage data center infrastructure lifecycle. All of the above is in service of this purpose. An architecturally beautiful system that cannot provision a VM, track its drift, and decommission it cleanly has failed at its reason for existing. + +**What this means:** + +Design decisions that serve priorities 1โ€“3 but break the end-to-end lifecycle (request โ†’ provision โ†’ operate โ†’ decommission) are not acceptable. Every capability added must have a clear answer to "how does this serve the lifecycle management mission?" + +Fit for purpose is not a fourth priority that can be traded against the first three โ€” it is a precondition. If a design cannot fulfill its stated purpose, priorities 1โ€“3 become irrelevant. This is why it is listed fourth rather than first: it is assumed, not aspirational. + +--- + +## Applying the Priorities โ€” Decision Framework + +When facing a design decision where priorities seem to conflict, apply this sequence: + +``` +1. Does this design decision compromise a non-negotiable security property? + YES โ†’ redesign until it does not. No exceptions. + +2. Does the secure option create significant operational burden? + YES โ†’ design the ease-of-use mechanism simultaneously. + The secure path must also be the easy path. + If you cannot make it easy enough, reconsider whether the + security property is correctly scoped. + +3. Can this behavior be expressed through the existing profile/policy/extension system? + YES โ†’ use it. Do not add new mechanisms when existing ones suffice. + NO โ†’ extend the existing mechanism before creating a new one. + +4. Does this design decision support the complete lifecycle? + NO โ†’ do not proceed until it does. +``` + +### Common Misapplications + +**"We can disable X in the minimal profile for simplicity."** +Wrong application. The minimal profile scales down operational burden, not security properties. The question is: what is the minimum viable implementation of X that requires no operational overhead? That is what minimal profile gets. + +**"Security is too complex for our users, so we'll make it optional."** +Wrong application. If security is too complex, the design of the security mechanism needs to improve (priority 2). Making security optional removes it โ€” that fails priority 1. Design a simpler mechanism that achieves the same security outcome. + +**"We need a new mechanism for this capability."** +Wrong starting point (priority 3 failure). The question is: can this be expressed through profiles, policies, provider capability extensions, or compliance overlays? Usually yes. If genuinely not, extend the nearest existing mechanism rather than creating a new one. + +**"This edge case isn't part of the lifecycle."** +Wrong framing (priority 4). Every edge case in the lifecycle โ€” partial realization, compensation, drift remediation, credential revocation on decommission โ€” is part of the lifecycle. Fit for purpose means handling the complete lifecycle, not just the happy path. + +--- + +## Profile Scaling Model + +The profile system is the primary mechanism for expressing priorities 1โ€“3 simultaneously. Understanding what profiles control โ€” and what they do not โ€” is essential to applying the priority order correctly. + +**Profiles control:** +- Enforcement strictness (how strictly a security property is enforced) +- Threshold values (how long, how often, how many) +- Automation level (automated vs manual trigger) +- Approval tier (auto-approve vs human review vs verified vs authorized) +- Review periods (how long shadow mode runs before promotion) + +**Profiles do not control:** +- Whether a security property is present (it always is) +- Which non-negotiable constraints apply (CPX-001, SMX-004, SMX-008, etc.) +- Whether the audit trail is maintained (always maintained; retention varies) +- Whether the data model is valid (schema conformance is not profile-dependent) + +### Profile Scaling Table + +This table shows how representative security properties scale across profiles. "Present" means the property is architecturally required โ€” what varies is the configuration. + +| Security Property | minimal | dev | standard | prod | fsi | sovereign | +|------------------|---------|-----|----------|------|-----|-----------| +| Credential rotation | Required; P365D max; manual OK | Required; P180D max; manual OK | Required; automated | Required; strict interval | Required; P90D max | Required; hardware-triggered | +| Idle detection threshold | P30D | P14D | P7D | P3D | P1D | PT12H | +| Algorithm baseline | Forbidden list | Forbidden list | Approved list | Approved list | FIPS-only | HSM-generated only | +| Shadow mode on contribution | Always on | Always on | Always on | Always on | Always on | Always on | +| First retrieval audit | Always | Always | Always | Always | Always | Always | +| Revocation cache TTL | PT5M | PT5M | PT1M | PT1M | PT30S | PT15S | +| Auto-approve threshold | โ‰ค 45 | โ‰ค 40 | โ‰ค 25 | โ‰ค 15 | โ‰ค 10 | โ‰ค 5 | +| Step-up MFA for credentials | Optional | Optional | Sensitive types | All types | Hardware MFA | mTLS | +| FIPS level | None required | None required | None required | Level 1 | Level 2 | Level 3 | +| IP binding | Not required | Not required | Not required | Not required | Required | Required | +| Hub contribution auto-approve | Yes | Yes | Yes | No (human review) | No (verified) | No (authorized) | + +--- + +--- + +## Approval Tier Model + +DCM defines four approval tiers that apply to requests, policy contributions, provider registrations, and any pipeline decision requiring human authorization. Understanding the model is critical: **DCM provides the gate and the audit trail. The review process is the organization's responsibility.** + +> **Full specification:** See [Authority Tier Model](32-authority-tier-model.md) for the complete ordered tier list, custom tier contribution model, dynamic threshold format, and ATM-001โ€“ATM-008 system policies. + +### What DCM Does vs What Organizations Provide + +| Tier | Required authority level | DCM provides | Organization provides | DCM gate condition | +|------|-------------------------|-------------|----------------------|-------------------| +| `auto` | None โ€” `decision_gravity: none`; system confidence sufficient | Structural and governance validation; automatic activation on pass | Nothing โ€” fully automated | All validation checks pass | +| `reviewed` | Standard authority โ€” `decision_gravity: routine`; one qualified reviewer in the relevant domain | Approval record; eligible reviewer notification via Notification Provider; pipeline hold; decision recording via Admin API; activation or rejection | Who constitutes a qualified reviewer; the review process; recording the decision via DCM API or an external system that calls it | One actor with reviewer role records a decision via the Admin API | +| `verified` | Elevated authority โ€” `decision_gravity: elevated`; independent confirmation required; separation of duties | Approval record requiring two independent decisions; enforces distinct actors (same actor cannot provide both); eligible reviewer notification; pipeline hold | Who constitutes qualified reviewers; both review processes; may use external workflow tools that call the DCM API | Two distinct actors with reviewer role each record a decision via the Admin API | +| `authorized` | Senior/governing authority โ€” `decision_gravity: critical`; highest organizational weight; most consequential decisions | Approval record specifying the required DCMGroup and threshold (N of M); group member notification; pipeline hold; individual decision tracking via Admin API; threshold evaluation; activation when N reached | Who constitutes the authority group (one person with delegated authority, a CTO, a CISO and legal counsel, a change board โ€” the organization decides); how they deliberate; what external tools they use; DCM records decisions, not deliberation | N members of the declared DCMGroup record decisions via the Admin API within the declared window | + +### Tier Extensibility + +The four default tiers (`auto`, `reviewed`, `verified`, `authorized`) are DCM system defaults. Organizations can add custom tiers by inserting them into the ordered list between existing tiers. The tier name is stable; numeric weight is derived from list position at evaluation time. + +Example: An organization adds `compliance_reviewed` between `verified` and `authorized`: +``` +auto โ†’ reviewed โ†’ verified โ†’ compliance_reviewed โ†’ authorized +``` +All existing references to `authorized` continue to work. Only the threshold ranges in the affected profile need updating. See [Authority Tier Model](32-authority-tier-model.md). + +### The `authorized` Tier โ€” What DCM Builds vs What It Does Not Not + +DCM does **not** build an authorized group management system. It does not track deliberation, run voting sessions, manage agendas, or coordinate review meetings. + +DCM builds: + +1. **DCMGroup membership management** โ€” which actors constitute the authorized group; configurable by platform admins +2. **Quorum declaration** โ€” `N of M` threshold declared in the profile or per-decision configuration +3. **Notification routing** โ€” when a decision enters `pending_authorized` state, the Notification Provider fires to all DCMGroup members +4. **Vote recording API** โ€” the Admin API endpoint that authorized group members (or external systems acting on their behalf) call to record `approve` or `reject` +5. **Quorum tracking** โ€” DCM counts votes and advances the pipeline when N is reached +6. **Audit trail** โ€” every vote is audited with actor UUID, timestamp, decision, and the system that recorded it + +External systems (ServiceNow, Jira, email workflows, Slack bots) connect to DCM by calling the vote recording API. A Slack bot that collects emoji reactions from group members and then calls DCM's Admin API is a valid implementation โ€” DCM doesn't care how the organization collected the vote, only that an authorized group member recorded it. + +### Admin API as the Integration Point + +The Admin API approval endpoint is designed to be called by external systems, not only by humans in a DCM UI: + +``` +POST /api/v1/admin/approvals/{approval_uuid}/vote +Authorization: Bearer # any actor who is a member of the required DCMGroup + +{ + "decision": "approve | reject", + "reason": "", + "recorded_via": "dcm_admin_ui | servicenow | jira | slack_bot | api_direct | other", + "external_reference": "" +} + +Response 200: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 2, + "quorum_required": 3, + "quorum_reached": false, + "pipeline_status": "pending_authorized" +} + +# When quorum is reached: +{ + "approval_uuid": "", + "voter_uuid": "", + "decision": "approve", + "votes_recorded": 3, + "quorum_required": 3, + "quorum_reached": true, + "pipeline_status": "activating" +} +``` + +The `recorded_via` field provides the audit trail provenance โ€” DCM knows whether the vote came through its own UI, a ServiceNow integration, a Jira plugin, or a direct API call. This is not enforced โ€” it is informational for audit purposes. + +### Deadline and Escalation + +DCM manages the approval window (the time within which a decision must be reached) and fires escalation notifications when the window is approaching: + +```yaml +approval_window: + reviewed: PT72H # 3 days; configurable per profile + verified: PT72H + authorized: P7D # 7 days for deliberation; configurable + on_expiry: + reviewed: escalate # escalate to platform admin + verified: escalate + authorized: reject # authorized tier that cannot reach threshold in window โ†’ reject +``` + +When the window expires without a decision, DCM fires an escalation notification and either rejects (for authorized) or escalates to the next approval tier (for reviewed and verified). The organization can configure these windows to match their actual governance processes. + +### DPO Alignment + +The approval tier model directly implements all four design priorities: + +1. **Security:** Approval tiers are the enforcement mechanism for governance. The gate is DCM's responsibility โ€” it cannot be bypassed, and every decision is audited. +2. **Ease of use:** The Admin API as integration point means organizations use whatever workflow tools they already have. DCM does not require them to adopt a new process tool. +3. **Extensibility:** The `recorded_via` field and `external_reference` field make the approval tier model composable with arbitrary external systems without DCM needing to integrate with each one. +4. **Fit for purpose:** The tier model enables governance of every pipeline decision (request approval, policy contribution, provider registration, federation contribution) through a single consistent mechanism. + +## Documentation Requirements + +Every document in the DCM data model should: + +1. **Reference the priority order** where design decisions are made that involve tradeoffs +2. **Explain non-negotiable security properties** with clear rationale +3. **Document what profiles control vs what they do not** for each security-relevant configuration +4. **Identify the ease-of-use mechanism** that accompanies every security requirement +5. **State fit-for-purpose scope** explicitly โ€” what lifecycle operations does this document govern? + +--- + +## System Policies + +| Policy | Rule | +|--------|------| +| `DPO-001` | Security properties are architecturally present in all profiles. Profiles control enforcement strictness, thresholds, and automation level โ€” not whether the property exists. | +| `DPO-002` | Every security requirement must be accompanied by an ease-of-use mechanism that makes compliance effortless for the common case. A security model routinely bypassed because of complexity has failed. | +| `DPO-003` | New capabilities should be expressed through the existing profile/policy/provider extension system before creating new mechanisms. Extensibility is achieved through composition, not proliferation. | +| `DPO-004` | Fit for purpose is a precondition, not a priority. All four priorities apply only within the constraint that the system can fulfill its lifecycle management mission. | +| `DPO-005` | The `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security." Design decisions that disable security properties rather than scaling them violate DPO-001. | +| `DPO-006` | When security and ease of use conflict, redesign the ease-of-use mechanism โ€” not the security requirement. The secure path must also be the easy path. | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/00-foundations.md b/content/docs/data-model/00-foundations.md new file mode 100644 index 0000000..5f75acd --- /dev/null +++ b/content/docs/data-model/00-foundations.md @@ -0,0 +1,305 @@ +# DCM โ€” Foundational Abstractions + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** โœ… Complete +**Document Type:** Architecture Foundation โ€” Read This First +**Related Documents:** [Data Model Context](00-context-and-purpose.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) + +--- + +## 1. The Three Abstractions + +DCM is built on three foundational abstractions. Every concept in the architecture is an instance of one of these three โ€” or a combination of them. There is no fourth. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ DATA โ”‚ +โ”‚ โ”‚ +โ”‚ Everything that exists, is stored, has a lifecycle, and flows โ”‚ +โ”‚ through the system. Entities, layers, policies, accreditations, โ”‚ +โ”‚ audit records, groups, relationships โ€” all Data. โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ flows through + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ–ผ โ–ผ +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ PROVIDER โ”‚ โ”‚ POLICY โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ Every external โ”‚ โ”‚ Every rule that fires on Data, โ”‚ +โ”‚ component DCM โ”‚ โ”‚ decides what happens, transforms โ”‚ +โ”‚ calls or that โ”‚ โ”‚ values, or enforces constraints. โ”‚ +โ”‚ calls DCM. โ”‚ โ”‚ โ”‚ +โ”‚ Eleven typed โ”‚ โ”‚ Seven typed output schemas. โ”‚ +โ”‚ capability โ”‚ โ”‚ One evaluation algorithm. โ”‚ +โ”‚ extensions. โ”‚ โ”‚ Same lifecycle for all. โ”‚ +โ”‚ One base contract. โ”‚ โ”‚ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +**The runtime that connects them:** + +``` +Event (Data state change) + โ†’ Policy Engine evaluates all matching Policies + โ†’ Policies produce decisions / mutations / actions + โ†’ Actions invoke Providers or produce new Data + โ†’ New Data triggers new Events + โ†’ Repeat +``` + +This is the complete DCM operational model. Everything else is a typed specialization of these three abstractions operating through this loop. + +--- + +## 2. DATA โ€” Everything That Exists + +**Definition:** Data is any structured artifact in DCM with a type, fields, classification, provenance, and lifecycle state. Data is always versioned, always identified by UUID, and always carries provenance describing where each field value came from. + +**The universal properties of all Data:** +- **UUID** โ€” every Data artifact has a universally unique identifier, stable across its full lifecycle +- **Type** โ€” every Data artifact has a declared type that determines its schema and valid field set +- **Lifecycle state** โ€” every Data artifact is in exactly one lifecycle state at any moment +- **Artifact metadata** โ€” every Data artifact carries a standard metadata block (handle, version, status, owned_by, created_by, created_via) +- **Provenance** โ€” every field in every Data artifact carries lineage metadata describing its origin and all modifications +- **Data classification** โ€” every field carries a classification (public โ†’ classified) governing what may cross interaction boundaries +- **Immutability if versioned** โ€” once a version is published, it cannot be modified; changes produce new versions +- **Contributor identity** โ€” every Data artifact records who contributed it (platform admin, consumer/tenant, service provider, or peer DCM) and what review it received before activation. DCM defaults to a federated contribution model โ€” all authorized actor types can create Data within the bounds their role permits. See [Federated Contribution Model](28-federated-contribution-model.md). + +**The complete Data taxonomy:** + +| Data Type | Description | Storage | +|-----------|-------------|---------| +| **Resource Entity** | A realized infrastructure resource; the primary managed thing | Realized Store | +| **Process Entity** | An ephemeral execution (job, playbook, pipeline) | Realized Store | +| **Composite Entity** | A Meta Provider composition of Resource Entities | Realized Store | +| **Intent State** | Consumer's raw declaration before processing | Intent Store (GitOps) | +| **Requested State** | Fully assembled, policy-validated provider payload | Requested Store | +| **Discovered State** | What actually exists per discovery observation | Discovered Store | +| **Data Layer** | A versioned artifact contributing fields to assembly | Layer Store (GitOps) | +| **Resource Type Specification** | Schema definition for a resource type | Registry | +| **Provider Catalog Item** | Provider-specific instantiation of a Resource Type Spec | Registry | +| **Policy** | A rule artifact with match conditions and output schema | Policy Store (GitOps) | +| **Policy Group** | A collection of policies grouped by concern_type | Policy Store (GitOps) | +| **Policy Profile** | A composition: one posture + zero or more compliance domains | Policy Store (GitOps) | +| **Accreditation** | A compliance certification artifact | Accreditation Store | +| **Sovereignty Zone** | A geopolitical/regulatory boundary artifact | Config Store | +| **Registration Token** | A scoped authorization artifact for provider registration | Token Store | +| **DCMGroup** | A grouping artifact (tenant_boundary, resource_grouping, etc.) | Config Store | +| **Drift Record** | A comparison result artifact | Operational Store | +| **Audit Record** | An immutable event record | Audit Store | +| **Governance Matrix Rule** | A boundary control rule artifact | Policy Store (GitOps) | +| **Orphan Candidate** | A potentially untracked resource artifact | Operational Store | + +**How Data flows โ€” the four lifecycle stages:** + +Every Resource Entity flows through four stages. These are not four separate things โ€” they are the same entity at four different lifecycle stages, stored in specialized stores optimized for each stage's access pattern: + +``` +Consumer Intent + โ”‚ raw consumer declaration + โ–ผ +Intent State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ GitOps Store + โ”‚ layer assembly + policy evaluation + โ–ผ +Requested State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Write-once Store + โ”‚ provider execution + โ–ผ +Realized State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Snapshot Store + โ”‚ independent observation + โ–ผ +Discovered State โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Ephemeral Stream +``` + +**How Data is composed โ€” the layering model:** + +Data fields are assembled from multiple contributing layers in a deterministic precedence order. See [Data Model Context](00-context-and-purpose.md) and [Layering and Versioning](03-layering-and-versioning.md) for the complete assembly algorithm. + +--- + +## 3. PROVIDER โ€” Everything External + +**Definition:** A Provider is any external component that DCM interacts with through a defined contract. Providers receive Data from DCM, act on it, and return Data to DCM. The contract governs how this exchange happens โ€” not what the Provider does internally. + +**The universal properties of all Providers:** +- **Registration** โ€” every Provider registers with DCM, declaring its capabilities, sovereignty, and accreditation +- **Health check** โ€” every Provider exposes a health endpoint; DCM monitors it continuously +- **Sovereignty declaration** โ€” every Provider declares where it operates and what jurisdictions it covers +- **Accreditation** โ€” every Provider declares its compliance certifications; DCM enforces these via the Governance Matrix +- **Governance Matrix enforcement** โ€” every interaction with a Provider is subject to the Governance Matrix before data crosses the boundary +- **Zero trust** โ€” every Provider interaction is authenticated and authorized; no implicit trust from network position +- **Lifecycle** โ€” every Provider registration goes through a defined lifecycle (SUBMITTED โ†’ VALIDATING โ†’ ACTIVE โ†’ DEREGISTERED) + +**The complete Provider taxonomy:** + +| Provider Type | Capability | Data direction | +|--------------|-----------|---------------| +| **Service Provider** | Realizes infrastructure resources | DCM โ†’ Provider โ†’ DCM | +| **Information Provider** | Serves authoritative external data | DCM queries โ†’ Provider responds | +| **Storage Provider** | Persists DCM state | DCM reads/writes โ†” Provider | +| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Children โ†’ DCM | +| **Policy Provider** | Evaluates policies externally | DCM sends payload โ†’ Provider decides | +| **Credential Provider** | Manages secrets and credentials | DCM requests โ†’ Provider issues | +| **Auth Provider** | Authenticates identities | DCM verifies โ†’ Provider confirms | +| **Notification Provider** | Delivers notifications | DCM sends envelope โ†’ Provider delivers | +| **Message Bus Provider** | Async event streaming | DCM publishes/subscribes โ†” Provider | +| **Registry Provider** | Serves the resource type registry | DCM pulls โ†’ Provider serves | +| **Peer DCM** | Another DCM instance (federation) | DCM โ†” DCM via federation tunnel | +| **ITSM Provider** | Bidirectional integration with ITSM systems (ServiceNow, Jira, Remedy, etc.); creates/updates ITSM records from DCM events; routes ITSM approvals back to DCM | DCM โ†’ ITSM (outbound) / ITSM โ†’ DCM (inbound) | + +**The unified Provider base contract** is defined in [A-provider-contract.md](A-provider-contract.md). All eleven Provider types implement this base contract. What varies is the capability declaration โ€” what operations the Provider exposes and what data flows in which direction. + +**Peer DCM as Provider:** A federated DCM instance is a typed Provider. The federation tunnel is the Provider's communication channel. Federation routing is policy-governed provider selection. There is no separate "federation abstraction" โ€” federation is the Provider abstraction applied across DCM instances. + +--- + +## 4. POLICY โ€” Everything That Decides + +**Definition:** A Policy is a rule artifact that fires when Data matches declared conditions, produces a typed output (decision, mutation, action, or directive), and is enforced according to a declared level. Policies govern every transition, transformation, and constraint in DCM. + +**The universal properties of all Policies:** +- **Match conditions** โ€” every Policy declares when it fires, using the four governance matrix axes (subject, data, target, context) or payload type + field conditions +- **Typed output schema** โ€” every Policy produces one of seven output types; the output type determines how the Policy Engine applies the result +- **Enforcement level** โ€” hard (cannot be overridden) or soft (can be tightened by more-specific policies) +- **Domain precedence** โ€” policies at more-specific domains win within their concern type; system > platform > tenant > resource_type > entity +- **Lifecycle** โ€” every Policy follows the standard artifact lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired) +- **Shadow mode** โ€” proposed Policies execute against real traffic without applying results; safe validation before activation +- **Audit** โ€” every Policy evaluation produces an audit record regardless of outcome + +**The complete Policy taxonomy:** + +| Policy Type | Fires on | Output | +|-------------|---------|--------| +| **GateKeeper** | Request payload | `allow` or `deny` with reason | +| **Validation** | Request payload | `pass` or `fail` with field-level details | +| **Transformation** | Request payload | `mutations[]` โ€” field additions, changes, locks | +| **Recovery** | Failure/timeout trigger condition | `action` + parameters (DRIFT_RECONCILE, DISCARD_AND_REQUEUE, etc.) | +| **Orchestration Flow** | Payload type events | `flow_directive` โ€” sequence ordering for pipeline steps | +| **Governance Matrix Rule** | Any cross-boundary interaction | `ALLOW / DENY / ALLOW_WITH_CONDITIONS / STRIP_FIELD / REDACT / AUDIT_ONLY` | +| **Lifecycle Policy** | Relationship events | `action` on the related entity (save, destroy, notify, cascade) | + +**The unified Policy base contract** is defined in [B-policy-contract.md](B-policy-contract.md). All seven Policy types implement this base contract. What varies is the output schema. + +**Policies as orchestration โ€” two levels that compose:** + +*Level 1 โ€” Named Workflow Artifacts (explicit, visible, auditable):* +An Orchestration Flow Policy with `concern_type: orchestration_flow` and `ordered: true` is a named workflow. It declares steps in explicit sequence. Named workflows are first-class Data artifacts โ€” versioned, GitOps-managed, profile-bound. Adding an explicit pipeline step = adding a step to a workflow Policy artifact. + +*Level 2 โ€” Dynamic Policies (conditional, inline):* +GateKeeper, Transformation, Recovery, and Governance Matrix Policies fire when their match conditions are satisfied โ€” within or alongside workflow steps, without being declared in the workflow. Adding conditional behavior = writing a dynamic policy. + +Both levels are evaluated by the same Policy Engine and triggered through the same Request Orchestrator event bus. They compose naturally: a named workflow provides the sequence skeleton; dynamic policies provide conditional behavior within it. + +**The Governance Matrix as Policy:** The Governance Matrix rules (doc 27) are typed Policies with the `boundary_control` output schema. They fire at every cross-boundary interaction. They follow the same match conditions, enforcement levels, and lifecycle as all other Policies. The governance matrix is not a separate system โ€” it is the Policy abstraction applied at interaction boundaries. + +--- + +## 5. The Runtime โ€” Connecting the Three + +The Request Orchestrator and Policy Engine are the runtime that connects the three abstractions. They are not a fourth abstraction โ€” they are the implementation machinery. + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ Request Orchestrator โ”‚ +โ”‚ (event bus โ€” not a sequencer) โ”‚ +โ”‚ โ”‚ +โ”‚ Receives events โ†’ routes to Policy Engine โ”‚ +โ”‚ Policy Engine evaluates all matching Policies โ”‚ +โ”‚ Results: invoke Providers OR produce new Data โ”‚ +โ”‚ New Data โ†’ new events โ†’ new Policy evaluations โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +**Key runtime properties:** +- The Request Orchestrator contains no pipeline logic โ€” Policies define what happens +- Every pipeline step is a Policy firing on a payload type event +- Parallel execution: Policies with no data dependencies evaluate concurrently +- Static flows: Orchestration Flow Policies with `ordered: true` +- Dynamic flows: conditional Policies that fire based on payload state + +**Control plane components as runtime specializations (nine total):** + +The components in [Control Plane Components](25-control-plane-components.md) are specialized runtime implementations, not separate abstractions: + +| Component | Abstraction it implements | +|-----------|--------------------------| +| Request Orchestrator | The runtime event bus | +| Policy Engine | The runtime Policy evaluator | +| Placement Engine | Policy evaluation specialized for provider selection | +| Cost Analysis | Information Provider (internal; data derivation) | +| Lifecycle Constraint Enforcer | Scheduled Recovery Policy trigger | +| Discovery Scheduler | Scheduled Provider invocation | +| Notification Router | Transformation Policy + Notification Provider invocation | +| Drift Reconciliation | Data comparison producing new Data (drift records) | +| Search Index | Storage Provider sub-type (queryable projection) | + +--- + +## 6. Extension Points + +DCM is designed to be extended without modifying the core. Every extension fits within the three abstractions: + +**Extending Data:** New entity types, new artifact types, new resource types, new group classes โ€” all are typed extensions of the Data abstraction. Register them in the Resource Type Registry or DCMGroup registry. + +**Extending Providers:** New provider types (a Billing Provider, a CMDB Provider, an AI/ML Provider) โ€” implement the unified Provider base contract with a new capability declaration extension. Register in the Provider Type Registry. + +**Extending Policies:** New policy types, new governance matrix rules, new orchestration flows โ€” implement the unified Policy base contract with a new output schema. Register in the Policy Store via GitOps. + +**The extension principle:** If you can express it as Data, Provider, or Policy โ€” it belongs in DCM. If you cannot express it within these three abstractions, it is either a runtime implementation detail or a genuinely novel concept that should be explicitly identified and documented as such. + +--- + +## 7. The Core Ethos + +These three abstractions serve DCM's core ethos: + +**Effective at the core mission** โ€” managing the lifecycle of infrastructure resources across a sovereign private cloud. The Data abstraction ensures every resource is tracked, versioned, and auditable. The Provider abstraction ensures every external integration is governed and trustworthy. The Policy abstraction ensures every decision is declared, reproducible, and auditable. + +**Easy to use** โ€” consumers interact with Data (submit an intent, receive a resource). Policies govern what happens without consumers needing to understand them. Providers handle the implementation details. + +**Easy to implement** โ€” implementors implement one base contract (Provider) with a typed capability extension. The Policy Engine handles all policy evaluation. The Data model handles all storage and provenance. + +**Easy to extend and integrate** โ€” add a new provider type by implementing the base contract. Add a new policy type by defining an output schema. Add a new data type by defining a schema. No core changes required. + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* + + +--- + +## Design Priority Order + +> **Full specification:** See [Design Priorities](00-design-priorities.md) for the complete priority framework, decision framework, profile scaling table, and DPO-001โ€“006 system policies. + + +Every design decision in DCM is evaluated against this priority order. When priorities conflict, higher priorities win. When there is no conflict, all four apply simultaneously. + +**1. Industry best practices for security** +Security is not a feature or a profile option. It is the baseline that every other design decision must respect. Where security and convenience conflict, security wins โ€” but the design must find a way to make the secure path the easy path. A security model that is routinely bypassed because it is too burdensome has failed at both security and usability. + +*In practice:* Security properties โ€” value separation, non-transferable credentials, scoped permissions, rotation, audit, revocation propagation โ€” are architecturally present in every profile. What profiles control is the enforcement strictness, operational automation, and threshold values. A `minimal` profile does not disable security; it implements security with minimal operational overhead. + +**2. Ease of use** +DCM exists to enable self-service for application teams. If the right path is also the hard path, teams will find other paths. The goal is to make secure, governed, auditable infrastructure management the path of least resistance โ€” not the path of compliance obligation. + +*In practice:* Profile defaults should eliminate configuration burden for common cases. The standard pipeline should auto-approve ordinary requests without human intervention. Policy authoring should not require Rego expertise for common patterns. The Flow GUI, scoring model, and contribution endpoints all serve this priority. + +**3. Extensibility and capability grouping** +The profile system, compliance domain overlays, policy groups, and registry governance exist to make DCM adaptable to arbitrary organizational requirements without code changes. This priority serves at scale โ€” a platform that can only be configured by modifying source code is not a platform. + +*In practice:* New compliance requirements should be expressible as policy additions within the existing framework. New provider types should fit the existing Provider base contract. New deployment contexts should be addressable through profile configuration. + +**4. Fit for purpose (always required)** +DCM must actually manage data center infrastructure lifecycle. All of the above is in service of this purpose โ€” not independent of it. An architecturally beautiful system that cannot provision a VM, track its drift, and decommission it cleanly has failed at its reason for existing. + +*In practice:* Design decisions that serve priorities 1โ€“3 but break the end-to-end lifecycle (request โ†’ provision โ†’ operate โ†’ decommission) are not acceptable. Every capability added must have a clear answer to "how does this serve the lifecycle management mission?" + +--- + +**The implication for profiles:** A `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security." The security architecture is present and correct in every profile. What varies is how much automation, how strict the thresholds, and how much manual intervention is acceptable. This is the principle that makes DCM trustworthy in a homelab and in a sovereign government deployment using the same codebase. + diff --git a/content/docs/data-model/01-entity-types.md b/content/docs/data-model/01-entity-types.md new file mode 100644 index 0000000..7ade494 --- /dev/null +++ b/content/docs/data-model/01-entity-types.md @@ -0,0 +1,351 @@ +# DCM Data Model โ€” Entity Types + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + +**Document Status:** โœ… Complete +**Document Type:** Architecture Reference + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” typed entity extensions (Infrastructure Resource, Composite, Process) + + +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) + +--- + +## 1. Purpose + +This document defines the complete taxonomy of entity types in DCM. Every resource, service, group, and process managed by DCM is an entity โ€” and every entity belongs to one of the types defined here. The entity type determines the lifecycle state machine, the ownership model, the decommission behavior, and which data model fields are applicable. + +Understanding entity types is prerequisite to understanding: +- How lifecycle states are assigned and transition +- How ownership and allocation interact +- How drift detection operates at different levels +- How decommission cascades through dependent entities + +--- + +## 2. The Three Primary Entity Types + +DCM defines three primary entity types. Every entity is exactly one of these. + +### 2.1 Infrastructure Resource Entity + +An **Infrastructure Resource Entity** is a realized physical or virtual infrastructure resource that persists across time and has a full operational lifecycle. + +**Characteristics:** +- Persists after provisioning โ€” it continues to exist and consume resources until explicitly decommissioned +- Owned by exactly one Tenant at any point in time +- Has a full bidirectional lifecycle including OPERATIONAL and SUSPENDED states +- Subject to drift detection โ€” its Realized State is continuously compared against Discovered State +- Subject to TTL management โ€” may declare an expiry after which decommission is triggered +- May have relationships to other entities โ€” dependencies, attachments, allocations, business data +- Carries field-level provenance across its full lifecycle + +**Lifecycle State Machine:** + +``` + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ REQUESTED โ”‚ + โ”‚ (Intent State assembled, โ”‚ + โ”‚ Requested State committed) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Provider dispatch + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ PENDING โ”‚ + โ”‚ (Awaiting provider capacity โ”‚ + โ”‚ or dependency resolution) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Provider begins work + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ PROVISIONING โ”‚ + โ”‚ (Provider actively realizing) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Provider confirms realization + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ REALIZED โ”‚ + โ”‚ (Provider-confirmed, DCM has โ”‚ + โ”‚ full Realized State record) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ Passes health checks + โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ—„โ”€โ”€ Primary operational state + โ”‚ OPERATIONAL โ”‚ Drift detection active + โ”‚ (Active, healthy, in use) โ”‚ Cost analysis active + โ””โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ Policy evaluation active + โ”‚ โ”‚ + Suspend โ”‚ โ”‚ Decommission request + request โ–ผ โ–ผ + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ SUSPENDED โ”‚ โ”‚ DECOMMISSIONING โ”‚ + โ”‚ (Paused, not in โ”‚ โ”‚ (Provider removing, โ”‚ + โ”‚ active use, โ”‚ โ”‚ dependencies โ”‚ + โ”‚ may be billed โ”‚ โ”‚ being released) โ”‚ + โ”‚ at reduced rate)โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ + โ”‚ Resume โ”‚ Provider confirms removal + โ”‚ or decommission โ–ผ + โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บ DECOMMISSIONED โ”‚ โ—„โ”€โ”€ Terminal state + โ”‚ (Removed from infra, โ”‚ + โ”‚ audit records โ”‚ + โ”‚ preserved) โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + +Any state except DECOMMISSIONED: + PROVISIONING_FAILED โ†’ rolls back to REQUESTED or terminal FAILED + PENDING_REVIEW โ†’ sovereignty/tenancy conflict during rehydration (see Section 2.1.2) +``` + +**Applicable to:** VirtualMachine, VLAN, IPAddress, StorageVolume, Container, LoadBalancer, DNSRecord, FirewallRule, NetworkPort, Subnet, and all other persistent infrastructure resource types. + +#### 2.1.1 Infrastructure Resource Entity Data Model + +```yaml +infrastructure_resource_entity: + # Universal artifact metadata + uuid: # stable across full lifecycle including rehydration + handle: # human-readable stable identifier + resource_type: # e.g., Compute.VirtualMachine + resource_type_spec_version: + lifecycle_state: + created_at: + updated_at: + + # Ownership + owned_by_tenant_uuid: # exactly one Tenant; mandatory + created_by_actor_uuid: + + # Ownership model โ€” see doc 04b + ownership_model: + # whole_allocation: consumer owns this entity outright + # allocation: this entity is an allocation carved from a pool (consumer owns it) + # shareable: consumer has a stake; ownership remains with pool owner + + # If this is an allocation from a pool resource + allocated_from_pool_uuid: # UUID of the pool entity; null if not an allocation + allocation_ref_uuid: # UUID of the AllocationRecord relationship + + # If this is a shareable stake + shared_resource_uuid: # UUID of the shared resource; null if not a stake + + # Provider details (populated after REALIZED) + provider_uuid: + provider_entity_id: # provider's own identifier (e.g., "vm-12345") + provider_entity_id_history: [...] # history of provider IDs (rehydration changes these) + + # Lifecycle constraints + ttl: + ttl_expires_at: + on_expiry: + billing_state: + + # Rehydration + rehydration_constraints: + min_auth_level: + allow_delegated_rehydration: + rehydration_history: [...] + + # Drift tracking + last_discovered_at: + drift_status: + last_drift_severity: + + # Relationships (see doc 09) + relationships: [...] + + # Field-level provenance on all data fields (see doc 00, Section 4) + # [all resource-type-specific fields carry provenance metadata] +``` + +#### 2.1.2 PENDING_REVIEW State + +`PENDING_REVIEW` is a formal lifecycle state for Infrastructure Resource Entities (not Process Resources). An entity enters `PENDING_REVIEW` when an automated operation detects a conflict that requires human resolution before the operation can proceed: + +| Trigger | Description | +|---------|-------------| +| Rehydration sovereignty conflict | Rehydration discovers the target provider no longer satisfies the entity's sovereignty constraints | +| Cross-tenant authorization revoked | An authorization enabling a cross-tenant resource reference was revoked while the resource is still allocated | +| Ownership transfer conflict | An ownership transfer request conflicts with active relationships that prevent transfer | + +An entity in `PENDING_REVIEW`: +- Is not actively drifting from its Realized State (the underlying resource is unchanged) +- Has an active `pending_review_record` on the entity with trigger, timestamp, and resolution options +- Generates notifications to the entity owner, Tenant admin, and platform admin +- Remains in `PENDING_REVIEW` until a resolution action is taken (re_authorize, release, escalate, or manual override) +- Is never automatically resolved โ€” all resolutions require explicit human or policy authorization + +### 2.2 Composite Resource Entity + +A **Composite Resource Entity** is produced by a Meta Provider that orchestrates multiple constituent Infrastructure Resource Entities to deliver a higher-order service. The composite is a first-class entity โ€” it has its own UUID, Tenant ownership, and lifecycle. Its constituents each retain their own entity identity. + +**Characteristics:** +- Represents the logical aggregate, not a physical resource +- Owned by exactly one Tenant (the Tenant that requested the composite service) +- Constituents may be owned by the same Tenant or may be allocations/stakes in pool resources owned by another Tenant +- Drift detection operates at two levels: the composite level (is the composite healthy as a whole?) and the constituent level (is each underlying resource still in its expected state?) +- Decommission is staged: composite decommissioned first, then constituents in reverse dependency order + +**Lifecycle state machine:** Same as Infrastructure Resource Entity. The composite's `lifecycle_state` reflects the aggregate health of all constituents โ€” a composite is OPERATIONAL only when all required constituents are OPERATIONAL. + +**Constituent relationship:** Each constituent is recorded as a `constituent_of` relationship from the constituent to the composite. The composite holds `has_constituent` relationships to each constituent. The composite UUID is the correlation key across all constituent audit records. + +```yaml +composite_resource_entity: + uuid: + resource_type: # e.g., ApplicationStack.WebApp + lifecycle_state: + owned_by_tenant_uuid: + composition_visibility: + # opaque: consumers see composite only; constituents hidden + # transparent: consumers see composite and all constituents + # selective: policy declares which constituents are visible + + constituents: + - constituent_entity_uuid: + role: + required_for_composite_operational: + # If a required constituent fails, the composite enters DEGRADED + constituent_lifecycle_state: + composite_health: +``` + +### 2.3 Process Resource Entity + +A **Process Resource Entity** represents an ephemeral execution โ€” an automation job, playbook, pipeline, workflow, or script execution. It does not persist after completion. Its lifecycle is terminal-focused: every Process Resource Entity ends in either COMPLETED, FAILED, or CANCELLED. + +**Characteristics:** +- Does not persist after reaching a terminal state โ€” no ongoing Realized State to manage +- Must declare `max_execution_time` โ€” mandatory, not optional +- If max_execution_time is exceeded, the process enters FAILED state and DCM generates a `PROCESS_TIMEOUT` event +- If the process modifies any Infrastructure Resource Entity, it must record the modified entity UUIDs in its provenance +- Owned by the Tenant that initiated the execution +- Subject to audit โ€” every process execution produces a full audit trail + +**Lifecycle state machine:** + +``` +REQUESTED โ†’ INITIATED โ†’ EXECUTING โ†’ COMPLETED (terminal) + โ†’ FAILED (terminal) + โ†’ CANCELLED (terminal โ€” requires explicit cancel request) +``` + +No SUSPENDED state. No PENDING_REVIEW state. Process Resources are ephemeral โ€” they do not enter states that require ongoing management. + +```yaml +process_resource_entity: + uuid: + resource_type: # e.g., Automation.AnsiblePlaybook + lifecycle_state: + owned_by_tenant_uuid: + created_by_actor_uuid: + + max_execution_time: # mandatory + started_at: + completed_at: + execution_timeout_at: # computed: started_at + max_execution_time + + # Entities this process modified (mandatory if any modifications made) + affected_entity_uuids: [, ...] + + # Execution details + provider_uuid: # which automation provider executed this + provider_job_id: # provider's own job identifier + exit_status: + execution_log_ref: # reference to log store entry + + # Provenance on all execution parameters carries field-level lineage +``` + +--- + +## 3. Sub-Types and Specializations + +### 3.1 Shared Resource Entity (Infrastructure Resource sub-type) + +A **Shared Resource Entity** is an Infrastructure Resource Entity where multiple consumers hold stakes โ€” references, attachments, or dependencies โ€” without any consumer owning an allocation of the resource. The resource has a single owner (typically a platform or network operations Tenant). Consumers reference it through relationships. + +See [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) for the complete model. + +**Examples:** VLAN, NetworkSegment, SharedStorageCluster, DNS Zone, NTP Server, Certificate Authority. + +**Key property:** `ownership_model: shareable` + +Decommission is deferred while any active stakeholder relationships exist. The `minimum_relationship_count` on the resource type spec declares the safe minimum โ€” typically 0 (can be decommissioned when all stakes are released) but may be higher for infrastructure that must always have at least one consumer. + +### 3.2 Allocatable Pool Resource (Infrastructure Resource sub-type) + +An **Allocatable Pool Resource** is an Infrastructure Resource Entity that serves as a pool from which consumers receive owned allocations. The pool itself is owned by a platform Tenant. Each allocation request produces a new, independently owned Infrastructure Resource Entity carved from the pool. + +See [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) for the complete model. + +**Examples:** IPAddressPool (allocates IPAddress entities), SubnetPool (allocates Subnet entities), VLANPool (allocates VLAN entities), StoragePool (allocates StorageVolume entities). + +**Key property:** `ownership_model: whole_allocation` on the pool entity; allocation products have `ownership_model: allocation`. + +The pool tracks available capacity. Allocation requests go through the placement engine like any other resource request. The produced allocation entity is owned by the requesting Tenant. + +--- + +## 4. Entity Identity Invariants + +These invariants apply to all entity types without exception: + +| Invariant | Rule | +|-----------|------| +| UUID stability | An entity's UUID never changes across its full lifecycle, including rehydration and provider migration | +| Single Tenant ownership | Every Infrastructure Resource Entity and Process Resource Entity is owned by exactly one Tenant at all times | +| Composite constituent ownership | A Composite Resource Entity's constituents are owned individually โ€” the composite UUID does not override constituent Tenant ownership | +| Immutable Realized State | Realized State events are append-only; a new event is created for every state change | +| Audit trail preservation | Audit records for an entity are never destroyed while any related entity is active; preservation policy governs post-terminal retention | +| Provider ID separation | The entity UUID is the DCM stable identity; the provider entity ID is the provider's own reference. These are separate and the provider ID may change on rehydration | + +--- + +## 5. Entity Type to Resource Type Mapping + +Not all resource types produce the same entity type. The entity type is declared in the Resource Type Specification: + +```yaml +resource_type_spec: + fqn: Compute.VirtualMachine + entity_type: infrastructure_resource # infrastructure_resource | composite_resource | process_resource + ownership_model: whole_allocation # whole_allocation | allocation | shareable + allocatable_from_pool_type: null # if allocation: the pool resource type this comes from + pool_resource_type: null # if pool: declare this is a pool resource + shareable: false # if shareable: true +``` + +--- + +## 6. Related Policies + +| Policy | Rule | +|--------|------| +| `ENT-001` | Every Infrastructure Resource Entity must be owned by exactly one Tenant at all times | +| `ENT-002` | Process Resource Entities must declare max_execution_time โ€” this field has no default and is not optional | +| `ENT-003` | Process Resource Entities must record all affected entity UUIDs if any infrastructure modifications are made during execution | +| `ENT-004` | Composite Resource Entity lifecycle_state reflects aggregate constituent health โ€” OPERATIONAL only when all required constituents are OPERATIONAL | +| `ENT-005` | PENDING_REVIEW is a valid Infrastructure Resource Entity state requiring human resolution โ€” it is never an error state and never automatically resolved | +| `ENT-006` | The entity UUID is immutable across the full entity lifecycle including rehydration, provider migration, and ownership transfer | + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/02-four-states.md b/content/docs/data-model/02-four-states.md new file mode 100644 index 0000000..5cabbdc --- /dev/null +++ b/content/docs/data-model/02-four-states.md @@ -0,0 +1,862 @@ +# DCM Data Model โ€” The Four States + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** โœ… Complete +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” four lifecycle stages and their storage models + + + +--- + +## 1. Purpose + +The four states are the foundational model for how DCM tracks the complete lifecycle of any resource or service. Every entity in DCM exists in one or more of these states simultaneously. The states are not sequential stages โ€” they are parallel, independently maintained records that together provide a complete, auditable picture of what was requested, what was approved, what was built, and what actually exists. + +The four states answer four distinct questions: + +| State | Question Answered | Store Type | +|-------|------------------|------------| +| **Intent State** | What did the consumer ask for? | GitOps Store (required) | +| **Requested State** | What was approved and dispatched to the provider? | Write-once Storage Provider (GitOps reference impl) | +| **Realized State** | What did the provider actually build? | Write-once Snapshot Store (authorized changes only) | +| **Discovered State** | What does DCM observe actually existing right now? | Ephemeral Snapshot Store | + +--- + +## 2. State Definitions + +### 2.1 Intent State + +The **Intent State** is the immutable record of a consumer's original declaration. It is captured at the moment a request is submitted โ€” before any layer assembly, before any policy evaluation, before any provider selection. + +**Characteristics:** +- Immutable once created โ€” the consumer's original intent is never modified +- Stored in a GitOps store โ€” branched, reviewed, merged +- The CI/CD pipeline operates on the Intent State โ€” policy pre-validation, cost estimation, sovereignty check, approval workflow +- Versioned via Git history โ€” every revision of an intent is traceable +- Supports human review and debate via the PR mechanism +- The entity UUID is assigned at Intent State creation โ€” it follows the entity through all subsequent states + +**When created:** Every request submission, every rehydration operation, every drift remediation authorization + +**Content:** The consumer's raw declaration in DCM Unified Data Model format โ€” what they want, not what will be built + +### 2.2 Requested State + +The **Requested State** is the fully assembled, policy-processed, provider-ready payload. It is produced by the Request Payload Processor from the Intent State โ€” after layer assembly, after all policy evaluation, after provider selection. + +**Characteristics:** +- Immutable once created โ€” a new Requested State is created for each request cycle +- Stored in a GitOps store โ€” committed, versioned, triggering CD pipeline +- The CD pipeline dispatches from the Requested State to the provider +- Contains the complete assembled payload with full field-level provenance +- Contains the results of all policy evaluations โ€” which policies ran, what they did, what they locked +- Contains provider selection โ€” which provider will realize this request +- Is the authoritative record of what DCM instructed a provider to build + +**When created:** After Intent State approval (merge), after successful policy processing + +**Content:** The complete assembled payload in DCM Unified Data Model format, with full provenance chain, policy evaluation results, provider selection, and override control metadata + +### 2.3 Realized State + +The **Realized State** is the provider-confirmed record of what was actually built. It is produced by the provider after successful realization โ€” the denaturalized result of the provider's execution, translated back to DCM Unified Data Model format. + +**Characteristics:** +- Write-once complete snapshots โ€” each Realized State record is a full entity state, never modified after writing +- Every Realized State record is traceable to exactly one Requested State record โ€” no exceptions +- Stored in a Write-once Snapshot Store keyed by entity UUID +- Contains provider-specific details not in the Requested State โ€” assigned IPs, generated passwords, actual storage sizes, provider-internal IDs +- Is the authoritative record of what actually exists from DCM's perspective +- Drift is detected by comparing the most recent Realized State snapshot against Discovered State +- Carries a supersession chain โ€” each snapshot knows which snapshot it superseded and which superseded it + +**Three write sources (all require a corresponding Requested State record):** + +| Source | Requested State record type | Example | +|--------|---------------------------|---------| +| Initial realization | `initial_realization` | Consumer provisions a new VM | +| Consumer update request | `consumer_update` | Consumer patches an editable field | +| Provider update notification | `provider_update` | Provider reports an authorized state change (auto-healing, maintenance) | + +**What does NOT write to the Realized Store:** +- Drift detection โ€” drift only compares, never writes +- Discovery cycles โ€” discovery writes to Discovered Store only +- Unsanctioned provider changes โ€” these are drift events until DCM evaluates and explicitly approves them + +**When created:** When a provider confirms realization of any authorized request (initial, consumer update, or approved provider update notification) + +**Content:** Complete entity state snapshot in DCM Unified Data Model format, with provider-added fields, full field-level provenance including provider attribution, and supersession chain references + +### 2.4 Discovered State + +The **Discovered State** is what DCM observes actually existing through active discovery โ€” polling providers, querying Kubernetes APIs, interrogating infrastructure. It is the ground truth of what physically exists, independent of what DCM thinks exists. + +**Characteristics:** +- Append-only snapshot stream โ€” each discovery cycle produces a new snapshot +- Stored in an Event Stream Store (ephemeral) โ€” recent history retained, older snapshots archived or discarded +- High-frequency and machine-generated โ€” not appropriate for human review +- Used exclusively for drift detection โ€” comparing against Realized State +- May contain resources DCM did not provision โ€” brownfield resources discovered for ingestion + +**When created:** On every discovery cycle, on demand for specific entities + +**Content:** Raw discovered resource state in DCM Unified Data Model format, with discovery metadata (timestamp, discovery method, provider interrogated) + + +### 2.5 Recovery States + +Five additional states apply to Infrastructure Resource Entities when the normal provisioning lifecycle encounters timeouts, cancellation failures, or partial realization. These states are governed by Recovery Policies (see [Operational Models](24-operational-models.md) Section 5). + +| State | Meaning | Entry Trigger | +|-------|---------|--------------| +| `TIMEOUT_PENDING` | Dispatch timeout fired; cancellation sent to provider | `DISPATCH_TIMEOUT` recovery trigger | +| `LATE_REALIZATION_PENDING` | Provider responded after timeout; NOTIFY_AND_WAIT active | `LATE_RESPONSE_RECEIVED` recovery trigger | +| `INDETERMINATE_REALIZATION` | State ambiguous; drift detection resolving | `DRIFT_RECONCILE` recovery action | +| `COMPENSATION_IN_PROGRESS` | Compound service rollback underway | `PARTIAL_REALIZATION` trigger | +| `COMPENSATION_FAILED` | Rollback itself failed; orphaned resources possible | Compensation step failure | + +See [Operational Models](24-operational-models.md) for the complete recovery state machine and Recovery Policy model. + + +--- + +## 3. The Entity UUID โ€” Universal Linking Key + +Every entity has a single UUID assigned at Intent State creation. This UUID is the universal key linking the entity across all four states and all stores: + +``` +Intent Store: file path includes entity_uuid, content declares entity_uuid +Requested Store: file path includes entity_uuid, content declares entity_uuid +Realized Store: event stream keyed by entity_uuid +Discovered Store: snapshot stream keyed by entity_uuid (matched via provider labels) +Audit Store: all provenance events indexed by entity_uuid +Search Index: entity_uuid โ†’ git_path mapping for Git stores +``` + +Given an entity UUID, DCM can reconstruct the complete history of that entity across its entire lifecycle โ€” from the consumer's original intent through every state transition to the current discovered state. + +--- + +## 4. Physical Representation โ€” Storage Provider Model + +DCM describes store **contracts**, not implementations. Each store is a Storage Provider โ€” a formal DCM provider type with registration, health check, and trust obligations. Implementors choose the technology that satisfies the contract. + +See [Storage Providers](11-storage-providers.md) for the complete contract specifications. + +### 4.1 GitOps Stores (Intent and Requested) + +**Contract characteristics:** +- Branch-per-request โ€” each request is a branch in the store +- Pull Request semantics โ€” review, comment, approve, merge +- Immutable history โ€” commits are permanent records +- CI/CD hook support โ€” commits trigger pipeline execution +- Indexed for query โ€” a Search Index projection enables field-based queries at scale +- Entity UUID โ†’ file path mapping maintained in the Search Index + +**Intent Store โ€” GitOps implementation required:** +The PR workflow, branch-per-request semantics, and human review flow are first-class features of the Intent Store โ€” not implementation details. GitOps is the only supported implementation for the Intent Store. + +**Requested Store โ€” write-once Storage Provider (GitOps is the reference implementation):** +The Requested Store requires write-once semantics and hash-chain integrity but does not require GitOps PR mechanics โ€” the Requested State is machine-generated output, not consumer input. The GitOps implementation is the reference implementation and suitable for small-to-medium deployments. For production scale (thousands of requests per day), a purpose-built write-once document store is explicitly supported. + +The distinction matters for the following reasons: +- Git performance degrades at scale (large repo size, high-frequency machine writes) +- PR semantics (branch creation, merge, CI hooks) add latency and overhead with no workflow benefit for machine-generated content +- Sensitive assembled payload data may warrant stricter field-level access control than Git provides +- A write-once document store with hash-chain integrity satisfies all Requested Store contracts without Git's operational constraints + +**Typical implementations (Intent Store):** GitHub, GitLab, Gitea, Forgejo + +**Typical implementations (Requested Store):** GitHub/GitLab/Gitea (reference implementation); PostgreSQL with write-once enforcement + Merkle hash chain (production scale); CockroachDB (geo-distributed) + +**Repository structure:** Resolved. See [Worked Examples](04-examples.md) Section 2 for the complete Git directory layout. Provider selection is recorded in the assembled payload (placement.yaml), not in the directory structure โ€” directories are independent of provider selection (Q54 resolved). + +### 4.2 Write-once Snapshot Store (Realized State) + +The Realized Store uses a **write-once snapshot model** โ€” each Realized State record is a complete entity state snapshot, not a field-level event. This model aligns with the constraint that Realized State only changes via authorized requests. + +**Contract characteristics:** +- Write-once โ€” each snapshot record is immutable after creation; updates create new snapshot records +- Complete snapshots โ€” each record captures the full entity state, not a delta from previous state +- Entity-UUID-keyed โ€” O(1) lookup of all snapshots for a given entity +- Supersession chain โ€” each record references the snapshot it superseded and the Requested State record that authorized the change +- Queryable by timestamp โ€” point-in-time state reconstruction is a direct lookup, not a replay +- Traceable โ€” every record has a non-nullable `corresponding_requested_state_uuid` field + +**Realized State snapshot record structure:** + +```yaml +realized_state_snapshot: + realized_state_uuid: # this snapshot's identity + entity_uuid: # entity this belongs to + realized_at: + + # Always traceable to a request โ€” mandatory, not nullable + source_type: + corresponding_requested_state_uuid: + + # Supersession chain + supersedes_realized_state_uuid: # null for first realization + superseded_by_realized_state_uuid: # null for current record + + # Complete entity state at this point โ€” all fields, all provenance + fields: + # [full entity state in DCM Unified Data Model format] + + # Provider-added fields + provider_entity_id: + provider_reported_at: +``` + +**Why snapshots instead of events:** +Rehydration from Realized State requires a complete entity state โ€” not a replay of field-level events. A snapshot model makes rehydration a direct lookup rather than an event replay. Point-in-time queries ("what was the Realized State on March 15?") are also direct lookups. The Realized Store does not need the high-frequency write throughput of an event stream โ€” it is written only when an authorized change completes. + +**Typical implementations:** PostgreSQL with write-once constraints; CockroachDB; etcd (small deployments) + +### 4.3 Ephemeral Snapshot Store (Discovered State) + +The Discovered Store retains its event stream model โ€” discovery is high-frequency, machine-generated, and ephemeral. It is never a rehydration source by definition (discovered state was never authorized through DCM). + +**Contract characteristics:** +- Append-only snapshot stream โ€” each discovery cycle produces a new snapshot +- Entity-UUID-keyed streams โ€” each entity has its own snapshot stream +- Replayable โ€” for trending and historical discovery analysis +- High throughput โ€” designed for machine-generated, high-frequency writes +- Ephemeral โ€” retention policy governs how long snapshots are kept (see RHY-008) + +**Typical implementations:** Kafka with log compaction, EventStoreDB, Apache Pulsar + +### 4.3 Search Index (Git Store Projection) + +**Contract characteristics:** +- Derived from Git stores โ€” rebuilt from Git history on demand +- Explicitly non-authoritative โ€” Git always wins if index and Git disagree +- Queryable by indexed fields: entity_uuid, tenant_uuid, resource_type, lifecycle_state, timestamp, cost_center, business_unit, provider_uuid +- Lightweight โ€” stores indexed fields only, not full payloads +- Fast โ€” designed for millisecond query response at millions of records + +**Typical implementations:** Elasticsearch, OpenSearch, Meilisearch + +--- + +## 5. Rehydration + +Rehydration is the process of using a previously stored state record as the starting point for a new request. It is not a shortcut around governance โ€” **all relevant governance policies always apply regardless of rehydration source.** Rehydration is a new request that happens to start from a known prior state. + +### 5.1 Three Rehydration Sources + +**From Intent State:** +- The consumer's original declaration is replayed +- Full layer assembly runs โ€” current layers applied +- All governance policies run โ€” current policies applied +- Provider selection runs fresh +- Most likely to produce a different result than the original โ€” policies and layers may have changed +- Use cases: upgrade resource to current standards, apply new sovereignty constraints, environment refresh + +**From Requested State:** +- The previously assembled, policy-processed payload is loaded +- Layer assembly is skipped โ€” layers were already applied +- All governance policies run โ€” current policies applied +- Provider selection: configurable via flag (see Section 5.3) +- Use cases: reproduce a resource as closely as possible to the approved specification + +**From Realized State:** +- The provider-confirmed realized payload is loaded +- Provider-specific fields are stripped โ€” DCM unified format only +- Layer assembly is skipped +- All governance policies run โ€” current policies applied +- Provider selection: configurable via flag +- Use cases: exact reproduction for disaster recovery, environment cloning, replacing a failed resource + +### 5.2 The Common Governance Pipeline + +Regardless of rehydration source, all requests flow through the same governance pipeline: + +``` +Rehydration source selected and loaded + โ”‚ + โ”‚ Source payload becomes the basis for a new Intent State record + โ”‚ New entity UUID assigned (or existing UUID preserved โ€” policy decision) + โ”‚ Rehydration provenance recorded: source_store, source_record_uuid, + โ”‚ rehydration_reason, requested_by_uuid, rehydration_timestamp + โ–ผ +If source = Intent: + โ”‚ Full layer assembly runs (Steps 1-7) + โ”‚ Current layers applied + โ–ผ +If source = Requested or Realized: + โ”‚ Layer assembly skipped + โ”‚ Payload loaded as pre-assembled + โ”‚ If source = Realized: provider-specific fields stripped + โ–ผ +Placement evaluation + โ”‚ See Section 5.3 โ€” configurable + โ–ผ +Policy Engine โ€” ALL governance policies applied + โ”‚ Authorization policies: does this actor have permission to rehydrate? + โ”‚ Transformation policies: current enrichment applied + โ”‚ Validation policies: current constraints checked + โ”‚ GateKeeper policies: current field locks applied + โ”‚ Gatekeeping policies: is this resource type still permitted? + โ”‚ + โ”‚ Governance is NEVER skippable โ€” not for any rehydration source, + โ”‚ not for any actor, not for any urgency claim + โ–ผ +New Requested State produced and stored + โ”‚ New record โ€” never overwrites the source record + โ”‚ Source record remains immutable + โ”‚ Provenance chain links to source record + โ–ผ +Provider dispatch + โ”‚ Dispatched to selected provider + โ–ผ +New Realized State events produced + โ”‚ New event stream or continuation of existing stream + โ”‚ Provenance links to rehydration Requested State +``` + +### 5.3 Placement Flag โ€” Provider-Portable Rehydration + +When rehydrating from Requested State or Realized State, provider selection is configurable via an explicit flag in the rehydration request: + +```yaml +rehydration_request: + uuid: + source_store: + source_record_uuid: + + placement: + re_evaluate: false + # false (default): honor provider selection from source record + # Use when: original provider is available and appropriate + # Result: resource reproduced on same provider + # + # true: strip provider selection, run placement policies fresh + # Use when: original provider unavailable, decommissioned, + # at capacity, or no longer sovereign-compliant + # Result: placement policies select provider from current landscape + # Named concept: Provider-Portable Rehydration + + placement_constraints: + # Optional โ€” additional constraints for re-evaluation + # Only applicable when re_evaluate: true + exclude_provider_uuids: [, ...] + require_region: + require_sovereignty_capability: + + governance: + apply_all_policies: true + # Always true โ€” governance is never skippable + # Included explicitly for auditability โ€” the rehydration record + # must declare that governance was applied + + policy_version: current + # current (default): apply today's policies + # pinned: apply policies as of a specific timestamp + # Use when: exact historical reproduction required + # (audit evidence, regulatory examination, environment reconstruction) + # Requires elevated authorization โ€” bypasses current GateKeeper policies + # Only SRE and Admin actors may use pinned policy version + + pinned_timestamp: + # Required when policy_version: pinned + + rehydration_reason: + requested_by_uuid: +``` + +### 5.4 The Four Rehydration Modes + +Two independent axes โ€” placement and policy version โ€” produce four distinct rehydration configurations: + +| Mode | re_evaluate | policy_version | Use Case | +|------|-------------|----------------|----------| +| **Faithful** | false | current | Same provider, current governance | +| **Provider-Portable** | true | current | New provider, current governance | +| **Historical Exact** | false | pinned | Same provider, historical governance (audit evidence) | +| **Historical Portable** | true | pinned | New provider, historical governance | + +Historical modes require elevated authorization. All modes run governance โ€” the difference is whether governance uses current or pinned policies. + +### 5.5 Rehydration Tenancy and Sovereignty Controls + +**Tenancy controls, sovereignty directives, and cross-tenant authorizations are always evaluated against current policies during rehydration โ€” they cannot be pinned to historical versions.** + +The `policy_version: pinned` setting governs resource configuration policies only. It does not apply to: +- Tenancy boundary enforcement +- Sovereignty constraints +- Cross-tenant authorization requirements + +```yaml +rehydration: + policy_version: pinned # governs resource configuration policies + # The following ALWAYS use current policies โ€” cannot be pinned: + tenancy_controls: always_current + sovereignty_controls: always_current + cross_tenant_authorizations: always_current +``` + +**When rehydration conflicts with current tenancy controls:** + +If the current policy environment produces a tenancy or sovereignty constraint that conflicts with a cross-tenant allocation valid at original request time โ€” for example, the consuming Tenant's authorization was revoked since the original request โ€” the rehydration is **paused**, not failed or silently bypassed: + +``` +Rehydration detects cross-tenant authorization conflict + โ”‚ + โ–ผ +Entity enters PENDING_REVIEW state + โ”‚ Allocation is not automatically released + โ”‚ Rehydration_tenancy_conflict_record created + โ–ผ +Notifications dispatched: + โ”‚ entity owner, owning Tenant admin, + โ”‚ consuming Tenant admin, platform admin + โ–ผ +Resolution options: + re_authorize โ†’ issue new cross_tenant_authorization for this allocation + release โ†’ release the allocation, entity decommissioned + escalate โ†’ refer to platform admin for manual decision + โ”‚ + โ””โ”€โ”€ A policy may declare automatic resolution: + "on rehydration conflict โ†’ re_authorize if consuming Tenant + still meets sovereignty requirements" +``` + +**System policies for rehydration tenancy:** + +| Policy | Rule | +|--------|------| +| `RHY-001` | Tenancy, sovereignty, and cross-tenant authorizations always use current policies during rehydration โ€” cannot be pinned | +| `RHY-002` | Rehydration that conflicts with current tenancy/sovereignty pauses and enters PENDING_REVIEW | +| `RHY-003` | A paused rehydration allocation is not automatically released โ€” requires explicit resolution | +| `RHY-004` | A policy may declare automatic resolution behavior for rehydration tenancy conflicts | + +### 5.6 Partial Resolution of Q54 โ€” Provider Selection + +The placement flag model clarifies the Q54 question (selected_provider as policy output vs placement component). The emerging answer: + +**Policies set placement constraints โ€” the placement component selects the provider.** + +A GateKeeper policy may output: "must be in region EU-WEST, must support sovereignty capability PCI-DSS." The placement component reads these constraints and selects the specific provider within those constraints. The policy does not name the provider. The placement component names the provider. + +This is consistent with the portability model โ€” a policy that names a specific provider would be portability-breaking. Policies set constraints. Placement honors constraints and selects. + +--- + +## 6. Drift Detection + +Drift is the difference between what DCM believes exists (Realized State) and what actually exists (Discovered State). + +### 6.1 Drift Detection Flow + +``` +Discovery cycle completes + โ”‚ Provider interrogated โ†’ Discovered State snapshot written + โ–ผ +Drift Detection component + โ”‚ Loads latest Discovered State for entity UUID + โ”‚ Loads latest Realized State events for entity UUID + โ”‚ Field-by-field comparison + โ–ผ +No drift detected + โ”‚ Discovery timestamp updated + โ”‚ No action + โ–ผ +Drift detected + โ”‚ Drift record created with: + โ”‚ - entity_uuid + โ”‚ - drifted_fields: [{field_path, realized_value, discovered_value}] + โ”‚ - discovery_timestamp + โ”‚ - drift_severity: + โ–ผ +Policy Engine evaluates drift + โ”‚ Drift response policy determines action: + โ”‚ REVERT: submit a rehydration request from Realized State to restore + โ”‚ UPDATE_DEFINITION: promote discovered state to new Realized State + โ”‚ ALERT: notify personas, no automatic action + โ”‚ ESCALATE: trigger human review workflow + โ”‚ + โ”‚ Response determined by drift severity, resource type, + โ”‚ resource ownership, and organizational policy + โ–ผ +Audit Store records drift event with full provenance +``` + + +### 6.3 Drift Severity Classification + +Drift severity is determined by combining three independent tiers. The final severity is the highest tier that applies. + +**Tier 1 โ€” Field criticality (declared in Resource Type Specification):** + +```yaml +resource_type_spec: + fields: + display_name: + drift_criticality: minor # non-functional change + cpu_count: + drift_criticality: significant + memory_gb: + drift_criticality: significant + security_group_ids: + drift_criticality: critical # security-relevant change + firewall_rules: + drift_criticality: critical +``` + +**Tier 2 โ€” Profile/layer magnitude thresholds:** + +```yaml +# system/drift/severity-thresholds layer (overridable at platform/tenant domain) +drift_severity_thresholds: + significant_field_magnitude_upgrade: + percentage_change_threshold: 50 # >50% change upgrades significant โ†’ critical + minor_field_magnitude_upgrade: + item_count_threshold: 10 # 10+ changed items upgrades minor โ†’ significant +``` + +**Tier 3 โ€” Provider and consumer injection:** + +Providers may suggest severity in update notifications (raise only): +```yaml +provider_drift_hint: + field: memory_gb + suggested_severity: critical + reason: "Memory decrease on running workload risks OOM" +``` + +Consumers may override sensitivity on specific entities (raise or lower): +```yaml +entity: + drift_sensitivity_overrides: + - field: cpu_count + override_criticality: critical + reason: "Production payments workload โ€” any CPU change is critical" +``` + +**Resolution rule:** The Drift Detection component takes the highest severity from all three tiers. Provider injection can raise but not lower the Tier 1/2 result. Consumer injection can raise or lower (entity owner controls their own resource's sensitivity). Profile governs whether consumer lowering is permitted. + + + +### 6.3 Drift Severity Classification + +Drift severity is determined by two independent dimensions declared in the Resource Type Specification โ€” field criticality and change magnitude. The combination produces a deterministic severity classification for any drift event. + +#### Field Criticality (declared per field in Resource Type Spec) + +```yaml +resource_type_spec: + fqn: Compute.VirtualMachine + fields: + display_name: + drift_criticality: low # cosmetic; never affects function + cpu_count: + drift_criticality: medium # affects performance; not security + memory_gb: + drift_criticality: medium + security_group_ids: + drift_criticality: critical # security boundary field; always critical + os_image: + drift_criticality: critical # security posture; always critical + storage_gb: + drift_criticality: medium + network_interface_ids: + drift_criticality: high # connectivity; significant operational impact +``` + +**Criticality levels:** `low | medium | high | critical` + +#### Change Magnitude (profile-governed thresholds) + +```yaml +drift_magnitude_thresholds: + profile_defaults: + standard: + minor: change_pct < 10% + significant: change_pct 10-50% + critical: change_pct > 50% OR value_disappeared OR type_changed + prod: + minor: change_pct < 5% + significant: change_pct 5-25% + critical: change_pct > 25% OR value_disappeared OR type_changed +``` + +#### Severity Matrix + +| Field Criticality | Change Magnitude | Drift Severity | +|------------------|-----------------|----------------| +| low | any | minor | +| medium | minor | minor | +| medium | significant | significant | +| medium | critical | significant | +| high | minor | significant | +| high | significant | significant | +| high | critical | critical | +| critical | any | critical | + +**Unsanctioned changes** (no corresponding Requested State record) are always elevated one severity level above what the matrix produces. A `significant` unsanctioned change becomes `critical`. + +**Multi-field drift:** when multiple fields drift simultaneously, the overall severity is the highest severity among all drifted fields. + + +### 6.2 Unsanctioned Changes + +A specific category of drift โ€” a change made directly to a resource without a corresponding DCM request. Detected by: +- Kubernetes: CR spec change without DCM request annotation +- VMware/OpenStack: resource modification not traceable to a DCM Requested State record +- General: any Discovered State field value that differs from Realized State without a Requested State record explaining the change + +Unsanctioned changes are always reported to the Policy Engine as `UNSANCTIONED_CHANGE` events. Policy determines the response. + +--- + +## 7. CI/CD Integration + +The GitOps stores are the natural integration point for CI/CD pipelines. DCM does not prescribe a specific CI/CD tool โ€” the GitOps store contract requires hook support, and the CI/CD tool is a deployment choice. + +### 7.1 CI Pipeline (Intent State) + +Triggered on: branch creation or update (new or revised intent) + +``` +CI pipeline executes: + 1. Policy pre-validation (dry run โ€” no state changes) + โ†’ Reports: which policies would apply, what they would do + 2. Cost estimation + โ†’ Reports: estimated cost for lifecycle of this resource + 3. Dependency graph validation + โ†’ Reports: all required dependent resources, any conflicts + 4. Sovereignty constraint check + โ†’ Reports: which sovereignty constraints apply, any violations + 5. Authorization check + โ†’ Reports: does this actor have permission to request this resource type? + 6. Auto-approve evaluation + โ†’ Reports: can this be merged automatically, or does it require human review? + +All results posted as PR comments on the Intent State branch +Consumer and approvers can review and debate before merge +``` + +### 7.2 CD Pipeline (Requested State) + +Triggered on: Intent State merge (PR merged to main) + +``` +CD pipeline executes: + 1. Request Payload Processor assembles full payload + 2. Full policy evaluation (binding โ€” not dry run) + 3. Provider selection (or re-evaluation if placement flag set) + 4. Requested State committed to Git store + 5. Provider dispatch via API Gateway + 6. Status monitoring โ€” poll or receive callbacks until terminal state + 7. Status written back to PR or status file + 8. Consumer notification +``` + +### 7.3 The Third Rail โ€” Direct API Ingress + +Not all requests come through the GitOps PR workflow. Some requests come through direct API submission โ€” automated systems, CI/CD pipelines, Terraform providers, programmatic consumers. These bypass the human review workflow but not governance. + +Direct API ingress: +- Creates an Intent State record (the submitted payload becomes the intent) +- Runs the same CI validation pipeline but non-interactively +- If auto-approve policy permits: proceeds directly to assembly and dispatch +- If human review required: creates a PR for review before proceeding +- Same governance pipeline regardless of ingress path + +The three ingress paths โ€” PR workflow, direct API, and programmatic (Terraform/Ansible) โ€” all converge on the same governance pipeline. The ingress path affects the review workflow; it never affects governance. + +--- + +## 7a. Four States Operational Gaps โ€” Q75 through Q78 + +### 7a.1 Entity UUID Preservation on Rehydration (Q75) + +Entity UUIDs are **preserved on rehydration**. The UUID represents the stable logical identity of the resource across provider migrations, sovereignty changes, and lifecycle events. All external references โ€” CMDB records, cost attribution, audit trails, cross-tenant relationships, dependency declarations โ€” reference the entity by UUID. Generating a new UUID on rehydration would silently break all of those references. + +What changes on rehydration is the **provider-side identifier** โ€” the actual VM ID, container name, or resource handle at the provider. These are recorded in the rehydration history: + +```yaml +entity: + uuid: # PRESERVED across all rehydrations + rehydration_history: + - rehydration_uuid: + rehydrated_at: + trigger: + from_provider_uuid: + to_provider_uuid: + from_realized_entity_id: "vm-12345" # provider's ID โ€” no longer valid + to_realized_entity_id: "vm-67890" # new provider's ID after rehydration + rehydrated_by: + intent_state_ref: + previous_requested_state_ref: + new_requested_state_ref: +``` + +**Rehydration is transactional:** If the target provider cannot accept the entity (capacity unavailable, sovereignty mismatch discovered mid-rehydration), the original entity remains in its current state with no UUID change and no partial state. Failure preserves the pre-rehydration state completely. + +### 7a.2 Pinned Authentication Level for Rehydration (Q76) + +Entities may declare a minimum authentication level required to rehydrate them. This prevents escalation of privilege through the rehydration mechanism โ€” a resource provisioned with hardware-token MFA authorization should not be re-instantiatable by a simple API key. + +```yaml +entity: + rehydration_constraints: + min_auth_level: hardware_token_mfa + # Ascending levels: api_key | ldap_password | oidc | oidc_mfa | + # hardware_token | hardware_token_mfa + auth_level_source: + allow_delegated_rehydration: false + # true = DCM service accounts may rehydrate if explicitly authorized +``` + +**Profile-governed enforcement:** + +| Profile | Enforcement | +|---------|------------| +| `minimal` | Not enforced โ€” any auth level may rehydrate | +| `dev` | Not enforced | +| `standard` | Advisory โ€” warn if rehydrating actor has lower auth | +| `prod` | Enforced โ€” reject if rehydrating actor has lower auth | +| `fsi` | Enforced โ€” dual approval required if auth level mismatch | +| `sovereign` | Enforced โ€” dual approval always; logged in classified audit | + +**Automated rehydration:** When DCM triggers rehydration automatically (sovereignty violation, provider decommission), the rehydration uses DCM's internal service account. This requires `allow_delegated_rehydration: true` OR a platform admin must manually authorize the operation. Authorization produces an audit record preserving accountability even when the action is automated. + +### 7a.3 Concurrent Rehydration Handling (Q77) + +Rehydration requests acquire an **exclusive rehydration lease** per entity. Only one rehydration may be active per entity at any time. + +```yaml +rehydration_lease: + entity_uuid: + lease_uuid: + acquired_by: + acquired_at: + lease_ttl: PT2H # expires after 2 hours if not released + trigger: + status: +``` + +**Concurrent request handling:** + +``` +Second rehydration attempt arrives for entity + โ”‚ + โ”œโ”€โ”€ No active lease โ†’ acquire lease; proceed + โ”‚ + โ””โ”€โ”€ Active lease exists: + Priority higher than active โ†’ escalate to platform admin; queue + Same or lower priority โ†’ reject: + "Rehydration in progress โ€” lease held since ; retry after PT2H" + REHYDRATION_BLOCKED audit event recorded +``` + +**Priority ordering:** +1. Security/compliance emergency (sovereignty violation at fsi/sovereign) +2. Manual platform admin rehydration +3. Automated sovereignty migration +4. Provider decommission migration +5. Manual consumer rehydration request + +**Lease TTL expiry:** If rehydration hangs or crashes, the lease expires after TTL. DCM marks the rehydration `failed` in rehydration_history, releases the lease, and triggers drift detection to assess partial completion at the provider. + +### 7a.4 Discovered State Retention (Q78) + +Discovered State is ephemeral operational data โ€” not the authoritative source of truth (Realized State is). It is a snapshot used for drift detection. Three retention modes, all profile-governed: + +```yaml +discovered_state_retention: + mode: # hybrid recommended + + rolling_window: + retention: P7D # keep last 7 days; useful for trending + + event_driven: + retain_until: drift_resolved # keep until associated drift record resolved + # Ensures drift investigation has the discovery snapshot that triggered it + + hybrid: # recommended โ€” combines both + minimum_retention: P24H + retain_until: drift_resolved # extend beyond minimum until drift resolved + maximum_retention: P30D # hard ceiling regardless of drift status +``` + +**Profile-governed defaults:** + +| Profile | Mode | Min Retention | Max Retention | +|---------|------|--------------|--------------| +| `minimal` | `rolling_window` | โ€” | P3D | +| `dev` | `rolling_window` | โ€” | P7D | +| `standard` | `hybrid` | P24H | P30D | +| `prod` | `hybrid` | P48H | P30D | +| `fsi` | `hybrid` | P7D | P90D | +| `sovereign` | `hybrid` | P7D | P90D | + +**Discovered State and the Audit Store:** + +Discovered State records are **NOT** stored in the Audit Store โ€” they are too high-volume and too ephemeral for compliance-grade storage. However, drift events triggered by Discovered State ARE recorded in the Audit Store with a reference to the discovery snapshot UUID. After the Discovered State expires, the audit record still exists โ€” it cannot link to the full snapshot, but the drift event itself is preserved. + +--- + +## 7b. Rehydration System Policies โ€” Complete Set + +| Policy | Rule | +|--------|------| +| `RHY-001` | Tenancy and sovereignty are always current on rehydration โ€” they cannot be pinned to historical state. | +| `RHY-002` | Sovereignty conflicts discovered during rehydration place the entity in PENDING_REVIEW state. | +| `RHY-003` | Resource allocations are not automatically released on rehydration. | +| `RHY-004` | Rehydration leases have TTL to prevent orphaned lease states. | +| `RHY-005` | Entity UUIDs are preserved on rehydration. The UUID represents stable logical identity across provider migrations. Provider-side identifiers change on rehydration and are recorded in rehydration_history. Rehydration is transactional โ€” failure preserves pre-rehydration state without UUID change. | +| `RHY-006` | Entities may declare min_auth_level for rehydration. Profile governs enforcement. Automated rehydration by DCM service accounts requires allow_delegated_rehydration: true OR platform admin manual authorization with full audit trail. | +| `RHY-007` | Rehydration requests acquire an exclusive lease per entity before proceeding. Only one rehydration may be active per entity. Concurrent requests are queued (higher priority) or rejected (same/lower). Lease TTL prevents indefinite blocking. Expiry triggers drift detection for partial completion assessment. | +| `RHY-008` | Discovered State retention is profile-governed: rolling_window, event_driven, or hybrid. Discovered State is never stored in the Audit Store. Drift events triggered by Discovered State are recorded in the Audit Store with discovery snapshot UUID reference. Maximum retention: P30D for standard/prod; P90D for fsi/sovereign. | + +--- + +## 8. Open Questions + +| # | Question | Impact | Status | +|---|----------|--------|--------| +| 1 | Git repository structure for Intent and Requested stores | Store design | โœ… Resolved โ€” handle-based directory structure; 4 repos; tenant isolation (STO-005) | +| 2 | Should the entity UUID be preserved or regenerated on rehydration? | Entity identity | โœ… Resolved โ€” UUID preserved; rehydration_history records provider-side ID changes; transactional (RHY-005) | +| 3 | For pinned policy version rehydration โ€” what is the minimum authorization level required? | Security | โœ… Resolved โ€” min_auth_level on entity; profile-governed enforcement; delegated rehydration requires explicit authorization (RHY-006) | +| 4 | How are concurrent rehydration requests for the same entity handled? | Concurrency | โœ… Resolved โ€” exclusive rehydration lease; priority ordering; TTL expiry triggers drift detection (RHY-007) | +| 5 | Should the Discovered Store retain full history or only a configurable window? | Retention | โœ… Resolved โ€” hybrid mode recommended; profile-governed min/max; event-driven until drift resolved; max P30-90D (RHY-008) | +| 6 | How does the Search Index handle Git store unavailability? | Reliability | โœ… Resolved โ€” serve degraded (warn + direct to authoritative); rebuild on recovery (STO-002) | + +--- + +## 9. Related Concepts + +- **Storage Provider** โ€” the formal provider type for all DCM stores +- **Entity UUID** โ€” the universal linking key across all four states +- **Rehydration** โ€” using a prior state record as the starting point for a new request +- **Provider-Portable Rehydration** โ€” rehydration with provider selection re-evaluated +- **Drift Detection** โ€” comparing Realized State against Discovered State +- **Unsanctioned Change** โ€” a resource modification not traceable to a DCM request +- **CI/CD Integration** โ€” GitOps stores as the natural CI/CD integration point +- **Search Index** โ€” queryable projection of GitOps stores, explicitly non-authoritative + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/content/docs/data-model/03-layering-and-versioning.md b/content/docs/data-model/03-layering-and-versioning.md new file mode 100644 index 0000000..36ac3e3 --- /dev/null +++ b/content/docs/data-model/03-layering-and-versioning.md @@ -0,0 +1,2019 @@ +# DCM Data Model โ€” Data Layers and the Assembly Process + +> **โš ๏ธ Active Development Notice** +> +> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made. +> +> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project). + + +**Document Status:** โœ… Complete +**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) + +> **Foundation Document Reference** +> +> This document is a detailed reference for a specific domain of the DCM architecture. +> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in +> [00-foundations.md](00-foundations.md). All concepts in this document map to one or +> more of those three abstractions. +> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) +> +> **This document maps to: DATA** +> +> The Data abstraction โ€” how Data is assembled from layers + + + +--- + +## 1. Purpose + +Data Layers are the mechanism by which DCM assembles a complete, contextually correct request payload from a set of composable, reusable data definitions. Rather than requiring consumers to specify every field of every resource they request, layers allow standards, organizational context, service-specific configuration, and consumer intent to be declared independently and merged into a unified payload at request time. + +Layers are the answer to the question: **how does a single consumer request become a complete, policy-validated, provider-ready payload?** + +The layering model enables: +- **Reuse** โ€” a base configuration defined once is inherited by thousands of resources +- **Standardization** โ€” organizational standards are encoded in layers, not in every individual request +- **Separation of concerns** โ€” infrastructure teams own core and service layers; consumers own request layers; policy owners own policy layers +- **Scale** โ€” 36 layer definitions can govern 40,000 VMs without duplication +- **Auditability** โ€” every field in the merged payload knows which layer set it and why + +--- + +## 1a. Layers vs Policies โ€” The Clear Distinction + +Layers and policies are the two foundational mechanisms of DCM's assembly process. They are complementary and distinct โ€” understanding the difference is critical to using DCM correctly. + +### Layers Are Data + +A layer is a **declarative, immutable, versioned unit of data**. It carries static configuration values, organizational defaults, compliance metadata, and contextual information. A layer answers the question: **"what values should these fields have?"** + +Layers are **passive** โ€” they declare values but do not execute logic. They do not evaluate the payload, make branching decisions, or enforce rules. The assembly process merges them in priority order. Layers come first. + +**What belongs in a layer:** +- Infrastructure defaults (DNS servers, NTP servers, MTU values) +- Organizational context (data center location, rack assignment, environment tier) +- Service-specific configuration defaults (VM sizing defaults, storage class preferences) +- Compliance metadata (data classification labels, retention tags, jurisdiction markers) +- Provider-specific configuration (provider default settings, tooling parameters) +- Business context (cost center defaults, environment labels, team tags) + +### Policies Are Logic + +A policy is an **executable rule** that evaluates the assembled payload and takes action. A policy answers the question: **"given this data, is it valid? what should change? should this proceed?"** + +Policies **execute** โ€” they run logic (OPA Rego, DCM native rules, Mode 4 black box calls). They can read every layer-provided value, validate correctness, transform fields, inject derived values, and gate requests. Policies come after layers โ€” they operate on the assembled result. + +**What belongs in a policy:** +- Validation rules ("this field must be present and within these bounds") +- Compliance enforcement ("all resources must have a classification label") +- Derived value injection ("inject cost center from OIDC claims") +- Placement constraints ("must be in EU sovereignty zone") +- Approval gates ("resources above X size require manager approval") +- Security enforcement ("encryption must be enabled โ€” if not, enable it or reject") + +### The Flow Is Strictly Unidirectional + +``` +Steps 1-4: LAYERS assembled โ†’ merged payload produced + โ”‚ Layers contribute field values + โ”‚ Higher priority layers override lower priority + โ”‚ Immutable fields locked at this stage + โ”‚ + โ–ผ +Steps 5-9: POLICIES execute โ†’ payload evaluated and acted upon + โ”‚ Policies read assembled payload + โ”‚ Transformation: modify/inject derived fields + โ”‚ Validation: verify correctness + โ”‚ GateKeeper: approve or reject + โ”‚ + โ–ผ +Provider-ready payload dispatched +``` + +Policies cannot set static configuration โ€” that is a layer's job. A policy that finds itself repeatedly injecting the same static value into every request should be refactored: that value belongs in a layer. + +Layers cannot enforce rules โ€” that is a policy's job. A layer that contains conditional logic or rule evaluation is being misused โ€” that logic belongs in a policy. + +### The Decision Rule for Practitioners + +> "Is this a **value** that should appear in the payload? โ†’ **Layer** +> Is this a **rule** about whether the payload is correct? โ†’ **Policy** +> Is this a **value derived by evaluating** the payload? โ†’ **Policy** (Transformation type)" + +### The Analogy + +- Layers are the **ingredients** โ€” pre-measured, pre-arranged, versioned +- Policies are the **chef** โ€” decides what to do with the ingredients, can add derived elements, makes judgment calls, can reject the dish entirely + +Both are necessary. Neither replaces the other. + +--- + +## 2. What is a Layer? + +A Layer is a **declarative, immutable, versioned unit of data** that contributes some or all of its fields to a merged payload. Layers do not execute โ€” they declare. The assembly process is what merges them. + +Every layer: +- Has a **UUID** that uniquely identifies it +- Has a **version** following the universal Major.Minor.Revision scheme +- Is **immutable once published** โ€” changes produce a new version +- Carries a **reference to its parent entity** (UUID and version) +- Has an **origination timestamp** +- Can be **deprecated** following the universal deprecation model +- Contributes **provenance metadata** for every field it sets โ€” any field set by a layer records that layer's UUID as its source + +Layers are stored in Git following GitOps practices. They are the configuration source of truth โ€” not the assembled payload. + +--- + +## 3a. Provenance Model Configuration + +### 3a.1 The Three Provenance Models + +Field-level provenance tracks which layer set each field, which policy modified it, and the full change history. DCM supports three configurable models โ€” organizations choose based on their scale, compliance requirements, and operational preferences. The active Profile provides a recommended default via its activated Policy Group. + +**Model A โ€” Full Inline** +All provenance stored explicitly on every entity record. Every field carries its complete provenance inline: source layer, modifying policies, previous values, timestamps, actor chain. + +| Aspect | Detail | +|--------|--------| +| Storage cost | Very high โ€” scales with entities ร— fields ร— changes | +| Query simplicity | Highest โ€” all provenance in one record, no traversal | +| Write performance | Lowest โ€” every field change requires provenance write | +| Audit clarity | Highest โ€” regulators see everything in one record | +| Tooling required | Minimal | +| Best for | Small deployments; FSI/sovereign (regulatory clarity); home lab | + +**Model B โ€” Deduplicated (Content-Addressed) โ† RECOMMENDED** +Classical content-addressed deduplication applied to provenance. The layer chain is the deduplication key โ€” every entity sharing the same configuration references the same chain rather than storing a copy. Only fields deviating from the chain store unique delta records. + +``` +Full provenance = layer chain content (deduplicated, shared) + entity deltas (unique per entity) +``` + +**Why lossless:** Layer chains are immutable. A reference to `layer-chain-abc123` always resolves to exactly the same data โ€” no cache invalidation, no drift. This is what makes the deduplication lossless for audit. The reference always reconstructs the original. + +**Storage reduction:** 95-99% for standardized deployments (many entities, few unique chains). 36 layer definitions serving 40,000 VMs produces 36 chain references, not 8 million field provenance entries. + +| Aspect | Detail | +|--------|--------| +| Storage cost | Low โ€” scales with unique configurations, not entity count | +| Query simplicity | Medium โ€” chain traversal required for layer-set fields | +| Write performance | Highest โ€” only deltas write; chain-matching fields are free | +| Audit clarity | Complete โ€” full reconstruction always possible | +| Tooling required | Moderate โ€” chain traversal tooling | +| Best for | Standard and prod deployments; large-scale environments | + +**Analogous to:** Git content-addressed objects, Docker image layers, ZFS block deduplication โ€” all content-addressed, deduplicated, lossless. + +**Model C โ€” Tiered Archive** +Hot/warm/cold storage tiers with decreasing detail. Recent provenance at full detail and fast access; older provenance compressed to change events; oldest compressed to hash anchors only (tamper-evidence without full reconstruction). + +| Aspect | Detail | +|--------|--------| +| Storage cost | Medium โ€” time-dependent, degrades gracefully | +| Query simplicity | Medium โ€” cross-tier joins for long time ranges | +| Write performance | Medium | +| Audit clarity | Full detail in hot tier; change events in warm; anchors in cold | +| Tooling required | Moderate โ€” tier promotion jobs, consistency checks | +| Best for | Large deployments with long retention requirements | + +**Models B and C are orthogonal** โ€” combine them for maximum efficiency: deduplicate at the entity level (Model B) AND tier the storage of chains and deltas (Model C). This is the highest-efficiency option for very large-scale deployments with long retention requirements. + +### 3a.2 Configurable Provenance Model + +The provenance model is declared in the DCM deployment configuration and activated via a Policy Group: + +```yaml +provenance_config: + model: + + # Model A โ€” Full Inline + full_inline: + include_previous_values: true + include_actor_chain: true + include_policy_rationale: true + + # Model B โ€” Deduplicated (Content-Addressed) + layer_chain_ref: + store_layer_derivable: false # do not store fields matching chain default + delta_detail_level: + history_document_retention: P7Y + chain_store_retention: P7Y # chains retained while any entity references them + + # Model C โ€” Tiered Archive + tiered: + hot_tier_duration: P30D # full detail, fast access + warm_tier_duration: P365D # change events only + cold_tier_duration: P10Y # hash anchors only + warm_tier_detail: + + # Model B + C โ€” Deduplicated + Tiered (maximum efficiency) + layer_chain_ref_tiered: + chain_store_hot: P365D # chains fast for 1 year + chain_store_warm: P7Y # chains slower for 7 years + delta_store_hot: P90D # deltas fast for 90 days + delta_store_warm: P7Y # deltas slower for 7 years +``` + +### 3a.3 Profile-Appropriate Provenance Policy Groups + +DCM ships four provenance Policy Groups. The active Profile activates the appropriate group by default. Organizations override by swapping the active group. + +| Group Handle | Model | Profile Default | Concern Type | +|-------------|-------|----------------|-------------| +| `system/group/provenance-full-inline` | A โ€” Full Inline | minimal, dev, fsi, sovereign | implementation_posture | +| `system/group/provenance-deduplicated` | B โ€” Deduplicated | standard, prod | implementation_posture | +| `system/group/provenance-tiered-archive` | C โ€” Tiered | (available โ€” not default) | implementation_posture | +| `system/group/provenance-deduplicated-tiered` | B+C โ€” Combined | (available for large-scale) | implementation_posture | + +**To change provenance model:** +```yaml +# Override profile default โ€” swap the active provenance group +tenant_config: + policy_group_overrides: + replace: + - from: system/group/provenance-full-inline + to: system/group/provenance-deduplicated + reason: "Deploying at scale โ€” switching to deduplicated model" +``` + +### 3a.4 The Audit Completeness Guarantee + +Regardless of provenance model, full provenance must always be reconstructable: + +``` +OPS-002 Regardless of provenance model, full provenance must always be + reconstructable for any entity from the combination of: entity + record, layer chain store, and Audit Store. The provenance model + governs where data is stored and how it is accessed โ€” not whether + it is available. +``` + +For Model B: chain reference + entity deltas โ†’ full provenance (lossless, immutable source) +For Model C: hot tier (full) OR warm tier (events) + cold tier (anchors prove integrity) +For Model A: entity record alone is sufficient + +### 3a.5 System Policies + +| Policy | Rule | +|--------|------| +| `OPS-001` | Field-level provenance model is configurable: full_inline, layer_chain_ref (deduplicated), tiered, or layer_chain_ref_tiered. Profile activates the appropriate Policy Group as default. Organizations override by replacing the active provenance group. Model B (layer_chain_ref) is the recommended default for standard+ profiles. | +| `OPS-002` | Regardless of provenance model, full provenance must always be reconstructable from the combination of entity record, layer chain store, and Audit Store. The provenance model governs storage location and access pattern โ€” not data availability. | + +--- + + +### 2a. Layer Contributors + +Every layer type has a declared contributor type. The contributor determines what review is required before the layer becomes active in assembly. See [Federated Contribution Model](28-federated-contribution-model.md) Section 3 for the full contributor permission table. + +| Layer Type | Contributor | Domain | Review | +|-----------|-------------|--------|--------| +| Base Layer | Platform Admin | system | auto | +| Core Layer | Platform Admin | platform | auto | +| Intermediate / Customization Layer | Platform Admin, Consumer/Tenant | platform, tenant | per profile | +| Service Layer | Platform Admin, Service Provider | provider | reviewed (standard+) | +| Request Layer | Consumer/Tenant | tenant | auto (applied directly to request) | +| Policy Layer | All contributor types | per contributor role | per profile + contributor type | + +The Request Layer is the only layer type that does not require a PR review โ€” it is a consumer's direct field declarations on a specific request. All other layers flow through the GitOps PR model. + + +## 3. Layer Types + +DCM defines six layer types. Each has a distinct purpose, scope, ownership model, and position in the assembly precedence chain. + +### 3.1 Base Layer + +**Purpose:** The foundation entity for a resource. Defines the minimum required fields and their default values for a given resource context. Everything starts with a Base Layer. + +**Scope:** Can be type-agnostic (a universal base) or type-scoped (a base specific to a Resource Type). A Base Layer that is type-scoped must declare its Resource Type. + +**Ownership:** DCM platform or platform implementor. + +**Characteristics:** +- Every layer chain must begin with a Base Layer +- Base Layers contain only universal fields โ€” no provider-specific data +- A Base Layer for a typed resource must conform to the Resource Type Specification's universal field requirements +- Multiple Base Layers can exist for the same context โ€” the applicable one is selected based on the request context + +**Examples:** +- CIS Benchmark base configuration +- Baseline OS configuration +- DMZ network base configuration + +--- + +### 3.2 Core Layers + +**Purpose:** Provide data that is applicable across any resource type. Core Layers carry organizational, infrastructure, and contextual data that is not specific to any one service. + +**Scope:** Type-agnostic by default. Core Layers apply to all resource types unless explicitly scoped. This is the primary distinction from Service Layers. + +**Ownership:** DCM platform, infrastructure teams, or platform implementors. + +**Characteristics:** +- Applied to every request regardless of resource type +- Cannot contain service-specific or provider-specific data +- Carry location, organizational, and infrastructure context +- Stored in the Core Layer Store +- Cached in the Service Layer Cache at deployment time + +**Examples:** +- Data Center layer (DC1, DC2) +- Zone layer (Zone 1, Zone 2) +- Rack layer +- Geographic region layer +- Environment layer (production, staging, development) + +--- + +### 3.3 Intermediate / Customization Layers + +**Purpose:** Provide organizational or contextual overrides and customizations that sit between the base standards and the service-specific configuration. These layers encode the organizational hierarchy and deployment context. + +**Scope:** Can be type-agnostic or type-scoped. Scope is declared per layer. + +**Ownership:** Organizational teams, domain owners, platform implementors. + +**Characteristics:** +- Stack between Core Layers and Service Layers in the precedence chain +- Encode organizational structure (business unit, enclave, logical unit) +- Allow organizational customization without modifying base standards +- The Git repo hierarchy typically mirrors the intermediate layer hierarchy + +**Examples:** +- Ship layer (in Navy context: specific vessel configuration) +- Enclave layer (isolated network segment configuration) +- Business unit layer +- DMZ customization layer +- Production web tier layer + +--- + +### 3.4 Service Layers + +**Purpose:** Provide service-specific data required to build a complete request payload for a specific Resource Type. Service Layers are the bridge between general organizational context and provider-ready configuration. + +**Scope:** **Must be type-scoped.** A Service Layer without a declared Resource Type scope is invalid. The scope inheritance behavior is configurable per Service Layer declaration. + +**Type Scope Declaration:** +```yaml +type_scope: + resource_type_uuid: + resource_type_fully_qualified_name: + scope_inheritance: + # exact: applies only to the declared Resource Type + # descendants: applies to the declared Resource Type and all child types via inheritance +``` + +**Ownership:** Service Providers or service domain teams. Stored in Service Layer SCM (source control management). Registered with DCM as part of Service Provider registration. + +**Characteristics:** +- Only applied when the request resource type matches the layer's declared type scope +- Carry service-specific configuration, defaults, and constraints +- Must not contain provider-specific data unless marked as portability-breaking +- Cached in the Service Layer Cache at Service Provider registration time + +**Examples:** +- VM sizing layer (small, medium, large configurations for `Compute.VirtualMachine`) +- Web server configuration layer for `Compute.VirtualMachine` +- Network port configuration layer for `Network.Port` +- CL Web Service Data Layer for `Compute.VirtualMachine` (exact scope) +- General compute placement layer for `Compute.VirtualMachine` and descendants + +--- + +### 3.5 Request Layer + +**Purpose:** Carries the consumer's declared intent. The Request Layer is what the consumer provides โ€” the fields they explicitly specify for their resource request. + +**Scope:** Scoped to the Resource Type the consumer is requesting. + +**Ownership:** Consumer (via Web UI or Consumer API). + +**Characteristics:** +- Created at the time the consumer submits a request +- Contains only what the consumer explicitly declares โ€” it does not need to be complete +- The gap between what the consumer declares and what the provider needs is filled by the lower layers in the chain +- Has higher precedence than all data layers below it โ€” consumer-declared values override layer defaults +- Is the direct source of the **Intent State** โ€” the Request Layer as submitted by the consumer is stored in the Intent Store before any processing occurs +- After assembly and policy processing, the enriched payload becomes the **Requested State** + +**Examples:** +- Consumer requests a VM with `cpu_count: 8`, `ram_gb: 32`, `os: RHEL9`, `environment: production` +- Consumer requests a firewall rule with source/target network and port + +--- + +### 3.6 Policy Layers + +**Purpose:** Policy Layers are not data layers in the traditional sense โ€” they do not add fields to the merge chain. Instead, they operate on the assembled payload after the data layers have been merged. They are the governance layer of the assembly process. + +**Scope:** Scoped by policy type and domain. Core Policies apply to all requests. Service Policies apply to specific Resource Types. Organizational and domain policies apply to specific organizational scopes. + +**Ownership:** Policy creators, security teams, compliance teams, organizational domain owners. + +**Policy Layer Types and Their Behavior:** + +| Policy Type | Behavior | Precedence Effect | +|-------------|----------|-------------------| +| **Validation** | Checks data against rules. Does not modify data. Returns pass/fail. If fail, request is rejected. | No precedence โ€” pass/fail only | +| **Transformation** | Enriches or modifies data in the payload. Adds missing fields, applies standards, fills gaps. | Adds to or modifies the assembled payload โ€” recorded in provenance | +| **GateKeeper** | Highest authority. Can override any field regardless of what was declared in lower layers or the Request Layer. Can halt execution entirely. Used for sovereignty constraints, security mandates, and hard compliance rules. | Overrides everything โ€” including consumer input | + +**Characteristics:** +- Policies operate only on the policy definition, core data, and the data in the request payload +- Policy outcomes are deterministic โ€” same input always produces same output for a given policy version +- All policy modifications are recorded in field-level provenance with policy UUID, operation type, and reason +- Policies are versioned using the universal versioning scheme +- Policies are maintained via GitOps practices + +--- + +## 4. Layer Identity โ€” Domain, Handle, and Priority + +Every layer has a formal identity model with three components that together make it uniquely identifiable, locatable, and orderable within DCM. + +### 4.1 Layer Domain + +The **Layer Domain** mirrors the Policy domain model exactly. It declares ownership, storage location, and authorization scope. The same domain hierarchy, the same authority model, the same override precedence. + +| Domain | Meaning | Authorization | Can Override | +|--------|---------|--------------|-------------| +| `system` | DCM built-in layers โ€” ship with DCM | DCM maintainers only | Nothing above system | +| `platform` | Platform team layers โ€” apply across all Tenants | Platform team | tenant, service, provider | +| `tenant` | Tenant-specific layers โ€” scoped to one Tenant | Tenant Admin | service, provider within Tenant | +| `service` | Service Provider contributed layers | Service Provider owner | provider | +| `provider` | Provider Catalog Item layers | Provider owner | Nothing above provider | +| `request` | Consumer-declared values in the request itself | Consumer | Nothing above request โ€” lowest authority | + +A lower-domain layer cannot override a higher-domain layer. A `tenant` layer cannot override a `platform` layer. This is enforced at ingestion โ€” the conflict detection pipeline checks domain authority before allowing a merge. + +**Domain mirrors policy authority:** Just as system-domain policies have highest authority in the Policy Engine, system-domain layers have highest authority in the assembly process. The same mental model applies to both. + +### 4.2 Layer Groups โ€” DCMGroup with group_class: layer_grouping + +Just as Policy Groups organize policies into cohesive concern-based collections, **Layer Groups** organize layers. A Layer Group is a `DCMGroup` with `group_class: layer_grouping` โ€” a versioned, audited, GitOps-managed collection of related layers. + +Layer Groups enable: +- **Discovery** โ€” "show me all layers related to PCI compliance" +- **Composition** โ€” include a group in a profile rather than listing individual layers +- **Governance** โ€” activate or deactivate a concern's worth of layers in one operation + +```yaml +# A Layer Group โ€” DCMGroup with group_class: layer_grouping +dcm_group: + artifact_metadata: + uuid: + handle: "platform/layer-groups/pci-network-standards" + version: "1.0.0" + status: active + group_class: layer_grouping + concern_tags: [pci-dss, networking, standards] + members: + - member_uuid: + member_type: layer + member_role: network_segmentation_defaults + - member_uuid: + member_type: layer + member_role: firewall_baseline + - member_uuid: + member_type: layer + member_role: tls_minimum_version +``` + +### 4.3 The Full Layer Structure + +Every layer carries: identity, domain and authority, compatibility metadata, per-field override metadata, and usage context. This mirrors the richness of a Policy registration. + +```yaml +layer: + artifact_metadata: + uuid: + handle: "platform/core/default-dns-config" + version: "1.2.0" + status: active + owned_by: + display_name: "Platform Infrastructure Team" + notification_endpoint: + created_via: pr # pr | api | migration | system + + # DOMAIN AND AUTHORITY + domain: platform + priority: + value: "500.20.0" + label: "platform.networking.dns" + category: platform + rationale: "Platform DNS infrastructure โ€” primary and secondary resolvers" + + # CONCERN TAGS โ€” for discoverability and grouping + concern_tags: [networking, dns, platform-defaults] + + # COMPATIBILITY METADATA โ€” what this layer applies to + compatibility: + resource_types: [Compute.VirtualMachine, Compute.Container] + resource_type_versions: "^1.0.0" + provider_types: [] # empty = all providers + profile_constraints: [] # empty = all profiles; or: [standard, prod, fsi] + domains_applicable: [platform, tenant, service, provider] # which domains may use this + + # CONDITIONAL INCLUSION (Q23) โ€” activation condition + activation_condition: + # Layer only included if this condition evaluates true during Step 2 (Layer Resolution) + field: tenant.tags + operator: not_contains # equals|not_equals|exists|not_exists|contains|in|not_in + value: custom-dns + # Compound conditions: + # conditions: + # operator: and # and | or + # rules: + # - field: request.gpu_requested + # operator: equals + # value: true + # - field: ingress.actor.roles + # operator: contains + # value: developer + + # FIELDS โ€” with per-field override metadata + fields: + dns_servers: + value: [10.0.0.53, 10.0.0.54] + metadata: + override: allow # allow | constrained | immutable + basis_for_value: "Platform DNS infrastructure โ€” primary and secondary" + provenance_note: "Set by platform infrastructure team per INFRA-2024-089" + dns_search_domain: + value: corp.example.com + metadata: + override: immutable # lower layers cannot override this field + locked_by_policy_uuid: + basis_for_value: "Corporate domain โ€” cannot be customized per SECURITY-2024-034" + + # USAGE CONTEXT โ€” human documentation embedded in the artifact + usage: + description: "Default DNS configuration for all platform VMs and containers" + applies_when: "All requests unless consumer declares layer exclusion or tenant has custom-dns tag" + excludes_when: "Tenant has custom_dns tag; consumer declares explicit layer exclusion" + supersedes: [] # handles of layers this replaces + conflicts_with: [] # handles of layers this conflicts with โ€” detected at ingestion + + # SOURCE OF TRUTH + scm_location: + repository: https://git.corp.example.com/dcm-layers + path: platform/core/default-dns-config/v1.2.0.yaml + commit: +``` + +### 4.4 Layer Handle + +The **Layer Handle** is the human-readable, stable identifier for a layer within DCM. + +**Format:** `{domain}/{concern_or_type}/{name}` + +**Examples:** +``` +platform/core/cis-benchmark-linux +platform/core/security-cpu-limits +tenant/service/payments-vm-standards +system/base/universal-defaults +service/service/kubevirt-vm-defaults +provider/service/cloudnativepg-database-config +``` + +**Git path from handle:** +``` +{layer_store_root}/{domain}/{concern_or_type}/{name}/v{Major}.{Minor}.{Revision}.yaml + +# Example: +dcm-layers/platform/core/security-cpu-limits/v1.2.0.yaml +dcm-layers/tenant/{tenant-uuid}/service/payments-vm-standards/v1.0.0.yaml +``` + +### 4.5 Priority Schema + +The **Priority Schema** is the deterministic ordering mechanism for resolving conflicts between layers of the same type and scope. + +**Format:** `{integer}.{integer}.{integer}...` โ€” unlimited depth + +**Comparison:** Left-to-right, segment by segment. **Higher numeric value = higher priority** (higher value = higher priority). No ceiling โ€” you can always go higher. + +``` +900.10 beats 800.10 (900 > 800 at segment 1) +900.20 beats 900.10 (20 > 10 at segment 2) +900.10.5 beats 900.10 (longer path with matching prefix) +900.10.10 beats 900.10.5 (10 > 5 at segment 3) +``` + +**Reference Priority Taxonomy (advisory โ€” not enforced by DCM):** + +| Suggested Range | Category | Rationale | +|-----------------|----------|-----------| +| `900.*` | Compliance | Regulatory mandates โ€” highest authority | +| `800.*` | Security | Security standards | +| `700.*` | Sovereignty | Data residency constraints | +| `600.*` | Operations | SRE and operational standards | +| `500.*` | Platform | Platform-level defaults | +| `400.*` | Service | Service-specific configuration | +| `300.*` | Organization | Organizational defaults | +| `200.*` | Site | Location-specific overrides | +| `100.*` | Custom | Implementor-defined โ€” lowest standard category | + +Higher number = higher priority. Organizations adopt, adapt, or ignore this taxonomy โ€” DCM resolves conflicts purely by numeric comparison. + +--- + +## 4b. Artifact Metadata Standard + +Every DCM artifact โ€” layers, policies, resource types, catalog items, provider registrations, entity definitions, and all other defined or stored objects โ€” carries a standard **Artifact Metadata** block. This is a structural requirement, not optional. + +The artifact metadata block answers: **who created this, when, who owns it, what changed, and how do we contact them?** + +### 4b.1 Universal Artifact Metadata Structure + +```yaml +artifact_metadata: + + # Identity + uuid: + handle: + + # Versioning + version: + status: + + # Status detail โ€” populated per status + status_detail: + # When status: proposed + proposed_at: + proposed_by: + uuid: + display_name: + email: + shadow_execution: + enabled: + started_at: + evaluation_count: + validation_dashboard_url: + + # When status: deprecated + deprecated_at: + deprecated_by: + uuid: + display_name: + replacement_uuid: + replacement_handle: + deprecation_reason: + migration_guidance: + sunset_date: + + # When status: retired + retired_at: + retired_by: + uuid: + display_name: + + # Origination + created_by: + uuid: + display_name: + email: + notification_endpoint: + created_at: + created_via: + # pr: submitted via GitOps PR workflow โ€” full review history available + # api: submitted via direct API + # migration: imported from external system โ€” provenance depth may be limited + # system: created by DCM itself (entity stubs, system artifacts) + + # Ownership โ€” may differ from creator + owned_by: + uuid: + display_name: + email: + notification_endpoint: + # Note: created_by is the audit record (who physically submitted it) + # owned_by is the accountability record (who is responsible and gets notified) + + # Modification history โ€” append-only + modifications: + - sequence: 1 + modified_by: + uuid: + display_name: + email: + modified_at: + modification_type: + version_before: + version_after: + change_summary: + pr_reference: + reason: +``` + +### 4b.2 The Five Artifact Statuses + +| Status | Meaning | Executes? | Output Applied? | Output Captured? | Merges to Active? | +|--------|---------|-----------|----------------|-----------------|------------------| +| `developing` | In active development. Development mode / dev pipeline only. | Dev mode only | No | Dev logs only | No โ€” must transition to proposed first | +| `proposed` | Development complete. Submitted for validation. Shadow mode for policies. | Yes (shadow) | No | Yes โ€” validation report | Yes โ€” after review approval | +| `active` | Live and governing. Applied to all relevant requests. | Yes | Yes | Yes โ€” audit/provenance | N/A | +| `deprecated` | Being phased out. Replacement available. Works but warns. | Yes | Yes | Yes โ€” with deprecation warning | N/A | +| `retired` | End of life. Cannot be used. | No | No | No | No | + +**Status transition rules:** +``` +developing โ†’ proposed (author submits for review) +developing โ†’ retired (author abandons without proposing) +proposed โ†’ active (reviewers approve โ€” via PR merge or API approval) +proposed โ†’ developing (returned for rework) +active โ†’ deprecated (replacement available โ€” sunset date declared) +deprecated โ†’ retired (sunset date reached or manual retirement) +retired โ†’ (terminal โ€” no transitions out) +``` + +### 4b.3 Proposed Status โ€” Shadow Execution for Policies + +When a policy artifact is in `proposed` status, it runs in **shadow mode** against real request traffic: + +- Executes alongside active policies on every relevant request +- Output is captured in a `proposed_evaluation_record` โ€” what it would have done +- Output is **never applied** to the actual request +- Shadow output feeds the Validation Dashboard for reviewer analysis +- Policy authors can see aggregate impact before activation + +```yaml +# Shadow output record โ€” captured per real request evaluated +proposed_evaluation_record: + policy_uuid: + policy_version: + request_uuid: + tenant_uuid: + evaluated_at: + would_have_applied: + shadow_output: + would_have_rejected: + rejection_reason: + would_have_patched: + - field: + current_value: + would_have_set: + reason: + would_have_locked: + - field: + lock_type: + reason: + would_have_selected_provider: + impact_assessment: + category: + # none: policy would not have applied to this request + # low: minor enrichment only + # medium: significant field modifications + # high: would have rejected or locked critical fields + # critical: would have rejected or overridden consumer intent +``` + +### 4b.4 Contact Info โ€” Two Modes + +Contact information supports both IdP-backed and standalone deployments: + +**Mode 1 โ€” Identity Provider backed:** +The `uuid` field contains the DCM external entity reference UUID linking to an Identity.Person or Identity.Team in a registered Information Provider. The `display_name` is cached non-authoritatively for UI display. DCM can resolve the full identity record via the Information Provider on demand. + +**Mode 2 โ€” Standalone (no Identity Provider):** +The `uuid` field is absent. `display_name`, `email`, and `notification_endpoint` are the primary identity fields. DCM accepts and records these directly without external verification. This mode supports bootstrapping, air-gapped deployments, and organizations that have not yet registered an Identity Information Provider. + +Both modes are fully supported. An organization can start in standalone mode and migrate to IdP-backed mode by adding `uuid` fields to existing artifact metadata โ€” no other changes required. + +### 4b.5 Notifications from Artifact Metadata + +The `owned_by.notification_endpoint` is the target for all proactive DCM notifications about an artifact: + +| Event | Who Is Notified | +|-------|----------------| +| Layer conflict detected at ingestion | Owner of new layer AND owner of conflicting existing layer | +| Layer deprecated | Owners of all artifacts that reference the deprecated layer | +| Provider deregistered | Owners of all catalog items backed by that provider | +| Policy violation | Owner of the entity that violated the policy | +| Drift detected | Owner of the entity that drifted | +| Proposed policy shadow shows high/critical impact | Policy owner and designated reviewers | +| Artifact approaching sunset date | Artifact owner | + +--- + +## 4c. Conflict Detection at Ingestion + +Conflict detection runs at layer ingestion time โ€” not at request assembly time. This ensures all layers in DCM are conflict-free before they are ever used. + +### 4c.1 Ingestion CI Pipeline + +When a layer is committed to the Layer Store (Git branch created or updated): + +``` +Layer committed to Git branch + โ”‚ + โ–ผ +CI Pipeline fires automatically + โ”‚ + โ”œโ”€โ”€ 1. Schema validation + โ”‚ Is the layer well-formed per the layer schema? + โ”‚ Does it carry required artifact metadata? + โ”‚ Is the version correctly incremented? + โ”‚ + โ”œโ”€โ”€ 2. Handle validation + โ”‚ Is the handle unique in DCM? + โ”‚ Does the handle match the Git path? + โ”‚ Does the domain match the submitting actor's authorization? + โ”‚ + โ”œโ”€โ”€ 3. Scope validation + โ”‚ If type-scoped: do declared resource types exist in the registry? + โ”‚ Is the layer type consistent with the domain? + โ”‚ + โ”œโ”€โ”€ 4. Priority validation + โ”‚ Is the priority value in valid dotted-notation format? + โ”‚ Does the priority category match the domain advisory range? + โ”‚ (Warning only if category/domain mismatch โ€” not a block) + โ”‚ + โ”œโ”€โ”€ 5. Conflict detection + โ”‚ For each field in this layer: + โ”‚ Find all active layers of the same type and overlapping scope + โ”‚ Check if any declare the same field + โ”‚ If conflict found: + โ”‚ โ†’ Does the new layer declare a higher priority? โ†’ Allowed, documented + โ”‚ โ†’ Does the existing layer declare a higher priority? โ†’ Allowed, documented + โ”‚ โ†’ Neither declares priority? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ โ†’ Both declare equal priority? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ โ†’ Domain authority violation? โ†’ CONFLICT ERROR โ€” PR blocked + โ”‚ + โ”‚ Conflict notification: + โ”‚ Posted as PR comment with: conflicting layer UUID, handle, owner + โ”‚ Both layer owners notified via notification_endpoint + โ”‚ + โ”œโ”€โ”€ 6. Deprecation reference validation + โ”‚ If status: deprecated โ€” does replacement UUID exist? + โ”‚ + โ””โ”€โ”€ 7. Result + All checks pass โ†’ PR approved for merge + Any check fails โ†’ PR blocked, detailed error comment posted +``` + +### 4c.2 Conflict Resolution Rules + +| Situation | Resolution | Action | +|-----------|-----------|--------| +| New layer and existing layer conflict, no priority on either | CONFLICT ERROR | PR blocked. Both owners notified. One must declare priority or remove the conflicting field. | +| New layer has higher priority (higher value) than existing | Allowed โ€” new layer wins | Documented in provenance. Warning posted if domain authority is unusual. | +| Existing layer has higher priority | Allowed โ€” existing layer wins | New layer is a lower-priority alternative. Documented. | +| Both layers have equal priority | CONFLICT ERROR | PR blocked. Priority must be differentiated. | +| New layer from lower domain overrides higher domain | CONFLICT ERROR | Domain authority violation. Platform cannot be overridden by service layer. | +| Priority category suggests domain mismatch | WARNING | PR comment posted, not blocked. Merge allowed but reviewers are notified. | + +### 4c.3 Pre-Validation of All Layers + +Because conflict detection runs at ingestion, all layers resident in DCM are pre-validated: + +- No two active layers of the same type and scope conflict without explicit priority resolution +- The assembly process never encounters an ambiguous merge โ€” all conflicts are resolved at definition time +- If a conflict is discovered after the fact (e.g., a new layer is activated that conflicts with an existing one that was already active when the new layer was ingested), the newer layer's ingestion pipeline should have caught this. A background validation job runs periodically to detect any edge cases. + +--- + +## 4d. Complete Layer Definition Structure + +Combining all elements โ€” identity, artifact metadata, scope, priority, and fields: + +```yaml +# Complete layer definition +layer: + # === ARTIFACT METADATA (universal โ€” required on all artifacts) === + artifact_metadata: + uuid: "layer-uuid-001" + handle: "platform/core/security-cpu-limits" + version: "1.2.0" + status: active + created_by: + uuid: "actor-uuid-001" # Optional โ€” present if IdP registered + display_name: "Jane Smith" + email: "jane.smith@example.com" + notification_endpoint: "https://notify.example.com/webhooks/jane" + created_at: "2026-01-15T10:30:00Z" + created_via: pr + owned_by: + uuid: "team-uuid-security" # Optional โ€” present if IdP registered + display_name: "Platform Security Team" + email: "platform-security@example.com" + notification_endpoint: "https://notify.example.com/webhooks/platform-security" + modifications: + - sequence: 1 + modified_by: + display_name: "Jane Smith" + email: "jane.smith@example.com" + modified_at: "2026-01-15T10:30:00Z" + modification_type: create + version_before: null + version_after: "1.0.0" + change_summary: "Initial creation โ€” CPU limits per CISO mandate SEC-2024-047" + pr_reference: "https://github.com/org/dcm-layers/pull/42" + reason: "CISO mandate SEC-2024-047 requires CPU limits on all containers" + - sequence: 2 + modified_by: + display_name: "Bob Jones" + email: "bob.jones@example.com" + modified_at: "2026-02-20T14:00:00Z" + modification_type: update + version_before: "1.0.0" + version_after: "1.2.0" + change_summary: "Increased CPU limit from 4 to 8 per updated mandate" + pr_reference: "https://github.com/org/dcm-layers/pull/67" + reason: "Updated CISO mandate SEC-2024-047-rev2 allows 8 CPU" + + # === LAYER IDENTITY === + domain: platform + layer_type: core + + scope: + resource_types: + - Compute.Container + - Compute.Pod + # Empty list = type-agnostic (applies to all resource types) + + priority: + value: "200.30.10" + label: "security.container.cpu_limits" + category: security + rationale: > + CPU limit enforcement for container workloads per + CISO mandate SEC-2024-047. Overrides platform defaults. + + # === LAYER CHAIN === + parent_chain: + - uuid: "base-layer-uuid-001" + handle: "system/base/universal-defaults" + version: "1.0.0" + layer_type: base + + # === FIELDS === + fields: + cpu_limit: + value: 8 + metadata: + basis_for_value: "CISO mandate SEC-2024-047-rev2" + baseline_value: 4 + override: constrained + constraint_schema: + minimum: 1 + maximum: 8 +``` + +--- + +--- + +## 5. Precedence and Merge Rules + +When layers are merged to produce the assembled payload, fields from higher-precedence layers override fields from lower-precedence layers. The precedence order from lowest to highest is: + +``` +1. Base Layer (lowest precedence โ€” foundation defaults) +2. Core Layers (organizational and infrastructure context) +3. Intermediate/Customization (organizational hierarchy overrides) +4. Service Layers (service-specific configuration) +5. Request Layer (consumer intent โ€” overrides all data layers) +6. Transformation Policies (enrichment โ€” adds or modifies fields) +7. Validation Policies (pass/fail โ€” no field modification) +8. GateKeeper Policies (highest authority โ€” overrides everything) +``` + +### 5.1 Override Behavior + +- A higher-precedence layer that declares a field **overrides** the value from all lower-precedence layers +- A higher-precedence layer that does **not** declare a field leaves the lower-precedence value intact +- Fields not declared at any layer level are absent from the payload โ€” providers must declare all required fields as being covered by at least one layer in the chain +- GateKeeper policies can override **any** field including consumer-declared Request Layer values โ€” this is the mechanism for enforcing sovereignty constraints, security mandates, and hard compliance rules + +### 5.2 Additive vs. Override Fields + +Some fields are **scalar** (a single value โ€” one layer wins) and some are **additive** (a list or set โ€” layers contribute to a collection). The field type in the Resource Type Specification declares which behavior applies: + +```yaml +field_name: + type: + merge_behavior: + # override: higher precedence layer's value replaces lower precedence value + # additive: all layers contribute their values to a merged collection +``` + +### 5.3 Conflict Resolution + +When two layers at the same precedence level declare conflicting values for the same field: +- The conflict is recorded and surfaced as a validation error +- The request is not processed until the conflict is resolved +- Conflict resolution is never silent โ€” it is always recorded in provenance + +--- + +## 5a. Field Override Control + +Field override control is the mechanism by which DCM governs **who can change what, under what conditions**, across the layer precedence chain. It was present in the original data model rules as "override preference" metadata on fields โ€” this section formalizes that concept as a graduated model that is **simple by default and powerful when needed**. + +**Design Principle:** A field with no override declaration is fully overridable by anyone. Restrictions are always opt-in. The model has three levels โ€” you use only the level you need. Levels 1 and 2 cover the vast majority of real-world cases. Level 3 exists for fields that genuinely require nuanced, actor-specific governance. + +--- + +### 5a.1 Two Categories of Override Rule + +**Category 1 โ€” Structural Rules (Request Payload Processor โ€” non-overridable)** + +Enforced by the Request Payload Processor as DCM System behavior. Not configurable. Always applied: + +- A layer entity is immutable once versioned โ€” no override can modify a published version +- A child layer cannot remove a field declared in a parent layer โ€” it can only override the value +- The layer precedence order is fixed โ€” Base โ†’ Core โ†’ Intermediate โ†’ Service โ†’ Request โ†’ Policy +- Circular layer references are rejected unconditionally +- A Service Layer without a declared type scope is rejected unconditionally + +**Category 2 โ€” Business Rules (Policy Engine โ€” configurable)** + +Enforced by the Policy Engine using the Validation/Transformation/GateKeeper mechanism. Override control metadata is set exclusively by the Policy Engine and carried in the payload as part of field-level provenance. Data layers and the Request Payload Processor never set override control. + +--- + +### 5a.2 Where Override Control is Declared + +Override control can be declared at two static levels and applied dynamically at runtime: + +**Level A โ€” Resource Type Specification (portable, sets the ceiling)** +Declares the default override behavior for a field across all implementations of that Resource Type. These defaults travel with the type definition and apply to all providers and catalog items that implement the type. This sets the maximum permissiveness ceiling โ€” lower levels can only restrict further. + +**Level B โ€” Catalog Item (offering-specific, can only restrict)** +Declares additional restrictions for a specific curated offering beyond the Resource Type defaults. A "PCI Production VM" catalog item can lock `encryption_standard` to a single value even if the VM Resource Type allows a broader enum. Cannot expand beyond what the Resource Type permits. + +**Level C โ€” Policy Engine (runtime, within static bounds)** +Applies override control at request processing time based on current organizational policies. Can only restrict within the bounds established by the Catalog Item (or Resource Type if no Catalog Item restriction exists). Higher-authority policy levels (Global) can grant expansion to trusted actors within their authority scope. + +**Inheritance Rule:** Override control can only be made more restrictive as it flows down the declaration hierarchy โ€” Resource Type โ†’ Catalog Item โ†’ Runtime Policy. The sole exception is explicit trusted grants made by higher-authority actors (see Section 5a.6). + +--- + +### 5a.3 Level 1 โ€” No Declaration (Default) + +No override control declaration on a field means it is fully overridable by any actor. This is the default for all fields. Zero configuration required. + +```yaml +# Level 1 โ€” fully overridable, no declaration needed +cpu_count: + value: 4 +``` + +This covers the majority of fields in most implementations. + +--- + +### 5a.4 Level 2 โ€” Simple Declaration + +A single `override` property covers the most common governance needs without requiring a full matrix. Sufficient for most governed fields. + +```yaml +# Level 2a โ€” nobody can change this +sovereignty_zone: + value: us-east + override: immutable + +# Level 2b โ€” anyone can change but only within these values +encryption_standard: + value: AES-256 + override: constrained + constraint_schema: + enum: [AES-256, AES-128] + +# Level 2c โ€” explicit allow (same as default, but self-documenting) +display_name: + value: my-vm + override: allow +``` + +| Value | Meaning | Enforcement | +|-------|---------|-------------| +| `allow` | Default. Any actor may override. | Structural rules | +| `constrained` | Any actor may override within `constraint_schema` | Policy Engine โ€” Validation | +| `immutable` | No actor may override at any level | Policy Engine โ€” GateKeeper | + +--- + +### 5a.5 Level 3 โ€” Matrix Declaration + +Full actor-level control for fields that require nuanced governance. Used only when Level 2 is insufficient. + +```yaml +billing_tag: + value: engineering + override_matrix: + default: allow + # Default permission for any actor not explicitly listed + # Options: allow | constrained | deny + + inheritance: restrict_only + # Catalog Items and lower-level declarations can only restrict + # Higher-authority actors can grant expansion via trusted_grants + + actors: + - actor: policy.global + permission: allow + can_expand: true + # Global policies can always override and can grant expansion + # to lower actors via trusted_grants + + - actor: policy.tenant + permission: allow + can_expand: true + # Tenant policies can override and grant within global ceiling + + - actor: policy.user + permission: deny + can_expand: false + # User policies cannot override and cannot grant to others + + - actor: consumer_request + permission: constrained + constraint_schema: + pattern: "^[a-z0-9-]+$" + can_expand: false + # Consumers can override within pattern, cannot grant expansion + + - actor: process_resource + permission: deny + can_expand: false + # Automation denied by default โ€” grant via trusted_grants + + - actor: provider + permission: deny + can_expand: false + # Providers cannot modify this field + + - actor: sre_override + permission: allow + can_expand: false + # SREs have operational authority but cannot grant to others + + - actor: admin_override + permission: allow + can_expand: true + # Admins can override and grant within their scope level + + trusted_grants: + # Explicit expansion grants from higher-authority actors + # Used when an actor needs more permission than their default + - granted_to_uuid: + actor_type: process_resource + permission: allow + granted_by_policy_uuid: + reason: Patching automation trusted to update billing_tag + expires: + + constraint_schema: + pattern: "^[a-z0-9-]+$" + # Applied to all actors with permission: constrained +``` + +--- + +### 5a.6 Actor Registry + +The actor list is extensible. DCM ships with built-in actors. Organizations register custom actors following the same model. Custom actors default to `deny` until explicitly granted permissions. + +**Built-in actors:** + +| Actor | Default Scope | Can Expand | Notes | +|-------|--------------|------------|-------| +| `policy.global` | All tenants | โœ… | Highest authority โ€” can grant to any actor | +| `policy.tenant` | Single tenant | โœ… | Within global ceiling | +| `policy.user` | Single user | โŒ | Can only restrict | +| `consumer_request` | Request submitter | โŒ | Can only restrict | +| `process_resource` | Automation execution | โŒ by default | Requires trusted grant | +| `provider` | Service Provider | โŒ | Can only restrict | +| `sre_override` | SRE team | โŒ | Operational authority, cannot grant | +| `admin_override` | DCM Admin | โœ… | Within their scope level | + +**Custom actor registration:** + +```yaml +custom_actor: + uuid: + name: + description: + registered_by_tenant_uuid: + default_permission: deny + # Custom actors always default to deny until explicitly granted + can_expand: false + # Custom actors cannot expand by default โ€” requires explicit grant + version: + status: + provenance: + +``` + +Custom actors follow the universal versioning and deprecation model. A custom actor registered at Tenant scope cannot be granted Global-level authority. + +--- + +### 5a.7 Expansion Rules + +Actor expansion follows a strict hierarchy: + +- **`policy.global`** and **`admin_override`** at global scope โ€” can grant expansion to any actor for any field, including fields declared `immutable` at lower levels +- **`policy.tenant`** and **`admin_override`** at tenant scope โ€” can grant expansion within their tenant, cannot expand beyond what Global permits +- **`policy.user`**, **`consumer_request`**, **`provider`** โ€” can never grant expansion regardless of what they receive +- **`sre_override`** โ€” can never grant expansion but can be granted expansion by Tenant or Global +- **`process_resource`** โ€” denied by default, can be granted expansion by Tenant or Global via `trusted_grants` +- **Custom actors** โ€” denied by default, can be granted expansion by the level that registered them or higher + +**Trusted grants expire** โ€” if an `expires` timestamp is set, the grant is automatically revoked at that time. Expired grants are retained in provenance for audit purposes but are no longer applied. + +--- + +### 5a.8 Override Control in the Assembly Process + +Override control is applied during Step 5 (Policy Processing) of the assembly process: + +``` +Layer Merge complete (Steps 1-4) + โ”‚ Fields have values โ€” all fields default to Level 1 (allow) + โ”‚ Static override declarations from Resource Type and Catalog Item are loaded + โ–ผ +Transformation Policies + โ”‚ May set override: constrained or override_matrix on fields + โ”‚ May set baseline_value and basis_for_value metadata + โ”‚ Records policy UUID, level, and reason in field provenance + โ–ผ +Validation Policies + โ”‚ Verify existing override declarations are not violated + โ”‚ Verify actor permissions against current override_matrix + โ”‚ Pass/fail โ€” no modification to override control + โ–ผ +GateKeeper Policies + โ”‚ May set override: immutable on fields + โ”‚ May override field values before locking + โ”‚ May issue trusted_grants to specific actors + โ”‚ Records policy UUID, level, lock type, and reason in provenance + โ–ผ +Requested State + โ”‚ All governed fields carry full override control metadata + โ”‚ Provenance chain complete โ€” every lock and grant is traceable + โ–ผ +``` + +--- + +### 5a.9 Override Control and Rehydration + +During rehydration, the Intent State is replayed through the **current** Policy Engine. Override control declared in current policies is applied fresh. A field that was `allow` in the original request may be `immutable` if a new GateKeeper policy was added since. This is by design โ€” rehydration applies current governance standards, not historical ones. + +The original consumer intent is preserved unchanged in the Intent Store. The new realized state reflects current governance. Both are auditable and traceable. + +The one exception is `pinned` policy version rehydration (Historical Exact or Historical Portable modes) โ€” this deliberately replays historical policies and may bypass current immutable locks. Pinned rehydration requires elevated authorization precisely for this reason. + +--- + +### 5a.11 Global Policy Self-Override โ€” The Immutable Ceiling Model + +**Q51 resolved:** When a Global GateKeeper policy sets `override: immutable` on a field, can a higher-priority Global policy still override it? + +**The answer emerges from execution order.** Policies execute highest-priority-first (highest numeric value first within a tier). The first policy to set `override: immutable` on a field locks it. All subsequent policies โ€” including other Global policies with lower priority values โ€” find the field locked and cannot modify it. In normal request processing, **default `immutable` is effectively absolute** โ€” not through a special rule, but through execution order. + +**The `immutable_ceiling` declaration** is a forward-looking protection for fields that must remain locked even if a higher-priority policy is **added to the system later**: + +```yaml +# Default immutable โ€” protected by execution order during this request +# The highest-priority Global GateKeeper to run first locks it +# No subsequent policy in this execution can change it +sovereignty_zone: + value: eu-west + override: immutable + # Safe in practice โ€” execution order guarantees the first-runner wins + # Does NOT protect against a new higher-priority policy being added tomorrow + +# Absolute immutable โ€” explicit forward-looking protection +# Protected even if a new higher-priority policy is added to the system +classification_level: + value: RESTRICTED + override: immutable + immutable_ceiling: absolute + # Cannot be overridden by ANY policy, ever + # If a policy attempts to override this, it receives a hard rejection + # The attempted override is logged in audit with full provenance + # Use for: sovereignty zones, data classification, hard compliance mandates +``` + +**The formal rule:** + +| Declaration | Protected During Execution? | Protected Against Future Policies? | Use Case | +|-------------|----------------------------|-------------------------------------|---------| +| `override: immutable` (default) | โœ… Yes โ€” execution order | โŒ No | Most governed fields | +| `override: immutable` + `immutable_ceiling: absolute` | โœ… Yes | โœ… Yes โ€” hard rejection | True non-negotiables | + +**`immutable_ceiling: absolute` is the nuclear option.** Use it sparingly โ€” only for fields where the governance requirement is genuinely non-negotiable regardless of any future organizational policy change. Sovereignty zone on a sovereign deployment. Data classification on a restricted system. Encryption standard under a regulatory mandate with no variance permitted. + +**Audit behavior:** When a policy attempts to override a field with `immutable_ceiling: absolute`, the attempt is rejected silently from the requesting policy's perspective (the field simply doesn't change) but is fully logged in the Audit Store with the policy UUID, the attempted value, the rejection reason, and the UUID of the policy that set the ceiling. + +--- + +### 5a.10 Override Control Metadata โ€” Full Structure + +The complete field metadata structure carrying override control in the payload: + +```yaml +field_name: + value: + metadata: + # Simple declaration (Level 2) โ€” set by Policy Engine at runtime + override: + # OR matrix declaration (Level 3) โ€” set by Policy Engine at runtime + override_matrix: + + + # Always present regardless of level + basis_for_value: + baseline_value: + locked_by_policy_uuid: + locked_at_level: + constraint_schema: + + provenance: + origin: + value: + source_type: + source_uuid: + timestamp: + modifications: + - sequence: 1 + previous_value: + modified_value: + source_uuid: + operation_type: + actor: + timestamp: + reason: +``` + +--- + +The Request Payload Processor assembles the final payload by executing the following **nine steps** in order. Each step is recorded in the payload's provenance chain. + +### Step 1 โ€” Intent Capture +The consumer's Request Layer is received and stored as the **Intent State** in the Intent Store. No modification occurs at this step. The Intent State is the immutable record of what the consumer asked for. + +### Step 2 โ€” Layer Resolution +The Request Payload Processor determines which layers apply to this request: +- Identifies the Resource Type from the Request Layer +- Retrieves the applicable Base Layer for the request context +- Retrieves all applicable Core Layers (type-agnostic โ€” all apply) +- Retrieves applicable Intermediate/Customization Layers based on organizational context +- Retrieves applicable Service Layers whose declared type scope matches the request Resource Type +- Orders all retrieved layers according to the precedence chain + +### Step 3 โ€” Layer Merge +Layers are merged in precedence order (lowest to highest). For each field: +- The value from the highest-precedence layer that declares it is used +- The source layer UUID and layer type are recorded in the field's provenance metadata +- Additive fields accumulate values from all layers that declare them + +### Step 4 โ€” Request Layer Application +The consumer's Request Layer is applied last in the data layer merge. Consumer-declared values override all data layer values. Each override is recorded in provenance. + +### Step 5 โ€” Pre-Placement Policy Processing +Policies matching the `request.layers_assembled` payload type are evaluated against the merged payload before any provider is known. GateKeeper, Transformation, Validation, and Governance Matrix policies may all fire at this stage โ€” evaluated by the Policy Engine in domain precedence order: + +1. **Transformation Policies** โ€” enrich and modify the payload. May set `override: constrained` on fields. Each transformation records the policy UUID, operation type, reason, and any override control declarations in provenance. +2. **Validation Policies** โ€” check the payload against rules. Pass/fail only โ€” no field modification. Failures reject the request. +3. **GateKeeper Policies** โ€” apply hard overrides and blocks. May set `override: immutable`. All overrides recorded in provenance. + +Pre-placement policies produce **placement constraints** โ€” declarative requirements a provider must satisfy (sovereignty zone, hardware class, conformance level, etc.). These constraints are carried forward as inputs to the Placement Engine. + +### Step 6 โ€” Placement Engine โ€” Placement Loop + +The Placement Engine takes the policy-processed payload and placement constraints, builds a candidate provider list (filtered by constraints, ordered by scoring criteria), and iterates through candidates until placement is confirmed or all candidates are exhausted. + +**Placement loop governance** (configurable by policy): +```yaml +placement_loop_config: + max_iterations: 5 # maximum candidates to attempt + max_duration_seconds: 30 # timeout for entire loop + on_exhaustion: + hold_ttl_seconds: 300 # how long provider holds resources +``` + +**Per-candidate iteration:** + +``` +โ”€โ”€ RESERVE QUERY (single atomic call to provider) โ”€โ”€ + Request: constraints + resource spec + hold TTL + metadata_requested + Response status: + confirmed: resources held, constraints satisfied, metadata returned + partial: hold confirmed, some metadata unavailable + insufficient: provider lacks capacity โ€” skip to next candidate + refused: provider cannot satisfy constraints โ€” skip to next candidate + +โ”€โ”€ POLICY PHASE (placement_phase: loop) โ”€โ”€ + Policies evaluate: payload + constraints + reserve query response + For each field declared in policy required_context: + Field present: evaluate normally + Field absent, required_context declared: + if_absent: gatekeep โ†’ release hold, abort loop, REJECT REQUEST + if_absent: warn โ†’ record warning, continue + if_absent: skip โ†’ record as skipped, continue + Field absent, no policy declares required_context: + โ†’ record policy_gap_record (implicit_approval), continue + Policy outcomes: + gatekeep โ†’ release hold, abort loop, REJECT REQUEST + reject_candidate โ†’ release hold, skip to next candidate + pass / warn โ†’ PLACEMENT CONFIRMED โ€” exit loop +``` + +**Reserve query structure:** +```yaml +reserve_query_request: + request_uuid: + hold_uuid: + resource_type: + placement_constraints: + resource_spec: + cpu: 16 + ram_gb: 64 + storage_gb: 500 + hold_ttl_seconds: 300 + metadata_requested: + - capacity_available + - topology + - sovereignty_certifications + - patch_level + - maintenance_windows + +reserve_query_response: + hold_uuid: + provider_hold_reference: + hold_status: + hold_confirmed_spec: + cpu: 16 + ram_gb: 64 + storage_gb: 500 + zone: eu-west-1a + rack: rack-07 + metadata: + topology: + zone: eu-west-1a + rack: rack-07 + network_segment: vlan-142 + available_ips: ["10.20.4.0/24"] + sovereignty_certifications: + - cert: ISO-27001 + valid_until: "2027-06-30" + missing_metadata: + - field: patch_level + reason: "Provider does not track patch metadata at this conformance level" +``` + +**Non-hold queries** (available outside the placement loop for capacity checks, provider health, cost estimation, and pre-filtering): + +| Query Type | Hold? | Purpose | +|-----------|-------|---------| +| `reserve_query` | Yes โ€” atomic | Primary placement loop query | +| `capacity_query` | No | Pre-loop filtering, dashboard, cost estimation | +| `metadata_query` | No | Provider health checks, audit, policy pre-evaluation | +| `constraint_verification` | No | Rapid pre-filter before entering the loop | + +**Policy gap records** โ€” when a field is absent and no policy declares `required_context` for it: +```yaml +policy_gap_record: + request_uuid: + field: patch_level + field_value: null + evaluation_result: implicit_approval + reason: > + No active policy declared required_context for this field. + Field was absent in reserve query response. + Request proceeded without policy evaluation of this field. + provider_uuid: + recorded_at: + resolution_expected: realized_payload + # Provider expected to supply this field in the realized payload or discovery +``` + +**Provider metadata completeness โ€” eventual consistency:** +Fields missing from the reserve query response are expected to be completed in: +1. **Realized payload** (primary) โ€” provider returns full metadata when confirming realization +2. **Discovery loop** (fallback) โ€” periodic discovery fills remaining gaps + +The realized entity carries `enrichment_status: pending | partial | complete` reflecting how complete its metadata is. This is the same pattern as the ingestion model. + +### Step 7 โ€” Post-Placement Policy Processing +Policies with `placement_phase: post` (or `both`) execute after the Placement Engine has confirmed a provider selection. These policies have full access to the `placement` block of the payload including the provider selection, hold confirmation, and all returned metadata. + +1. **Transformation Policies** โ€” provider-aware enrichment. Inject zone-specific configuration, provider-specific defaults, topology-derived values that are only knowable after provider selection. +2. **Validation Policies** โ€” post-placement checks. Verify the selected provider meets requirements that couldn't be expressed as pre-placement constraints. +3. **GateKeeper Policies** โ€” post-placement hard overrides. May inject mandatory fields triggered by the specific provider selected (e.g., additional data handling requirements for a provider in a specific jurisdiction). + +**Policy `placement_phase` values:** +```yaml +policy: + placement_phase:
+  # pre:  steps 5 โ€” before provider known (default)
+  # loop: step 6 โ€” inside placement loop, evaluates reserve query response
+  # post: step 7 โ€” after placement confirmed, provider known
+  # both: pre and post (not loop)
+```
+
+**Policy `required_context` for missing metadata:**
+```yaml
+policy:
+  placement_phase: loop
+  required_context:
+    - field: placement.provider_metadata.sovereignty_certifications
+      if_absent: gatekeep
+      if_absent_reason: >
+        Cannot evaluate sovereignty compliance without provider
+        certification data. Blocking request. Provider must register
+        this metadata to participate in sovereignty-scoped requests.
+    - field: placement.provider_metadata.patch_level
+      if_absent: warn
+      if_absent_reason: >
+        Patch level not available. Proceeding with warning.
+        Provider notified to register patch metadata.
+```
+
+### Step 8 โ€” Requested State Storage
+The fully assembled, policy-processed, placement-confirmed payload is stored as the **Requested State** in the Request Store. The Requested State includes:
+- All assembled resource fields with full provenance chain
+- Complete `placement` block: selected provider, hold UUID, all reserve query responses per iteration, all policy evaluations per iteration, placement constraints applied, alternatives considered
+- All `policy_gap_record` entries for implicit approvals
+- `enrichment_status` reflecting metadata completeness at dispatch time
+
+### Step 9 โ€” Provider Dispatch
+The Requested State payload is dispatched to the selected Service Provider via the API Gateway. The resource hold placed during the Placement Loop is confirmed by dispatch. The provider uses the hold reference to fulfill the request against the reserved resources.
+
+---
+
+---
+
+## 7. Layer Assembly Diagram
+
+```
+Consumer Request
+      โ”‚
+      โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚  REQUEST LAYER  โ”‚  โ† Consumer declared intent โ†’ stored as INTENT STATE (Step 1)
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+         โ”‚
+         โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚             LAYER RESOLUTION + MERGE (Steps 2-4)         โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Base Layer          (lowest precedence)                 โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  Core Layers         (type-agnostic context)             โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  Intermediate Layers (organizational context)            โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  Service Layers      (type-scoped service config)        โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  Request Layer       (consumer intent โ€” highest          โ”‚
+โ”‚                       data layer precedence)             โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+         โ”‚  Merged payload with full provenance
+         โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚          PRE-PLACEMENT POLICY PROCESSING (Step 5)        โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Transformation Policies  (enrich / modify)              โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  Validation Policies      (pass / fail check)            โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  GateKeeper Policies      (override / block)             โ”‚
+โ”‚       โ†“ outputs: placement constraints                   โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+         โ”‚  Policy-processed payload + placement constraints
+         โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚              PLACEMENT ENGINE โ€” LOOP (Step 6)            โ”‚
+โ”‚                                                          โ”‚
+โ”‚  For each candidate provider (filtered + scored):        โ”‚
+โ”‚    โ”‚                                                     โ”‚
+โ”‚    โ”œโ”€โ”€ Reserve Query (atomic: verify + metadata + hold)  โ”‚
+โ”‚    โ”‚     confirmed / partial โ†’ policy phase              โ”‚
+โ”‚    โ”‚     insufficient / refused โ†’ next candidate         โ”‚
+โ”‚    โ”‚                                                     โ”‚
+โ”‚    โ””โ”€โ”€ Loop Policy Phase (placement_phase: loop)         โ”‚
+โ”‚          Field present โ†’ evaluate normally               โ”‚
+โ”‚          Field absent + required_context โ†’ if_absent     โ”‚
+โ”‚          Field absent + no policy โ†’ implicit_approval    โ”‚
+โ”‚          pass/warn โ†’ PLACEMENT CONFIRMED                 โ”‚
+โ”‚          reject_candidate โ†’ release hold, next           โ”‚
+โ”‚          gatekeep โ†’ release hold, REJECT REQUEST         โ”‚
+โ”‚                                                          โ”‚
+โ”‚  No candidates remain โ†’ on_exhaustion behavior           โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+         โ”‚  selected_provider_uuid + placement block
+         โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚         POST-PLACEMENT POLICY PROCESSING (Step 7)        โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Transformation Policies  (provider-aware enrichment)    โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  Validation Policies      (post-placement checks)        โ”‚
+โ”‚       โ†“                                                  โ”‚
+โ”‚  GateKeeper Policies      (provider-triggered overrides) โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+         โ”‚  Complete, validated, placement-confirmed payload
+         โ–ผ
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚ REQUESTED STATE โ”‚  โ† Stored in Request Store (Step 8)
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜    includes: placement block, hold records,
+         โ”‚             policy gap records, enrichment_status
+         โ–ผ
+   Service Provider  (Step 9 โ€” dispatch, hold confirmed)
+```
+
+---
+
+## 8. Layer Scope and Type Enforcement
+
+### 8.1 Core Layer Scope Enforcement
+Core Layers are type-agnostic by default. They are applied to every request regardless of Resource Type. A Core Layer that contains service-specific or provider-specific data is invalid and must be rejected.
+
+### 8.2 Service Layer Scope Enforcement
+Service Layers must declare a Resource Type scope. The Request Payload Processor enforces this during Layer Resolution:
+- A Service Layer whose declared Resource Type does not match the request Resource Type is excluded from the merge
+- A Service Layer with `scope_inheritance: exact` is only included if the request Resource Type exactly matches the declared type
+- A Service Layer with `scope_inheritance: descendants` is included if the request Resource Type is the declared type or any descendant type in the inheritance hierarchy
+- A Service Layer with no declared type scope is invalid and must be rejected
+
+### 8.3 Unanticipated Data Interaction Prevention
+The type scoping rules for Service Layers are the primary mechanism for preventing unanticipated data interactions โ€” one of the core data model objectives. Because Service Layers can only contribute to requests of their declared type, data from one service domain cannot inadvertently affect requests in another service domain.
+
+---
+
+## 9. Layer Versioning
+
+All layers follow the universal DCM versioning scheme: **Major.Minor.Revision**
+
+| Component | Trigger |
+|-----------|---------|
+| **Major** | Breaking changes โ€” removing fields, changing field types, changing a field from optional to required |
+| **Minor** | Additive changes โ€” adding new optional fields, adding new contextual data |
+| **Revision** | Data/configuration changes โ€” updating field values, updating descriptions, updating metadata |
+
+**Immutability:** Once a layer version is published it cannot be modified. Any change produces a new version. Previous versions remain accessible and can be referenced by existing realized entities.
+
+**Parent Chain Versioning:** A layer's parent chain references specific versions of parent layers. Updating a parent layer does not automatically update child layers โ€” child layers must be explicitly updated to reference the new parent version, producing a new version of the child layer.
+
+---
+
+## 10. Artifact Lifecycle โ€” The Five Statuses
+
+All DCM artifacts โ€” layers, policies, resource types, catalog items, and all other defined objects โ€” follow a five-status lifecycle. The statuses are defined in Section 4b.2. For layers specifically:
+
+| Status | Layer Behavior |
+|--------|---------------|
+| `developing` | Layer is in active development. Only usable in development mode pipelines. Not loaded by the assembly process in production. |
+| `proposed` | Layer has been submitted for review (PR open). Not yet active. For policy layers: shadow execution runs. For data layers: layer is visible in the registry but not applied. Cannot merge to active until PR is approved. |
+| `active` | Layer is current and applied in assembly. Can be included in new layer chains. |
+| `deprecated` | Layer is being phased out. Existing chains using it continue to function. New chains should use the replacement. Deprecation warning recorded in assembly provenance. Must include replacement UUID, reason, migration guidance, and sunset date. |
+| `retired` | Layer cannot be included in new layer chains. Existing realized entities that reference it retain the reference for audit purposes but cannot be used for new requests. |
+
+**Status transition rules for layers:**
+```
+developing โ†’ proposed   (author submits PR)
+developing โ†’ retired    (author abandons)
+proposed   โ†’ active     (PR merged โ€” approval complete)
+proposed   โ†’ developing (PR returned for rework)
+active     โ†’ deprecated (replacement available โ€” sunset declared)
+deprecated โ†’ retired    (sunset date reached or manual retirement)
+```
+
+---
+
+## 11. Scale Example โ€” 40,000 Linux VMs
+
+This example illustrates the power of the layering model at scale. 40,000 distinct VM configurations are governed by 36 layer definitions:
+
+```
+Base Entity (3 variants)
+โ”œโ”€โ”€ CIS Benchmark
+โ”œโ”€โ”€ Baseline
+โ””โ”€โ”€ DMZ / Payments
+
+  โ””โ”€โ”€ Layer Entity โ€” OS Family (3 variants per base = 9 total)
+      โ”œโ”€โ”€ Common Linux Config / RHEL
+      โ”œโ”€โ”€ Common Linux Config / CoreOS
+      โ””โ”€โ”€ Common Linux Config / OEL
+
+        โ””โ”€โ”€ Layer Entity โ€” OS Version (4 variants per OS layer = 36 total)
+            โ”œโ”€โ”€ RHEL 6
+            โ”œโ”€โ”€ RHEL 7
+            โ”œโ”€โ”€ RHEL 8
+            โ””โ”€โ”€ RHEL 9
+
+              โ””โ”€โ”€ Realized Entity โ€” one per VM (40,000 total)
+                  Each realized entity carries FK references to its
+                  full layer chain (Base UUID + Layer UUIDs)
+                  and is stored in the CMDB
+```
+
+**Result:** 3 ร— 3 ร— 4 = **36 layer definitions** govern **40,000 VM configurations**. Each VM's realized entity is a lightweight reference to its layer chain โ€” not a copy of all the configuration data.
+
+This also means:
+- Updating the CIS Benchmark base layer creates one new layer version that cascades to all 40,000 VMs at their next realization
+- Drift detection compares each VM's discovered state against its realized entity's layer chain
+- Any VM can be reproduced exactly by replaying its layer chain through the assembly process
+
+---
+
+## 12. Relationship to the Four States
+
+| Layer | State Relationship |
+|-------|-------------------|
+| Request Layer (as submitted) | Directly captured as **Intent State** โ€” stored in Intent Store before any processing |
+| Assembled payload (post-merge, pre-policy) | Intermediate โ€” not a named state, internal to assembly process |
+| Assembled payload (post-policy) | Becomes **Requested State** โ€” stored in Request Store |
+| Provider execution result | Becomes **Realized State** โ€” stored in Realized Store |
+| Discovery interrogation result | Becomes **Discovered State** โ€” stored in Discovered Store |
+
+The layer chain of a Realized Entity is always traceable โ€” given a Realized State record, the complete layer chain that produced it can be reconstructed, providing full audit capability back to the original Base Layer.
+
+---
+
+## 13. Layer Gaps โ€” Q21 through Q24
+
+### 13.1 Consumer Layer Exclusion (Q21)
+
+Consumers may explicitly exclude specific layers from their request. Each exclusion carries a mandatory human-readable reason recorded in provenance and the audit trail.
+
+```yaml
+request:
+  resource_type: Compute.VirtualMachine
+  layer_exclusions:
+    - layer_handle: "platform/networking/default-dns-config"
+      reason: "This VM uses custom DNS โ€” default config conflicts with application requirements"
+    - layer_uuid: 
+      reason: "Dev environment โ€” monitoring layer not required"
+```
+
+**Exclusion mechanics:**
+- Excluded layers are removed from the candidate set during **Step 2 (Layer Resolution)** before priority ordering
+- Excluded layers produce no fields in the assembled payload
+- If a validation policy requires a field that would have been injected by an excluded layer, the validation fails with a clear message identifying the excluded layer
+- Exclusion is different from override โ€” exclusion removes the entire layer; override changes specific field values
+
+**Policy enforcement:** GateKeeper policies may declare specific layers non-excludable:
+
+```yaml
+policy:
+  type: gatekeeper
+  rule: >
+    If request.layer_exclusions CONTAINS layer.concern_tags CONTAINS "security-baseline"
+    THEN gatekeep: "Security baseline layers cannot be excluded"
+  immutable_ceiling: absolute
+```
+
+### 13.2 Service Layer Versioning (Q22)
+
+Service Layers are **independently versioned artifacts** โ€” not coupled to Service Provider versions. Service Providers declare semver-compatible version constraints for the layers they use.
+
+```yaml
+# Service Provider registration โ€” layer compatibility declarations
+provider_registration:
+  layer_compatibility:
+    - layer_handle: "layers/vm-compute-defaults"
+      compatible_versions: "^1.0.0"    # any 1.x version
+    - layer_handle: "layers/vm-networking-config"
+      compatible_versions: "~1.2"      # any 1.2.x revision
+```
+
+**Version lifecycle:** Service Layers follow the standard five-status artifact lifecycle. A deprecated Service Layer continues to work for existing realizations until retired. If a provider bumps to a new major version and updates its compatibility declaration, the old layer version is no longer used for new requests via that provider but continues to work for existing realizations.
+
+**Cache invalidation:** Service Layer Cache entries carry the layer version. When the registered layer version increments, the cache entry is invalidated and refreshed before the next assembly.
+
+### 13.3 Conditional Layer Inclusion (Q23)
+
+Layers may declare an `activation_condition` โ€” a field comparison evaluated during **Step 2 (Layer Resolution)**. Layers whose condition evaluates false are excluded from the candidate set.
+
+```yaml
+layer:
+  handle: "platform/compute/gpu-config"
+  activation_condition:
+    field: request.gpu_requested
+    operator: equals
+    value: true
+```
+
+**Compound conditions:**
+```yaml
+activation_condition:
+  conditions:
+    operator: and   # and | or
+    rules:
+      - field: request.gpu_requested
+        operator: equals
+        value: true
+      - field: request.resource_class
+        operator: in
+        value: [ml-training, gpu-compute]
+```
+
+**Condition field scope** โ€” activation conditions may reference:
+- Request fields (`request.gpu_requested`)
+- Tenant attributes (`tenant.tags`, `tenant.profile`)
+- Resource type fields (`resource_type.version`)
+- Core Layer fields already resolved in Step 1 (`core_layers.location_region`)
+- Ingress fields (`ingress.actor.roles`) โ€” enabling role-specific layers
+
+**Condition vs consumer exclusion:** Conditional inclusion is declared by the layer author and evaluated automatically. Consumer exclusion (Q21) is declared at request time by the consumer. Both result in the layer being absent from the candidate set โ€” but for different reasons, recorded differently in provenance.
+
+### 13.4 Layer Chain and Service Dependencies (Q24)
+
+Each service dependency executes its **own independent layer chain** during assembly. Dependencies do not share the parent request's layer chain.
+
+**Dependencies inherit from parent (read-only context):**
+- Tenant UUID and sovereignty context
+- Parent's resolved placement fields (declared by Resource Type Specification as `propagated_to_dependencies`)
+- Parent's resolved identity fields (hostname, etc.)
+- Active Profile
+
+**Dependencies do NOT inherit:**
+- Parent consumer declarations
+- Resource-type-specific layers (each type has its own)
+- Provider-specific layers (each provider has its own)
+
+**Dependency assembly flow:**
+```
+Parent request: Compute.VirtualMachine
+  โ”‚
+  โ–ผ  Steps 1-4: Parent layer chain โ†’ parent_assembled_payload
+  โ”‚
+  โ–ผ  Step 5: Pre-placement policies on parent
+  โ”‚
+  โ–ผ  Step 6: Placement loop โ€” parent provider selected
+  โ”‚           Also identifies required dependency providers
+  โ”‚
+  โ–ผ  For each dependency (parallel where ordering allows):
+  โ”‚  โ”œโ”€โ”€ Network.IPAddress โ†’ own layer chain (Steps 1-4)
+  โ”‚  โ”‚     Context: inherits parent resolved placement fields
+  โ”‚  โ”œโ”€โ”€ Network.Port โ†’ own layer chain (Steps 1-4)
+  โ”‚  โ”‚     Context: inherits parent + IP resolution result
+  โ”‚  โ””โ”€โ”€ DNS.Record โ†’ own layer chain (Steps 1-4)
+  โ”‚         Context: inherits parent + IP + Port results
+  โ”‚
+  โ–ผ  Steps 7-9: Post-placement, storage, dispatch
+       Parent + all dependency payloads dispatched together
+```
+
+**Layer exclusions on dependencies** โ€” consumers may declare per-dependency exclusions:
+```yaml
+request:
+  resource_type: Compute.VirtualMachine
+  dependencies:
+    - resource_type: Network.IPAddress
+      layer_exclusions:
+        - layer_handle: "layers/ip-default-ttl-config"
+          reason: "Custom TTL required โ€” excluding default"
+```
+
+---
+
+## 13b. Override Control and Constraint Visibility Gaps
+
+### 13b.1 Override Preference Enforcement (Q50)
+
+Layer fields declare override intent using three values. The Request Payload Processor enforces this during assembly Step 3 (Layer Merge) โ€” no separate GateKeeper policy required.
+
+```yaml
+fields:
+  dns_servers:
+    value: [10.0.0.53, 10.0.0.54]
+    metadata:
+      override: allow          # lower layers and consumers may change this
+
+  encryption_at_rest:
+    value: true
+    metadata:
+      override: immutable      # no lower-domain layer or consumer may change this
+      lock_reason: "CISO mandate SEC-2024-047 โ€” encryption always required"
+
+  cpu_count:
+    value: 4
+    metadata:
+      override: constrained    # may change within declared bounds
+      constraint:
+        type: range
+        min: 1
+        max: 32
+        step: 1
+      constraint_reason: "Platform capacity planning bounds"
+```
+
+**Authority rule:** `immutable` prevents overrides only from *lower-authority domains*. A `platform` domain field marked `immutable` cannot be changed by `tenant`, `service`, `provider`, or `request` layers โ€” but a `system` domain layer above it can still override it. Higher authority always wins.
+
+**GateKeeper escalation:** A GateKeeper policy at a higher authority level may additionally lock a field that a layer marked `allow` โ€” this is the compliance escape hatch for mandates the layer author did not anticipate.
+
+**Enforcement point:** Step 3 (Layer Merge) โ€” if a lower-priority layer or consumer request sets a field marked `immutable`, assembly halts with a clear error identifying the conflicting layer and the locking layer.
+
+### 13b.2 Constraint Schema Visibility (Q52)
+
+Constrained fields expose their constraint schema to consumers in the Service Catalog UI and Consumer API at a policy-governed disclosure level.
+
+```yaml
+constraint_visibility:
+  level: 
+  # full:    Show constraint type, bounds, constraint_reason, suggested values
+  # summary: Show bounds only โ€” no reason, no suggestions
+  # hidden:  Field appears free-form; constraint silently enforced at submission
+```
+
+**Profile defaults:**
+
+| Profile | Default Level | Rationale |
+|---------|--------------|-----------|
+| `minimal` | `full` | All context helpful |
+| `dev` | `full` | Developers benefit from full schema |
+| `standard` | `full` | Good developer experience |
+| `prod` | `summary` | Bounds visible; reasons may be sensitive |
+| `fsi` | `summary` | Regulatory constraints may not need full exposure |
+| `sovereign` | `hidden` | Constraint details may be operationally sensitive |
+
+**UI rendering (full mode):**
+```
+VM Size โ€” CPU Count
+  Enter a value between 1 and 32 (whole numbers)
+  Suggested: 2, 4, 8, 16
+  Reason: Platform capacity planning bounds
+```
+
+**API endpoint:** `GET /api/v1/catalog/items/{id}/schema` returns field schemas at the declared visibility level for the authenticated consumer's Tenant profile.
+
+Policy may override the profile default per field or resource type:
+```yaml
+policy:
+  type: transformation
+  rule: >
+    If resource_type == Compute.VirtualMachine
+    AND field.name == cpu_count
+    THEN set: constraint_visibility.level = full
+```
+
+### 13b.3 System Policies โ€” Override Control
+
+| Policy | Rule |
+|--------|------|
+| `LAY-005` | Layer fields declare override intent as allow, constrained, or immutable. The Request Payload Processor enforces override declarations during assembly Step 3. immutable prevents overrides from lower-authority domains only โ€” higher-domain layers may always override. GateKeeper policies may additionally lock allow or constrained fields at runtime. |
+| `LAY-006` | Constraint schemas on constrained fields are visible to consumers in the Service Catalog UI and Consumer API at a policy-governed disclosure level: full (constraint, bounds, reason, suggestions), summary (bounds only), or hidden (enforced but not displayed). Profile sets the default. Policy may override per field or resource type. |
+
+
+## 13a. Layer System Policies
+
+| Policy | Rule |
+|--------|------|
+| `LAY-001` | Consumers may declare `layer_exclusions` in their request. Each exclusion must carry a human-readable reason recorded in provenance. GateKeeper policies may declare specific layers non-excludable. Excluded layers produce no fields in the assembled payload. |
+| `LAY-002` | Service Layers are independently versioned artifacts. Service Providers declare layer compatibility using semver constraints. Service Layer Cache entries carry the layer version and are invalidated when the registered version changes. |
+| `LAY-003` | Service Layers may declare `activation_condition` evaluated during Step 2 (Layer Resolution). Layers whose conditions evaluate false are excluded from the candidate set. Condition evaluation results are recorded in the assembly provenance. Conditions may reference request fields, tenant attributes, resource type fields, resolved core layer fields, and ingress fields. |
+| `LAY-004` | Each service dependency executes its own independent layer chain during assembly. Dependencies inherit the parent's resolved placement and identity fields as declared by the Resource Type Specification. Dependencies do not inherit parent consumer declarations, resource-type-specific layers, or provider-specific layers. Layer exclusions may be declared per-dependency. |
+
+---
+
+## 14. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | How are conflicting Service Layers at the same precedence level resolved? | Assembly determinism | โœ… Resolved โ€” priority schema + conflict detection at ingestion |
+| 2 | Should Core Layers be ordered within their precedence level? | Merge determinism | โœ… Resolved โ€” priority schema provides deterministic ordering |
+| 3 | Can a consumer explicitly exclude a layer from their request? | Consumer control vs. standardization | โœ… Resolved โ€” layer_exclusions with mandatory reason; GateKeeper can lock layers as non-excludable (LAY-001) |
+| 4 | How are Service Layers registered and versioned relative to Service Provider registration? | Provider contract | โœ… Resolved โ€” independently versioned; provider declares semver compatibility; cache invalidation on version change (LAY-002) |
+| 5 | Should assembly support conditional layer inclusion? | Assembly flexibility | โœ… Resolved โ€” activation_condition on layers; evaluated in Step 2; references request, tenant, resource type, core layer, and ingress fields (LAY-003) |
+| 6 | How does the layer chain interact with service dependencies? | Dependency model | โœ… Resolved โ€” each dependency has its own layer chain; inherits parent resolved placement context; no consumer declaration inheritance (LAY-004) |
+| 7 | Should `override_preference` be declarable in layer definitions as a hint to the Policy Engine? | Override control | โœ… Resolved โ€” override: allow/constrained/immutable enforced by Request Payload Processor at Step 3; GateKeeper may additionally lock (LAY-005) |
+| 8 | When `override_preference: immutable` is set โ€” can a higher-priority policy still override it? | Override control precedence | โœ… Resolved โ€” immutable prevents lower-authority overrides only; higher-domain layers always win; GateKeeper can additionally lock (LAY-005) |
+| 9 | Should the `constraint_schema` on a constrained field be visible to consumers in the Service Catalog UI? | Consumer experience | โœ… Resolved โ€” full/summary/hidden disclosure levels; profile-governed defaults; API endpoint returns schema at declared visibility (LAY-006) |
+| 10 | Should the background validation job for detecting post-ingestion conflicts run on a schedule or be event-triggered? | Operational | โœ… Resolved โ€” event-triggered primary (on layer ingestion/update) + weekly scheduled sweep safety net; async non-blocking; both produce same conflict record format (OPS-003) |
+| 11 | What is the minimum validation review period for a proposed policy before it can be activated? | Policy governance | โœ… Resolved โ€” GateKeeper=14d, Validation=7d, Transformation=3d ร— profile multiplier (minimal=0ร—, dev=0.5ร—, standard=1ร—, prod=1.5ร—, fsi/sovereign=2ร—); DCM enforces; emergency bypass requires dual-approval audit (OPS-004) |
+
+---
+
+## 14. Related Concepts
+
+- **Request Payload Processor** โ€” the control plane component that executes the assembly process; enforces structural layer rules
+- **Policy Engine** โ€” executes Policy Layers (Validation, Transformation, GateKeeper) during the assembly process; the sole authority for setting field override control
+- **Field Override Control** โ€” the mechanism governing who can change what field, under what conditions, at what policy level
+- **Override Preference** โ€” per-field metadata declaring `allow`, `constrained`, or `immutable` โ€” the formalization of the original data model "override preference" subtag
+- **Service Layer Cache** โ€” caches Service Layer data at Service Provider registration time for efficient retrieval during assembly
+- **Core Layer Store** โ€” stores all Core Layer definitions
+- **Intent State** โ€” the Request Layer as submitted, before assembly
+- **Requested State** โ€” the fully assembled, policy-processed payload
+- **Field-Level Provenance** โ€” every field in the assembled payload records which layer set it and which policy modified it
+- **Resource Type Hierarchy** โ€” defines the type scope that Service Layers must declare and that the assembly process enforces
+- **GitOps** โ€” all layers are stored in Git, versioned and immutable
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/04-examples.md b/content/docs/data-model/04-examples.md
new file mode 100644
index 0000000..1f63ab7
--- /dev/null
+++ b/content/docs/data-model/04-examples.md
@@ -0,0 +1,490 @@
+# DCM Data Model โ€” Worked Examples
+
+> **โš ๏ธ Active Development Notice**
+>
+> Examples in this document are illustrative of the intended architecture. YAML structures, field names, and sequences represent the design intent and will be refined as implementation proceeds.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Reference Examples
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + PROVIDER + POLICY**
+>
+> Worked examples showing all three abstractions in operation
+
+
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md)
+
+---
+
+## 1. Purpose
+
+This document provides end-to-end worked examples that make the DCM data model concrete. Each example traces the complete lifecycle of a resource through DCM โ€” from consumer intent through the four states, showing exactly what data exists at each stage.
+
+These examples also resolve outstanding implementation details deferred from other documents, specifically the Git repository structure for the Intent and Requested stores.
+
+---
+
+## 2. Git Repository Structure
+
+This resolves the deferred Q54 item from the Four States document (Section 4.1).
+
+The Intent and Requested stores use a handle-based directory structure within Git. Tenant isolation is enforced at the directory level. Provider selection (the Q54 concern) is recorded in the assembled payload, not in the directory structure โ€” so the directory structure is independent of which provider was selected.
+
+### 2.1 Intent Store Layout
+
+```
+intent-store/
+โ”œโ”€โ”€ {tenant-uuid}/
+โ”‚   โ”œโ”€โ”€ {resource-type-category}/
+โ”‚   โ”‚   โ”œโ”€โ”€ {resource-type}/
+โ”‚   โ”‚   โ”‚   โ”œโ”€โ”€ {entity-uuid}/
+โ”‚   โ”‚   โ”‚   โ”‚   โ”œโ”€โ”€ intent.yaml          โ† consumer's raw declaration
+โ”‚   โ”‚   โ”‚   โ”‚   โ””โ”€โ”€ .metadata.yaml       โ† intent metadata (created_by, timestamp, ingress surface)
+โ”‚   โ”‚   โ”‚   โ””โ”€โ”€ {entity-uuid-2}/
+โ”‚   โ”‚   โ”‚       โ”œโ”€โ”€ intent.yaml
+โ”‚   โ”‚   โ”‚       โ””โ”€โ”€ .metadata.yaml
+โ”‚   โ”‚   โ””โ”€โ”€ ...
+โ”‚   โ””โ”€โ”€ ...
+โ””โ”€โ”€ ...
+
+# Example:
+intent-store/
+โ””โ”€โ”€ a1b2c3d4-tenant-uuid/
+    โ””โ”€โ”€ Compute/
+        โ””โ”€โ”€ VirtualMachine/
+            โ””โ”€โ”€ f5e6d7c8-entity-uuid/
+                โ”œโ”€โ”€ intent.yaml
+                โ””โ”€โ”€ .metadata.yaml
+```
+
+**Branch naming:** `intent/{tenant-uuid}/{entity-uuid}` for new requests. `intent/{tenant-uuid}/{entity-uuid}/v{n}` for revisions.
+
+**Merge to main:** Triggers the CD pipeline โ€” Request Payload Processor begins assembly.
+
+### 2.2 Requested Store Layout
+
+```
+requested-store/
+โ””โ”€โ”€ {tenant-uuid}/
+    โ””โ”€โ”€ {resource-type-category}/
+        โ””โ”€โ”€ {resource-type}/
+            โ””โ”€โ”€ {entity-uuid}/
+                โ”œโ”€โ”€ requested.yaml          โ† fully assembled payload
+                โ”œโ”€โ”€ assembly-provenance.yaml โ† complete layer chain and policy evaluation record
+                โ”œโ”€โ”€ placement.yaml           โ† provider selection and placement constraints
+                โ””โ”€โ”€ dependencies.yaml        โ† resolved dependency graph
+
+# Example:
+requested-store/
+โ””โ”€โ”€ a1b2c3d4-tenant-uuid/
+    โ””โ”€โ”€ Compute/
+        โ””โ”€โ”€ VirtualMachine/
+            โ””โ”€โ”€ f5e6d7c8-entity-uuid/
+                โ”œโ”€โ”€ requested.yaml
+                โ”œโ”€โ”€ assembly-provenance.yaml
+                โ”œโ”€โ”€ placement.yaml
+                โ””โ”€โ”€ dependencies.yaml
+```
+
+### 2.3 Layer and Policy Store Layout
+
+```
+layers/
+โ”œโ”€โ”€ system/
+โ”‚   โ”œโ”€โ”€ core/
+โ”‚   โ”‚   โ”œโ”€โ”€ datacenter-layer.yaml
+โ”‚   โ”‚   โ””โ”€โ”€ environment-layer.yaml
+โ”‚   โ””โ”€โ”€ compliance/
+โ”‚       โ””โ”€โ”€ pci-dss-layer.yaml
+โ”œโ”€โ”€ {tenant-uuid}/
+โ”‚   โ””โ”€โ”€ org/
+โ”‚       โ””โ”€โ”€ payments-team-layer.yaml
+โ””โ”€โ”€ providers/
+    โ””โ”€โ”€ {provider-uuid}/
+        โ””โ”€โ”€ vm-defaults-layer.yaml
+
+policies/
+โ”œโ”€โ”€ system/
+โ”‚   โ”œโ”€โ”€ gatekeeper/
+โ”‚   โ”‚   โ””โ”€โ”€ vm-size-limits.yaml
+โ”‚   โ””โ”€โ”€ transformation/
+โ”‚       โ””โ”€โ”€ inject-monitoring.yaml
+โ””โ”€โ”€ {tenant-uuid}/
+    โ””โ”€โ”€ gatekeeper/
+        โ””โ”€โ”€ approved-os-images.yaml
+```
+
+---
+
+## 3. Example 1 โ€” VM Provision End-to-End
+
+A developer on the AppTeam Tenant requests a standard Linux VM. This example traces the complete lifecycle through all four states.
+
+### 3.1 Consumer Submits Intent (Intent State)
+
+The developer submits the following intent via the Consumer API:
+
+```yaml
+# intent-store/a1b2c3d4-tenant/Compute/VirtualMachine/f5e6d7c8-entity/intent.yaml
+
+apiVersion: dcm.io/v1
+kind: ResourceIntent
+metadata:
+  entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0
+  resource_type: Compute.VirtualMachine
+  tenant_uuid: a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6   # AppTeam Tenant
+  submitted_by: b2c3d4e5-actor-uuid
+  submitted_at: 2026-03-15T09:00:00Z
+  ingress_surface: consumer_api
+
+spec:
+  # Consumer declares what they need โ€” not how to provision it
+  cpu_count: 4
+  memory_gb: 8
+  storage_gb: 100
+  os_family: rhel
+  environment: production
+  name: "payments-api-server-01"
+  # No provider specified โ€” consumer does not choose the provider
+```
+
+**CI pipeline runs immediately:**
+- Policy pre-validation: no GateKeeper violations detected (4 CPU is within AppTeam's quota)
+- Cost estimation: ~$0.32/hour based on current provider rates
+- Dependency check: no dependencies declared โ€” clean
+- Sovereignty check: AppTeam's Tenant has `data_residency: EU-WEST` โ€” placement must honor this
+- Authorization check: actor b2c3d4e5 has `request:compute:vm` permission in AppTeam Tenant
+- Auto-approve evaluation: meets all auto-approve criteria โ†’ PR auto-merged
+
+### 3.2 Assembly Produces Requested State
+
+After intent merge, the Request Payload Processor runs the nine-step assembly:
+
+**Step 3 โ€” Layer Resolution and Merge:**
+
+```yaml
+# Layer chain assembled (in precedence order, highest to lowest):
+# 1. system/core/datacenter-layer.yaml        (system domain)
+# 2. system/core/environment-layer.yaml        (system domain)
+# 3. system/compliance/eu-west-layer.yaml      (system domain)
+# 4. org/appteam-defaults-layer.yaml           (tenant domain)
+# 5. providers/openstack/vm-defaults-layer.yaml (provider domain โ€” pre-selected by policy)
+# 6. Consumer intent                            (request domain)
+
+# Resulting merged fields before policy evaluation:
+cpu_count:
+  value: 4                         # from consumer intent
+  provenance.origin.source_type: consumer
+  provenance.origin.source_uuid: f5e6d7c8-entity
+
+memory_gb:
+  value: 8                         # from consumer intent
+  provenance.origin.source_type: consumer
+
+storage_gb:
+  value: 100                       # from consumer intent
+
+data_center:
+  value: "EU-WEST-DC1"             # from datacenter layer
+  provenance.origin.source_type: base_layer
+  provenance.origin.source_uuid: dc-layer-uuid
+
+environment:
+  value: production                # from consumer intent (overrides layer default "dev")
+  provenance.modifications:
+    - sequence: 1
+      previous_value: dev          # layer default
+      modified_value: production   # consumer override
+      source_type: consumer
+
+monitoring_agent:
+  value: "datadog-agent:7.42"     # injected by org layer โ€” consumer did not declare this
+  provenance.origin.source_type: intermediate_layer
+  provenance.origin.source_uuid: appteam-defaults-layer-uuid
+
+backup_policy:
+  value: "daily-30d-eu-west"      # injected by compliance layer
+  provenance.origin.source_type: intermediate_layer
+  provenance.origin.source_uuid: eu-west-compliance-layer-uuid
+```
+
+**Step 5-7 โ€” Policy Evaluation:**
+
+```yaml
+# GateKeeper policy: vm-size-limits evaluates
+# Result: APPROVED (4 CPU within AppTeam's 16 CPU limit)
+
+# Transformation policy: inject-monitoring evaluates
+# Result: monitoring_endpoint field injected
+monitoring_endpoint:
+  value: "https://metrics.internal.eu-west.example.com"
+  provenance.modifications:
+    - sequence: 1
+      previous_value: null
+      modified_value: "https://metrics.internal.eu-west.example.com"
+      source_type: policy
+      source_uuid: inject-monitoring-policy-uuid
+      operation_type: enrichment
+      reason: "Standard monitoring endpoint for EU-WEST production resources"
+
+# GateKeeper policy: approved-os-images evaluates (AppTeam's tenant policy)
+# Result: APPROVED (rhel is in AppTeam's approved images list)
+```
+
+**Step 6 โ€” Placement Engine selects provider:**
+- Sovereignty pre-filter: eligible providers must satisfy `data_residency: EU-WEST`
+- Reserve query to 3 eligible OpenStack instances
+- EU-WEST-Prod-1 responds: capacity available, confidence 94
+- EU-WEST-Prod-2 responds: capacity available, confidence 87
+- EU-WEST-Prod-3: insufficient capacity
+- Tie-breaking: EU-WEST-Prod-1 selected (highest confidence score)
+
+**Requested State committed:**
+
+```yaml
+# requested-store/a1b2c3d4-tenant/Compute/VirtualMachine/f5e6d7c8-entity/requested.yaml
+
+apiVersion: dcm.io/v1
+kind: RequestedState
+metadata:
+  entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0
+  resource_type: Compute.VirtualMachine
+  tenant_uuid: a1b2c3d4-e5f6-a7b8-c9d0-e1f2a3b4c5d6
+  assembled_at: 2026-03-15T09:00:47Z
+  intent_state_ref: f5e6d7c8-intent-ref-uuid
+
+spec:
+  cpu_count: { value: 4, provenance: {...} }
+  memory_gb: { value: 8, provenance: {...} }
+  storage_gb: { value: 100, provenance: {...} }
+  os_family: { value: rhel, provenance: {...} }
+  environment: { value: production, provenance: {...} }
+  name: { value: "payments-api-server-01", provenance: {...} }
+  data_center: { value: "EU-WEST-DC1", provenance: {...} }
+  monitoring_agent: { value: "datadog-agent:7.42", provenance: {...} }
+  backup_policy: { value: "daily-30d-eu-west", provenance: {...} }
+  monitoring_endpoint: { value: "https://metrics...", provenance: {...} }
+
+placement:
+  selected_provider_uuid: eu-west-prod-1-provider-uuid
+  placement_reason: "highest confidence score among eligible providers"
+  sovereignty_satisfied: true
+  reserve_query_response_ref: 
+```
+
+### 3.3 Provider Realizes the Resource (Realized State)
+
+OpenStack EU-WEST-Prod-1 receives the payload, naturalizes it to OpenStack format, provisions the VM, and returns the denaturalized result:
+
+```yaml
+# Event written to Realized Store event stream (entity_uuid key)
+
+event_type: REALIZED
+entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0
+realized_at: 2026-03-15T09:03:12Z
+provider_uuid: eu-west-prod-1-provider-uuid
+
+# DCM unified fields
+cpu_count: { value: 4, provenance: { ...plus provider attribution } }
+memory_gb: { value: 8, provenance: {...} }
+storage_gb: { value: 100, provenance: {...} }
+
+# Provider-added fields (not in Requested State โ€” added by provider after realization)
+provider_entity_id: "vm-0a1b2c3d"              # OpenStack's internal VM ID
+assigned_ip_address: "10.1.45.23"              # IP assigned by provider at realization
+hypervisor_host: "compute-node-07.eu-west"      # where the VM was physically placed
+actual_storage_gb: 102                          # actual allocated (rounded up)
+console_url: "https://console.eu-west.example.com/vm/0a1b2c3d"
+```
+
+### 3.4 Discovery Cycle (Discovered State)
+
+24 hours after realization, the discovery cycle runs:
+
+```yaml
+# Snapshot written to Discovered Store
+
+snapshot_type: DISCOVERED
+entity_uuid: f5e6d7c8-e9f0-a1b2-c3d4-e5f6a7b8c9d0
+discovered_at: 2026-03-16T09:00:00Z
+discovery_method: openstack_api_query
+provider_uuid: eu-west-prod-1-provider-uuid
+
+cpu_count: 4          # matches Realized State โ€” no drift
+memory_gb: 8          # matches
+storage_gb: 102       # matches (actual_storage_gb from provider)
+provider_entity_id: "vm-0a1b2c3d"
+status: ACTIVE
+```
+
+Drift Detection runs field-by-field comparison: all fields match Realized State. No drift event generated.
+
+---
+
+## 4. Example 2 โ€” IP Address Allocation
+
+An allocation request showing the `allocation` ownership model (pool โ†’ owned allocation).
+
+```yaml
+# Consumer submits intent for an IP address
+# intent-store/a1b2c3d4-tenant/Network/IPAddress/ip-entity-uuid/intent.yaml
+
+spec:
+  requested_from: network                   # request from the network pool
+  address_family: IPv4
+  purpose: vm_interface
+  attachment_ref: f5e6d7c8-entity-uuid     # the VM this IP will be assigned to
+
+# Assembly runs โ€” placement engine finds eligible IPAddressPool
+# Pool: NetworkOps/Network/IPAddressPool/10.1.0.0-16 (owned by NetworkOps Tenant)
+# Available capacity: 65420 addresses
+
+# Provider carves allocation:
+# New entity created: IPAddress 10.1.45.23/32
+# Owned by: AppTeam Tenant (a1b2c3d4)
+# AllocationRecord relationship created:
+#   IPAddress 10.1.45.23/32 --[allocated_from]--> IPAddressPool 10.1.0.0/16
+
+# Realized State event for the new IPAddress entity:
+entity_uuid: ip-entity-uuid
+resource_type: Network.IPAddress
+ownership_model: allocation
+owned_by_tenant_uuid: a1b2c3d4-appteam-uuid    # AppTeam owns this
+allocated_from_pool_uuid: pool-entity-uuid      # NetworkOps owns the pool
+address: "10.1.45.23"
+prefix_length: 32
+address_family: IPv4
+```
+
+When AppTeam decommissions their VM, the IP address entity can also be decommissioned. The pool's available capacity increases by 1. NetworkOps Tenant is unaffected.
+
+---
+
+## 5. Example 3 โ€” VLAN Attachment (Shareable)
+
+A VM attaches to an existing VLAN โ€” the `shareable` ownership model (stake, not ownership).
+
+```yaml
+# VLAN-100 exists โ€” owned by NetworkOps Tenant
+# entity_uuid: vlan-100-entity-uuid
+# ownership_model: shareable
+
+# Consumer (AppTeam) requests VM attachment to VLAN-100
+# No new VLAN entity is created โ€” a stake relationship is established:
+
+relationship:
+  type: attached_to
+  source_entity_uuid: f5e6d7c8-vm-entity-uuid   # AppTeam's VM
+  target_entity_uuid: vlan-100-entity-uuid        # NetworkOps's VLAN
+  source_tenant_uuid: a1b2c3d4-appteam-uuid
+  target_tenant_uuid: netops-tenant-uuid
+  stake:
+    is_active: true
+    stake_strength: required                       # VM cannot function without VLAN
+    staked_at: 2026-03-15T09:03:12Z
+
+# If NetworkOps tries to decommission VLAN-100:
+# active required stakes: 3 (VM-A, VM-B, VM-C all have required stakes)
+# Result: DECOMMISSION_DEFERRED
+# NetworkOps notified: "VLAN-100 has 3 required stakeholders. Decommission deferred."
+# Each stakeholder (AppTeam, DevTeam, OpsTeam) notified:
+# "NetworkOps has requested decommission of VLAN-100. Please migrate your workloads."
+```
+
+---
+
+## 6. Example 4 โ€” Brownfield Ingestion
+
+A VM discovered by the provider that DCM did not provision is brought under DCM lifecycle management.
+
+```yaml
+# Step 1: INGEST โ€” discovery finds unknown VM
+discovered_entity:
+  provider_entity_id: "vm-legacy-0001"
+  resource_type: Compute.VirtualMachine
+  lifecycle_state: OPERATIONAL          # it's running
+  discovered_at: 2026-03-15T06:00:00Z
+  discovery_confidence: low             # no DCM provenance
+  transitional_tenant: __transitional__ # held in transitional Tenant during ingestion
+
+# Step 2: ENRICH โ€” CMDB Information Provider enriches the entity
+# CMDB lookup by IP address finds the business owner record:
+enrichment:
+  owner_business_unit: "Payments Platform"
+  cost_center: "PAYM-4421"
+  product_owner: "Jane Smith"
+  compliance_scope: PCI-DSS
+  confidence_descriptor:
+    authority_level: primary            # CMDB is primary authority for ownership data
+    corroboration: single_source        # only CMDB has this data
+    source_trust: verified
+
+# Step 3: PROMOTE โ€” operator assigns to AppTeam Tenant, creates entity record
+promotion:
+  target_tenant_uuid: a1b2c3d4-appteam-uuid
+  created_via: ingestion
+  intent_state_created: true            # Intent State created from discovered configuration
+  provenance_basis: discovered          # provenance chain starts from discovery
+  promoted_by: operator-actor-uuid
+  promoted_at: 2026-03-15T11:30:00Z
+```
+
+After promotion, the entity is a full DCM-managed entity. Drift detection is active. The operator can now request updates (targeted delta) or decommission through DCM.
+
+---
+
+## 7. Example 5 โ€” Drift Detection and Remediation
+
+Six hours after the VM from Example 1 was realized, discovery finds a discrepancy:
+
+```yaml
+# Discovery finds:
+cpu_count: 4       # matches
+memory_gb: 16      # DRIFT โ€” realized says 8, discovered says 16
+
+# Drift record created:
+drift_record:
+  entity_uuid: f5e6d7c8-entity-uuid
+  detected_at: 2026-03-15T15:00:00Z
+  drifted_fields:
+    - field_path: memory_gb
+      realized_value: 8
+      discovered_value: 16
+  drift_severity: significant      # memory doubling is significant
+  unsanctioned: true               # no DCM Requested State explains this change
+```
+
+**Policy Engine evaluates the drift record:**
+
+```yaml
+# Drift response policy for Compute.VirtualMachine at significant severity:
+# action: ESCALATE for unsanctioned changes
+
+escalation:
+  entity_uuid: f5e6d7c8-entity-uuid
+  notified:
+    - actor: b2c3d4e5-consumer-actor   # the entity owner
+    - actor: appteam-admin-actor        # AppTeam admin
+    - actor: sre-oncall-actor           # SRE on-call
+  escalation_reason: "Unsanctioned memory change: 8Gi โ†’ 16Gi"
+  resolution_options:
+    - REVERT: "Submit rehydration from Realized State to restore 8Gi memory"
+    - UPDATE_DEFINITION: "Promote discovered state โ€” update entity definition to 16Gi"
+    - ACCEPT: "Accept the change; add to next review cycle"
+```
+
+The consumer reviews and chooses UPDATE_DEFINITION โ€” the memory was legitimately increased by the infrastructure team for a critical workload. They submit an UPDATE_DEFINITION resolution, which creates a new Requested State reflecting 16Gi memory and updates the Realized State record. Future drift detection will compare against 16Gi.
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/04b-ownership-sharing-allocation.md b/content/docs/data-model/04b-ownership-sharing-allocation.md
new file mode 100644
index 0000000..23b1f8f
--- /dev/null
+++ b/content/docs/data-model/04b-ownership-sharing-allocation.md
@@ -0,0 +1,312 @@
+# DCM Data Model โ€” Ownership, Sharing, and Allocation
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA**
+>
+> The Data abstraction โ€” ownership models for entity data
+
+
+**Related Documents:** [Entity Types](01-entity-types.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md)
+
+---
+
+## 1. Purpose
+
+This document defines the complete ownership model for DCM entities โ€” specifically the three ownership patterns that govern how resources are owned, shared, and allocated across Tenants. It establishes precise vocabulary and clear boundaries between concepts that are frequently conflated:
+
+- **Ownership** โ€” who is accountable for a resource's lifecycle and costs
+- **Shareable** โ€” multiple entities have a stake in a single resource that they do not own
+- **Allocatable** โ€” a pool resource yields independently owned sub-resources to consumers
+
+Getting this model right is foundational. It governs decommission safety (can this resource be removed?), cost attribution (who pays for this?), cross-tenant visibility (who can see this?), drift accountability (whose responsibility is remediation?), and placement decisions (which providers serve allocation requests?).
+
+---
+
+## 2. The Three Ownership Patterns
+
+### 2.1 Whole Allocation (Consumer Owns the Entity)
+
+**What it means:** The consumer receives the entire resource entity. It belongs exclusively to them. They own it outright โ€” it is in their Tenant, they control its lifecycle, they bear its costs.
+
+**Structural model:**
+```
+Platform Tenant owns: the infrastructure, compute capacity, network fabric
+Consumer Tenant owns: VirtualMachine-A (a distinct entity)
+Consumer Tenant owns: VirtualMachine-B (another distinct entity)
+```
+There is no relationship between VirtualMachine-A and the infrastructure Tenant โ€” the consumer simply used DCM's Service Provider to provision a resource. Once provisioned, the entity belongs to the consumer's Tenant entirely. The platform Tenant has no visibility into the consumer's entity unless an explicit Information Provider or cross-tenant relationship is established.
+
+**Examples:** VirtualMachine, Container, StorageVolume, NetworkInterface, DNSRecord.
+
+**Resource Type Spec declaration:**
+```yaml
+resource_type_spec:
+  fqn: Compute.VirtualMachine
+  ownership_model: whole_allocation
+  # Each consumer request produces an entity owned entirely by the requesting Tenant
+```
+
+**Decommission behavior:** Straightforward. The owning Tenant decommissions the entity. The provider releases the underlying physical resources. No other Tenant is affected.
+
+---
+
+### 2.2 Allocation (Consumer Owns a Carved Portion)
+
+**What it means:** A *pool* resource is owned by a platform or provider Tenant (the pool owner). When a consumer requests a resource of this type, they receive an *allocation* โ€” a new, distinct entity carved from the pool. The consumer **owns** their allocation. The pool owner retains ownership of the pool.
+
+**The key distinction from Shareable:** The consumer's allocation is an independent entity with its own UUID, its own Tenant membership, its own lifecycle. It is not a reference to the pool โ€” it is a new thing that came from the pool.
+
+**Structural model:**
+```
+NetworkOps Tenant owns: IPAddressPool 10.0.0.0/16 (pool entity)
+  โ”‚
+  โ”œโ”€โ”€ AppTeam Tenant owns: IPAddress 10.0.1.45/32  โ† allocation entity (new UUID, AppTeam's Tenant)
+  โ”œโ”€โ”€ DevTeam Tenant owns: IPAddress 10.0.1.46/32  โ† allocation entity (new UUID, DevTeam's Tenant)
+  โ””โ”€โ”€ OpsTeam Tenant owns: IPAddress 10.0.1.47/32  โ† allocation entity (new UUID, OpsTeam's Tenant)
+```
+
+The IPAddress entities are not pointers to the pool โ€” they are real entities owned by their Tenants. If AppTeam decommissions 10.0.1.45/32, it is released back to the pool. The pool capacity increases. No other Tenant's allocation is affected.
+
+**Examples:** IPAddress (from IPAddressPool), Subnet (from SubnetPool), VLAN ID (from VLANIDPool), StorageVolume (from StoragePool), PublicCertificate (from CertificateAuthorityPool).
+
+**Resource Type Spec declarations (both pool and allocation):**
+```yaml
+# The pool resource type
+resource_type_spec:
+  fqn: Network.IPAddressPool
+  ownership_model: whole_allocation    # the pool entity is owned outright by the platform Tenant
+  is_pool: true
+  allocation_produces_type: Network.IPAddress
+  capacity_tracking: true              # DCM tracks used/available capacity
+
+# The allocation resource type
+resource_type_spec:
+  fqn: Network.IPAddress
+  ownership_model: allocation          # each instance is an allocation from a pool
+  allocated_from_pool_type: Network.IPAddressPool
+  # When a consumer requests Network.IPAddress, DCM:
+  # 1. Runs placement to find an eligible IPAddressPool
+  # 2. The pool provider carves out a specific IP
+  # 3. DCM creates a new IPAddress entity owned by the requesting Tenant
+  # 4. Records an AllocationRecord relationship between the entity and the pool
+```
+
+**AllocationRecord relationship:**
+Every allocation entity carries an `allocated_from` relationship to its source pool:
+
+```yaml
+relationship:
+  relationship_uuid: 
+  type: allocated_from
+  source_entity_uuid:        # the allocation (e.g., IPAddress 10.0.1.45/32)
+  target_entity_uuid:        # the pool (e.g., IPAddressPool 10.0.0.0/16)
+  source_tenant_uuid:        # AppTeam Tenant
+  target_tenant_uuid:        # NetworkOps Tenant โ€” cross-tenant relationship
+  allocation_ref:
+    allocation_size: "1/32"        # what was carved from the pool
+    allocated_at: 
+    allocated_by_actor_uuid: 
+    allocation_metadata:
+      pool_capacity_before: 65534
+      pool_capacity_after: 65533
+```
+
+**Decommission behavior:** When the consumer decommissions their allocation entity, DCM dispatches a decommission payload to the provider. The provider releases the specific allocated resource back to the pool. The pool's available capacity increases. The AllocationRecord relationship is terminated. The allocation entity enters DECOMMISSIONED state. The pool entity is unaffected.
+
+**Cross-tenant visibility:** The allocation entity is in the consumer's Tenant. The consumer cannot see the pool entity unless an explicit cross-tenant relationship or Information Provider is configured. The pool owner can see allocation counts and capacity via the Cost Analysis component and the provider's capacity reporting API โ€” they cannot see the consumer's entity data.
+
+---
+
+### 2.3 Shareable (Consumer Has a Stake, Not Ownership)
+
+**What it means:** A single resource entity is owned by one Tenant (the resource owner) and multiple consumers attach to, depend on, or reference it. Consumers have a *stake* โ€” a relationship that affects the resource's lifecycle โ€” but they do not own any portion of it. The resource's lifecycle is governed entirely by its owner.
+
+**The key distinction from Allocation:** No new entity is created for the consumer. The consumer receives a relationship to the existing resource, not a new sub-entity. The consumer does not own anything โ€” they hold a stake.
+
+**Structural model:**
+```
+NetworkOps Tenant owns: VLAN-100 (single entity โ€” there is only one VLAN-100)
+  โ”‚
+  โ”œโ”€โ”€ AppTeam has stake: VM-A attached to VLAN-100   (relationship, not ownership)
+  โ”œโ”€โ”€ DevTeam has stake: VM-B attached to VLAN-100   (relationship, not ownership)
+  โ””โ”€โ”€ OpsTeam has stake: VM-C attached to VLAN-100   (relationship, not ownership)
+```
+
+VLAN-100 belongs to NetworkOps. AppTeam, DevTeam, and OpsTeam each have a VM attached to it. If DevTeam decommissions VM-B, VLAN-100 is unaffected โ€” it still exists and serves VM-A and VM-C. If NetworkOps wants to decommission VLAN-100, they cannot do so while VMs are attached. Decommission is deferred until all stakes are released.
+
+**Examples:** VLAN (network fabric shared by many VMs), NetworkSegment, SharedFileSystem, DNS Zone, NTP Server, Certificate Authority (as a service), Transit Gateway.
+
+**Resource Type Spec declaration:**
+```yaml
+resource_type_spec:
+  fqn: Network.VLAN
+  ownership_model: shareable
+  # A single VLAN entity exists; consumers attach to it via relationships
+  # Consumers do not receive their own VLAN entity
+  decommission_policy:
+    defer_while_active_stakes: true
+    minimum_stake_count: 0       # can decommission when all stakes released
+    # Some resources may require minimum_stake_count: 1
+    # e.g., a DNS Zone that should never be empty
+```
+
+**Stake relationship:**
+```yaml
+relationship:
+  relationship_uuid: 
+  type: attached_to          # or: depends_on, references, uses
+  source_entity_uuid:    # VM-A (consumer's entity)
+  target_entity_uuid:    # VLAN-100 (shared resource)
+  source_tenant_uuid:    # AppTeam Tenant
+  target_tenant_uuid:    # NetworkOps Tenant
+  stake:
+    is_active: true
+    staked_at: 
+    staked_by_actor_uuid: 
+    stake_strength: 
+    # required: VM cannot function without VLAN attachment (blocks VLAN decommission)
+    # preferred: VM prefers attachment but can function without it
+    # optional: informational stake only
+```
+
+**Decommission behavior:** VLAN-100 cannot be decommissioned while any `stake_strength: required` stakes exist. The decommission attempt is deferred โ€” not rejected โ€” and a `DECOMMISSION_DEFERRED` event is generated. DCM notifies the resource owner of all active required stakes. The owner can request that stakeholders release their stakes (by decommissioning their VMs or migrating to a different VLAN) before decommission proceeds.
+
+**Cross-tenant visibility:** The consumer's VM can reference VLAN-100 (read access for attachment purposes). The consumer cannot modify VLAN-100 or see its owner's configuration details unless an explicit cross-tenant authorization grants that. The resource owner (NetworkOps) can see all active stakes on their resource โ€” this is how they know which consumers are affected by a planned decommission.
+
+---
+
+## 3. Hybrid Case โ€” Allocation from a Shareable Pool
+
+Some resources combine both patterns. A Subnet Pool is an allocatable pool. Each allocation (a specific /28) is owned by the consumer. But the /28 has a stake relationship to the parent /16 (which is shareable โ€” owned by the network team, referenced by all subnets).
+
+```
+NetworkOps Tenant owns: SupernetPool 10.0.0.0/8 (allocatable pool)
+  โ”‚
+  โ”œโ”€โ”€ NetworkOps Tenant owns: 10.0.0.0/16 (allocation from /8 โ€” NetworkOps-owned)
+  โ”‚     NetworkOps Tenant owns: 10.0.0.0/24 (allocation from /16 โ€” NetworkOps-owned)
+  โ”‚
+  โ””โ”€โ”€ NetworkOps Tenant owns: 10.1.0.0/16 (allocation from /8 โ€” NetworkOps-owned, shared)
+        โ”œโ”€โ”€ AppTeam Tenant owns: 10.1.0.0/24  โ† consumer allocation (owned by AppTeam)
+        โ”‚     โ””โ”€โ”€ stake: attached to 10.1.0.0/16 (shareable โ€” NetworkOps)
+        โ””โ”€โ”€ DevTeam Tenant owns: 10.1.1.0/24  โ† consumer allocation (owned by DevTeam)
+              โ””โ”€โ”€ stake: attached to 10.1.0.0/16 (shareable โ€” NetworkOps)
+```
+
+The consumer-facing 10.1.0.0/24 is an allocation โ€” AppTeam owns it. The parent 10.1.0.0/16 is shareable โ€” NetworkOps owns it, AppTeam and DevTeam both have stakes. The 10.1.0.0/24 has both an `allocated_from` relationship (to the /24 pool that produced it) and an `attached_to` relationship (stake in the parent /16).
+
+---
+
+## 4. Ownership Model Summary
+
+| Pattern | Consumer Gets | Consumer Owns | New Entity Created | Lifecycle Governed By | Cost Attribution |
+|---------|--------------|--------------|-------------------|----------------------|-----------------|
+| **Whole Allocation** | The entire resource | Yes, outright | Yes (same type) | Consumer Tenant | Consumer Tenant |
+| **Allocation** | A carved sub-resource | Yes, the allocation | Yes (sub-type) | Consumer Tenant | Consumer Tenant |
+| **Shareable** | A stake/relationship | No โ€” stake only | No new entity | Resource Owner Tenant | Resource Owner Tenant (shared cost attribution possible via policy) |
+
+---
+
+## 5. Placement Engine Interaction
+
+The placement engine handles all three ownership models, but the selection criteria differ:
+
+**Whole Allocation:** Standard placement. The placement engine selects a provider with available capacity. The provider provisions the resource and returns it owned by the requesting Tenant.
+
+**Allocation:** The placement engine selects an eligible pool resource owned by a platform Tenant with sufficient available capacity. The pool provider carves an allocation and returns it. DCM creates the new allocation entity in the requesting Tenant.
+
+```yaml
+# Placement engine for allocation requests:
+# Step 1: Find providers that offer Network.IPAddressPool
+# Step 2: Filter by sovereignty constraints
+# Step 3: Filter by available capacity (pool.available_count > 0)
+# Step 4: Apply tie-breaking hierarchy
+# Step 5: Dispatch allocation request to selected pool provider
+# Step 6: Provider returns the specific carved allocation
+# Step 7: DCM creates IPAddress entity in requesting Tenant
+# Step 8: AllocationRecord relationship created
+```
+
+**Shareable:** The placement engine finds the shareable resource instance that satisfies the consumer's attachment constraints. No new entity is provisioned โ€” the provider registers the stake relationship. If no eligible shareable instance exists, the request fails with a clear error (unlike Allocation where failure means insufficient pool capacity).
+
+---
+
+## 6. Decommission Safety Model
+
+The three patterns have different decommission safety behaviors:
+
+**Whole Allocation decommission:**
+- Owner Tenant initiates decommission
+- Policy checks for required relationships (do other entities depend on this?)
+- If required dependencies exist โ†’ `DECOMMISSION_DEFERRED` until dependencies release
+- If no required dependencies โ†’ dispatch decommission to provider โ†’ DECOMMISSIONED
+
+**Allocation decommission:**
+- Consumer Tenant initiates decommission of their allocation entity
+- DCM dispatches decommission to pool provider
+- Pool provider releases the resource back to pool
+- AllocationRecord relationship terminated
+- Pool `available_count` increases
+- Allocation entity โ†’ DECOMMISSIONED
+- Pool entity is unaffected
+
+**Shareable decommission:**
+- Resource owner Tenant initiates decommission of the shared resource
+- DCM checks `active_stake_count` for `stake_strength: required` stakes
+- If required stakes > 0 โ†’ `DECOMMISSION_DEFERRED`
+  - Notifications to all required stakeholders
+  - Owner retries decommission after stakeholders release
+- If required stakes == 0 โ†’ dispatch decommission to provider โ†’ DECOMMISSIONED
+  - Any remaining `optional` stakes are automatically terminated
+
+---
+
+## 7. Cost Attribution Model
+
+**Whole Allocation and Allocation:** Cost is attributed entirely to the owning Tenant. Standard Cost Analysis billing. The entity's `billing_state` governs the rate (billable/non_billable/reduced_rate).
+
+**Shareable:** The shared resource's cost is attributed to its owner Tenant by default. Organizations that want to distribute shared resource costs to stakeholders configure a cost attribution policy:
+
+```yaml
+cost_attribution_policy:
+  resource_type: Network.VLAN
+  model: 
+  # owner_bears_all: NetworkOps Tenant pays for VLAN regardless of how many VMs attach
+  # equal_split:     cost divided equally among active stakeholders
+  # proportional_by_usage: cost allocated by traffic volume or similar metric
+  # chargeback:      each stakeholder is invoiced for their declared portion
+```
+
+---
+
+## 8. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `OWN-001` | `whole_allocation` resources are owned entirely by the requesting Tenant from the moment of realization. The providing platform Tenant has no ownership claim. |
+| `OWN-002` | `allocation` resources are owned entirely by the requesting Tenant. The pool owner retains ownership of the pool entity only. AllocationRecord relationships are the only cross-Tenant link. |
+| `OWN-003` | `shareable` resources are owned by a single Tenant. Consumers hold stakes (relationships) only. No consumer owns any portion of a shareable resource. |
+| `OWN-004` | Decommission of a shareable resource is deferred while any `required` strength stakes are active. Optional stakes are terminated automatically on shareable resource decommission. |
+| `OWN-005` | Allocation entity decommission releases the allocation back to the source pool. The pool entity is never decommissioned by an allocation decommission. |
+| `OWN-006` | Cost attribution for shareable resources defaults to the resource owner Tenant. A cost attribution policy may redistribute costs to stakeholders. |
+| `OWN-007` | The ownership model for a resource type is declared in the Resource Type Specification and cannot be changed at the entity level. Ownership model is a type-level invariant. |
+| `OWN-008` | Cross-tenant AllocationRecord and stake relationships require that the requesting Tenant has either an active cross-tenant authorization or the resource type is declared as publicly allocatable/stakeable in its Resource Type Spec. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/05-resource-type-hierarchy.md b/content/docs/data-model/05-resource-type-hierarchy.md
new file mode 100644
index 0000000..2591aaf
--- /dev/null
+++ b/content/docs/data-model/05-resource-type-hierarchy.md
@@ -0,0 +1,504 @@
+# DCM Data Model โ€” Resource Type Hierarchy and Service Catalog
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Entity Types](01-entity-types.md) | [Four States](02-four-states.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Examples](04-examples.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA**
+>
+> The Data abstraction โ€” Resource Type Specifications and Provider Catalog Items
+
+
+
+---
+
+## 1. Purpose
+
+The DCM Resource Type Hierarchy is the structural model that defines how services and resources are categorized, specified, and exposed through the DCM Service Catalog. It is the mechanism by which DCM achieves **resource portability** โ€” the ability to express what a consumer needs independently of which specific provider delivers it.
+
+The hierarchy serves four goals:
+
+1. **Portability** โ€” consumer intent can be fulfilled by any provider that satisfies the resource type contract, without the consumer needing to know which provider that is
+2. **Standardization** โ€” a common vocabulary and data contract for all resource types encourages interoperability across providers, implementors, and the broader DCM community
+3. **Extensibility** โ€” the model can be extended at every level without breaking existing definitions
+4. **Transparency** โ€” any deviation from full portability is explicitly declared, versioned, and surfaced to consumers
+
+---
+
+
+### 1a. Precise Vocabulary โ€” Resource Type vs Catalog Item
+
+These terms are frequently conflated. The distinction is architectural:
+
+**Resource Type** โ€” the classification category. Groups catalog items for portability and discovery. Vendor-neutral by requirement. Defines the field schema that any provider offering this type must support. Examples: `Compute.VirtualMachine`, `Network.IPAddress`, `Process.AnsiblePlaybook`.
+
+**Resource Type Specification** โ€” the versioned, formal definition of a Resource Type: field schema, constraints, lifecycle rules, portability classification, and allowed relationship types. Stored in the Resource Type Registry. Providers implement against a specific version. Example: `Compute.VirtualMachine v2.1.0`.
+
+**Provider Catalog Item** โ€” what a specific Service Provider is offering to consumers. The provider's declaration: "I can fulfill `Compute.VirtualMachine v2.1.0` with these specific options, at this cost, with these availability characteristics, in this region." A catalog item is always linked to a specific Resource Type Specification version. Catalog items can represent resource allocations (a VM, a subnet) or processes (an automation job, a playbook execution, a pipeline run) โ€” anything a provider offers for consumption.
+
+**The key relationship:** Consumers request by Resource Type (or Resource Type Specification version). DCM resolves to a Provider Catalog Item through the specificity narrowing algorithm. The catalog item is what actually gets provisioned. The resource type is the portable, vendor-neutral expression of intent.
+
+**Anti-vocabulary update:** Never say "catalog item" when you mean "resource type specification." Never say "resource type" when you mean a specific provider offering โ€” use "catalog item" or "provider catalog item."
+
+
+## 2. The DCM Resource Type Registry
+
+DCM maintains an official **Resource Type Registry** โ€” the authoritative source of standard resource type definitions. The registry is the foundation of portability across the DCM ecosystem.
+
+
+### 2.1a Catalog Item vs Resource Type Specification โ€” Critical Distinction
+
+These two terms are frequently conflated throughout the documentation. They are distinct concepts at different levels of the hierarchy:
+
+**Resource Type Specification (Registry entry):**
+- Vendor-neutral definition of a resource type's fields, constraints, lifecycle rules, and portability classification
+- Lives in the Resource Type Registry (Tier 1, 2, or 3)
+- Examples: `Compute.VirtualMachine v2.1.0`, `Network.VLAN v1.0.0`
+- Defines what the resource TYPE is, not what any specific provider offers
+
+**Provider Catalog Item (Service Catalog entry):**
+- A specific provider's offering implementing a Resource Type Specification
+- Includes provider-specific pricing, availability, SLAs, and performance characteristics
+- What consumers actually request via the Service Catalog
+- Examples: "EU-WEST-Prod-1's 4-CPU VM offering", "NetworkOps's VLAN service"
+- Tied to a specific provider; multiple providers can offer catalog items for the same Resource Type Spec
+
+**When to use each term:**
+- "The consumer requests a catalog item" โœ“ โ€” they request a provider's specific offering
+- "The resource type specification defines the field schema" โœ“ โ€” the spec defines structure
+- "The catalog item schema" โœ— โ€” should be "the resource type specification schema"
+- "The consumer browses resource types" โœ“ โ€” they browse the type hierarchy
+- "The consumer selects a catalog item" โœ“ โ€” they select a specific provider offering
+
+**In the anti-vocabulary:** "Catalog Item" should not be used when "Resource Type Specification" is meant, and vice versa. The hierarchy is: Resource Type Category โ†’ Resource Type โ†’ Resource Type Specification โ†’ Provider Catalog Item.
+
+
+### 2.1 Registry Principles
+
+- The registry is **open** โ€” third parties, implementors, and the community can propose new resource type definitions
+- Registry entries are **versioned and immutable** once published โ€” changes produce new versions
+- Registry definitions are **vendor-neutral by hard requirement** โ€” no vendor-specific data is permitted in a DCM-specified resource type unless that vendor is the exclusive provider of that technology stack
+- The registry itself is subject to the same **deprecation model** as all other DCM definitions
+- All registry entries follow the **universal versioning scheme** (Major.Minor.Revision)
+
+### 2.2 Default Resource Type Categories
+
+DCM ships with a default set of Resource Type Categories. Implementors may define additional categories following the specification. The registry contains both **Resource Types** (for provisioned resources) and **Information Types** (for external data references) โ€” distinguished by category prefix.
+
+**Resource Type Categories:**
+
+| Category | Description |
+|----------|-------------|
+| `Compute` | Processing resources โ€” virtual machines, containers, bare metal |
+| `Network` | Networking resources โ€” IP addresses, VLANs, firewall rules, load balancers |
+| `Storage` | Storage resources โ€” block, object, file, databases |
+| `Platform` | Platform services โ€” Kubernetes clusters, application platforms |
+| `Security` | Security resources โ€” certificates, secrets, HSMs, identity |
+| `Observability` | Monitoring and logging resources |
+| `Data` | Data services โ€” streams, queues, pipelines |
+
+**Information Type Categories:**
+
+| Category | Description |
+|----------|-------------|
+| `Business` | Business organizational data โ€” BusinessUnit, CostCenter, ProductOwner |
+| `Identity` | Identity and access data โ€” Person, ServiceAccount, Group |
+| `Compliance` | Regulatory and compliance data โ€” RegulatoryScope, AuditFramework |
+| `Operations` | Operational reference data โ€” Runbook, SLA, SupportContract |
+
+All categories follow the same versioning, deprecation, and registry governance model. The `implements_type` field on provider registrations distinguishes whether a provider is a Service Provider (`service`) or an Information Provider (`information`).
+
+### 2.3 Registry Entry Structure
+
+Every entry in the Resource Type Registry carries the following metadata:
+
+```yaml
+registry_entry:
+  uuid: 
+  name: 
+  fully_qualified_name: 
+  version: 
+  parent_uuid: 
+  status:
+    state: 
+    deprecation_date: 
+    sunset_date: 
+    replacement_uuid: 
+    replacement_version: 
+    deprecation_reason: 
+    migration_guidance: 
+  portability:
+    classification: 
+    portability_breaking: 
+    portability_notes: 
+  ownership:
+    owner: 
+    owner_uuid: 
+    origination_date: 
+  description: 
+  specification_ref: 
+```
+
+---
+
+## 3. Resource Type Hierarchy Levels
+
+The hierarchy has four levels, from most abstract to most concrete. Each level builds on the one above it.
+
+### Level 1 โ€” Resource Type Category
+
+The broadest classification. Defines the domain of a resource without any specificity about what the resource is.
+
+- DCM ships with default categories (see Section 2.2)
+- Implementors may define additional categories
+- Categories have no data fields โ€” they are organizational containers
+- Categories are versioned and can be deprecated
+
+**Example:** `Compute`, `Network`, `Storage`
+
+---
+
+### Level 2 โ€” Resource Type
+
+Defines an abstract resource within a category. A Resource Type represents a class of resource that multiple providers can implement. Resource Types are the primary unit of portability in DCM.
+
+- DCM maintains default Resource Types in the registry
+- Community and implementors can define and register new Resource Types
+- Resource Types must be **vendor-neutral** โ€” no provider-specific data
+- Resource Types declare their **base field specification** (universal fields only)
+- Resource Types are versioned and can be deprecated
+
+**Example:** `Compute.VirtualMachine`, `Network.IPAddress`, `Network.FirewallRule`
+
+---
+
+### Level 3 โ€” Resource Type Specification
+
+The data contract for a Resource Type. Defines all fields โ€” universal, conditional, and any declared extension points โ€” along with their types, constraints, and portability classifications.
+
+- Every field in a specification carries a **portability classification** (see Section 4)
+- Specifications define which fields are required vs. optional
+- Specifications define validation constraints for each field
+- Specifications declare **extension points** where providers may add fields
+- Specifications are versioned independently of their Resource Type
+- Specifications can be deprecated
+
+**Example:** `Compute.VirtualMachine` specification defines: `cpu_count` (universal, required), `ram_gb` (universal, required), `storage_gb` (universal, required), `os_image` (universal, required), `high_availability` (conditional, optional)
+
+---
+
+### Level 4 โ€” Provider Catalog Item
+
+A specific provider's concrete implementation of a Resource Type Specification. This is where provider-specific detail lives and where the abstract becomes actionable.
+
+- Provider Catalog Items are registered against a specific Resource Type Specification version
+- They must implement **all universal fields** of the parent specification
+- They may implement **conditional fields** (declared in their registration)
+- They may add **provider-specific extension fields** (must be marked portability-breaking)
+- They are versioned and can be deprecated
+- They declare their **sovereignty capabilities** (see Section 6)
+- They declare their **supported lifecycle operations** (see Section 7)
+
+**Example:** `Nutanix.VM.Small` implements `Compute.VirtualMachine` with `cpu_count: 4`, `ram_gb: 16`, `storage_gb: 60`
+
+---
+
+## 4. Portability Classification
+
+Every field in every Resource Type Specification carries a portability classification. This classification is part of the field's metadata and is immutable once published for a given version.
+
+### 4.1 Classification Levels
+
+| Classification | Description | Portability Impact |
+|---|---|---|
+| `universal` | Part of the DCM standard spec. All providers implementing this type must support it. | Fully portable across all implementing providers |
+| `conditional` | Supported by multiple providers but not all. Providers declare support in their registration. | Portable across providers that declare support |
+| `provider-specific` | Specific to one provider or technology stack. Using this field locks the request to that provider. | Portability-breaking โ€” must be explicitly marked |
+| `exclusive` | Only one provider supports this technology stack. Portability is not applicable by definition. | Not applicable โ€” acknowledged and declared |
+
+### 4.2 Hard Portability Requirements
+
+The following are non-negotiable requirements for any DCM-specified Resource Type:
+
+1. All **universal** fields MUST be supported by ALL providers implementing that Resource Type
+2. **Provider-specific** fields MUST be explicitly marked as portability-breaking in the field metadata
+3. Consumers MUST be warned when their request contains portability-breaking fields
+4. The only exception to vendor-neutrality is the **exclusive** classification โ€” where one provider is the sole implementor of a technology stack, explicitly acknowledged and declared in the registry
+5. Any Resource Type in the DCM registry that contains provider-specific fields as universal fields is invalid and must be rejected
+
+### 4.3 Portability Field Metadata
+
+Every field in a Resource Type Specification carries the following portability metadata:
+
+```yaml
+field_name:
+  type: 
+  required: 
+  description: 
+  portability:
+    classification: 
+    portability_breaking: 
+    portability_notes: 
+    supported_by: 
+  constraints:
+    - 
+  default_value: 
+  provenance:
+    
+```
+
+---
+
+## 5. Inheritance Model
+
+Resource Types support inheritance, enabling specialization without duplication. A child type inherits all fields from its parent and may add new fields.
+
+### 5.1 Inheritance Rules
+
+1. A child type inherits **all fields** from its parent type โ€” no field can be removed or redefined
+2. A child type may **add new fields** beyond its parent's specification
+3. A child type's portability classification can only be **equal to or more restrictive** than its parent โ€” a child of a `universal` type may be `conditional`, but not vice versa
+4. Each level of the hierarchy is **independently versioned**
+5. Each level maintains a **reference to its parent UUID and version**
+6. Deprecating a parent type **does not automatically deprecate child types** โ€” each must be independently deprecated with appropriate migration guidance
+
+### 5.2 Inheritance Example
+
+```
+Compute                                        # Category
+  โ””โ”€โ”€ VirtualMachine                           # Base Resource Type
+        โ”œโ”€โ”€ VirtualMachine.GPU                 # Inherits VirtualMachine
+        โ”‚     โ”œโ”€โ”€ gpu_count (conditional)
+        โ”‚     โ”œโ”€โ”€ gpu_memory_gb (conditional)
+        โ”‚     โ””โ”€โ”€ VirtualMachine.GPU.HighMemory  # Inherits VirtualMachine.GPU
+        โ”‚           โ””โ”€โ”€ extended_memory_gb (conditional)
+        โ””โ”€โ”€ VirtualMachine.HighAvailability    # Inherits VirtualMachine
+              โ”œโ”€โ”€ ha_mode (conditional)
+              โ””โ”€โ”€ failover_policy (conditional)
+```
+
+### 5.3 Inheritance Metadata
+
+Every Resource Type that inherits from a parent carries the following inheritance metadata:
+
+```yaml
+inheritance:
+  parent_uuid: 
+  parent_version: 
+  parent_fully_qualified_name: 
+  inherited_fields: 
+  added_fields: 
+```
+
+---
+
+## 6. Provider Registration and Catalog Item Declaration
+
+For a provider to participate in the DCM ecosystem and have its catalog items available for request resolution, it must register against the Resource Type Hierarchy.
+
+### 6.1 Provider Registration Declaration
+
+A provider's registration is a machine-readable declaration that DCM consumes to understand what the provider offers and how to route requests to it:
+
+```yaml
+provider_registration:
+  uuid: 
+  name: 
+  version: 
+  status:
+    state: 
+    deprecation_date: 
+    sunset_date: 
+    replacement_uuid: 
+    deprecation_reason: 
+    migration_guidance: 
+  catalog_items:
+    - 
+  sovereignty_capabilities:
+    
+  supported_lifecycle_operations:
+    
+  trust_declaration:
+    
+```
+
+### 6.2 Catalog Item Declaration
+
+Each catalog item a provider offers is declared against a specific Resource Type Specification version:
+
+```yaml
+catalog_item:
+  uuid: 
+  name: 
+  version: 
+  implements:
+    resource_type_uuid: 
+    resource_type_version: 
+    resource_type_fully_qualified_name: 
+  status:
+    state: 
+    deprecation_date: 
+    sunset_date: 
+    replacement_uuid: 
+    deprecation_reason: 
+    migration_guidance: 
+  universal_fields:
+    
+  conditional_fields_supported:
+    
+  provider_specific_extensions:
+    
+    
+  portability_warning: 
+```
+
+---
+
+## 7. Request Resolution โ€” Specificity Narrowing
+
+Provider selection in DCM is never explicit. The consumer declares intent using Resource Types and field values. The appropriate provider catalog item is selected by the DCM Policy Engine through progressive specificity narrowing.
+
+### 7.1 Resolution Steps
+
+```
+Step 1: Resource Type declared
+        โ†’ matches all providers implementing that Resource Type
+
+Step 2: Universal fields specified
+        โ†’ still matches all providers (all must support universal fields)
+
+Step 3: Conditional fields specified
+        โ†’ narrows to providers that declare support for those fields
+
+Step 4: Provider-specific fields used
+        โ†’ narrows to single provider
+        โ†’ portability warning issued and recorded in request provenance
+        โ†’ enforcement mode applied (block|warn|allow) per organizational policy
+
+Step 5: Placement and sovereignty constraints applied
+        โ†’ Policy Engine applies placement policies
+        โ†’ Provider sovereignty capabilities matched against request requirements
+        โ†’ Final provider catalog item selected
+
+Step 6: Provider catalog item UUID recorded in request payload provenance
+```
+
+### 7.2 Portability Warning Enforcement
+
+When a request contains portability-breaking fields, the Policy Engine applies the configured enforcement mode. This is organizational policy โ€” configurable at the organization, domain, or service level:
+
+| Enforcement Mode | Behavior |
+|---|---|
+| `block` | Request is rejected. Consumer must remove portability-breaking fields or explicitly acknowledge the lock-in. |
+| `warn` | Request proceeds. Portability warning is recorded in request provenance and surfaced to the consumer. |
+| `allow` | Request proceeds silently. Portability-breaking fields are still recorded in provenance but no warning is surfaced. |
+
+The enforcement mode is itself a versioned, auditable policy โ€” subject to the same provenance tracking as all other data in DCM.
+
+---
+
+## 8. Deprecation Model
+
+Every definition at every level of the Resource Type Hierarchy can be deprecated. Deprecation is a first-class concept in DCM โ€” not an afterthought.
+
+### 8.1 Deprecation Lifecycle
+
+```
+active โ†’ deprecated โ†’ retired
+```
+
+| State | Meaning | System Behavior |
+|---|---|---|
+| `active` | Definition is current and fully supported | Normal operation |
+| `deprecated` | Definition is being phased out. Replacement is available. | Deprecation warning surfaced to consumers. Requests still processed. Warning recorded in provenance. |
+| `retired` | Definition is no longer honored. | Requests using retired definitions are rejected by the Policy Engine. |
+
+### 8.2 Deprecation Cascade Rules
+
+- Deprecating a **Resource Type** does not automatically deprecate its child types or provider catalog items โ€” each must be independently deprecated
+- Deprecating a **Provider Catalog Item** does not affect other catalog items implementing the same Resource Type
+- Retiring a **Resource Type Specification version** causes all catalog items registered against that version to require re-registration against a current version
+- **Sunset dates** must provide sufficient migration runway โ€” minimum notice periods may be defined by organizational policy
+
+### 8.3 Migration Guidance Requirement
+
+Any definition marked `deprecated` MUST include:
+- A reference to the replacement definition (UUID and version)
+- A human-readable deprecation reason
+- Human-readable migration guidance explaining how to transition
+- A sunset date giving consumers time to migrate
+
+---
+
+## 9. Versioning
+
+All definitions in the Resource Type Hierarchy follow the universal DCM versioning scheme.
+
+### 9.1 Version Scheme
+
+`Major.Minor.Revision`
+
+| Component | Trigger |
+|---|---|
+| **Major** | Breaking changes to the contract โ€” removing fields, changing field types, changing required/optional status of universal fields |
+| **Minor** | Additive changes, backward compatible โ€” adding new optional fields, adding new conditional fields, adding new extension points |
+| **Revision** | Data or configuration changes with no contract impact โ€” updating descriptions, updating constraints that don't break existing data, updating metadata |
+
+### 9.2 Version Constraints in Requests
+
+Consumers and dependencies may declare version constraints in their requests:
+
+```yaml
+resource_type:
+  uuid: 
+  version_constraint: 
+  version: 
+```
+
+### 9.3 Version Immutability
+
+Once a version is published it is immutable. Any change โ€” even a documentation correction โ€” produces a new version. This applies to all definitions at all levels of the hierarchy.
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | What is the governance model for proposing and approving new Resource Types to the DCM registry? | Community adoption, quality control | โœ… Resolved โ€” three-tier registry (DCM Core / Verified Community / Organization); PR-based proposals with automated validation gates; shadow validation period before active promotion; see doc 20 (REG-001, REG-002) |
+| 2 | Should the registry support a formal review/approval workflow before a Resource Type becomes `active`? | Registry integrity, community trust | โœ… Resolved โ€” PR-based workflow with automated gates (schema, FQN conflict, dependency resolution) and mandatory shadow validation before active; review periods by change type; see doc 20 (REG-002) |
+| 3 | What is the minimum sunset period for deprecated definitions? | Migration planning, operational stability | โœ… Resolved โ€” default sunset policies REG-DP-002: Tier 1=P12M, Tier 2=P6M; overridable via standard policy priority; locked as immutable in fsi/sovereign profiles; see doc 20 |
+| 4 | Should version constraints in requests be strictly enforced or advisory? | Operational flexibility vs. predictability | โœ… Resolved โ€” strictly enforced; version_policy options: exact/compatible/latest_minor/latest; DCM never auto-upgrades across major versions; profile-governed defaults (fsi/sovereign=exact); see doc 20 (REG-004) |
+| 5 | How are conflicts resolved when multiple providers satisfy all narrowing criteria equally? | Request resolution determinism | โœ… Resolved โ€” six-step tie-breaking: policy preference โ†’ provider priority โ†’ tenant affinity โ†’ cost analysis (if available) โ†’ least loaded โ†’ consistent hash on request_uuid; see doc 20 (REG-005) |
+| 6 | Should the registry be distributed or centralized? How does this interact with sovereignty requirements? | Registry availability, sovereignty | โœ… Resolved โ€” federated model: DCM Project registry โ†’ Organization mirror โ†’ Sovereign DCM (offline/signed bundles); air-gap via signed bundle import; see doc 20 (REG-006) |
+
+---
+
+## 11. Related Concepts
+
+- **Portability** โ€” the ability to fulfill a resource intent using any provider that satisfies the resource type contract
+- **Naturalization** โ€” provider's responsibility to transform DCM unified data into provider-specific format
+- **Denaturalization** โ€” provider's responsibility to transform provider-specific results back into DCM unified format
+- **Sovereign Execution Posture** โ€” sovereignty capabilities declared in provider registration inform placement decisions
+- **Policy Engine** โ€” applies portability enforcement, placement policies, and request resolution logic
+- **Field-Level Provenance** โ€” every field modification during request resolution is recorded with source UUID and operation type
+- **Universal Versioning** โ€” Major.Minor.Revision applies to all definitions at all levels of the hierarchy
+- **Deprecation** โ€” universal model for phasing out definitions at any level with migration guidance
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/06-resource-service-entities.md b/content/docs/data-model/06-resource-service-entities.md
new file mode 100644
index 0000000..1a19392
--- /dev/null
+++ b/content/docs/data-model/06-resource-service-entities.md
@@ -0,0 +1,894 @@
+# DCM Data Model โ€” Resource/Service Entities
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Operational Models](24-operational-models.md) | [Entity Types](01-entity-types.md) | [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + PROVIDER**
+>
+> Data: entity lifecycle. Provider: lifecycle events and update notifications
+
+
+
+---
+
+## 1. Purpose
+
+This document defines the two fundamental transactional concepts in DCM โ€” the **Resource/Service Request** and the **Resource/Service Entity** โ€” and establishes the ownership models, lifecycle principles, and provider relationship rules that govern them.
+
+Understanding the distinction between a Request and an Entity, and understanding DCM's role as the authoritative owner of all resource data regardless of operational ownership, is essential to understanding how DCM achieves its core goals of auditability, lifecycle management, and sovereignty.
+
+---
+
+## 2. Core Terminology
+
+### 2.1 Resource/Service Request
+
+A **Resource/Service Request** is what a consumer submits to DCM โ€” the declared intent to consume a resource or service. It is the consumer side of the transaction.
+
+- Created when a consumer submits a request via the Web UI or Consumer API
+- Captured as the **Intent State** before any processing
+- Processed into the **Requested State** after assembly and policy validation
+- Is the initiating event that causes a Resource/Service Entity to be created
+
+A Request is not a thing โ€” it is an **instruction**. It describes what the consumer wants. The provider acts on the Requested State to produce an Entity.
+
+### 2.2 Resource/Service Entity
+
+A **Resource/Service Entity** is the "thing" produced by a provider as a result of fulfilling a Resource/Service Request. It is the provider side of the transaction โ€” the allocation made real.
+
+- Created when a provider fulfills a Requested State payload
+- Returned to DCM in unified data model format via Denaturalization
+- Captured as the **Realized State** in the Realized Store
+- Assigned to a **DCM Tenant** โ€” the ownership boundary
+- Has a UUID, full provenance chain, and complete lifecycle from creation to decommission
+- Is the unit of consumption, cost attribution, drift detection, and audit in DCM
+
+A Resource/Service Entity IS a thing โ€” it exists, it has state, it has an owner, and DCM manages its lifecycle.
+
+### 2.3 The Critical Distinction
+
+```
+Consumer submits        โ†’  Resource/Service REQUEST  โ†’  Intent/Requested State
+Provider fulfills       โ†’  Resource/Service ENTITY   โ†’  Realized State
+DCM manages lifecycle   โ†’  ENTITY persists            โ†’  Drift/Audit/Cost/Rehydration
+```
+
+---
+
+## 3. DCM as Authoritative Owner of All Resource Data
+
+This is the most fundamental principle governing Resource/Service Entities:
+
+**DCM is ALWAYS the system of record for Resource/Service Entity data. DCM is ALWAYS authoritative for the resource definition. DCM ALWAYS owns the lifecycle. This applies regardless of the operational ownership model.**
+
+The operational ownership model (described in Section 4) determines who has authority to operate on a Resource/Service Entity. It does not affect DCM's data ownership. Specifically:
+
+- DCM owns the **data definition** of every Resource/Service Entity โ€” what it is, what it should be, what it was
+- DCM owns the **lifecycle** โ€” from Requested through Realized to Decommissioned
+- DCM is **authoritative** โ€” if a provider reports a change DCM was not aware of, DCM acts on it according to policy
+- DCM acts as the **Tenant advocate** โ€” it protects the Tenant's interests in all provider interactions
+- Providers are **custodians** of the underlying infrastructure โ€” they are not the system of record
+
+**When a provider reports an unsanctioned change:**
+
+If a provider reports a state change that was not initiated by a DCM request, the Policy Engine evaluates the change and determines the appropriate response:
+
+| Response | Description |
+|----------|-------------|
+| `ALERT` | Notify appropriate personas โ€” Tenant owner, SRE, Auditor |
+| `REVERT` | Instruct provider to revert to DCM-declared realized state |
+| `UPDATE_DEFINITION` | Accept the change and update the realized state definition |
+| `INVESTIGATE` | Flag for human review before action |
+| `DECOMMISSION` | Initiate decommission if the change represents unrecoverable deviation |
+| `ESCALATE` | Escalate to higher policy tier for decision |
+
+The response is determined by Policy Engine evaluation against:
+- The Resource/Service definition
+- Service/Resource dependencies
+- Consumer preferences
+- Organizational and Tenant policies
+- Sovereignty requirements
+
+---
+
+## 4. Ownership Models
+
+DCM supports four ownership models for Resource/Service Entities. Every Provider Catalog Item must declare which ownership model(s) it supports. The ownership model is recorded in the Resource/Service Entity's provenance at creation time.
+
+### 4.1 Allocation Model
+
+The provider retains internal ownership of the underlying infrastructure. The consumer owns the Resource/Service Entity (the allocation) in their DCM Tenant. The provider can reclaim the underlying resource when the entity is decommissioned.
+
+**Characteristics:**
+- Provider retains asset ownership
+- Consumer owns the allocation โ€” the Entity in their DCM Tenant
+- Provider has reclaim rights on decommission
+- Underlying infrastructure may be shared or subdivided
+- DCM manages the Entity lifecycle; provider manages the underlying resource
+
+**Examples:** Virtual Machine, Container, Network Port, IP Address, Firewall Rule, Database Instance
+
+---
+
+### 4.2 Whole Allocation Model
+
+The entire physical or logical resource is allocated as a single indivisible unit to one consumer's DCM Tenant. The provider retains internal ownership but the consumer has exclusive use of the whole resource. The resource cannot be subdivided or shared during the allocation period.
+
+**Characteristics:**
+- Provider retains asset ownership
+- Consumer has exclusive, indivisible use
+- The resource is not shared or subdivided
+- Provider has reclaim rights on decommission
+- DCM manages the Entity lifecycle
+
+**Examples:** Dedicated Bare Metal server (provider-owned), Dedicated Network appliance, Whole storage array allocation
+
+---
+
+### 4.3 Full Transfer Model
+
+The provider transfers complete ownership of the underlying resource to the consumer's DCM Tenant. The Resource/Service Entity IS the resource โ€” there is no separation between the allocation and the underlying infrastructure from DCM's perspective. The consumer controls the full lifecycle including decommissioning. The provider has no reclaim rights after transfer.
+
+**Characteristics:**
+- Ownership of the underlying resource transfers to consumer's DCM Tenant
+- The Entity IS the resource โ€” no allocation/infrastructure separation
+- Consumer controls full lifecycle including decommission
+- Provider has no reclaim rights post-transfer
+- Transfer is recorded in provenance โ€” permanent audit record
+- DCM remains authoritative for data and lifecycle regardless of transfer
+
+**Examples:** Transferred Bare Metal server, Licensed software asset, Dedicated hardware appliance transferred to consumer
+
+---
+
+### 4.4 Hybrid Transfer Model
+
+Ownership can transfer multiple times across the lifecycle of the Resource/Service Entity. The current owner is always exactly one DCM Tenant, but ownership can be formally reassigned through a DCM-governed ownership transfer process. Every transfer is tracked, auditable, and policy-governed.
+
+**Characteristics:**
+- Ownership is held by exactly one DCM Tenant at any point in time
+- Ownership can be transferred to another DCM Tenant through a formal DCM process
+- Every transfer is recorded in the Entity's provenance chain โ€” complete ownership history
+- Transfer requires Policy Engine validation and authorization
+- The receiving Tenant must accept the transfer โ€” it cannot be forced
+- DCM remains authoritative for data and lifecycle through all transfers
+
+**Transfer Provenance Record:**
+```yaml
+ownership_transfer:
+  sequence: 
+  from_tenant_uuid: 
+  to_tenant_uuid: 
+  transfer_timestamp: 
+  authorized_by: 
+  transfer_reason: 
+  policy_uuid: 
+```
+
+**Examples:** Bare Metal server reallocated between tenants, Hardware asset transferred between business units, Licensed resource reassigned
+
+---
+
+### 4.5 Ownership Model Declaration
+
+Every Provider Catalog Item must declare the ownership model(s) it supports:
+
+```yaml
+catalog_item:
+  uuid: 
+  ownership_models_supported:
+    - allocation
+    - whole_allocation
+    - full_transfer
+    - hybrid_transfer
+  default_ownership_model: 
+  transfer_policy_required: 
+  # If true, a policy must be referenced in any transfer request
+```
+
+---
+
+## 5. Resource/Service Entity Lifecycle
+
+Every Resource/Service Entity progresses through a defined lifecycle. The lifecycle states are:
+
+```
+REQUESTED โ†’ PENDING โ†’ PROVISIONING โ†’ REALIZED โ†’ OPERATIONAL
+                                                      โ”‚
+                                          โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+                                          โ–ผ           โ–ผ           โ–ผ
+                                      DEGRADED   MAINTENANCE  SUSPENDED
+                                          โ”‚           โ”‚           โ”‚
+                                          โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+                                                      โ–ผ
+                                                DECOMMISSIONING
+                                                      โ”‚
+                                                      โ–ผ
+                                                DECOMMISSIONED
+```
+
+| State | Description |
+|-------|-------------|
+| `REQUESTED` | Request submitted, Intent State captured |
+| `PENDING` | Requested State assembled, awaiting provider dispatch |
+| `PROVISIONING` | Provider is fulfilling the request |
+| `REALIZED` | Provider has fulfilled the request, Entity exists, Realized State captured |
+| `OPERATIONAL` | Entity is in active use |
+| `DEGRADED` | Entity is functioning but below expected operational characteristics |
+| `MAINTENANCE` | Entity is undergoing planned maintenance |
+| `SUSPENDED` | Entity is temporarily suspended โ€” not operational but not decommissioned |
+| `DECOMMISSIONING` | Decommission process initiated |
+| `DECOMMISSIONED` | Entity no longer exists. Record retained permanently for audit. |
+
+**Terminal states:** `DECOMMISSIONED` is the only terminal state. Once decommissioned, the Entity record is immutable and retained permanently.
+
+---
+
+## 6. Process Resource Entities
+
+A **Process Resource Entity** is a distinct class of Resource/Service Entity representing ephemeral execution resources โ€” automation jobs, playbooks, pipelines, workflows, and similar process-oriented resources.
+
+### 6.1 Characteristics
+
+- **Ephemeral lifecycle** โ€” exists for the duration of execution, then terminates
+- **No ongoing realized state to manage** โ€” lifecycle ends at COMPLETED or FAILED
+- **Execution record retained permanently** โ€” the record of what the process did is immutable and permanent
+- **Must belong to a DCM Tenant** โ€” even ephemeral resources must be owned
+- **Must be in the provenance chain** of any Resource/Service Entity they affect
+
+### 6.2 Process Resource Lifecycle
+
+```
+REQUESTED โ†’ INITIATED โ†’ EXECUTING โ†’ COMPLETED
+                                  โ†’ FAILED
+                                  โ†’ CANCELLED
+```
+
+| State | Description |
+|-------|-------------|
+| `REQUESTED` | Process request submitted |
+| `INITIATED` | Provider has begun execution |
+| `EXECUTING` | Process is actively running |
+| `COMPLETED` | Process completed successfully โ€” terminal |
+| `FAILED` | Process failed โ€” terminal |
+| `CANCELLED` | Process cancelled before completion โ€” terminal |
+
+All terminal states are permanent. The execution record is immutable after reaching a terminal state.
+
+### 6.3 Process Resource Entity Data Model
+
+```yaml
+process_resource_entity:
+  uuid: 
+  entity_class: process
+  process_type: 
+  tenant_uuid: 
+  version: 
+  lifecycle_state: 
+  input_payload:
+    
+  output_payload:
+    
+  affected_entities:
+    - entity_uuid: 
+      effect_type: 
+      effect_description: 
+  execution_record:
+    initiated_timestamp: 
+    completed_timestamp: 
+    executing_provider_uuid: 
+    authorized_by_policy_uuid: 
+  provenance:
+    
+```
+
+### 6.4 Provenance Obligation for Process Resources
+
+If a Process Resource modifies the state of a Resource/Service Entity, that Entity's realized state provenance MUST reference the Process Resource Entity UUID as the source of the modification. This ensures that every change to an Infrastructure Entity can be traced back to the Process that caused it.
+
+---
+
+## 7. Provider Internal Lifecycle Model
+
+Providers have their own internal infrastructure that underpins the Resource/Service Entities they create. While that internal infrastructure is opaque to consumers, DCM needs visibility into it for placement, cost analysis, and operational governance.
+
+### 7.1 Provider Capacity Model
+
+DCM supports three capacity information modes. Mode 3 is mandatory for all providers. Modes 1 and 2 are configurable per provider registration.
+
+**Mode 1 โ€” Dynamic Query (on-demand)**
+DCM queries the provider for current capacity as part of request processing. Used when real-time accuracy is critical or when the provider cannot maintain a registration schedule.
+
+```yaml
+capacity_query_response:
+  provider_uuid: 
+  resource_type_uuid: 
+  location_uuid: 
+  query_timestamp: 
+  available_capacity: 
+  reserved_capacity: 
+  committed_capacity: 
+  sovereignty_capabilities: 
+```
+
+**Mode 2 โ€” Provider Registration (scheduled, preferred)**
+Provider registers capacity data with DCM on a configurable schedule. DCM maintains an internal capacity rating per provider, per Resource Type, per location. Default minimum update frequency: twice daily. Update frequency is configurable per provider registration.
+
+```yaml
+capacity_registration:
+  provider_uuid: 
+  registration_timestamp: 
+  next_scheduled_registration: 
+  capacity_by_resource_type:
+    - resource_type_uuid: 
+      location_uuid: 
+      available_capacity: 
+      reserved_capacity: 
+      committed_capacity: 
+      sovereignty_capabilities: 
+```
+
+**Mode 3 โ€” Provider Denial (reactive, mandatory)**
+The provider validates it can fulfill a request before executing. If it cannot, it denies the request with reason `INSUFFICIENT_RESOURCES`. DCM receives the denial and can retry with an alternative provider. The denial triggers an immediate update to DCM's internal capacity rating for that provider.
+
+```yaml
+provider_denial:
+  provider_uuid: 
+  request_uuid: 
+  denial_reason: INSUFFICIENT_RESOURCES
+  denial_timestamp: 
+  resource_type_uuid: 
+  location_uuid: 
+  estimated_available_at: 
+```
+
+### 7.2 Provider Lifecycle Events
+
+Any provider event that affects Resource/Service Entity availability or operational characteristics MUST be reported to DCM immediately. Providers have a contractual obligation to report these events โ€” this is non-negotiable.
+
+**Reportable Event Types:**
+
+| Event Type | Description | DCM Response |
+|------------|-------------|--------------|
+| `CAPACITY_CHANGE` | Available capacity increased or decreased | Update internal capacity rating |
+| `DEGRADATION` | Underlying resource is degraded | Policy Engine evaluation โ†’ ALERT/REVERT/ESCALATE |
+| `MAINTENANCE_SCHEDULED` | Planned maintenance window declared | Policy Engine evaluation โ†’ notify, migrate if needed |
+| `MAINTENANCE_STARTED` | Maintenance has begun | Update Entity state to MAINTENANCE |
+| `MAINTENANCE_COMPLETED` | Maintenance completed | Restore Entity state, trigger drift detection |
+| `UNSANCTIONED_CHANGE` | Change occurred that was not initiated by DCM | Policy Engine evaluation โ†’ REVERT/UPDATE/ALERT |
+| `ENTITY_HEALTH_CHANGE` | Entity health status changed | Policy Engine evaluation |
+| `PROVIDER_DEGRADATION` | Provider itself is degraded | Policy Engine evaluation โ†’ reroute new requests |
+| `DECOMMISSION_NOTICE` | Provider is decommissioning underlying resource | Policy Engine evaluation โ†’ migrate or decommission Entity |
+
+**Event Payload Format:**
+All provider lifecycle events must be reported in DCM unified data model format:
+
+```yaml
+provider_lifecycle_event:
+  event_uuid: 
+  event_type: 
+  provider_uuid: 
+  affected_entity_uuids:
+    - 
+  event_timestamp: 
+  event_details:
+    
+  severity: 
+  requires_immediate_action: 
+```
+
+**Maximum Reporting Latency:**
+Providers must report lifecycle events within the timeframe declared in their provider registration. For CRITICAL severity events, immediate reporting is required. The reporting latency SLA is part of the Provider SLA/Operational Contract.
+
+### 7.3 DCM Capacity Rating
+
+DCM maintains an internal capacity rating per provider, per Resource Type, per location. This rating is used by the Policy Engine for placement decisions.
+
+```yaml
+dcm_capacity_rating:
+  provider_uuid: 
+  resource_type_uuid: 
+  location_uuid: 
+  last_updated: 
+  update_source: 
+  available_capacity: 
+  capacity_confidence: 
+  # high: updated within last scheduled window
+  # medium: updated within 2x scheduled window
+  # low: stale โ€” beyond 2x scheduled window
+  next_scheduled_update: 
+```
+
+---
+
+
+---
+
+## 7a. Provider Update Notification Model
+
+### 7a.1 The Fundamental Constraint โ€” Realized State Only Changes via a Request
+
+DCM enforces a single foundational rule for the Realized Store:
+
+> **Realized State only changes when an authorized request produces a corresponding Requested State record. No exceptions.**
+
+This constraint unifies all state change pathways and eliminates ambiguity:
+
+- **Drift is always unsanctioned** โ€” if Discovered State differs from Realized State and there is no corresponding Requested State record explaining the difference, it is drift. There is no such thing as "legitimate drift."
+- **Discovery does not update Realized State** โ€” discovery writes only to the Discovered Store. It never updates the Realized Store, even if discovery shows an authorized change (the authorization produces its own Requested State and Realized State records).
+- **Providers cannot write directly to Realized State** โ€” providers report changes via the Provider Update Notification API. DCM evaluates the notification and creates a Requested State record if approved. Only then does a new Realized State record get written.
+
+### 7a.2 Provider Update Notification
+
+A **Provider Update Notification** is a formal mechanism by which a Service Provider reports an authorized state change to DCM. This is distinct from a lifecycle event (which reports provider health) and distinct from an unsanctioned change (which triggers drift). A Provider Update Notification is the provider saying: "I made an authorized change to this entity โ€” please record it as the new Realized State."
+
+**When is a Provider Update Notification appropriate:**
+
+| Scenario | Correct mechanism | Why |
+|----------|------------------|-----|
+| Provider auto-heals a failed disk | Provider Update Notification | Authorized maintenance action; new disk is the correct state |
+| Provider scales resources per pre-authorized auto-scale policy | Provider Update Notification | DCM pre-authorized the scaling policy; each scaling event is an authorized change |
+| Provider performs planned maintenance that changes an IP assignment | Provider Update Notification | Planned, coordinated change |
+| Unauthorized human modifies VM configuration at provider console | Drift event | No DCM authorization; treated as unsanctioned change |
+| Provider silently changes configuration without notifying DCM | Drift event (detected by discovery) | Unreported change is unsanctioned until evaluated |
+
+### 7a.3 Provider Update Notification API
+
+Service Providers submit update notifications via a dedicated endpoint on the DCM API Gateway:
+
+```
+POST /api/v1/provider/entities/{entity_uuid}/update-notification
+Authorization: 
+
+Request body:
+{
+  "provider_uuid": "",
+  "notification_uuid": "",      # idempotency key
+  "notification_type": "",
+  "changed_fields": {
+    "memory_gb": {
+      "previous_value": 8,
+      "new_value": 16,
+      "change_reason": "Auto-scale policy: payments-api-scale-up triggered at 85% memory utilization",
+      "authorizing_policy_ref": ""
+    }
+  },
+  "effective_at": "",
+  "provider_evidence_ref": ""
+}
+```
+
+### 7a.4 DCM Processing of Provider Update Notifications
+
+```
+Provider submits update notification
+  โ”‚
+  โ–ผ Authentication and authorization check
+  โ”‚   Verify: provider UUID is registered and active
+  โ”‚   Verify: provider has authority over this entity
+  โ”‚
+  โ–ผ Policy Engine evaluates notification
+  โ”‚   Evaluate: is this type of change pre-authorized for this entity/provider?
+  โ”‚   Evaluate: does the change violate any GateKeeper constraints?
+  โ”‚   Evaluate: does this change require consumer notification or approval?
+  โ”‚
+  โ”œโ”€โ”€ REJECTED
+  โ”‚   The change is not authorized.
+  โ”‚   DCM does NOT update Realized State.
+  โ”‚   The discrepancy between provider state and DCM Realized State becomes drift.
+  โ”‚   Provider receives rejection response with reason.
+  โ”‚   UNSANCTIONED_CHANGE event logged.
+  โ”‚
+  โ”œโ”€โ”€ REQUIRES_CONSUMER_APPROVAL
+  โ”‚   The change is plausible but requires consumer sign-off.
+  โ”‚   Notification queued. Consumer notified.
+  โ”‚   Entity enters PENDING_REVIEW state.
+  โ”‚   Provider receives "pending_approval" response.
+  โ”‚   On consumer approval โ†’ proceeds to APPROVED path.
+  โ”‚   On consumer rejection โ†’ treated as REJECTED.
+  โ”‚
+  โ””โ”€โ”€ APPROVED
+      DCM creates a Requested State record:
+        source_type: provider_update
+        actor: provider (service account)
+        authorizing_policy_uuid: 
+        changed_fields: [as reported by provider]
+      
+      DCM writes new Realized State snapshot:
+        source_type: provider_update
+        corresponding_requested_state_uuid: 
+        supersedes_realized_state_uuid: 
+      
+      Audit record written: PROVIDER_UPDATE_APPLIED
+      Provider receives "accepted" response.
+```
+
+### 7a.5 Pre-Authorization of Provider Updates
+
+Organizations can pre-authorize categories of provider updates through policy, eliminating the need for per-change human approval:
+
+```yaml
+policy:
+  type: gatekeeper
+  handle: "tenant/payments/allow-auto-scale"
+  rules:
+    - condition:
+        notification_type: auto_scale
+        provider_uuid: 
+        entity.resource_type: Compute.VirtualMachine
+        changed_fields: [memory_gb, cpu_count]
+        change_within_bounds:
+          memory_gb: { max_increase_factor: 2 }
+          cpu_count: { max_increase_factor: 2 }
+      action: approve
+      audit_note: "Auto-scale approved per payments team scaling policy"
+```
+
+This pre-authorization pattern allows providers to implement auto-scaling, auto-healing, and maintenance operations without requiring per-change manual approval, while keeping DCM's Realized Store accurate and traceable.
+
+### 7a.6 Updated Provider Lifecycle Events Table
+
+The following table supersedes the table in Section 7.2 with clearer DCM response categorization:
+
+| Event Type | Mechanism | DCM Response | Realized Store Updated? |
+|------------|-----------|-------------|------------------------|
+| `CAPACITY_CHANGE` | Lifecycle event | Update internal capacity rating | No |
+| `DEGRADATION` | Lifecycle event | Policy Engine โ†’ ALERT/ESCALATE | No |
+| `MAINTENANCE_SCHEDULED` | Lifecycle event | Notify, plan migration if needed | No |
+| `MAINTENANCE_CHANGE` | **Provider Update Notification** | Evaluate โ†’ Requested State if approved | Yes (if approved) |
+| `AUTO_SCALE` | **Provider Update Notification** | Evaluate per pre-auth policy โ†’ Requested State if approved | Yes (if approved) |
+| `AUTO_HEAL` | **Provider Update Notification** | Evaluate per pre-auth policy โ†’ Requested State if approved | Yes (if approved) |
+| `UNSANCTIONED_CHANGE` | Lifecycle event (no notification) | Drift event โ†’ Policy Engine โ†’ REVERT/ALERT/ESCALATE | No (drift, not update) |
+| `ENTITY_HEALTH_CHANGE` | Lifecycle event | Policy Engine evaluation | No |
+| `DECOMMISSION_NOTICE` | Lifecycle event | Policy Engine โ†’ migrate or decommission | No |
+
+### 7a.7 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `RSE-010` | Realized State only changes via an authorized request that produces a corresponding Requested State record. Drift detection, discovery cycles, and lifecycle events do not write to the Realized Store. |
+| `RSE-011` | Provider Update Notifications are evaluated by the Policy Engine before any Realized State change. Rejected notifications do not update Realized State โ€” the discrepancy becomes drift. |
+| `RSE-012` | Categories of provider updates may be pre-authorized via GateKeeper policy. Pre-authorized updates are processed automatically without per-change human approval. |
+| `RSE-013` | Provider Update Notifications that require consumer approval place the entity in PENDING_REVIEW state. The provider receives a "pending_approval" response and the change is queued until resolution. |
+
+
+
+### 7c. Provider Accreditation Registration
+
+Every Service Provider must declare its accreditation status during registration. Accreditation declarations are references to accreditation records registered in DCM's accreditation registry (see [doc 26](26-accreditation-and-authorization-matrix.md)).
+
+```yaml
+provider_registration:
+  # ... existing fields ...
+  accreditations:
+    - accreditation_uuid:        # reference to registered accreditation record
+      framework: fedramp_high
+      status: active
+      valid_until: "2026-12-31"
+
+    - accreditation_uuid: 
+      framework: hipaa
+      accreditation_type: baa
+      status: active
+
+  # Self-declared compliance (lowest trust; used when no formal accreditation exists)
+  self_declared_compliance:
+    frameworks: [iso_27001]
+    last_self_review: "2026-01-15"
+    evidence_ref: 
+
+  # Maximum data classification this provider is permitted to handle
+  # DCM computes this from active accreditations; self_declared_max is the fallback
+  self_declared_max_data_classification: confidential
+```
+
+Providers without any accreditation records are treated as `self_declared` level and are subject to the most restrictive authorization matrix rules. They may only receive data classified as `public` or `internal`.
+
+
+## 8. Entity Relationships
+
+Every Resource/Service Entity carries a `relationships` section declaring its relationships to other entities โ€” internal DCM entities, external data entities, and business context entities. The relationship model is universal โ€” the same structure is used for all relationship types.
+
+See [Entity Relationships](09-entity-relationships.md) for the complete relationship model.
+
+```yaml
+resource_service_entity:
+  uuid: 
+  # ... other entity fields ...
+  relationships:
+    - relationship_uuid: 
+      this_entity_uuid: 
+      this_role: 
+      related_entity_uuid: 
+      related_entity_type: 
+      relationship_type: 
+      nature: 
+      lifecycle_policy:
+        on_related_destroy: 
+        on_related_suspend: 
+        on_related_modify: 
+      status: 
+      provenance:
+        
+```
+
+---
+
+## 9. DCM System Policies for Resource/Service Entities
+
+The following are **non-overridable DCM System Policies** that apply to all Resource/Service Entities:
+
+| Policy | Rule | Enforcement |
+|--------|------|-------------|
+| `RSE-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation โ€” no Tenant = request rejected |
+| `RSE-002` | Every Resource/Service Entity must have a UUID | Enforced at Entity creation |
+| `RSE-003` | Every Resource/Service Entity must have a complete provenance chain | Enforced at every state transition |
+| `RSE-004` | Realized State payloads must be complete โ€” not a status code | Enforced at provider response receipt |
+| `RSE-005` | Decommissioned Entity records are immutable and permanent | Enforced at decommission โ€” records cannot be deleted |
+| `RSE-006` | Provider lifecycle events must be recorded in Entity provenance | Enforced at event receipt |
+| `RSE-007` | Ownership transfers must be authorized by policy | Enforced at transfer initiation |
+| `RSE-008` | Process Resource Entities must reference all affected Entity UUIDs | Enforced at process completion |
+
+---
+
+## 9a. Lifecycle Time Constraints
+
+### 9a.1 Concept
+
+**Lifecycle time constraints** declare when a resource should cease to exist or trigger a lifecycle action. They are a first-class field on any resource entity โ€” governed, provenance-tracked, and subject to the standard override control model.
+
+Any source in the data model precedence chain can declare a time constraint: a consumer request, a Core Layer, a Service Layer, or a policy. The Policy Engine has full authority over constraints โ€” a GateKeeper can lock a TTL immutable or set `immutable_ceiling: absolute` on an expiry date.
+
+### 9a.2 Constraint Structure
+
+```yaml
+lifecycle_constraints:
+  ttl:
+    duration: P14D                            # ISO 8601 duration
+    reference_point: realization_timestamp    # created_at | realization_timestamp | last_modified
+    on_expiry: 
+    metadata:
+      override: allow                         # standard override control
+      basis_for_value: "Consumer declared ephemeral โ€” 14-day lab resource"
+
+  expires_at:
+    timestamp: "2026-06-30T23:59:59Z"         # absolute calendar date
+    on_expiry: notify
+    metadata:
+      override: immutable
+      locked_by_policy_uuid: 
+      basis_for_value: "Project deadline โ€” resource must not persist beyond Q2"
+
+  enforcement:
+    warn_before_expiry: P1D                   # warn 1 day before expiry
+    grace_period: PT1H                        # 1 hour grace after expiry before action
+    on_grace_period_expiry: 
+```
+
+When both `ttl` and `expires_at` are declared, the earliest expiry wins (LTC-004).
+
+### 9a.3 Precedence
+
+Time constraints follow the same precedence as all other resource fields:
+
+```
+Base Layer (lowest โ€” e.g., no TTL by default)
+  โ†“  Core Layer (e.g., all dev resources: TTL 90 days)
+  โ†“  Service Layer (e.g., ephemeral compute: TTL 7 days)
+  โ†“  Request Layer (consumer declared)
+  โ†“  Transformation Policy (enrich from business context)
+  โ†“  GateKeeper Policy (highest โ€” may lock immutable)
+```
+
+### 9a.4 Expiry Enforcement
+
+The **Lifecycle Constraint Enforcer** is a DCM control plane component โ€” not a provider concern. It monitors realized entities, fires `on_expiry` actions when constraints are reached, and records all enforcement in provenance and the Audit Store.
+
+Entities whose `on_expiry` action fails to execute enter `PENDING_EXPIRY_ACTION` state and trigger an escalation (LTC-005).
+
+### 9a.5 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `LTC-001` | Lifecycle time constraints follow standard data model precedence |
+| `LTC-002` | GateKeeper policies may lock lifecycle constraints as immutable |
+| `LTC-003` | Expiry enforcement is a DCM control plane function |
+| `LTC-004` | When multiple time constraints exist, the earliest expiry wins |
+| `LTC-005` | Failed expiry action execution triggers `PENDING_EXPIRY_ACTION` state and escalation |
+
+---
+
+## 9a. Lifecycle Time Constraints โ€” Process Resources (Q28)
+
+Process Resource entities must declare a maximum execution time. This is a mandatory field โ€” not optional. A Process Resource with no execution time limit creates operational blindness (DCM cannot know if it is hung).
+
+```yaml
+process_resource_entity:
+  resource_type: Process.AnsiblePlaybook
+  execution_constraints:
+    max_execution_time: PT2H          # mandatory โ€” ISO 8601 duration
+    expected_completion: PT30M        # advisory โ€” when we expect completion
+    grace_period: PT15M               # grace period after max before action fires
+    on_max_exceeded: 
+    # escalate:  notify platform admin and provider; human decides
+    # terminate: DCM instructs provider to terminate the process
+    # notify:    notify consumer and wait; no automatic action
+    escalation_recipient: 
+```
+
+The Lifecycle Constraint Enforcer handles this โ€” process execution time is a `lifecycle_constraint.ttl` with `reference_point: realization_timestamp`. The `on_max_exceeded` action maps to the standard `on_expiry` lifecycle action vocabulary.
+
+**Profile-governed default `on_max_exceeded`:**
+
+| Profile | Default Action |
+|---------|---------------|
+| `minimal` | `notify` |
+| `dev` | `notify` |
+| `standard` | `escalate` |
+| `prod` | `escalate` |
+| `fsi` | `terminate` |
+| `sovereign` | `terminate` |
+
+---
+
+## 9b. Billing State and SUSPENDED Entities (Q29)
+
+DCM carries billing state as a first-class field โ€” the Cost Analysis component consumes it. Organizations declare billing behavior via policy โ€” DCM does not decide what is billable.
+
+```yaml
+entity:
+  lifecycle_state: SUSPENDED
+  billing_state: 
+  billing_metadata:
+    billing_rate_multiplier: 0.3       # 30% of normal rate if reduced_rate
+    billing_reason: "Reserved capacity โ€” suspended but resources held"
+    billing_policy_uuid:         # policy that determined this billing state
+    billable_components: [storage, ip_address]   # which sub-resources are billed
+    non_billable_components: [compute]
+```
+
+**Three billing models for SUSPENDED:**
+- **`billable`** โ€” resources reserved and capacity held (stopped VM still consuming reserved IP and storage)
+- **`non_billable`** โ€” resources fully released on suspension (spot/ephemeral resource)
+- **`reduced_rate`** โ€” partial resources held (storage retained, compute released)
+
+Policy injects `billing_state` and `billing_metadata` during state transitions. A GateKeeper can declare: "all suspended VMs in the payments Tenant are billed at 30% โ€” compute released but storage and IP retained."
+
+---
+
+## 9c. Bare Metal Indivisibility (Q26)
+
+Bare metal Whole Allocation uses the same `shareability.allowed: false` mechanism as any non-shareable resource (REL-017), plus an explicit `allocation_model` declaration:
+
+```yaml
+resource_type_spec:
+  fully_qualified_name: Compute.BareMetal
+  allocation_model: whole_unit         # whole_unit | fractional | pooled
+  shareability:
+    allowed: false                     # structural lock โ€” cannot be changed by policy
+    indivisibility_reason: "Physical hardware โ€” cannot be partitioned"
+  capacity:
+    unit: server
+    minimum_allocation: 1
+    maximum_allocation: 1              # whole unit only
+
+# Provider contract obligations for bare metal:
+provider_contract_obligations:
+  - Report full physical identity in realized payload (serial_number, hardware_profile)
+  - Exclusive placement hold during reserve_query โ€” no concurrent holds on same server
+  - Notify DCM immediately if any sharing attempt is detected (drift trigger)
+```
+
+---
+
+## 9d. Capacity Confidence โ€” Automatic Actions (Q27)
+
+Capacity confidence ratings trigger policy-governed automatic actions. Policy determines the action per confidence level; the active Profile sets defaults.
+
+```yaml
+capacity_confidence_policy:
+  HIGH:
+    action: proceed
+    max_data_age: PT5M
+  MEDIUM:
+    action: proceed_with_warning      # default โ€” overridable by policy
+    max_data_age: PT30M
+  LOW:
+    action: refresh_before_placement  # default โ€” trigger Mode 1 query
+    max_data_age: PT1H
+    trigger_mode1_query: true
+```
+
+**Profile-governed defaults:**
+
+| Profile | HIGH | MEDIUM | LOW |
+|---------|------|--------|-----|
+| `minimal` | proceed | proceed | proceed_with_warning |
+| `dev` | proceed | proceed | refresh_before_placement |
+| `standard` | proceed | proceed_with_warning | refresh_before_placement |
+| `prod` | proceed | refresh_before_placement | reject |
+| `fsi` | proceed | refresh_before_placement | reject |
+| `sovereign` | proceed | refresh_before_placement | reject |
+
+---
+
+## 9e. Ownership Transfer Count (Q25)
+
+Ownership transfers are unlimited by default. Each transfer is immutably recorded with a monotonically incrementing `transfer_number`. Policy may declare a maximum per resource type.
+
+```yaml
+ownership_transfer_record:
+  transfer_uuid: 
+  transfer_number: 3              # monotonically incrementing โ€” never resets
+  from_tenant_uuid: 
+  to_tenant_uuid: 
+  authorized_by: 
+  transfer_timestamp: 
+  reason: 
+  policy_uuid: 
+```
+
+Policy-governed maximum when needed:
+```yaml
+policy:
+  type: gatekeeper
+  rule: >
+    If resource.ownership_transfer_count > 5
+    AND resource_type == Compute.VirtualMachine
+    THEN gatekeep: "VM has exceeded 5 ownership transfers โ€” manual review required"
+```
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | For Hybrid Transfer โ€” what is the maximum number of ownership transfers allowed, or is it unlimited? | Operational complexity | โœ… Resolved โ€” unlimited by default; policy may declare maximum; monotonically incrementing transfer_number (ENT-001) |
+| 2 | For Whole Allocation of bare metal โ€” how is the indivisibility enforced at the provider level? | Provider contract | โœ… Resolved โ€” allocation_model: whole_unit; shareability.allowed: false; exclusive hold; provider reports physical identity (ENT-002) |
+| 3 | Should capacity confidence ratings trigger automatic actions? | Capacity model | โœ… Resolved โ€” policy-governed actions per confidence level; LOW triggers Mode 1 query by default in standard+; profile-governed (ENT-003) |
+| 4 | For Process Resources โ€” should there be a maximum execution time? | Operational governance | โœ… Resolved โ€” mandatory max_execution_time; enforced by Lifecycle Constraint Enforcer; profile-governed on_max_exceeded (ENT-004) |
+| 5 | How does the SUSPENDED state interact with cost analysis? | Cost model | โœ… Resolved โ€” billing_state field (billable/non_billable/reduced_rate); policy injects on state transition; Cost Analysis consumes (ENT-005) |
+
+---
+
+## 11. DCM System Policies โ€” Entity and Dependency Gaps
+
+| Policy | Rule |
+|--------|------|
+| `ENT-001` | Ownership transfer count is unlimited by default. Policy may declare a maximum transfer count per resource type. Each transfer is immutably recorded with a monotonically incrementing transfer_number and mandatory reason field. |
+| `ENT-002` | Bare metal resources declare `allocation_model: whole_unit` and `shareability.allowed: false`. Placement holds are exclusive. Providers must report the server's physical identity in the realized payload and notify DCM of any sharing attempt. |
+| `ENT-003` | Capacity confidence ratings trigger policy-governed automatic actions. LOW confidence triggers a Mode 1 Information Provider query by default in standard+ profiles. Profile determines the default action per confidence level. |
+| `ENT-004` | Process Resource entities must declare `max_execution_time`. This field is mandatory. Execution time is enforced by the Lifecycle Constraint Enforcer. Profile governs the default `on_max_exceeded` action. |
+| `ENT-005` | Entity `billing_state` (billable, non_billable, or reduced_rate) is a first-class field injected by policy during state transitions. The Cost Analysis component consumes `billing_state` for cost attribution. DCM does not decide billing policy โ€” it carries the billing signal. |
+
+---
+
+
+
+- **DCM Tenant** โ€” the mandatory ownership boundary for all Resource/Service Entities
+- **Four States** โ€” Intent, Requested, Realized, Discovered โ€” the state lifecycle of a Resource/Service Request and Entity
+- **Field-Level Provenance** โ€” every state transition and ownership transfer is recorded in Entity provenance
+- **Policy Engine** โ€” evaluates provider events and unsanctioned changes, determines response actions
+- **Service Dependencies** โ€” Resource/Service Entities declare dependencies on other Entities
+- **Resource Grouping** โ€” Entities belong to a Tenant and optionally to additional Resource Groups
+- **Provider Contract** โ€” governs provider obligations including capacity reporting and event notification
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/07-service-dependencies.md b/content/docs/data-model/07-service-dependencies.md
new file mode 100644
index 0000000..57a1a87
--- /dev/null
+++ b/content/docs/data-model/07-service-dependencies.md
@@ -0,0 +1,553 @@
+# DCM Data Model โ€” Service Dependencies
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Entity Relationships](09-entity-relationships.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA**
+>
+> The Data abstraction โ€” dependency graph as embedded data structure
+
+
+
+> **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](09-entity-relationships.md). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document.
+
+---
+
+## 1. Purpose
+
+This document defines how service dependencies are declared, resolved, and managed within DCM. Dependencies are a core data model concern โ€” not an orchestration concern. The data structures defined here enable DCM to know the complete resource footprint of any request before execution begins, which is essential for cost analysis, placement decisions, rehydration, and audit.
+
+---
+
+## 2. Why Dependencies Must Be Declared in Advance
+
+Dependencies must be declared in the data model โ€” not discovered at runtime by providers. This is a hard requirement driven by four core DCM goals:
+
+**Auditability** โ€” the complete dependency graph must be known before execution. Every resource that will be created as part of fulfilling a request must be visible in the request's provenance chain from the start.
+
+**Cost Analysis** โ€” accurate cost estimation and cost-based placement require knowing the full resource footprint before provisioning. Hidden dependencies produce hidden costs that only become visible after the fact.
+
+**Placement** โ€” the Policy Engine cannot make optimal placement decisions without knowing all resources that will be created. A Web Server request that implicitly spawns a VM, IP address, and firewall rule has placement requirements that span multiple resource types.
+
+**Idempotency and Consistency** โ€” if dependencies are declared in the service definition, the same request always produces the same dependency graph. Provider-driven dependency discovery at runtime breaks idempotency โ€” different provider implementations could produce different dependency graphs for the same logical request.
+
+---
+
+## 3. Hybrid Dependency Declaration Model
+
+DCM uses a hybrid model for dependency declaration that operates at two levels:
+
+### 3.1 Type-Level Dependencies (Resource Type Specification)
+
+Dependencies declared at the Resource Type Specification level are **portable and provider-agnostic**. They define what kinds of resources are needed โ€” not which specific provider supplies them.
+
+- Declared in the Resource Type Specification
+- Apply to all Provider Catalog Items implementing that Resource Type
+- Use Resource Type UUIDs โ€” not provider-specific references
+- Required for all implementations of the Resource Type
+- Portable โ€” the dependency can be fulfilled by any provider implementing the required Resource Type
+
+**Example:**
+```yaml
+resource_type: Compute.VirtualMachine
+type_level_dependencies:
+  - dependency_uuid: 
+    required_resource_type_uuid: 
+    required_resource_type_name: Network.IPAddress
+    dependency_type: hard
+    cardinality: one_to_one
+    description: Every VM requires exactly one IP address
+  - dependency_uuid: 
+    required_resource_type_uuid: 
+    required_resource_type_name: Network.FirewallRule
+    dependency_type: hard
+    cardinality: one_to_many
+    description: Every VM requires at least one firewall rule
+```
+
+### 3.2 Provider-Specific Dependencies (Provider Catalog Item)
+
+Dependencies declared at the Provider Catalog Item level are **provider-specific additions** beyond the type-level dependencies. They must be marked as portability-breaking.
+
+- Declared in the Provider Catalog Item registration
+- Apply only to requests fulfilled by that specific provider
+- Must be marked `portability_breaking: true`
+- Visible to the Policy Engine for governance decisions
+- Surfaced to consumers as portability warnings
+
+**Example:**
+```yaml
+catalog_item: Nutanix.VM.Small
+provider_specific_dependencies:
+  - dependency_uuid: 
+    required_resource_type_uuid: 
+    required_resource_type_name: Nutanix.StorageContainer
+    dependency_type: hard
+    portability_breaking: true
+    description: Nutanix VMs require a Nutanix Storage Container
+    portability_warning: This dependency locks this request to Nutanix providers
+```
+
+---
+
+## 4. Dependency Types
+
+Every declared dependency must specify its type:
+
+| Type | Description | Behavior |
+|------|-------------|----------|
+| `hard` | Must be realized before or alongside the dependent resource | Failure of dependency fails the dependent resource |
+| `soft` | Preferred but not blocking | Failure of dependency is recorded but does not block the dependent resource |
+| `conditional` | Required only if specific conditions in the request payload are met | Evaluated by Policy Engine against request data |
+
+---
+
+## 5. Dependency Cardinality
+
+Every declared dependency must specify its cardinality:
+
+| Cardinality | Description | Example |
+|-------------|-------------|---------|
+| `one_to_one` | Exactly one dependency resource required | One VM needs exactly one primary IP |
+| `one_to_many` | One or more dependency resources required | One VM needs one or more firewall rules |
+| `one_to_optional` | Zero or one dependency resource | One VM may optionally have a secondary IP |
+| `one_to_range` | A specific numeric range required | One load balancer needs 2-6 backend VMs |
+
+---
+
+## 6. Dependency Graph
+
+When a request is processed, the Request Payload Processor constructs a **Dependency Graph** โ€” a complete map of all resources that must be created to fulfill the request, including all transitive dependencies.
+
+### 6.1 Dependency Graph Structure
+
+```yaml
+dependency_graph:
+  graph_uuid: 
+  root_request_uuid: 
+  tenant_uuid: 
+  created_timestamp: 
+  nodes:
+    - node_uuid: 
+      resource_type_uuid: 
+      resource_type_name: 
+      request_uuid: 
+      entity_uuid: 
+      lifecycle_state: 
+      dependencies:
+        - dependency_uuid: 
+          dependent_node_uuid: 
+          dependency_type: 
+          status: 
+  edges:
+    - from_node_uuid: 
+      to_node_uuid: 
+      dependency_uuid: 
+      dependency_type: 
+```
+
+### 6.2 Transitive Dependencies
+
+DCM resolves transitive dependencies โ€” the full chain of dependencies, not just direct ones.
+
+**Example โ€” Web Server request:**
+```
+Web Server (requested)
+  โ”œโ”€โ”€ VM (hard dependency of Web Server)
+  โ”‚     โ”œโ”€โ”€ IP Address (hard dependency of VM)
+  โ”‚     โ”‚     โ””โ”€โ”€ Network (hard dependency of IP Address)
+  โ”‚     โ””โ”€โ”€ Firewall Rule (hard dependency of VM)
+  โ”‚           โ””โ”€โ”€ IP Address (reference โ€” already in graph)
+  โ””โ”€โ”€ DNS Record (soft dependency of Web Server)
+        โ””โ”€โ”€ IP Address (reference โ€” already in graph)
+```
+
+The dependency graph contains each resource exactly once โ€” circular references and duplicate nodes are detected and resolved. A resource that appears as a dependency of multiple nodes is represented as a single node with multiple incoming edges.
+
+### 6.3 Dependency Graph and the Four States
+
+The dependency graph is part of the request's data from the moment it is constructed:
+
+- **Intent State** โ€” consumer's request, no dependency graph yet
+- **Requested State** โ€” dependency graph constructed and attached, all nodes in PENDING state
+- **Realized State** โ€” nodes updated to REALIZED as providers fulfill each dependency
+- **Discovered State** โ€” dependency graph used to scope discovery โ€” discover all nodes in the graph
+
+---
+
+## 7. Dependency Payload Passing
+
+When a dependency resource is realized, its realized payload must be passed to the dependent resource's provider. This is how a provider knows the details of the resources it depends on โ€” IP addresses, network configurations, security group IDs, etc.
+
+### 7.1 The Payload Passing Mechanism
+
+```
+Dependency Resource realized
+  โ”‚
+  โ–ผ
+Realized State payload captured in Realized Store
+  โ”‚
+  โ–ผ
+Dependency node in graph updated: entity_uuid recorded, status โ†’ SATISFIED
+  โ”‚
+  โ–ผ
+Dependent resource's Requested State payload enriched with dependency data
+  โ”‚  Recorded in field-level provenance โ€” source_type: dependency_payload
+  โ”‚  source_uuid: 
+  โ–ผ
+Enriched payload dispatched to dependent resource's provider
+```
+
+### 7.2 Dependency Data in Request Payloads
+
+When a dependency is satisfied, the dependent resource's Requested State payload is enriched with the dependency entity's UUID and relevant realized data:
+
+```yaml
+# Original request payload for VM
+vm_request:
+  cpu_count: 8
+  ram_gb: 32
+  os: RHEL9
+
+# After IP Address dependency is realized
+vm_request:
+  cpu_count: 8
+  ram_gb: 32
+  os: RHEL9
+  dependencies:
+    ip_address:
+      entity_uuid: 
+      ip_address: 192.168.1.45
+      network_uuid: 
+      subnet: 192.168.1.0/24
+      provenance:
+        source_type: dependency_payload
+        source_uuid: 
+        timestamp: 
+```
+
+---
+
+## 8. Dependency Resolution Order
+
+The dependency graph determines resolution order. Resources with no unsatisfied hard dependencies can be dispatched immediately. Resources with unsatisfied hard dependencies wait until their dependencies are satisfied.
+
+### 8.1 Resolution Rules
+
+- A resource node can only be dispatched when all its `hard` dependencies are in SATISFIED state
+- `soft` dependencies do not block dispatch โ€” they are attempted but failure does not block
+- `conditional` dependencies are evaluated by the Policy Engine before the graph is constructed โ€” if conditions are not met, the conditional dependency node is not added to the graph
+- Independent branches of the dependency graph can be resolved in parallel โ€” the Orchestration component determines parallelism
+- Circular dependencies are invalid โ€” the Policy Engine rejects any dependency graph with circular references
+
+### 8.2 Failure Handling
+
+Dependency failure handling is **configurable per request or per policy**:
+
+| Failure Mode | Behavior |
+|-------------|---------|
+| `fail_all` | Any hard dependency failure fails the entire request. All partially realized nodes are decommissioned. |
+| `fail_dependent` | A hard dependency failure fails only the dependent resource and its dependents. Independent branches continue. |
+| `retry` | Failed dependencies are retried with the same or alternative provider before failing. Retry count and provider selection policy are configurable. |
+| `partial_complete` | Request is marked partially complete. Failed nodes are flagged for retry or manual intervention. |
+
+The failure mode is declared in the request payload or in an applicable organizational policy.
+
+---
+
+## 9. Rehydration and the Dependency Graph
+
+The dependency graph is the primary mechanism enabling **DC Rehydration** โ€” the ability to reconstruct any resource and its dependencies from scratch.
+
+### 9.1 Rehydration Process
+
+Rehydration uses the **Intent State** of the original request โ€” not the Realized State โ€” to reconstruct the dependency graph. This ensures that rehydration applies current policies and standards rather than replaying an old realized state.
+
+```
+Rehydration initiated for a Tenant / Group / Entity
+  โ”‚
+  โ–ผ
+Intent State(s) retrieved from Intent Store
+  โ”‚
+  โ–ผ
+Dependency graphs reconstructed from Intent States
+  โ”‚
+  โ–ผ
+Graphs processed through current Policy Engine
+  โ”‚  Current policies applied โ€” may differ from original request
+  โ”‚  Current placement policies applied
+  โ”‚  Current sovereignty constraints applied
+  โ–ผ
+New Requested State payloads generated
+  โ”‚
+  โ–ผ
+Resources realized in dependency order
+  โ”‚
+  โ–ผ
+New Realized States recorded
+```
+
+### 9.2 Intent Portability in Rehydration
+
+Because rehydration uses Intent State rather than Realized State:
+- Resources can be rehydrated to a different provider โ€” as long as the provider supports the required Resource Types
+- Current organizational standards and policies are applied โ€” ensuring rehydrated resources meet current compliance requirements
+- Provider-specific dependencies (portability-breaking) may prevent rehydration to a different provider โ€” this is surfaced as a portability warning during rehydration planning
+
+### 9.3 Rehydration Scope
+
+Rehydration can be scoped to:
+- A single Resource/Service Entity and its full dependency graph
+- A Resource Group โ€” all entities in the group and their dependency graphs
+- A Tenant โ€” all entities owned by the Tenant
+- A full Data Center โ€” all entities across all Tenants in a location
+
+The dependency graph ensures that rehydration is always complete โ€” no orphaned resources, no missing dependencies.
+
+---
+
+## 10. Dependency Declaration in Service Catalog Items
+
+Service Catalog Items must declare their dependencies as part of their definition. A catalog item with undeclared dependencies is invalid and will be rejected by the Policy Engine.
+
+```yaml
+catalog_item:
+  uuid: 
+  name: Web Server Service
+  resource_type_uuid: 
+  type_level_dependencies:
+    - dependency_uuid: 
+      required_resource_type_uuid: 
+      dependency_type: hard
+      cardinality: one_to_one
+  provider_specific_dependencies: []
+  conditional_dependencies:
+    - dependency_uuid: 
+      required_resource_type_uuid: 
+      dependency_type: conditional
+      condition:
+        field: high_availability
+        operator: equals
+        value: true
+      description: Load balancer required when high_availability is true
+```
+
+---
+
+## 11. DCM System Policies for Dependencies
+
+| Policy | Rule | Enforcement |
+|--------|------|-------------|
+| `DEP-001` | All dependencies must be declared before a catalog item is active | Enforced at catalog item registration |
+| `DEP-002` | Circular dependencies are invalid | Enforced at dependency graph construction |
+| `DEP-003` | Provider-specific dependencies must be marked portability-breaking | Enforced at provider catalog item registration |
+| `DEP-004` | Dependency payloads must be passed to dependent providers in DCM unified format | Enforced at dependency satisfaction |
+| `DEP-005` | Every node in a dependency graph must have a UUID | Enforced at graph construction |
+
+---
+
+## 11a. Dependency Graph Versioning (Q30)
+
+Dependency graphs are versioned as properties of their parent catalog item โ€” not as independent artifacts. When the dependency graph changes, the catalog item version increments following standard semver semantics:
+
+| Change | Semver Impact | Reason |
+|--------|--------------|--------|
+| Dependency version constraint tightened | Revision bump | Compatible โ€” narrower constraint |
+| New optional dependency added | Minor bump | Compatible โ€” additive |
+| New required dependency added | **Major bump** | Breaking โ€” consumers must update |
+| Required dependency removed | **Major bump** | Breaking โ€” consumers may depend on it |
+| Dependency type changed | **Major bump** | Breaking โ€” structural change |
+
+**At request time:** The catalog item version determines the dependency graph. A consumer pinning to `catalog_item_version: "1.5.3"` gets exactly the dependency graph declared in that version.
+
+**For existing realizations:** The dependency graph version is captured in the Requested State assembly provenance. Rehydration with `re_evaluate: false` replays from the Requested State. Rehydration with `re_evaluate: true` uses the current dependency graph for the selected version.
+
+---
+
+## 11b. Dependency Graph Storage (Q31)
+
+The dependency graph is embedded in assembly provenance โ€” not a separate entity.
+
+| Level | What is stored | Where |
+|-------|---------------|-------|
+| Declared dependency graph | Part of Resource Type Specification | GitOps Layer/Policy Store |
+| Resolved dependency graph | `placement.yaml` in Requested State | GitOps Requested Store |
+| Realized dependency graph | Realized State events per dependency | Event Stream / Realized Store |
+
+```yaml
+# In placement.yaml โ€” resolved dependency graph
+dependency_resolution:
+  - dependency_role: storage
+    resource_type: Storage.Block
+    resolved_provider_uuid: 
+    resolved_catalog_item_version: "1.2.0"
+    reserved_entity_uuid: 
+    reservation_hold_uuid: 
+  - dependency_role: networking
+    resource_type: Network.IPAddress
+    resolved_provider_uuid: 
+    reserved_entity_uuid: 
+```
+
+The full dependency chain is always traceable from the Requested State record โ€” no separate entity needed.
+
+---
+
+## 11c. Dependency Graph Depth (Q33)
+
+Dependency graph depth is limited to a profile-governed maximum. Circular dependency detection is always enforced regardless of depth configuration.
+
+```yaml
+dependency_depth_policy:
+  max_depth: 10                  # configurable via Policy Group
+  on_max_exceeded: reject        # reject with clear error identifying depth + chain
+  cycle_detection: always        # non-configurable โ€” always enforced
+```
+
+**Profile-governed defaults:**
+
+| Profile | Default Max Depth | Rationale |
+|---------|-----------------|-----------|
+| `minimal` | 20 | Home lab โ€” free composition |
+| `dev` | 15 | Development โ€” generous |
+| `standard` | 10 | Production baseline |
+| `prod` | 10 | Production |
+| `fsi` | 7 | Tight โ€” complex dependencies harder to audit |
+| `sovereign` | 7 | Maximum control |
+
+In practice, well-designed service compositions rarely exceed 5-6 levels. Depth 10 provides headroom without allowing pathological compositions.
+
+---
+
+## 11d. Meta Provider Composition Visibility (Q34)
+
+Meta Providers declare how their internal composition is exposed to DCM. This determines whether sub-resources are DCM entities subject to standard lifecycle management, or opaque to DCM.
+
+```yaml
+meta_provider_registration:
+  composition_visibility:
+    mode: 
+    # opaque:      Consumer sees only top-level service entity
+    #              Sub-resources not visible in DCM
+    # transparent: All sub-resources registered as DCM entities
+    #              Full dependency graph visible; drift detection on all
+    # selective:   Provider declares which sub-resources are DCM-visible
+    dcm_visible_sub_resources:    # if selective
+      - resource_type: Compute.VirtualMachine
+        role: control_plane_node
+      - resource_type: Network.LoadBalancer
+        role: api_endpoint
+```
+
+**Drift detection interaction:**
+- `opaque` โ€” drift detection only on what the Meta Provider reports via realized payload; sub-resources are provider's responsibility
+- `transparent` โ€” drift detection on all sub-resources as full DCM entities
+- `selective` โ€” drift detection on declared DCM-visible sub-resources only
+
+---
+
+## 12. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | How are dependency graphs versioned โ€” does a new version of a catalog item invalidate existing dependency graphs? | Versioning model | โœ… Resolved โ€” versioned as part of catalog item; semver semantics; captured in assembly provenance (ENT-006) |
+| 2 | Should the dependency graph be stored as a separate entity or embedded in the request payload? | Data model structure | โœ… Resolved โ€” embedded in assembly provenance; declared in Resource Type Spec; resolved in placement.yaml (ENT-007) |
+| 3 | How are cross-tenant dependencies handled? | Multi-tenancy | โœ… Resolved โ€” governed by REL-010/011/012 and DEP-001/002/003; see Entity Relationships doc |
+| 4 | Should there be a maximum dependency graph depth? | Operational complexity | โœ… Resolved โ€” profile-governed max (10 standard/prod, 7 fsi/sovereign); circular detection always enforced (ENT-008) |
+| 5 | How does the dependency graph interact with the Meta Provider model? | Provider model | โœ… Resolved โ€” composition_visibility (opaque/transparent/selective); transparent/selective registers sub-resources as DCM entities (ENT-009) |
+
+---
+
+## 13. DCM System Policies โ€” Dependency Gaps
+
+| Policy | Rule |
+|--------|------|
+| `ENT-006` | Dependency graphs are versioned as properties of their parent catalog item. New required dependency or removed dependency is a major (breaking) version bump. The dependency graph version used in a realization is captured in assembly provenance. |
+| `ENT-007` | The declared dependency graph is embedded in the Resource Type Specification. The resolved dependency graph is embedded in the Requested State assembly provenance (placement.yaml). No separate dependency graph entity is required. |
+| `ENT-008` | Dependency graph depth is limited to a profile-governed maximum (default: 10 for standard/prod; 7 for fsi/sovereign). Requests exceeding the maximum depth are rejected with a clear error. Circular dependency detection is always enforced regardless of depth configuration. |
+| `ENT-009` | Meta Providers declare composition_visibility as opaque, transparent, or selective. Transparent and selective modes register sub-resources as DCM entities subject to standard lifecycle management and drift detection. Opaque mode delegates sub-resource management entirely to the provider. |
+
+---
+
+
+
+- **Resource Type Specification** โ€” declares type-level dependencies for a Resource Type
+- **Provider Catalog Item** โ€” declares provider-specific additional dependencies
+- **Request Payload Processor** โ€” constructs the dependency graph during assembly
+- **Policy Engine** โ€” evaluates conditional dependencies, enforces dependency policies, governs failure handling
+- **Intent Portability** โ€” rehydration uses Intent State to allow replay with different providers
+- **Field-Level Provenance** โ€” dependency payload data is recorded with source Entity UUID
+- **Resource Grouping** โ€” rehydration can be scoped to groups and tenants
+
+
+---
+
+## 8. Compound Service Compensation Declaration
+
+### 8.1 Overview
+
+Compound services (delivered by Meta Providers) must declare compensation behavior for each component. This declaration is part of the service definition โ€” not discovered at runtime. See [Operational Models](24-operational-models.md) Section 6 for the full compensation execution model.
+
+### 8.2 Compensation Fields on Service Components
+
+```yaml
+service_component:
+  id: vm
+  resource_type: Compute.VirtualMachine
+  required_for_delivery: 
+  # atomic: must succeed; failure triggers full compensation rollback
+  # partial: failure โ†’ DEGRADED state; compound service delivered partially
+
+  compensation_on_failure: 
+  # decommission_immediately: decommission this component as part of rollback
+  # release_allocation:       release allocation back to pool (for allocatable resources)
+  # skip:                     do not compensate; used for partial delivery components
+  # notify:                   notify owner; human decides compensation
+
+  compensation_order: 
+  # Lower numbers compensate first; higher numbers compensate last
+  # Reverse dependency order is the default if not declared
+
+  depends_on: []
+```
+
+### 8.3 Partial Delivery Policy
+
+```yaml
+partial_delivery_policy:
+  min_required_components: [vm, ip]  # compound DEGRADED if only these succeed
+  degraded_is_acceptable: true
+  auto_retry_optional_components:
+    enabled: true
+    max_attempts: 3
+    interval: PT15M
+    on_exhaustion: notify_owner
+```
+
+### 8.4 System Policies โ€” Compensation
+
+| Policy | Rule |
+|--------|------|
+| `DEP-010` | Compensation executes in reverse dependency order (highest compensation_order first). |
+| `DEP-011` | Compensation failure triggers COMPENSATION_FAILED state and immediate orphan detection. |
+| `DEP-012` | Components with required_for_delivery: partial are not compensation-triggering. Their failure produces a DEGRADED compound entity. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/08-resource-grouping.md b/content/docs/data-model/08-resource-grouping.md
new file mode 100644
index 0000000..a35c8b5
--- /dev/null
+++ b/content/docs/data-model/08-resource-grouping.md
@@ -0,0 +1,393 @@
+# DCM Data Model โ€” Resource Grouping
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+> **Universal Group Model:** The constructs defined in this document (DCM Tenant, Resource Groups) are superseded by the [Universal Group Model](15-universal-groups.md) for new implementations. Existing constructs map 1:1 to `group_class` values in the universal model โ€” UUIDs, handles, and API references are preserved. This document remains valid as a reference for pre-universal implementations and for understanding the migration path.
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA**
+>
+> The Data abstraction โ€” DCMGroup typed extensions (Tenant, Resource Group, Cross-Tenant Auth)
+
+
+
+---
+
+## 1. Purpose
+
+This document defines how Resource/Service Entities are organized into groups within DCM. Grouping provides the ownership, organizational context, cost attribution, policy scope, and rehydration targeting that makes DCM operationally meaningful at scale.
+
+Two concepts are defined here:
+1. **DCM Tenant** โ€” the mandatory, first-class ownership boundary for all Resource/Service Entities
+2. **Resource Groups** โ€” flexible, composable grouping entities that provide additional organizational context
+
+---
+
+## 2. DCM Tenant
+
+### 2.1 Definition
+
+A **DCM Tenant** is the primary ownership and isolation boundary for Resource/Service Entities in DCM. Every Resource/Service Entity โ€” including Process Resources โ€” must belong to exactly one DCM Tenant at any point in time.
+
+Tenant membership is the answer to the question: **who owns this resource?**
+
+### 2.2 Tenant as a DCM System Policy
+
+Mandatory Tenant membership is a **non-overridable DCM System Policy**:
+
+| Policy | Rule | Enforcement |
+|--------|------|-------------|
+| `TEN-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation โ€” no Tenant = request rejected |
+| `TEN-002` | Tenant membership cannot be empty โ€” a Tenant must exist before resources can be created in it | Enforced at request processing |
+| `TEN-003` | A Resource/Service Entity cannot exist without a Tenant | Enforced at all lifecycle states |
+
+### 2.3 What Tenant Provides
+
+The Tenant boundary enables the following DCM capabilities for all resources it owns:
+
+| Capability | Description |
+|------------|-------------|
+| **Ownership** | Unambiguous answer to "who owns this resource" โ€” always answerable, always auditable |
+| **Isolation** | Resources in one Tenant are isolated from resources in another โ€” hard tenancy enforcement |
+| **Cost Attribution** | All resource costs roll up to the owning Tenant |
+| **Policy Scope** | Tenant-level policies apply to all resources in the Tenant |
+| **Drift Detection Scope** | Drift detection can be scoped to a Tenant |
+| **Rehydration Scope** | A full Tenant can be targeted for rehydration |
+| **Audit Scope** | All activity within a Tenant is auditable as a unit |
+| **Sovereignty Boundary** | Sovereignty constraints can be applied at the Tenant level |
+
+### 2.4 Tenant Entity Definition
+
+```yaml
+dcm_tenant:
+  uuid: 
+  name: 
+  description: 
+  version: 
+  status:
+    state: 
+    deprecation_date: 
+    sunset_date: 
+    replacement_uuid: 
+    deprecation_reason: 
+    migration_guidance: 
+  ownership:
+    owner_uuid: 
+    owner_type: 
+    created_timestamp: 
+  membership_policy:
+    exclusive: true
+    # A resource belongs to exactly one Tenant
+    # This is non-overridable
+  sovereignty_constraints:
+    
+  policies:
+    
+  provenance:
+    
+```
+
+### 2.5 Tenant and Resource Consumption
+
+A resource belongs to exactly one Tenant โ€” its **owner**. However, a resource can be **consumed** by multiple Tenants via the DCM Service Catalog. Ownership and consumption are distinct:
+
+- **Ownership** (Tenant membership) โ€” who is responsible for the lifecycle, cost, and compliance of this resource
+- **Consumption** โ€” who uses or depends on this resource as a service
+
+Cross-tenant consumption is tracked through service requests and cost attribution โ€” not through Tenant membership. A shared DNS service owned by a Platform Tenant can be consumed by any number of application Tenants. The DNS Entity belongs to the Platform Tenant. Consumption is tracked via service requests from each consuming Tenant.
+
+---
+
+## 3. Resource Groups
+
+### 3.1 Definition
+
+A **Resource Group** is a flexible, composable grouping entity that provides organizational context, operational scope, and policy targeting beyond what Tenant membership provides.
+
+Resource Groups function like **structured tags** โ€” a resource accumulates group memberships that describe its context from multiple dimensions simultaneously. A VM could simultaneously belong to:
+- `Deployment: WebApp-v2` (what deployment it is part of)
+- `BusinessUnit: Payments` (which business unit owns the workload)
+- `RegulatoryScope: PCI-DSS` (which compliance regime applies)
+- `CostCenter: CC-4421` (where costs are attributed)
+
+Each group membership is a different dimension of context โ€” not a hierarchy within a single dimension.
+
+### 3.2 Resource Group Classes
+
+DCM defines two classes of Resource Group, both implementing the same **Resource Group Interface**:
+
+**Class 1 โ€” DCM Default Resource Group**
+Built into DCM. The standard mechanism for grouping resources. No implementor customization required to use it.
+
+**Class 2 โ€” Custom Resource Group**
+Implementor-defined grouping entities. Tied to internal business structures โ€” business units, product lines, regulatory scopes, cost centers, etc. Full parity with DCM Default Resource Groups in terms of DCM capabilities.
+
+Both classes implement the same interface. The DCM Default Resource Group is simply DCM's own implementation of the Resource Group Interface. Custom groups are implementor-defined implementations of the same interface.
+
+### 3.3 The Resource Group Interface
+
+Every Resource Group โ€” both DCM default and custom โ€” must implement this interface:
+
+```yaml
+resource_group:
+  uuid: 
+  name: 
+  description: 
+  group_class: 
+  group_type: 
+  version: 
+  status:
+    state: 
+    deprecation_date: 
+    sunset_date: 
+    replacement_uuid: 
+    deprecation_reason: 
+    migration_guidance: 
+  nesting:
+    supported: 
+    # If true, this group can contain other groups as members
+    max_depth: 
+  membership:
+    members:
+      - member_uuid: 
+        member_type: 
+        joined_timestamp: 
+        joined_by_uuid: 
+    membership_policy:
+      exclusive: 
+      # If true, a resource can only belong to one group of this type at a time
+      # If false, a resource can belong to multiple groups of this type
+      max_memberships: 
+      # Maximum number of groups of this type a resource can belong to
+      allowed_entity_types:
+        
+  policies:
+    
+  provenance:
+    
+```
+
+### 3.4 Multi-Group Membership
+
+A Resource/Service Entity can belong to multiple Resource Groups across all classes. This multi-dimensional membership is what gives groups their tag-like flexibility.
+
+**Membership constraints are configurable per group definition:**
+- A group can declare `exclusive: true` โ€” meaning a resource can only belong to one group of that type at a time
+- Example: A `RegulatoryScope` group might declare `exclusive: true` โ€” a resource cannot be in both EU-GDPR and US-FISMA regulatory scopes simultaneously
+- Example: A `Deployment` group might declare `exclusive: false` โ€” a resource could participate in multiple deployments
+
+**Policy-governed membership:**
+Organizational policies can further restrict multi-group membership. For example, a sovereignty policy could declare that resources in a PCI-DSS scope cannot be in the same group as resources in a non-PCI scope.
+
+### 3.5 Nesting
+
+Resource Groups that declare `nesting: true` can contain other Resource Groups as members in addition to individual Resource/Service Entities.
+
+**Example nesting structure:**
+```
+Tenant: Payments Platform
+  โ”‚
+  โ””โ”€โ”€ Resource Group: Deployment โ€” WebApp-v2         (nesting: true)
+        โ”œโ”€โ”€ Resource Group: Service โ€” Frontend        (nesting: true)
+        โ”‚     โ”œโ”€โ”€ Entity: Web Server VM 1
+        โ”‚     โ”œโ”€โ”€ Entity: Web Server VM 2
+        โ”‚     โ””โ”€โ”€ Entity: Load Balancer
+        โ””โ”€โ”€ Resource Group: Service โ€” Backend         (nesting: true)
+              โ”œโ”€โ”€ Entity: App Server VM 1
+              โ”œโ”€โ”€ Entity: App Server VM 2
+              โ””โ”€โ”€ Entity: Database
+```
+
+**Nesting rules:**
+- Circular nesting is invalid โ€” a group cannot contain itself directly or transitively
+- Nesting depth is declared per group โ€” `max_depth: unlimited` allows arbitrary depth
+- A child group inherits policy scope from parent groups โ€” policies applied to a parent group propagate to all child groups and their members
+- Cost rollup propagates up the nesting hierarchy
+
+---
+
+## 4. DCM System Policies for Resource Grouping
+
+| Policy | Rule | Enforcement |
+|--------|------|-------------|
+| `GRP-001` | Every Resource/Service Entity must belong to exactly one DCM Tenant | Enforced at Entity creation |
+| `GRP-002` | A Resource/Service Entity cannot be removed from its Tenant without being transferred to another Tenant | Enforced at all lifecycle states |
+| `GRP-003` | Circular nesting in Resource Groups is invalid | Enforced at group membership modification |
+| `GRP-004` | Custom Resource Groups must implement the full Resource Group Interface | Enforced at group registration |
+| `GRP-005` | Exclusive membership groups must reject membership requests that violate exclusivity | Enforced at group membership addition |
+
+---
+
+## 5. Grouping and DCM Capabilities
+
+Resource Groups enable the following DCM capabilities at the group scope:
+
+| Capability | Tenant | Resource Group |
+|------------|--------|---------------|
+| Cost Attribution | โœ… Primary | โœ… Rollup within group |
+| Policy Scope | โœ… | โœ… |
+| Drift Detection Scope | โœ… | โœ… |
+| Rehydration Scope | โœ… Full Tenant | โœ… Group and dependencies |
+| Audit Scope | โœ… | โœ… |
+| Placement Constraints | โœ… | โœ… |
+| Sovereignty Boundary | โœ… | โœ… |
+
+---
+
+## 6. Process Resources and Grouping
+
+Process Resource Entities follow the same grouping rules as Infrastructure Resource Entities:
+
+- Must belong to exactly one DCM Tenant โ€” non-overridable
+- Can optionally belong to Resource Groups
+- Typically grouped under the same Deployment or Service group as the resources they operate on
+- Tenant membership ensures cost attribution for execution resources
+- Group membership enables operational scoping โ€” "show me all automation jobs that ran against this Deployment"
+
+---
+
+## 7. Custom Resource Group Registration
+
+Implementors register custom Resource Group types as part of their DCM implementation. Custom group types must declare their full interface implementation:
+
+```yaml
+custom_group_type_registration:
+  uuid: 
+  type_name: 
+  version: 
+  description: 
+  implementing_organization_uuid: 
+  interface_version: 
+  default_membership_policy:
+    exclusive: 
+    max_memberships: 
+  nesting_supported: 
+  allowed_entity_types: 
+  status:
+    state: 
+```
+
+---
+
+## 8. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should there be a DCM-maintained registry of well-known custom group types to encourage standardization? | Interoperability | โœ… Resolved โ€” group_subclass open and advisory; community subclass catalog as non-authoritative reference; no validation or enforcement; see doc 15 (GRP-011) |
+| 2 | How does group membership interact with sovereignty โ€” can a group span sovereignty boundaries? | Sovereignty model | โœ… Resolved โ€” class-specific: tenant_boundary never cross-sovereignty (structural); resource_grouping permitted+policy restriction; policy_collection always permitted; composite governed by most restrictive member; see doc 15 (GRP-012) |
+| 3 | When a Tenant is decommissioned, what happens to its resources and group memberships? | Lifecycle management | โœ… Resolved โ€” four-phase staged decommission: pre-validation โ†’ resource decommission โ†’ membership cleanup โ†’ audit archival; child groups must be resolved first; audit records never destroyed; see doc 15 (GRP-013) |
+| 4 | Should Resource Groups support time-bounded membership โ€” a resource belongs to a group for a defined period? | Operational flexibility | โœ… Resolved โ€” valid_from/valid_until already in Universal Group Model; on_expiry (remove/notify/suspend_member); Lifecycle Constraint Enforcer handles; MEMBER_REMOVE audit record; see doc 15 (GRP-014) |
+| 5 | How are group-level policies inherited by nested child groups โ€” is inheritance opt-in or opt-out? | Policy model | โœ… Resolved โ€” class-specific defaults profile-governed; tenant_boundary: opt_out (standard/prod), opt_in (fsi/sovereign); federation always opt_in; composite opt_out; see doc 15 (GRP-015) |
+
+---
+
+## 9. Related Concepts
+
+- **DCM Tenant** โ€” primary ownership boundary, mandatory for all entities
+- **Resource/Service Entity** โ€” the thing being grouped
+- **Policy Engine** โ€” enforces grouping system policies and evaluates group-level organizational policies
+- **Cost Analysis** โ€” rolls up costs through group hierarchies
+- **Drift Detection** โ€” can be scoped to a group
+- **Rehydration** โ€” can target a group as the unit of reconstruction
+- **Field-Level Provenance** โ€” group membership changes are recorded in entity provenance
+
+
+---
+
+## 10. Cross-Tenant Authorization Lifecycle
+
+Cross-tenant authorizations are the formal mechanism by which one Tenant grants another Tenant access to a shared resource or allocation. They are DCMGroup instances with `group_class: cross_tenant_authorization`.
+
+### 10.1 Creation
+
+Cross-tenant authorizations are created by:
+
+| Actor | Mechanism | When |
+|-------|-----------|------|
+| Granting Tenant Admin | Explicit manual grant via Admin API | Normal cross-tenant sharing setup |
+| Platform Admin | Emergency authorization | Operational incident; requires dual approval in fsi/sovereign |
+| Policy (pre-authorization) | GateKeeper policy auto-creates authorization | Pre-approved sharing patterns |
+
+```yaml
+cross_tenant_authorization:
+  artifact_metadata:
+    uuid: 
+    handle: "xta/networkops/appteam/vlan-100"
+    version: "1.0.0"
+    status: active
+  group_class: cross_tenant_authorization
+  granting_tenant_uuid: 
+  consuming_tenant_uuid: 
+  authorized_resource_types: [Network.VLAN, Network.IPAddressPool]
+  authorized_entity_uuids: []    # null = all resources of declared types
+  duration: P1Y                                   # null = perpetual until revoked
+  expires_at: 
+  created_by: 
+  created_at: 
+  purpose: "AppTeam VMs require VLAN-100 attachment for production network access"
+```
+
+### 10.2 Duration and Renewal
+
+- **Fixed duration** (`duration: P1Y`): expires automatically. Notification sent P30D before expiry. Consuming Tenant must request renewal. If not renewed, enters EXPIRING state, then EXPIRED.
+- **Perpetual** (`duration: null`): active until explicitly revoked. No automatic expiry.
+- **Renewal**: consuming Tenant submits a new authorization request. Granting Tenant approves. New authorization created; old one superseded.
+
+### 10.3 Revocation
+
+```
+Granting Tenant admin revokes authorization
+  โ”‚
+  โ–ผ Authorization status โ†’ REVOKED
+  โ”‚
+  โ–ผ Identify active allocations and stakes under this authorization
+  โ”‚   All cross-tenant allocations/stakes enter PENDING_REVIEW
+  โ”‚
+  โ–ผ Notifications sent to:
+  โ”‚   Consuming Tenant Admin (action required: migrate or release)
+  โ”‚   Affected resource owners in consuming Tenant
+  โ”‚   Platform Admin (informational)
+  โ”‚
+  โ–ผ Resolution deadline: P30D (configurable; P7D for fsi/sovereign)
+  โ”‚
+  โ”œโ”€โ”€ Consuming Tenant releases stakes/allocations โ†’ authorization closes cleanly
+  โ””โ”€โ”€ Deadline exceeded โ†’ Platform Admin escalation
+      Policy may declare automatic release on deadline exceeded
+```
+
+### 10.4 What Happens to Active Allocations on Revocation
+
+Resources already allocated under a now-revoked authorization are NOT immediately decommissioned โ€” this would break production workloads. Instead:
+- The allocation/stake relationship enters PENDING_REVIEW
+- The authorization revocation is recorded as the `pending_review_trigger`
+- The consuming Tenant has a grace period to migrate or release
+- Automatic decommission on deadline exceeded is a policy declaration, not a default
+
+### 10.5 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `XTA-001` | Cross-tenant authorizations require explicit creation by the granting Tenant admin, a platform admin, or a pre-authorization policy. They are never implicitly created. |
+| `XTA-002` | Fixed-duration authorizations generate a P30D expiry warning notification. Non-renewal results in EXPIRING then EXPIRED states. |
+| `XTA-003` | Authorization revocation places active allocations and stakes in PENDING_REVIEW with a policy-governed grace period. Resources are not automatically decommissioned on revocation. |
+| `XTA-004` | Automatic decommission of resources on authorization expiry or revocation requires explicit policy declaration. It is not the default behavior. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/09-entity-relationships.md b/content/docs/data-model/09-entity-relationships.md
new file mode 100644
index 0000000..5fdf8cf
--- /dev/null
+++ b/content/docs/data-model/09-entity-relationships.md
@@ -0,0 +1,1076 @@
+# DCM Data Model โ€” Entity Relationships
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Resource Grouping](08-resource-grouping.md) | [Information Providers](10-information-providers.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + POLICY**
+>
+> Data: relationship records. Policy: Lifecycle Policy output schema
+
+
+
+---
+
+## 1. Purpose
+
+The DCM Entity Relationship model is the **universal mechanism for expressing relationships between any two entities in DCM** โ€” whether between two Resource/Service Entities, between an entity and external business data, or between entities at the service definition level.
+
+A single relationship model is used everywhere. There is no separate binding mechanism for storage, no separate dependency graph structure, no separate business data association mechanism. One model serves all relationship types across the full lifecycle โ€” from pre-realization planning through to post-realization management, drift detection, cost rollup, and rehydration.
+
+This document supersedes the dependency graph concept from the Service Dependencies document for data structure purposes. The Service Dependencies document retains content on rehydration ordering and failure handling, which operate on the relationship graph defined here.
+
+---
+
+## 2. Design Principle
+
+**Single model. Minimum variance. Simple by default.**
+
+The worst outcome is a data model with different mechanisms for expressing similar concepts. Every relationship in DCM โ€” whether a VM requires storage, an application contains a web server, or a resource references a Business Unit โ€” is expressed using the same structure. The only things that vary are the relationship type, role, and nature โ€” all of which are declared fields, not structural differences.
+
+---
+
+## 3. The Universal Relationship Structure
+
+Every relationship is a first-class data object with its own UUID. It is recorded **bidirectionally** โ€” on both participating entities. The same `relationship_uuid` appears on both sides, identifying the relationship itself.
+
+### 3.1 Relationship Record Structure
+
+```yaml
+relationship:
+  relationship_uuid: 
+  
+  # This entity's perspective
+  this_entity_uuid: 
+  this_role: 
+  
+  # The related entity
+  related_entity_uuid: 
+  related_entity_type: 
+  related_entity_role: 
+  
+  # For external entities only
+  information_provider_uuid: 
+  information_type: 
+  lookup_method: 
+  
+  # Relationship semantics
+  relationship_type: 
+  nature: 
+  
+  # Lifecycle policy โ€” for constituent and operational relationships only
+  lifecycle_policy:
+    on_related_destroy: 
+    on_related_suspend: 
+    on_related_modify: 
+  
+  # Metadata
+  version: 
+  status: 
+  created_timestamp: 
+  created_by_uuid: 
+  
+  provenance:
+    
+```
+
+### 3.2 Bidirectional Recording
+
+Every relationship is recorded on both participating entities. The `relationship_uuid` is identical on both sides โ€” it identifies the relationship itself, not one side of it.
+
+**Example โ€” VM requires Storage:**
+
+```yaml
+# On the VM Entity
+relationships:
+  - relationship_uuid: "rel-uuid-001"
+    this_entity_uuid: "vm-uuid-001"
+    this_role: compute
+    related_entity_uuid: "storage-uuid-001"
+    related_entity_type: internal
+    related_entity_role: storage
+    relationship_type: requires
+    nature: constituent
+    lifecycle_policy:
+      on_related_destroy: destroy
+      on_related_suspend: suspend
+      on_related_modify: notify
+
+# On the Storage Entity
+relationships:
+  - relationship_uuid: "rel-uuid-001"
+    this_entity_uuid: "storage-uuid-001"
+    this_role: storage
+    related_entity_uuid: "vm-uuid-001"
+    related_entity_type: internal
+    related_entity_role: compute
+    relationship_type: required_by
+    nature: constituent
+    lifecycle_policy:
+      on_related_destroy: destroy
+      on_related_suspend: suspend
+      on_related_modify: notify
+```
+
+---
+
+## 4. Relationship Types
+
+Relationship types form a fixed standard vocabulary. Every type has an inverse โ€” when you record the relationship on both entities, the type is expressed from each entity's perspective.
+
+| Type | Inverse | Meaning |
+|------|---------|---------|
+| `requires` | `required_by` | This entity cannot function without the related entity |
+| `depends_on` | `dependency_of` | This entity uses the related entity but can degrade without it |
+| `contains` | `contained_by` | This entity is a logical container for the related entity |
+| `references` | `referenced_by` | This entity references the related entity without owning or requiring it |
+| `peer` | `peer` | Equal relationship โ€” neither owns, requires, or contains the other |
+| `manages` | `managed_by` | This entity has lifecycle management authority over the related entity |
+
+---
+
+## 5. Relationship Roles
+
+Roles describe the **function** a related entity serves in a relationship. They are semantic labels that carry meaning for humans and for policy evaluation โ€” they do not affect system behavior directly.
+
+### 5.1 Standard Roles (DCM-defined)
+
+| Role | Description |
+|------|-------------|
+| `compute` | Processing resource โ€” VM, container, bare metal |
+| `storage` | Storage resource โ€” block, object, file |
+| `networking` | Network resource โ€” IP, VLAN, subnet, port |
+| `security` | Security resource โ€” firewall rule, certificate, HSM |
+| `database` | Database resource โ€” relational, NoSQL, time-series |
+| `web` | Web tier resource โ€” web server, reverse proxy, CDN |
+| `app` | Application tier resource โ€” app server, runtime |
+| `cache` | Caching resource โ€” in-memory cache, CDN layer |
+| `queue` | Messaging resource โ€” message queue, event stream |
+| `pipeline` | Pipeline resource โ€” CI/CD, data pipeline |
+| `identity` | Identity resource โ€” service account, credential |
+| `monitoring` | Monitoring resource โ€” metrics, logging, alerting |
+| `business_unit` | Business Unit association |
+| `cost_center` | Cost Center association |
+| `product_owner` | Product Owner association |
+| `regulatory_scope` | Regulatory or compliance scope association |
+
+### 5.2 Custom Roles (extensible)
+
+Organizations register custom roles for domain-specific relationship semantics. Custom roles are semantic labels only โ€” they do not change system behavior. DCM core ignores unknown custom roles in operational decisions but carries them in payloads for downstream consumers.
+
+```yaml
+custom_role_registration:
+  uuid: 
+  name: 
+  description: 
+  registered_by_tenant_uuid: 
+  category: 
+  version: 
+  status: 
+```
+
+---
+
+## 6. Relationship Nature
+
+Nature describes the **structural character** of a relationship โ€” what it means for the entities involved.
+
+| Nature | Meaning | Lifecycle Policy | Example |
+|--------|---------|-----------------|---------|
+| `constituent` | The related entity is a required component of this entity's definition | Required โ€” declared on relationship | VM requires its boot disk |
+| `operational` | The related entity is needed for operation but is not part of the definition | Required โ€” declared on relationship | Web server depends on load balancer |
+| `informational` | The related entity provides context or reference only โ€” no operational dependency | Not applicable | Resource references its Business Unit |
+
+---
+
+## 6a. Relationship Type ร— Nature Matrix
+
+The two dimensions of every relationship โ€” type and nature โ€” form a matrix of valid combinations. This matrix makes explicit what each combination means semantically and what behavioral rules apply. Not all 18 combinations are valid.
+
+| | `constituent` | `operational` | `informational` |
+|---|---|---|---|
+| **`requires`** | โœ… **Core constituent** โ€” entity cannot function without this component; component is part of its definition. Lifecycle policy required. | โœ… **Hard operational dependency** โ€” entity cannot function without this but it is not a component. Lifecycle policy required. | โš ๏ธ **Invalid** โ€” if an entity truly requires something, it has an operational or constituent dependency, not merely informational context. |
+| **`depends_on`** | โœ… **Soft constituent** โ€” entity degrades without this component but is not fully broken. Lifecycle policy required. | โœ… **Primary cell for allocated resources** โ€” soft operational dependency. Cross-tenant allocations live here. Lifecycle policy required. | โœ… **Awareness dependency** โ€” entity is aware of and tracks this entity but has no hard operational dependency. No lifecycle policy. |
+| **`contains`** | โœ… **Ownership container** โ€” this entity logically owns and contains the related entity as a component. Lifecycle policy required. | โš ๏ธ **Rare** โ€” containing something operationally is unusual; most containment is constituent. Use with explicit justification. | โŒ **Invalid** โ€” containing something purely informational has no semantic meaning. |
+| **`references`** | โŒ **Invalid** โ€” a reference implies no ownership or dependency; constituent implies the opposite. | โŒ **Invalid** โ€” if there is an operational dependency, use `depends_on`. A reference that creates operational coupling is mismodeled. | โœ… **Pure informational reference** โ€” primary cell for Business Unit, Cost Center, Product Owner relationships. No lifecycle policy. |
+| **`peer`** | โŒ **Invalid** โ€” peers cannot be constituent components of each other. | โœ… **Operational peers** โ€” equal entities with mutual operational interdependency. Lifecycle policy on each side. | โœ… **Informational peers** โ€” equal entities that are aware of each other. No lifecycle policy. |
+| **`manages`** | โœ… **Component management** โ€” this entity has lifecycle authority over a component it manages. Lifecycle policy required. | โœ… **Operational management** โ€” this entity manages the operations of another entity without owning it. Lifecycle policy required. | โœ… **Audit/reporting management** โ€” management relationship for visibility only. No lifecycle policy. |
+
+**Key behavioral rules derived from the matrix:**
+
+- Any `constituent` or `operational` relationship **must** declare a lifecycle policy (REL-004, REL-008)
+- `constituent` + `requires` is the strongest possible relationship โ€” both the entity and its component are mutually dependent; cross-tenant is prohibited (REL-010)
+- `operational` + `depends_on` is the **allocated resource cell** โ€” this is where cross-tenant allocations are modeled
+- `informational` + `references` is the **business context cell** โ€” Business Unit, Cost Center, Person relationships live here
+- `โŒ Invalid` combinations must be rejected by the Policy Engine at request time
+
+---
+
+## 6b. Cross-Tenant Relationships
+
+### 6b.1 The Governing Principle
+
+The relationship **nature** determines whether a cross-tenant relationship is permitted:
+
+| Nature | Cross-Tenant Permitted? | Governing Rule |
+|--------|------------------------|---------------|
+| `constituent` | โŒ Never | REL-010 โ€” DCM System Policy |
+| `operational` | โœ… With explicit dual authorization | REL-011 โ€” both Tenants must authorize |
+| `informational` | โœ… Unless denied by hard tenancy | REL-012 โ€” blocked only by `deny_all` |
+
+### 6b.2 Hard Tenancy Declaration
+
+Tenants declare their cross-tenant relationship policy. This is enforced by the GateKeeper Policy Engine at request time:
+
+```yaml
+tenant:
+  uuid: 
+  hard_tenancy:
+    cross_tenant_relationships: explicit_only
+    # deny_all:            no relationships of any nature may cross this boundary
+    # explicit_only:       ALL cross-tenant must be explicitly authorized (DEFAULT)
+    # operational_permitted: operational cross-tenant permitted; informational requires explicit auth
+    # allow_all:           all cross-tenant permitted โ€” requires justification
+```
+
+**Default is `explicit_only` โ€” informational sharing is not open by default.** Every cross-tenant relationship of any nature requires an explicit `cross_tenant_authorization` record. This closes the model โ€” cross-tenant access must be deliberately granted, not passively permitted.
+
+### 6b.3 DCM System Policies for Cross-Tenant Relationships
+
+| Policy | Rule |
+|--------|------|
+| `REL-010` | Constituent relationships may not cross Tenant boundaries |
+| `REL-011` | Cross-tenant operational relationships require explicit authorization from both the owning Tenant and the consuming Tenant |
+| `REL-012` | A Tenant with `hard_tenancy.cross_tenant_relationships: deny_all` may not participate in any cross-tenant relationship in any direction |
+| `XTA-001` | Cross-tenant information sharing is closed by default โ€” explicit authorization required for all cross-tenant relationships of any nature (see Policy Organization document Section 6) |
+| `XTA-002` | Cross-tenant authorizations must specify who, what, when, and where |
+| `XTA-003` | More specific authorizations take precedence: field_specific > resource_specific > tenant_global |
+| `XTA-004` | All cross-tenant authorization decisions are policy-driven and DCM-enforced |
+| `XTA-005` | Sovereignty constraints declared by either Tenant must be honored by all cross-tenant relationships |
+
+---
+
+## 6c. Allocated Resources โ€” Cross-Tenant Operational Model
+
+### 6c.1 Concept
+
+An **Allocated Resource** is a pre-defined, discrete slice of a parent resource โ€” provisioned by the owning Tenant and made available for consuming Tenants to claim. The allocated resource becomes a **first-class entity** in the consuming Tenant's scope with its own UUID, its own lifecycle, and its own governance โ€” while maintaining a formal `depends_on` + `operational` relationship to the parent resource across the Tenant boundary.
+
+This models real infrastructure practice: the network team pre-carves VLANs, the storage team pre-partitions pools, the platform team pre-defines availability zones. Consumers claim from what is available.
+
+The relationship type is `depends_on` + `operational` โ€” the allocated entity depends on the parent operationally but is not a constituent component of it. The allocation is the relationship; the entity itself is independently governed.
+
+### 6c.2 Parent Resource โ€” Available Allocations
+
+The owning Tenant pre-defines allocations on the parent resource:
+
+```yaml
+parent_resource_entity:
+  uuid: 
+  tenant_uuid: 
+
+  available_allocations:
+    - allocation_uuid: 
+      allocation_type: Network.VLANRange
+      allocation_spec:
+        vlan_range: "100-199"
+        bandwidth: "10Gbps"
+      status: 
+      claimable_by:
+        - tenant_uuid: 
+        - tenant_uuid: 
+        # Empty list = any authorized Tenant may claim
+
+  active_allocations:
+    - allocation_uuid: 
+      claimed_by_tenant_uuid: 
+      claimed_entity_uuid: 
+      claimed_at: 
+      notification_endpoint: 
+      # Parent uses this to notify Tenant A of lifecycle changes
+```
+
+### 6c.3 Allocated Entity โ€” In the Consuming Tenant
+
+When a consuming Tenant claims an available allocation, DCM creates a first-class entity in the consuming Tenant's scope:
+
+```yaml
+allocated_entity:
+  uuid: 
+  entity_type: allocated_resource
+  resource_type_uuid: 
+  tenant_uuid:   # Belongs to the consuming Tenant
+
+  allocation_spec:
+    vlan_range: "100-199"
+    bandwidth: "10Gbps"
+    # The specific slice allocated to this Tenant
+
+  parent_allocation:
+    parent_entity_uuid: 
+    parent_tenant_uuid: 
+    allocation_uuid: 
+
+  lifecycle_state: OPERATIONAL
+
+  parent_lifecycle_policy:
+    on_parent_destroy: notify_then_detach
+    on_parent_suspend: suspend
+    on_parent_maintenance: notify
+    on_parent_degrade: notify
+    on_parent_capacity_change: notify
+
+  relationships:
+    - relationship_uuid: 
+      related_entity_uuid: 
+      related_entity_type: internal
+      related_entity_tenant_uuid: 
+      relationship_type: depends_on
+      nature: operational
+      cross_tenant: true
+      allocation_uuid: 
+      authorized_by:
+        owning_tenant_policy_uuid: 
+        consuming_tenant_policy_uuid: 
+
+  artifact_metadata:
+    
+```
+
+### 6c.4 Lifecycle Event Propagation
+
+When the parent resource changes state, DCM iterates all active allocations and propagates according to each allocation's `parent_lifecycle_policy`:
+
+```
+Parent resource enters MAINTENANCE
+  โ”‚
+  โ–ผ
+DCM iterates active_allocations
+  โ”‚
+  For each active allocation:
+  โ”‚  Read parent_lifecycle_policy.on_parent_maintenance
+  โ”‚  โ†’ notify: dispatch lifecycle event to consuming Tenant
+  โ”‚  โ†’ suspend: transition allocated entity to SUSPENDED state
+  โ”‚  โ†’ detach: terminate relationship, allocated entity becomes independent
+  โ”‚
+  Policy Engine evaluates each propagation:
+  โ”‚  SLA commitments that gate maintenance?
+  โ”‚  Override policies in consuming Tenant?
+  โ–ผ
+Events dispatched via notification_endpoint on each active_allocation record
+```
+
+### 6c.5 Claiming Flow
+
+```
+Parent Tenant pre-defines available_allocations on parent resource
+  โ”‚
+  โ–ผ
+Consuming Tenant A submits claim request
+  โ”‚  Specifies: parent_entity_uuid, allocation_uuid
+  โ–ผ
+Policy Engine evaluates:
+  โ”‚  Is allocation_uuid still available?
+  โ”‚  Is Tenant A in claimable_by list (or list is open)?
+  โ”‚  Does Tenant A's cross_tenant policy permit this?
+  โ”‚  Does Infrastructure Tenant's cross_tenant policy permit this?
+  โ–ผ
+DCM creates:
+  โ”‚  Allocated Entity (owned by Tenant A) with UUID
+  โ”‚  depends_on / dependency_of relationship (bidirectional, cross_tenant: true)
+  โ”‚  Updates parent's available_allocation status: available โ†’ claimed
+  โ”‚  Adds record to parent's active_allocations
+  โ”‚  Provenance recorded on both entities
+  โ–ผ
+Infrastructure Tenant owner notified of new claim
+  โ”‚  Via owned_by.notification_endpoint on the parent entity
+```
+
+---
+
+## 7. Lifecycle Policies
+
+Lifecycle policies declare what happens to an entity when its related entity changes state. They apply to `constituent` and `operational` relationships only โ€” `informational` relationships have no lifecycle implications.
+
+### 7.1 Policy Actions
+
+| Action | Meaning |
+|--------|---------|
+| `destroy` | Destroy this entity when the related entity is destroyed |
+| `retain` | Keep this entity when the related entity is destroyed โ€” it becomes independent |
+| `detach` | Detach this entity from the relationship โ€” relationship terminated, entity retained |
+| `notify` | Notify appropriate personas and trigger Policy Engine evaluation โ€” no automatic action |
+| `suspend` | Suspend this entity when the related entity is suspended |
+| `cascade` | Cascade the change from the related entity to this entity |
+| `ignore` | Take no action โ€” the change to the related entity does not affect this entity |
+
+### 7.2 Lifecycle Action Hierarchy โ€” Save Overrides Destroy
+
+When a shared resource has multiple active relationships and a lifecycle event triggers, each relationship may produce a different action recommendation. DCM resolves conflicts using a deterministic hierarchy โ€” **the most conservative action always wins**:
+
+```
+retain        โ† most conservative โ€” entity preserved unconditionally
+  โ”‚
+notify        โ† inform and wait โ€” human decision required
+  โ”‚
+suspend       โ† temporarily inactive โ€” reversible
+  โ”‚
+detach        โ† relationship released โ€” entity becomes independent
+  โ”‚
+cascade       โ† propagate state change from related entity
+  โ”‚
+destroy       โ† least conservative โ€” entity terminated
+```
+
+**The save_overrides_destroy rule (REL-018):** If any active relationship recommends `retain`, the entity is retained regardless of what any other relationship recommends โ€” including relationships with `override: immutable` lifecycle policies. `retain` is the save. It always beats `destroy`.
+
+This rule applies automatically and silently when the hierarchy resolves cleanly (e.g., `retain` beats `destroy`). It is recorded in the `lifecycle_conflict_record` with severity `info` for audit purposes but requires no notification.
+
+### 7.3 Lifecycle Conflict Detection
+
+**Not all multi-recommendation scenarios are conflicts.** The hierarchy resolves most cases deterministically. A conflict worth surfacing occurs when:
+
+1. **Adjacent hierarchy levels** โ€” two relationships recommend actions that are one step apart (e.g., `notify` vs `suspend`) โ€” the hierarchy resolves it but the ambiguity is worth surfacing
+2. **An immutable lifecycle lock couldn't be honored** โ€” a GateKeeper set `on_related_destroy: destroy` with `immutable_ceiling: absolute` but `retain` from another relationship won per REL-018
+3. **`notify` is the winning action** โ€” inherently means human decision required; the notification should include the full conflict picture
+
+**Conflict severity:**
+
+| Scenario | Severity | Action |
+|----------|---------|--------|
+| `retain` beats `destroy` โ€” non-adjacent levels | `info` | Logged only โ€” working as designed |
+| All relationships agree | None | No record needed |
+| Adjacent levels (e.g., `notify` vs `suspend`) | `warning` | Notify entity owner and affected policy owners |
+| `notify` is the winning action | `warning` | Notify owner โ€” human decision required |
+| Immutable lifecycle lock overridden by REL-018 | `critical` | Notify entity owner, policy owner, and platform admin |
+
+**Lifecycle conflict record:**
+
+```yaml
+lifecycle_conflict_record:
+  entity_uuid: 
+  event_trigger: 
+  triggering_entity_uuid: 
+  action_recommendations:
+    - relationship_uuid: 
+      related_entity_uuid: 
+      recommended_action: destroy
+      source: lifecycle_policy
+    - relationship_uuid: 
+      related_entity_uuid: 
+      recommended_action: retain
+      source: gatekeeper_policy
+      policy_uuid: 
+    - relationship_uuid: 
+      related_entity_uuid: 
+      recommended_action: notify
+      source: lifecycle_policy
+  resolved_action: retain
+  resolution_rule: save_overrides_destroy
+  conflict_detected: true
+  conflict_severity: info
+  notifications_sent:
+    - recipient_uuid: 
+      message: "Lifecycle conflict resolved: retain overrode destroy and notify."
+  recorded_at: 
+```
+
+### 7.4 Lifecycle Policy Authority Hierarchy
+
+Lifecycle policies follow the same three-tier authority model as override control:
+
+```
+Resource Type Specification default (lowest โ€” portable default)
+  โ”‚
+  โ–ผ
+Provider Catalog Item default (provider preference)
+  โ”‚
+  โ–ผ
+Consumer declaration (at request time โ€” within Resource Type bounds)
+  โ”‚
+  โ–ผ
+DCM System Policy (non-overridable โ€” sovereignty and compliance mandates)
+```
+
+**Example:** A DCM System Policy might declare that all storage entities in a PCI-DSS scope must `retain` when their parent VM is destroyed โ€” regardless of what the provider default or consumer declared.
+
+---
+
+## 7a. Shared Resource Model โ€” Same-Tenant
+
+### 7a.1 Concept
+
+A **Shared Resource** is an entity within a single Tenant that has active relationships from multiple parent entities. Rather than being exclusively owned by one parent, it is referenced by N parents โ€” each with its own lifecycle relationship.
+
+This is the same-tenant counterpart to the cross-tenant Allocated Resource model. Both use reference counting to defer destructive actions. The sharing model applies within a Tenant; the allocation model applies across Tenant boundaries.
+
+**Examples:** Shared NFS volume mounted by multiple VMs. Shared database cluster used by multiple application services. Shared VLAN used by multiple VMs. Shared TLS certificate used by multiple services.
+
+### 7a.2 The `sharing_model` Declaration
+
+The Resource Type Specification declares whether instances of a type can be shared. Individual entities carry the runtime sharing state:
+
+```yaml
+# On the Resource Type Specification
+resource_type_spec:
+  fully_qualified_name: Storage.SharedVolume
+  shareability:
+    allowed: true
+    default_sharing_scope: tenant    # tenant | cross_tenant
+    max_active_relationships: null   # null = unlimited; integer = cap (e.g., license seats)
+
+# On the entity instance
+entity:
+  uuid: 
+  sharing_model:
+    shareable: true
+    sharing_scope: tenant
+    active_relationship_count: 3     # DCM maintains this โ€” do not set manually
+    minimum_relationship_count: 0    # below this, on_last_relationship_released fires
+    on_last_relationship_released: 
+    # destroy: entity destroyed when last relationship is released
+    # retain:  entity persists independently โ€” becomes unowned
+    # notify:  notify owner, entity enters PENDING_DECISION
+```
+
+**`shareability.allowed: false`** on a Resource Type (e.g., `Compute.BootDisk`) means the Policy Engine rejects any attempt to create a second active constituent or operational relationship to an instance. Boot disks, primary network interfaces, and similar exclusively-owned resources are non-shareable by type definition (REL-017).
+
+### 7a.3 Reference Count Lifecycle
+
+DCM maintains `active_relationship_count` automatically:
+
+- **Relationship created** โ†’ `active_relationship_count` incremented
+- **Relationship released** (parent decommissioned, relationship detached) โ†’ `active_relationship_count` decremented
+- **Informational relationships** โ†’ never counted (REL-016)
+- **Count reaches `minimum_relationship_count`** โ†’ `on_last_relationship_released` fires
+
+When a parent entity is destroyed and has a relationship to a shared resource:
+
+```
+Parent entity destroyed
+  โ”‚
+  โ–ผ
+DCM collects action recommendations from all active relationships on shared resource
+  โ”‚  Each relationship's lifecycle policy produces one recommendation
+  โ”‚  Informational relationships excluded
+  โ”‚
+  โ–ผ
+Action resolution โ€” save_overrides_destroy hierarchy (REL-018)
+  โ”‚  Most conservative recommendation wins
+  โ”‚  Lifecycle conflict record created if multiple recommendations differ
+  โ”‚
+  โ–ผ
+Execute winning action
+  โ”‚  retain โ†’ shared resource unaffected
+  โ”‚  notify โ†’ PENDING_DECISION state, notifications dispatched
+  โ”‚  suspend โ†’ shared resource suspended
+  โ”‚  detach โ†’ parent's relationship released, count decremented
+  โ”‚  destroy โ†’ only if count reaches minimum_relationship_count (REL-015)
+  โ”‚
+  โ–ผ
+Deferred destruction record created (if action was deferred)
+```
+
+### 7a.4 Deferred Destruction Records
+
+Every time a destructive action is deferred by the reference count mechanism:
+
+```yaml
+deferred_destruction_record:
+  entity_uuid: 
+  triggering_request_uuid: 
+  triggering_relationship_uuid: 
+  relationship_count_before: 3
+  relationship_count_after: 2
+  action_taken: deferred
+  reason: "active_relationship_count above minimum. Destruction deferred."
+  remaining_relationships:
+    - relationship_uuid: 
+      related_entity_uuid: 
+      relationship_type: required_by
+    - relationship_uuid: 
+      related_entity_uuid: 
+      relationship_type: dependency_of
+  recorded_at: 
+```
+
+When the last relationship is released:
+
+```yaml
+deferred_destruction_record:
+  relationship_count_before: 1
+  relationship_count_after: 0
+  action_taken: "on_last_relationship_released โ†’ destroy"
+  reason: "Last active relationship released. Executing on_last_relationship_released."
+  recorded_at: 
+```
+
+### 7a.5 Unified with the Allocated Resource Model
+
+The same-tenant sharing model and the cross-tenant allocated resource model are the same concept at different scopes:
+
+| Dimension | Same-Tenant Sharing | Cross-Tenant Allocation |
+|-----------|--------------------|-----------------------|
+| Scope | Within one Tenant | Across Tenant boundaries |
+| Pre-definition | Not required โ€” relationships declared at request time | Parent pre-defines `available_allocations` |
+| Reference tracking | `active_relationship_count` on entity | `active_allocations` list on parent |
+| Destruction deferral | Deferred until count reaches minimum | Deferred until last allocation released |
+| Lifecycle events | `on_last_relationship_released` | `parent_lifecycle_policy` per allocation |
+| Governed by | REL-015 through REL-019 | REL-011, REL-014 |
+
+---
+
+## 8. Relationship Declarations โ€” Where They Live
+
+Relationship declarations exist at multiple levels, each building on the previous:
+
+### 8.1 Resource Type Specification (structural ceiling)
+
+Declares what relationships are **possible** for a resource type. Sets the ceiling โ€” lower levels can only declare relationships within these bounds.
+
+```yaml
+resource_type: Compute.VirtualMachine
+possible_relationships:
+  - role: storage
+    relationship_type: requires
+    nature: constituent
+    permitted_related_types:
+      - Storage.Block
+      - Storage.File
+    default_lifecycle_policy:
+      on_related_destroy: destroy
+      on_related_suspend: suspend
+    binding_types_permitted: [owned, referenced]
+    consumer_declarable: true
+    # Consumer can declare binding_type and lifecycle_policy override
+
+  - role: networking
+    relationship_type: requires
+    nature: constituent
+    permitted_related_types:
+      - Network.IPAddress
+    default_lifecycle_policy:
+      on_related_destroy: destroy
+    consumer_declarable: false
+    # DCM manages this automatically โ€” consumer cannot override
+```
+
+### 8.2 Catalog Item (offering-specific)
+
+Declares the **actual relationships** for a specific curated offering. Can only be more restrictive than the Resource Type Specification.
+
+```yaml
+catalog_item: Production VM
+relationships:
+  - role: storage
+    relationship_type: requires
+    nature: constituent
+    related_catalog_item_uuid: 
+    lifecycle_policy:
+      on_related_destroy: retain
+      # Overrides Resource Type default of destroy
+      # Storage persists even if VM is destroyed โ€” production data protection
+    binding_type: owned
+```
+
+### 8.3 Request Time (consumer-declared)
+
+The consumer declares relationships in their request. Bundled declarations (storage fields within a VM request) are automatically expanded into relationship records by the Request Payload Processor.
+
+```yaml
+# Explicit relationship declaration in a request
+request:
+  resource_type: Compute.VirtualMachine
+  # ... other fields ...
+  relationships:
+    - role: storage
+      relationship_type: requires
+      binding_type: referenced
+      related_entity_uuid: 
+      # Consumer referencing existing storage โ€” not creating new
+
+# Bundled declaration โ€” expanded automatically
+request:
+  resource_type: Compute.VirtualMachine
+  storage:
+    disks:
+      - name: boot
+        capacity: 100GB
+        # Processor expands this into a Storage Entity stub
+        # and a relationship record with binding_type: owned
+```
+
+### 8.4 External Data Relationships
+
+Relationships to external data entities follow the same structure with `related_entity_type: external`:
+
+```yaml
+# On a VM Entity โ€” relationship to external Business Unit
+relationships:
+  - relationship_uuid: 
+    this_entity_uuid: 
+    this_role: 
+    related_entity_uuid: 
+    related_entity_type: external
+    information_provider_uuid: 
+    information_type: Business.BusinessUnit
+    relationship_type: references
+    role: business_unit
+    nature: informational
+    lookup_method: primary_key
+```
+
+---
+
+## 9. Bundled Declaration Expansion
+
+When a consumer includes resource configuration as bundled fields (e.g., storage within a VM request), the Request Payload Processor expands these into first-class entities and relationship records.
+
+### 9.1 Expansion Process
+
+```
+Consumer submits bundled VM request with storage fields
+  โ”‚
+  โ–ผ
+Request Payload Processor
+  โ”‚  Reads expansion rules from Resource Type Specification
+  โ”‚  For each expandable field:
+  โ”‚    1. Creates a Resource/Service Entity stub (PENDING state)
+  โ”‚       with its own UUID, Tenant membership, Resource Type
+  โ”‚    2. Creates a Relationship record on both the parent stub
+  โ”‚       and the child stub
+  โ”‚    3. Applies lifecycle policy from:
+  โ”‚       consumer declaration โ†’ provider default โ†’ Resource Type default
+  โ”‚       โ†’ DCM System Policy override
+  โ”‚    4. Adds the child entity stub to the relationship graph
+  โ–ผ
+Policy Engine validates:
+  โ”‚  Binding type is permitted by Resource Type Specification
+  โ”‚  Consumer has override_matrix permission to declare binding type
+  โ”‚  Lifecycle policy is not overridden by a DCM System Policy
+  โ–ผ
+Service Provider receives:
+  โ”‚  Parent entity request payload
+  โ”‚  Child entity stub UUIDs embedded in parent payload
+  โ”‚  Provisions resources natively
+  โ”‚  Returns realized payloads for all entities in DCM unified format
+  โ–ผ
+DCM updates:
+  โ”‚  Parent entity: PENDING โ†’ REALIZED
+  โ”‚  Child entities: PENDING โ†’ REALIZED
+  โ”‚  All relationship records: status โ†’ active
+  โ”‚  Full provenance recorded on all entities and relationships
+```
+
+### 9.2 Expansion Rules in Resource Type Specification
+
+The expansion rule declares which fields expand into entities and how:
+
+```yaml
+field_definition:
+  field_name: storage
+  type: object
+  expansion:
+    expand_to_entity: true
+    entity_resource_type_uuid: 
+    entity_resource_type_name: Storage.Block
+    default_binding_type: owned
+    binding_types_permitted: [owned, referenced]
+    default_lifecycle_policy:
+      on_related_destroy: destroy
+      on_related_suspend: suspend
+    consumer_can_override_lifecycle: true
+    consumer_can_override_binding_type: true
+```
+
+---
+
+## 10. The Entity Relationship Graph
+
+All relationships across all entities form a traversable **Entity Relationship Graph** โ€” the complete map of how all entities in DCM relate to each other.
+
+### 10.1 Graph Properties
+
+- Every node is a Resource/Service Entity (internal or external reference)
+- Every edge is a Relationship with a UUID
+- The graph is bidirectional โ€” traversable from any node in any direction
+- Every node exists exactly once โ€” shared entities appear once with multiple relationship edges
+- Circular relationships are invalid and must be rejected
+
+### 10.2 Graph and the Four States
+
+The relationship graph exists across all four states:
+
+| State | Graph Role |
+|-------|-----------|
+| Intent State | Graph declared at request time โ€” nodes are intent stubs |
+| Requested State | Graph fully assembled โ€” nodes are PENDING entity stubs with UUIDs |
+| Realized State | Graph populated โ€” nodes are REALIZED entities with full provenance |
+| Discovered State | Graph used for comparison โ€” discovered entities matched against realized graph |
+
+### 10.3 Graph Applications
+
+| Application | How the Graph is Used |
+|-------------|----------------------|
+| **Rehydration** | Full graph traversal from a root entity โ€” all related entities identified and realized in dependency order |
+| **Cost Rollup** | Graph traversal accumulates costs across all related constituent entities |
+| **Drift Detection** | Discovered State graph compared against Realized State graph โ€” structural and data differences identified |
+| **Decommission** | Graph traversal determines decommission order โ€” lifecycle policies applied at each edge |
+| **Placement** | Pre-realization graph used to understand full resource footprint for placement decisions |
+| **Impact Analysis** | Graph traversal from any node identifies all entities affected by a change |
+
+---
+
+## 11. Relationship Integrity
+
+### 11.1 DCM System Policies for Relationships
+
+| Policy | Rule |
+|--------|------|
+| `REL-001` | Every relationship must have a UUID |
+| `REL-002` | Every relationship must be recorded on both participating entities |
+| `REL-003` | Circular relationships are invalid and must be rejected |
+| `REL-004` | A constituent or operational relationship must have a lifecycle policy declared somewhere in the authority chain before provider dispatch |
+| `REL-005` | External relationships must reference a registered Information Provider |
+| `REL-006` | Relationship types must be from the standard vocabulary |
+| `REL-007` | Consumer-declared binding types must be permitted by the Resource Type Specification |
+| `REL-008` | A constituent relationship lifecycle policy may not be set to `ignore` for `on_related_destroy` |
+| `REL-009` | Lifecycle policy conflicts between policies are resolved by the standard Policy Engine authority hierarchy โ€” no special case |
+| `REL-010` | Constituent relationships may not cross Tenant boundaries |
+| `REL-011` | Cross-tenant operational relationships require explicit authorization from both the owning Tenant and the consuming Tenant |
+| `REL-012` | A Tenant with `hard_tenancy.cross_tenant_relationships: deny_all` may not participate in any cross-tenant relationship in any direction |
+| `REL-013` | `โŒ Invalid` relationship type ร— nature combinations (per the matrix in Section 6a) must be rejected by the Policy Engine at request time |
+| `REL-014` | An allocated resource claim requires a matching `available` allocation record on the parent entity |
+| `REL-015` | A destructive lifecycle action on a shared resource entity (`ownership_model: shareable` (see [Ownership, Sharing, and Allocation](04b-ownership-sharing-allocation.md))) is deferred until `active_relationship_count` reaches `minimum_relationship_count` |
+| `REL-016` | Informational relationships do not contribute to `active_relationship_count` on shared resource entities |
+| `REL-017` | A Resource Type Specification with `shareability.allowed: false` must reject any attempt to create more than one active constituent or operational relationship to an instance of that type |
+| `REL-018` | When a lifecycle event produces multiple action recommendations on a shared resource, the most conservative action wins per the hierarchy: `retain > notify > suspend > detach > cascade > destroy` (save_overrides_destroy) |
+| `REL-019` | When lifecycle action recommendations conflict, a `lifecycle_conflict_record` is created. Conflicts at `warning` or `critical` severity trigger notifications to the entity owner and affected policy owners |
+
+### 11.2 Lifecycle Policy Conflict Resolution
+
+Lifecycle policy fields on relationships are fields. They carry the same `override` metadata, the same provenance obligations, and resolve under the same Policy Engine authority hierarchy as any other field in DCM. There is no special case โ€” minimum variance applies.
+
+**Authority chain for a relationship lifecycle policy field (lowest to highest):**
+
+```
+Resource Type Specification default
+  โ†’ Provider Catalog Item default
+    โ†’ Consumer declaration at request time
+      โ†’ Transformation Policy (may set override: constrained)
+        โ†’ Validation Policy (checks โ€” no modification)
+          โ†’ GateKeeper Policy (may set override: immutable)
+            โ†’ DCM System Policies REL-008, REL-009 (non-overridable)
+```
+
+**Within the Policy Engine**, the priority schema governs conflicts between policies at the same tier. Highest numeric priority value within a tier runs first. The first policy to set `override: immutable` on a lifecycle policy field locks it โ€” all subsequent policies in that execution find it locked and cannot modify it.
+
+**Conflict detection at ingestion** applies to lifecycle policy declarations in policies exactly as it does to layer fields:
+- Two policies both declare `on_related_destroy` for the same relationship type without priority differentiation โ†’ CONFLICT ERROR at ingestion โ€” both owners notified
+- One has higher priority value โ†’ Higher wins, documented in provenance
+- Equal priority โ†’ CONFLICT ERROR
+
+**`immutable_ceiling: absolute` applies here.** A sovereign compliance mandate that storage must always be retained when a VM is destroyed โ€” `on_related_destroy: retain` with `immutable_ceiling: absolute` โ€” cannot be overridden by any future policy regardless of priority.
+
+**Example โ€” compliant lifecycle policy field with override control:**
+
+```yaml
+lifecycle_policy:
+  on_related_destroy:
+    value: retain
+    metadata:
+      override: immutable
+      locked_by_policy_uuid: 
+      locked_at_level: global
+      basis_for_value: "Compliance mandate โ€” storage must outlive VM for audit retention"
+      immutable_ceiling: absolute
+    provenance:
+      origin:
+        source_type: policy
+        source_uuid: 
+        timestamp: 
+      modifications: []
+```
+
+### 11.2a Cross-Tenant Dependency System Policies
+
+| Policy | Rule |
+|--------|------|
+| `DEP-001` | Cross-tenant constituent dependencies are prohibited โ€” a dependency that would produce a constituent cross-tenant relationship is rejected at dependency graph construction time |
+| `DEP-002` | Cross-tenant operational dependencies require a valid available allocation record on the target resource โ€” failure returns `CROSS_TENANT_DEPENDENCY_UNAVAILABLE` |
+| `DEP-003` | A Resource Type Specification may only declare cross-tenant dependencies if explicitly marked `cross_tenant: permitted` โ€” default is `cross_tenant: not_permitted` |
+
+### 11.3 Relationship Versioning and Deprecation
+
+Relationships follow the universal versioning and deprecation model. A relationship version changes when its lifecycle policy, nature, or role changes. Terminated relationships are retained in provenance permanently.
+
+---
+
+## 12. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | How are relationship conflicts resolved โ€” two policies declare different lifecycle policies for the same relationship? | Policy model | โœ… Resolved โ€” standard Policy Engine authority hierarchy; REL-008 and REL-009 |
+| 2 | Should relationship roles be validated against the role registry at request time, or is validation advisory? | Operational complexity | โœ… Resolved โ€” advisory default; Resource Type Spec may declare permitted_relationship_roles with role_validation: advisory/enforced; community role catalog; see doc 09 Section 12 (REL-020) |
+| 3 | How does the relationship graph interact with multi-tenant scenarios โ€” can a relationship cross Tenant boundaries? | Multi-tenancy | โœ… Resolved โ€” nature governs; constituent never; operational with dual auth; informational unless deny_all; REL-010/011/012 |
+| 4 | Should there be a maximum relationship graph depth to prevent runaway complexity? | Operational governance | โœ… Resolved โ€” profile-governed max depth: 15 standard/prod, 10 fsi/sovereign; circular detection always enforced; depth = traversal distance; see doc 09 Section 12 (REL-021) |
+| 5 | How are shared entities represented in the relationship graph โ€” an entity required by multiple parents? | Graph model | โœ… Resolved โ€” sharing_model declaration; active_relationship_count; save_overrides_destroy hierarchy (REL-018); lifecycle_conflict_record; REL-015 through REL-019 |
+
+
+---
+
+## 14. Notification Traversal Rules
+
+The entity relationship graph is the source of truth for notification audiences. This section defines how relationships govern notification traversal for the Notification Model (doc 23).
+
+### 14.1 Relationship Properties Relevant to Notifications
+
+Every relationship carries two properties that the Notification Router uses for audience resolution:
+
+```yaml
+relationship:
+  type: attached_to
+  stake_strength: 
+  notification_relevance:
+    # Declared in the Resource Type Spec for this relationship type
+    # Can be overridden per relationship instance
+    notifiable_events: [entity.decommissioning, entity.state_changed, entity.ttl_expired]
+    traversal_depth: 1               # how many hops from this relationship
+    audience_role: stakeholder       # role assigned to notified party
+```
+
+### 14.2 Stake Strength and Notification Threshold
+
+Different event types use different minimum stake strengths for notification:
+
+| Event Category | Minimum Stake Strength | Rationale |
+|---------------|----------------------|-----------|
+| `entity.decommissioning` | optional | All stakeholders should know |
+| `entity.decommissioned` | optional | All stakeholders should know |
+| `entity.state_changed` (to FAILED/DEGRADED) | required | Only required stakeholders are affected |
+| `entity.state_changed` (to OPERATIONAL) | preferred | Recovery notification broader |
+| `entity.ttl_expired` | required | Only required stakeholders need to act |
+| `drift.detected` | โ€” (owner only) | Drift is the owner's concern |
+| `dependency.state_changed` | required | Only affects required dependents |
+
+The minimum stake strength threshold per event type is declared in the resource type specification and can be overridden by a platform-domain policy.
+
+### 14.3 Notification Traversal and Graph Depth
+
+Notification traversal respects the same depth limits as other graph operations (REL-021: max depth 15 standard/prod, 10 fsi/sovereign). However, notification traversal depth is typically much shallower โ€” most event types only traverse depth 1 (direct relationships).
+
+```
+VLAN-100 decommissioning (depth 1 traversal):
+  Direct relationships:
+    โ”œโ”€โ”€ VM-A (attached_to, required) โ†’ AppTeam notified as stakeholder
+    โ”œโ”€โ”€ VM-B (attached_to, required) โ†’ DevTeam notified as stakeholder
+    โ””โ”€โ”€ VM-C (attached_to, optional) โ†’ OpsTeam notified as observer
+  No depth-2 traversal โ€” VM-A's dependencies are not notified about VLAN changes
+```
+
+Security events (sovereignty violation, audit chain break) use depth 0 (system audiences only โ€” no relationship traversal needed).
+
+### 14.4 Notification Traversal Policies
+
+| Policy | Rule |
+|--------|------|
+| `REL-022` | Notification traversal follows relationship edges from the changed entity. Traversal depth per event type is declared in the Resource Type Specification. Default traversal depth is 1. |
+| `REL-023` | Notification traversal respects sovereignty boundaries. Cross-tenant notifications carry only content authorized for the receiving Tenant. |
+| `REL-024` | The same actor reached via multiple relationship paths receives a single notification with all applicable audience_roles listed. |
+
+
+---
+
+## 13. Related Concepts
+
+- **Entity Relationship Graph** โ€” the complete traversable graph of all entity relationships in DCM
+- **Information Provider** โ€” provider type for external data entities referenced in relationships
+- **Bundled Declaration Expansion** โ€” processor mechanism for expanding bundled fields into entities and relationships
+- **Lifecycle Policy** โ€” declares what happens to an entity when its related entity changes state
+- **Service Dependencies** โ€” document covering rehydration ordering and failure handling on the relationship graph
+- **Resource Type Specification** โ€” declares possible relationships for a resource type
+- **External Entity Reference** โ€” stable pointer to data owned by an external system
+
+
+## 12. Relationship Gap Resolutions โ€” Q58 and Q60
+
+### 12.1 Relationship Role Validation (Q58)
+
+Relationship roles are semantic labels โ€” human-readable identifiers for the function a member plays in a relationship. By default, role validation is advisory. Resource Type Specifications may declare a closed set of permitted roles with enforced validation.
+
+```yaml
+resource_type_spec:
+  fully_qualified_name: Compute.VirtualMachine
+  permitted_relationship_roles:
+    - role: storage
+      relationship_types: [requires]
+      permitted_related_types: [Storage.Block, Storage.File]
+    - role: networking
+      relationship_types: [requires]
+      permitted_related_types: [Network.IPAddress, Network.Port]
+    - role: dns
+      relationship_types: [depends_on]
+      permitted_related_types: [DNS.Record]
+    - role: load_balancer
+      relationship_types: [depends_on]
+      permitted_related_types: [Network.LoadBalancer]
+  role_validation: advisory   # advisory | enforced
+  # advisory: unknown roles produce a warning in assembly provenance
+  # enforced: unknown roles are rejected at request time
+```
+
+**Community role catalog:** DCM ships a non-authoritative reference list of commonly-used roles. Organizations freely declare roles not in the catalog when role_validation is advisory.
+
+### 12.2 Maximum Relationship Graph Depth (Q60)
+
+Relationship graph depth is limited to a profile-governed maximum. Circular relationship detection is always enforced regardless of depth configuration.
+
+```yaml
+relationship_depth_policy:
+  max_depth: 15                  # configurable via Policy Group
+  on_max_exceeded: reject        # reject with clear error
+  cycle_detection: always        # non-configurable โ€” always enforced
+  # Depth = maximum traversal distance between any two entities
+  # NOT the count of relationships on one entity
+```
+
+**Profile-governed defaults:**
+
+| Profile | Max Depth | Rationale |
+|---------|----------|-----------|
+| `minimal` | 25 | Home lab โ€” free composition |
+| `dev` | 20 | Development โ€” generous |
+| `standard` | 15 | Production baseline |
+| `prod` | 15 | Production |
+| `fsi` | 10 | Tighter โ€” complex graphs harder to audit |
+| `sovereign` | 10 | Maximum control |
+
+**Note:** Relationship depth differs from dependency depth (ENT-008). Dependency depth counts the provisioning chain. Relationship depth counts the graph traversal distance between any two entities. A VM with 50 IP address relationships has depth 1, not 50.
+
+---
+
+## 13. System Policies โ€” Relationship Gaps
+
+| Policy | Rule |
+|--------|------|
+| `REL-020` | Relationship roles are semantic labels. Resource Type Specifications may declare permitted_relationship_roles with advisory or enforced validation. Advisory produces assembly warnings for unknown roles. Enforced rejects unknown roles at request time. DCM maintains a community role catalog as a non-authoritative reference. |
+| `REL-021` | Relationship graph depth is limited to a profile-governed maximum (default: 15 for standard/prod; 10 for fsi/sovereign). Circular relationship detection is always enforced regardless of depth configuration. Depth is measured as the maximum traversal distance between any two entities in the relationship graph. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
\ No newline at end of file
diff --git a/content/docs/data-model/10-information-providers.md b/content/docs/data-model/10-information-providers.md
new file mode 100644
index 0000000..aabd078
--- /dev/null
+++ b/content/docs/data-model/10-information-providers.md
@@ -0,0 +1,451 @@
+# DCM Data Model โ€” Information Providers
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: PROVIDER**
+>
+> The Provider abstraction โ€” Information Provider capability extension
+
+
+
+---
+
+## 1. Purpose
+
+An **Information Provider** is a registered DCM provider that serves as the authoritative source for a specific category of data that DCM needs to reference but does not own. It exposes external data to DCM through a standard interface, enabling DCM to look up, verify, and relate external records without caching or owning them.
+
+Information Providers are a first-class provider type in DCM alongside Service Providers and Meta Providers. They follow the same registration, health check, trust, and contract model as Service Providers โ€” adapted where applicable to the lookup-only nature of information retrieval.
+
+---
+
+## 2. Why Information Providers Exist
+
+DCM manages the lifecycle of resources it provisions. But resources exist in a broader organizational context โ€” they are owned by business units, attributed to cost centers, associated with product owners, governed by regulatory scopes. This contextual data lives in authoritative external systems (HR systems, finance systems, CMDBs, ITSM tools) that DCM does not and should not own.
+
+Without a formal model for referencing external data, organizations face two bad choices:
+- **Copy the data into DCM** โ€” creating duplication, staleness, and an ownership conflict with the authoritative system
+- **Ignore the data** โ€” losing business context, cost attribution, and compliance traceability
+
+Information Providers solve this by giving DCM a standard, stable, governed interface to external data without requiring ownership transfer.
+
+---
+
+## 3. Information Provider in the Provider Ecosystem
+
+DCM defines eleven provider types, all implementing the unified Provider base contract. This document focuses on the Information Provider type. For the complete provider type list see [A-provider-contract.md](A-provider-contract.md).
+
+| Provider Type | Purpose | Data Direction | DCM Owns Result? |
+|--------------|---------|---------------|-----------------|
+| **Service Provider** | Executes work, realizes resources | DCM โ†’ Provider โ†’ DCM | Yes โ€” DCM owns the realized entity |
+| **Information Provider** | Serves authoritative external data | DCM โ†’ Provider (lookup only) | No โ€” external system is authoritative |
+| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Child Providers โ†’ DCM | Yes โ€” DCM owns the composite result |
+
+---
+
+## 4. Information Provider Contract
+
+Information Providers follow the same provider contract model as Service Providers where applicable. The contract dimensions are:
+
+### 4.1 Registration Contract
+Same model as Service Providers. Information Providers register with DCM declaring their endpoint, the information types they implement, their lookup capabilities, and their extended schema.
+
+### 4.2 Health Check Contract
+Same model as Service Providers. Information Providers expose a `/health` endpoint. DCM polls it on the same configurable interval. `Ready`/`NotReady` state machine applies. An `NotReady` Information Provider is excluded from lookups โ€” relationships referencing it are flagged for on-demand verification fallback.
+
+### 4.3 Trust Contract
+Same model as Service Providers. Information Providers must be registered, validated, and certified before DCM will accept their data. The chain of trust applies to data returned by Information Providers โ€” provenance records the provider UUID for every field sourced from an Information Provider.
+
+### 4.4 Capacity Contract
+Adapted for lookup capacity rather than resource provisioning capacity. Information Providers declare and report their query capacity โ€” requests per second, rate limits, availability windows.
+
+```yaml
+capacity_registration:
+  provider_uuid: 
+  registration_timestamp: 
+  capacity_by_information_type:
+    - information_type_uuid: 
+      queries_per_second: 1000
+      rate_limit_window: 60s
+      availability: 99.9%
+```
+
+### 4.5 Lifecycle Event Contract
+Same model as Service Providers. Information Providers have a contractual obligation to notify DCM when records they have provided references for change status. DCM receives the notification and updates the external entity reference record accordingly.
+
+**Reportable event types for Information Providers:**
+
+| Event Type | Description | DCM Response |
+|------------|-------------|--------------|
+| `RECORD_DEACTIVATED` | A referenced record has been deactivated | Update reference status, Policy Engine evaluation |
+| `RECORD_MERGED` | Two records merged โ€” UUID may change | Update external_uuid in reference record |
+| `RECORD_SPLIT` | One record split into multiple | Policy Engine evaluation โ€” which new record applies? |
+| `UUID_CHANGED` | Record UUID changed in external system | Update external_uuid, re-verify all references |
+| `DATA_UPDATED` | Standard field values changed | Update last_verified, notify relationships |
+| `PROVIDER_DEGRADED` | Provider is degraded but operational | DCM flags affected references for on-demand verification |
+
+### 4.6 Naturalization/Denaturalization Contract
+Information Providers translate their native data format (HR system JSON, finance system XML, LDAP records, REST APIs) into the DCM unified data model format. The translation is the provider's responsibility โ€” DCM always receives data in DCM format.
+
+---
+
+## 5. Standard vs Extended Data
+
+### 5.1 Standard Data (DCM-defined)
+
+Fields that are part of the DCM-specified schema for an information type. DCM core uses these fields for lookups, relationship matching, policy evaluation, and display. They are portable across all implementations of that information type.
+
+DCM only relies on standard data for operational decisions. Extended data is carried in the payload but is not used for DCM core operations.
+
+### 5.2 Extended Data (organization-defined)
+
+Additional fields organizations add to enrich the standard schema for their specific needs. Declared in the provider's extended schema registration. DCM carries extended data in the payload for downstream consumers โ€” policy engines, cost analysis tools, reporting โ€” that know how to use them.
+
+```yaml
+# Standard + Extended data example โ€” Business.BusinessUnit
+business_unit_record:
+  # Standard fields โ€” DCM defined, used for lookups
+  uuid: "bu-uuid-001"
+  name: "Payments Platform"
+  code: "BU-PAY"
+  parent_uuid: "bu-uuid-root"
+  organization_uuid: "org-uuid-001"
+  status: active
+
+  # Extended fields โ€” organization defined
+  extensions:
+    profit_center_code: "PC-4421"
+    regulatory_jurisdiction: "EU"
+    trading_desk_id: "TD-007"
+    risk_tier: 1
+    internal_charge_code: "IC-PAY-001"
+```
+
+---
+
+## 6. Lookup Key Model
+
+DCM looks up external records using a stable primary key โ€” always the external UUID where available โ€” with a fallback chain for systems that don't support UUID-based lookup.
+
+### 6.1 External Entity Reference Structure
+
+```yaml
+external_entity_reference:
+  uuid: 
+  # DCM UUID is what gets stored in relationship declarations
+  # If the external system changes its UUID, only this record changes
+  # All relationships pointing to dcm-uuid remain valid
+
+  external_uuid: 
+  information_provider_uuid: 
+  information_type_uuid: 
+  information_type_name: Business.BusinessUnit
+
+  lookup_method:
+    primary_key: external_uuid
+    # Always attempted first
+    fallback_keys:
+      - field: code
+        value: "BU-PAY"
+      - field: name
+        value: "Payments Platform"
+    # Fallback keys tried in order if primary_key lookup fails
+
+  # Non-authoritative display cache โ€” for UI convenience only
+  display_name: "Payments Platform"
+  display_name_authoritative: false
+
+  verification:
+    last_verified: 
+    last_verified_method: 
+    verification_status: 
+    next_scheduled_verification: 
+
+  status:
+    state: 
+
+  provenance:
+    
+```
+
+### 6.2 Why DCM UUID Wraps External UUID
+
+The DCM-generated UUID is the stable internal anchor. This means:
+- All relationship declarations inside DCM reference the DCM UUID
+- If the external system changes its UUID (migration, system upgrade), only the `external_entity_reference` record needs updating
+- All relationships pointing to the DCM UUID remain valid without modification
+- The provenance chain tracks the change via the `UUID_CHANGED` lifecycle event
+
+---
+
+## 7. Three-Mode Verification Model
+
+DCM uses a trust-but-verify approach to external entity references. The external system is trusted as authoritative for the data โ€” DCM does not validate content. But DCM verifies that references remain valid โ€” the UUID still exists and the record is still active.
+
+### 7.1 Mode 1 โ€” Scheduled Verification (DCM-initiated)
+
+DCM calls the Information Provider's `/verify/{uuid}` endpoint on a configurable schedule for all registered external entity references. Default frequency: configurable โ€” suggested minimum twice daily. Updates `last_verified` and `verification_status`.
+
+### 7.2 Mode 2 โ€” Provider Push (Information Provider obligation)
+
+The Information Provider notifies DCM when a referenced record changes status. This is a contractual obligation โ€” same model as Service Provider lifecycle events. DCM receives the notification, updates the external entity reference, and the Policy Engine evaluates the appropriate response.
+
+### 7.3 Mode 3 โ€” On-Demand Verification (fallback)
+
+When a relationship involving an external entity reference is accessed during request processing, policy evaluation, or drift detection, DCM can verify the reference in real time before relying on it. Used when:
+- `last_verified` is beyond the acceptable staleness window
+- The operation requires high confidence
+- Scheduled verification returned `stale` or `unverifiable`
+
+### 7.4 Verification Fallback Chain
+
+```
+External entity reference accessed
+  โ”‚
+  โ–ผ
+Is verification_status: verified AND last_verified within window?
+  โ”‚ Yes โ†’ proceed with reference
+  โ”‚ No โ†“
+  โ–ผ
+Mode 3 โ€” on-demand verify via Information Provider /verify/{uuid}
+  โ”‚ Success โ†’ update last_verified, verification_status: verified, proceed
+  โ”‚ Failure โ†“
+  โ–ผ
+Policy Engine evaluates:
+  Options (configurable per information type and organizational policy):
+    block_request   โ€” reject request until reference is verified
+    warn_and_proceed โ€” proceed with warning recorded in provenance
+    use_display_only โ€” use display_name only, no operational reliance
+    escalate        โ€” notify appropriate personas for human resolution
+```
+
+---
+
+## 8. Information Type Registry
+
+Information types live in the same DCM Resource Type Registry as Resource Types, distinguished by category prefix. Same versioning, same deprecation model, same governance.
+
+### 8.1 Standard Information Type Categories
+
+| Category | Description | Examples |
+|----------|-------------|---------|
+| `Business.*` | Business organizational data | BusinessUnit, CostCenter, ProductOwner |
+| `Identity.*` | Identity and access data | Person, ServiceAccount, Group |
+| `Compliance.*` | Regulatory and compliance data | RegulatoryScope, AuditFramework |
+| `Operations.*` | Operational reference data | Runbook, SLA, SupportContract |
+
+### 8.2 DCM Default Information Types
+
+```yaml
+# Business.BusinessUnit
+information_type:
+  uuid: 
+  name: Business.BusinessUnit
+  category: Business
+  version: 1.0.0
+  standard_fields:
+    - name: uuid
+      type: string
+      required: true
+      lookup_supported: true
+    - name: name
+      type: string
+      required: true
+      lookup_supported: true
+    - name: code
+      type: string
+      required: false
+      lookup_supported: true
+    - name: parent_uuid
+      type: string
+      required: false
+      lookup_supported: false
+    - name: organization_uuid
+      type: string
+      required: true
+      lookup_supported: false
+    - name: status
+      type: enum
+      values: [active, inactive]
+      required: true
+      lookup_supported: false
+  extended_fields_permitted: true
+  status: active
+
+# Business.CostCenter
+information_type:
+  uuid: 
+  name: Business.CostCenter
+  standard_fields:
+    - name: uuid
+      lookup_supported: true
+    - name: name
+      lookup_supported: true
+    - name: code
+      lookup_supported: true
+    - name: owner_uuid
+      lookup_supported: false
+    - name: budget_period
+      lookup_supported: false
+    - name: status
+      lookup_supported: false
+
+# Identity.Person
+information_type:
+  uuid: 
+  name: Identity.Person
+  standard_fields:
+    - name: uuid
+      lookup_supported: true
+    - name: name
+      lookup_supported: true
+    - name: email
+      lookup_supported: true
+    - name: employee_id
+      lookup_supported: true
+    - name: department_uuid
+      lookup_supported: false
+    - name: status
+      lookup_supported: false
+```
+
+### 8.3 Custom Information Types
+
+Organizations register custom information types following the same model:
+
+```yaml
+custom_information_type:
+  uuid: 
+  name: 
+  # Must use a non-reserved category prefix or register a new one
+  category: 
+  version: 
+  registered_by_tenant_uuid: 
+  standard_fields:
+    
+  extended_fields_permitted: 
+  status: 
+```
+
+---
+
+## 9. Information Provider Registration
+
+```yaml
+information_provider_registration:
+  uuid: 
+  name: 
+  display_name: 
+  version: 
+
+  implements:
+    - information_type_uuid: 
+      information_type_name: Business.BusinessUnit
+      information_type_version: 
+      lookup_methods_supported: [primary_key, code]
+      extended_fields_supported: true
+      extended_schema:
+        
+
+  endpoint: 
+
+  capacity:
+    queries_per_second: 
+    rate_limit_window: 
+    update_frequency: 
+
+  sovereignty_constraints:
+    
+
+  trust_declaration:
+    
+
+  health_check:
+    endpoint: /health
+    poll_interval_seconds: 
+
+  status:
+    state: 
+    deprecation_date: 
+    sunset_date: 
+    replacement_uuid: 
+    deprecation_reason: 
+    migration_guidance: 
+
+  provenance:
+    
+```
+
+---
+
+## 10. Mandatory Information Provider API Endpoints
+
+All Information Providers must implement these endpoints as part of their provider contract:
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `GET` | `/health` | Provider health check โ€” same as Service Provider |
+| `GET` | `/lookup/{uuid}` | Returns standard + extended data for a record by external UUID |
+| `GET` | `/verify/{uuid}` | Lightweight โ€” confirms UUID exists and is active |
+| `POST` | `/search` | Finds records matching standard field criteria (fallback lookup) |
+| `POST` | `/notify` | DCM calls this to acknowledge receipt of provider push events |
+
+---
+
+## 11. Internally Owned Business Data
+
+When an organization decides to manage business context data in DCM rather than reference an external system, they define it as a DCM Resource Type in the `Business.*` or custom category. Internally owned business data follows the **standard resource entity model** exactly:
+
+- Has a UUID
+- Has a Resource Type (`Business.BusinessUnit`, `Business.CostCenter`, etc.)
+- Has provenance
+- Has versioning
+- Has relationships to other entities
+- Follows the universal lifecycle (active โ†’ deprecated โ†’ retired)
+- Can be grouped under Tenants and Resource Groups
+
+The relationship model is identical whether the related entity is internal or external โ€” the `related_entity_type` field (`internal` vs `external`) is the only difference from the consuming entity's perspective.
+
+This means an organization can start with an external Information Provider reference and migrate to internally owned business data later โ€” relationships remain structurally the same, only the `related_entity_type` changes.
+
+---
+
+## 12. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | How are conflicting provider push events handled โ€” two Information Providers claim authority for the same record? | Data integrity | โœ… Resolved โ€” authority_level (primary/secondary/advisory) + authority_scope; conflict_resolution strategies; ingestion-time conflict detection; conflict records; see doc 21 (INF-001) |
+| 2 | Should Information Providers support write-back โ€” DCM updating external records via the provider? | Scope expansion | โœ… Resolved โ€” optional declared capability; policy-triggered write-back; audit records produced; credentials via Credential Provider; see doc 21 (INF-002) |
+| 3 | How is the extended schema versioned โ€” if a provider adds or removes extended fields, how are existing references affected? | Versioning | โœ… Resolved โ€” semver semantics on extended schema; field removal/type change = major; new optional field = minor; migration plan required for major bumps; see doc 21 (INF-003) |
+| 4 | Should DCM maintain a registry of well-known Information Providers (HR systems, finance systems) to simplify onboarding? | Adoption | โœ… Resolved โ€” three-tier Information Provider Registry (Core/Community/Organization); same governance model as Resource Type Registry; separate registries; see doc 21 (INF-004) |
+| 5 | How does the verification model interact with air-gapped environments where Information Providers may be unreachable? | Sovereignty | โœ… Resolved โ€” three air-gap modes: pre-verified signed bundle, internal mTLS, periodic online re-verification with cached tokens; profile-governed cache expiry (prod/fsi/sovereign=suspend on expiry); see doc 21 (INF-005) |
+
+---
+
+## 13. Related Concepts
+
+- **External Entity Reference** โ€” the stable pointer record DCM uses to reference external data
+- **Entity Relationships** โ€” the universal relationship model that uses Information Provider references
+- **Service Provider** โ€” counterpart provider type for resource provisioning
+- **Resource Type Registry** โ€” the unified registry containing both Resource Types and Information Types
+- **Trust Contract** โ€” the provider trust model shared across all provider types
+- **Naturalization/Denaturalization** โ€” translation between external native format and DCM unified format
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/11-storage-providers.md b/content/docs/data-model/11-storage-providers.md
new file mode 100644
index 0000000..756e511
--- /dev/null
+++ b/content/docs/data-model/11-storage-providers.md
@@ -0,0 +1,796 @@
+# DCM Data Model โ€” Storage Providers
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Four States](02-four-states.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Information Providers](10-information-providers.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: PROVIDER**
+>
+> The Provider abstraction โ€” Storage Provider capability extension
+
+
+
+---
+
+> **Operational guidance:** GitOps store-at-scale and migration are covered in [Operational Reference](41-operational-reference.md) Sections 1 and 2.
+
+## 1. Purpose
+
+A **Storage Provider** is the fourth formal DCM provider type. It is the interface through which DCM persists, retrieves, and streams all state data. DCM defines the contract โ€” the characteristics, capabilities, and obligations each store must satisfy. The implementation technology is a deployment choice made by implementors.
+
+This is consistent with DCM's governing framework philosophy: DCM does not prescribe technology. It defines what is required and what is guaranteed. An organization using GitHub and Kafka satisfies the same contracts as one using Gitea and EventStoreDB.
+
+---
+
+## 2. Storage Provider Sub-Types
+
+Storage Providers are one of eleven DCM provider types (see [Unified Provider Contract](A-provider-contract.md)). Within the Storage Provider type, four storage sub-types are defined, each optimized for different access patterns and consistency requirements:
+
+| Provider Type | Purpose | Data Direction | DCM Owns Result? |
+|--------------|---------|---------------|-----------------|
+| **Service Provider** | Realizes resources | DCM โ†’ Provider โ†’ DCM | Yes |
+| **Information Provider** | Serves external authoritative data | DCM โ†’ Provider (lookup) | No |
+| **Meta Provider** | Composes multiple providers | DCM โ†’ Meta โ†’ Children โ†’ DCM | Yes |
+| **Storage Provider** | Persists and streams DCM state | DCM โ†” Provider | Yes โ€” DCM is authoritative |
+
+---
+
+## 3. Storage Provider Contract โ€” Base Requirements
+
+All Storage Providers share these base contract requirements regardless of store type:
+
+### 3.1 Registration
+Same model as Service and Information Providers. Storage Providers register with DCM declaring their endpoint, store type, capabilities, and sovereignty characteristics.
+
+```yaml
+storage_provider_registration:
+  uuid: 
+  name: 
+  display_name: 
+  store_type: 
+  version: 
+  endpoint: 
+  capabilities: 
+  sovereignty_constraints: 
+  trust_declaration: 
+  status: 
+```
+
+### 3.2 Health Check
+Same model as all providers. `GET /health` endpoint, DCM polls on configurable interval.
+
+### 3.3 Trust
+Same model as all providers. DCM validates Storage Provider identity before writing or reading state data. A compromised Storage Provider is treated as a sovereignty incident.
+
+### 3.4 Provenance Emission Obligation
+Every Storage Provider that holds state data has a contractual obligation to emit provenance events to the Audit component when state is written or modified. This is not optional โ€” it is part of the Storage Provider contract.
+
+```yaml
+# Provenance emission event โ€” sent to Audit component on every write
+provenance_emission:
+  store_type: 
+  operation: 
+  entity_uuid: 
+  record_uuid: 
+  actor_uuid: 
+  timestamp: 
+  payload_hash: 
+  store_reference: 
+```
+
+### 3.5 Consistency Guarantee Declaration
+Each Storage Provider must declare its consistency model in registration. DCM components read this declaration and adapt their behavior accordingly.
+
+```yaml
+consistency_declaration:
+  consistency_model: 
+  replication_factor: 
+  durability_guarantee: 
+  max_data_loss_window: 
+```
+
+---
+
+## 4. GitOps Store Contract
+
+Used for: Intent State, Requested State, Layer Store, Policy Store
+
+### 4.1 Required Capabilities
+
+```yaml
+gitops_capabilities:
+  branching: true              # Branch-per-request support
+  pull_request: true           # PR creation, review, merge
+  immutable_history: true      # Commits are permanent
+  ci_cd_hooks: true            # Webhook triggers on push/merge
+  search_index_integration: true # Search Index companion required
+  access_control: true         # Per-branch, per-path access control
+  signed_commits: optional     # Recommended for audit integrity
+```
+
+### 4.2 Required API Operations
+
+| Operation | Description | Used By |
+|-----------|-------------|---------|
+| `create_branch` | Create a new branch from main | Intent State creation |
+| `commit_file` | Commit a file to a branch | Intent and Requested State write |
+| `create_pr` | Open a Pull Request for review | Intent State review workflow |
+| `merge_pr` | Merge an approved PR to main | Intent State approval |
+| `get_file` | Retrieve a file by path or commit | State retrieval |
+| `get_history` | Retrieve commit history for a path | Audit and rehydration |
+| `trigger_ci` | Trigger CI pipeline on branch | Policy pre-validation |
+| `trigger_cd` | Trigger CD pipeline on merge | Requested State assembly and dispatch |
+| `post_comment` | Post a comment on a PR | CI pipeline result reporting |
+
+### 4.3 File Structure Convention
+
+```
+{store_root}/
+  tenants/
+    {tenant_uuid}/
+      {entity_uuid}/
+        intent.yaml          # Intent State record
+        # OR
+        requested-state.yaml # Requested State record
+```
+
+### 4.4 Search Index Companion
+
+Every GitOps store deployment requires a companion Search Index. The Search Index is a separate Storage Provider that maintains a queryable projection of the GitOps store. See Section 6.
+
+---
+
+## 5. Event Stream Store Contract
+
+Used for: Realized State, Discovered State
+
+### 5.1 Required Capabilities
+
+```yaml
+event_stream_capabilities:
+  append_only: true            # Events are never modified or deleted
+  entity_keyed_streams: true   # Each entity has its own event stream
+  stream_replay: true          # Streams can be replayed from any offset
+  entity_uuid_lookup: true     # O(1) lookup of stream by entity UUID
+  at_least_once_delivery: true # Events are never silently lost
+  configurable_retention: true # Retention period configurable per stream type
+  distributed_replication: true # Data replicated across nodes
+  high_throughput_write: true  # Optimized for machine-generated writes
+```
+
+### 5.2 Stream Naming Convention
+
+```
+dcm.realized.{entity_uuid}    # Realized State stream per entity
+dcm.discovered.{entity_uuid}  # Discovered State stream per entity
+dcm.audit.{tenant_uuid}       # Audit event stream per tenant
+dcm.system                    # DCM system-level events
+```
+
+### 5.3 Required API Operations
+
+| Operation | Description | Used By |
+|-----------|-------------|---------|
+| `append_event` | Append an event to an entity stream | Provider callbacks, discovery |
+| `read_stream` | Read events from an entity stream from offset | State retrieval, drift detection |
+| `read_latest` | Read the most recent event in a stream | Current state queries |
+| `replay_stream` | Replay all events from beginning | Audit, historical reconstruction |
+| `list_streams` | List streams matching a pattern | Tenant-level queries |
+| `get_stream_metadata` | Get stream statistics and metadata | Health monitoring |
+
+### 5.4 Event Envelope
+
+Every event written to the Event Stream Store uses this envelope:
+
+```yaml
+event_envelope:
+  event_uuid: 
+  stream_id: 
+  entity_uuid: 
+  tenant_uuid: 
+  event_type: 
+  sequence_number: 
+  timestamp: 
+  schema_version: 
+  payload_hash: 
+  payload: 
+  provenance:
+    written_by_uuid: 
+    triggered_by_request_uuid: 
+    triggered_by_actor_uuid: 
+```
+
+### 5.5 Retention Model
+
+| Stream Type | Default Retention | Rationale |
+|-------------|------------------|-----------|
+| Realized State | Permanent | Complete audit trail required |
+| Discovered State | Configurable window | Operational use only โ€” older snapshots archived |
+| Audit | Regulatory period (configurable โ€” minimum 7 years for FSI) | Compliance requirement |
+
+---
+
+
+---
+
+## 5a. Write-once Snapshot Store Contract (Realized Store)
+
+The Realized Store is a **write-once snapshot store** โ€” distinct from the Event Stream Store used for Discovered State. It holds complete entity state snapshots where every record is traceable to an authorized DCM request.
+
+### 5a.1 Store Characteristics
+
+The Realized Store occupies a middle ground between the GitOps store (structured, human-navigable, PR-mediated) and the Event Stream store (high-frequency, field-level events, ephemeral). The Realized Store is:
+
+- **Write-once** โ€” records are immutable after creation; a new record is created for each authorized state change
+- **Snapshot-based** โ€” each record is a complete entity state, not a field-level delta
+- **Request-traceable** โ€” every record has a non-nullable `corresponding_requested_state_uuid`
+- **Moderate frequency** โ€” written only on authorized changes (initial realization, consumer updates, approved provider updates), not on every event
+- **Long-lived** โ€” records persist for the full entity lifetime plus audit retention period
+
+### 5a.2 Write Authorization Model
+
+The Realized Store has a strictly controlled set of write sources. No component may write to the Realized Store without a corresponding Requested State record:
+
+```yaml
+realized_store_write_authorization:
+  allowed_sources:
+    - source_type: initial_realization
+      trigger: provider_confirms_realization
+      required: corresponding_requested_state_uuid (type: initial_realization)
+
+    - source_type: consumer_update
+      trigger: provider_confirms_targeted_delta
+      required: corresponding_requested_state_uuid (type: consumer_update)
+
+    - source_type: provider_update
+      trigger: dcm_approves_provider_update_notification
+      required: corresponding_requested_state_uuid (type: provider_update)
+
+  explicitly_forbidden:
+    - drift_detection          # drift only reads, never writes
+    - discovery_cycles         # discovery writes to Discovered Store only
+    - unsanctioned_changes     # unsanctioned changes remain drift events
+    - direct_admin_writes      # no bypassing the request pipeline
+```
+
+**Enforcement:** The write authorization model is enforced at the Realized Store API level โ€” not by convention. A write without a valid `corresponding_requested_state_uuid` is rejected with `401 Unauthorized`.
+
+### 5a.3 Required Capabilities
+
+```yaml
+write_once_snapshot_store:
+  write_once_enforcement: true       # writes without corresponding_requested_state_uuid rejected
+  entity_uuid_keyed: true            # O(1) lookup by entity UUID
+  snapshot_retention: per_entity     # all snapshots for an entity retained per retention policy
+  point_in_time_query: true          # query state as of any timestamp
+  supersession_chain: true           # each record knows predecessor and successor
+  hash_chain_integrity: true         # each record hashes previous โ€” tamper evident
+  concurrent_write_handling: optimistic_lock  # retry on conflict, no silent overwrite
+```
+
+### 5a.4 Required API Operations
+
+| Operation | Description |
+|-----------|-------------|
+| `write_snapshot(entity_uuid, payload, requested_state_uuid)` | Write new snapshot; validate non-null requested_state_uuid; return snapshot UUID |
+| `get_current(entity_uuid)` | Return the most recent snapshot for entity |
+| `get_at_timestamp(entity_uuid, timestamp)` | Return snapshot valid at given timestamp |
+| `get_by_uuid(realized_state_uuid)` | Return specific snapshot |
+| `list_history(entity_uuid)` | Return supersession chain for entity |
+| `verify_chain(entity_uuid)` | Verify hash chain integrity for entity's snapshots |
+
+### 5a.5 Retention Policy
+
+Realized State snapshots are retained for the full entity lifecycle. After an entity is DECOMMISSIONED, snapshots are retained per the audit retention policy โ€” the same policy that governs Audit Store records. The entity's final Realized State snapshot is preserved even after all preceding snapshots are archived.
+
+### 5a.6 Relationship to the Audit Store
+
+The Realized Store and Audit Store are complementary โ€” not redundant:
+
+| Aspect | Realized Store | Audit Store |
+|--------|---------------|-------------|
+| Content | Complete entity state snapshots | Atomic action records (who did what when) |
+| Query pattern | "What was the state of X at time T?" | "Who changed X and why?" |
+| Write frequency | Per authorized change | Per every DCM action |
+| Hash chain | Per entity | Per instance |
+| Rehydration source | Yes | No |
+
+The Audit Store records the action. The Realized Store records the result. Both are required for complete auditability.
+
+### 5a.7 Typical Implementations
+
+| Scale | Implementation | Notes |
+|-------|---------------|-------|
+| Minimal / dev | SQLite or PostgreSQL single-instance | Simple; acceptable for evaluation |
+| Standard | PostgreSQL with write-once constraints | Reliable; familiar operational model |
+| Production | CockroachDB or PostgreSQL HA | Geo-distributed; strong consistency |
+| FSI / Sovereign | PostgreSQL with HSM-backed encryption | Encryption at rest required |
+
+
+## 6. Search Index Contract
+
+Used for: Queryable projection of GitOps stores
+
+### 6.1 Role and Authority
+
+The Search Index is explicitly **non-authoritative**. If the Search Index and the GitOps store disagree on any record, the GitOps store wins unconditionally. The Search Index is a performance layer โ€” it is never the source of truth.
+
+The Search Index can be rebuilt from scratch from Git history at any time. This replaceability is a contract requirement โ€” implementors must support full index rebuild from the GitOps store.
+
+### 6.2 Required Indexed Fields
+
+At minimum the Search Index must index these fields from Intent and Requested State records:
+
+```yaml
+indexed_fields:
+  - entity_uuid          # Universal linking key
+  - tenant_uuid          # Tenant ownership
+  - resource_type_name   # e.g., Compute.VirtualMachine
+  - resource_type_uuid   # Registry UUID
+  - lifecycle_state      # Current state
+  - provider_uuid        # Selected provider
+  - created_timestamp    # When the record was created
+  - updated_timestamp    # When the record was last updated
+  - cost_center          # Business context (if declared)
+  - business_unit_uuid   # Business context (if declared)
+  - git_path             # Path in GitOps store โ€” used to retrieve full record
+  - git_commit_hash      # Specific commit โ€” used for point-in-time retrieval
+```
+
+### 6.3 Required Query Operations
+
+| Operation | Example | Used By |
+|-----------|---------|---------|
+| `find_by_entity_uuid` | Find all records for entity xyz | Rehydration, audit |
+| `find_by_tenant` | All entities for Tenant A | Tenant management |
+| `find_by_resource_type` | All VMs across all tenants | Catalog reporting |
+| `find_by_lifecycle_state` | All PENDING entities | Operational monitoring |
+| `find_by_field` | All entities with cost_center=BU-PAY | FinOps reporting |
+| `full_text_search` | Search across all indexed text fields | Discovery, debugging |
+| `count_by_field` | Count entities grouped by resource_type | Analytics |
+
+---
+
+## 7. DCM-Internal Caches
+
+DCM may maintain internal performance caches between components and stores. These are not Storage Providers โ€” they are internal implementation details that do not require external registration or trust.
+
+### 7.1 Cache Characteristics
+
+- **Non-authoritative** โ€” explicitly marked. Cache hits are not treated as ground truth.
+- **Cache-aside pattern** โ€” DCM checks cache first; on miss, reads from authoritative store and populates cache
+- **Invalidation on write** โ€” any write to an authoritative store invalidates the corresponding cache entry
+- **Bounded staleness** โ€” maximum staleness window configured per cache; entries older than the window are treated as misses
+- **Rebuildable** โ€” any cache can be cleared and rebuilt from its authoritative store
+
+### 7.2 Candidate Cache Locations
+
+| Cache | Authoritative Source | Purpose |
+|-------|---------------------|---------|
+| Layer Cache | Layer Store (Git) | Avoid repeated Git reads for frequently used layers |
+| Policy Cache | Policy Store (Git) | OPA policy bundles cached in Policy Engine memory |
+| Catalog Cache | Catalog Store (Git) | Service catalog items cached for presentation |
+| Provider Registry Cache | Provider Registry | Registered provider list cached for routing |
+| Search Index | GitOps stores | Queryable projection (also functions as a cache) |
+
+---
+
+## 8. Storage Provider vs Service Provider โ€” Key Differences
+
+| Dimension | Service Provider | Storage Provider |
+|-----------|-----------------|-----------------|
+| **Purpose** | Realizes resources | Persists DCM state |
+| **Data direction** | DCM sends, provider executes | DCM reads and writes |
+| **Naturalization** | Required โ€” DCM format โ†’ native | Not required โ€” DCM format throughout |
+| **Denaturalization** | Required โ€” native โ†’ DCM format | Not required |
+| **Provenance emission** | Required (realized state) | Required (all writes) |
+| **Capacity model** | Resource capacity | Storage capacity and throughput |
+| **Health model** | Is provider healthy? | Is store reachable and consistent? |
+
+---
+
+## 9. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should Storage Providers support multi-region replication as a declared capability? | Sovereignty | โœ… Resolved โ€” declared capability in registration; Profile determines minimum (STO-001) |
+| 2 | How are Storage Provider failures handled โ€” failover, queuing, or rejection? | Reliability | โœ… Resolved โ€” per store type: Commit Log aborts; GitOps queues; Event Stream queues; Audit accumulates; Search degrades (STO-002) |
+| 3 | Should the Search Index be a separate registered Storage Provider or bundled with the GitOps store? | Architecture | โœ… Resolved โ€” separate sub-type; non-authoritative; rebuildable (STO-003) |
+| 4 | How does the Storage Provider model interact with air-gapped environments? | Sovereignty | โœ… Resolved โ€” sovereignty_declaration on all providers; air_gap_capable flag; offline registry; signed bundles (SOV-001) |
+
+---
+
+
+## 10. Storage Architecture โ€” Q79 through Q83
+
+### 10.1 Git Repository Structure โ€” Intent and Requested Stores (Q79)
+
+GitOps stores use a deterministic handle-based directory structure. Every artifact lives at a path derivable from its identity โ€” human-navigable, diff-readable, and independently verifiable without DCM tooling.
+
+```
+dcm-intent/                              โ† Intent Store repository
+  tenants/
+    {tenant-uuid}/
+      requests/
+        {request-uuid}/
+          intent.yaml                    โ† Consumer's original submission
+          metadata.yaml                  โ† Timestamp, actor, ingress block
+
+dcm-requested/                           โ† Requested Store repository
+  tenants/
+    {tenant-uuid}/
+      requests/
+        {request-uuid}/
+          requested-payload.yaml         โ† Fully assembled, policy-processed payload
+          assembly-provenance.yaml       โ† Layer chain, policy evaluation results
+          placement.yaml                 โ† Selected provider, placement constraints
+          dependencies/
+            {dependency-uuid}/
+              requested-payload.yaml     โ† Each dependency's assembled payload
+
+dcm-layers/                              โ† Layer Store repository
+  {domain}/
+    {concern-or-type}/
+      {name}/
+        v{Major}.{Minor}.{Revision}.yaml
+
+dcm-policies/                            โ† Policy Store repository
+  {domain}/
+    {concern-or-type}/
+      {name}/
+        v{Major}.{Minor}.{Revision}.yaml
+```
+
+**Tenant isolation:** Each Tenant's requests live under their `{tenant-uuid}` directory. Access control is enforced at the DCM API layer โ€” the Git repository uses DCM's service account for all reads/writes. Individual Tenants never have direct Git access.
+
+**Branching model:** `main` is the authoritative branch. No feature branches for operational stores โ€” all writes go directly to `main` via the DCM service account. The Git history IS the audit trail for the GitOps stores.
+
+**Repository count:**
+- `minimal` and `dev` profiles: monorepo (all four stores in one repository โ€” simpler, single backup target)
+- `standard` and above: separate repositories per store type (cleaner governance boundaries, independent scaling, independent access control)
+
+**System policy (STO-005):** GitOps stores use a handle-based directory structure. `main` is authoritative. Minimal/dev may use monorepo; standard+ should use separate repos.
+
+### 10.2 Multi-Region Replication Capability Declaration (Q80)
+
+Storage Providers declare their replication capabilities in their registration. The active Profile determines the minimum replication requirement.
+
+```yaml
+storage_provider_registration:
+  capabilities:
+    replication:
+      multi_region: true
+      supported_regions: [eu-west-1, eu-west-2, us-east-1]
+      replication_modes: [active_active, active_passive]
+      consistency_model: 
+    redundancy:
+      replicas: 3
+      write_quorum: 2
+      zone_spread: required
+    backup:
+      automated: true
+      schedule: "0 */6 * * *"
+      retention: P30D
+      cross_region: true
+```
+
+**Profile minimum replication requirements:**
+
+| Profile | Multi-Region | Min Replicas | Consistency |
+|---------|-------------|-------------|------------|
+| `minimal` | No | 1 | Any |
+| `dev` | No | 1 | Any |
+| `standard` | No | 3 | Strong or bounded |
+| `prod` | Yes | 3 | Strong |
+| `fsi` | Yes | 5 | Strong |
+| `sovereign` | Yes (within boundary) | 5 | Strong |
+
+For `sovereign` profile: `multi_region: true` is required but all regions must be within the declared sovereignty boundary. Cross-boundary replication is blocked by sovereign policy groups.
+
+**System policy (STO-001):** Storage Providers must declare replication capabilities. Active Profile determines minimum requirements. Providers not meeting Profile minimum cannot be activated for that Profile's stores.
+
+### 10.3 Storage Provider Failure Handling (Q81)
+
+Failure behavior is declared per store type and governed by the active Profile.
+
+```yaml
+storage_failure_policy:
+  commit_log:
+    on_quorum_unavailable: abort_operation      # hard โ€” no silent changes
+    on_minority_failure: continue               # transparent via Raft
+  gitops_store:
+    on_unavailable: queue_writes                # local buffer; serve reads from cache
+    max_queue_size: 10000
+    max_queue_age: PT1H                         # reject if queued > 1 hour
+    on_queue_exhausted: reject                  # explicit rejection โ€” not silent drop
+  event_stream:
+    on_unavailable: queue_locally               # producer-side queuing
+    consumer_behavior: resume_from_offset       # no data loss on recovery
+  audit_store:
+    on_unavailable: accumulate_in_commit_log    # two-stage audit handles this
+    max_accumulation_age: P7D                   # alert if pending > 7 days
+  search_index:
+    on_unavailable: serve_degraded              # warn + direct to authoritative
+    on_recovery: rebuild_from_authoritative     # full index rebuild
+```
+
+**By store type:**
+- **Commit Log:** Quorum unavailable โ†’ operation aborted. Minority failure โ†’ continues via Raft reelection.
+- **GitOps Stores:** Unavailable โ†’ writes queue locally; reads serve from cache. Queue exhausted โ†’ explicit rejection.
+- **Event Stream (Discovered Store):** Producer queues locally. Consumer resumes from last committed offset on recovery. No data loss.
+- **Write-once Snapshot Store (Realized Store):** Writes queue locally with the same WAL pattern as the Audit Store. Write retry on recovery. Rejected writes (missing requested_state_uuid) are never retried โ€” they are logged as security events.
+- **Audit Store:** Two-stage model โ€” Commit Log accumulates `pending_forward` entries. Operations not blocked.
+- **Search Index:** Non-authoritative. Unavailable โ†’ degraded response + reference to authoritative store. Recovery โ†’ full index rebuild.
+
+`fsi` and `sovereign` profiles tighten GitOps failure policy: write buffer age limit drops to PT15M; queue exhaustion triggers platform alert and operator notification.
+
+**System policy (STO-002):** Storage Provider failure behavior declared per store type and governed by Profile. GitOps unavailability queues writes โ€” does not silently drop. Commit Log quorum loss aborts operation. Audit Store unavailability accumulates in Commit Log. Search Index unavailability degrades queries without impacting writes.
+
+### 10.4 Search Index โ€” Separate Storage Provider Sub-Type (Q82)
+
+The Search Index is a **separate Storage Provider sub-type** distinct from GitOps stores. Treating them as the same type would obscure the critical distinction: GitOps stores are authoritative and cannot be lost; the Search Index is non-authoritative and rebuildable.
+
+```yaml
+search_index_provider:
+  provider_type: search_index              # distinct sub-type of storage_provider
+  implementation: elasticsearch           # or: opensearch
+  authoritative: false                    # explicitly non-authoritative
+  rebuildable_from: [gitops_store, event_stream]
+  rebuild_trigger: 
+  rebuild_schedule: "0 3 * * 0"          # weekly full rebuild
+  consistency_lag: PT5M                   # acceptable lag behind authoritative stores
+```
+
+**Indexing model:** DCM control plane emits index update events on every authoritative store write. Search Index Storage Provider consumes these events and updates incrementally. On failure, rebuilds from authoritative stores.
+
+**API freshness:** Queries may specify `freshness: authoritative` to bypass the index and query the GitOps store directly for guaranteed-current results.
+
+**System policy (STO-003):** Search Index is a separate Storage Provider sub-type โ€” non-authoritative, rebuildable, distinct backend from GitOps. API queries may specify `freshness: authoritative` to bypass the index.
+
+### 10.5 Audit Store โ€” Specialized Storage Provider Sub-Type (Q83)
+
+The Audit Store is a **specialized Storage Provider sub-type** with compliance properties no general Event Stream Store satisfies:
+
+- **Append-only with immutability enforcement** โ€” records cannot be modified or deleted while retention obligations apply
+- **Hash chain integrity** โ€” maintains and verifies the per-entity hash chain (AUD-006)
+- **Reference-based retention tracking** โ€” tracks entity lifecycle states for retention eligibility (AUD-003)
+- **Compliance-grade query** โ€” multi-dimensional queries by entity_uuid, actor_uuid, action, timestamp range, tenant_uuid
+
+The Event Stream (Kafka) is the **delivery channel** to the Audit Store โ€” transient, cleared after Audit Store confirms receipt. The Audit Store is the **compliance destination** โ€” permanent for the duration of retention obligations.
+
+```yaml
+audit_store_provider:
+  provider_type: audit_store             # specialized sub-type of storage_provider
+  implementation: elasticsearch          # or: opensearch
+  authoritative: true
+  append_only_enforced: true
+  hash_chain_verification: true
+  retention_tracking: reference_based
+  query_capabilities:
+    - entity_uuid
+    - actor_uuid
+    - action
+    - timestamp_range
+    - tenant_uuid
+    - request_uuid
+    - retention_status
+    - ingress_surface
+    - auth_provider_type
+  compliance_certifications: [SOC2, ISO-27001, PCI-DSS]
+```
+
+```
+Commit Log โ†’ Audit Forward Service โ†’ Event Stream โ†’ Audit Store
+(Stage 1)    (enrichment + hash)     (delivery)     (compliance storage)
+```
+
+**System policy (STO-004):** Audit Store is a specialized Storage Provider sub-type โ€” append-only, hash chain integrity, reference-based retention, compliance-grade queries. Event Stream is the delivery channel only.
+
+---
+
+## 11. Provider Sovereignty Declaration
+
+### 11.1 Obligation
+
+Every provider registration โ€” Service Provider, Information Provider, Message Bus Provider, Policy Provider, and Auth Provider โ€” must include a `sovereignty_declaration` block. This is a contractual obligation, not optional metadata. DCM uses sovereignty declarations to make placement decisions, enforce Tenant sovereignty requirements, and detect drift between declared and actual posture.
+
+### 11.2 Sovereignty Declaration Structure
+
+```yaml
+sovereignty_declaration:
+  # JURISDICTIONAL DATA
+  operating_jurisdictions:
+    - country: DE
+      legal_system: eu_gdpr
+      data_center_location: Frankfurt
+    - country: FR
+      legal_system: eu_gdpr
+      data_center_location: Paris
+  # Does data ever transit through other jurisdictions?
+  data_transit_jurisdictions: []          # empty = data stays in declared jurisdictions
+  data_residency_guarantee: true          # data never leaves declared jurisdictions
+  
+  # LEGAL FRAMEWORKS
+  legal_frameworks: [eu_gdpr, eu_nis2]
+  excluded_frameworks: []                 # frameworks this provider explicitly cannot support
+
+  # EXTERNAL DEPENDENCIES โ€” does the provider require external connectivity?
+  external_dependencies:
+    air_gap_capable: false               # true = can operate without external connectivity
+    external_services:
+      - service: licensing_server
+        jurisdiction: US
+        data_shared: [license_key, hostname]
+      - service: telemetry_endpoint
+        jurisdiction: US
+        data_shared: [usage_metrics]
+    opt_out_available:
+      telemetry: true                    # telemetry can be disabled
+
+  # THIRD-PARTY SUB-PROCESSORS
+  sub_processors:
+    - name: "Acme Cloud Storage"
+      jurisdiction: US
+      data_handled: [vm_disk_images]
+      gdpr_dpa_in_place: true
+
+  # GOVERNMENT ACCESS RISK
+  government_access_risk:
+    jurisdictions_with_compelled_access: [US]
+    # US CLOUD Act, FISA Section 702, etc.
+    legal_challenge_policy: notify_customer_where_legally_permitted
+
+  # CERTIFICATIONS โ€” with validity periods
+  certifications:
+    - name: ISO-27001
+      issuer: BSI
+      valid_from: "2024-03-01"
+      valid_until: "2027-03-01"
+      scope: "Cloud Infrastructure Operations"
+      certificate_ref:
+        credential_provider_uuid: 
+        path: "dcm/providers/kubevirt/certs/iso27001"
+    - name: SOC2-Type-II
+      issuer: Deloitte
+      valid_from: "2025-01-01"
+      valid_until: "2026-01-01"
+      scope: "Infrastructure as a Service"
+
+  # AUDIT RIGHTS
+  audit_rights:
+    customer_audit_right: true
+    audit_notice_days: 30
+    third_party_audit_accepted: true
+
+  # CHANGE NOTIFICATION OBLIGATION
+  change_notification:
+    # Provider MUST notify DCM when any sovereignty data changes
+    notification_endpoint: 
+    # Changes that MUST be notified:
+    mandatory_notification_events:
+      - certification_expiry
+      - new_jurisdiction_added
+      - jurisdiction_removed
+      - new_sub_processor
+      - sub_processor_removed
+      - new_external_dependency
+      - government_access_event
+    notification_sla: PT24H              # must notify within 24 hours of change
+```
+
+### 11.3 Change Notification and DCM Response
+
+When a provider notifies DCM of a sovereignty change (or DCM discovers one via periodic verification):
+
+```
+Provider sovereignty change detected
+  โ”‚
+  โ–ผ
+Policy Engine evaluates: does the change violate any Tenant's
+sovereignty requirements for resources currently placed with this provider?
+  โ”‚
+  โ”œโ”€โ”€ No violations:
+  โ”‚     Update sovereignty record
+  โ”‚     Emit: provider.sovereignty_changed webhook event
+  โ”‚     Notify: affected Tenants (informational)
+  โ”‚
+  โ””โ”€โ”€ Violations found โ€” for each affected resource:
+        Policy determines action:
+          notify_only       โ€” inform Tenant; no automatic action
+          pause             โ€” suspend resource; Tenant must act
+          migrate           โ€” Provider-Portable Rehydration to compliant provider
+          emergency_migrate โ€” immediate parallel provisioning; decommission after
+        Record: sovereignty_violation_record (in Audit Store)
+        Notify: Tenant owner, platform admin, data_protection_officer (if declared)
+```
+
+### 11.4 Sovereignty Violation Record
+
+```yaml
+sovereignty_violation_record:
+  record_uuid: 
+  detected_at: 
+  detection_method: 
+  provider_uuid: 
+  change_type: 
+  previous_value: 
+  new_value: 
+  affected_resources:
+    - entity_uuid: 
+      tenant_uuid: 
+      sovereignty_requirement_violated: "Data must not transit US jurisdiction"
+      policy_action: migrate
+      migration_request_uuid:    # if migrate or emergency_migrate
+  notifications_sent:
+    - recipient: tenant_owner
+    - recipient: platform_admin
+    - recipient: data_protection_officer
+```
+
+### 11.5 Auto-Migration
+
+Auto-migration (`migrate` or `emergency_migrate` policy action) uses Provider-Portable Rehydration:
+
+```
+Sovereignty violation detected โ†’ policy declares: migrate
+  โ”‚
+  โ–ผ
+DCM assembles migration request:
+  โ”‚  Same entity declaration โ€” new placement constraints
+  โ”‚  Placement engine excludes non-compliant provider from candidate set
+  โ”‚  Selects compliant alternative provider
+  โ”‚
+  โ–ผ
+emergency_migrate: parallel provisioning
+  โ”‚  New resource provisioned BEFORE old one decommissioned
+  โ”‚  Traffic/workload cutover coordinated with Tenant
+  โ”‚
+standard migrate: sequential
+  โ”‚  Old resource suspended โ†’ new resource provisioned โ†’ old decommissioned
+  โ”‚
+  โ–ผ
+Full audit trail: sovereignty_violation_record links to migration request
+```
+
+### 11.6 System Policies โ€” Provider Sovereignty
+
+| Policy | Rule |
+|--------|------|
+| `SOV-001` | All provider registrations must include a `sovereignty_declaration` block covering operating_jurisdictions, legal_frameworks, data_residency_guarantees, external_dependencies, certifications with validity periods, and government_access_risk. |
+| `SOV-002` | Providers must notify DCM when any declared sovereignty data changes. Sovereignty change notifications are treated as discovered drift and trigger Policy Engine re-evaluation. Notification SLA is declared in the provider registration. |
+| `SOV-003` | When a provider sovereignty change violates a Tenant's sovereignty requirements, the Policy Engine evaluates affected resources and applies the declared action: notify_only, pause, migrate, or emergency_migrate. |
+| `SOV-004` | Auto-migration triggered by SOV-003 uses Provider-Portable Rehydration. The non-compliant provider is excluded from the placement candidate set. The migration is a first-class DCM operation with full audit trail. |
+| `SOV-005` | Certification validity periods are tracked by DCM. Certifications expiring within P30D trigger a warning notification to the provider and affected Tenants. Expired certifications trigger SOV-003 re-evaluation. |
+
+---
+
+## 12. System Policies โ€” Storage Architecture
+
+| Policy | Rule |
+|--------|------|
+| `STO-001` | Storage Providers must declare replication capabilities. Active Profile determines minimum replication requirements. Providers not meeting Profile minimum cannot be activated for that Profile's stores. |
+| `STO-002` | Storage Provider failure behavior is declared per store type and governed by the active Profile. GitOps unavailability queues writes locally โ€” does not silently drop. Commit Log quorum loss aborts the triggering operation. Audit Store unavailability accumulates entries in the Commit Log. Search Index unavailability degrades query responses without impacting write operations. |
+| `STO-003` | The Search Index is a separate Storage Provider sub-type โ€” non-authoritative and rebuildable. API queries may specify `freshness: authoritative` to bypass the index. |
+| `STO-007` | The Realized Store is a write-once snapshot store. Every write requires a non-nullable corresponding_requested_state_uuid. Drift detection, discovery cycles, and unsanctioned provider changes do not write to the Realized Store. Enforcement is at the store API level, not by convention. |
+| `STO-008` | The Intent Store requires a GitOps implementation. The Requested Store requires write-once semantics; GitOps is the reference implementation; write-once document stores are supported for production scale. |
+| `STO-004` | The Audit Store is a specialized Storage Provider sub-type โ€” append-only, hash chain integrity, reference-based retention, compliance-grade queries. The Event Stream is the delivery channel only. |
+| `STO-005` | GitOps stores use a handle-based directory structure. The main branch is authoritative. Minimal and dev profiles may use a monorepo; standard and above should use separate repositories per store type. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/12-audit-provenance-observability.md b/content/docs/data-model/12-audit-provenance-observability.md
new file mode 100644
index 0000000..d1f3409
--- /dev/null
+++ b/content/docs/data-model/12-audit-provenance-observability.md
@@ -0,0 +1,466 @@
+# DCM Data Model โ€” Audit, Provenance, and Observability
+
+> **โš ๏ธ Active Development Notice**
+> 
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+> 
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Four States](02-four-states.md) | [Storage Providers](11-storage-providers.md) | [Context and Purpose](00-context-and-purpose.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA**
+>
+> The Data abstraction โ€” audit records, provenance as structural data
+
+
+
+---
+
+## 1. Purpose
+
+Audit, Provenance, and Observability are three distinct but related capabilities in DCM. They are often conflated โ€” this document separates them precisely, defines their relationship, and establishes the architectural model for each.
+
+| Capability | Question Answered | Audience | Time Orientation |
+|------------|------------------|----------|-----------------|
+| **Provenance** | Where did this data come from and how did it change? | System โ€” embedded in data | Embedded in every payload |
+| **Audit** | What happened, who authorized it, can you prove it? | Auditors, Compliance, Security | Backward-looking |
+| **Observability** | Is the system healthy and performing within expectations? | SRE, Platform Engineers | Forward-looking, real-time |
+
+---
+
+## 2. Provenance
+
+### 2.1 Definition
+
+Provenance is the structural data lineage mechanism embedded in every field of every DCM payload. It is not a separate system โ€” it is part of the data itself. Every field that can be created or modified by any DCM process carries provenance metadata alongside its value.
+
+Provenance answers: "where did this value come from, what modified it, and why?"
+
+Audit queries provenance to answer its questions. Observability does not use provenance directly โ€” it operates on event streams and metrics.
+
+### 2.2 Provenance Structure
+
+See [Context and Purpose โ€” Section 4.4](00-context-and-purpose.md) for the complete field-level provenance structure. The key elements:
+
+```yaml
+field_name:
+  value: 
+  metadata:
+    override: 
+    basis_for_value: 
+    baseline_value: 
+    locked_by_policy_uuid: 
+    locked_at_level: 
+  provenance:
+    origin:
+      value: 
+      source_type: 
+      source_uuid: 
+      timestamp: 
+    modifications:
+      - sequence: 1
+        previous_value: 
+        modified_value: 
+        source_uuid: 
+        operation_type: 
+        actor_uuid: 
+        timestamp: 
+        reason: 
+```
+
+### 2.3 Provenance Obligations
+
+Every DCM component that modifies data carries a provenance obligation โ€” it must record its UUID, operation type, actor, timestamp, and reason for every field it touches. A component that modifies data without recording provenance violates the data model contract.
+
+| Component | Provenance Obligation |
+|-----------|----------------------|
+| Request Payload Processor | Record source UUID and type for every field assembled from layers |
+| Policy Engine | Record policy UUID, level, operation type, and reason for every field modified or locked |
+| Service Provider (Denaturalization) | Record provider UUID and timestamp for every field in the realized payload |
+| Storage Provider | Emit provenance event to Audit component on every write |
+| Resource Discovery | Record provider UUID, timestamp, and method for every discovered field |
+| Rehydration Pipeline | Record source store, source record UUID, rehydration reason, and actor UUID |
+
+### 2.4 Provenance Across the Full Lifecycle
+
+The provenance chain for a single field may span multiple lifecycle stages:
+
+```
+Base Layer sets encryption_standard: AES-128
+  origin: {source_type: base_layer, source_uuid: layer-uuid-001}
+
+Transformation Policy enriches to AES-256
+  modification: {source_uuid: policy-uuid-001, operation: transformation,
+                 reason: "Security standard requires AES-256 minimum"}
+
+GateKeeper Policy locks as immutable
+  modification: {source_uuid: policy-uuid-002, operation: lock,
+                 reason: "CISO mandate โ€” encryption standard non-negotiable"}
+
+Provider reports realized value: AES-256
+  modification: {source_uuid: provider-uuid-001, operation: denaturalization,
+                 reason: "Provider confirmed encryption standard applied"}
+
+Drift detected: discovered value AES-128
+  modification: {source_uuid: discovery-uuid-001, operation: discovery,
+                 reason: "Direct modification detected outside DCM lifecycle"}
+```
+
+The complete chain tells the full story of that field across its entire existence.
+
+---
+
+## 3. Audit
+
+### 3.1 Definition
+
+Audit is the compliance-grade, queryable record of all significant actions across the DCM lifecycle. It is backward-looking, human-readable, and access-controlled by persona. It answers: "what happened, who authorized it, can you prove it?"
+
+Audit is a **separate component** โ€” not a query against the GitOps stores, not a view into provenance directly. It aggregates and indexes provenance events from all stores and presents them through a structured query API surfaced by the DCM API Gateway.
+
+### 3.2 Architecture
+
+```
+All Storage Providers emit provenance events (contractual obligation)
+  โ”‚
+  โ”‚  Events include: entity_uuid, operation, actor_uuid,
+  โ”‚  timestamp, payload_hash, store_reference
+  โ–ผ
+Audit Component
+  โ”‚  Receives provenance events from all stores
+  โ”‚  Correlates events by entity_uuid across all stores
+  โ”‚  Indexes for structured query: by entity, tenant, actor,
+  โ”‚  time range, operation type, policy UUID
+  โ”‚  Maintains immutable records โ€” audit records are never modified
+  โ”‚  Enforces long retention (regulatory periods โ€” configurable,
+  โ”‚  minimum 7 years for FSI deployments)
+  โ”‚  Verifies payload hashes โ€” detects store tampering
+  โ–ผ
+DCM API Gateway
+  โ”‚  Surfaces Audit query API with persona-based access control
+  โ”‚  Auditor: full access โ€” all entities, all tenants, all time
+  โ”‚  SRE: full access within operational scope
+  โ”‚  Admin: full access within administrative scope
+  โ”‚  Consumer: own entities and requests only
+  โ”‚  Provider: own provider's operations only
+```
+
+### 3.3 Audit API (via DCM API Gateway)
+
+```
+GET  /api/v1/audit/entities/{uuid}/history
+     Returns: complete lifecycle history for an entity
+     Fields: all state transitions, all provenance events, all actor actions
+
+GET  /api/v1/audit/requests/{uuid}/provenance
+     Returns: complete provenance chain for a specific request
+     Fields: intent, assembly, policy evaluation, provider dispatch, realization
+
+GET  /api/v1/audit/policies/{uuid}/evaluations
+     Returns: all evaluations of a specific policy across all requests
+     Fields: when it ran, what it did, which entities it affected
+
+GET  /api/v1/audit/actors/{uuid}/activity
+     Returns: all actions taken by a specific actor
+     Fields: requests submitted, approvals given, policy evaluations triggered
+
+GET  /api/v1/audit/tenants/{uuid}/activity
+     Returns: all activity within a specific tenant
+     Fields: requests, realizations, drift events, policy violations
+
+POST /api/v1/audit/query
+     Body: structured audit query with field filters, time range, pagination
+     Returns: matching audit records
+```
+
+### 3.4 Audit Record Structure
+
+```yaml
+audit_record:
+  audit_uuid: 
+  entity_uuid: 
+  tenant_uuid: 
+  event_type: 
+  timestamp: 
+  actor_uuid: 
+  actor_type: 
+
+  source_store:
+    store_type: 
+    store_uuid: 
+    store_reference: 
+    payload_hash: 
+
+  provenance_summary:
+    
+
+  policy_context:
+    policies_evaluated: []
+    policies_applied: []
+    policies_rejected: []
+    override_control_changes: []
+
+  related_records:
+    intent_record_uuid: 
+    requested_record_uuid: 
+    realized_event_uuid: 
+    rehydration_source_uuid: 
+```
+
+### 3.5 Audit Integrity
+
+Audit records are immutable. The Audit component verifies payload hashes against the Storage Provider's stored values on every read โ€” if a hash mismatch is detected, the Audit component flags the record as potentially tampered and escalates to the Policy Engine.
+
+The Audit Store itself is a Storage Provider with the highest consistency and durability requirements โ€” linearizable consistency, synchronous replication, cryptographic payload hashing, and compliance-grade retention.
+
+---
+
+## 4. Observability
+
+### 4.1 Definition
+
+Observability is real-time insight into the health, performance, and behavior of the DCM system. It is forward-looking, machine-readable, and aggregated. It answers: "is the system healthy, where are the bottlenecks, what is the error rate?"
+
+Observability is operationally oriented โ€” SREs and platform engineers use it to understand system behavior and respond to incidents. It does not carry the compliance obligations of Audit.
+
+### 4.2 The Three Pillars
+
+**Metrics** โ€” quantitative measurements of system state over time
+- Request throughput: requests/second by resource type, tenant, provider
+- Latency: assembly time, policy evaluation time, provider dispatch time, end-to-end time
+- Error rates: policy rejection rate, provider failure rate, drift detection rate
+- Capacity: provider utilization, store capacity, queue depth
+- Cost: accumulated cost by tenant, resource type, provider
+
+**Traces** โ€” distributed traces of request execution across components
+- Full request trace from Intent State creation through provider dispatch
+- Policy evaluation trace โ€” which policies ran, in what order, how long each took
+- Assembly trace โ€” which layers were applied, in what order, what each contributed
+
+**Logs** โ€” structured event logs from all DCM components
+- Component startup and shutdown
+- Registration events (provider registration, deregistration)
+- Error conditions
+- Drift detection events
+- Unsanctioned change events
+
+### 4.3 Architecture
+
+```
+DCM components emit metrics, traces, and logs
+  โ”‚
+  โ”‚  All telemetry in OpenTelemetry format
+  โ”‚  Standardized metric names, trace context propagation,
+  โ”‚  structured log format
+  โ–ผ
+Observability Store
+  โ”‚  Time-series metrics store (Prometheus-compatible)
+  โ”‚  Distributed trace store (Jaeger/Zipkin compatible)
+  โ”‚  Log aggregation (structured, indexed)
+  โ”‚  Short-to-medium retention (configurable โ€” typically 90 days)
+  โ–ผ
+DCM API Gateway
+  โ”‚  GET /api/v1/observability/metrics
+  โ”‚  GET /api/v1/observability/traces/{request_uuid}
+  โ”‚  GET /api/v1/observability/health
+  โ”‚  GET /api/v1/observability/providers/{uuid}/performance
+  โ–ผ
+Dashboards and alerting (external tooling)
+  โ”‚  Grafana, DataDog, Splunk โ€” implementor choice
+  โ”‚  DCM provides OpenTelemetry-compatible telemetry
+  โ”‚  Dashboards are deployment artifacts, not DCM artifacts
+```
+
+### 4.4 Standard DCM Metrics
+
+```
+# Request lifecycle
+dcm_requests_total{resource_type, tenant, status}
+dcm_request_duration_seconds{resource_type, stage}
+dcm_requests_in_flight{resource_type, tenant}
+
+# Policy Engine
+dcm_policy_evaluations_total{policy_type, result}
+dcm_policy_evaluation_duration_seconds{policy_type}
+dcm_policy_rejections_total{policy_uuid, resource_type}
+
+# Provider
+dcm_provider_requests_total{provider_uuid, resource_type, status}
+dcm_provider_response_duration_seconds{provider_uuid}
+dcm_provider_capacity_available{provider_uuid, resource_type}
+dcm_provider_health_status{provider_uuid}
+
+# Drift
+dcm_drift_detections_total{resource_type, severity}
+dcm_unsanctioned_changes_total{resource_type, provider_uuid}
+dcm_drift_resolution_duration_seconds{resolution_type}
+
+# Storage
+dcm_store_write_duration_seconds{store_type, store_uuid}
+dcm_store_read_duration_seconds{store_type, store_uuid}
+dcm_store_health_status{store_type, store_uuid}
+
+# Rehydration
+dcm_rehydrations_total{source_store, placement_mode, policy_version}
+dcm_rehydration_duration_seconds{source_store}
+```
+
+### 4.5 Observability vs Audit โ€” The Key Distinctions
+
+| Dimension | Audit | Observability |
+|-----------|-------|---------------|
+| **Retention** | Regulatory period (years) | Operational window (days-months) |
+| **Access control** | Strict persona-based | Operational teams |
+| **Data volume** | Moderate โ€” per-entity events | High โ€” continuous time series |
+| **Query model** | Structured, entity-centric | Aggregated, time-series |
+| **Immutability** | Absolute โ€” records never modified | Aggregated data may be downsampled |
+| **Compliance** | Compliance-grade โ€” hash-verified | Operational โ€” best effort |
+| **Use case** | Prove what happened | Understand what is happening |
+
+---
+
+## 5. The API Gateway โ€” Unified Access
+
+All three capabilities โ€” Provenance (embedded in data), Audit (structured history), and Observability (operational telemetry) โ€” are surfaced through the DCM API Gateway. There is no separate endpoint for audit or observability. All DCM capabilities live in a unified API hierarchy.
+
+```
+DCM API Gateway
+  โ”‚
+  โ”œโ”€โ”€ /api/v1/catalog/          # Service Catalog
+  โ”œโ”€โ”€ /api/v1/requests/         # Request submission and management
+  โ”œโ”€โ”€ /api/v1/entities/         # Entity lifecycle management
+  โ”œโ”€โ”€ /api/v1/providers/        # Provider registration and management
+  โ”œโ”€โ”€ /api/v1/policies/         # Policy management
+  โ”œโ”€โ”€ /api/v1/audit/            # Audit queries
+  โ”œโ”€โ”€ /api/v1/observability/    # Operational metrics and traces
+  โ””โ”€โ”€ /api/v1/admin/            # Administrative functions
+```
+
+Persona-based access control is enforced at the API Gateway level for all endpoints. The same authentication and authorization model applies across the entire API surface.
+
+---
+
+## 6. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should the Audit Store be a specialized Storage Provider or can a general Event Stream Store satisfy the audit contract? | Architecture | โœ… Resolved โ€” specialized Storage Provider sub-type; append-only; hash chain integrity; reference-based retention; compliance queries; see doc 11 (STO-004) |
+| 2 | How are audit records replicated across sites in air-gapped or geographically distributed deployments? | Sovereignty | โœ… Resolved โ€” live sync for Regional DCMs; signed bundle for Sovereign DCMs; sovereignty check required; hash chain preserved across transport (AUD-018) |
+| 3 | Should DCM provide a default observability dashboard or only the telemetry? | Deployment | โœ… Resolved โ€” default Grafana dashboard for minimal/dev/standard; enterprise integration recommended for prod; required for fsi; local-only for sovereign (OBS-002) |
+| 4 | How does the Audit component handle provenance events from a Storage Provider that has been deregistered? | Operational | โœ… Resolved โ€” two-stage model handles this; Commit Log independent of Storage Providers; gap record inserted on Audit Store recovery; chain makes gap explicit (AUD-019) |
+
+---
+
+## 7. Related Concepts
+
+- **Provenance** โ€” field-level data lineage embedded in every DCM payload
+- **Audit Store** โ€” compliance-grade, immutable store of all audit records
+- **Observability Store** โ€” time-series metrics, traces, and logs
+- **Storage Provider** โ€” formal provider type for all DCM stores
+- **API Gateway** โ€” unified access point for all DCM capabilities including audit and observability
+- **Drift Detection** โ€” uses discovered vs realized state comparison; drift events feed the Audit component
+- **Unsanctioned Change** โ€” a specific audit event type triggered by unauthorized resource modification
+
+
+## 7. Audit Provenance Observability Gap Resolutions
+
+### 7.1 Audit Store Architecture (Q1)
+
+Resolved as STO-004 in doc 11 (Storage Providers). The Audit Store is a specialized Storage Provider sub-type โ€” append-only with immutability enforcement, hash chain integrity, reference-based retention tracking, and compliance-grade multi-dimensional queries. The Event Stream is the delivery channel only, not the compliance destination.
+
+### 7.2 Audit Record Replication Across Sites (Q2)
+
+Each DCM instance maintains its own Audit Store. Replication uses live sync (Regional DCMs with connectivity) or signed bundle export (Sovereign DCMs without connectivity).
+
+```yaml
+audit_replication:
+  model: 
+
+  live_sync:                        # Regional DCMs with Hub connectivity
+    direction: regional_to_hub      # Regional pushes aggregated view to Hub
+    filters:
+      include: [SECURITY, GATEKEEPER_TRIGGERED, SOVEREIGNTY_VIOLATION]
+    sovereignty_check: required     # before any replication
+
+  signed_bundle:                    # Sovereign DCMs
+    export_on: [scheduled, connectivity_window, on_demand]
+    schedule: "0 0 * * 0"           # weekly during connectivity window
+    encryption: required
+    hash_chain_preserved: true      # chain integrity maintained across transport
+    import_at: hub_dcm_audit_store
+
+  per_instance_only:                # fully isolated Sovereign DCMs
+    export_on_request: via_signed_bundle_manual_transfer
+```
+
+### 7.3 Default Observability Dashboard (Q3)
+
+DCM ships a default Grafana-based observability dashboard for minimal/dev/standard profiles.
+
+```yaml
+default_observability_dashboard:
+  implementation: grafana
+  pre_built_dashboards:
+    - dcm_overview              # request throughput, error rates, component health
+    - resource_lifecycle        # entity state transitions, rehydration activity
+    - policy_evaluation         # GateKeeper triggers, shadow results, validation failures
+    - provider_health           # provider availability, capacity confidence, trust scores
+    - audit_integrity           # hash chain status, pending forwards, chain breaks
+    - federation_status         # federation tunnel health, cross-DCM traffic
+  profile_behavior:
+    minimal: included
+    dev: included
+    standard: included_optional         # shipped; organizations may substitute
+    prod: integration_recommended       # integrate with enterprise observability
+    fsi: integration_required
+    sovereign: local_only               # local Grafana; no external connections
+  export_formats: [prometheus, opentelemetry, json]
+```
+
+### 7.4 Audit Component Handling Failing Storage Provider (Q4)
+
+The two-stage audit model handles this by design โ€” the Stage 1 Commit Log (etcd) has no dependency on any Storage Provider.
+
+```
+Storage Provider failure detected
+  โ”‚
+  โ–ผ Stage 1 Commit Log (etcd) โ€” independent of Storage Provider
+  โ”‚   Records: STORAGE_PROVIDER_FAILURE event immediately
+  โ”‚
+  โ–ผ Stage 2 Audit Forward Service โ€” async, after recovery
+  โ”‚   Forwards accumulated events including the failure event itself
+  โ”‚
+  โ–ผ For Audit Store self-failure specifically:
+      Commit Log accumulates events as pending_forward
+      On recovery: queue drains in order
+      AUDIT_STORE_UNAVAILABLE gap record inserted with exact outage timestamps
+      Hash chain makes the gap explicitly visible โ€” not hidden
+```
+
+The gap record is not a failure โ€” it is evidence of correct behavior. Auditors can see exactly when the Audit Store was unavailable and that no records were lost (all arrived after recovery).
+
+### 7.5 System Policies โ€” Audit Provenance Gaps
+
+| Policy | Rule |
+|--------|------|
+| `STO-004` | The Audit Store is a specialized Storage Provider sub-type โ€” append-only, hash chain integrity, reference-based retention, compliance-grade queries. Event Stream is the delivery channel only. (See doc 11) |
+| `AUD-018` | Audit records are replicated using live sync (Regional DCMs) or signed bundle export (Sovereign DCMs). Sovereignty checks required before any replication. Hash chain integrity preserved across transport. Fully isolated Sovereign DCMs maintain local-only audit stores with manual export. |
+| `OBS-002` | DCM ships a default Grafana-based observability dashboard for minimal/dev/standard profiles. Standard+ profiles may substitute enterprise platforms. FSI requires enterprise observability. Sovereign DCMs use local dashboard only with no external connections. |
+| `AUD-019` | Storage Provider failures are recorded via the Stage 1 Commit Log (etcd), which is independent of all Storage Providers. Audit Store self-failures produce pending_forward records. On recovery, a gap record (AUDIT_STORE_UNAVAILABLE) is inserted with the outage window timestamps. The hash chain gap is explicit and auditable. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/13-ingestion-model.md b/content/docs/data-model/13-ingestion-model.md
new file mode 100644
index 0000000..4620593
--- /dev/null
+++ b/content/docs/data-model/13-ingestion-model.md
@@ -0,0 +1,481 @@
+# DCM Data Model โ€” Ingestion Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + PROVIDER**
+>
+> Data: ingestion state artifacts. Provider: discovery provider invocation
+
+
+
+---
+
+## 1. Purpose
+
+The DCM Ingestion Model is the **unified mechanism for bringing entities that exist outside DCM's lifecycle control into DCM's governance model**. It applies to three distinct sources:
+
+- **V1 Migration** โ€” entities from a DCM V1 deployment that predate the mandatory Tenant model
+- **Brownfield Discovery** โ€” entities discovered by a Service Provider that already exist in the infrastructure but are unknown to DCM
+- **Manual Import** โ€” entities imported from external systems (CMDBs, spreadsheets, legacy records) during onboarding
+
+All three sources follow the same pattern: ingest, enrich, and promote. The same data structures, the same governance policies, the same audit trail, and the same transitional holding mechanism apply regardless of source.
+
+**The three-step pattern:**
+
+```
+1. INGEST   โ€” bring the entity into DCM with whatever identity and metadata is available
+2. ENRICH   โ€” associate business data, ownership, Tenant assignment, and relationships
+3. PROMOTE  โ€” transition from holding state to full DCM lifecycle ownership
+```
+
+---
+
+## 2. Design Principles
+
+**Unified model โ€” minimum variance.** V1 migration and brownfield ingestion are the same fundamental operation. One model, one audit record structure, one set of governance policies.
+
+**Non-blocking.** Entities that cannot be immediately assigned a Tenant do not block migration or discovery. They land in the `__transitional__` Tenant and are resolved progressively. Migration does not require every entity to be assigned before any entity can proceed.
+
+**Provenance transparency.** Ingested entities are honest about their provenance depth. `created_via: migration` or `created_via: discovery` on the artifact metadata signals that the chain has limited depth. The ingestion record carries the confidence level.
+
+**Promotion gates governance.** An entity in a holding state cannot be the parent of a new allocated resource claim, cannot be used as a hard dependency by new requests, and cannot receive new operational relationships until promoted. Informational relationships are permitted โ€” entities can be referenced during enrichment.
+
+**Audit completeness.** Every ingested entity carries an `ingestion_record` in its provenance. Every Tenant assignment, enrichment action, and promotion event is recorded with actor, timestamp, and reason.
+
+---
+
+## 3. Ingestion Lifecycle States
+
+Entities going through ingestion follow a distinct mini-lifecycle before entering the standard entity lifecycle:
+
+```
+INGESTED
+  โ”‚  Entity exists in DCM. Minimal metadata. Tenant may be __transitional__.
+  โ”‚  Action: enrich โ€” add business data, assign relationships, assign real Tenant
+  โ–ผ
+ENRICHING
+  โ”‚  Tenant assigned. Metadata being completed. Relationships being established.
+  โ”‚  Action: complete enrichment, satisfy governance requirements
+  โ–ผ
+PROMOTED
+  โ”‚  All required fields present. Governance satisfied. Full DCM lifecycle assumed.
+  โ–ผ
+OPERATIONAL  (standard entity lifecycle from here)
+```
+
+### 3.1 State Behavior
+
+| State | Tenant | New Requests Can Use? | Parent for Allocations? | New Relationships? |
+|-------|--------|----------------------|------------------------|-------------------|
+| `INGESTED` | `__transitional__` or assigned | No | No | Informational only |
+| `ENRICHING` | Assigned | No | No | Operational (read-only) |
+| `PROMOTED` | Assigned | Yes | Yes | All types |
+| `OPERATIONAL` | Assigned | Yes | Yes | All types |
+
+### 3.2 Promotion Requirements
+
+Before an entity can be promoted, the following must be satisfied:
+
+- Assigned to a real Tenant (not `__transitional__`)
+- All `universal` fields on the Resource Type Specification are populated
+- All `constituent` relationships declared on the Resource Type Specification are resolved
+- At least one actor has reviewed and authorized the promotion
+- `ingestion_record.enrichment_status` is `complete`
+
+---
+
+## 4. The `__transitional__` Tenant
+
+The `__transitional__` Tenant is a DCM System artifact โ€” a system-managed holding area for entities that have been ingested but not yet assigned to a real Tenant.
+
+```yaml
+tenant:
+  uuid: 
+  handle: "__transitional__"
+  type: system_managed
+  purpose: ingestion_holding
+  governance:
+    max_residency_days: 90          # configurable per deployment
+    on_max_residency: escalate      # escalate | block | alert
+    escalation_endpoint: 
+  hard_tenancy:
+    cross_tenant_relationships: operational_only
+  artifact_metadata:
+    created_by:
+      display_name: "DCM Ingestion System"
+    created_via: system
+    status: active
+```
+
+**Properties:**
+- Cannot be deleted
+- Cannot be renamed
+- Cannot be used for new resource provisioning โ€” only ingestion assignment
+- Entities in `__transitional__` are fully auditable and visible in DCM
+- Governance policy enforces maximum residency and escalation
+
+---
+
+## 5. The Ingestion Record
+
+Every ingested entity carries an `ingestion_record` in its provenance chain. This is the audit record of how the entity entered DCM.
+
+```yaml
+ingestion_record:
+  ingestion_uuid: 
+  resource_entity_uuid: 
+  ingestion_timestamp: 
+
+  ingestion_source: 
+
+  # V1 migration fields (when ingestion_source: v1_migration)
+  v1_identifier: 
+  v1_metadata_snapshot: 
+
+  # Brownfield discovery fields (when ingestion_source: brownfield_discovery)
+  discovered_state_uuid: 
+  discovery_provider_uuid: 
+  discovery_timestamp: 
+
+  # Manual import fields (when ingestion_source: manual_import)
+  import_source_system: 
+  import_reference: 
+
+  # Common fields
+  assigned_tenant_uuid: 
+  assignment_method: 
+  assignment_signal: >
+    Human-readable description of what drove auto-assignment.
+    e.g., "Resource group membership: payments-group โ†’ Payments Tenant"
+    e.g., "Business unit metadata: BU-PAY โ†’ Payments Tenant"
+    e.g., "No signal found โ€” assigned to __transitional__"
+  assigned_by:
+    uuid: 
+    display_name: 
+    timestamp: 
+
+  ingestion_confidence: 
+  # high:   strong unambiguous signal โ€” auto-assignment reliable
+  # medium: inferred from metadata โ€” reasonable confidence, human review recommended
+  # low:    orphaned or conflicting signals โ€” assigned to __transitional__
+
+  enrichment_status: 
+  enrichment_history:
+    - sequence: 1
+      action: 
+      performed_by:
+        display_name: 
+      timestamp: 
+      detail: 
+
+  promoted_at: 
+  promoted_by:
+    display_name: 
+```
+
+---
+
+## 6. Auto-Assignment Signals
+
+When DCM ingests an entity, it attempts auto-assignment to a real Tenant using the following signals in priority order:
+
+| Signal | Confidence | Description |
+|--------|-----------|-------------|
+| Explicit ownership metadata | High | Business unit, cost center, or team tag on the resource maps unambiguously to a Tenant |
+| Resource group membership | High | Resource belongs to a group that maps to a known Tenant |
+| Request history | High | V1 request record identifies the requesting team, which maps to a Tenant |
+| Network / location context | Medium | Resource's location, VLAN, or network segment maps to a Tenant by convention |
+| Naming convention | Medium | Resource name matches a known Tenant naming pattern |
+| Provider context | Medium | Resource was provisioned by a known provider associated with a Tenant |
+| No signal found | Low | No auto-assignment possible โ€” entity goes to `__transitional__` |
+
+Multiple signals can be combined. If signals conflict, the higher-confidence signal wins and the conflict is recorded in the ingestion record with `ingestion_confidence: medium` regardless of individual signal strengths.
+
+---
+
+## 7. V1 Migration
+
+### 7.1 Overview
+
+V1 resources have no `tenant_uuid`. V2 requires one (`TEN-001`). The V1 migration process uses the ingestion model to assign every V1 resource a Tenant before it can participate in V2 operations.
+
+### 7.2 Resource Categories
+
+| Category | Description | Assignment Path |
+|----------|-------------|----------------|
+| **Auto-assignable** | Clear ownership signals โ€” resource group, business unit, request history | Auto-assigned during migration analysis pass |
+| **Manually assignable** | Ambiguous signals โ€” multiple possible owners, or medium-confidence signals only | Surfaced in admin assignment queue |
+| **Orphaned** | No signals โ€” no ownership data available | Assigned to `__transitional__` |
+
+### 7.3 Migration Flow
+
+```
+V1 estate
+  โ”‚
+  โ–ผ  Step 1 โ€” Pre-migration analysis pass
+  โ”‚  Inventory all V1 resources
+  โ”‚  Attempt auto-assignment via signals (Section 6)
+  โ”‚  Classify each resource: auto_assignable | manually_assignable | orphaned
+  โ”‚  Produce migration readiness report
+  โ”‚
+  โ–ผ  Step 2 โ€” Auto-assignment
+  โ”‚  Create or map to existing V2 Tenants
+  โ”‚  Assign auto_assignable resources in bulk
+  โ”‚  Create ingestion_record per resource (ingestion_source: v1_migration)
+  โ”‚  State: INGESTED โ†’ ENRICHING (for auto-assigned)
+  โ”‚
+  โ–ผ  Step 3 โ€” Manual assignment queue
+  โ”‚  manually_assignable resources surfaced in admin UI
+  โ”‚  Administrators review and assign Tenants
+  โ”‚  Each assignment recorded in enrichment_history
+  โ”‚
+  โ–ผ  Step 4 โ€” Transitional fallback
+  โ”‚  orphaned resources โ†’ __transitional__ Tenant
+  โ”‚  ingestion_record.assignment_method: transitional
+  โ”‚  ingestion_record.ingestion_confidence: low
+  โ”‚  Governance timer starts
+  โ”‚
+  โ–ผ  Step 5 โ€” Enrichment and promotion
+  โ”‚  Relationships established, missing fields populated
+  โ”‚  Each entity reviewed and promoted when complete
+  โ”‚  State: ENRICHING โ†’ PROMOTED โ†’ OPERATIONAL
+  โ”‚
+  โ–ผ  Migration complete when __transitional__ Tenant is empty
+```
+
+### 7.4 Migration System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ING-001` | Every entity ingested into V2 from V1 must be assigned to exactly one Tenant โ€” either a real Tenant or `__transitional__` โ€” before it is eligible for new V2 requests |
+| `ING-002` | Entities in `INGESTED` or `ENRICHING` state may not be the parent resource for a new allocated resource claim |
+| `ING-003` | The `__transitional__` Tenant is system-managed and cannot be deleted, renamed, or used for new resource provisioning |
+| `ING-004` | Every ingested entity must carry an `ingestion_record` in its provenance chain |
+| `ING-005` | Entities in `__transitional__` for longer than `max_residency_days` must trigger the configured escalation action |
+
+---
+
+## 8. Brownfield Ingestion
+
+### 8.1 Overview
+
+Brownfield ingestion brings infrastructure that already exists in the real world โ€” but is unknown to DCM โ€” under DCM lifecycle management. The source is the **Discovered State**: a Service Provider interrogates existing infrastructure and creates Discovered State records for everything it finds.
+
+This is the "greening the brownfield" use case โ€” taking an unmanaged estate and progressively bringing it under DCM governance without requiring a big-bang cutover.
+
+### 8.2 Brownfield Flow
+
+```
+Service Provider performs discovery scan
+  โ”‚  Interrogates existing infrastructure
+  โ”‚  Creates Discovered State records for all found entities
+  โ”‚
+  โ–ผ  DCM identifies "unmanaged" discovered entities
+  โ”‚  Discovered State records with no matching Realized State = unmanaged
+  โ”‚  These are brownfield candidates
+  โ”‚
+  โ–ผ  Ingestion initiation
+  โ”‚  Platform admin or automated policy initiates ingestion
+  โ”‚  DCM creates entity stubs with:
+  โ”‚    - New UUID (DCM-assigned)
+  โ”‚    - ingestion_source: brownfield_discovery
+  โ”‚    - State: INGESTED
+  โ”‚    - Tenant: __transitional__ (pending enrichment)
+  โ”‚    - ingestion_record linking to Discovered State UUID
+  โ”‚
+  โ–ผ  Enrichment
+  โ”‚  Business data associated (owner, cost center, purpose)
+  โ”‚  Tenant assigned based on auto-assignment signals
+  โ”‚  Relationships established to other entities
+  โ”‚  Missing fields populated from discovery data
+  โ”‚
+  โ–ผ  Promotion
+  โ”‚  Review and authorization by responsible actor
+  โ”‚  State: ENRICHING โ†’ PROMOTED
+  โ”‚  DCM assumes lifecycle ownership:
+  โ”‚    - Discovered State record becomes the initial Realized State
+  โ”‚    - Entity enters standard DCM lifecycle (OPERATIONAL)
+  โ”‚    - Drift detection active from this point forward
+  โ”‚
+  โ–ผ  OPERATIONAL
+     DCM now manages the full lifecycle of this previously unmanaged entity
+```
+
+### 8.3 Discovered โ†’ Realized Promotion
+
+When a brownfield entity is promoted, its Discovered State record is promoted to become the initial Realized State. This is the moment DCM assumes lifecycle authority:
+
+```yaml
+realized_state_record:
+  entity_uuid: 
+  source: brownfield_promotion
+  ingestion_uuid: 
+  discovered_state_uuid: 
+  promoted_at: 
+  promoted_by:
+    display_name: 
+  initial_realized_payload: 
+  provenance:
+    origin:
+      source_type: brownfield_discovery
+      source_uuid: 
+      timestamp: 
+```
+
+From this point, the standard drift detection cycle runs: future discoveries are compared against the Realized State and any deviations are flagged as drift.
+
+---
+
+## 9. Relationship to the Four States
+
+Ingestion interacts with the Four States model as follows:
+
+| Ingestion Source | States Involved | Flow |
+|-----------------|----------------|------|
+| V1 Migration | Intent โ†’ Requested โ†’ (no Realized yet) | V1 records treated as incomplete Requested State; migration creates minimal Realized State |
+| Brownfield Discovery | Discovered โ†’ Realized | Discovered State is promoted to Realized State at promotion |
+| Manual Import | None initially | Entity stub created; no prior state records; Realized State created at promotion from import data |
+
+In all cases: once an entity reaches `PROMOTED`, it has a Realized State record and full Four States tracking begins.
+
+---
+
+## 10. DCM System Policies โ€” Full List
+
+| Policy | Rule |
+|--------|------|
+| `ING-001` | Every entity ingested into DCM must be assigned to exactly one Tenant โ€” either a real Tenant or `__transitional__` โ€” before it is eligible for new requests |
+| `ING-002` | Entities in `INGESTED` or `ENRICHING` state may not be the parent resource for a new allocated resource claim |
+| `ING-003` | The `__transitional__` Tenant is system-managed โ€” cannot be deleted, renamed, or used for new resource provisioning |
+| `ING-004` | Every ingested entity must carry an `ingestion_record` in its provenance chain |
+| `ING-005` | Entities in `__transitional__` beyond `max_residency_days` must trigger the configured escalation action |
+| `ING-006` | A brownfield entity may not be promoted to `PROMOTED` state without explicit actor authorization |
+| `ING-007` | At promotion, the Discovered State record must be promoted to Realized State โ€” this is the moment DCM assumes lifecycle ownership |
+
+---
+
+## 11. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should the auto-assignment signal priority order be configurable per deployment? | Migration flexibility | โœ… Resolved โ€” platform domain layer declares priority; explicit_tenant_tag fixed first; default_tenant fixed last; middle signals configurable (ING-012) |
+| 2 | Can multiple entities be promoted in bulk? | Operational efficiency | โœ… Resolved โ€” bulk promotion supported; profile-governed max batch sizes; preview required; PT24H rollback; BULK_PROMOTE audit (ING-013) |
+| 3 | Should there be a maximum number of ingestion sources per entity? | Data integrity | โœ… Resolved โ€” profile-governed max (5 standard/prod, 3 fsi/sovereign); warn or reject on exceed (ING-014) |
+| 4 | How does ingestion interact with the Service Catalog? | Catalog model | โœ… Resolved โ€” ingested entities promotable to catalog items; bidirectional drift detection (ING-015) |
+
+---
+
+## 12. Related Concepts
+
+- **`__transitional__` Tenant** โ€” system-managed holding Tenant for unassigned ingested entities
+- **Ingestion Record** โ€” provenance record carried by every ingested entity
+- **Four States** โ€” Discovered State is the entry point for brownfield ingestion; Realized State is the output of promotion
+- **Brownfield** โ€” existing infrastructure not yet under DCM lifecycle management
+- **Drift Detection** โ€” begins for brownfield entities at the moment of promotion
+- **V1 Migration** โ€” migration of pre-Tenant DCM V1 entities to V2 using the ingestion model
+- **Greening the Brownfield** โ€” the progressive process of bringing unmanaged infrastructure under DCM lifecycle control
+
+
+## 8. Ingestion Gap Resolutions
+
+### 8.1 Configurable Signal Priority Order (Q1)
+
+The ingestion signal priority order is declared in a platform-domain layer and configurable per deployment. `explicit_tenant_tag` always has highest priority; `default_tenant` always has lowest. The middle signals may be reordered.
+
+```yaml
+layer:
+  handle: "platform/ingestion/signal-priority"
+  domain: platform
+  fields:
+    ingestion_signal_priority:
+      - explicit_tenant_tag         # fixed: always first
+      - provider_declared_tenant    # configurable order
+      - network_segment_mapping     # configurable order
+      - hardware_class_mapping      # configurable order
+      - geographic_location         # configurable order
+      - default_tenant              # fixed: always last
+```
+
+### 8.2 Bulk Entity Promotion (Q2)
+
+Bulk promotion is supported with profile-governed limits and approval requirements.
+
+```yaml
+bulk_promotion_config:
+  max_entities_per_bulk: 500        # configurable per profile
+  requires_approval: true
+  preview_required: true            # must review bulk preview before confirming
+  rollback_window: PT24H
+  audit_record: BULK_PROMOTE        # single audit event with full member list
+```
+
+| Profile | Max per Bulk | Approval Required |
+|---------|-------------|-----------------|
+| minimal | Unlimited | No |
+| dev | 1000 | No |
+| standard | 500 | Recommended |
+| prod | 100 | Yes |
+| fsi | 50 | Yes + dual approval |
+| sovereign | 25 | Yes + dual approval |
+
+### 8.3 Maximum Ingestion Sources per Entity (Q3)
+
+Profile-governed maximum to encourage clear data ownership and manageable conflict resolution.
+
+```yaml
+ingestion_source_limits:
+  max_sources_per_entity: 5         # default for standard/prod
+  on_max_exceeded: 
+  profile_defaults:
+    minimal: unlimited
+    dev: 10
+    standard: 5
+    prod: 5
+    fsi: 3
+    sovereign: 3
+```
+
+### 8.4 Ingestion to Service Catalog Promotion (Q4)
+
+Ingested entities may be promoted to Service Catalog items โ€” the pathway from brownfield discovery to catalog-driven management.
+
+```
+Ingested entity
+  โ†’ Operator associates entity with Resource Type Specification
+  โ†’ Fields validated against spec
+  โ†’ Service Catalog item created from entity's configuration
+  โ†’ Entity becomes template ("golden example") for this catalog item
+  โ†’ Future requests use catalog item
+  โ†’ Drift detection bidirectional:
+      entity drifts from catalog item โ†’ drift event
+      catalog item updated โ†’ entity flagged for review
+```
+
+### 8.5 System Policies โ€” Ingestion Gaps
+
+| Policy | Rule |
+|--------|------|
+| `ING-012` | Ingestion signal priority order is declared in a platform domain layer and configurable per deployment. explicit_tenant_tag always has highest priority. default_tenant always has lowest priority. Middle signals are reorderable. |
+| `ING-013` | Bulk entity promotion is supported with profile-governed maximum batch sizes and approval requirements. Preview required before confirmation. Rollback window PT24H. Single BULK_PROMOTE audit record with full member list. |
+| `ING-014` | Maximum ingestion sources per entity is profile-governed (default: 5 for standard/prod; 3 for fsi/sovereign). Exceeding the maximum triggers warn or reject per policy. |
+| `ING-015` | Ingested entities may be associated with Resource Type Specifications and promoted to Service Catalog items. Drift detection operates bidirectionally between the ingested entity and its associated catalog item. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/14-policy-profiles.md b/content/docs/data-model/14-policy-profiles.md
new file mode 100644
index 0000000..d6af6b9
--- /dev/null
+++ b/content/docs/data-model/14-policy-profiles.md
@@ -0,0 +1,1653 @@
+# DCM Data Model โ€” Policy Organization: Groups, Profiles, and Policy Providers
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+> **Universal Group Model:** Policy Groups (`group_class: policy_collection`) and Policy Profiles (`group_class: policy_profile`) are expressions of the [Universal Group Model](15-universal-groups.md). The structures defined in this document remain authoritative for policy-specific behavior; the universal model adds composability, cross-type membership, and the ability to include policy groups within composite groups.
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Scoring Model](29-scoring-model.md) | [Context and Purpose](00-context-and-purpose.md) | [Data Layers and Assembly](03-layering-and-versioning.md) | [Entity Relationships](09-entity-relationships.md) | [Storage Providers](11-storage-providers.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + POLICY**
+>
+> Data: Policy Group and Profile artifacts. Policy: concern types and composition
+
+
+
+---
+
+> **Standards and Compliance Reference:** See [Standards and Compliance Catalog](40-standards-catalog.md) for the complete mapping of compliance frameworks (HIPAA, PCI DSS, FedRAMP, NIST SP 800-53, GDPR, DoD IL4) to DCM profiles and system requirements.
+
+## 1. Purpose
+
+DCM's Policy Engine is powerful โ€” but power without usability is a barrier to adoption. This document defines the **policy organization model**: the structures that make DCM easy to configure correctly for any use case, from a home lab evaluation to a sovereign financial services deployment.
+
+Three concepts work together:
+
+- **Policy Groups** โ€” cohesive collections of policies addressing a single identifiable concern (a technology, a compliance standard, a sovereignty requirement, a business process)
+- **Policy Profiles** โ€” complete DCM configurations for a specific use case, composed of Policy Groups
+- **Policy Providers** โ€” external authoritative sources that supply policies directly into DCM, extending the provider model to its fifth type
+
+The relationship is compositional:
+
+```
+Policy Profile     โ€” complete use-case configuration
+  โ”‚  composed of
+  โ–ผ
+Policy Groups      โ€” single-concern policy collections
+  โ”‚  composed of
+  โ–ผ
+Policies           โ€” individual Transformation / Validation / GateKeeper rules
+  โ”‚  optionally sourced from
+  โ–ผ
+Policy Providers   โ€” external authoritative policy sources
+```
+
+---
+
+## 1a. Design Priority Order in Policy Profiles
+
+Profiles implement the DCM design priority order (see [Foundational Abstractions](00-foundations.md)):
+
+1. **Security:** Profile defaults implement security correctly. Lower profiles have less strict enforcement โ€” not absent security.
+2. **Ease of use:** Profile defaults minimize configuration burden. `standard` profile should work for most deployments without customization.
+3. **Extensibility:** Profiles compose with compliance domain overlays. Organizations add compliance requirements additively without rewriting base configuration.
+4. **Fit for purpose:** Every profile must support the complete DCM lifecycle.
+
+**The `minimal` profile is not "security optional"** โ€” it is the security model with minimal operational overhead. All security properties are present; thresholds and automation levels are relaxed.
+
+---
+
+## 1b. Policy Authorship โ€” Federated Contribution Model
+
+Policies in DCM are not exclusively authored by platform admins. The DCM federated contribution model enables all actor types to author policies within their permitted domain scope:
+
+- **Platform admins** โ€” all domains, all policy types
+- **Consumers / Tenant admins** โ€” tenant domain policies (GateKeeper, Transformation, Recovery, Lifecycle, Orchestration Flow, Governance Matrix rules scoped to their Tenant)
+- **Service Providers** โ€” provider-domain GateKeeper and Validation policies for their resource types
+- **Peer DCM instances** โ€” policy templates contributed through verified federation relationships
+
+This is not a special case โ€” it is the standard GitOps PR model applied to all contributor types. Consumer-authored policies go through the same lifecycle (developing โ†’ proposed โ†’ active) with appropriate review requirements per the active profile. See [Federated Contribution Model](28-federated-contribution-model.md) for the complete specification.
+
+---
+
+## 2. Policy Groups
+## 1a. Two-Dimensional Profile Model
+
+
+### 1a.0 Profile โ€” One Posture, Multiple Compliance Domains
+
+**A DCM deployment runs exactly one Deployment Posture and zero or more Compliance Domain Groups simultaneously.** This is the complete profile model. No new concept is needed.
+
+```
+Active DCM Governance = one Deployment Posture + [zero or more Compliance Domains]
+
+Examples:
+  prod + hipaa                         โ† healthcare production
+  prod + hipaa + gdpr                  โ† EU healthcare production  
+  sovereign + fedramp-high + dod-il5   โ† classified federal
+  standard                             โ† general enterprise, no compliance overlay
+  dev + hipaa                          โ† healthcare development (hipaa policies active,
+                                          but operational cost reduced by dev posture)
+```
+
+**Deployment Postures are mutually exclusive** โ€” you cannot be both `prod` and `dev`. One posture governs the operational characteristics of the entire DCM deployment.
+
+**Compliance domains are additive** โ€” HIPAA + FSI is valid and common. Each compliance domain group adds its own set of policies and constraints on top of the posture. They do not conflict with each other at the domain level (they govern different aspects of data handling); they may produce policy conflicts at the field level, which are resolved through the standard policy conflict resolution process.
+
+**The dev posture and compliance domains** โ€” applying `dev` posture to a HIPAA-scoped deployment does not remove HIPAA obligations. It relaxes the *operational cost* of meeting them: less redundancy, shorter retention windows where permitted, advisory enforcement where HIPAA allows flexibility. The HIPAA compliance domain group remains active and its mandatory controls remain enforced.
+
+**Modules vs Profiles** โ€” DCM uses both concepts with distinct meanings:
+- A **Profile** is a governance configuration: it declares how DCM behaves and what operational and compliance requirements apply.
+- A **Module** is a capability extension: it adds new functions to DCM (e.g., a HIPAA record validator, a custom resource type). Modules are not profiles and do not configure DCM behavior โ€” they extend what DCM can do.
+
+
+### 1a.1 The Gap in the Original Model
+
+The original six profiles (minimal โ†’ sovereign) are organized around **deployment posture** โ€” how strict, how redundant, how governed. But organizations need compliance governance that is orthogonal to posture. A healthcare organization needs HIPAA controls regardless of whether they deploy at `standard` or `sovereign` posture. A payment processor needs PCI-DSS regardless of their redundancy profile.
+
+The new model makes compliance a first-class dimension that composes with posture:
+
+```
+Complete Profile = Deployment Posture Group + Compliance Domain Group(s)
+```
+
+### 1a.2 Dimension 1 โ€” Deployment Posture Groups
+
+Posture groups govern how the DCM infrastructure itself behaves โ€” redundancy, enforcement strictness, audit retention, tenancy model, cross-tenant defaults. These are the vertical axis from least to most governed.
+
+| Group Handle | Posture | Key Behaviors |
+|-------------|---------|--------------|
+| `system/group/posture-minimal` | Minimal | Advisory only; single instance; no redundancy |
+| `system/group/posture-dev` | Development | Warn-not-block; basic logging; ephemeral defaults |
+| `system/group/posture-standard` | Standard | Full enforcement; 3-replica; explicit cross-tenant |
+| `system/group/posture-prod` | Production | Full enforcement + SLA; geo-replicated; cost governance |
+| `system/group/posture-hardened` | Enterprise Hardened | 5-replica; 7-year audit; hard tenancy; dual approval |
+| `system/group/posture-sovereign` | Sovereign | Air-gap; deny_all cross-tenant; 10-year audit; signed bundles |
+
+### 1a.3 Dimension 2 โ€” Compliance Domain Groups
+
+Compliance domain groups govern what data handling, audit, and control requirements apply to resources managed by DCM. Multiple compliance domains can apply simultaneously.
+
+| Group Handle | Domain | Key Controls |
+|-------------|--------|-------------|
+| `system/group/compliance-fsi` | Financial Services | Basel III, SOX, Dodd-Frank financial controls |
+| `system/group/compliance-pci-dss` | Payment Card | PCI-DSS v4 โ€” payment card industry |
+| `system/group/compliance-hipaa` | Healthcare | HIPAA/HITECH PHI handling and audit |
+| `system/group/compliance-fedramp-moderate` | US Federal Moderate | FedRAMP Moderate โ€” NIST 800-53 Moderate baseline |
+| `system/group/compliance-fedramp-high` | US Federal High | FedRAMP High โ€” NIST 800-53 High baseline |
+| `system/group/compliance-dod-il2` | DoD IL2 | Public/unclassified defense data |
+| `system/group/compliance-dod-il4` | DoD IL4 | Controlled Unclassified Information (CUI) |
+| `system/group/compliance-dod-il5` | DoD IL5 | National Security Systems non-classified |
+| `system/group/compliance-dod-il6` | DoD IL6 | Classified โ€” maximum sovereign posture |
+| `system/group/compliance-government` | Government General | General government/public sector controls |
+| `system/group/compliance-gdpr` | EU Data Protection | GDPR data residency and rights |
+| `system/group/compliance-iso27001` | ISO 27001 | Information security management |
+| `system/group/compliance-nist-800-53` | NIST 800-53 | NIST security control framework |
+| `system/group/compliance-soc2` | SOC 2 | Service organization controls |
+| `system/group/compliance-nerc-cip` | Critical Infrastructure | Energy/utilities NERC CIP |
+| `system/group/compliance-sovereign` | Sovereign/Classified | Sovereign deployment, air-gap, classified |
+
+### 1a.4 Compliance Domain Group Contents
+
+#### `system/group/compliance-hipaa`
+
+HIPAA/HITECH controls for Protected Health Information (PHI):
+- PHI field classification enforcement (fields containing PHI must be tagged `phi: true`)
+- PHI access control (only roles with `phi_authorized: true` may access PHI-tagged fields)
+- Audit retention: P6Y minimum (HIPAA requires 6 years from creation or last use)
+- Encryption at rest: AES-256 required for all PHI storage
+- Transmission security: TLS 1.3 minimum for PHI in transit
+- Breach notification workflow: sovereignty_violation_record triggers HIPAA breach assessment
+- Business Associate Agreement (BAA) tracking: providers handling PHI must declare `baa_in_place: true` in sovereignty_declaration
+- Right to Access: consumer data export capability required for PHI entities
+- Minimum Necessary standard: data_request_spec on Mode 4 providers limited to minimum PHI fields
+
+#### `system/group/compliance-government`
+
+General government and public sector controls:
+- Data classification mandatory on all resources (classification_level field required)
+- Cross-boundary controls: data cannot cross classification levels without explicit policy
+- Audit retention: P10Y minimum
+- Air-gap capability required for Sensitive compartments
+- Actor authentication must declare clearance level in external_identity claims
+- All provider sovereignty_declarations must declare government_access_risk
+
+#### `system/group/compliance-fedramp-moderate`
+
+FedRAMP Moderate authorization controls:
+- NIST 800-53 Rev 5 Moderate baseline implemented as policy group
+- FedRAMP-authorized provider preference injected as placement constraint
+- Continuous monitoring: drift detection mandatory; P24H maximum drift resolution window
+- Incident response: webhook required for security events (audit.chain_break, drift.escalated)
+- POA&M tracking: `poam_status` field in status_metadata on all policy artifacts
+- Boundary protection: explicit ingress/egress control documentation
+
+#### `system/group/compliance-dod-il4`
+
+DoD Impact Level 4 โ€” Controlled Unclassified Information:
+- Inherits compliance-fedramp-moderate
+- CUI handling markers on all data fields containing controlled information
+- Sovereign posture within US jurisdiction boundary
+- Provider sovereignty_declaration must exclude foreign sub-processors
+- CMMC Level 2 cyber hygiene controls
+
+#### `system/group/compliance-sovereign`
+
+Sovereign and classified deployment controls:
+- All data must remain within declared sovereignty boundary
+- Air-gap capability: provider air_gap_capable: true required
+- Signed bundle import only โ€” no live registry connectivity
+- deny_all cross-tenant cross-boundary data flows
+- Hardware security module (HSM) required for key management
+- Audit records: 10-year retention; cryptographic signing required
+
+### 1a.5 Profile Composition Model
+
+The six built-in profiles become explicit posture+compliance compositions:
+
+```yaml
+system/profile/minimal:
+  policy_groups: [system/group/posture-minimal]
+
+system/profile/dev:
+  extends: system/profile/minimal
+  policy_groups: [system/group/posture-dev]
+
+system/profile/standard:
+  extends: system/profile/dev
+  policy_groups: [system/group/posture-standard]
+
+system/profile/prod:
+  extends: system/profile/standard
+  policy_groups: [system/group/posture-prod]
+
+system/profile/fsi:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/posture-hardened
+    - system/group/compliance-fsi
+    - system/group/compliance-pci-dss
+    - system/group/compliance-iso27001
+
+system/profile/sovereign:
+  extends: system/profile/fsi
+  policy_groups:
+    - system/group/posture-sovereign
+    - system/group/compliance-sovereign
+```
+
+### 1a.6 DCM Built-In Extended Profiles
+
+In addition to the six core profiles, DCM ships extended profiles for common compliance domains:
+
+```yaml
+system/profile/hipaa-prod:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/compliance-hipaa
+    - system/group/compliance-iso27001
+  description: "Production infrastructure with HIPAA/HITECH compliance"
+
+system/profile/hipaa-sovereign:
+  extends: system/profile/sovereign
+  policy_groups:
+    - system/group/compliance-hipaa
+  description: "Sovereign deployment with HIPAA/HITECH compliance โ€” highest healthcare posture"
+
+system/profile/fedramp-moderate:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/compliance-fedramp-moderate
+    - system/group/compliance-nist-800-53
+  description: "FedRAMP Moderate authorized deployment"
+
+system/profile/fedramp-high:
+  extends: system/profile/sovereign
+  policy_groups:
+    - system/group/compliance-fedramp-high
+    - system/group/compliance-nist-800-53
+  description: "FedRAMP High authorized deployment"
+
+system/profile/government:
+  extends: system/profile/prod
+  policy_groups:
+    - system/group/compliance-government
+    - system/group/compliance-nist-800-53
+  description: "General government and public sector deployment"
+
+system/profile/dod-il4:
+  extends: system/profile/sovereign
+  policy_groups:
+    - system/group/compliance-dod-il4
+    - system/group/compliance-fedramp-high
+    - system/group/compliance-nist-800-53
+  description: "DoD Impact Level 4 โ€” Controlled Unclassified Information"
+
+system/profile/dod-il5:
+  extends: system/profile/dod-il4
+  policy_groups:
+    - system/group/compliance-dod-il5
+  description: "DoD Impact Level 5 โ€” National Security Systems non-classified"
+
+system/profile/dod-il6:
+  extends: system/profile/dod-il5
+  policy_groups:
+    - system/group/compliance-dod-il6
+    - system/group/compliance-sovereign
+  description: "DoD Impact Level 6 โ€” Classified"
+```
+
+### 1a.7 Organization Custom Profiles โ€” Composition Examples
+
+```yaml
+# Healthcare with federal cloud authorization
+org/profile/hipaa-fedramp:
+  extends: system/profile/fedramp-moderate
+  policy_groups:
+    - system/group/compliance-hipaa
+  description: "Healthcare workloads on FedRAMP Moderate platform"
+
+# Multi-compliance financial + healthcare
+org/profile/fsi-hipaa:
+  extends: system/profile/fsi
+  policy_groups:
+    - system/group/compliance-hipaa
+  description: "FSI-grade deployment for organizations managing both financial and health data"
+
+# Tenant-level compliance override โ€” platform is standard; this Tenant is PCI-scoped
+# (declared in tenant_config โ€” not profile)
+tenant_compliance_overlay:
+  active_profile: system/profile/standard    # platform posture
+  compliance_groups:
+    - system/group/compliance-pci-dss        # this Tenant handles payment cards
+    - system/group/compliance-hipaa          # this Tenant also handles PHI
+  # Other Tenants on same platform have standard posture, no compliance overlay
+```
+
+### 1a.8 Compliance at Tenant Level
+
+Compliance domain groups may apply at Tenant level โ€” different Tenants on the same DCM deployment can have different compliance domains:
+
+```yaml
+tenant_config:
+  active_profile: system/profile/prod        # posture from platform
+  compliance_groups:
+    - system/group/compliance-hipaa          # this Tenant handles PHI
+    - system/group/compliance-pci-dss        # this Tenant processes payments
+```
+
+This is the critical capability: **one DCM deployment, multiple compliance postures per Tenant**. A hospital system can run a single DCM with: clinical Tenants (HIPAA), billing Tenants (HIPAA + PCI-DSS), and administrative Tenants (standard) โ€” all on the same platform profile.
+
+### 1a.9 System Policies โ€” Profile Composition
+
+| Policy | Rule |
+|--------|------|
+| `PROF-001` | Profiles compose a Deployment Posture Group with zero or more Compliance Domain Groups. Posture groups govern DCM infrastructure behavior. Compliance domain groups govern data handling, audit, and control requirements. |
+| `PROF-002` | Compliance Domain Groups may be applied at platform level (all Tenants) or Tenant level (specific Tenants). Tenant-level compliance groups are additive โ€” they do not replace platform-level groups. |
+| `PROF-003` | DCM ships built-in Compliance Domain Groups for: FSI, PCI-DSS, HIPAA/HITECH, FedRAMP Moderate, FedRAMP High, DoD IL2-IL6, Government, GDPR, ISO 27001, NIST 800-53, SOC2, NERC-CIP, and Sovereign/Classified. Organizations extend these groups or compose them into custom profiles. |
+| `PROF-004` | The `implementation_posture` concern_type Policy Groups (provenance model, auth simplicity, deployment complexity) are independent of compliance domain โ€” organizations select their implementation posture separately from their compliance requirements. |
+
+---
+
+
+### 2.1 Definition
+
+A **Policy Group** is a versioned, cohesive collection of policies that together address a **single identifiable concern**. The group is the unit of reuse โ€” activate a group to enable a concern, not individual policies.
+
+```yaml
+policy_group:
+  artifact_metadata:
+    uuid: 
+    handle: "system/group/pci-dss"
+    version: "1.2.0"
+    status: active
+    owned_by:
+      display_name: "DCM Project Team"
+      notification_endpoint: 
+
+  name: "PCI-DSS v4"
+  description: >
+    Policy group implementing PCI-DSS v4 controls relevant to
+    DCM-managed infrastructure. Enforces encryption standards,
+    network segmentation, access control, and audit requirements
+    for resources in PCI scope.
+
+  concern_type: compliance
+  concern_tags: [pci-dss, financial, encryption, network-segmentation]
+  extends: null   # or another group handle โ€” inherits all parent policies
+
+  # Source โ€” locally authored or from a Policy Provider
+  source:
+    type: 
+    provider_uuid: 
+    provider_group_reference: 
+    on_provider_update: 
+    # proposed: provider updates require local review before activation
+    # active:   provider updates activate immediately (trusted providers only)
+
+  # Constituent policies
+  policies:
+    - policy_uuid: 
+      handle: "system/gatekeeper/pci-encryption-aes256"
+      description: "Enforce AES-256 on all PCI-scoped storage"
+      placement_phase: pre
+    - policy_uuid: 
+      handle: "system/validation/pci-network-segmentation"
+      description: "Validate network segment isolation for PCI resources"
+      placement_phase: pre
+    - policy_uuid: 
+      handle: "system/transformation/pci-classification-inject"
+      description: "Auto-inject PCI classification on scoped resources"
+      placement_phase: pre
+    - policy_uuid: 
+      handle: "system/gatekeeper/pci-audit-retention"
+      description: "Enforce 10-year audit retention for PCI evidence"
+      placement_phase: pre
+
+  # Activation scope โ€” surgical application within a profile
+  activation_scope:
+    resource_types: []          # empty = all resource types
+    tenant_tags: [pci-scope]    # only Tenants tagged pci-scope
+    regions: []                 # empty = all regions
+
+  # Conflict declarations
+  conflicts_with:
+    - group_handle: "system/group/dev-defaults"
+      reason: "PCI requires blocking enforcement; dev-defaults uses warn-only"
+      resolution: this_group_wins
+```
+
+### 2.2 Concern Types
+
+| Type | Description | Examples |
+|------|-------------|---------|
+| `technology` | Policies specific to a technology or provider | kubevirt, openstack, kubernetes, vmware |
+| `compliance` | Regulatory or standards compliance | pci-dss, iso-27001, nist-800-53, fedramp |
+| `sovereignty` | Data residency, jurisdictional, air-gap | gdpr-eu, air-gap, data-residency-uk |
+| `business` | Business process, cost, lifecycle governance | cost-governance, ephemeral-resources, chargeback |
+| `operational` | Operational posture, defaults, SLAs | dev-defaults, hard-tenancy, sla-enforcement |
+| `security` | Security controls and posture | data-classification, zero-trust, encryption-baseline |
+| `implementation_posture` | Implementation complexity vs capability trade-offs | provenance-full-inline, provenance-deduplicated, single-instance-deployment, advisory-policies-only |
+
+### 2.3 Group Inheritance
+
+A Policy Group may extend another group โ€” inheriting all its policies and overriding or adding to them:
+
+```yaml
+policy_group:
+  handle: "org/group/pci-dss-extended"
+  extends: "system/group/pci-dss"
+  # Inherits all system/group/pci-dss policies
+  # Additional policies added below are on top of the parent
+  policies:
+    - policy_uuid: 
+      handle: "org/gatekeeper/our-pci-additional-control"
+```
+
+### 2.4 DCM Built-In Policy Groups
+
+DCM ships the following policy groups as part of its standard distribution:
+
+| Handle | Concern | Description |
+|--------|---------|-------------|
+| `system/group/core-minimal` | operational | Absolute minimum โ€” UUID requirements, basic well-formedness |
+| `system/group/dev-defaults` | operational | Warn-not-block, 90-day TTL defaults, single-auth cross-tenant |
+| `system/group/ephemeral-resources` | business | TTL enforcement, auto-expiry, short-lived resource defaults |
+| `system/group/audit-basic` | operational | Basic audit logging, 90-day retention |
+| `system/group/audit-compliance` | compliance | Compliance-grade audit, configurable retention |
+| `system/group/data-classification` | security | Data classification tagging and handling rules |
+| `system/group/cost-governance` | business | Budget enforcement, cost attribution, TTL governance |
+| `system/group/sla-enforcement` | operational | SLA tracking, availability commitments |
+| `system/group/hard-tenancy` | operational | Full tenant isolation, deny_all cross-tenant default |
+| `system/group/explicit-cross-tenant` | operational | Explicit cross-tenant authorization requirement (XTA-001 through XTA-005) |
+| `system/group/zero-trust` | security | Zero-trust network and identity enforcement |
+| `system/group/encryption-baseline` | security | AES-256 minimum, TLS 1.3, key rotation |
+| `system/group/pci-dss` | compliance | PCI-DSS v4 controls |
+| `system/group/gdpr-eu` | sovereignty | GDPR data residency and handling |
+| `system/group/nist-800-53` | compliance | NIST 800-53 control implementation |
+| `system/group/iso-27001` | compliance | ISO 27001 controls |
+| `system/group/fedramp-moderate` | compliance | FedRAMP Moderate baseline |
+| `system/group/air-gap` | sovereignty | Air-gapped deployment constraints |
+| `system/group/fsi-audit` | compliance | Financial services audit retention (7-year minimum) |
+| `system/group/lifecycle-ttl-enforcement` | operational | Lifecycle time constraint enforcement (LTC-001 through LTC-004) |
+| `system/group/kubevirt` | technology | KubeVirt provider-specific policies |
+| `system/group/openstack` | technology | OpenStack provider-specific policies |
+| `system/group/vmware` | technology | VMware provider-specific policies |
+| `system/group/provenance-full-inline` | implementation_posture | Model A โ€” all provenance inline on entity records; simplest; highest storage cost |
+| `system/group/provenance-deduplicated` | implementation_posture | Model B โ€” content-addressed deduplication; recommended; 95-99% storage reduction; lossless |
+| `system/group/provenance-tiered-archive` | implementation_posture | Model C โ€” hot/warm/cold tiers; balances cost and access speed |
+| `system/group/provenance-deduplicated-tiered` | implementation_posture | Model B+C โ€” maximum efficiency for very large-scale deployments |
+
+---
+
+## 3. Policy Profiles
+
+### 3.1 Definition
+
+A **Policy Profile** is a named, versioned, curated composition of Policy Groups that together configure DCM for a specific use case. Activating a profile is the primary configuration mechanism โ€” most deployments should activate a profile and then add organization-specific groups on top rather than configuring policies individually.
+
+```yaml
+policy_profile:
+  artifact_metadata:
+    uuid: 
+    handle: "system/profile/fsi"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "DCM Project Team"
+
+  name: "FSI Production"
+  description: >
+    Policy profile for Financial Services production deployments.
+    Enforces hard tenancy, regulatory-grade audit retention,
+    full sovereignty controls, and explicit cross-tenant authorization.
+
+  target_use_case: fsi_production
+  extends: "system/profile/prod"   # inherits all prod groups + overrides
+
+  enforcement_summary:
+    tenancy: hard_tenancy_required
+    audit_retention_years: 7
+    sovereignty: full
+    cross_tenant_default: explicit_only
+    policy_enforcement: blocking
+    immutable_fields: [sovereignty_zone, classification_level, audit_retention]
+    policy_version_pinning: permitted_with_elevation
+
+  policy_groups:
+    - group_handle: "system/group/fsi-audit"
+    - group_handle: "system/group/pci-dss"
+    - group_handle: "system/group/gdpr-eu"
+    - group_handle: "system/group/hard-tenancy"
+    - group_handle: "system/group/explicit-cross-tenant"
+    - group_handle: "system/group/encryption-baseline"
+```
+
+### 3.2 DCM Built-In Profiles
+
+DCM ships six profiles covering the spectrum from minimal to sovereign:
+
+#### `system/profile/minimal` โ€” Home Lab / Evaluation
+
+```yaml
+handle: "system/profile/minimal"
+name: "Minimal"
+extends: null
+description: >
+  Minimal configuration for home lab, local testing, and evaluation.
+  Most controls advisory only. Single Tenant auto-created on first use.
+  No audit requirements. No sovereignty enforcement.
+
+enforcement_summary:
+  tenancy: optional
+  audit_retention: none
+  sovereignty: none
+  cross_tenant_default: allow_all
+  policy_enforcement: advisory     # policies warn but do not block
+  time_constraints: optional
+
+auto_tenant:
+  enabled: true
+  default_tenant_handle: "default"
+  # TEN-001 satisfied silently โ€” no explicit Tenant declaration required
+
+policy_groups:
+  - group_handle: "system/group/core-minimal"
+```
+
+#### `system/profile/dev` โ€” Development Environments
+
+```yaml
+handle: "system/profile/dev"
+name: "Development"
+extends: "system/profile/minimal"
+description: >
+  Development environment profile. Tenancy recommended but not blocking.
+  Basic logging. Ephemeral resource defaults. Warn-not-block enforcement.
+
+enforcement_summary:
+  tenancy: recommended
+  audit_retention: basic
+  sovereignty: none
+  cross_tenant_default: operational_only
+  policy_enforcement: warn_only
+  default_ttl: P90D              # dev resources default to 90-day TTL
+
+policy_groups:
+  - group_handle: "system/group/dev-defaults"
+  - group_handle: "system/group/ephemeral-resources"
+  - group_handle: "system/group/audit-basic"
+```
+
+#### `system/profile/standard` โ€” General Enterprise Production
+
+```yaml
+handle: "system/profile/standard"
+name: "Standard"
+extends: "system/profile/dev"
+description: >
+  General enterprise production profile. Full policy enforcement.
+  Tenancy required. Explicit cross-tenant authorization. Basic
+  data classification. Compliance-grade audit.
+
+enforcement_summary:
+  tenancy: required
+  audit_retention: compliance_grade
+  sovereignty: configurable
+  cross_tenant_default: explicit_only
+  policy_enforcement: blocking
+
+policy_groups:
+  - group_handle: "system/group/data-classification"
+  - group_handle: "system/group/audit-compliance"
+  - group_handle: "system/group/explicit-cross-tenant"
+  - group_handle: "system/group/encryption-baseline"
+```
+
+#### `system/profile/prod` โ€” Production with SLA Requirements
+
+```yaml
+handle: "system/profile/prod"
+name: "Production"
+extends: "system/profile/standard"
+description: >
+  Production profile with SLA enforcement, cost governance, and
+  full lifecycle constraint enforcement.
+
+enforcement_summary:
+  tenancy: required
+  audit_retention: compliance_grade
+  sovereignty: configurable
+  cross_tenant_default: explicit_only
+  policy_enforcement: blocking
+
+policy_groups:
+  - group_handle: "system/group/cost-governance"
+  - group_handle: "system/group/sla-enforcement"
+  - group_handle: "system/group/lifecycle-ttl-enforcement"
+```
+
+#### `system/profile/fsi` โ€” Financial Services Production
+
+```yaml
+handle: "system/profile/fsi"
+name: "FSI Production"
+extends: "system/profile/prod"
+description: >
+  Financial services production. Hard tenancy. 7-year audit retention.
+  Full sovereignty enforcement. PCI-DSS and GDPR compliance.
+
+enforcement_summary:
+  tenancy: hard_tenancy_required
+  audit_retention_years: 7
+  sovereignty: full
+  cross_tenant_default: explicit_only
+  policy_enforcement: blocking
+
+policy_groups:
+  - group_handle: "system/group/fsi-audit"
+  - group_handle: "system/group/pci-dss"
+  - group_handle: "system/group/gdpr-eu"
+  - group_handle: "system/group/hard-tenancy"
+```
+
+#### `system/profile/sovereign` โ€” Air-Gapped / Sovereign Deployments
+
+```yaml
+handle: "system/profile/sovereign"
+name: "Sovereign"
+extends: "system/profile/fsi"
+description: >
+  Maximum control profile for air-gapped, sovereign, and highest-security
+  deployments. Complete tenant isolation. Zero external dependencies.
+  Maximum audit and sovereignty enforcement.
+
+enforcement_summary:
+  tenancy: hard_tenancy_required
+  audit_retention_years: 10
+  sovereignty: maximum
+  cross_tenant_default: deny_all
+  policy_enforcement: blocking
+  immutable_ceiling_on_all_sovereignty_fields: true
+
+policy_groups:
+  - group_handle: "system/group/air-gap"
+  - group_handle: "system/group/zero-trust"
+```
+
+### 3.3 Profile Inheritance Chain
+
+```
+system/profile/sovereign
+  extends: system/profile/fsi
+    extends: system/profile/prod
+      extends: system/profile/standard
+        extends: system/profile/dev
+          extends: system/profile/minimal
+            extends: null (base)
+```
+
+Each level adds groups without replacing parent groups. An organization extending a DCM profile only needs to declare what differs:
+
+```yaml
+# Organization custom profile
+org/profile/my-prod:
+  extends: system/profile/prod
+  policy_groups:
+    - group_handle: "org/group/our-naming-conventions"
+    - group_handle: "org/group/our-cost-centers"
+    - group_handle: "system/group/iso-27001"    # add a DCM compliance group
+```
+
+### 3.4 Profile Activation
+
+Profiles activate at three levels โ€” more specific takes precedence:
+
+```yaml
+# DCM installation default
+installation_config:
+  default_profile: "system/profile/minimal"
+
+# Platform-level (applies to all Tenants)
+platform_config:
+  active_profile: "system/profile/prod"
+  minimum_tenant_profile: "system/profile/dev"   # Tenants cannot go below this
+  maximum_tenant_profile: null                    # null = no ceiling
+
+# Tenant-level override
+tenant_config:
+  active_profile: "system/profile/fsi"           # must be >= minimum_tenant_profile
+```
+
+**A Tenant cannot activate a profile less restrictive than the platform minimum.** A sovereign deployment can set `minimum_tenant_profile: system/profile/sovereign` โ€” no Tenant can drop below that level.
+
+### 3.5 Profile Conflict Resolution
+
+When a profile is activated, DCM runs conflict detection across all constituent group policies โ€” same ingestion conflict detection that layers use. Conflicts must be resolved before a profile is marked `active`.
+
+Conflict resolution order:
+1. **Explicit `conflicts_with` declarations** on groups โ€” use declared resolution rule
+2. **Priority schema** โ€” higher numeric priority wins
+3. **Domain authority** โ€” `system` beats `platform` beats `tenant`
+4. **Unresolved** โ€” profile activation fails with detailed conflict report
+
+### 3.6 Profile Shadow Validation
+
+When a profile is in `proposed` status, all its constituent policies run in shadow mode โ€” the same proposed policy shadow execution model. The Validation Dashboard shows the aggregate impact across all policies in the profile before activation. This enables safe preview of what a profile upgrade would do to an existing deployment.
+
+---
+
+## 4. Policy Providers
+
+### 4.1 Policy Provider (one of eleven DCM provider types)
+
+A **Policy Provider** is a fifth DCM provider type โ€” an external authoritative source that supplies policies directly into DCM or evaluates and enriches DCM data through an external logic engine.
+
+| Type | Purpose | DCM Owns Result? |
+|------|---------|-----------------|
+| **Service Provider** | Realizes resources | Yes |
+| **Information Provider** | Serves authoritative external data | No |
+| **Meta Provider** | Composes multiple providers | Yes |
+| **Storage Provider** | Persists DCM state | Yes |
+| **Policy Provider** | Supplies policies from external authoritative sources, or evaluates and enriches data via external logic | Policies and enrichment data become DCM-owned on import |
+
+**Why Policy Providers?** Organizations should not need to manually translate regulatory controls, security benchmarks, or vendor-specific policies into DCM format. A Policy Provider is the authoritative source โ€” DCM subscribes to it and receives updates automatically. For Mode 4, the external system is the authoritative logic engine โ€” DCM queries it and acts on the result.
+
+### 4.2 The Four Policy Provider Delivery Modes
+
+| Mode | Name | How it works | Logic lives in |
+|------|------|-------------|---------------|
+| **Mode 1** | DCM Native Push/Pull | Provider delivers DCM-format policy artifacts | DCM Policy Engine |
+| **Mode 2** | OPA/Rego Bundle | Provider delivers OPA Rego bundles | DCM Policy Engine (OPA) |
+| **Mode 3** | External Schema | Provider delivers external-format policies requiring naturalization | DCM Policy Engine (post-translation) |
+| **Mode 4** | Black Box Query-Enrichment | DCM sends a query, provider evaluates and/or enriches, returns structured result | External provider โ€” logic is opaque to DCM |
+
+Modes 1-3 are **policy delivery** modes โ€” the provider sends rules, DCM stores and executes them. Mode 4 is fundamentally different: the policy logic lives in the external system. DCM sends data, receives a result, and acts on it.
+
+### 4.3 Policy Provider Contract โ€” Modes 1-3
+
+Policy Providers in Modes 1-3 follow the same base contract as all providers: registration, health check, trust, and provenance emission.
+
+```yaml
+policy_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/policy/grc-platform-001"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Security Team"
+      notification_endpoint: 
+
+  name: "Enterprise GRC Platform"
+  description: "Publishes compliance controls from GRC platform to DCM"
+
+  delivery:
+    mode: 
+    pull_schedule: "0 2 * * *"    # cron โ€” if mode: pull
+    endpoint: 
+
+  policy_format: 
+
+  managed_domains:
+    concern_types: [compliance, sovereignty]
+    concern_tags: [pci-dss, gdpr, iso-27001]
+
+  trust_level: 
+  max_policy_authority: gatekeeper
+
+  on_update: proposed
+  on_provider_failure:
+    action: 
+    sunset_days: 30
+```
+
+### 4.4 Policy Naturalization (Mode 3)
+
+When a Policy Provider delivers policies in an external format, DCM applies **Policy Naturalization** โ€” translating external policy schemas into DCM policy format.
+
+```
+External Policy Format (OSCAL, XACML, CIS JSON, STIG XCCDF)
+  โ”‚
+  โ–ผ  Policy Naturalization (translator component)
+  โ”‚
+  โ–ผ
+DCM Policy Format (standard DCM policy artifact)
+  โ”‚
+  โ–ผ  Trust validation + conflict detection
+  โ”‚
+  โ–ผ
+DCM Policy Engine
+```
+
+```yaml
+naturalization:
+  source_schema: oscal           # oscal | xccdf | cis-json | xacml | custom
+  translator_uuid: 
+  validation_on_import: strict   # strict | lenient
+```
+
+### 4.5 Policy Provider and Policy Groups
+
+A Modes 1-3 Policy Provider can deliver at three levels of granularity:
+
+- **Individual policies** โ€” administrator manually assigns to groups
+- **Complete Policy Groups** โ€” provider supplies group definitions alongside policies
+- **Complete Policy Profiles** โ€” provider supplies a full deployment profile for one-step activation
+
+```yaml
+policy_group:
+  handle: "org/group/grc-pci-dss"
+  source:
+    type: policy_provider
+    provider_uuid: 
+    provider_group_reference: "pci-dss-v4-full"
+    on_provider_update: proposed
+    last_synced: 
+    provider_version: "4.0.1"
+```
+
+### 4.6 Trust Levels and Policy Authority
+
+| Trust Level | Max Policy Authority | Requires | Use Case |
+|-------------|---------------------|----------|---------|
+| `trusted` | GateKeeper | Manual elevation + dual approval | Regulatory body, certified compliance package |
+| `verified` | Validation | Registration + health check | Security vendor, GRC platform |
+| `untrusted` | Advisory (logged but not executed) | Registration only | Evaluation, new providers |
+
+Trust elevation requires explicit authorization from both a platform admin and a security owner.
+
+### 4.7 Policy Provider Health and Lifecycle
+
+- **Healthy** โ€” policies current, delivery working normally
+- **Degraded** โ€” delivery delayed or partial โ€” warnings emitted
+- **Unhealthy** โ€” policies move toward `deprecated` per `on_provider_failure` declaration
+- **Deregistered** โ€” policies deprecated with configured sunset
+
+---
+
+### 4.8 Mode 4 โ€” Black Box Query-Enrichment
+
+#### 4.8.1 Concept
+
+A **Mode 4 Policy Provider** is an external system that DCM queries during the assembly process to evaluate request data, return a decision, enrich the payload with additional fields, or do both simultaneously.
+
+**The key distinction from Modes 1-3:** The policy logic lives in the external system and is opaque to DCM. DCM does not receive or store the rules โ€” it sends a query and receives a structured result. The external system is the authoritative evaluator and enricher.
+
+**Mode 4 providers can:**
+- **Evaluate** โ€” return a pass/fail, score, or recommendation based on the query
+- **Enrich** โ€” inject additional fields into the payload (risk scores, compliance citations, cost predictions, organizational context, case references)
+- **Do both** โ€” evaluate and enrich in a single atomic query-response cycle
+
+**Examples:**
+- AI/ML risk scoring engine โ€” returns risk score AND injects mitigation recommendations
+- Compliance oracle โ€” returns pass/fail AND injects compliance citations
+- FinOps cost predictor โ€” returns predicted cost AND injects cost allocation metadata
+- Sovereignty verification service โ€” returns jurisdiction compliance AND injects residency certificates
+- Fraud detection system โ€” returns anomaly score AND injects case reference number
+- Identity enrichment service โ€” returns authorization AND injects organizational context (business unit, cost center, project codes)
+- Privileged access management system โ€” returns allow/deny AND injects access justification record
+
+#### 4.8.2 Governance Concerns
+
+Mode 4 introduces governance concerns that Modes 1-3 do not:
+
+**Data sovereignty on outbound data:** DCM is sending request payload data โ€” potentially sensitive โ€” to an external system. Before any data is sent, DCM must verify the provider is authorized to receive the data classifications present in the query. A sovereign deployment must prevent any data from leaving its boundary without explicit authorization.
+
+**Result integrity:** The black box returns a result that DCM acts on. The logic is opaque โ€” DCM cannot inspect it. The provenance chain must record the full query-response cycle: what was sent, what was returned, what action was taken, and the provider's audit token for cross-system correlation.
+
+**Enrichment governance:** Fields injected by a Mode 4 provider carry the same provenance obligations as fields injected by a Transformation Policy. The override control model applies โ€” a GateKeeper can refuse black box enrichment on sovereignty-sensitive fields. Enrichment output may itself carry classification implications and must be governed accordingly.
+
+**Failure behavior:** The black box is external and can be unavailable, slow, or malformed. Failure behavior must be explicitly declared โ€” the default is `gatekeep` (unknown is not safe).
+
+**Non-determinism:** A Mode 4 provider may return different results for the same input at different times (e.g., a risk model updated overnight). Result caching must be declared โ€” and cached results carry a validity period.
+
+#### 4.8.3 Registration โ€” Mode 4 Specific Fields
+
+```yaml
+policy_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/policy/risk-scoring-engine"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Security Engineering Team"
+      notification_endpoint: 
+
+  name: "Enterprise Risk Scoring Engine"
+  description: >
+    ML-based risk scoring and enrichment for infrastructure requests.
+    Returns risk score and injects mitigation recommendations.
+
+  delivery:
+    mode: black_box_query           # Mode 4
+    endpoint: 
+    query_protocol: 
+    timeout_seconds: 30
+    on_timeout: gatekeep            # gatekeep | allow | escalate
+    on_error: gatekeep
+    on_unavailable: gatekeep
+
+  # What data this provider is authorized to receive
+  data_request_spec:
+    fields_requested:
+      - field: resource_type
+        classification_ceiling: unclassified
+      - field: placement.selected_provider_uuid
+        classification_ceiling: internal
+      - field: requester.tenant_uuid
+        classification_ceiling: internal
+      - field: lifecycle_constraints.ttl
+        classification_ceiling: unclassified
+    # Fields NOT declared here are NEVER sent โ€” DCM enforces data minimization
+    # classification_ceiling: maximum classification level this provider may receive
+
+  # Where the provider operates โ€” sovereignty gating
+  operational_sovereignty:
+    jurisdiction: eu-west
+    certifications: [ISO-27001, GDPR-compliant, SOC2-Type2]
+    # DCM checks these against Tenant sovereignty requirements
+    # before authorizing any query
+
+  # Result and enrichment capabilities
+  result_capabilities:
+    result_type: 
+    # pass_fail:    decision only
+    # score:        numeric score with optional threshold
+    # recommendation: structured recommendation
+    # enrichment:   data injection only โ€” no decision
+    # multi_factor: decision + enrichment combined
+
+    # Decision component schema (if applicable)
+    decision_schema:
+      outcome_field: outcome        # field name in response
+      score_field: score            # field name for numeric score
+      confidence_field: confidence
+      citations_field: citations
+      valid_until_field: valid_until
+      audit_token_field: audit_token
+
+    # Enrichment component schema (if applicable)
+    enrichment_schema:
+      fields_injected:
+        - field: risk_score
+          type: float
+          classification: internal
+        - field: risk_citations
+          type: array
+          classification: internal
+        - field: recommended_mitigations
+          type: array
+          classification: internal
+
+  # Result caching
+  result_caching:
+    enabled: true
+    ttl_seconds: 300
+    cache_key_fields: [resource_type, requester.tenant_uuid]
+
+  trust_level: verified
+  max_policy_authority: transformation   # enrichment = transformation authority
+  # A Mode 4 provider that only evaluates can have validation or gatekeeper authority
+  # A Mode 4 provider that enriches requires at minimum transformation authority
+```
+
+#### 4.8.4 Data Sovereignty Governance โ€” Pre-Query Evaluation
+
+Before DCM sends any data to a Mode 4 provider, the Policy Engine evaluates:
+
+```
+Query to Mode 4 provider proposed
+  โ”‚
+  โ–ผ
+Data classification check (BBQ-001)
+  โ”‚  What classification levels are in the query payload fields?
+  โ”‚  Is each field's classification โ‰ค provider's declared ceiling?
+  โ”‚  โ†’ Any field exceeds ceiling: strip field or reject query
+  โ”‚
+  โ–ผ
+Sovereignty check (BBQ-003)
+  โ”‚  Does the provider's operational_sovereignty.jurisdiction
+  โ”‚  satisfy the requesting Tenant's sovereignty requirements?
+  โ”‚  โ†’ Incompatible: block query, apply on_sovereign_mismatch behavior
+  โ”‚
+  โ–ผ
+Data minimization (BBQ-002)
+  โ”‚  Strip all fields not in provider's data_request_spec
+  โ”‚  Apply field-level filtering per cross_tenant_authorization if applicable
+  โ”‚
+  โ–ผ
+Authorized โ†’ send minimized query
+Unauthorized โ†’ apply on_unavailable behavior (typically gatekeep)
+```
+
+#### 4.8.5 Assembly Process Integration
+
+Mode 4 providers participate in any assembly phase โ€” most usefully inside the placement loop where provider-specific data is available:
+
+```yaml
+policy:
+  placement_phase: loop         # pre | loop | post | both
+  evaluation_type: black_box_query
+  black_box_provider_uuid: 
+
+  # What to send โ€” must be subset of provider's data_request_spec
+  query_fields: [resource_type, placement.selected_provider_uuid, requester.tenant_uuid]
+
+  # How to act on the decision component
+  on_decision:
+    pass: continue
+    fail: 
+    score_below_threshold:
+      threshold: 0.7
+      action: reject_candidate   # try next provider candidate
+    score_above_threshold:
+      threshold: 0.9
+      action: continue
+
+  # How to act on the enrichment component
+  on_enrichment:
+    inject_fields: true          # inject returned fields into payload
+    override_existing: false     # do not overwrite fields already set
+    # Each injected field carries source_type: black_box_provider + audit_token
+```
+
+#### 4.8.6 Result Schema and Provenance
+
+**Full result structure:**
+
+```yaml
+black_box_result:
+  # Decision component (optional)
+  decision:
+    outcome: 
+    score: 0.83
+    confidence: 
+    citations:
+      - "Provider certification ISO-27001 current as of 2026-01-15"
+      - "No open security incidents in region eu-west-1a"
+    valid_until: 
+    audit_token: "BB-2026-03-26-00847-A"  # provider's internal reference
+
+  # Enrichment component (optional)
+  enrichment:
+    fields_to_inject:
+      - field: risk_score
+        value: 0.83
+        provenance_note: "Returned by risk scoring engine v2.3"
+      - field: risk_citations
+        value: ["ISO-27001:A.12.1", "No active incidents"]
+        provenance_note: "Risk scoring engine evidence"
+      - field: recommended_mitigations
+        value: ["Enable MFA", "Restrict egress to known endpoints"]
+        provenance_note: "Risk scoring engine recommendations"
+```
+
+**Provenance on injected enrichment fields:**
+
+Each field injected by a Mode 4 provider carries standard field-level provenance:
+
+```yaml
+risk_score:
+  value: 0.83
+  metadata:
+    override: allow              # standard override control applies
+    basis_for_value: "ML risk scoring engine evaluation"
+  provenance:
+    origin:
+      source_type: black_box_provider
+      source_uuid: 
+      timestamp: 
+      audit_token: "BB-2026-03-26-00847-A"
+      query_uuid: 
+```
+
+#### 4.8.7 Audit Record
+
+Every Mode 4 query-response cycle produces a `black_box_evaluation_record` in the Audit Store regardless of outcome:
+
+```yaml
+black_box_evaluation_record:
+  record_uuid: 
+  policy_uuid: 
+  request_uuid: 
+  provider_uuid: 
+  evaluated_at: 
+  placement_phase: loop
+
+  query_sent:
+    fields_included: [resource_type, placement.selected_provider_uuid]
+    # Field NAMES only โ€” not raw values. Values stored in provider's system.
+    # Full correlation via audit_token.
+    data_minimization_applied: true
+    sovereignty_check: passed
+    classification_ceiling_honored: true
+
+  result_received:
+    result_type: multi_factor
+    decision:
+      outcome: pass
+      score: 0.83
+      confidence: high
+      valid_until: 
+      audit_token: "BB-2026-03-26-00847-A"
+    enrichment:
+      fields_injected: [risk_score, risk_citations, recommended_mitigations]
+      override_existing_applied: false
+
+  action_taken: continue_with_enrichment
+  cached_result: false
+  cache_stored: true
+  cache_expires_at: 
+```
+
+The `audit_token` is the **cross-system audit bridge** โ€” DCM's record references the provider's internal record. Auditors can correlate DCM's audit trail with the black box provider's own logs for full end-to-end traceability.
+
+#### 4.8.8 Failure and Fallback Behavior
+
+| Condition | Default Behavior | Rationale |
+|-----------|-----------------|-----------|
+| `on_timeout` | `gatekeep` | Unknown is not safe |
+| `on_error` | `gatekeep` | Malformed response is not safe |
+| `on_unavailable` | `gatekeep` | External unavailability cannot bypass governance |
+| `on_sovereign_mismatch` | `gatekeep` | Sovereignty cannot be bypassed |
+| `on_classification_exceeded` | strip field or `gatekeep` | Data cannot be sent to unauthorized recipient |
+
+All failure behaviors are configurable. `allow` is available for non-critical enrichment where the enrichment is additive and the request can proceed safely without it. Organizations must explicitly declare `allow` โ€” it is never the default.
+
+#### 4.8.9 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `BBQ-001` | Before sending any data to a Mode 4 provider, the Policy Engine must verify the provider is authorized to receive the data classification levels present in the query |
+| `BBQ-002` | Data sent to a Mode 4 provider must be minimized to only the fields declared in the provider's `data_request_spec` |
+| `BBQ-003` | A Mode 4 provider's `operational_sovereignty` must be compatible with the requesting Tenant's sovereignty requirements before any query is dispatched |
+| `BBQ-004` | All Mode 4 query-response cycles must produce a `black_box_evaluation_record` in the Audit Store |
+| `BBQ-005` | Mode 4 provider failure behavior (`on_timeout`, `on_error`, `on_unavailable`) must be explicitly declared โ€” default is `gatekeep` |
+| `BBQ-006` | Cached Mode 4 results must include the original query timestamp and validity period in provenance |
+| `BBQ-007` | Fields injected into the payload by a Mode 4 provider enrichment must carry standard field-level provenance: `source_type: black_box_provider`, `source_uuid`, and `audit_token` |
+| `BBQ-008` | The override control model applies to fields injected by Mode 4 enrichment โ€” a GateKeeper policy may restrict or refuse black box enrichment on specific fields |
+| `BBQ-009` | A Mode 4 provider that performs enrichment requires at minimum `transformation` trust level authority |
+
+---
+
+## 5. Lifecycle Time Constraints
+
+### 5.1 Concept
+
+Lifecycle time constraints declare **when a resource should cease to exist or trigger a lifecycle action**. They are a first-class field on any resource entity โ€” not metadata, not a tag, but a governed field that follows the standard data model precedence and override control.
+
+### 5.2 Constraint Types
+
+| Type | Format | Description |
+|------|--------|-------------|
+| `ttl` | ISO 8601 duration (e.g., `P14D`) | Relative โ€” expires N time after the reference point |
+| `expires_at` | ISO 8601 timestamp | Absolute โ€” expires at a specific calendar date/time |
+
+When both are declared, the **earliest expiry wins** (LTC-004).
+
+### 5.3 Data Model
+
+```yaml
+lifecycle_constraints:
+  ttl:
+    duration: P14D                       # ISO 8601 duration โ€” 14 days
+    reference_point: realization_timestamp  # created_at | realization_timestamp | last_modified
+    on_expiry: 
+    metadata:
+      override: allow                    # standard override control applies
+      basis_for_value: "Consumer declared ephemeral โ€” 14-day lab resource"
+
+  expires_at:
+    timestamp: "2026-06-30T23:59:59Z"
+    on_expiry: notify
+    metadata:
+      override: immutable
+      locked_by_policy_uuid: 
+      basis_for_value: "Project deadline โ€” resource must not persist beyond Q2"
+
+  # Enforcement behavior
+  enforcement:
+    warn_before_expiry: P1D              # warn 1 day before expiry
+    warn_notification_endpoint: 
+    grace_period: PT1H                  # 1 hour grace after expiry before action
+    on_grace_period_expiry: 
+```
+
+### 5.4 Precedence
+
+Lifecycle time constraints follow the standard data model precedence chain:
+
+```
+Base Layer default (lowest โ€” e.g., "no TTL by default")
+  โ†“
+Core Layer (e.g., "all dev environment resources: TTL 90 days")
+  โ†“
+Service Layer (e.g., "ephemeral compute: TTL 7 days")
+  โ†“
+Request Layer (consumer declared TTL)
+  โ†“
+Transformation Policy (enrich TTL from business context)
+  โ†“
+GateKeeper Policy (highest โ€” may lock TTL as immutable)
+```
+
+A consumer can declare `ttl: P14D` in their request. A GateKeeper policy can override this to `P7D` and lock it immutable if organizational policy mandates shorter maximum lifetimes. A Core Layer can set default TTLs for resource classes. The provenance chain records every modification.
+
+### 5.5 Expiry Enforcement
+
+The **Lifecycle Constraint Enforcer** is a DCM control plane component that:
+- Monitors all realized entities against their declared lifecycle constraints
+- Fires the configured `on_expiry` action when a constraint is reached
+- Records the enforcement action in provenance and the Audit Store
+- Emits expiry warnings `warn_before_expiry` duration before the deadline
+
+Expiry enforcement is a DCM concern โ€” not a provider concern. The provider does not need to know about or implement TTL logic.
+
+### 5.6 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `LTC-001` | Lifecycle time constraints follow standard data model precedence โ€” layers, request, policies |
+| `LTC-002` | GateKeeper policies may lock lifecycle constraints as `override: immutable` or `immutable_ceiling: absolute` |
+| `LTC-003` | Expiry enforcement is a DCM control plane function โ€” not a provider concern |
+| `LTC-004` | When multiple time constraints exist on an entity, the earliest expiry wins |
+| `LTC-005` | Expired entities that fail to execute their `on_expiry` action enter `PENDING_EXPIRY_ACTION` state and trigger an escalation |
+
+---
+
+## 6. Cross-Tenancy Authorization Model
+
+### 6.1 Default Stance โ€” Closed
+
+Cross-tenant information sharing is **closed by default**. No cross-tenant relationship of any nature is permitted unless explicitly authorized. This applies to both operational dependencies and informational relationships.
+
+The hard tenancy spectrum:
+
+| Setting | Meaning |
+|---------|---------|
+| `deny_all` | No cross-tenant relationships of any nature |
+| `explicit_only` | All cross-tenant must be explicitly authorized (DEFAULT) |
+| `operational_permitted` | Operational cross-tenant permitted; informational requires explicit auth |
+| `allow_all` | All cross-tenant permitted โ€” requires justification; not available in sovereign profile |
+
+The default shifts from the Q59 model's `operational_only` to `explicit_only`. Informational sharing is no longer implicitly open โ€” every informational cross-tenant relationship requires an explicit authorization record.
+
+### 6.2 Cross-Tenant Authorization Record
+
+```yaml
+cross_tenant_authorization:
+  artifact_metadata:
+    uuid: 
+    handle: "tenant-a/auth/shared-network-read"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Infrastructure Tenant Admin"
+
+  # WHO
+  authorized_consumer_tenant_uuid: 
+  authorized_actor_constraint:
+    roles: [service_account, automation]    # null = any actor in the tenant
+    specific_uuids: []                      # specific actor UUIDs if needed
+
+  # WHAT
+  resource_entity_uuid: 
+  resource_type_uuid: 
+  permitted_fields:
+    - field: network_segment
+    - field: vlan_id
+    # empty list = all fields permitted
+  permitted_relationship_natures: [informational, operational]
+
+  # WHEN
+  valid_from: 
+  valid_until: 
+
+  # WHERE
+  permitted_in_regions: [eu-west, eu-central]   # null = any region
+  sovereignty_constraints:
+    must_honor_consuming_tenant_sovereignty: true
+    must_honor_owning_tenant_sovereignty: true
+
+  # GOVERNANCE
+  authorized_by_policy_uuid: 
+  authorization_level: 
+  # Hierarchy: field_specific > resource_specific > tenant_global
+  # More specific = higher precedence
+```
+
+### 6.3 Authorization Hierarchy
+
+More specific authorizations take precedence over broader ones:
+
+```
+field_specific     โ† highest precedence โ€” only these exact fields on this entity
+  โ”‚
+resource_specific  โ† this entity โ€” all permitted fields
+  โ”‚
+tenant_global      โ† all entities in this Tenant โ€” broadest
+```
+
+If a tenant-global policy says "allow informational sharing with Tenant B" but a resource-specific policy says "this resource is not shareable with anyone," the resource-specific policy wins.
+
+### 6.4 System Policies โ€” Cross-Tenancy
+
+| Policy | Rule |
+|--------|------|
+| `XTA-001` | Cross-tenant information sharing is closed by default โ€” explicit authorization required |
+| `XTA-002` | Cross-tenant authorizations must specify who, what, when, and where |
+| `XTA-003` | More specific authorizations take precedence: field_specific > resource_specific > tenant_global |
+| `XTA-004` | All cross-tenant authorization decisions are policy-driven and DCM-enforced |
+| `XTA-005` | Sovereignty constraints declared by either Tenant must be honored by all cross-tenant relationships |
+
+---
+
+## 7. Rehydration Tenancy Controls
+
+### 7.1 Tenancy and Sovereignty Are Always Current
+
+Tenancy controls, sovereignty directives, and cross-tenant authorizations are **always evaluated against current policies during rehydration**. They cannot be pinned to historical versions.
+
+```yaml
+rehydration:
+  re_evaluate: true/false          # governs placement
+  policy_version: current/pinned   # governs resource configuration policies
+  # The following are ALWAYS current โ€” cannot be pinned:
+  tenancy_controls: always_current
+  sovereignty_controls: always_current
+  cross_tenant_authorizations: always_current
+```
+
+### 7.2 Rehydration Tenancy Conflict
+
+When rehydration produces a tenancy or sovereignty constraint that conflicts with an existing cross-tenant allocation:
+
+```yaml
+rehydration_tenancy_conflict_record:
+  rehydration_request_uuid: 
+  entity_uuid: 
+  conflict_type: cross_tenant_authorization_conflict
+  original_authorization_uuid: 
+  current_policy_violation:
+    policy_uuid: 
+    violation: "Consuming Tenant no longer has authorization for this allocation"
+  action_taken: paused
+  entity_state: PENDING_REVIEW
+  notifications_sent:
+    - entity_owner
+    - owning_tenant_admin
+    - consuming_tenant_admin
+    - platform_admin
+  resolution_options:
+    - re_authorize
+    - release
+    - escalate
+  policy_override_available: true
+```
+
+### 7.3 System Policies โ€” Rehydration
+
+| Policy | Rule |
+|--------|------|
+| `RHY-001` | Tenancy, sovereignty, and cross-tenant authorizations always use current policies during rehydration |
+| `RHY-002` | Rehydration that conflicts with current tenancy/sovereignty pauses and enters PENDING_REVIEW |
+| `RHY-003` | A paused rehydration allocation is not automatically released โ€” requires explicit resolution |
+| `RHY-004` | A policy may declare automatic resolution behavior for rehydration tenancy conflicts |
+
+---
+
+## 8. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should organizations be able to submit custom profiles and groups back to the DCM project registry? | Community | โœ… Resolved โ€” community submissions via PR-based workflow; Tier 2; documented use case + deployment reference + test results + named maintainer (PROF-005) |
+| 2 | Should there be a certified profile program โ€” profiles that have been validated against specific regulatory frameworks? | Compliance | โœ… Resolved โ€” certified profile program with third-party certification metadata; certified profiles promoted to Tier 1; applies to artifact not deployment (PROF-006) |
+| 3 | Should Policy Provider trust elevation require a formal approval workflow in DCM UI or is out-of-band approval sufficient? | Security | โœ… Resolved โ€” formal approval workflow; profile-governed approvers (1 standard โ†’ 3 sovereign); P7D shadow period; POLICY_PROVIDER_ELEVATED audit (PROF-007) |
+| 4 | Should the default TTL for dev profile resources be configurable at the platform level or only at the group level? | Configuration | โœ… Resolved โ€” overridable at platform domain layer; per-resource-type TTL overrides; on_expiry action configurable (PROF-008) |
+| 5 | How does Policy Provider delivery interact with air-gapped deployments โ€” pull from internal mirror? | Sovereignty | โœ… Resolved โ€” signed bundle model identical to registry; Mode 4 in sovereign restricted to within-boundary endpoints (PROF-009) |
+
+---
+
+## 9. Related Concepts
+
+- **Policy Engine** โ€” executes the policies organized by groups and profiles
+- **Policy Layers** โ€” the assembly process step where policies execute
+- **Artifact Metadata** โ€” universal metadata on all policy artifacts
+- **Five Artifact Statuses** โ€” developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired
+- **Shadow Execution** โ€” proposed policies and profiles run in shadow mode for validation
+- **Override Control** โ€” policies set override control on fields; immutable_ceiling: absolute for non-negotiables
+- **Ingestion Model** โ€” policy profile requirements may gate brownfield promotion
+- **Four States** โ€” rehydration tenancy controls govern how historical states are replayed
+
+
+## 8. Policy Profile Gap Resolutions
+
+### 8.1 Community Profile and Group Submissions (Q1)
+
+Organizations may submit custom profiles and policy groups to the DCM community registry following the same PR-based proposal workflow as Resource Types. Community contributions live in Tier 2 (Verified Community).
+
+```yaml
+community_profile_submission:
+  profile:
+    handle: "community/profile/hipaa-openstack"
+    extends: system/profile/hipaa-prod
+    description: "HIPAA production profile for OpenStack deployments"
+    policy_groups:
+      - system/group/compliance-hipaa
+      - community/group/openstack-security-baseline
+    contributed_by: "Healthcare IT Community Group"
+    tested_with: [OpenStack 2024.1, DCM 1.0]
+  required_for_submission:
+    - documented_use_case
+    - at_least_one_real_deployment_reference
+    - test_results_against_reference_implementation
+    - named_maintainer
+```
+
+Community profiles carry the same lifecycle as Resource Types โ€” shadow validation before active, deprecation policies, sunset periods. Organizations adopt them directly or extend them further.
+
+### 8.2 Certified Profile Program (Q2)
+
+DCM supports a certified profile program where profiles carry formal third-party certification metadata against compliance frameworks. Certified profiles are promoted to Tier 1 (DCM Core).
+
+```yaml
+profile_certification:
+  certifications:
+    - framework: HIPAA
+      certifying_body: "Coalfire Systems"
+      certification_date: "2025-11-01"
+      valid_until: "2027-11-01"
+      certification_scope: "PHI data lifecycle management via DCM"
+      certificate_ref:
+        credential_provider_uuid: 
+        path: "dcm/registry/certifications/hipaa-prod-2025"
+```
+
+**Important:** Profile certification applies to the profile artifact only โ€” it does not certify the deploying organization's compliance posture. A certified profile is evidence that the profile implements the required controls; it is not a compliance certification of any specific deployment.
+
+### 8.3 Policy Provider Trust Elevation Approval (Q3)
+
+Policy Provider trust elevation (increasing the mode level) requires a formal approval workflow. Approval requirements are profile-governed.
+
+```yaml
+policy_provider_trust_elevation:
+  elevation_request:
+    from_mode: 1
+    to_mode: 3
+    justification: "Need OPA Rego for complex placement constraints"
+
+  approval_requirements:
+    standard:
+      approvers: [platform_admin]
+      min_approvers: 1
+    prod:
+      approvers: [platform_admin, security_owner]
+      min_approvers: 2
+    fsi:
+      approvers: [platform_admin, security_owner, compliance_officer]
+      min_approvers: 2
+      verified_required: true
+    sovereign:
+      approvers: [platform_admin, security_owner, compliance_officer]
+      min_approvers: 3
+      requires_change_control_ticket: true
+
+  shadow_period_after_elevation: P7D    # elevated mode runs in shadow before active
+  audit_record: POLICY_PROVIDER_ELEVATED
+```
+
+The P7D shadow period catches unintended consequences before elevated outputs become binding on production requests.
+
+### 8.4 Dev Profile Resource TTL Configurability (Q4)
+
+The default TTL for dev profile resources is declared in the system domain layer and overridable at the platform domain level.
+
+```yaml
+layer:
+  handle: "platform/dev-profile/resource-ttl-override"
+  domain: platform
+  fields:
+    dev_profile_resource_ttl:
+      default_ttl: P30D               # platform override: 30d instead of system default P7D
+      max_ttl: P90D                   # consumers cannot declare TTL > 90 days in dev
+      on_expiry: notify               # notify (consumers can extend) vs destroy
+      per_resource_type_overrides:
+        Compute.VirtualMachine: P7D
+        Storage.Block: P14D
+        DNS.Record: P3D
+```
+
+### 8.5 Air-Gapped Policy Provider Delivery (Q5)
+
+Policy Provider delivery in air-gapped deployments uses signed bundles โ€” same model as the registry bundle system.
+
+```yaml
+policy_provider_airgap:
+  delivery_mode: signed_bundle
+  bundle_contents:
+    - provider_registration_yaml
+    - policy_artifacts_zip
+    - mode_specific_package:
+        mode_3: opa_rego_bundle       # OPA Rego files + data
+        mode_4: endpoint_config       # endpoint declaration (must be within boundary)
+  signing_key_ref: 
+  valid_until: 
+```
+
+**Mode 4 sovereign constraint:** In sovereign profiles, Mode 4 Policy Providers may only call endpoints within the sovereignty boundary. External AI service calls are blocked by the BBQ-001 sovereignty check before any Mode 4 query.
+
+### 8.6 System Policies โ€” Policy Profile Gaps
+
+| Policy | Rule |
+|--------|------|
+| `PROF-005` | Organizations may submit custom profiles and policy groups to the DCM community registry via the same PR-based proposal workflow as Resource Types. Community contributions live in Tier 2. Submissions require documented use case, at least one production deployment reference, test results, and a named maintainer. |
+| `PROF-006` | DCM supports a certified profile program where profiles carry formal third-party certification metadata. Certified profiles are promoted to Tier 1. Profile certification applies to the artifact only โ€” it does not certify the deploying organization's compliance posture. |
+| `PROF-007` | Policy Provider trust elevation requires a formal approval workflow (standard: 1 platform admin; prod: platform admin + security owner; fsi/sovereign: dual approval + compliance officer). Elevated providers run in shadow mode for P7D before activation. All elevations produce a POLICY_PROVIDER_ELEVATED audit record. |
+| `PROF-008` | The default TTL for dev profile resources is declared in the system domain layer and overridable at the platform domain level. Per-resource-type TTL overrides are supported. The on_expiry action is configurable. |
+| `PROF-009` | Policy Provider delivery in air-gapped deployments uses signed bundles identical to the registry bundle model. Mode 4 providers in sovereign profiles may only call endpoints within the sovereignty boundary. |
+
+
+
+---
+
+## 9. Recovery Posture Policy Groups
+
+### 9.1 recovery_posture as a Concern Type
+
+`recovery_posture` is a Policy Group concern_type that governs how DCM responds to provisioning failures, timeouts, and ambiguous states. It is the fifth concern type alongside security, compliance, operational, and implementation posture.
+
+Recovery posture groups contain Recovery Policies โ€” a formal DCM policy type that maps trigger conditions (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, etc.) to response actions (DRIFT_RECONCILE, DISCARD_AND_REQUEUE, NOTIFY_AND_WAIT, etc.).
+
+See [Operational Models](24-operational-models.md) Section 5 for the complete Recovery Policy model, trigger vocabulary, and action vocabulary.
+
+### 9.2 Four Built-in Recovery Posture Groups
+
+| Group Handle | Posture | Appropriate For |
+|-------------|---------|----------------|
+| `system/group/recovery-automated-reconciliation` | Let drift detection converge on correct state | Dev, standard environments |
+| `system/group/recovery-discard-and-requeue` | Clean up and restart on any ambiguity | Consistency-critical environments |
+| `system/group/recovery-notify-and-wait` | Always notify human; never act automatically | FSI, sovereign, regulated environments |
+| `system/group/recovery-aggressive-retry` | Retry everything before giving up | High-transient-failure environments |
+
+### 9.3 Profile Binding Defaults
+
+| Profile | Default Recovery Posture |
+|---------|------------------------|
+| `minimal` | recovery-automated-reconciliation |
+| `dev` | recovery-automated-reconciliation |
+| `standard` | recovery-automated-reconciliation |
+| `prod` | recovery-notify-and-wait |
+| `fsi` | recovery-notify-and-wait |
+| `sovereign` | recovery-notify-and-wait |
+
+### 9.4 Override Hierarchy
+
+Organizations override recovery posture at Tenant or resource-type level without changing the deployment profile:
+
+```yaml
+# Tenant override โ€” all resources in this Tenant use discard-and-requeue
+tenant_config:
+  recovery_profile_override: recovery-discard-and-requeue
+
+# Resource-type override โ€” VMs get aggressive retry regardless of Tenant/profile
+resource_type_recovery_override:
+  resource_type: Compute.VirtualMachine
+  recovery_profile: recovery-aggressive-retry
+```
+
+Resource-type override wins over Tenant override wins over profile default.
+
+
+
+---
+
+## 10. Zero Trust Posture Policy Groups
+
+### 10.1 zero_trust_posture as a Concern Type
+
+`zero_trust_posture` is the sixth Policy Group concern type. It governs authentication requirements, credential lifetime, revocation check frequency, and hardware attestation requirements for all DCM interactions.
+
+See [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) Section 5 for the complete zero trust model.
+
+### 10.2 Four Zero Trust Posture Levels
+
+| Posture | Boundary | Internal | Hardware | Profile Default |
+|---------|---------|----------|----------|----------------|
+| `none` | Perimeter model | Trusted | Not required | minimal |
+| `boundary` | Zero trust at external boundaries | Service mesh | Not required | dev, standard |
+| `full` | Zero trust everywhere | Per-call auth | Not required | prod, fsi |
+| `hardware_attested` | Zero trust everywhere | Per-call auth | Required (TPM/HSM) | sovereign |
+
+### 10.3 Credential Lifetime Defaults
+
+| Profile | Max credential lifetime |
+|---------|------------------------|
+| minimal | PT8H |
+| dev | PT4H |
+| standard | PT1H |
+| prod | PT30M |
+| fsi | PT15M |
+| sovereign | PT15M + hardware attestation |
+
+### 10.4 Hard Data Boundary Constraints
+
+The sovereign profile enforces a hard constraint via the Data/Capability Authorization Matrix: **data classified as `sovereign` or `classified` never crosses any interaction boundary**. This constraint is declared with `hard_constraint: true` in the federation boundary matrix and cannot be overridden by any policy, profile, or operator action. It is enforced at the matrix level, not the policy level.
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/15-universal-groups.md b/content/docs/data-model/15-universal-groups.md
new file mode 100644
index 0000000..c3222e7
--- /dev/null
+++ b/content/docs/data-model/15-universal-groups.md
@@ -0,0 +1,682 @@
+# DCM Data Model โ€” Universal Group Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Grouping](08-resource-grouping.md) | [Entity Relationships](09-entity-relationships.md) | [Policy Organization](14-policy-profiles.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA**
+>
+> The Data abstraction โ€” DCMGroup as universal grouping artifact
+
+
+
+---
+
+## 1. Purpose
+
+DCM previously used eight distinct grouping constructs โ€” Tenant, Resource Groups, Custom Resource Groups, Policy Groups, Policy Profiles, Layer Domain, Activation Scope, and Cross-Tenant Authorization. Each had its own structure, its own API, its own documentation, and its own membership model. This fragmentation created cognitive overhead for operators and prevented natural organizational structures (such as "everything related to Payments") from being expressed as a single construct.
+
+The **Universal Group Model** collapses all grouping into a single `DCMGroup` entity distinguished by `group_class` metadata. One mental model. One API. One registry. The same UUID, versioning, lifecycle, policy targeting, and audit trail apply to every group regardless of its class.
+
+**What changes:**
+- All grouping constructs become `group_class` values on `DCMGroup`
+- Existing UUIDs, handles, and API references are preserved
+- Existing APIs become class-filtered views of the universal group store
+- Migration is additive โ€” no breaking changes
+
+**What does NOT change:**
+- The structural invariants of each construct (one Tenant per resource, no constituent cross-tenant, etc.)
+- Policy enforcement behavior โ€” governed by the active Profile, not per-group configuration
+- The hard-tenancy model โ€” tenant_boundary groups retain all isolation guarantees
+
+---
+
+## 2. The DCMGroup Entity
+
+### 2.1 Universal Structure
+
+```yaml
+dcm_group:
+  artifact_metadata:
+    uuid: 
+    handle: 
+    version: 
+    status: 
+    created_by: 
+    owned_by: 
+    created_via: 
+    modifications: 
+
+  # IDENTITY
+  name: 
+  description: 
+  concern_tags: [payments, pci-scope, eu-west]  # free tagging โ€” discoverability
+
+  # WHAT KIND OF GROUP
+  group_class: 
+  group_subclass: 
+  # group_subclass examples: cost_center, business_unit, compliance_scope, project
+
+  # MEMBERSHIP
+  member_types_permitted: [resource_entity, policy, layer, group, tenant]
+  # Determines what can be a member of this group
+  # Single-type groups declare one type (e.g., [policy] for policy_collection)
+  # Composite groups declare multiple types
+
+  exclusivity:
+    per_member: 
+    # one:  a member can belong to only one group of this class at a time
+    # many: a member can belong to multiple groups of this class simultaneously
+    per_group: 
+    cap: 
+
+  members:
+    - member_uuid: 
+      member_type: 
+      member_role: 
+      added_at: 
+      added_by: 
+      valid_from: 
+      valid_until: 
+      membership_status: 
+
+  # BEHAVIOR
+  enforcement_model: 
+  # advisory:   group is a tag โ€” no system behavior enforced by DCM
+  # enforced:   group drives policy scoping and system behavior
+  # mandatory:  group membership is non-optional (structural requirement)
+  # NOTE: For tenant_boundary groups, enforcement_model is profile-governed
+  # โ€” the active Profile sets the enforcement floor, not per-group configuration
+
+  cross_boundary:
+    tenant_spanning: 
+    sovereignty_spanning: 
+
+  lifecycle_coupling:
+    on_group_destroy: 
+    # detach (DEFAULT): destroying the group releases memberships but
+    #                   does NOT destroy members
+    # cascade: destroying the group destroys all members
+    # notify:  destroying the group notifies owners and waits for confirmation
+    # retain:  group cannot be destroyed while it has members
+    on_member_destroy: 
+
+  # INHERITANCE AND COMPOSITION
+  extends: 
+  includes_groups:
+    - group_uuid: 
+      member_type_filter: [resource_entity]  # optional โ€” only include this type
+      # If omitted: all member types from the included group are pulled in
+
+  # NESTING (for tenant_boundary groups)
+  parent_group_uuid: 
+  child_groups: [, ...]  # populated by DCM โ€” do not set manually
+
+  # POLICY TARGETING
+  # Any policy can target this group by UUID or handle โ€” no special declaration
+  # Policy targeting a composite group applies to all member types by default
+  # Policy can narrow with: member_type_filter: [resource_entity]
+```
+
+### 2.2 Group Classes
+
+| group_class | Replaces | member_types_permitted | exclusivity.per_member | enforcement_model |
+|-------------|---------|----------------------|----------------------|------------------|
+| `tenant_boundary` | Tenant | resource_entity, group | one (structural lock) | profile-governed |
+| `resource_grouping` | Resource Group, Custom Resource Group | resource_entity | many | advisory |
+| `policy_collection` | Policy Group | policy | many | enforced |
+| `policy_profile` | Policy Profile | group (policy_collection only) | many | enforced |
+| `layer_grouping` | Layer Domain grouping | layer | many | enforced |
+| `provider_grouping` | Provider collections | provider | many | advisory |
+| `composite` | (new) | all types | many | configurable |
+| `federation` | (new) | group (tenant_boundary) | many | advisory |
+
+### 2.3 Structural Invariants โ€” Non-Overridable
+
+Regardless of `enforcement_model`, `group_class`, or active Profile, the following structural invariants always hold:
+
+| Invariant | Applies To | Rule |
+|-----------|-----------|------|
+| `GRP-INV-001` | `tenant_boundary` | A resource_entity may belong to exactly one active tenant_boundary group |
+| `GRP-INV-002` | `tenant_boundary` | Constituent relationships may not cross tenant_boundary group boundaries |
+| `GRP-INV-003` | `tenant_boundary` | Destroying a parent tenant_boundary group requires explicit resolution of all child groups first โ€” no silent cascade |
+| `GRP-INV-004` | `tenant_boundary` | A resource in a child tenant_boundary group belongs to the child โ€” never the parent |
+| `GRP-INV-005` | All | Circular group membership is invalid |
+| `GRP-INV-006` | All | A group cannot be a member of itself |
+
+---
+
+## 3. Group Class Reference
+
+### 3.1 tenant_boundary
+
+**Replaces:** Tenant entity  
+**Purpose:** Ownership boundary, isolation enforcement, cost attribution, audit scope, sovereignty boundary
+
+```yaml
+dcm_group:
+  group_class: tenant_boundary
+  member_types_permitted: [resource_entity, group]
+  exclusivity:
+    per_member: one   # STRUCTURAL LOCK โ€” cannot be changed by policy
+  enforcement_model: mandatory   # set by active Profile โ€” not configurable per-group
+  cross_boundary:
+    tenant_spanning: false   # STRUCTURAL LOCK
+  lifecycle_coupling:
+    on_group_destroy: notify   # requires explicit resolution
+    on_member_destroy: remove_from_group
+
+  # Tenant-specific fields preserved from original model
+  tenant_config:
+    hard_tenancy:
+      cross_tenant_relationships: explicit_only
+    active_profile: system/profile/standard
+    minimum_child_profile: null
+```
+
+**Profile-governed enforcement:**
+- `minimal` profile โ†’ `enforcement_model: advisory` (tenancy optional)
+- `dev` profile โ†’ `enforcement_model: enforced` (tenancy recommended)
+- `standard` and above โ†’ `enforcement_model: mandatory` (tenancy required)
+
+### 3.2 resource_grouping
+
+**Replaces:** DCM Default Resource Group, Custom Resource Group  
+**Purpose:** Flexible composable grouping of resource entities โ€” structured tagging
+
+```yaml
+dcm_group:
+  group_class: resource_grouping
+  group_subclass: cost_center   # advisory โ€” CostCenter, BusinessUnit, Project, Team...
+  member_types_permitted: [resource_entity]
+  exclusivity:
+    per_member: many   # a resource can be in multiple resource groups
+  enforcement_model: advisory
+```
+
+### 3.3 policy_collection
+
+**Replaces:** Policy Group  
+**Purpose:** Cohesive collection of policies addressing a single concern
+
+```yaml
+dcm_group:
+  group_class: policy_collection
+  concern_tags: [pci-dss, encryption, network-segmentation]
+  member_types_permitted: [policy]
+  enforcement_model: enforced
+  # Source โ€” local or Policy Provider
+  source:
+    type: 
+    provider_uuid: 
+    on_provider_update: 
+```
+
+### 3.4 policy_profile
+
+**Replaces:** Policy Profile  
+**Purpose:** Complete DCM configuration for a use case, composed of policy_collection groups
+
+```yaml
+dcm_group:
+  group_class: policy_profile
+  member_types_permitted: [group]   # only policy_collection groups
+  extends:     # inherits all parent's groups
+  enforcement_model: enforced
+```
+
+### 3.5 composite
+
+**New concept:** A group whose members span multiple member types โ€” the organizational unit for a complete concern.
+
+```yaml
+dcm_group:
+  group_class: composite
+  name: "Payments Platform"
+  concern_tags: [payments, pci-scope]
+  member_types_permitted: [resource_entity, policy, layer, group, provider]
+  enforcement_model: advisory   # composite groups are organizational โ€” advisory default
+
+  members:
+    - member_uuid: 
+      member_type: resource_entity
+      member_role: compute
+    - member_uuid: 
+      member_type: group
+      member_role: compliance_governance
+    - member_uuid: 
+      member_type: group
+      member_role: resource_inventory
+    - member_uuid: 
+      member_type: layer
+      member_role: configuration
+```
+
+**Policy targeting composite groups:**
+```yaml
+policy:
+  target_groups:
+    - group_uuid: 
+      member_type_filter: [resource_entity]   # narrow to resources only
+      # Omit member_type_filter to apply to ALL member types (default)
+```
+
+### 3.6 federation
+
+**New concept:** A group of tenant_boundary groups that share governance, visibility, and resources while maintaining complete independence.
+
+```yaml
+dcm_group:
+  group_class: federation
+  name: "Global FSI Federation"
+  member_types_permitted: [group]   # tenant_boundary groups only
+  enforcement_model: advisory       # federation cannot override member Tenant isolation
+
+  members:
+    - member_uuid: 
+      member_type: group
+      member_role: member_tenant
+    - member_uuid: 
+      member_type: group
+      member_role: member_tenant
+    - member_uuid: 
+      member_type: group
+      member_role: shared_governance
+
+  federation_config:
+    shared_policy_inheritance: 
+    # opt_in:  member Tenants must explicitly adopt shared policies
+    # opt_out: shared policies apply to all members unless explicitly excluded
+    cross_member_visibility: 
+    consolidated_reporting: true
+```
+
+---
+
+## 4. Nested Tenants
+
+### 4.1 Concept
+
+A **Nested Tenant** is a `tenant_boundary` group that is a member of a parent `tenant_boundary` group. The child Tenant maintains complete isolation โ€” its resources belong to it, not the parent. The parent Tenant has governance overlay, cost rollup authority, and audit aggregation across all children.
+
+```
+corporate_tenant (tenant_boundary)
+  โ”‚  child_groups:
+  โ”œโ”€โ”€ business_unit_a_tenant (tenant_boundary)
+  โ”‚     โ””โ”€โ”€ resources, policies, layers owned by BU-A
+  โ””โ”€โ”€ business_unit_b_tenant (tenant_boundary)
+        โ””โ”€โ”€ resources, policies, layers owned by BU-B
+```
+
+### 4.2 Structural Invariants for Nested Tenants
+
+- A resource belongs to the **leaf** tenant_boundary group โ€” never the parent (GRP-INV-004)
+- Parent Tenant has **governance overlay** โ€” not ownership
+- Parent Tenant destruction requires all child Tenants to be resolved first (GRP-INV-003)
+- Constituent relationships cannot cross any tenant_boundary boundary โ€” including parent-child (GRP-INV-002)
+
+### 4.3 Policy Inheritance Direction
+
+Policy inheritance from parent to child Tenant is profile-governed:
+
+| Profile | Default | Meaning |
+|---------|---------|---------|
+| `minimal`, `dev` | `opt_in` | Child Tenants must explicitly adopt parent policies |
+| `standard`, `prod` | `opt_out` | Parent policies cascade to children unless child excludes |
+| `fsi`, `sovereign` | `opt_in` | Nothing crosses without consent |
+
+```yaml
+nested_tenant_config:
+  parent_group_uuid: 
+  policy_inheritance: opt_out   # governed by active Profile
+  parent_policy_exclusions:
+    - policy_uuid:    # explicitly excluded from cascading to this child
+  cost_rollup_to_parent: true
+  audit_visible_to_parent: true
+  sovereign_boundary: independent   # child sovereignty independent of parent
+```
+
+### 4.4 Nested Tenant Use Cases
+
+- **Enterprise structure:** Corporate โ†’ Business Unit โ†’ Team Tenants
+- **Multi-region deployment:** Global Tenant โ†’ Regional Tenants โ†’ Zone Tenants
+- **Multi-tier compliance:** Organization Tenant โ†’ PCI-scope Tenant โ†’ Payment-processing Tenant
+- **Partner/customer isolation:** Platform Tenant โ†’ Customer A Tenant โ†’ Customer B Tenant
+
+---
+
+## 5. Federated Tenants
+
+### 5.1 Concept
+
+A **Federated Tenant** structure is a `federation` group containing multiple independent `tenant_boundary` groups. Member Tenants maintain complete independence โ€” the federation provides shared governance, consolidated visibility, and mutual cross-tenant authorization within the federation scope.
+
+### 5.2 Federation Capabilities
+
+- **Shared policy application:** `policy_collection` groups included in the federation apply to all member Tenants (per `shared_policy_inheritance` setting)
+- **Cross-member visibility:** federation members can declare mutual `cross_tenant_authorization` scoped to federation membership โ€” without requiring separate bilateral authorizations
+- **Consolidated reporting:** cost, audit, and observability queries scoped to the federation group return aggregated results across all member Tenants
+- **Federation-level governance:** policies targeting the federation group apply to all member Tenants
+
+### 5.3 Federation vs Nesting
+
+| Dimension | Nested Tenants | Federated Tenants |
+|-----------|---------------|-----------------|
+| Relationship | Parent-child hierarchy | Peer membership |
+| Governance direction | Top-down from parent | Shared among peers |
+| Independence | Child subordinate to parent | Members fully independent |
+| Cost rollup | Mandatory to parent | Configurable |
+| Use case | Enterprise hierarchy | Multi-organization collaboration |
+
+---
+
+## 6. Group Registry and API
+
+### 6.1 Universal Registry
+
+All groups are stored in a single **Group Registry** โ€” a GitOps store following the standard Storage Provider contract. The registry is queryable by any combination of fields.
+
+### 6.2 Class-Filtered API Views
+
+The universal registry exposes class-filtered views that preserve backward compatibility with existing API consumers:
+
+| Endpoint | Equivalent Query |
+|----------|----------------|
+| `GET /tenants` | `GET /groups?group_class=tenant_boundary` |
+| `GET /resource-groups` | `GET /groups?group_class=resource_grouping` |
+| `GET /policy-groups` | `GET /groups?group_class=policy_collection` |
+| `GET /policy-profiles` | `GET /groups?group_class=policy_profile` |
+| `GET /federations` | `GET /groups?group_class=federation` |
+
+Existing API references continue to work unchanged. New API consumers can use the universal endpoint.
+
+---
+
+## 7. Migration from Current Constructs
+
+Existing constructs migrate to the universal model with UUID preservation:
+
+| Current Construct | Migration | UUID Preserved? |
+|------------------|-----------|----------------|
+| Tenant entity | `group_class: tenant_boundary` | Yes |
+| DCM Default Resource Group | `group_class: resource_grouping` | Yes |
+| Custom Resource Group | `group_class: resource_grouping, group_subclass: ` | Yes |
+| Policy Group | `group_class: policy_collection` | Yes |
+| Policy Profile | `group_class: policy_profile` | Yes |
+
+Migration uses the standard ingestion model โ€” existing constructs are ingested as `ingestion_source: migration` with `ingestion_confidence: high` (UUID preservation, no ambiguity).
+
+---
+
+## 8. DCM System Policies
+
+| Policy | Rule |
+|--------|------|
+| `GRP-INV-001` | A resource_entity may belong to exactly one active tenant_boundary group |
+| `GRP-INV-002` | Constituent relationships may not cross tenant_boundary group boundaries |
+| `GRP-INV-003` | Destroying a parent tenant_boundary group requires explicit resolution of all child groups first |
+| `GRP-INV-004` | A resource in a child tenant_boundary group belongs to the child โ€” never the parent |
+| `GRP-INV-005` | Circular group membership is invalid and must be rejected |
+| `GRP-INV-006` | A group cannot be a member of itself |
+| `GRP-007` | Composite group `on_group_destroy` default is `detach` โ€” destroying a group releases memberships but does not destroy members |
+| `GRP-008` | Policies targeting a composite group apply to all member types by default; `member_type_filter` narrows scope |
+| `GRP-009` | Federation groups cannot override member Tenant isolation boundaries |
+| `GRP-010` | Nested Tenant policy inheritance direction is governed by the active Profile โ€” not per-group configuration |
+
+---
+
+## 9. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should composite group policy targeting emit a linting warning when no member_type_filter is declared? | Operational safety | โœ… Resolved โ€” linting warning (not error) when composite policy targeting has no member_type filter; suppress with explicit_no_filter: true (GRP-016) |
+| 2 | Should there be a maximum nesting depth for tenant_boundary groups? | Operational governance | โœ… Resolved โ€” Maximum nesting depth is profile-governed: standard/prod = 5 levels; fsi/sovereign = 3 levels. Deeper nesting creates policy inheritance complexity and audit graph depth issues. Enforced at group creation time. |
+| 3 | How does group membership interact with the Search Index? | Performance | โœ… Resolved โ€” Group membership is indexed in the Search Index as a field on each entity record (member_of_groups: [uuid, ...]). The Search Index supports querying by group_uuid. Group membership changes trigger an incremental index update (not full rebuild). Staleness follows the standard Search Index model (PT5M standard profile). |
+| 4 | Should time-bounded memberships (valid_until) trigger notifications before expiry? | Consumer experience | โœ… Resolved โ€” warn_before_expiry field on membership (GRP-014) |
+
+---
+
+## 9. Grouping and Relationship Gap Resolutions
+
+### 9.1 Community Subclass Catalog (Q35)
+
+The `group_class` set is closed โ€” system behavior is tied to declared classes only. `group_subclass` is open and advisory. DCM maintains a community subclass catalog as a non-authoritative reference shipped with the well-known Information Provider Registry:
+
+```yaml
+# Community subclass catalog (advisory โ€” not enforced, not validated)
+common_group_subclasses:
+  resource_grouping:
+    - subclass: cost_center
+      description: "Financial cost attribution grouping"
+    - subclass: business_unit
+      description: "Organizational business unit"
+    - subclass: project
+      description: "Project-scoped resource collection"
+    - subclass: environment
+      description: "Environment grouping (prod/staging/dev)"
+    - subclass: application
+      description: "Application component grouping"
+  policy_collection:
+    - subclass: compliance_framework
+      description: "Policies implementing a compliance framework"
+    - subclass: technology_baseline
+      description: "Technology-specific policy baseline"
+```
+
+Organizations freely declare subclasses not in the catalog โ€” there is no validation or enforcement on subclass values.
+
+### 9.2 Group Sovereignty Interaction (Q36)
+
+Sovereignty interaction is group_class-specific:
+
+| group_class | Cross-Sovereignty | Notes |
+|-------------|-----------------|-------|
+| `tenant_boundary` | **Never** | Structural โ€” not configurable |
+| `resource_grouping` | Permitted by default | Policy may restrict for classified resources |
+| `policy_collection` | Always permitted | Policies have no sovereignty โ€” governance artifacts |
+| `layer_grouping` | Always permitted | Layers have no sovereignty |
+| `composite` | Governed by most restrictive member | If contains cross-sovereignty resources, resource rules apply |
+| `federation` | Permitted with DCM federation rules | DCM-003 governs data flows |
+
+```yaml
+# Policy restricting cross-sovereignty resource group membership
+policy:
+  type: gatekeeper
+  rule: >
+    If group.group_class == resource_grouping
+    AND member.classification_level IN [confidential, restricted]
+    AND member.sovereignty_zone != group.primary_sovereignty_zone
+    THEN gatekeep: "Classified resources cannot join cross-sovereignty resource groups"
+```
+
+### 9.3 Tenant Decommission Lifecycle (Q37)
+
+Tenant decommission is the highest-stakes lifecycle operation in DCM. It requires mandatory pre-decommission validation and follows a staged sequence.
+
+**Phase 1 โ€” Pre-decommission validation (blocking):**
+- All resources in decommissionable state (not PROVISIONING or active incidents)
+- Cross-tenant operational relationships accounted for (consuming Tenants notified)
+- Allocated resources claimed by other Tenants addressed (returned or migrated)
+- Active rehydration leases released
+- Compliance holds reviewed (HIPAA/PCI records may need archival)
+- Child tenant_boundary groups resolved first (GRP-INV-003)
+
+**Phase 2 โ€” Resource decommission (per lifecycle policy):**
+```
+For each resource in the Tenant:
+  cascade โ†’ decommission resource (default for tenant_boundary)
+  retain  โ†’ resource enters ORPHANED state (operator must rehome or destroy)
+  notify  โ†’ alert owner; resource enters PENDING_DECOMMISSION
+```
+
+**Phase 3 โ€” Group membership cleanup:**
+- Remove Tenant from all group memberships
+- Empty federation groups enter EMPTY state
+- Orphaned child groups must have been resolved in Phase 1
+
+**Phase 4 โ€” Audit record archival:**
+All audit records enter `all_retired` retention_status. They are **never destroyed** as part of Tenant decommission. Post-lifecycle retention clock starts per governing policy.
+
+### 9.4 Time-Bounded Group Membership (Q38)
+
+Group memberships already support time-bounded validity via `valid_from` and `valid_until` in the Universal Group Model. The Lifecycle Constraint Enforcer handles expiry.
+
+```yaml
+member:
+  member_uuid: 
+  member_type: resource_entity
+  valid_from: "2026-01-01T00:00:00Z"
+  valid_until: "2026-12-31T23:59:59Z"
+  membership_status: 
+  on_expiry: 
+  # remove:         member silently removed from group on expiry
+  # notify:         notify group owner; member remains with expired status (default)
+  # suspend_member: transition the member entity to SUSPENDED state
+  warn_before_expiry: P7D          # notify 7 days before expiry
+```
+
+Membership expiry produces a `MEMBER_REMOVE` audit record with `reason: membership_ttl_expired`.
+
+### 9.5 Group Policy Inheritance โ€” Nested Groups (Q39)
+
+Policy inheritance for nested groups is group_class-specific and profile-governed:
+
+| group_class | Default | Profile Override |
+|-------------|---------|----------------|
+| `tenant_boundary` | `opt_out` (parent cascades unless child excludes) | `opt_in` for minimal/dev/fsi/sovereign |
+| `resource_grouping` | Not applicable | Resource groups are tags โ€” policies target them, not inherit through them |
+| `policy_collection` | Not applicable | Policy collections use `extends` for inheritance |
+| `composite` | `opt_out` | Configurable per group |
+| `federation` | `opt_in` | Peer consent always required โ€” not configurable |
+
+```yaml
+# Nested group policy inheritance declaration
+dcm_group:
+  group_class: tenant_boundary
+  parent_group_uuid: 
+  policy_inheritance: opt_out     # governed by active Profile
+  parent_policy_exclusions:
+    - policy_uuid:          # explicitly excluded from cascading to this child
+```
+
+---
+
+## 10. System Policies โ€” Grouping Gaps
+
+| Policy | Rule |
+|--------|------|
+| `GRP-011` | The group_class set is closed โ€” system behavior is tied to declared classes only. group_subclass is open and advisory. DCM maintains a community subclass catalog as a non-authoritative reference. No validation or enforcement on subclass values. |
+| `GRP-012` | Sovereignty interaction is group_class-specific. tenant_boundary groups never span sovereignty boundaries (structural). resource_grouping groups may span sovereignty boundaries by default โ€” policy may restrict for classified resources. policy_collection and layer_grouping groups always permitted cross-sovereignty. composite groups are governed by the sovereignty rules of their most restrictive member type. |
+| `GRP-013` | Tenant decommission requires pre-decommission validation (resource state, cross-tenant relationships, compliance holds, child group resolution). Resources follow declared lifecycle policy. Child tenant_boundary groups must be resolved before parent decommission. Audit records enter post-lifecycle retention โ€” never destroyed as part of Tenant decommission. |
+| `GRP-014` | Group memberships support time-bounded validity via valid_from and valid_until. Membership expiry is enforced by the Lifecycle Constraint Enforcer. Expiry produces a MEMBER_REMOVE audit record. on_expiry action (remove, notify, suspend_member) declared per membership. Default: notify. |
+| `GRP-015` | Group policy inheritance is group_class-specific and profile-governed. tenant_boundary: opt_out (standard/prod) or opt_in (minimal/dev/fsi/sovereign). federation: always opt_in โ€” peer consent required. composite: opt_out by default. resource_grouping and policy_collection: not applicable. |
+
+
+## 11. Related Concepts
+
+- **Resource Grouping** (doc 08) โ€” original resource grouping model, now implemented via `group_class: resource_grouping`
+- **Policy Organization** (doc 14) โ€” Policy Groups and Profiles, now implemented via `group_class: policy_collection` and `group_class: policy_profile`
+- **Entity Relationships** (doc 09) โ€” cross-tenant authorized relationships between groups
+- **Universal Audit Model** (doc 16) โ€” all group changes produce audit records
+- **Ingestion Model** (doc 13) โ€” migration of existing constructs to universal groups
+
+
+---
+
+## 13. Cross-Tenant Authorization Lifecycle
+
+### 13.1 What Cross-Tenant Authorizations Are
+
+A `cross_tenant_authorization` is a DCMGroup with `group_class: cross_tenant_authorization`. It is the formal mechanism by which one Tenant grants another Tenant permission to reference, allocate from, or stake a resource that belongs to the granting Tenant.
+
+Without a cross-tenant authorization, entities in different Tenants cannot form relationships. The authorization is the bridge that enables cross-Tenant resource sharing while maintaining isolation.
+
+### 13.2 Authorization Lifecycle
+
+```yaml
+cross_tenant_authorization:
+  artifact_metadata:
+    uuid: 
+    handle: "org/cross-tenant-auth/networkops-to-appteam-vlan100"
+    version: "1.0.0"
+    status: active
+
+  granting_tenant_uuid: 
+  receiving_tenant_uuid: 
+  authorized_resources:
+    - resource_uuid: 
+      permitted_operations: [stake, read]
+    - resource_type: Network.IPAddress
+      source_pool_uuid: 
+      permitted_operations: [allocate]
+
+  # Duration
+  valid_from: 
+  valid_until:        # null = perpetual until revoked
+  auto_renew: false
+
+  # Who created this
+  granted_by_actor_uuid: 
+  granted_at: 
+```
+
+### 13.3 Who Creates Cross-Tenant Authorizations
+
+| Creator | Scenario | Authorization type |
+|---------|---------|-------------------|
+| Granting Tenant Admin | Standard: NetworkOps authorizes AppTeam to use VLAN-100 | explicit |
+| Platform Admin | Emergency or platform-managed shared infrastructure | platform_managed |
+| Pre-authorization policy | Policy automatically authorizes based on conditions | policy_auto |
+
+### 13.4 Revocation and Its Consequences
+
+When a cross-tenant authorization is revoked:
+
+```
+Authorization revoked (by granting Tenant admin, platform admin, or expiry)
+  โ”‚
+  โ–ผ All active allocations and stakes under this authorization are identified
+  โ”‚
+  โ–ผ For each active allocation / stake:
+  โ”‚   Entity enters PENDING_REVIEW state
+  โ”‚   pending_review_record created:
+  โ”‚     trigger: cross_tenant_auth.revoked
+  โ”‚     resolution_options: [re_authorize, release, migrate, escalate]
+  โ”‚
+  โ–ผ Notifications sent:
+  โ”‚   Granting Tenant Admin
+  โ”‚   Receiving Tenant Admin
+  โ”‚   Each affected resource owner
+  โ”‚   Platform Admin (if platform_managed authorization)
+  โ”‚
+  โ–ผ Resolution deadline: PT72H (configurable per profile)
+  โ”‚
+  โ””โ”€โ”€ On deadline: on_deadline_exceeded recovery policy fires
+```
+
+### 13.5 System Policies โ€” Cross-Tenant Authorization
+
+| Policy | Rule |
+|--------|------|
+| `CTX-001` | Cross-tenant relationships require an active cross-tenant authorization or a resource type declared publicly_stakeable / publicly_allocatable in its Resource Type Spec. |
+| `CTX-002` | Cross-tenant authorization revocation places all active dependent entities in PENDING_REVIEW. Revocation does not immediately release allocations. |
+| `CTX-003` | Cross-tenant authorization expiry is treated identically to explicit revocation. |
+| `CTX-004` | Platform Admin may create cross-tenant authorizations on behalf of any Tenant. All platform-managed authorizations carry a platform_managed flag and are visible in the platform admin audit log. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/16-universal-audit.md b/content/docs/data-model/16-universal-audit.md
new file mode 100644
index 0000000..0449ec7
--- /dev/null
+++ b/content/docs/data-model/16-universal-audit.md
@@ -0,0 +1,618 @@
+# DCM Data Model โ€” Universal Audit Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Storage Providers](11-storage-providers.md) | [Universal Groups](15-universal-groups.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA**
+>
+> The Data abstraction โ€” Audit Record structure and tamper-evident chain
+
+
+
+---
+
+## 1. Purpose
+
+The Universal Audit Model defines the **unconditional obligation** for every DCM component to record every change to every artifact in a uniform, tamper-evident, retention-governed audit trail. No change is silent. No change is exempt.
+
+**The four required fields for every audit record:**
+- **Date and time** โ€” when the change occurred (ISO 8601 with milliseconds)
+- **Who** โ€” the complete actor chain (immediate actor + human authorization chain)
+- **What** โ€” the subject of the change (entity UUID, type, handle)
+- **Action** โ€” what happened (closed vocabulary โ€” not free text)
+
+**The retention requirement:** Audit records must survive at least as long as any referenced resource or group is live. Policy governs what happens after all affected parties reach terminal state.
+
+---
+
+## 2. Design Principles
+
+**Universal โ€” no exceptions.** Every mutation to every DCM artifact produces an audit record. Resources, policies, layers, groups, relationships, providers, configurations, authorizations, mode 4 queries, ingestion events, rehydration events, drift events, login events โ€” all covered.
+
+**Append-only โ€” tamper-evident.** Audit records are never modified or deleted while retention obligations apply. Each record carries a hash of its own content and a reference to the previous record's hash โ€” forming a tamper-evident chain per entity.
+
+**Guaranteed delivery โ€” not guaranteed synchrony.** Audit writes use a write-ahead log (WAL) pattern โ€” the change and its audit record are written to a local WAL first, then delivered to the Audit Store asynchronously with retry. A change is never silent โ€” it may be briefly buffered, but delivery is guaranteed before the WAL is cleared.
+
+**Reference-based retention โ€” not time-based.** Audit records are retained while any referenced entity is live. Fixed time schedules (7 years, 10 years) are applied only after all referenced entities reach terminal state. A record created 20 years ago is retained unconditionally if any entity it references is still operational.
+
+**Policy-governed post-lifecycle retention.** After all referenced entities reach terminal state, policy determines how long to keep the audit record. Default is 7 years post-retirement (FSI-aligned). Organizations configure per Profile.
+
+---
+
+## 3. The Universal Audit Record
+
+```yaml
+audit_record:
+  # IDENTITY โ€” immutable once written
+  record_uuid: 
+  record_timestamp: 
+  dcm_version: 
+
+  # WHO โ€” composite actor chain
+  actor:
+    # The immediate actor that performed the change
+    immediate:
+      type: 
+      uuid: 
+      display_name: 
+      session_uuid: 
+
+    # The human who ultimately authorized this action (traceable chain)
+    authorized_by:
+      uuid: 
+      display_name: 
+      authorization_method: 
+      # direct_action:      human directly performed this
+      # request_submission: human submitted the request that triggered this
+      # policy_activation:  human activated the policy that triggered this
+      # system_policy:      DCM System Policy โ€” no individual human
+      # scheduled:          scheduled job โ€” authorized by job owner
+
+    # Links to originating context
+    request_uuid: 
+    policy_uuid: 
+    policy_version: 
+    correlation_id: 
+
+  # WHAT โ€” the subject of the change
+  subject:
+    entity_uuid: 
+    entity_type: 
+    entity_handle: 
+    entity_version_before: 
+    entity_version_after: 
+
+  # ACTION โ€” closed vocabulary
+  action: 
+
+  # ACTION DETAIL โ€” structured per action type
+  action_detail:
+    # For MODIFY, ENRICH, LOCK
+    field_changes:
+      - field: 
+        previous_value: 
+        new_value: 
+        change_reason: 
+        locked_after: 
+
+    # For STATE_TRANSITION
+    state_transition:
+      from_state: 
+      to_state: 
+      transition_reason: 
+      triggered_by: 
+
+    # For RELATIONSHIP_CREATE, RELATIONSHIP_RELEASE
+    relationship_detail:
+      related_entity_uuid: 
+      related_entity_type: 
+      relationship_type: 
+      relationship_nature: 
+      cross_tenant: 
+
+    # For MEMBER_ADD, MEMBER_REMOVE
+    membership_detail:
+      group_uuid: 
+      group_class: 
+      member_role: 
+      time_bounded: 
+      valid_until: 
+
+    # For EVALUATE (policy evaluation)
+    evaluation_detail:
+      policy_uuid: 
+      policy_version: 
+      outcome: 
+      placement_phase: 
+      missing_fields: []
+
+    # For QUERY (Mode 4 black box)
+    query_detail:
+      provider_uuid: 
+      fields_queried: []
+      result_type: 
+      outcome: 
+      audit_token: 
+      cached_result: 
+
+    # For DRIFT_DETECT
+    drift_detail:
+      drifted_fields:
+        - field: 
+          realized_value: 
+          discovered_value: 
+      drift_severity: 
+      policy_response: 
+
+  # CONTEXT
+  context:
+    tenant_uuid: 
+    request_uuid: 
+    session_uuid: 
+    profile_active: 
+    tags: [...]   # arbitrary searchable tags
+
+  # RETENTION
+  retention:
+    referenced_entities:
+      - entity_uuid: 
+        entity_type: 
+        last_known_state: 
+    retention_status: 
+    # live:           at least one referenced entity is non-retired โ€” retain unconditionally
+    # all_retired:    all referenced entities have reached terminal state
+    # policy_governed: apply governing_policy after all_retired
+    governing_policy_uuid: 
+    retain_until: 
+
+  # INTEGRITY โ€” tamper-evident hash chain
+  integrity:
+    record_hash: 
+    previous_record_hash: 
+    # Forms a per-entity hash chain โ€” inserting, modifying, or deleting a
+    # historical record breaks the chain, detectable by verification
+    chain_sequence: 
+    signed_by: 
+    signature: 
+```
+
+---
+
+## 4. Action Vocabulary
+
+The `action` field uses a closed vocabulary. Free-text actions are invalid and rejected at write time (AUD-007).
+
+| Action | Applies To | Description |
+|--------|-----------|-------------|
+| `CREATE` | All | New artifact created |
+| `MODIFY` | All | Artifact field changed |
+| `STATE_TRANSITION` | Entities, groups | Lifecycle state changed |
+| `DELETE` | All | Artifact destroyed / decommissioned |
+| `ACTIVATE` | Policies, profiles, groups | Artifact made active |
+| `DEACTIVATE` | Policies, profiles, groups | Artifact made inactive |
+| `DEPRECATE` | All | Artifact deprecated |
+| `RETIRE` | All | Artifact retired |
+| `MEMBER_ADD` | Groups | Member added to group |
+| `MEMBER_REMOVE` | Groups | Member removed from group |
+| `RELATIONSHIP_CREATE` | Entities | Relationship established |
+| `RELATIONSHIP_RELEASE` | Entities | Relationship released |
+| `AUTHORIZE` | Cross-tenant, actors | Authorization granted |
+| `REVOKE` | Cross-tenant, actors | Authorization revoked |
+| `EVALUATE` | Policies | Policy evaluated (with outcome) |
+| `ENRICH` | Fields | Field enriched by policy, layer, or Mode 4 provider |
+| `LOCK` | Fields | Field locked (override: immutable set) |
+| `HOLD_PLACE` | Resources | Resource hold placed with provider |
+| `HOLD_CONFIRM` | Resources | Resource hold confirmed |
+| `HOLD_RELEASE` | Resources | Resource hold released |
+| `DRIFT_DETECT` | Entities | Drift detected between Realized and Discovered |
+| `DRIFT_RESOLVE` | Entities | Drift resolved |
+| `INGEST` | Entities | Entity ingested (brownfield or V1 migration) |
+| `PROMOTE` | Entities | Ingested entity promoted to full lifecycle |
+| `EXPIRE` | Entities | Lifecycle time constraint expiry action fired |
+| `REHYDRATE` | Entities | Rehydration requested |
+| `QUERY` | Mode 4 | Black box query sent and result received |
+| `DISCOVER` | Entities | Discovery cycle completed |
+| `LOGIN` | Actors | Actor authentication event |
+| `LOGOUT` | Actors | Actor session ended |
+| `CONFIG_CHANGE` | Platform | DCM configuration changed (profile activated, etc.) |
+
+---
+
+## 5. The "Who" โ€” Composite Actor Record
+
+The `who` in an audit record is not a single identity โ€” it is a **composite actor chain** tracing from the immediate action back to the human who ultimately authorized it.
+
+### 5.1 Actor Types
+
+| Type | Example | authorized_by |
+|------|---------|--------------|
+| `human` | Platform admin changes a policy | Self |
+| `system_component` | Lifecycle Constraint Enforcer fires expiry | Policy that set the constraint โ†’ human who activated policy |
+| `policy` | Transformation Policy enriches a field | Human who activated the policy |
+| `provider` | Service Provider updates Realized State | Dispatch that triggered it โ†’ human who submitted request |
+| `scheduled_job` | Discovery cycle runs | Owner of the scheduled job |
+| `mode4_provider` | Black box enriches a field | Policy that triggered the query โ†’ human who activated policy |
+
+### 5.2 System-Initiated Actions
+
+For system-initiated changes where there is no immediate human actor, the authorization chain traces back as far as possible:
+
+```yaml
+actor:
+  immediate:
+    type: system_component
+    uuid: 
+    display_name: "Lifecycle Constraint Enforcer"
+  authorized_by:
+    uuid: null   # no specific human โ€” system policy
+    display_name: "DCM System Policy LTC-003"
+    authorization_method: system_policy
+  policy_uuid: 
+  policy_version: "1.0.0"
+```
+
+---
+
+## 6. Retention Model
+
+### 6.1 Reference-Based Retention
+
+Audit records are retained based on the lifecycle state of all referenced entities โ€” not on a fixed time schedule.
+
+```
+Audit record created
+  โ”‚  retention_status: live (all referenced entities tracked)
+  โ”‚
+  โ–ผ  [continuous monitoring]
+  โ”‚
+  As referenced entities change state:
+  โ”‚  DCM updates last_known_state on each referenced_entity
+  โ”‚  When all reach terminal state โ†’ retention_status: all_retired
+  โ”‚
+  โ–ผ  retention_status: all_retired
+  โ”‚  Governing policy determines retain_until date
+  โ”‚  retention_status: policy_governed
+  โ”‚
+  โ–ผ  retain_until reached
+     Audit record eligible for destruction / archival
+```
+
+### 6.2 Post-Lifecycle Retention Options
+
+| Policy Setting | Meaning | Default Profile |
+|---------------|---------|----------------|
+| `destroy_immediately` | Destroy when last entity retires | (not available in standard+) |
+| `retain_for: P90D` | 90 days post-retirement | dev profile |
+| `retain_for: P3Y` | 3 years post-retirement | standard profile |
+| `retain_for: P7Y` | 7 years post-retirement | prod, fsi profiles (DEFAULT) |
+| `retain_for: P10Y` | 10 years post-retirement | sovereign profile |
+| `retain_indefinitely` | Never destroy | optional โ€” maximum compliance |
+| `archive_after: P1Y` | Move to cold storage 1 year post-retirement | configurable |
+
+### 6.3 Retention Shorter Than Referenced Entity Lifetime
+
+This cannot happen. While any referenced entity is live, `retention_status: live` and the record is retained unconditionally. The retention policy only applies **after** all referenced entities reach terminal state. A 90-day retention policy means "90 days after the last referenced entity is retired" โ€” not "90 days after creation."
+
+---
+
+## 7. Two-Stage Audit โ€” Synchronous Commit + Async Enrichment
+
+### 7.1 The Design
+
+DCM uses a **two-stage audit model** that provides synchronous durability guarantees without impacting request processing performance.
+
+```
+Stage 1 โ€” Commit Log (synchronous, in critical path, < 1ms)
+Stage 2 โ€” Audit Store (asynchronous, out of critical path, full record)
+```
+
+**Stage 1** writes a minimal Commit Log entry synchronously using consensus protocol (Raft). The write is confirmed when a quorum of Commit Log replicas acknowledges it. The operation returns success after Stage 1 confirms โ€” not after the Audit Store write.
+
+**Stage 2** runs asynchronously via the Audit Forward Service: enriches the minimal Commit Log entry into a full audit_record, computes the hash chain, and writes to the Audit Store with retry.
+
+### 7.2 Stage 1 โ€” Commit Log Entry (minimal, ultra-fast)
+
+```yaml
+commit_log_entry:
+  entry_uuid:             # links to full audit_record in Stage 2
+  sequence:            # monotonically increasing โ€” global ordering
+  timestamp:    # authoritative audit timestamp
+  entity_uuid: 
+  entity_type: 
+  action: 
+  actor_uuid:             # immediate actor only
+  request_uuid:           # if applicable
+  tenant_uuid: 
+  change_fingerprint: 
+  # change_fingerprint enables Stage 2 to verify full record matches Stage 1
+
+  status: 
+  forwarded_at:       # populated by Audit Forward Service
+  audit_record_uuid:      # UUID of full audit_record in Audit Store
+```
+
+**Stage 1 guarantees:** the change happened, at this exact time, this actor performed it, this entity was affected, this action was taken. Full detail follows in Stage 2.
+
+**Commit Log quorum write** (distributed deployment):
+```
+Write confirmed when quorum acknowledges:
+  โ”œโ”€โ”€ Replica 1 (local node)     โ†’ ACK โ”€โ”
+  โ”œโ”€โ”€ Replica 2 (different node) โ†’ ACK โ”€โ”ค quorum (2/3) โ€” write confirmed
+  โ””โ”€โ”€ Replica 3 (different zone) โ†’ (async best-effort)
+```
+
+### 7.3 Stage 2 โ€” Audit Forward Service
+
+```
+Audit Forward Service reads pending_forward Commit Log entries
+  โ”‚
+  โ”œโ”€โ”€ Retrieve full change context from DCM internal state
+  โ”‚   (field values before/after, complete actor chain, relationship detail)
+  โ”‚
+  โ”œโ”€โ”€ Construct complete audit_record (full structure per Section 3)
+  โ”‚   - Compute record_hash + previous_record_hash (hash chain)
+  โ”‚   - Set retention.referenced_entities
+  โ”‚
+  โ”œโ”€โ”€ Write to Audit Store
+  โ”‚   โ†’ Success: mark commit_log_entry status: forwarded
+  โ”‚   โ†’ Failure: retry with exponential backoff
+  โ”‚              N retries exhausted โ†’ status: forward_failed, alert admin
+  โ”‚
+  โ””โ”€โ”€ Commit Log entry eligible for cleanup after:
+      status: forwarded AND entry age > Commit Log retention window
+```
+
+### 7.4 Recoverability
+
+| Failure Scenario | Recovery |
+|-----------------|---------|
+| DCM crashes after Stage 1, before Stage 2 | On restart, Audit Forward Service replays all `pending_forward` entries |
+| Audit Store unavailable | Commit Log accumulates; Audit Forward Service retries when Audit Store recovers |
+| Stage 2 fails mid-enrichment | Commit Log entry remains `pending_forward`; retried from committed Stage 1 data |
+| Commit Log quorum unavailable | Stage 1 fails โ†’ operation aborted โ†’ no silent change |
+| All Commit Log replicas lost | Recovery from replica backup; forward_failed entries investigated |
+
+### 7.5 Performance Characteristics
+
+| Component | Latency | In Critical Path? |
+|-----------|---------|-----------------|
+| Stage 1 โ€” Commit Log quorum write | < 1ms (local NVMe + Raft) | Yes |
+| Stage 2 โ€” Audit Store write | 5โ€“50ms (network + indexing) | No |
+| Full audit record visible | Seconds to minutes after Stage 1 | No |
+
+**The Stage 1 timestamp is the authoritative audit timestamp.** Stage 2 write time is when the full record became queryable โ€” not when the change occurred.
+
+---
+
+---
+
+## 8. Tamper-Evidence โ€” Hash Chain
+
+Each audit record carries:
+- `record_hash` โ€” SHA-256 of the record's content
+- `previous_record_hash` โ€” hash of the immediately preceding audit record for this entity
+- `chain_sequence` โ€” monotonically increasing integer per entity
+
+Together these form a **per-entity hash chain**. To verify integrity:
+
+```
+For each entity:
+  Load all audit records ordered by chain_sequence
+  For each record:
+    Verify record_hash == SHA-256(record content)
+    Verify previous_record_hash == record_hash of sequence N-1
+  If any verification fails:
+    โ†’ Chain broken โ€” tampering detected
+    โ†’ Alert dispatched to security and platform admin
+    โ†’ Affected records flagged in audit dashboard
+```
+
+Inserting, modifying, or deleting any historical record breaks the chain at that point and all subsequent records for that entity. The breach is detectable at the next verification run.
+
+---
+
+## 9. DCM System Policies
+
+| Policy | Rule |
+|--------|------|
+| `AUD-001` | Every modification to any DCM artifact must produce a Commit Log entry synchronously before the operation returns success. Commit Log write failure aborts the operation โ€” no silent unaudited changes. |
+| `AUD-002` | Audit records are append-only and immutable. No audit record may be modified or deleted while retention_status is `live` or `policy_governed`. |
+| `AUD-003` | Audit records must survive at least as long as any referenced entity is in a non-retired/non-decommissioned state (retention_status: live). |
+| `AUD-004` | Post-lifecycle retention is governed by policy. Default is `retain_for: P7Y` after all referenced entities reach terminal state. |
+| `AUD-005` | The actor field must identify both the immediate actor and the authorized_by human actor chain to the extent traceable. |
+| `AUD-006` | Audit records must carry a `record_hash` and `previous_record_hash` forming a tamper-evident hash chain per entity. |
+| `AUD-007` | The action field must use the closed vocabulary โ€” free-text action fields are invalid and must be rejected at write time. |
+| `AUD-008` | Audit Store implementations must support queries by: entity_uuid, actor_uuid, action, timestamp range, tenant_uuid, request_uuid, and retention_status. |
+| `AUD-009` | The Audit Forward Service must deliver all Commit Log entries to the Audit Store with exponential backoff retry. Commit Log entries may only be cleared after both: (a) Audit Store confirms receipt AND (b) entry has aged beyond the Commit Log retention window. |
+| `AUD-010` | Hash chain verification must be available as a first-class DCM operation. Chain breaks must trigger immediate security alerts. |
+| `AUD-011` | On DCM restart, the Audit Forward Service must replay all `status: pending_forward` Commit Log entries before accepting new operations. |
+| `AUD-012` | The Commit Log must use consensus protocol (Raft or equivalent) with quorum writes. A write is confirmed durable only when a quorum of replicas acknowledges it. |
+| `AUD-013` | The Stage 1 timestamp in the Commit Log is the authoritative audit timestamp. Stage 2 enrichment timestamps record when the full audit record became queryable โ€” not when the change occurred. |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should hash chain verification run continuously or on-demand? | Security | โœ… Resolved โ€” three levels: continuous write (chain construction), scheduled sweep (weekly to 6-hourly per profile), on-demand (operator-triggered); failure โ†’ security alert + integrity incident (AUD-014) |
+| 2 | Should the WAL have a configurable maximum capacity, and what happens when it is reached? | Availability | โœ… Resolved โ€” configurable max capacity; alert_and_continue (standard/prod); reject_new_ops (fsi/sovereign); backpressure at 75%/90%; P7D max age escalation (AUD-015) |
+| 3 | Should audit records for system-initiated changes (no human actor) be flagged differently in the dashboard? | Operational | โœ… Resolved โ€” actor.type: human/service_account/system; system_actor block with component/trigger/policy; full audit records; enables filtering in queries and dashboards (AUD-016) |
+| 4 | How does hash chain verification interact with distributed DCM deployments where audit records may be written to multiple regional stores? | Architecture | โœ… Resolved โ€” per-instance hash chains; daily Merkle root federation integrity proof at Hub DCM; cross-instance queries via parallel chains + correlation_id (AUD-017) |
+
+---
+
+## 11. Related Concepts
+
+- **Audit, Provenance, and Observability** (doc 12) โ€” three distinct concerns; this document covers the audit concern in full
+- **Field-Level Provenance** โ€” data lineage embedded in every payload; separate from audit records
+- **Storage Providers** (doc 11) โ€” Audit Store contract: append-only, WAL delivery, hash chain, retention tracking
+- **Universal Groups** (doc 15) โ€” all group changes produce audit records per this model
+- **Policy Organization** (doc 14) โ€” policy activation, shadow evaluation, and Mode 4 queries all produce audit records
+
+
+## 10. Universal Audit Gap Resolutions
+
+### 10.1 Hash Chain Verification Modes (Q1)
+
+Hash chain verification operates at three independent levels:
+
+```yaml
+hash_chain_verification:
+  continuous_write: true              # always โ€” hash computed on every write (chain construction)
+
+  scheduled_sweep:
+    enabled: true
+    schedule:
+      standard: "0 2 * * 0"          # weekly
+      prod: "0 2 * * *"              # daily
+      fsi: "0 */6 * * *"            # every 6 hours
+      sovereign: "0 */6 * * *"
+
+  on_demand:
+    enabled: true                     # always available to platform admin
+    max_range: P365D                  # maximum time range per verification run
+
+  on_verification_failure:
+    action: alert_security_team
+    halt_new_writes: false            # do not halt โ€” alert and investigate
+    # Halting writes is itself a security risk; alerting is the correct response
+    create_integrity_incident: true
+```
+
+Continuous verification is part of chain construction (not a separate process). Scheduled sweep catches tampering between writes. On-demand is available for incident investigation, compliance audit, and pre-report verification.
+
+### 10.2 Commit Log Maximum Capacity (Q2)
+
+The Commit Log has a configurable maximum capacity with a declared overflow policy โ€” different profiles have different trade-offs between availability and audit completeness.
+
+```yaml
+commit_log_capacity:
+  max_size: 10Gi                      # configurable; profile-governed
+  max_age: P7D                        # records older than 7d escalate regardless
+  on_capacity_exceeded:
+    profile_defaults:
+      minimal: alert_and_continue     # availability priority
+      dev: alert_and_continue
+      standard: alert_and_continue
+      prod: alert_and_continue
+      fsi: reject_new_ops             # audit completeness priority
+      sovereign: reject_new_ops
+  warn_at_percent: 75                 # alert at 75% capacity
+  urgent_at_percent: 90              # urgent alert at 90%
+```
+
+**`reject_new_ops` for fsi/sovereign:** Operating without a functional audit trail is a compliance violation in regulated environments. Stopping operations is preferable to operating unaudited โ€” same principle as Commit Log quorum unavailability โ†’ abort operation (STO-002).
+
+### 10.3 System-Initiated Audit Records (Q3)
+
+Audit records for system-initiated changes declare `actor.type: system` with a `system_actor` block identifying the DCM component, trigger, and authorizing policy.
+
+```yaml
+audit_record:
+  action: REHYDRATE
+  actor:
+    uuid: 
+    type: system                      # human | service_account | system
+    system_actor:
+      component: lifecycle_constraint_enforcer
+      trigger: entity_ttl_expired
+      entity_uuid: 
+      policy_uuid: 
+    authorization: implicit           # implicit = authorized by DCM architecture
+                                      # explicit = authorized by named policy
+```
+
+System actor records are full audit records โ€” they appear in all queries and compliance reports. `actor.type` enables filtering:
+- `filter: actor.type = human` โ†’ all human-initiated changes
+- `filter: actor.type = system` โ†’ all automated lifecycle operations
+- `filter: actor.type = service_account` โ†’ all API/programmatic changes
+
+### 10.4 Distributed Hash Chain Integrity (Q4)
+
+In distributed DCM deployments (Hub + Regional + Sovereign DCMs), each instance maintains its own independent hash chain. Federation-level integrity is provided by daily Merkle root proofs.
+
+```yaml
+distributed_hash_chain:
+  model: per_instance               # each DCM instance has its own chain
+  instance_chain:
+    chain_id:    # chain scoped to this instance
+
+  federation_integrity_proof:
+    enabled: true
+    schedule: "0 0 * * *"           # daily
+    mechanism: merkle_root
+    # Hub DCM collects chain tip hashes from all Regional DCMs
+    # Computes Merkle root โ†’ stores as federation_integrity_record
+    # Any chain break in any instance is detectable against this root
+    stored_at: hub_dcm_audit_store
+    signed_by: hub_dcm_service_account
+```
+
+**Cross-instance queries:** Records from different chains are presented as parallel chains with cross-references via `correlation_id`. Not merged into a single chain โ€” each instance's chain remains independently verifiable. Federation-level verification requires Hub DCM connectivity; per-instance verification is always available locally.
+
+### 10.5 System Policies โ€” Universal Audit Gaps
+
+| Policy | Rule |
+|--------|------|
+| `AUD-014` | Hash chain verification operates at three levels: continuous (hash computed on every write), scheduled sweep (weekly to every 6 hours per profile), and on-demand (operator-triggered for any time range). Verification failure triggers a security alert and integrity incident. New audit writes continue โ€” halting writes is itself a security risk. |
+| `AUD-015` | The Commit Log has configurable maximum capacity with a declared overflow policy: alert_and_continue (standard/prod) or reject_new_ops (fsi/sovereign). Backpressure alerts fire at 75% and 90% capacity. Records older than P7D trigger escalation regardless of capacity. |
+| `AUD-016` | Audit records for system-initiated changes declare actor.type: system with a system_actor block identifying the DCM component, trigger, and authorizing policy. System actor records are full audit records appearing in all queries and compliance reports. actor.type enables filtering between human, service_account, and system-initiated changes. |
+| `AUD-017` | In distributed DCM deployments, each instance maintains its own independent hash chain scoped to that instance. Federation-level integrity is maintained via daily Merkle root proofs computed from all instance chain tips, stored at the Hub DCM. Cross-instance audit queries present parallel chains with cross-references via correlation_id. |
+
+
+## 9a. Audit vs Observability โ€” The Definitive Distinction (Q16)
+
+Audit and Observability are separate components with separate storage contracts, separate consumers, and opposite fundamental trade-offs. They cannot be combined without violating one contract or the other.
+
+### 9a.1 Comparison
+
+| Dimension | Audit | Observability |
+|-----------|-------|--------------|
+| **Purpose** | Immutable record of WHAT HAPPENED and WHO authorized it | Real-time visibility into SYSTEM HEALTH and PERFORMANCE |
+| **Primary consumers** | Auditors, compliance, security, legal, regulators | SREs, platform engineers, operators, dashboards |
+| **Write rate** | Low โ€” one record per action | Very high โ€” multiple per second per component |
+| **Retention** | Very long โ€” P7Y+ (compliance-driven) | Short โ€” days to months (operational) |
+| **Mutability** | Never โ€” append-only, hash-chained | Downsampling and aggregation acceptable |
+| **Accuracy** | 100% required โ€” no sampling | Statistical sampling acceptable |
+| **Compliance grade** | Required | Not required |
+| **Cost per event** | High โ€” hash chain computation | Low โ€” time series append |
+| **Failure behavior** | Missing audit = compliance violation | Missing observability = operational inconvenience |
+| **Query model** | Point-in-time, actor-based, compliance reports | Time-series, rate queries, anomaly detection |
+| **Data model** | Closed 30-action vocabulary, structured | Open schema โ€” any component emits any metric |
+| **Storage type** | Audit Store (specialized sub-type) | Time-series database (Prometheus, InfluxDB) |
+
+### 9a.2 The Relationship
+
+Observability data MAY reference audit record UUIDs for correlation โ€” a spike in error rate can link to audit records from that time window. But they live in separate stores with separate contracts.
+
+- **Audit** answers: "What happened and who authorized it?"
+- **Observability** answers: "Is the system healthy and how is it performing?"
+
+These are different questions requiring different storage architectures.
+
+### 9a.3 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `AUD-013` | Audit and Observability are separate components with separate storage contracts, consumers, and failure behaviors. Audit is compliance-grade, append-only, hash-chained, long-retention. Observability is operational, time-series, high-throughput, short-retention. They serve different consumers and cannot be combined without violating one contract or the other. Observability data may reference audit record UUIDs for correlation but is stored separately. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/17-deployment-redundancy.md b/content/docs/data-model/17-deployment-redundancy.md
new file mode 100644
index 0000000..fb21c54
--- /dev/null
+++ b/content/docs/data-model/17-deployment-redundancy.md
@@ -0,0 +1,791 @@
+# DCM Data Model โ€” Deployment and Redundancy Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit Model](16-universal-audit.md) | [Policy Organization](14-policy-profiles.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + PROVIDER**
+>
+> Data: deployment specification. Provider: Storage Provider redundancy
+
+
+
+---
+
+> **Operational guidance:** GitOps the disaster recovery runbook and RTO/RPO tables are in [Operational Reference](41-operational-reference.md) Section 3.
+
+## 1. Purpose
+
+Every DCM component, every data store, and every capability is designed for redundancy by default. Redundancy is not an add-on or an advanced configuration โ€” it is the baseline operational posture for all profiles above `minimal`.
+
+**The minimal profile** provides single-instance deployment for home lab and evaluation simplicity. All other profiles assume redundancy as the floor. The transition from `minimal` to `dev` is the transition from "works on my laptop" to "survives a node failure."
+
+**Everything in DCM runs as a container in a pod.** No bare-metal DCM components. No special-case deployment paths. Every DCM component follows the same container lifecycle, the same health check model, the same rolling update pattern. DCM runs on Kubernetes โ€” and manages Kubernetes.
+
+**DCM is self-hosting.** DCM's own deployment is expressible as DCM resources. DCM can manage its own lifecycle, detect drift in its own components, and rehydrate its own deployment from Git. This is the ultimate expression of the data center repave use case โ€” DCM restoring itself.
+
+---
+
+## 2. Design Principles
+
+**Redundant by default.** Every component, store, and capability has a redundancy model. The `minimal` profile sets `replicas: 1`. Every other profile sets `replicas: >= 3` with quorum writes and anti-affinity scheduling.
+
+**Everything containerized.** All DCM components run as containers in Kubernetes pods. No exceptions. This gives a consistent deployment model, rolling updates, health checks, and self-healing as first-class properties.
+
+**Profile-governed redundancy.** Replica counts, quorum thresholds, geo-replication, and anti-affinity requirements are declared by the active Profile โ€” not by per-component configuration. Activating a Profile configures redundancy for the entire deployment.
+
+**Self-hosting.** DCM's own deployment is a DCM resource. DCM manages itself through the same model it uses to manage customer infrastructure.
+
+**Stateless control plane.** All DCM control plane components are stateless โ€” all state lives in external stores. Any component instance can fail and be replaced without data loss. State recovery means restarting a pod โ€” not restoring a database.
+
+**Quorum writes for durability.** All durable stores use quorum writes โ€” a write is confirmed only when a majority of replicas acknowledge it. This ensures durability even if a minority of replicas fail simultaneously.
+
+---
+
+## 3. Component Redundancy Model
+
+### 3.1 Control Plane Components
+
+All control plane components are stateless, horizontally scalable, and deployed as Kubernetes Deployments with configurable replica counts.
+
+```yaml
+component_redundancy:
+  component: request_payload_processor   # same model for all components
+  deployment_type: kubernetes_deployment
+  replicas: 3                    # set by active Profile
+  affinity:
+    anti_affinity: required      # pods spread across nodes
+    zone_spread: preferred       # prefer spreading across availability zones
+  disruption_budget:
+    min_available: 2             # always keep 2 running during rolling updates
+  health_check:
+    liveness:
+      path: /healthz
+      interval_seconds: 10
+      failure_threshold: 3
+    readiness:
+      path: /readyz
+      interval_seconds: 5
+      failure_threshold: 2
+  rolling_update:
+    strategy: RollingUpdate
+    max_unavailable: 0           # never take a pod down before replacement is ready
+    max_surge: 1
+```
+
+**Control plane components:**
+
+| Component | Stateless? | Replica Model |
+|-----------|-----------|--------------|
+| API Gateway | Yes | Deployment + HorizontalPodAutoscaler |
+| Request Payload Processor | Yes | Deployment |
+| Policy Engine (OPA) | Yes | Deployment + PolicyBundle sidecar |
+| Placement Engine | Yes | Deployment |
+| Service Catalog | Yes | Deployment |
+| IDM / IAM | Yes | Deployment (external IdP recommended) |
+| Audit Forward Service | Yes | Deployment (1 active + 1 standby) |
+| Lifecycle Constraint Enforcer | Yes | Deployment (leader election) |
+| Drift Detection | Yes | Deployment (leader election for scheduling) |
+| Resource Discovery | Yes | Deployment (leader election) |
+| Message Bus Router | Yes | Deployment |
+
+**Leader election** for scheduler-type components (Lifecycle Constraint Enforcer, Drift Detection, Resource Discovery): multiple replicas run but only one holds the leader lease at a time. On leader failure, a replica acquires the lease within seconds.
+
+### 3.2 Data Store Redundancy
+
+All DCM data stores run as containers. Each store type has a declared replication and quorum model.
+
+#### Commit Log
+
+```yaml
+commit_log:
+  replicas: 3
+  write_quorum: 2               # confirmed durable when 2/3 replicas acknowledge
+  read_quorum: 1                # any replica can serve reads
+  affinity:
+    zone_spread: required       # replicas MUST span availability zones
+  implementation: etcd          # or equivalent consensus store
+  # etcd is purpose-built for this pattern: Raft consensus, quorum writes,
+  # sub-millisecond local writes, proven in Kubernetes itself
+```
+
+The Commit Log uses consensus protocol (Raft/equivalent). A write is confirmed when the quorum acknowledges โ€” ensuring durability even if a minority of replicas fail simultaneously.
+
+#### GitOps Store (Intent, Requested, Layers, Policies)
+
+```yaml
+gitops_store:
+  implementation: gitea          # or equivalent self-hosted Git
+  replicas: 3
+  replication_mode: active_active  # any node can accept writes
+  write_quorum: 2
+  backup:
+    enabled: true
+    schedule: "0 */6 * * *"    # every 6 hours
+    retention: 30d
+```
+
+#### Event Stream Store (Realized, Discovered, Audit Events)
+
+```yaml
+event_stream:
+  implementation: kafka          # or equivalent
+  brokers: 3
+  replication_factor: 3
+  min_insync_replicas: 2        # minimum replicas that must acknowledge a write
+  partitions: 12                # enables parallel consumption
+  retention:
+    bytes: -1                   # unlimited โ€” retention governed by policy
+    ms: -1                      # unlimited
+```
+
+#### Audit Store
+
+```yaml
+audit_store:
+  implementation: elasticsearch  # or equivalent โ€” optimized for queryable retention
+  replicas: 3
+  primary_shards: 5
+  replica_shards: 1             # each shard has 1 replica = 2 copies total
+  geo_replicated: true          # in prod/fsi/sovereign profiles
+  append_only_enforced: true    # storage layer enforces immutability
+```
+
+#### Search Index (Non-Authoritative)
+
+```yaml
+search_index:
+  implementation: elasticsearch
+  replicas: 2                   # lower redundancy โ€” can rebuild from Git
+  rebuild_from_git: true        # on data loss, rebuild from authoritative stores
+```
+
+### 3.3 Container Specification
+
+Every DCM component pod follows a common security and resource model:
+
+```yaml
+pod_spec:
+  security_context:
+    run_as_non_root: true
+    run_as_user: 65534           # nobody
+    run_as_group: 65534
+    fs_group: 65534
+    seccomp_profile:
+      type: RuntimeDefault
+    capabilities:
+      drop: [ALL]
+
+  containers:
+    - name: 
+      image: ghcr.io/dcm-project/:
+      image_pull_policy: IfNotPresent
+      security_context:
+        read_only_root_filesystem: true
+        allow_privilege_escalation: false
+      resources:
+        requests:
+          cpu: 500m
+          memory: 512Mi
+        limits:
+          cpu: 2000m
+          memory: 2Gi
+      liveness_probe: 
+      readiness_probe: 
+      volume_mounts:
+        - name: tmp
+          mount_path: /tmp       # writable temp โ€” no other writable paths
+
+  volumes:
+    - name: tmp
+      empty_dir: {}
+```
+
+---
+
+## 4. Redundancy by Profile
+
+Profile activation configures redundancy for the entire deployment. Organizations do not configure replica counts individually โ€” they activate a profile.
+
+```yaml
+# Redundancy matrix per profile
+redundancy_matrix:
+  minimal:
+    control_plane_replicas: 1
+    store_replicas: 1
+    write_quorum: false
+    zone_spread: false
+    geo_replication: false
+    anti_affinity: false
+    note: "Single-instance. No redundancy. Home lab and evaluation only."
+
+  dev:
+    control_plane_replicas: 1
+    store_replicas: 1
+    write_quorum: false
+    zone_spread: false
+    geo_replication: false
+    anti_affinity: false
+    note: "Single-instance with backup. Basic resilience for dev environments."
+
+  standard:
+    control_plane_replicas: 3
+    store_replicas: 3
+    write_quorum: 2             # 2 of 3
+    zone_spread: preferred
+    geo_replication: false
+    anti_affinity: required
+    note: "Production baseline. Survives single node or zone failure."
+
+  prod:
+    control_plane_replicas: 3
+    store_replicas: 3
+    write_quorum: 2
+    zone_spread: required
+    geo_replication: true
+    anti_affinity: required
+    sla_disruption_budget: "always 2 replicas available"
+    note: "Production with SLA. Geo-replicated stores."
+
+  fsi:
+    control_plane_replicas: 5
+    store_replicas: 5
+    write_quorum: 3             # 3 of 5
+    zone_spread: required
+    geo_replication: true
+    anti_affinity: required
+    audit_store_replicas: 5
+    note: "FSI-grade. Higher quorum threshold. Compliance-grade audit."
+
+  sovereign:
+    control_plane_replicas: 5
+    store_replicas: 5
+    write_quorum: 3
+    zone_spread: required
+    geo_replication: true        # within sovereignty boundary only
+    anti_affinity: required
+    air_gap_backup: true
+    sovereignty_boundary_enforced: true
+    note: "Maximum. All geo-replication within sovereignty boundary."
+```
+
+---
+
+## 5. The DCM Deployment Specification
+
+The DCM deployment itself is a DCM resource โ€” declared in YAML, stored in Git, governed by Policy, subject to the same four-state lifecycle as any other resource.
+
+```yaml
+dcm_deployment:
+  artifact_metadata:
+    uuid: 
+    handle: "deployments/primary/dcm-control-plane"
+    version: "1.0.0"
+    status: active
+
+  profile: system/profile/standard
+  kubernetes_namespace: dcm-system
+
+  # Redundancy โ€” set by active Profile, overridable per component
+  redundancy:
+    control_plane:
+      replicas: 3
+      affinity:
+        anti_affinity: required
+        zone_spread: preferred
+      disruption_budget:
+        min_available: 2
+
+    stores:
+      commit_log:
+        replicas: 3
+        write_quorum: 2
+        implementation: etcd
+      gitops_store:
+        replicas: 3
+        write_quorum: 2
+        implementation: gitea
+      event_stream:
+        brokers: 3
+        replication_factor: 3
+        min_insync_replicas: 2
+        implementation: kafka
+      audit_store:
+        replicas: 3
+        geo_replicated: false     # standard profile default
+        implementation: elasticsearch
+      search_index:
+        replicas: 2
+        implementation: elasticsearch
+
+  # Provider health configuration
+  providers:
+    health_check_interval_seconds: 30
+    unhealthy_threshold: 3
+    automatic_failover: true
+
+  # Self-hosting: DCM manages its own deployment
+  self_managed: true
+  drift_detection_enabled: true
+  rehydration_enabled: true
+  # DCM detects if its own components drift from declared spec
+  # and can rehydrate (redeploy) from this declaration
+```
+
+---
+
+## 6. Self-Hosting โ€” DCM Managing Itself
+
+DCM's own deployment is managed through the same model it uses to manage customer infrastructure. This is the **self-hosting principle** โ€” DCM eats its own cooking.
+
+### 6.1 What Self-Hosting Means
+
+- DCM control plane components are defined as Resource Entities in DCM
+- DCM data stores are defined as Storage Provider resources in DCM
+- DCM's own Policy Groups govern DCM's own deployment constraints
+- DCM runs drift detection on its own components โ€” a component running the wrong image version is drift
+- DCM can rehydrate its own deployment from the `dcm_deployment` declaration in Git
+
+### 6.2 The Bootstrap Problem
+
+DCM cannot manage itself before it exists. The bootstrap sequence:
+
+```
+1. Bootstrap installer deploys minimal DCM (single instance, no redundancy)
+   from a declarative bootstrap manifest
+   โ”‚
+2. Bootstrap DCM reads the target dcm_deployment declaration from Git
+   โ”‚
+3. Bootstrap DCM provisions itself to the target state:
+   - Scales from 1 to N replicas
+   - Provisions redundant stores
+   - Configures quorum
+   โ”‚
+4. Bootstrap instance hands off to the now-redundant DCM
+   โ”‚
+5. DCM manages its own lifecycle from this point forward
+```
+
+The bootstrap manifest is the only thing that exists outside DCM's management scope. It is minimal by design โ€” just enough to get DCM running.
+
+### 6.3 Self-Hosted Drift Detection
+
+DCM continuously compares its own running state against the `dcm_deployment` declaration:
+
+| Drift Type | Example | Response |
+|-----------|---------|---------|
+| Wrong image version | Component running v1.1.0, declared v1.2.0 | Rolling update triggered |
+| Wrong replica count | 2 replicas running, declared 3 | Scale-up triggered |
+| Wrong resource limits | Component using more than declared limits | Alert + potential eviction |
+| Store replication mismatch | Store has 2 replicas, declared 3 | Replication repair triggered |
+
+### 6.4 The Repave Scenario
+
+The ultimate test of self-hosting: DCM is lost entirely (ransomware, catastrophic failure). Recovery:
+
+```
+1. Deploy bootstrap installer to new Kubernetes cluster
+   โ”‚
+2. Bootstrap DCM reads dcm_deployment declaration from Git backup
+   โ”‚
+3. DCM provisions itself โ€” full redundant deployment
+   โ”‚
+4. DCM reads all resource declarations from Git
+   โ”‚
+5. DCM rehydrates customer workloads in dependency order
+   โ”‚
+6. Drift detection validates recovered state matches declared state
+```
+
+The recovery time is bounded by infrastructure provisioning speed โ€” not by backup restoration or manual configuration. Everything is code. Everything is declarative. Everything is in Git.
+
+---
+
+## 7. Commit Log Redundancy โ€” Two-Stage Audit Integration
+
+The Commit Log is the synchronous component of the two-stage audit model. In a distributed deployment, "synchronous durable write" means quorum acknowledgment:
+
+```
+DCM component initiates change
+  โ”‚
+  โ–ผ
+Write to Commit Log (Raft consensus)
+  โ”‚  Propose to leader
+  โ”‚  Leader replicates to followers
+  โ”‚  Write confirmed when quorum (2/3 or 3/5) acknowledge
+  โ”‚
+  โ”œโ”€โ”€ Replica 1 (local node)     โ†’ ACK โ”€โ”
+  โ”œโ”€โ”€ Replica 2 (different node) โ†’ ACK โ”€โ”ค quorum reached
+  โ””โ”€โ”€ Replica 3 (different zone) โ†’ ACK โ”€โ”˜
+  โ”‚
+  โ–ผ  Operation returns success (< 1ms typical with NVMe)
+  โ”‚
+  โ–ผ  [async โ€” Audit Forward Service]
+Read from any surviving Commit Log replica
+  โ”‚  Enrich โ†’ write to Audit Store
+  โ””โ”€โ”€ Clear Commit Log entry after Audit Store confirms
+```
+
+**Failure scenarios and recovery:**
+
+| Failure | During Stage 1 | Effect |
+|---------|---------------|--------|
+| Single replica fails before quorum | Quorum still achievable | No impact |
+| Majority fail before quorum | Commit Log unavailable | Operation aborted โ€” no silent change |
+| Leader fails after quorum | New leader elected (seconds) | In-flight writes complete on new leader |
+| All replicas fail after quorum | Audit Forward Service reads from backup | Full recovery on restart |
+| Audit Store unavailable | Commit Log accumulates | Forward resumes when Audit Store recovers |
+
+---
+
+## 8. Network Architecture
+
+> **Full internal auth specification:** See [Internal Component Authentication](36-internal-component-auth.md) for component identity model, Internal CA, bootstrap protocol, and ICOM-001โ€“ICOM-009 system policies.
+
+### 8.1 Service Mesh
+
+All DCM component-to-component communication uses a service mesh (Istio or equivalent):
+- mTLS everywhere (RFC 8446 TLS 1.3 + RFC 5280 X.509) โ€” no plaintext internal communication
+- Traffic policies enforced at mesh level
+- Observability: traces, metrics, logs for all inter-component calls
+- Circuit breaking: prevent cascade failures
+
+### 8.2 Ingress
+
+External traffic enters through a redundant Ingress layer:
+
+```
+External clients
+  โ”‚
+  โ–ผ
+Load Balancer (external โ€” cloud or on-premises)
+  โ”‚
+  โ–ผ
+Ingress Controller (replicated โ€” 2+ instances)
+  โ”‚
+  โ–ผ
+API Gateway pods (3+ instances, anti-affinity)
+  โ”‚
+  โ–ผ
+Internal service mesh
+```
+
+### 8.3 DNS and Service Discovery
+
+All DCM components address each other via Kubernetes Service DNS. No hardcoded IPs. Service discovery is automatic โ€” a new pod replica is immediately addressable.
+
+---
+
+## 9. DCM System Policies โ€” Redundancy
+
+| Policy | Rule |
+|--------|------|
+| `RED-001` | All DCM control plane components must run as containers in Kubernetes pods |
+| `RED-002` | All control plane components must be stateless โ€” all persistent state in external stores |
+| `RED-003` | In profiles above `minimal`, all control plane components must have `replicas >= 3` with anti-affinity |
+| `RED-004` | All durable stores in profiles above `minimal` must use quorum writes with `write_quorum >= 2` |
+| `RED-005` | The Commit Log must use consensus protocol (Raft or equivalent) with quorum writes |
+| `RED-006` | The DCM deployment must be declared as a DCM resource in Git โ€” self-hosting required |
+| `RED-007` | DCM must run drift detection on its own components โ€” version drift is treated as resource drift |
+| `RED-008` | A rolling update of any DCM component must not reduce available replicas below `min_available` |
+| `RED-009` | All DCM component communication must use mTLS โ€” no plaintext internal communication |
+| `RED-010` | The bootstrap manifest is the only DCM configuration outside DCM's management scope |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should the bootstrap manifest be version-controlled and verifiable? | Bootstrap integrity | โœ… Resolved โ€” GitOps store + hash verification at every startup; tampering prevents start; operator-signed (RED-011) |
+| 2 | How does DCM handle Kubernetes cluster upgrades in sovereign deployments? | Operational | โœ… Resolved โ€” pre-staged images via signed bundles; maintenance mode during upgrade; startup verification before resume (RED-012) |
+| 3 | Should DCM support non-Kubernetes container runtimes? | Portability | โœ… Resolved โ€” Kubernetes primary/required for production; Podman/Docker Compose for dev/community only (RED-013) |
+| 4 | What is the minimum hardware specification per profile? | Implementation | โœ… Resolved โ€” declared as DCM Resource definitions; enforced by placement engine; table documented (RED-014) |
+| 5 | How does DCM's self-hosted drift detection handle DCM drifting from its own state? | Self-hosting | โœ… Resolved โ€” DCM is DCM-managed resource; Operator reconciles; bootstrap hash provides independent check; audit hash chain externally verifiable (RED-015) |
+
+---
+
+## 11. Related Concepts
+
+- **Universal Audit Model** (doc 16) โ€” two-stage audit; Commit Log quorum model
+- **Policy Organization** (doc 14) โ€” Profile-governed redundancy configuration
+- **Storage Providers** (doc 11) โ€” Store contracts include replication requirements
+- **Four States** (doc 02) โ€” all state stores are redundant per this model
+- **Ingestion Model** (doc 13) โ€” DCM's own deployment recovery uses the repave/rehydration pattern
+
+
+## 8. Deployment Redundancy Gap Resolutions
+
+### 8.1 Bootstrap Manifest Verification (Q1)
+
+The bootstrap manifest is stored in the GitOps store, hash-verified at DCM startup and every restart. Tampering prevents DCM from starting.
+
+```yaml
+bootstrap_manifest:
+  version: "1.0.0"
+  manifest_uuid: 
+  manifest_hash:           # computed at creation; verified at every startup
+  signed_by:    # signed by deploying operator
+```
+
+DCM startup sequence: verify manifest hash โ†’ verify manifest signature โ†’ proceed with initialization. On failure: refuse to start; emit security alert; notify platform admin.
+
+### 8.2 Kubernetes Cluster Upgrades in Sovereign Deployments (Q2)
+
+Sovereign DCM (air-gapped) uses pre-staged container images and the maintenance mode pattern:
+
+```
+Pre-upgrade:
+  1. Pre-stage all DCM container images in local registry (signed bundles)
+  2. DCM enters maintenance mode โ€” new requests queued; in-flight complete
+
+During upgrade:
+  3. Kubernetes upgrade proceeds using pre-staged images
+  4. DCM components restart against new cluster version
+
+Post-upgrade:
+  5. Startup verification (bootstrap manifest hash check)
+  6. DCM exits maintenance mode โ€” queued requests resume
+```
+
+Same signed bundle model as registry updates โ€” no new pattern needed.
+
+### 8.3 Non-Kubernetes Container Runtime Support (Q3)
+
+Kubernetes is DCM's primary and recommended container runtime. Non-Kubernetes runtimes (Podman, Docker Compose) are supported for development and community extension only. The DCM Operator is Kubernetes-native and not supported on other runtimes. Production deployments must use Kubernetes.
+
+### 8.4 Minimum Hardware Specifications (Q4)
+
+Expressed as DCM Resource definitions per profile โ€” machine-readable and enforced by the placement engine during self-deployment.
+
+| Profile | CPU | Memory | Storage | Replicas |
+|---------|-----|--------|---------|---------|
+| minimal | 2 cores | 4 Gi | 20 Gi | 1 |
+| dev | 4 cores | 8 Gi | 50 Gi | 1 |
+| standard | 8 cores | 16 Gi | 100 Gi | 3 |
+| prod | 16 cores | 32 Gi | 200 Gi | 3 |
+| fsi | 32 cores | 64 Gi | 500 Gi | 5 |
+| sovereign | 32 cores | 64 Gi | 500 Gi | 5 |
+
+These are minimums. Production workloads may require significantly more based on managed resource count.
+
+### 8.5 DCM Self-Hosted Drift Detection (Q5)
+
+DCM's own deployment is a DCM-managed resource subject to the same drift detection as any other resource. The DCM Operator continuously reconciles running components against the declared deployment manifest.
+
+**The bootstrap paradox โ€” who watches the watchman:**
+- DCM Operator drifts โ†’ bootstrap manifest hash verification (RED-011) detects independently
+- Audit component compromised โ†’ Audit Store hash chain break is detectable by external verification
+- Full DCM compromise โ†’ signed bundle verification at import time provides external trust anchor
+
+### 8.6 System Policies โ€” Deployment Redundancy Gaps
+
+| Policy | Rule |
+|--------|------|
+| `RED-011` | The bootstrap manifest is version-controlled in the GitOps store, hash-verified at DCM startup and every restart. Bootstrap manifest tampering prevents DCM from starting and triggers a security alert. |
+| `RED-012` | Kubernetes cluster upgrades in Sovereign DCM deployments use pre-staged container images from the local signed bundle registry. DCM enters maintenance mode during upgrade. In-flight operations complete before upgrade. DCM exits maintenance mode on successful startup verification. |
+| `RED-013` | Kubernetes is DCM's primary and recommended container runtime. Non-Kubernetes runtimes are supported for development and community extension purposes only. Production deployments must use Kubernetes. |
+| `RED-014` | Minimum hardware specifications are expressed as DCM Resource definitions per profile and enforced by the placement engine during DCM self-deployment. |
+| `RED-015` | DCM's own deployment is a DCM-managed resource subject to the same drift detection as any other resource. Bootstrap manifest hash verification (RED-011) provides independent verification. Audit Store hash chain breaks are detectable externally. |
+
+
+
+---
+
+## 9. Bootstrap Tenant Creation Sequence
+
+### 9.1 The Bootstrap Problem
+
+DCM requires every entity to belong to exactly one Tenant. But during initial deployment, no Tenants exist. The bootstrap sequence resolves this by creating the foundational Tenants as part of DCM startup, declared in the bootstrap manifest.
+
+### 9.2 The Three Foundation Tenants
+
+The bootstrap manifest declares three system Tenants that are created before any consumer can submit requests:
+
+```yaml
+bootstrap_tenants:
+  - handle: "__platform__"
+    display_name: "DCM Platform"
+    purpose: "Owns DCM's own control plane resources (components, stores, providers)"
+    automatically_created: true
+    cannot_be_decommissioned: true
+
+  - handle: "__transitional__"
+    display_name: "Transitional"
+    purpose: "Holds brownfield entities during ingestion before promotion to a real Tenant"
+    automatically_created: true
+    cannot_be_decommissioned: true
+
+  - handle: "__system__"
+    display_name: "System"
+    purpose: "Owns system-level artifacts (system layers, system policies, system workflows)"
+    automatically_created: true
+    cannot_be_decommissioned: true
+```
+
+### 9.3 Bootstrap Startup Sequence
+
+```
+DCM starts
+  โ”‚
+  โ–ผ Step 1: Verify bootstrap manifest hash and signature
+  โ”‚
+  โ–ผ Step 2: Initialize storage providers
+  โ”‚   GitOps stores initialized
+  โ”‚   Audit Store initialized
+  โ”‚   Commit Log initialized
+  โ”‚
+  โ–ผ Step 3: Create foundation Tenants (if not already existing)
+  โ”‚   __platform__, __transitional__, __system__
+  โ”‚
+  โ–ผ Step 4: Create initial Platform Admin actor
+  โ”‚   Declared in bootstrap manifest
+  โ”‚   Assigned to __platform__ Tenant
+  โ”‚   Given platform_admin role
+  โ”‚
+  โ–ผ Step 5: Activate system domain layers and policies
+  โ”‚   System layers loaded from GitOps store
+  โ”‚   System policies activated
+  โ”‚   Built-in recovery profiles activated
+  โ”‚
+  โ–ผ Step 6: Register built-in providers
+  โ”‚   Built-in Auth Provider
+  โ”‚   Search Index Storage Provider
+  โ”‚   Audit Store Storage Provider
+  โ”‚   (All owned by __platform__ Tenant)
+  โ”‚
+  โ–ผ Step 7: DCM ready
+      Consumer API, Provider API, Admin API accepting requests
+      Platform Admin can now create organization Tenants
+      Organization Tenants can request resources
+```
+
+### 9.4 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `RED-016` | The three foundation Tenants (__platform__, __transitional__, __system__) are created during bootstrap and cannot be decommissioned. All DCM control plane resources are owned by __platform__. All brownfield ingested entities enter __transitional__ before promotion. |
+
+
+
+---
+
+## 9. Bootstrap Sequence and Initial Tenant Creation
+
+### 9.1 The Bootstrap Problem
+
+The DCM data model requires every entity to be owned by a Tenant. But Tenants are themselves DCM entities. The bootstrap sequence defines how the initial Tenants and platform admin actor are created before DCM can accept external requests.
+
+### 9.2 Bootstrap Manifest
+
+The bootstrap manifest (RED-011) declares the initial state required for DCM to start. It includes:
+
+```yaml
+bootstrap_manifest:
+  version: "1.0.0"
+  signed_by: 
+
+  # Initial system Tenants (created before any external requests)
+  system_tenants:
+    - uuid: 
+      handle: "__platform__"
+      display_name: "DCM Platform"
+      description: "System Tenant owning DCM's own control plane resources"
+      immutable: true             # cannot be decommissioned or modified by regular operators
+
+    - uuid: 
+      handle: "__transitional__"
+      display_name: "Transitional"
+      description: "Holding Tenant for brownfield ingestion (INGEST phase)"
+      immutable: true
+
+  # Initial platform admin actor
+  bootstrap_admin:
+    uuid: 
+    username: "dcm-bootstrap-admin"
+    auth_provider: builtin
+    roles: [platform_admin]
+    credential_ref: 
+    # This credential is rotated on first login
+
+  # Active profile for initial deployment
+  initial_profile:
+    deployment_posture: minimal    # or as declared; can be changed post-bootstrap
+    compliance_domains: []
+
+  # Bootstrap admin's initial Tenant
+  initial_tenant:
+    uuid: 
+    handle: "org-default"
+    display_name: "Default Organization Tenant"
+    owned_by: bootstrap_admin
+```
+
+### 9.3 Bootstrap Sequence
+
+```
+DCM starts โ†’ bootstrap manifest hash verified (RED-011)
+  โ”‚
+  โ–ผ System Tenants created (before Policy Engine active):
+  โ”‚   __platform__ Tenant โ€” owns DCM control plane resources
+  โ”‚   __transitional__ Tenant โ€” brownfield ingestion holding
+  โ”‚   These are created by the bootstrap process itself, not through the request pipeline
+  โ”‚
+  โ–ผ Bootstrap admin actor created
+  โ”‚   Auth Provider initialized with bootstrap credential
+  โ”‚   Platform Admin role assigned
+  โ”‚
+  โ–ผ Initial profile activated
+  โ”‚   Deployment posture policies loaded
+  โ”‚   Compliance domain policies loaded (if declared)
+  โ”‚
+  โ–ผ Policy Engine comes online
+  โ”‚   All subsequent operations go through the standard request pipeline
+  โ”‚
+  โ–ผ Bootstrap admin creates the initial organization Tenant (optional)
+  โ”‚   First real request through the pipeline
+  โ”‚   Creates the initial production Tenant for organizational resources
+  โ”‚
+  โ–ผ Bootstrap admin credential rotation notification sent
+  โ”‚   Bootstrap credential must be rotated on first login
+  โ”‚   After rotation, bootstrap_admin becomes a standard platform admin actor
+  โ”‚
+  โ–ผ DCM accepts external requests
+```
+
+### 9.4 System Tenants
+
+The `__platform__` and `__transitional__` Tenants are created by the bootstrap process and are immutable:
+
+| System Tenant | Purpose | Who can modify |
+|--------------|---------|---------------|
+| `__platform__` | Owns DCM's own control plane resources | Platform Admin (restricted operations only) |
+| `__transitional__` | Brownfield ingestion holding area | Ingestion pipeline only |
+
+These Tenants are exempt from the normal Tenant decommission workflow โ€” they cannot be decommissioned while DCM is operational.
+
+### 9.5 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `BOOT-001` | The __platform__ and __transitional__ system Tenants are created by the bootstrap process before the Policy Engine comes online. They are immutable and cannot be decommissioned while DCM is running. |
+| `BOOT-002` | The bootstrap admin credential must be rotated on first login. The bootstrap manifest declares the initial credential reference only; the credential itself is managed by the Credential Provider. |
+| `BOOT-003` | After bootstrap, all Tenant creation and modification goes through the standard request pipeline. The bootstrap process is a one-time operation. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/18-webhooks-messaging.md b/content/docs/data-model/18-webhooks-messaging.md
new file mode 100644
index 0000000..0b3d2f1
--- /dev/null
+++ b/content/docs/data-model/18-webhooks-messaging.md
@@ -0,0 +1,831 @@
+# DCM Data Model โ€” Webhooks, Messaging, and External Integration
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Related Documents (updated):** [Notification Model](23-notification-model.md) | [Entity Relationships](09-entity-relationships.md)  
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: PROVIDER**
+>
+> The Provider abstraction โ€” Message Bus and webhook delivery channels
+
+
+**Related Documents:** [Universal Audit Model](16-universal-audit.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Authentication and Authorization](19-auth-providers.md) | [Policy Organization](14-policy-profiles.md)
+
+---
+
+
+> **See [Event Catalog](33-event-catalog.md)** โ€” authoritative source for all DCM event types and payload schemas.
+
+## 1. Purpose
+
+DCM communicates with the outside world through three complementary mechanisms:
+
+- **Outbound Webhooks** โ€” DCM pushes event notifications to external HTTP endpoints
+- **Inbound Webhooks** โ€” External systems push requests, queries, and events to DCM HTTP endpoints
+- **Message Bus Providers** โ€” DCM integrates with external message buses for persistent, high-throughput bidirectional event streaming
+
+All three mechanisms are authenticated, authorized, and audited identically to any other DCM API call. There is no privileged back-channel. Every integration is a registered DCM actor subject to full Policy Engine evaluation.
+
+**Webhooks are optional and policy-governed.** The active Profile sets defaults โ€” `fsi` and `sovereign` profiles may require webhook or message bus coverage for audit events. `minimal` and `dev` profiles make them fully optional.
+
+---
+
+## 2. Ingress and Egress โ€” The Universal Actor Model
+
+### 2.1 The `ingress` Block
+
+Every request entering DCM carries an immutable `ingress` block set by the DCM ingress layer before Policy Engine evaluation. It is never consumer-declarable and never modifiable by policies โ€” policies may only read it.
+
+```yaml
+ingress:
+  # HOW it arrived
+  surface: 
+  protocol: 
+  authenticated_via: 
+  authorized_via: policy_engine
+
+  # WHO sent it โ€” fully resolved actor context
+  actor:
+    uuid: 
+    type: 
+    display_name: "Jane Smith"
+    identity_source: 
+    auth_provider_uuid:           # which Auth Provider authenticated
+    auth_provider_type: 
+
+    # Resolved DCM roles and scope (at authentication time)
+    roles: [sre]
+    tenant_scope: []
+    groups: []
+    permissions: [request.submit, query.entity_state]
+
+    # Authorization chain
+    authorized_by:
+      method: 
+      authorizing_entity_uuid: 
+      authorization_timestamp: 
+      expiry: 
+
+    # Session context (human actors)
+    session_uuid: 
+    session_started_at: 
+    mfa_verified: 
+
+    # External identity (federated actors)
+    external_identity:
+      provider: 
+      subject: "uid=jsmith,cn=users,cn=accounts,dc=corp,dc=example,dc=com"
+      claims:
+        email: jsmith@corp.example.com
+        display_name: "Jane Smith"
+        department: Engineering
+        cost_center: CC-1234
+        ldap_groups: [cn=dcm-sre,...]
+        sid: "S-1-5-21-..."        # AD Security Identifier
+
+    # Rate limit tracking
+    rate_limit_bucket: 
+
+  # Surface-specific detail
+  webhook_registration_uuid:    # if surface: webhook_inbound
+  message_bus_provider_uuid:    # if surface: message_bus_inbound
+  message_offset:             # if surface: message_bus_inbound
+  scheduler_job_uuid:           # if surface: scheduler
+  parent_request_uuid:          # if surface: policy_engine|rehydration
+  source_ip: 
+```
+
+### 2.2 The `egress` Block
+
+All outbound calls from DCM carry DCM's authenticated identity:
+
+```yaml
+egress:
+  surface: 
+  protocol: 
+  actor:
+    uuid: 
+    type: dcm_internal
+    component: 
+    authenticated_via: 
+    credential_ref:
+      credential_provider_uuid: 
+      secret_path: 
+  originating_request_uuid: 
+  originating_actor_uuid: 
+```
+
+### 2.3 Policy Engine Use Cases โ€” Ingress/Egress Fields
+
+The ingress block enables a rich class of governance rules:
+
+```yaml
+# Require specific auth for sensitive operations
+policy: "If action == decommission AND ingress.actor.mfa_verified == false THEN gatekeep"
+
+# Block legacy API keys from production Tenants
+policy: "If tenant.profile == prod AND ingress.actor.identity_source == static_api_key THEN gatekeep"
+
+# Require enterprise auth for security resources
+policy: "If resource_type IN [Network.FirewallRule] AND ingress.actor.auth_provider_type NOT IN [oidc, freeipa, active_directory] THEN gatekeep"
+
+# Enrich from OIDC claims
+policy: "If ingress.actor.external_identity.claims.department EXISTS THEN inject: business_context.department"
+
+# Block message bus inbound from non-service-accounts
+policy: "If ingress.surface == message_bus_inbound AND ingress.actor.type != webhook_service_account THEN gatekeep"
+
+# Sovereignty check on inbound message bus
+policy: "If ingress.surface == message_bus_inbound AND message_bus_provider.jurisdiction != tenant.sovereignty_zone THEN gatekeep"
+```
+
+
+### 2.5 Ingress API vs Consumer API โ€” Relationship Clarification
+
+These two terms refer to different architectural layers:
+
+**Ingress API (infrastructure layer):**
+The network-level entry point for all inbound requests to the DCM control plane. It handles:
+- TLS termination
+- Authentication token validation
+- Setting the immutable `ingress` block on every request (surface, actor, timestamp, mfa_verified)
+- Rate limiting at the network level
+- Routing to the appropriate internal component (Consumer API handlers, Provider API handlers, Admin API handlers)
+
+The Ingress API is infrastructure โ€” it is not directly defined in any consumer-facing specification.
+
+**Consumer API (application layer):**
+The logical REST API surface that consumers interact with, as defined in the [Consumer API Specification](../specifications/consumer-api-spec.md). The Consumer API is *served through* the Ingress API. When a consumer calls `POST /api/v1/requests`, that request enters through the Ingress API (which sets the ingress block) and is then handled by the Consumer API component.
+
+**Other APIs served through the Ingress API:**
+- **Provider API** โ€” the callback and notification endpoints that Service Providers call (`/api/v1/provider/...`)
+- **Admin API** โ€” platform administration operations (`/api/v1/admin/...`)
+- **Webhook Inbound** โ€” external systems calling DCM (`/api/v1/webhooks/...`)
+
+**The Ingress API is one, the Consumer API is one of several logical surfaces routed through it.**
+
+---
+
+### 2.6 Consumer Rate Limiting and Quota Model
+
+Consumer-side rate limiting and resource quotas are enforced by GateKeeper policies โ€” not hardcoded limits. This keeps quota enforcement consistent with DCM's policy-driven model.
+
+**Request rate limiting (per actor):**
+
+```yaml
+rate_limit_policy:
+  type: gatekeeper
+  handle: "system/quotas/api-rate-limit"
+  trigger: request.initiated
+  conditions:
+    - field: ingress.actor_uuid
+      rate_window: PT1M
+      max_requests: 60           # configurable per Tenant policy
+  action: reject
+  rejection_code: 429
+  rejection_message: "Rate limit exceeded. Retry after PT1M."
+```
+
+**Resource quotas (per Tenant per resource type):**
+
+```yaml
+quota_policy:
+  type: gatekeeper
+  handle: "tenant/payments/vm-quota"
+  trigger: request.initiated
+  conditions:
+    - field: request.resource_type
+      equals: Compute.VirtualMachine
+    - field: tenant.active_entity_count
+      resource_type: Compute.VirtualMachine
+      operator: gte
+      value: 100                 # max 100 concurrent VMs for this Tenant
+  action: reject
+  rejection_message: "VM quota exceeded (100). Request a quota increase via the Admin API."
+```
+
+**Quota increase process:** Tenants request quota increases through the standard request process. A quota change request produces a Requested State record, goes through policy evaluation, and requires Platform Admin approval for significant increases.
+
+**Profile-governed defaults:**
+
+| Profile | Default API rate limit | Default resource quota |
+|---------|----------------------|----------------------|
+| minimal | 10 req/min | Unlimited |
+| dev | 60 req/min | Unlimited |
+| standard | 60 req/min | Policy-declared |
+| prod | 120 req/min | Policy-declared |
+| fsi | 60 req/min | Strict policy-declared |
+| sovereign | 30 req/min | Strict policy-declared |
+
+
+### 2.4 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ING-008` | All DCM requests must carry an ingress block. The ingress block is set by the DCM ingress layer and is immutable โ€” policies may read but not modify it. |
+| `ING-009` | The ingress block must include a fully resolved actor context: uuid, type, identity_source, roles, tenant_scope, auth_provider_uuid, and authorized_by chain. |
+| `ING-010` | All egress calls from DCM must carry DCM's authenticated identity. Unauthenticated egress is rejected. |
+| `ING-011` | Authentication enforcement is profile-governed. Standard and above reject unauthenticated requests. Minimal and dev support lightweight authenticated modes. There is no anonymous access in any profile. |
+| `ING-012` | Webhook and message bus inbound surfaces always require authentication regardless of active Profile. |
+| `ING-013` | Rate limiting is enforced per registered actor. Exceeding rate limits returns 429 Too Many Requests. |
+
+---
+
+## 3. Outbound Webhooks
+
+> **โš ๏ธ Architecture Update โ€” Notification Model Supersedes Outbound Webhooks**
+>
+> The outbound webhook model described in Section 3 has been superseded by the **Unified Notification Model** (see [doc 23: Notification Model](23-notification-model.md)). Outbound webhooks are now one delivery channel type within the Notification Provider model rather than a parallel mechanism.
+>
+> **For new implementations:** Use the Notification Provider subscription model (doc 23, Section 6) with a webhook-type Notification Provider.
+>
+> **For existing webhook registrations:** The registration model below remains supported via a compatibility layer. Existing registrations are automatically treated as actor-level subscriptions with a webhook-type Notification Provider. No migration required for current deployments.
+>
+> The key improvement in the new model: audience is derived from the **entity relationship graph**, not from a manually maintained subscriber list. A webhook subscription for VLAN drift events will now automatically include all VMs attached to that VLAN as audience context.
+
+### 3.1 Concept
+
+DCM pushes event notifications to registered external HTTP endpoints. Outbound webhooks are **optional and policy-governed** โ€” the active Profile and Policy Groups determine which events require external notification.
+
+### 3.2 Webhook Registration
+
+```yaml
+webhook_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "org/webhooks/payments-drift-alerts"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Payments Platform Team"
+      notification_endpoint: 
+
+  name: "Payments Platform Drift Alerts"
+  description: "Notifies payments team of drift detection events"
+
+  # SCOPE
+  scope:
+    type: 
+    tenant_uuid: 
+    cross_tenant_authorization_uuid:    # if cross_tenant
+
+  # EVENT SUBSCRIPTIONS
+  event_subscriptions:
+    - event_type: drift.detected
+      schema_version: "1.0"
+      adapter: true              # DCM transforms newer schemas to 1.0
+      filter:
+        tenant_uuid: 
+        resource_types: [Compute.VirtualMachine, Storage.Block]
+    - event_type: request.realized
+      schema_version: "1.0"
+    - event_type: entity.state_transition
+      schema_version: "1.0"
+      filter:
+        to_states: [DEGRADED, FAILED]
+
+  # ENDPOINT
+  endpoint:
+    url: https://alerts.payments.corp.example.com/dcm/events
+    sovereignty_check: true     # verify endpoint jurisdiction before delivery
+
+  # AUTHENTICATION
+  authentication:
+    mode: 
+    secret_ref:
+      credential_provider_uuid: 
+      secret_path: "dcm/webhooks/payments-drift/hmac-secret"
+    rotation_policy:
+      automatic: true
+      interval: P90D
+      transition_window: P7D
+      notify_before: P14D
+
+  # RELIABILITY
+  retry_policy:
+    max_attempts: 7
+    backoff: exponential
+    initial_interval: PT5S
+    max_interval: PT1H
+    timeout_per_attempt: PT10S
+    on_exhaustion: dead_letter   # dead_letter | discard | escalate
+
+  # HEALTH
+  health:
+    failure_threshold: 10
+    suspension_notification: true
+    auto_deactivate_after: P30D
+    status: active
+
+  # SCHEMA COMPATIBILITY
+  schema_adapter:
+    enabled: true
+    # DCM maintains forward-compatibility adapters per schema version
+    # Consumer stays on declared schema_version indefinitely
+    deprecation_notice_days: 90
+```
+
+### 3.3 Event Taxonomy
+
+> **See [doc 23: Notification Model](23-notification-model.md) Section 4 for the authoritative event taxonomy.** The table below is the legacy taxonomy for existing webhook registrations.
+
+The event taxonomy maps onto the Universal Audit action vocabulary. All are versioned registry entries:
+
+| Category | Events |
+|----------|--------|
+| Entity lifecycle | `entity.created`, `entity.modified`, `entity.state_transition`, `entity.deleted`, `entity.expired`, `entity.rehydrated` |
+| Group | `group.member_added`, `group.member_removed`, `group.created`, `group.deleted` |
+| Relationship | `relationship.created`, `relationship.released` |
+| Policy | `policy.activated`, `policy.deactivated`, `policy.evaluated` (fail/gatekeep only), `policy.shadow_result` |
+| Provider | `provider.healthy`, `provider.degraded`, `provider.unhealthy`, `provider.registered`, `provider.deregistered` |
+| Audit/security | `audit.chain_break`, `audit.forward_failed` |
+| Drift | `drift.detected`, `drift.resolved`, `drift.escalated` |
+| Request | `request.submitted`, `request.approved`, `request.rejected`, `request.realized`, `request.failed` |
+| Rehydration | `rehydration.started`, `rehydration.completed`, `rehydration.paused`, `rehydration.interrupted` |
+| Authorization | `authorization.granted`, `authorization.revoked` |
+| Webhook | `webhook.secret_rotated`, `webhook.suspended`, `webhook.schema_deprecated` |
+
+### 3.4 Payload Format
+
+```yaml
+webhook_payload:
+  # Envelope
+  event_uuid:               # idempotency key
+  event_type: drift.detected
+  event_schema_version: "1.0"
+  timestamp:            # from Stage 1 Commit Log โ€” authoritative
+  dcm_version: 
+
+  # Subject
+  subject:
+    entity_uuid: 
+    entity_type: resource_entity
+    entity_handle: 
+    tenant_uuid: 
+
+  # Delta
+  delta:
+    drifted_fields:
+      - field: cpu_count
+        realized_value: 4
+        discovered_value: 8
+    drift_severity: significant
+
+  # Links
+  links:
+    self: 
+    audit_record: 
+```
+
+### 3.5 Delivery Guarantees
+
+- **At-least-once** โ€” not exactly-once; consumers must be idempotent using `event_uuid`
+- **Per-entity ordering** โ€” events for a given `entity_uuid` delivered in Commit Log sequence order
+- **Cross-entity ordering** โ€” not guaranteed; use `timestamp` for actual occurrence time
+- **Sovereignty-aware** โ€” delivery blocked if endpoint jurisdiction incompatible with Tenant sovereignty (WHK-004)
+
+---
+
+## 4. Inbound Webhooks
+
+### 4.1 Concept
+
+DCM exposes authenticated HTTP endpoints that external systems call to submit requests, queries, and events. Inbound webhooks are subject to full Policy Engine evaluation โ€” identical to any other API call.
+
+### 4.2 Inbound Endpoints
+
+| Endpoint | Purpose |
+|----------|---------|
+| `POST /webhooks/inbound/request` | Submit a service request |
+| `POST /webhooks/inbound/query` | Query entity state or catalog |
+| `POST /webhooks/inbound/event` | Push an event (provider state change, CI/CD signal) |
+| `POST /webhooks/inbound/ingestion` | Push brownfield ingestion data |
+| `POST /webhooks/inbound/data` | Push enrichment or information data |
+
+### 4.3 Webhook Actor Registration
+
+Every inbound webhook caller must be registered as a **webhook actor** โ€” a service account in the DCM identity model:
+
+```yaml
+webhook_actor:
+  artifact_metadata:
+    uuid: 
+    handle: "actors/webhook/cicd-pipeline-prod"
+    status: active
+
+  name: "CI/CD Pipeline Production"
+  actor_type: webhook_service_account
+
+  # Authentication
+  authentication:
+    mode: hmac_sha256
+    secret_ref:
+      credential_provider_uuid: 
+      secret_path: "dcm/webhooks/inbound/cicd-pipeline/hmac"
+
+  # Authorization
+  role: consumer
+  tenant_scope: []
+  permitted_operations:
+    - request.submit
+    - query.entity_state
+    - query.catalog
+
+  # Rate limiting
+  rate_limit:
+    requests_per_minute: 60
+    burst: 10
+
+  # Audit identity
+  audit_identity:
+    display_name: "CI/CD Pipeline (Production)"
+    system: "jenkins-prod-01"
+```
+
+### 4.4 Response Model
+
+- **Queries** โ€” synchronous response with result
+- **Requests and events** โ€” `202 Accepted` + `request_uuid`; caller polls status or registers outbound webhook for completion notification
+
+---
+
+## 5. Message Bus Provider
+
+### 5.1 Concept
+
+A **Message Bus Provider** is the sixth DCM provider type โ€” a persistent, high-throughput integration with an external message bus for bidirectional event streaming. Where webhooks are point-to-point HTTP calls, a Message Bus Provider is a durable pub/sub connection.
+
+### 5.2 Registration
+
+```yaml
+message_bus_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/messagebus/corporate-kafka"
+    version: "1.0.0"
+    status: active
+
+  name: "Corporate Kafka Cluster"
+  provider_type: message_bus
+
+  # Direction
+  direction: 
+
+  # Protocol
+  protocol: 
+
+  # Connection
+  connection:
+    brokers: [kafka-1.corp:9093, kafka-2.corp:9093, kafka-3.corp:9093]
+    credentials_ref:
+      credential_provider_uuid: 
+      secret_path: "dcm/providers/messagebus/corporate-kafka/credentials"
+    tls:
+      mode: mtls
+      ca_cert_ref:
+        credential_provider_uuid: 
+        secret_path: "dcm/providers/messagebus/corporate-kafka/ca-cert"
+
+  # Outbound โ€” DCM publishes to external bus
+  outbound:
+    topic_mapping:
+      entity.state_transition: "dcm.entities.state"
+      drift.detected: "dcm.drift.alerts"
+      request.realized: "dcm.requests.completed"
+      audit.chain_break: "dcm.security.alerts"
+    schema_version: "1.0"
+    delivery_guarantee: at_least_once
+
+  # Inbound โ€” DCM consumes from external bus
+  inbound:
+    consumer_group: "dcm-inbound-prod"
+    topic_mapping:
+      "cicd.deployment.completed": request.submit
+      "cmdb.discovery.update": ingestion.push
+      "itsm.change.approved": request.approve
+    # Inbound messages processed as authenticated API calls
+    actor_identity_uuid: 
+    # Same Policy Engine evaluation as inbound webhooks
+
+  # Sovereignty
+  operational_sovereignty:
+    jurisdiction: eu-west
+    certifications: [ISO-27001, GDPR-compliant]
+
+  # Health
+  health_check:
+    interval_seconds: 30
+    on_unhealthy: alert
+```
+
+### 5.3 Architecture
+
+```
+DCM internal Message Bus (internal pub/sub backbone)
+  โ”‚
+  โ”œโ”€โ”€ Webhook Delivery Service โ”€โ”€โ”€โ”€โ”€โ”€โ†’ External HTTP endpoints (outbound webhooks)
+  โ”‚
+  โ””โ”€โ”€ Message Bus Bridge Service โ”€โ”€โ”€โ”€โ†’ External message bus (Message Bus Provider)
+                                 โ†โ”€โ”€โ”€ External message bus (inbound)
+```
+
+The internal Message Bus is never exposed directly. All external event integration goes through either the Webhook Delivery Service or the Message Bus Bridge Service โ€” both of which handle authentication, authorization, sovereignty checks, and schema transformation.
+
+---
+
+## 6. Git PR Ingress โ€” Distributed Git Request Mechanism
+
+### 6.1 Concept
+
+DCM supports **git_pr_merge** as a twelfth ingress surface โ€” enabling teams to submit DCM resource definitions as Pull Requests to a DCM-watched Git repository. This is the native workflow for infrastructure-as-code teams: open a PR, get it reviewed by humans and DCM's policy engine simultaneously, merge to execute.
+
+**Why Git PR ingress matters:**
+- GitOps teams work in Git โ€” their deployment workflow is already PR-based
+- Security and compliance teams review infrastructure changes the same way they review code
+- The PR itself is the human review record; DCM's audit trail captures the automated processing
+- Rollback is a Git revert โ€” natural and familiar
+- Multi-team approval workflows use existing Git branch protection rules
+- The PR diff shows exactly what changes โ€” field-level visibility
+
+### 6.2 The Git Request Watcher
+
+A dedicated control plane component โ€” the **Git Request Watcher** โ€” monitors designated repositories via webhooks (preferred) or polling. It is policy-governed: which repositories it watches, which branches trigger processing, and which resource types may be submitted via Git PR.
+
+### 6.3 Request Repository Structure
+
+```
+dcm-requests/                              โ† DCM-watched request repository
+  {tenant-uuid}/
+    pending/
+      {resource-handle}/
+        request.yaml                       โ† Standard DCM resource definition
+    realized/
+      {resource-handle}/
+        realized.yaml                      โ† DCM writes realized state here on success
+    failed/
+      {resource-handle}/
+        request.yaml                       โ† Moved here on failure with error detail
+```
+
+### 6.4 Git Actor Identity Resolution
+
+**Authentication is always required.** Git PR ingress actors must be resolved to DCM actors through the registered Auth Provider โ€” DCM trusts the Git server's authentication assertion, not user-declared Git configuration. Anyone can set their local `git config user.email` to anything; DCM ignores self-declared identity.
+
+**The trust chain:**
+```
+Git server authenticates user (SSH key, OAuth token, password)
+  โ”‚  Git server's authentication is trusted โ€” not user's claimed identity
+  โ–ผ
+DCM Git Request Watcher receives PR merge webhook
+  โ”‚  Webhook payload contains: actor.login, actor.auth_method, actor.external_id
+  โ”‚  All verified by the Git server
+  โ–ผ
+Auth Provider resolution (same path as web UI login for the same user):
+  โ”œโ”€โ”€ OIDC/OAuth: Git server OAuth subject โ†’ OIDC Auth Provider userinfo lookup
+  โ”œโ”€โ”€ LDAP/AD: Git server username โ†’ LDAP lookup โ†’ DCM actor
+  โ”œโ”€โ”€ SSH key: Git server key fingerprint โ†’ DCM SSH key registry โ†’ DCM actor
+  โ””โ”€โ”€ Service account: Git service account โ†’ registered webhook actor
+  โ–ผ
+Fully resolved DCM actor โ€” same roles, groups, tenant scope as any other user
+```
+
+**Resolution methods:**
+
+```yaml
+git_actor_resolution:
+  # Method 1: OIDC/OAuth (recommended โ€” Git server uses same IdP as DCM)
+  method: oidc_subject_lookup
+  auth_provider_uuid: 
+
+  # Method 2: LDAP/AD (enterprise โ€” Git server authenticates via corporate directory)
+  method: ldap_username_lookup
+  auth_provider_uuid: 
+
+  # Method 3: SSH key fingerprint
+  method: ssh_key_fingerprint
+  # Keys registered in DCM SSH key registry, linked to actor UUIDs
+
+  # Method 4: Service account (automated workflows)
+  method: webhook_service_account
+  # Git service account mapped to registered webhook actor
+```
+
+**Identity resolution failure โ€” explicit rejection:**
+
+When DCM cannot map the merge actor to a DCM actor, the PR is rejected with an actionable comment. Never silently ignored.
+
+```
+โŒ DCM Identity Resolution Failed
+
+DCM could not map the merge actor "jsmith" to a DCM actor identity.
+
+Possible causes:
+  โ€ข Your Git account is not linked to a DCM actor via the corporate Auth Provider
+  โ€ข Your DCM actor account has been suspended or deactivated
+
+To resolve:
+  โ€ข Contact your platform administrator: https://dcm.corp.example.com/actors/git-identity-setup
+
+This PR will not be processed until identity resolution succeeds.
+```
+
+### 6.5 The ingress Block for Git PR
+
+```yaml
+ingress:
+  surface: git_pr_merge               # or: git_pr_open (for shadow validation)
+  protocol: https
+  authenticated_via: oidc             # or: ldap_direct_bind, active_directory, ssh_key
+  actor:
+    uuid: 
+    type: human
+    display_name: "Jane Smith"
+    identity_source: oidc
+    auth_provider_uuid: 
+    roles: [consumer]
+    tenant_scope: []
+    groups: []
+    # Groups and tenant scope: SAME mappings as web UI login for this user
+    external_identity:
+      provider: github                # or: gitlab, gitea, freeipa, active_directory
+      subject:         # verified by Git server
+      git_username: jsmith
+      git_verified_email: jsmith@corp.com  # from Git server record โ€” not git config
+    mfa_verified: true                # from Auth Provider session record
+  git_context:
+    repository: https://git.corp.example.com/dcm-requests/payments
+    pr_number: 142
+    pr_url: https://git.corp.example.com/payments/pulls/142
+    merge_commit: 
+    base_branch: main
+    pr_author: jsmith
+    pr_reviewers: [platform-team, security-team]
+    pr_approved_by: [, ]
+    # Approved by: DCM resolves Git reviewer identities via same Auth Provider
+```
+
+### 6.6 PR Lifecycle โ€” DCM Processing Flow
+
+```
+1. Developer creates resource definition YAML (standard DCM request format)
+   โ”‚
+2. Opens PR against dcm-requests/{tenant-uuid}/pending/
+   โ”‚
+3. DCM Git Watcher detects PR (git_pr_open event)
+   โ”‚
+4. DCM resolves PR author โ†’ DCM actor via Auth Provider
+   โ”‚  Failure โ†’ post rejection comment; stop processing
+   โ”‚
+5. DCM validates actor tenant scope against target Tenant
+   โ”‚  Failure โ†’ post rejection comment; stop processing
+   โ”‚
+6. Shadow policy evaluation (same nine-step assembly โ€” dry run)
+   โ”‚  Results posted as PR review comments:
+   โ”‚  "โœ… Schema valid"
+   โ”‚  "โœ… All policies pass"
+   โ”‚  "โš ๏ธ  Will be placed in eu-west-1 per sovereignty policy"
+   โ”‚  "โŒ GateKeeper: VM size exceeds quota โ€” reduce cpu_count"
+   โ”‚
+7. Human review and approval (standard Git PR workflow)
+   โ”‚  Branch protection enforces required reviewers
+   โ”‚  Policy may declare: require DCM-defined approvers in git_context.pr_approved_by
+   โ”‚
+8. PR merged to main (git_pr_merge event)
+   โ”‚  ingress_surface: git_pr_merge
+   โ”‚  Actor re-verified at merge time (not assumed from PR open time)
+   โ”‚
+9. DCM processes as standard nine-step assembly (real โ€” not shadow)
+   โ”‚
+10. DCM posts realization result as PR comment + status check
+    โ”‚  "โœ… VM payments-prod-01 realized โ€” UUID: "
+    โ”‚  "โŒ Realization failed โ€” audit record: "
+    โ”‚
+11. DCM commits realized state to realized/ directory (optional)
+    Git history = full lifecycle record
+```
+
+### 6.7 Policy Engine Use Cases โ€” Git PR Ingress
+
+```yaml
+# Require PR approval before processing production resources
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND tenant.profile IN [prod, fsi, sovereign]
+    AND ingress.git_context.pr_approved_by NOT CONTAINS required_approvers
+    THEN gatekeep: "PR requires approval from platform admin and security owner"
+
+# Require MFA for Git PR merges in production Tenants
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND tenant.profile IN [prod, fsi, sovereign]
+    AND ingress.actor.mfa_verified == false
+    THEN gatekeep: "MFA required for Git PR merges in production Tenants"
+
+# Restrict resource types submittable via Git PR
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND resource_type NOT IN [Compute.VirtualMachine, Storage.Block]
+    THEN gatekeep: "Only compute and storage resources may be submitted via Git PR"
+
+# Require actor to be in authorized Git team for target Tenant
+policy:
+  type: gatekeeper
+  rule: >
+    If ingress.surface == git_pr_merge
+    AND ingress.actor.groups NOT CONTAINS tenant.authorized_git_groups
+    THEN gatekeep: "Git PR author is not in an authorized group for this Tenant"
+
+# Post shadow results as PR comments (transformation)
+policy:
+  type: transformation
+  placement_phase: pre
+  rule: >
+    If ingress.surface IN [git_pr_merge, git_pr_open]
+    THEN inject: git_feedback.post_as_pr_comment = true
+```
+
+### 6.8 Updated Ingress Surface Taxonomy
+
+```yaml
+ingress_surface_taxonomy:
+  - web_ui                 # DCM's own web interface
+  - consumer_api           # Direct REST API call
+  - webhook_inbound        # Inbound webhook call
+  - message_bus_inbound    # Inbound message bus
+  - provider_callback      # Provider reporting back
+  - policy_engine          # Policy-generated sub-request
+  - scheduler              # Scheduled/timed trigger
+  - rehydration            # Rehydration-generated
+  - ingestion              # Ingestion pipeline
+  - dcm_internal           # DCM system-generated
+  - operator_cli           # Command line interface
+  - git_pr_merge           # Git PR merge โ†’ execute  โ† new
+  - git_pr_open            # Git PR open โ†’ shadow validation only  โ† new
+```
+
+### 6.9 System Policies โ€” Git PR Ingress
+
+| Policy | Rule |
+|--------|------|
+| `GIT-001` | DCM supports git_pr_merge and git_pr_open as ingress surfaces. Git PR ingress is subject to full Policy Engine evaluation identical to API ingress. |
+| `GIT-002` | DCM trusts the Git server's authentication assertion โ€” not user-declared Git configuration. DCM resolves the Git server's verified identity through the registered Auth Provider to produce a fully-resolved DCM actor with the same role, group, and tenant scope mappings as any other user authenticated via the same Auth Provider. |
+| `GIT-003` | Unresolvable Git actor identities are rejected with an actionable PR comment. PRs are never silently ignored. |
+| `GIT-004` | The resolved Git PR actor must have the target Tenant UUID in their tenant_scope. PRs targeting Tenants outside the actor's scope are rejected โ€” same enforcement as API tenant scope checks. |
+| `GIT-005` | DCM posts shadow policy evaluation results as PR review comments on git_pr_open. GateKeeper failures should be surfaced via repository branch protection integration. |
+| `GIT-006` | PR approval status may be declared as an authorization requirement by policy. DCM checks declared reviewer approvals against the PR's actual approval record before processing a merged PR. |
+| `GIT-007` | The Git Request Watcher component is policy-governed: which repositories it monitors, which branches trigger processing, and which resource types may be submitted via Git PR are declared via Policy Group. |
+| `GIT-008` | Actor identity is re-verified at merge time โ€” not assumed from PR open time. A user whose DCM actor is suspended between PR open and merge will be rejected at merge. |
+
+---
+
+
+
+| Policy | Rule |
+|--------|------|
+| `WHK-001` | Events for a given entity_uuid are delivered in Commit Log sequence order. Cross-entity ordering is not guaranteed. |
+| `WHK-002` | Webhook delivery uses at-least-once semantics. Consumers must be idempotent using event_uuid as the deduplication key. |
+| `WHK-003` | Outbound webhook authentication must be declared at registration. Supported: hmac_sha256 (default), mtls, bearer_token. Unauthenticated webhooks are rejected. |
+| `WHK-004` | Webhook endpoints that would deliver data outside a Tenant's sovereignty boundary are subject to sovereignty checks. |
+| `WHK-005` | Webhook governance is policy-driven. Profiles set defaults โ€” fsi/sovereign profiles may require webhook coverage for audit events via Policy Group. |
+| `WHK-006` | Webhook registrations must declare the event schema version expected. DCM supports current and N-1 schema versions simultaneously. |
+| `WHK-007` | Platform-scoped webhooks require Platform Admin role and are audit-logged as CONFIG_CHANGE. |
+| `WHK-008` | Cross-tenant webhooks require a valid cross_tenant_authorization record (XTA-001). |
+| `WHK-009` | Inbound webhook callers must be registered as webhook actors with explicit role, tenant scope, and permitted operations. Unregistered callers are rejected with 401. |
+| `WHK-010` | Inbound webhook calls are subject to full Policy Engine evaluation โ€” identical to any other API call. No bypass. |
+| `WHK-011` | All inbound webhook calls are recorded in the audit trail with the webhook actor as the immediate actor. |
+| `WHK-012` | Inbound webhook endpoints return 202 Accepted + request_uuid for async operations. |
+| `WHK-013` | Rate limiting is enforced per registered webhook actor. Exceeding rate limits returns 429 Too Many Requests. |
+| `WHK-014` | All credential references in webhook and message bus configurations must resolve through a registered Credential Provider. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/19-auth-providers.md b/content/docs/data-model/19-auth-providers.md
new file mode 100644
index 0000000..d6a23e7
--- /dev/null
+++ b/content/docs/data-model/19-auth-providers.md
@@ -0,0 +1,693 @@
+# DCM Data Model โ€” Authentication, Authorization, and Auth Providers
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).*
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Webhooks and Messaging](18-webhooks-messaging.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: PROVIDER**
+>
+> The Provider abstraction โ€” Auth Provider and Credential Provider extensions
+
+
+
+---
+
+## 1. Purpose
+
+DCM authenticates and authorizes every request โ€” inbound and outbound โ€” through a unified **Auth Provider** model. An Auth Provider is the eighth DCM provider type: an external (or built-in) system that answers two questions:
+
+1. **Authentication** โ€” is this identity who they claim to be?
+2. **Authorization** โ€” what is this identity permitted to do?
+
+Every authentication mode DCM supports โ€” static API key, local users, GitHub OAuth, LDAP, FreeIPA, Active Directory, OIDC, mTLS โ€” is an Auth Provider implementation. The built-in Auth Provider ships with DCM and requires zero external configuration, enabling immediate home lab and evaluation use. External Auth Providers are registered artifacts, versioned, GitOps-managed, and audited.
+
+**Authentication is always required โ€” there is no anonymous access in any DCM profile.** The difference between profiles is how much effort authentication setup requires, not whether it exists.
+
+---
+
+## 2. Auth and Credential Provider Types
+
+Auth Providers and Credential Providers are two of the eleven DCM provider types (see [Unified Provider Contract](A-provider-contract.md)). This section covers the authentication modes and configurations supported:
+
+Auth Provider completes the DCM provider ecosystem:
+
+| # | Type | Purpose |
+|---|------|---------|
+| 1 | **Service Provider** | Realizes resources |
+| 2 | **Information Provider** | Serves authoritative external data |
+| 3 | **Meta Provider** | Composes multiple providers |
+| 4 | **Storage Provider** | Persists DCM state |
+| 5 | **Policy Provider** | Evaluates policies externally |
+| 6 | **Credential Provider** | Manages secrets and credentials |
+| 7 | **Auth Provider** | Authenticates actor identities |
+| 8 | **Notification Provider** | Delivers notifications |
+| 9 | **Message Bus Provider** | Async event streaming |
+| 10 | **Registry Provider** | Serves the Resource Type Registry |
+| 11 | **Peer DCM** | Another DCM instance (federation) |
+
+---
+
+## 3. Auth Provider Registration
+
+```yaml
+auth_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/auth/corporate-freeipa"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Platform Team"
+
+  name: "Corporate FreeIPA"
+  description: "Primary enterprise directory โ€” FreeIPA with Kerberos"
+
+  # Capabilities
+  capabilities:
+    authentication: true
+    authorization: true
+    mfa: false                    # does this provider enforce MFA?
+    session_management: true
+    group_sync: true
+
+  # Provider type
+  provider_type: 
+
+  # What actor types this provider can authenticate
+  authenticates: [human, service_account, webhook_service_account]
+
+  # Trust level
+  trust_level: 
+  # authoritative: DCM accepts all decisions without re-evaluation
+  # verified:      DCM accepts with additional Policy Engine checks
+  # advisory:      DCM treats decisions as input โ€” full re-evaluation always
+
+  # Connection credentials
+  connection_credentials_ref:
+    credential_provider_uuid: 
+    secret_path: "dcm/auth/freeipa/bind-password"
+
+  # Health check
+  health_check:
+    interval_seconds: 30
+    on_unhealthy: 
+    fallback_provider_uuid: 
+
+  # Session configuration
+  session:
+    token_ttl: PT8H
+    refresh_enabled: true
+    refresh_ttl: P7D
+    concurrent_sessions: 3
+
+  # Role mapping โ€” external groups โ†’ DCM roles
+  role_mapping:
+    default_role: consumer
+    group_role_map:
+      - external_group: "cn=dcm-admins,cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+        dcm_role: platform_admin
+      - external_group: "cn=dcm-sre,cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+        dcm_role: sre
+      - external_group: "cn=dcm-consumers,cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+        dcm_role: consumer
+
+  # Tenant mapping โ€” external groups โ†’ DCM Tenants
+  tenant_mapping:
+    strategy: 
+    group_tenant_map:
+      - external_group: "cn=payments-team,cn=groups,..."
+        tenant_uuid: 
+      - external_group: "cn=platform-team,cn=groups,..."
+        tenant_scope: [all]
+
+  # Config changes go through shadow validation
+  on_config_change: proposed
+```
+
+---
+
+## 4. Authentication Modes
+
+### 4.1 Built-In Auth Provider (zero configuration)
+
+Ships with DCM. Always registered. Cannot be deregistered โ€” only deprioritized.
+
+```yaml
+built_in_auth_provider:
+  handle: "providers/auth/dcm-builtin"
+  provider_type: built_in
+  modes:
+    static_api_key:
+      enabled: true             # generated at bootstrap โ€” shown once
+    local_users:
+      enabled: true             # managed via: dcm user create
+    github_oauth:
+      enabled: false            # opt-in: requires client_id + secret
+    gitlab_oauth:
+      enabled: false            # opt-in: requires client_id + secret
+```
+
+**Static API Key** โ€” generated at bootstrap, shown once:
+```
+โœ“ DCM is ready.
+Your bootstrap API key (shown once โ€” store it securely):
+  dcm_sk_a1b2c3...f7g8
+
+curl -H "Authorization: Bearer dcm_sk_a1b2c3...f7g8" https://localhost:8443/api/v1/catalog
+```
+
+**Local Users** โ€” managed via CLI:
+```bash
+dcm user create --username admin --role platform_admin
+dcm user create --username developer --role consumer --tenant payments
+```
+
+### 4.2 GitHub / GitLab OAuth
+
+```yaml
+auth_provider:
+  provider_type: github_oauth
+  config:
+    client_id: 
+    client_secret_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/github/client-secret"
+    role_mapping:
+      default_role: consumer
+      org_role_map:
+        - github_org: "my-lab-org"
+          dcm_role: platform_admin
+```
+
+### 4.3 LDAP / FreeIPA (RFC 4511)
+
+```yaml
+auth_provider:
+  provider_type: freeipa          # or: ldap
+  config:
+    server: ldaps://freeipa.corp.example.com:636
+    tls:
+      mode: ldaps                 # ldaps | starttls
+      ca_cert_ref:
+        credential_provider: dcm_internal
+        path: "dcm/auth/freeipa/ca-cert"
+    bind_dn: "uid=dcm-service,cn=users,cn=accounts,dc=corp,dc=example,dc=com"
+    bind_password_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/freeipa/bind-password"
+
+    user_search:
+      base_dn: "cn=users,cn=accounts,dc=corp,dc=example,dc=com"
+      filter: "(uid={username})"
+      attributes:
+        username: uid
+        email: mail
+        display_name: cn
+
+    group_search:
+      base_dn: "cn=groups,cn=accounts,dc=corp,dc=example,dc=com"
+      filter: "(member={user_dn})"
+      attributes:
+        group_name: cn
+
+    # FreeIPA-specific integrations
+    kerberos:
+      enabled: true               # SSO for Linux CLI users
+      keytab_ref:
+        credential_provider: dcm_internal
+        path: "dcm/auth/freeipa/dcm.keytab"
+      service_principal: "HTTP/dcm.corp.example.com@CORP.EXAMPLE.COM"
+    hbac:
+      enforce: true               # Honor FreeIPA Host-Based Access Control
+    ca:
+      trust_freeipa_ca: true      # Trust FreeIPA CA for mTLS
+
+    group_sync:
+      enabled: true
+      interval_seconds: 300
+      on_group_change: reauthorize
+```
+
+### 4.4 Active Directory
+
+```yaml
+auth_provider:
+  provider_type: active_directory
+  config:
+    domain_controllers:
+      - ldaps://dc01.corp.example.com:636
+      - ldaps://dc02.corp.example.com:636   # automatic failover
+    tls:
+      mode: ldaps
+      ca_cert_ref:
+        credential_provider: dcm_internal
+        path: "dcm/auth/ad/ca-cert"
+    bind_dn: "CN=DCM Service,OU=Service Accounts,DC=corp,DC=example,DC=com"
+    bind_password_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/ad/bind-password"
+
+    user_search:
+      base_dn: "DC=corp,DC=example,DC=com"
+      filter: "(sAMAccountName={username})"
+      # UPN alternative: "(userPrincipalName={username}@corp.example.com)"
+      attributes:
+        username: sAMAccountName
+        email: userPrincipalName
+        display_name: displayName
+        sid: objectSid            # AD Security Identifier โ€” for audit
+
+    group_search:
+      base_dn: "DC=corp,DC=example,DC=com"
+      # LDAP_MATCHING_RULE_IN_CHAIN โ€” resolves nested AD group membership
+      filter: "(&(objectClass=group)(member:1.2.840.113556.1.4.1941:={user_dn}))"
+      attributes:
+        group_name: cn
+        group_dn: distinguishedName
+```
+
+### 4.5 OIDC
+
+```yaml
+auth_provider:
+  provider_type: oidc
+  config:
+    issuer: https://accounts.google.com      # or: Okta, Azure AD, Keycloak, Dex
+    client_id: dcm-production
+    client_secret_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/oidc/client-secret"
+    scopes: [openid, profile, email, groups]
+    claims_mapping:
+      username: preferred_username
+      email: email
+      display_name: name
+      groups: groups
+      department: department          # custom claims
+      cost_center: cost_center
+```
+
+### 4.6 mTLS
+
+```yaml
+auth_provider:
+  provider_type: mtls
+  config:
+    ca_cert_ref:
+      credential_provider: dcm_internal
+      path: "dcm/auth/mtls/ca-cert"
+    # Client certificate CN โ†’ DCM actor mapping
+    cn_actor_mapping:
+      - cn_pattern: "service-account-*"
+        actor_type: service_account
+        default_role: consumer
+      - cn_pattern: "provider-*"
+        actor_type: provider
+```
+
+---
+
+## 5. Multiple Auth Providers โ€” Priority and Routing
+
+DCM routes to the appropriate Auth Provider based on the authentication signal present in the request:
+
+```yaml
+auth_provider_resolution:
+  resolution_order:
+    - signal: mtls_client_cert
+      provider_uuid: 
+    - signal: bearer_token_oidc
+      provider_uuid: 
+    - signal: bearer_token_apikey
+      provider_uuid: 
+    - signal: basic_auth
+      provider_uuid: 
+    - signal: hmac_signature
+      provider_uuid: 
+    - signal: none
+      action: reject               # always โ€” no anonymous access
+```
+
+### 5.1 Auth Provider Chain
+
+Authentication and authorization enrichment can be chained:
+
+```yaml
+auth_provider_chain:
+  authentication:
+    provider_uuid:    # fast LDAP bind
+  enrichment:
+    provider_uuid:    # LDAP group membership
+  augmentation:
+    provider_uuid:  # OIDC userinfo for rich claims
+    # (department, cost_center, project codes from HR system)
+```
+
+---
+
+## 6. Credential Provider
+
+A **Credential Provider** is the seventh DCM provider type โ€” a cross-cutting dependency that any DCM component or provider registration references for secret resolution. DCM never stores credentials directly.
+
+```yaml
+credential_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/credentials/hashicorp-vault-prod"
+    status: active
+
+  name: "HashiCorp Vault Production"
+  backend_type: 
+
+  connection:
+    endpoint: https://vault.corp.example.com:8200
+    auth_method: 
+    namespace: 
+
+  credential_types: [hmac_secret, api_key, certificate, connection_string,
+                     bearer_token, private_key, username_password, ldap_bind]
+
+  health_check:
+    interval_seconds: 60
+    on_unhealthy: 
+    # suspend_dependents: suspend all components using this provider
+    # fail_open: continue using cached credentials (risk โ€” use cautiously)
+
+  caching:
+    enabled: true
+    ttl_seconds: 300             # refresh from vault every 5 minutes
+```
+
+**Credential references** โ€” used everywhere a secret is needed:
+
+```yaml
+# In webhook authentication
+secret_ref:
+  credential_provider_uuid: 
+  secret_path: "dcm/webhooks/payments/hmac-secret"
+  version: latest
+
+# In Auth Provider connection
+bind_password_ref:
+  credential_provider_uuid: 
+  secret_path: "dcm/auth/freeipa/bind-password"
+
+# In Service Provider registration
+credentials_ref:
+  credential_provider_uuid: 
+  secret_path: "dcm/providers/kubevirt/service-account"
+```
+
+Credentials are cached in memory per the configured TTL. On cache miss, DCM fetches from the Credential Provider. Credentials never appear in audit records (only the `secret_path` is recorded), never in Git, never in logs.
+
+---
+
+## 7. The Authentication Ladder
+
+Every rung is authenticated. The ladder is about setup effort โ€” not whether authentication exists.
+
+| Profile | Auth Modes Available | Setup Effort | Notes |
+|---------|---------------------|-------------|-------|
+| `minimal` | Static API key, Local user/password | 30 seconds โ€“ 2 minutes | Generated at bootstrap; zero external config |
+| `dev` | + GitHub/GitLab OAuth, FreeIPA/AD (direct bind) | 5โ€“15 minutes | OAuth requires app registration; LDAP requires server config |
+| `standard` | + OIDC via broker (Dex/Keycloak), AD/FreeIPA direct | 30โ€“60 minutes | Enterprise directory or IdP integration |
+| `prod` | + OIDC direct, MFA | 1โ€“2 hours | Full enterprise IdP; MFA configurable |
+| `fsi` | + mTLS required, MFA required | 4โ€“8 hours | Certificate infrastructure required |
+| `sovereign` | + Air-gapped OIDC/mTLS | 1โ€“2 days | No external auth dependencies |
+
+### 7.1 First-Run Setup
+
+```
+DCM First Run Setup
+โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•
+
+Welcome to DCM. Choose an authentication mode:
+
+  [1] Static API Key    โ€” Instant start. One key, full access.
+                          Best for: solo home lab, quick evaluation.
+
+  [2] Local Users       โ€” Create usernames and passwords.
+                          Best for: small team, dev environment.
+
+  [3] GitHub OAuth      โ€” Login with GitHub accounts.
+                          Best for: dev teams using GitHub.
+
+  [4] Configure later   โ€” Start with API key, switch to OIDC/LDAP later.
+
+Choice [1]:
+```
+
+### 7.2 The Upgrade Path
+
+Authentication configuration is a DCM artifact โ€” versioned, stored in Git, upgradeable via standard lifecycle:
+
+```bash
+# Upgrade from static API key to FreeIPA LDAP
+dcm auth configure \
+  --provider-type freeipa \
+  --server ldaps://freeipa.corp.example.com:636 \
+  --bind-dn "uid=dcm-service,..." \
+  --bind-password-ref "dcm/auth/freeipa/bind-password"
+# DCM validates, runs in shadow mode, cuts over โ€” API key deprecated on schedule
+```
+
+---
+
+## 8. Profile-Governed Enforcement
+
+| Feature | minimal | dev | standard | prod | fsi | sovereign |
+|---------|---------|-----|---------|------|-----|----------|
+| Static API key | โœ… | โœ… | โŒ | โŒ | โŒ | โŒ |
+| Local user/password | โœ… | โœ… | โŒ | โŒ | โŒ | โŒ |
+| GitHub/GitLab OAuth | โœ… | โœ… | โœ… | โŒ | โŒ | โŒ |
+| LDAP direct bind | โœ… | โœ… | โœ… | โœ… | โœ… | โœ… |
+| OIDC (any provider) | โœ… | โœ… | โœ… | โœ… | โœ… | โœ… |
+| mTLS | โŒ | โŒ | Optional | Recommended | Required | Required |
+| MFA | โŒ | โŒ | Optional | Configurable | Required | Required |
+| Air-gapped OIDC | โŒ | โŒ | โŒ | โŒ | Optional | Required |
+| Anonymous access | โŒ | โŒ | โŒ | โŒ | โŒ | โŒ |
+
+---
+
+> **Session revocation lifecycle:** See [Session Token Revocation](35-session-revocation.md) for the complete session revocation model including AUTH-016โ€“AUTH-022 (actor deprovisioning, revocation registry, token introspection, concurrent session enforcement).
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `AUTH-001` | All DCM authentication must be handled through a registered Auth Provider. The built-in Auth Provider is always available and cannot be deregistered. |
+| `AUTH-002` | Multiple Auth Providers may be registered simultaneously. The ingress layer routes to the appropriate provider based on the authentication signal in the request. |
+| `AUTH-003` | Auth Provider trust level governs how DCM treats decisions: authoritative (accepted as-is), verified (with Policy Engine augmentation), advisory (full re-evaluation). |
+| `AUTH-004` | Auth Provider role and tenant mappings are versioned artifacts subject to standard DCM artifact lifecycle. Changes go through proposed โ†’ active validation. |
+| `AUTH-005` | If an Auth Provider becomes unhealthy, existing sessions remain valid until TTL expiry. New authentication attempts route to the configured fallback provider or are rejected. |
+| `AUTH-006` | The Auth Provider used to authenticate a request is recorded in the ingress block and carried into the audit record. Policies may act on auth_provider_uuid and provider_type. |
+| `AUTH-007` | Auth Provider configuration credentials must reference a registered Credential Provider. Plaintext credentials are rejected. |
+| `AUTH-008` | There is no anonymous access in any DCM profile. Minimal and dev profiles support lightweight authenticated modes requiring minimal setup. |
+| `AUTH-009` | Webhook and message bus inbound surfaces always require authentication regardless of active Profile. Anonymous actors are never permitted on these surfaces. |
+| `AUTH-010` | Rate limiting is enforced per authenticated actor. Limits are declared on the Auth Provider or webhook actor registration. |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should DCM support SCIM for automated user provisioning from enterprise IdPs? | Enterprise integration | โœ… Resolved โ€” SCIM 2.0 (RFC 7643 / RFC 7644) optional capability; provisions actors and group memberships; roles not SCIM-provisioned; suspend on deprovision default (AUTH-012) |
+| 2 | How does Auth Provider failover interact with in-flight requests during the transition? | Reliability | โœ… Resolved โ€” in-flight requests complete on cached tokens; failover chain for new auth; session TTL respected during outage; all providers unavailable โ†’ reject (AUTH-013) |
+| 3 | Should MFA enforcement be per-operation (step-up MFA) or per-session? | Security UX | โœ… Resolved โ€” two-tier MFA: per-session + step-up; policy declares which operations require step-up; PT10M step-up token TTL; profile-governed defaults (AUTH-014) |
+| 4 | Should the built-in Auth Provider's local user store be backed by a pluggable database? | Architecture | โœ… Resolved โ€” pluggable storage backend; SQLite for minimal/dev; PostgreSQL for standard+; encryption required for fsi/sovereign; local store for bootstrap/service accounts only (AUTH-015) |
+
+---
+
+## 11. Related Concepts
+
+- **Webhooks and Messaging** (doc 18) โ€” ingress/egress actor model; webhook actor registration
+- **Policy Organization** (doc 14) โ€” policies act on auth_provider_type and ingress fields
+- **Universal Audit Model** (doc 16) โ€” auth provider and ingress context in every audit record
+- **Credential Provider** โ€” resolves all Auth Provider connection secrets
+- **Universal Group Model** (doc 15) โ€” group memberships resolved via Auth Provider group sync
+
+## 10. Git Identity Resolution
+
+When DCM processes Git PR ingress, it must resolve the Git server's verified actor identity to a DCM actor with full role, group, and tenant scope context โ€” identical to web UI or API login for the same user.
+
+### 10.1 The Trust Model
+
+DCM trusts the **Git server's authentication assertion** โ€” not user-declared Git configuration. The Git server has already authenticated the user (via SSH key, OAuth token, or LDAP password). DCM receives the Git server's verified identity from the PR merge webhook and resolves it through the registered Auth Provider.
+
+```
+Git server authenticates user โ†’ PR merge webhook โ†’ DCM Auth Provider resolution โ†’ DCM actor
+```
+
+### 10.2 Resolution Methods
+
+| Method | When Used | Auth Provider |
+|--------|----------|--------------|
+| `oidc_subject_lookup` | Git server uses same OIDC/OAuth IdP as DCM | OIDC Auth Provider |
+| `ldap_username_lookup` | Git server authenticates via LDAP/AD | LDAP/AD Auth Provider |
+| `ssh_key_fingerprint` | SSH key-authenticated Git workflows | DCM SSH key registry |
+| `webhook_service_account` | Automated CI/CD Git workflows | Registered webhook actor |
+
+### 10.3 The Resolved Actor
+
+The resolved Git actor carries **identical** role, group, and tenant scope to the same user authenticating via web UI:
+
+```yaml
+# A user who logs into the web UI via FreeIPA LDAP gets:
+# roles: [sre], tenant_scope: [payments-uuid], groups: [payments-team-uuid]
+
+# The same user merging a Git PR via their Git account gets:
+# roles: [sre], tenant_scope: [payments-uuid], groups: [payments-team-uuid]
+# โ€” identical โ€” because both resolve through the same FreeIPA Auth Provider
+```
+
+This is the key invariant: **Git PR ingress does not grant different permissions than any other ingress surface.** The same Auth Provider, the same group mappings, the same tenant scope enforcement.
+
+### 10.4 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `AUTH-011` | Git PR actor identity resolution must use the registered Auth Provider. DCM trusts the Git server's verified identity assertion โ€” not user-declared Git configuration. The resolved actor carries the same role, group, and tenant scope as any other user authenticated via the same Auth Provider. |
+
+
+## 11. Auth Provider Gap Resolutions
+
+### 11.1 SCIM 2.0 User Provisioning (Q1)
+
+DCM supports SCIM 2.0 as an optional Auth Provider capability for enterprise deployments. SCIM automates actor lifecycle management โ€” provisioning, attribute updates, and deprovisioning โ€” from enterprise IdPs (Okta, Azure AD, Ping Identity, JumpCloud).
+
+```yaml
+scim_provider_config:
+  enabled: true
+  scim_version: "2.0"
+  endpoint: https://dcm.corp.example.com/scim/v2
+  auth:
+    mode: bearer_token
+    token_ref:
+      credential_provider_uuid: 
+      path: "dcm/auth/scim/bearer-token"
+
+  provisioned_resources:
+    dcm_actors: true             # create/update/deactivate DCM actor records
+    group_memberships: true      # manage DCM group memberships from IdP groups
+    role_assignments: false      # roles managed by DCM policy โ€” not SCIM
+
+  attribute_mapping:
+    idp_userName: actor.username
+    idp_email: actor.email
+    idp_displayName: actor.display_name
+    idp_department: actor.status_metadata.department
+    idp_groups: actor.groups     # IdP groups โ†’ DCM group memberships (where mapped)
+
+  on_user_deprovisioned:
+    action: suspend              # suspend | deactivate | archive
+    # suspend: reversible; sessions terminated; leases released
+    in_flight_request_handling: complete_then_suspend
+```
+
+**What SCIM does NOT manage:** Roles are not SCIM-provisioned โ€” they require explicit DCM policy authorization. This prevents privilege escalation through the SCIM channel.
+
+### 11.2 Auth Provider Failover and In-Flight Requests (Q2)
+
+In-flight requests authenticated before Auth Provider failure continue to completion using cached session tokens. New requests follow the declared failover chain.
+
+```yaml
+auth_failover_config:
+  primary_provider_uuid: 
+  failover_chain:
+    - provider_uuid: 
+      promotion_delay: PT30S          # wait 30s before promoting failover
+    - provider_uuid: 
+  session_cache:
+    enabled: true
+    ttl: PT8H                         # valid sessions remain valid during outage
+```
+
+**Three scenarios:**
+- **Mid-assembly request (already authenticated):** Continues to completion โ€” session token carries resolved roles/groups/tenant scope; Auth Provider not needed for assembly
+- **New request, Auth Provider down:** Follows failover chain; served from session cache if session still valid
+- **Session expiry during outage:** Requires re-authentication via available failover provider; if all unavailable โ†’ reject with clear error
+
+### 11.3 Step-Up MFA (Q3)
+
+MFA enforcement is two-tier: per-session (validated at login, captured in `mfa_verified` field) and step-up (additional challenge at sensitive operations within an already-authenticated session).
+
+```yaml
+step_up_mfa_config:
+  step_up_required_for:
+    - platform_policy_activate
+    - provider_decommission
+    - tenant_decommission
+    - sovereignty_zone_change
+    - auth_provider_update
+    - manual_rehydration          # if entity min_auth_level = hardware_token_mfa
+  step_up_method: 
+  step_up_token_ttl: PT10M
+  step_up_challenge_max_age: PT5M
+```
+
+**Profile defaults:**
+
+| Profile | Per-Session MFA | Step-Up Required |
+|---------|----------------|-----------------|
+| minimal | No | No |
+| dev | No | No |
+| standard | Recommended | Optional |
+| prod | Required | Destructive operations |
+| fsi | Required | All policy changes |
+| sovereign | Required (hardware token) | All administrative operations |
+
+### 11.4 Built-In Auth Provider Storage Backend (Q4)
+
+The built-in Auth Provider's local user store uses a pluggable storage backend following the Storage Provider model.
+
+```yaml
+builtin_auth_provider_config:
+  user_store:
+    profile_defaults:
+      minimal: sqlite            # zero infrastructure; single-file
+      dev: sqlite
+      standard: postgresql       # durable; concurrent; backupable
+      prod: postgresql
+      fsi: postgresql            # encrypted storage (TDE required)
+      sovereign: postgresql      # HSM-backed encryption required
+    encryption_at_rest:
+      required_profiles: [fsi, sovereign]
+      key_ref:
+        credential_provider_uuid: 
+        path: "dcm/auth/builtin/encryption-key"
+```
+
+**The local user store should only contain:** bootstrap users, service accounts, and API key holders. Enterprise users belong in external Auth Providers (LDAP, OIDC, SCIM).
+
+### 11.5 System Policies โ€” Auth Provider Gaps
+
+| Policy | Rule |
+|--------|------|
+| `AUTH-012` | DCM supports SCIM 2.0 as an optional Auth Provider capability. SCIM provisions and deprovisions DCM actors and group memberships. Roles are not SCIM-provisioned โ€” they require explicit DCM policy authorization. SCIM deprovisioning suspends actors by default; in-flight requests complete before suspension. |
+| `AUTH-013` | In-flight requests authenticated before Auth Provider failure continue using cached session tokens. New requests follow the declared failover chain. Sessions remain valid for their declared TTL during outages. Session expiry during outage requires re-authentication via available failover provider. All providers unavailable โ†’ new authentication rejected. |
+| `AUTH-014` | MFA enforcement is two-tier: per-session MFA (captured in mfa_verified field) and step-up MFA (additional challenge at sensitive operations). Policy declares which operations require step-up regardless of session MFA status. Step-up tokens are short-lived (PT10M default). Profile governs default requirements. |
+| `AUTH-015` | The built-in Auth Provider uses a pluggable storage backend. SQLite is the default for minimal/dev profiles. PostgreSQL is the default for standard+ profiles. FSI and sovereign profiles require encryption at rest. The local user store should only contain bootstrap users, service accounts, and API key holders. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/20-registry-governance.md b/content/docs/data-model/20-registry-governance.md
new file mode 100644
index 0000000..b6ac267
--- /dev/null
+++ b/content/docs/data-model/20-registry-governance.md
@@ -0,0 +1,643 @@
+# DCM Data Model โ€” Registry Governance
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Auth Providers](19-auth-providers.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + PROVIDER**
+>
+> Data: registry artifacts. Provider: Registry Provider extension
+
+
+
+---
+
+## 1. Purpose
+
+The Resource Type Registry is the authoritative catalog of Resource Type Specifications available to DCM deployments. It governs what resources can be requested, how they are defined, and how those definitions evolve over time. Registry governance defines how new types are proposed, reviewed, approved, versioned, deprecated, and distributed โ€” including in air-gapped and sovereign deployments.
+
+Registry governance follows the same principles as all other DCM governance: GitOps-managed, policy-driven, profile-governed for ease of use, and audited.
+
+---
+
+## 2. The Three-Tier Registry
+
+### 2.1 Registry Tiers
+
+| Tier | Name | Maintained By | Contains | Governed By |
+|------|------|--------------|---------|------------|
+| 1 | **DCM Core** | DCM Project team | Universal resource types | DCM maintainers + community |
+| 2 | **Verified Community** | Named community maintainers | Technology/platform-specific types | Named maintainer(s) + DCM oversight |
+| 3 | **Organization** | Deploying organization | Organization-specific/proprietary types | Organization's own process |
+
+**Tier 1 examples:** `Compute.VirtualMachine`, `Network.VLAN`, `Network.IPAddress`, `Storage.Block`, `Storage.File`, `Container.Pod`
+
+**Tier 2 examples:** `OpenStack.HeatStack`, `VMware.NSXSegment`, `KubeVirt.VirtualMachine`, `Ansible.Playbook`
+
+**Tier 3 examples:** `Acme.LegacyMainframeJob`, `Corp.ServiceNowTicket`, `Internal.ComplianceReport`
+
+
+### 2a. Three-Tier Model Applied to All Artifact Types
+
+The three-tier registry model applies to all DCM artifact types, not just resource type specs. Every artifact in DCM has a tier that determines its trust level and the review requirements for changes:
+
+| Tier | Maintained by | Examples | Review for changes |
+|------|--------------|---------|-------------------|
+| **Core** | DCM Project | Built-in policies, base layers, system resource types | DCM project PR process |
+| **Verified Community** | Named community maintainers | Community resource types, shared policy templates, vetted provider specs | Community review + platform admin acceptance |
+| **Organization** | Deploying organization | Tenant policies, provider catalog items, org-specific specs | Per profile (auto โ†’ authorized) |
+
+**Contributor sub-tiers within Organization tier:**
+- `organization/platform` โ€” authored by platform admins; highest trust in org tier
+- `organization/provider` โ€” authored by registered Service Providers; scoped to their resource types
+- `organization/tenant` โ€” authored by Consumer/Tenant actors; scoped to their Tenant
+
+This means a tenant-authored GateKeeper policy is Organization/Tenant tier โ€” it has lower inherent trust than a platform-authored policy at the same domain level, and may require additional review per the active profile. See [Federated Contribution Model](28-federated-contribution-model.md).
+
+
+### 2.2 The Federated Registry Model
+
+The registry uses a federated model โ€” not centralized, not fully distributed. This supports air-gapped and sovereign deployments without external dependencies.
+
+```
+DCM Project Registry (authoritative origin)
+  Published at: registry.dcm-project.github.io
+  Contains: Tier 1 Core + Tier 2 Verified Community
+  โ”‚
+  โ–ผ  Sync (scheduled pull)
+Organization Registry (local mirror)
+  Hosted internally by the deploying organization
+  Adds: Tier 3 Organization-specific types
+  Authoritative for: this organization's DCM deployments
+  Can operate offline: yes โ€” pulls during sync windows
+  โ”‚
+  โ–ผ  Signed bundle transfer (for air-gapped)
+Air-gapped Registry (offline copy)
+  No external connectivity required
+  Updated via signed bundles verified against org public key
+  Authoritative for: this sovereign/air-gapped deployment
+```
+
+---
+
+## 3. Proposal and Review Workflow
+
+### 3.1 The PR-Based Proposal Flow
+
+Resource Type proposals follow a GitOps PR-based workflow โ€” not form submissions or tickets. A proposal is a Pull Request against the registry repository.
+
+```
+1. Author creates Resource Type Specification draft
+   โ”œโ”€โ”€ Standard artifact format (uuid, handle, version, status: developing)
+   โ”œโ”€โ”€ Schema definition
+   โ”œโ”€โ”€ Lifecycle declarations
+   โ”œโ”€โ”€ Declared dependencies (must exist in registry)
+   โ””โ”€โ”€ At least one example request payload
+
+2. Author opens Pull Request
+   โ”œโ”€โ”€ PR template: use case justification, example provider implementation,
+   โ”‚   test cases, schema validation passing
+   โ””โ”€โ”€ Status automatically set to: proposed (on PR open)
+
+3. Automated validation gates (must all pass before review begins)
+   โ”œโ”€โ”€ Schema validator passes
+   โ”œโ”€โ”€ No FQN conflict with existing active entries
+   โ”œโ”€โ”€ All declared dependencies resolve
+   โ”œโ”€โ”€ Breaking change detector (if version > 1.0.0)
+   โ””โ”€โ”€ Test case coverage (at least one valid example payload)
+
+4. Community review period (see Section 3.2)
+
+5. Maintainer approval + merge
+   โ””โ”€โ”€ Status: proposed โ†’ enters shadow validation
+
+6. Shadow validation period (same duration as review period)
+   โ”œโ”€โ”€ Specification available to DCM deployments opted into proposed feed
+   โ”œโ”€โ”€ Issues reported back as PR comments
+   โ””โ”€โ”€ Must pass without critical issues before promotion
+
+7. Promotion to active
+   โ””โ”€โ”€ Status: active โ†’ available in standard registry feed
+```
+
+### 3.2 Review Periods by Change Type
+
+| Change Type | Min Review Period | Shadow Validation | Approvers Required |
+|-------------|-----------------|-------------------|-------------------|
+| New Tier 1 resource type | 14 days | 14 days | 2 DCM maintainers |
+| New Tier 2 resource type | 7 days | 7 days | 1 DCM maintainer + named tier maintainer |
+| Minor version (non-breaking) | 7 days | 7 days | 1 DCM maintainer |
+| Revision (config data only) | 3 days | 3 days | 1 DCM maintainer (or auto-approve if CI passes) |
+| Breaking change (major version) | 21 days | 21 days | 2 DCM maintainers + community comment period |
+| Deprecation | 30 days | N/A | 2 DCM maintainers + affected provider notification |
+| Emergency (security) | Waived | 7 days minimum | 2 DCM maintainers + immediate notification |
+
+---
+
+## 4. Versioning
+
+### 4.1 Version Schema
+
+Resource Type Specifications use semantic versioning: `Major.Minor.Revision`
+
+| Component | Meaning | Compatibility |
+|-----------|---------|--------------|
+| **Major** | Breaking change โ€” field removed, type changed, behavior incompatible | Not compatible with previous major |
+| **Minor** | Non-breaking addition โ€” new optional fields, new lifecycle states | Compatible within major |
+| **Revision** | Configuration data change โ€” no structural change | Compatible within minor |
+
+### 4.2 Version Resolution Policy
+
+Version constraints in requests are **strictly enforced** โ€” DCM never silently resolves to a different version than declared. The resolution policy governs how much flexibility a consumer has:
+
+```yaml
+resource_type_version_constraint:
+  resource_type: Compute.VirtualMachine
+  version_policy: 
+  # exact:        Must match โ€” "1.2.3" means only 1.2.3
+  # compatible:   Same major โ€” "^1.2.3" means >= 1.2.3 < 2.0.0
+  # latest_minor: Latest revision of specified minor โ€” "~1.2" means 1.2.x
+  # latest:       Always use the latest active version
+  pinned_version: "1.2.3"   # required if version_policy: exact
+```
+
+**DCM never automatically upgrades across major versions regardless of version_policy.** Moving from v1.x to v2.x always requires explicit consumer action.
+
+### 4.3 Profile-Governed Version Policy Defaults
+
+| Profile | Default Version Policy | Rationale |
+|---------|----------------------|-----------|
+| `minimal` | `latest` | Home lab โ€” always current, no pinning overhead |
+| `dev` | `compatible` | Dev โ€” tracks major version, picks up fixes automatically |
+| `standard` | `compatible` | Production โ€” stable within major version |
+| `prod` | `compatible` | Production โ€” explicit major version control |
+| `fsi` | `exact` | Regulatory โ€” version-controlled for auditability |
+| `sovereign` | `exact` | Maximum control โ€” exact versions for reproducibility |
+
+---
+
+## 5. Deprecation Lifecycle
+
+### 5.1 The Default Deprecation Policy
+
+Deprecation lifecycle is governed by **default DCM system policies** โ€” not hard-coded values. These defaults can be overridden using the standard policy priority mechanism. Higher-priority organizational policies can shorten, extend, or lock any of these values.
+
+```yaml
+# Default deprecation lifecycle policies (platform domain โ€” overridable)
+deprecation_lifecycle_policies:
+
+  REG-DP-001:
+    name: "Default deprecation notification period"
+    value: P30D           # 30 days notice before deprecation status applied
+    override: allow       # organizations may change this
+
+  REG-DP-002:
+    name: "Default sunset period by tier"
+    values:
+      tier_1: P12M        # 12 months for Core registry types
+      tier_2: P6M         # 6 months for Verified Community types
+      tier_3: organization_governed
+    override: allow
+    profile_locks:
+      fsi: immutable      # FSI profile locks sunset periods
+      sovereign: immutable
+
+  REG-DP-003:
+    name: "Default migration window after retirement"
+    value: P90D           # 90 days after retirement โ€” realizations enter DEPRECATED_RUNTIME
+    override: allow
+
+  REG-DP-004:
+    name: "Migration target declaration"
+    requirement: required_in_deprecation_notice
+    # Deprecation notice must declare: successor type or explicit migration guidance
+    override: allow
+
+  REG-DP-005:
+    name: "Behavior on retirement โ€” new requests"
+    value: reject         # retired types reject new requests (not warn โ€” reject)
+    override: not_permitted   # this is structural โ€” cannot be changed
+
+  REG-DP-006:
+    name: "Behavior on retirement โ€” existing realizations"
+    value: deprecated_runtime_state
+    # Existing realizations enter DEPRECATED_RUNTIME state:
+    # - Eligible for: modify, decommission, drift detection
+    # - Not eligible for: rehydration using deprecated type
+    # - Not automatically destroyed
+    override: allow
+
+  REG-DP-007:
+    name: "Emergency deprecation migration window"
+    value: P30D           # minimum 30 days even for security emergency
+    override: not_permitted   # floor cannot be removed
+```
+
+### 5.2 Deprecation Lifecycle Flow
+
+```
+Resource Type in active status
+  โ”‚
+  โ–ผ  Deprecation proposal (PR + 30 day review)
+Status: deprecated
+  โ”‚  Notification dispatched to:
+  โ”‚  - All registered providers implementing this type
+  โ”‚  - All organizations with active realizations
+  โ”‚  - All webhook registrations subscribed to registry events
+  โ”‚
+  โ–ผ  Sunset period (P12M Tier 1 / P6M Tier 2 โ€” per REG-DP-002)
+  โ”‚  During sunset:
+  โ”‚  - New requests: succeed with deprecation warning
+  โ”‚  - Existing realizations: unaffected
+  โ”‚  - Drift detection: continues
+  โ”‚  - Provider implementations: remain valid
+  โ”‚
+  โ–ผ  Retirement (status: retired)
+  โ”‚  Existing realizations โ†’ DEPRECATED_RUNTIME state
+  โ”‚  New requests โ†’ rejected (REG-DP-005)
+  โ”‚
+  โ–ผ  Migration window (P90D โ€” per REG-DP-003)
+  โ”‚  Organizations migrate realizations to successor type
+  โ”‚  DEPRECATED_RUNTIME entities can be decommissioned or migrated
+  โ”‚
+  โ–ผ  Post-migration window
+     DEPRECATED_RUNTIME entities remain operational but unsupported
+     Drift detection: continues but remediation is manual
+```
+
+### 5.3 Overriding Deprecation Defaults
+
+Organizations use standard policy priority to customize deprecation behavior:
+
+```yaml
+# Organizational policy: extend Tier 2 sunset to 12 months
+policy:
+  domain: platform
+  priority: 600.0.0
+  type: gatekeeper
+  rule: >
+    If registry.deprecation.tier == tier_2
+    THEN override: sunset_period = P12M
+    basis: "Our tooling requires longer migration windows"
+```
+
+```yaml
+# FSI profile lock: sunset periods immutable
+policy:
+  domain: system
+  priority: 900.0.0
+  immutable_ceiling: absolute
+  rule: >
+    If active_profile IN [fsi, sovereign]
+    THEN lock: REG-DP-002 as immutable
+    rationale: "Regulatory change control requirements"
+```
+
+---
+
+## 6. Provider Selection Tie-Breaking
+
+When the placement engine has multiple viable provider candidates that satisfy all constraints equally, the following hierarchy resolves the tie deterministically:
+
+### 6.1 Tie-Breaking Hierarchy
+
+```
+Priority  Factor                    Condition
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€    โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+1         Policy preference         A Transformation policy injected a
+                                    preference_score or preferred_provider_uuid
+
+2         Provider priority         Providers declare a numeric priority
+                                    Higher value = preferred (default: 50)
+
+3         Tenant affinity           Tenant's Policy Group declares preferred
+                                    providers for specific resource types
+
+4         Cost analysis             Cost Analysis component has current data
+                                    AND cost is determinable for candidates
+                                    Prefer lower total cost (CapEx + OpEx)
+                                    SKIP if cost data absent or incomparable
+
+5         Least loaded              Current capacity utilization from reserve_query
+                                    If utilization differs > 10%: prefer less loaded
+                                    SKIP if utilization data unavailable
+
+6         Consistent hash           SHA-256(request_uuid + resource_type + sorted_candidate_uuids)
+                                    Deterministic โ€” same request always resolves
+                                    to same provider in a stable cluster
+                                    Never round-robin
+```
+
+### 6.2 Cost Analysis Integration
+
+Cost analysis ranks above operational load because cost is a business decision. When cost data is available and comparable:
+
+- **CapEx consideration:** provider infrastructure cost allocation per resource type
+- **OpEx consideration:** operational overhead, licensing, support costs per resource unit
+- **Comparability requirement:** cost must be expressed in the same currency and time period; if not comparable (different currencies, missing data), skip to step 5
+
+Cost data is sourced from the **Cost Analysis** control plane component. If Cost Analysis is not deployed or does not have current data for the candidate providers, this step is skipped without blocking placement.
+
+```yaml
+# Cost analysis in placement loop
+placement_cost_evaluation:
+  enabled: true                    # false if Cost Analysis unavailable
+  data_freshness_max: PT1H         # reject cost data older than 1 hour
+  comparison_threshold: 0.05       # 5% cost difference to trigger preference
+  # If candidates are within 5% cost: skip cost as a tiebreaker
+  cost_components:
+    - capex_allocation_per_unit
+    - opex_per_unit_per_hour
+    - licensing_per_unit
+```
+
+### 6.3 Provider Priority Declaration
+
+```yaml
+provider_registration:
+  provider_priority: 100   # default: 50; higher = preferred when equal
+  cost_metadata:
+    capex_allocation_per_unit: 12.50    # USD per VM-month
+    opex_per_unit_per_hour: 0.08       # USD per VM-hour
+    currency: USD
+    last_updated: 
+```
+
+---
+
+## 7. The Registry Provider
+
+### 7.1 Concept
+
+The Registry Provider is a specialized sub-type of Information Provider โ€” the mechanism through which a DCM deployment accesses its authoritative Resource Type Registry. Every DCM deployment has exactly one active Registry Provider.
+
+### 7.2 Registration
+
+```yaml
+registry_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/registry/org-primary"
+    version: "1.0.0"
+    status: active
+
+  name: "Organization Primary Registry"
+  provider_type: registry              # sub-type of information_provider
+
+  # Registry source
+  registry_url: https://registry.corp.example.com
+  tier_1_source: https://registry.dcm-project.github.io   # upstream pull
+  tier_2_sources:
+    - https://registry.dcm-project.github.io
+    - https://registry.partner-org.example.com            # verified partner
+
+  # Sync configuration
+  sync:
+    schedule: "0 2 * * *"            # nightly pull from upstream
+    on_sync_failure: 
+    cache_ttl: P7D                   # use cached if upstream unavailable
+
+  # Air-gapped / sovereign configuration
+  offline_mode: false                # true: no external connectivity
+  signed_bundle_import: false        # true: updates via signed bundles only
+  bundle_signing_key_ref:
+    credential_provider_uuid: 
+    secret_path: "dcm/registry/bundle-verification-key"
+
+  # Sovereignty filtering
+  sovereignty_filter:
+    enabled: true
+    permitted_jurisdictions: [eu-west, eu-central]
+    # Only activate resource types flagged as compatible with these jurisdictions
+
+  # Vendor approval list
+  vendor_allowlist:
+    enabled: false                   # true in prod/fsi/sovereign
+    permitted_vendors: [dcm-project, vmware, redhat, hashicorp]
+    # Resource types from non-listed vendors are not activated
+```
+
+### 7.3 Signed Bundle Model (Air-Gapped Updates)
+
+```
+Online workstation (with registry access)
+  โ”‚
+  Pull registry delta since last sync
+  Sign with organization private key (via Credential Provider)
+  Package: registry-update-YYYY-MM-DD.bundle
+  โ”‚
+  Transfer via approved secure channel
+  โ”‚
+Air-gapped DCM deployment
+  โ”‚
+  Verify signature against organization public key
+  Import bundle โ†’ update local registry
+  Emit: registry.sync_completed audit event
+```
+
+### 7.4 Registry Provider Policies
+
+The Registry Provider is fully policy-governed โ€” policies act on registry operations at every stage:
+
+**Sovereignty enforcement:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_sync
+  rule: "If resource_type.jurisdiction_compatibility NOT CONTAINS tenant.sovereignty_zone THEN reject_activation"
+```
+
+**Vendor allowlist:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_activation
+  rule: "If resource_type.publisher NOT IN approved_vendor_list THEN gatekeep: require_manual_approval"
+```
+
+**Bundle verification:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_bundle_import
+  rule: "If bundle.signature_valid == false THEN reject: unsigned bundles not permitted"
+```
+
+**Version pinning in production:**
+```yaml
+policy:
+  type: gatekeeper
+  target: registry_sync
+  rule: "If active_profile == prod AND resource_type.version_delta.type == major THEN gatekeep: major version upgrades require manual approval"
+```
+
+**Audit all syncs:**
+```yaml
+policy:
+  type: transformation
+  target: registry_sync
+  rule: "Always inject: sync_audit.required = true, sync_audit.reviewer = platform_admin"
+```
+
+### 7.5 Profile-Appropriate Registry Policy Groups
+
+DCM ships built-in registry policy groups โ€” one per profile, activated automatically:
+
+| Group Handle | Profile | Key Behaviors |
+|-------------|---------|--------------|
+| `system/group/registry-minimal` | minimal | Advisory only; pull everything; no restrictions; warn on unverified sources |
+| `system/group/registry-dev` | dev | Warn on unverified sources; pull Tier 1+2; no vendor restrictions |
+| `system/group/registry-standard` | standard | Block unverified sources; Tier 1+2 only; sovereignty filter enabled |
+| `system/group/registry-prod` | prod | Strict version pinning; approved vendor list; audit all syncs; major version manual approval |
+| `system/group/registry-fsi` | fsi | Exact version pinning; approved vendor list; immutable sunset periods; all syncs audited with dual approval |
+| `system/group/registry-sovereign` | sovereign | Signed bundles only; offline registry; no external connectivity; bundle verification required |
+
+Organizations can replace or extend these groups using standard Policy Group composition.
+
+---
+
+## 8. DCM System Policies
+
+| Policy | Rule |
+|--------|------|
+| `REG-001` | Resource Type proposals follow a PR-based GitOps workflow with automated validation gates (schema, FQN conflict, dependency resolution, breaking change detection) that must all pass before review begins. |
+| `REG-002` | All registry changes require a minimum review period by change type and a mandatory shadow validation period in `proposed` status before promotion to `active`. |
+| `REG-003` | Deprecation lifecycle is governed by default policies REG-DP-001 through REG-DP-007. These defaults are overridable via standard policy priority except where locked by active Profile. |
+| `REG-004` | Version constraints in requests are strictly enforced. DCM never automatically upgrades across major versions regardless of version_policy. Version resolution policy is profile-governed. |
+| `REG-005` | When multiple providers satisfy all placement criteria equally, the tie-breaking hierarchy applies: policy preference โ†’ provider priority โ†’ tenant affinity โ†’ cost analysis (if available) โ†’ least loaded โ†’ consistent hash on request_uuid. |
+| `REG-006` | The registry uses a federated model. Air-gapped and sovereign deployments use offline registries populated via signed bundles verified against the organization's public key. |
+| `REG-007` | The Registry Provider is policy-governed. Profile-appropriate registry policy groups are activated by default. Organizations may extend or replace these groups using standard Policy Group composition. |
+| `REG-DP-001` | Default deprecation notification period: P30D before deprecation status is applied. Overridable. |
+| `REG-DP-002` | Default sunset period: Tier 1 = P12M, Tier 2 = P6M. Overridable; locked as immutable in fsi and sovereign profiles. |
+| `REG-DP-003` | Default migration window after retirement: P90D. Overridable. |
+| `REG-DP-004` | Deprecation notices must declare a successor type or explicit migration guidance. Overridable. |
+| `REG-DP-005` | Retired resource types reject new requests. Not overridable โ€” structural. |
+| `REG-DP-006` | Existing realizations of retired types enter DEPRECATED_RUNTIME state โ€” eligible for modify and decommission, not rehydration. Overridable. |
+| `REG-DP-007` | Emergency deprecation minimum migration window: P30D. Not overridable โ€” floor cannot be removed. |
+
+---
+
+## 9. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should there be a certified registry tier between Tier 2 and DCM Core for formally audited types? | Ecosystem | โœ… Resolved โ€” no fourth tier; certification metadata within existing tier structure serves same purpose (REG-008) |
+| 2 | Should organizations be able to publish their Tier 3 types to the Verified Community registry? | Community | โœ… Resolved โ€” Tier 3 to Tier 2 promotion via PR pathway with additional requirements: production deployment + OSS license + named maintainer + migration path (REG-009) |
+| 3 | How does the Registry Provider handle a scenario where the upstream DCM Project Registry is permanently unavailable? | Resilience | โœ… Resolved โ€” Organization Registry mirror is self-sufficient; upstream loss is governance decision not operational crisis; three long-term options (REG-010) |
+| 4 | Should cost metadata on provider registrations be sourced from the Cost Analysis component or declared statically? | Architecture | โœ… Resolved โ€” static or Cost Analysis sourcing; hybrid with Cost Analysis preferred; placement engine uses freshest available (REG-011) |
+
+---
+
+## 10. Related Concepts
+
+- **Resource Type Hierarchy** (doc 05) โ€” the structure of Resource Type Specifications
+- **Policy Organization** (doc 14) โ€” Policy Groups governing registry behavior
+- **Deployment and Redundancy** (doc 17) โ€” registry sync and offline operation
+- **Auth Providers** (doc 19) โ€” authentication for registry access
+- **Universal Audit Model** (doc 16) โ€” all registry operations produce audit records
+
+
+## 11. Registry Governance Gap Resolutions
+
+### 11.1 No Fourth Registry Tier โ€” Certification Metadata Instead (Q1)
+
+A formal fourth registry tier is not introduced. Resource Type Specifications in any tier may carry certification metadata from recognized certifying bodies. Certification provides equivalent assurance to a separate tier without the governance complexity.
+
+```yaml
+resource_type_spec:
+  registry_tier: 2
+  tier_certifications:
+    - certifying_body: "OpenStack Foundation"
+      certification: "OpenStack Powered"
+      certified_versions: [">=2023.1"]
+      certificate_ref: 
+```
+
+Users seeking "formally audited types" filter on certification metadata โ€” same result as a separate tier, without the structural fragmentation.
+
+### 11.2 Tier 3 to Tier 2 Promotion Pathway (Q2)
+
+Organizations may promote Tier 3 Resource Type Specifications to Tier 2 (Verified Community) via the standard PR-based promotion pathway with additional requirements.
+
+```yaml
+tier_3_to_tier_2_promotion:
+  requirements:
+    - at_least_one_production_deployment: true
+    - documented_use_case: true
+    - open_source_license_compatible: true   # DCM is Apache 2.0
+    - named_community_maintainer: true
+    - test_suite_included: true
+    - migration_path_from_tier3: documented  # handles name changes, schema diffs
+  review_period: 14 days
+  existing_tier3_users_notified: true        # current users notified of promotion
+```
+
+The promotion pathway gives organizations a route from internal tooling to community contribution without requiring a ground-up rewrite. The migration path documentation ensures existing Tier 3 deployments can upgrade smoothly.
+
+### 11.3 Upstream Registry Permanently Unavailable (Q3)
+
+The Organization Registry mirror operates independently from the upstream DCM Project Registry. Permanent upstream loss is a governance decision, not an operational crisis.
+
+**Short-term:** Organization Registry mirror is self-sufficient for all operations. Existing types continue working normally.
+
+**Medium-term:** Registry Provider enters "independent operation" mode โ€” new Tier 1/2 types cannot be added (no upstream to sync from); existing types continue operating; Tier 3 unaffected (organization-governed).
+
+**Long-term governance options:**
+- **Option A:** Designate a community mirror as the new upstream (community self-governance)
+- **Option B:** Fork the registry โ€” organization takes ownership of their copy
+- **Option C:** Continue as independent installation (no new community types)
+
+The mirror's self-sufficiency means existing deployments never experience an operational outage due to upstream unavailability.
+
+### 11.4 Provider Cost Metadata Source (Q4)
+
+Provider cost metadata may be declared statically or sourced dynamically from the Cost Analysis component.
+
+```yaml
+provider_cost_metadata:
+  source: 
+
+  static:
+    capex_allocation_per_unit: 12.50   # USD per VM-month
+    opex_per_unit_per_hour: 0.08
+    currency: USD
+    last_updated: 
+
+  cost_analysis:
+    query_interval: PT1H               # refresh cost data hourly
+    fallback: static
+    fallback_max_age: PT24H            # use static if Cost Analysis data older than 24h
+
+  hybrid:
+    prefer: cost_analysis
+    static_for_unavailable: true
+```
+
+The placement engine's cost analysis step (tie-breaking step 4) uses whichever source is freshest and available โ€” Cost Analysis preferred, static as fallback. No changes required to the placement tie-breaking model.
+
+### 11.5 System Policies โ€” Registry Governance Gaps
+
+| Policy | Rule |
+|--------|------|
+| `REG-008` | A formal fourth registry tier is not introduced. Resource Type Specifications in any tier may carry certification metadata from recognized certifying bodies. Certification metadata is a filter criterion โ€” not a structural tier boundary. |
+| `REG-009` | Organizations may promote Tier 3 Resource Type Specifications to Tier 2 via the standard PR-based promotion pathway with additional requirements: at least one production deployment, OSS-compatible license, named community maintainer, and documented migration path from the Tier 3 handle. |
+| `REG-010` | The Organization Registry mirror operates independently from the upstream DCM Project Registry. Permanent upstream unavailability does not affect existing operations. New community type adoption requires a designated community mirror, organizational fork, or independent operation decision. |
+| `REG-011` | Provider cost metadata may be declared statically or sourced dynamically from the Cost Analysis component. Hybrid mode uses Cost Analysis when available and falls back to static. The placement engine uses whichever source is freshest and available. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/21-information-providers-advanced.md b/content/docs/data-model/21-information-providers-advanced.md
new file mode 100644
index 0000000..e3aa053
--- /dev/null
+++ b/content/docs/data-model/21-information-providers-advanced.md
@@ -0,0 +1,704 @@
+# DCM Data Model โ€” Information Providers: Confidence Scoring, Authority, and Conflict Resolution
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Information Providers](10-information-providers.md) | [Policy Organization](14-policy-profiles.md) | [Universal Audit Model](16-universal-audit.md) | [DCM Federation](22-dcm-federation.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: PROVIDER**
+>
+> The Provider abstraction โ€” Information Provider advanced capabilities
+
+
+
+---
+
+## 1. Purpose
+
+This document extends the base Information Provider model with the advanced concepts required for enterprise-grade information governance: confidence scoring for all provider-supplied data, authority and priority declarations as layer-defined organizational knowledge, ingestion-time conflict detection with policy-driven resolution, write-back capabilities, schema versioning, the well-known Information Provider Registry, and air-gapped verification.
+
+---
+
+## 2. Confidence Scoring โ€” The Hybrid Descriptor Model
+
+### 2.1 Purpose and Design Goals
+
+Every field value supplied by an Information Provider carries a confidence descriptor. DCM aggregates data from multiple external sources โ€” CMDB, HR systems, IPAM, asset management, monitoring tools โ€” each with different freshness, authority, and reliability. The confidence model answers: **how much should you trust this field value?**
+
+Three goals drive the design:
+- **Accuracy** โ€” each dimension of confidence is independently meaningful and auditable
+- **Reliability** โ€” a derived numeric score enables mathematical composition for placement decisions and conflict resolution
+- **Ease of use** โ€” a derived band (very_high through very_low) is what humans and policies work with day to day
+
+### 2.2 The Primary Data Model โ€” Confidence Descriptor
+
+The **confidence_descriptor** is the primary data model. The score and band are derived from it โ€” not the other way around. This separation removes false precision: the score is explicitly a convenience derivation, not an independent measurement.
+
+```yaml
+field_confidence:
+  # PRIMARY โ€” stored; set at specific lifecycle points
+  authority_level: primary          # set at provider registration from authority layer
+  corroboration: single_source      # set at ingestion; updated on subsequent pushes
+  source_trust: verified            # maintained by trust scoring system (INF-009)
+  last_updated_at:        # set at each push event
+  source_provider_uuid:       # set at ingestion
+
+  # DERIVED โ€” computed on demand; never stored as primary
+  freshness: high                   # computed: (now - last_updated_at) vs thresholds
+  data_age_minutes: 87              # computed: now - last_updated_at
+  score: 86                         # computed: from descriptor components
+  band: high                        # computed: from score vs band thresholds
+```
+
+### 2.3 Who Sets Each Descriptor Field
+
+| Field | Set By | When | How |
+|-------|--------|------|-----|
+| `authority_level` | Authority declaration layer | Provider registration | Organizational knowledge โ€” static per field per provider |
+| `corroboration` | DCM ingestion pipeline | Each push event | Compared against existing values from other providers |
+| `source_trust` | DCM trust scoring system | Event-triggered + weekly | Push failures, schema errors, health check, re-verification |
+| `last_updated_at` | DCM ingestion pipeline | Each push event | Timestamp of the push event |
+| `freshness` | DCM โ€” derived | Query time | Computed from `now - last_updated_at` vs thresholds |
+| `score` | DCM โ€” derived | Query time | Computed from descriptor components |
+| `band` | DCM โ€” derived | Query time | Computed from score vs band thresholds |
+
+**DCM computes all derived values โ€” providers never self-declare confidence.**
+
+### 2.4 Descriptor Component Values
+
+**`authority_level`** โ€” from the authority declaration layer:
+
+| Value | Meaning |
+|-------|---------|
+| `primary` | Declared primary authoritative source for this field |
+| `secondary` | Corroborating source; used if primary unavailable |
+| `advisory` | Context only; never used for decisions |
+| `discovered` | Value found via active interrogation |
+| `self_reported` | Entity reported its own value |
+| `inferred` | Value inferred from other data |
+
+**`corroboration`** โ€” computed at ingestion time:
+
+| Value | Condition | Confidence Effect |
+|-------|-----------|-----------------|
+| `confirmed` | 2+ providers agree on this value | Increases confidence |
+| `single_source` | Only one provider has asserted this value | Neutral |
+| `contested` | 2+ providers disagree on this value | Reduces confidence |
+
+**`source_trust`** โ€” maintained by trust scoring system:
+
+| Value | Condition | Confidence Effect |
+|-------|-----------|-----------------|
+| `verified` | Provider identity, sovereignty, certs all current | Full confidence |
+| `degraded` | Provider has elevated error/conflict rate | Reduced confidence |
+| `suspended` | Provider below trust threshold; pushes stopped | No new data |
+
+**`freshness`** โ€” computed from `data_age_minutes`:
+
+| Value | Age Threshold |
+|-------|-------------|
+| `high` | < 1 hour |
+| `medium` | 1 hour โ€“ 1 day |
+| `low` | 1 day โ€“ 7 days |
+| `stale` | > 7 days |
+
+### 2.5 The Score Derivation Formula
+
+The score is a convenience number derived deterministically from the descriptor. It enables mathematical composition (cross-DCM scoring, conflict resolution ordering) where a single number is needed.
+
+```
+score = min(100, base(authority_level)
+              ร— freshness_multiplier(freshness)
+              ร— corroboration_multiplier(corroboration)
+              ร— trust_multiplier(source_trust))
+```
+
+**Base values by authority_level:**
+
+| authority_level | Base Score |
+|----------------|-----------|
+| `primary` | 90 |
+| `secondary` | 70 |
+| `discovered` | 60 |
+| `advisory` | 50 |
+| `self_reported` | 40 |
+| `inferred` | 30 |
+
+**Freshness multipliers:**
+
+| freshness | Multiplier |
+|-----------|-----------|
+| `high` | 1.00 |
+| `medium` | 0.95 |
+| `low` | 0.85 |
+| `stale` | 0.50 |
+
+**Corroboration multipliers:**
+
+| corroboration | Multiplier |
+|--------------|-----------|
+| `confirmed` | 1.15 |
+| `single_source` | 1.00 |
+| `contested` | 0.60 |
+
+**Trust multipliers:**
+
+| source_trust | Multiplier |
+|-------------|-----------|
+| `verified` | 1.00 |
+| `degraded` | 0.75 |
+| `suspended` | 0.00 |
+
+**Example:** Primary authority, fresh data (30 min old), single source, verified:
+`min(100, 90 ร— 1.00 ร— 1.00 ร— 1.00)` = **90**
+
+**Example:** Primary authority, medium freshness (4 hours), two sources agree, verified:
+`min(100, 90 ร— 0.95 ร— 1.15 ร— 1.00)` = 98.3 โ†’ **98**
+
+**Example:** Secondary authority, stale data (10 days), contested, degraded:
+`min(100, 70 ร— 0.50 ร— 0.60 ร— 0.75)` = 15.75 โ†’ **16**
+
+### 2.6 Score Bands โ€” For Policy Use
+
+Policies use bands, not raw scores. This avoids the brittleness of threshold values like "reject if score < 73":
+
+| Band | Score Range | Policy Label |
+|------|------------|-------------|
+| Very High | 81-100 | `very_high` |
+| High | 61-80 | `high` |
+| Medium | 41-60 | `medium` |
+| Low | 21-40 | `low` |
+| Very Low | 0-20 | `very_low` |
+
+```yaml
+# Policy using band โ€” clear and maintainable
+policy:
+  type: gatekeeper
+  rule: >
+    If field.owner_business_unit.band IN [very_low, low]
+    THEN gatekeep: "Business unit confidence insufficient โ€” manual verification required"
+
+# Policy using individual descriptor dimensions โ€” most precise
+policy:
+  type: gatekeeper
+  rule: >
+    If field.cost_center.corroboration == contested
+    THEN gatekeep: "Cost center is contested between providers โ€” resolve before provisioning"
+
+# Policy using score โ€” for mathematical thresholds
+policy:
+  type: gatekeeper
+  rule: >
+    If field.cost_center.score < 60
+    THEN gatekeep: "Cost center confidence below required threshold"
+```
+
+### 2.7 Derivation Chain Summary
+
+```
+STORED (authoritative):
+  authority_level + corroboration + source_trust + last_updated_at
+
+DERIVED AT QUERY TIME (deterministic from stored):
+  freshness โ† (now - last_updated_at) vs thresholds
+  score     โ† base(authority_level) ร— freshness_mult ร— corroboration_mult ร— trust_mult
+  band      โ† score vs band thresholds
+
+AUDIT RECORD (what auditors can reconstruct from):
+  authority_level (from registration)
+  corroboration (from ingestion event)
+  source_trust (from trust audit at that time)
+  last_updated_at (from push event timestamp)
+  โ†’ score and band fully reconstructable from these four stored fields
+```
+
+### 2.8 Configurable Derivation
+
+Organizations may configure the base scores, multipliers, and band thresholds via Policy Group. This allows domain-specific calibration without changing the underlying descriptor model:
+
+```yaml
+confidence_derivation_config:
+  # Override defaults for this deployment
+  base_scores:
+    primary: 90           # default โ€” can increase to 95 for high-trust environments
+    secondary: 70
+  freshness_thresholds:
+    high_max_minutes: 60  # default 60; can tighten to 15 for real-time requirements
+    medium_max_minutes: 1440
+    low_max_minutes: 10080
+  band_thresholds:
+    very_high_min: 81     # default; adjust as needed
+    high_min: 61
+```
+
+Adjusted derivation configs are stored as Policy Group artifacts โ€” versioned, auditable, and profile-governed.
+
+## 3. Authority and Priority โ€” Layer-Defined
+
+### 3.1 Authority as Layer Data
+
+Information Provider authority scope and priority are **layer-defined** โ€” not just policy-driven. They represent static organizational knowledge about information architecture ("our CMDB is the authoritative source for business unit data"). This knowledge belongs in a `platform` domain layer โ€” versioned, GitOps-managed, and inherited by all requests.
+
+```yaml
+layer:
+  handle: "platform/information-authority/cmdb-authority"
+  domain: platform
+  priority: 600.0.0
+  concern_tags: [information-authority, cmdb, organizational-data]
+  fields:
+    information_authority:
+      primary_sources:
+        - provider_uuid: 
+          fields: [owner_business_unit, cost_center, cmdb_id, cmdb_location]
+          authority_level: primary
+          priority: 900.0.0
+        - provider_uuid: 
+          fields: [asset_tag, purchase_date, warranty_expiry, serial_number]
+          authority_level: primary
+          priority: 900.0.0
+      secondary_sources:
+        - provider_uuid: 
+          fields: [owner_business_unit, employee_id]
+          authority_level: secondary
+          priority: 500.0.0
+          # Secondary: corroborates primary; used if primary unavailable
+      advisory_sources:
+        - provider_uuid: 
+          fields: [reported_hostname, reported_ip]
+          authority_level: advisory
+          priority: 200.0.0
+          # Advisory: context only; never used for decisions
+```
+
+### 3.2 Priority Within Authority Level
+
+When multiple providers have the same `authority_level`, the `priority` field (using the same numeric priority schema as layers and policies) determines which value wins:
+
+```
+Higher priority value โ†’ wins when authority levels are equal
+Authority level hierarchy: primary > secondary > advisory
+Within same authority level: higher priority number wins
+```
+
+### 3.3 Policy Acting on Authority
+
+Policies can act on authority metadata at runtime:
+
+```yaml
+# Transformation: enrich payload with confidence-weighted values
+policy:
+  type: transformation
+  rule: >
+    If field.owner_business_unit.confidence_score < 60
+    AND field.owner_business_unit.authority_level != primary
+    THEN inject: request_flags.requires_manual_business_unit_verification = true
+
+# GateKeeper: require high confidence for financial operations
+policy:
+  type: gatekeeper
+  rule: >
+    If resource_type == Compute.VirtualMachine
+    AND field.cost_center.confidence_band IN [very_low, low]
+    THEN gatekeep: "Cost center assignment confidence insufficient for VM provisioning"
+```
+
+---
+
+## 4. Ingestion-Time Conflict Detection and Resolution
+
+### 4.1 Conflict Detection at Ingestion
+
+Conflict detection occurs at ingestion time โ€” when DCM receives a push event from an Information Provider. This is the correct architectural moment: before the data enters the entity record, not after.
+
+```
+Information Provider push event received
+  โ”‚
+  โ–ผ 1. Schema validation
+  โ”‚   Validate against provider's declared schema version
+  โ”‚   Reject on violation (strict) or warn (lenient per policy)
+  โ”‚
+  โ–ผ 2. Authority scope check
+  โ”‚   Is this provider authorized to assert values for these fields on this entity?
+  โ”‚   Reject unauthorized field assertions (INF-001)
+  โ”‚
+  โ–ผ 3. Confidence score computation
+  โ”‚   Compute per-field score using standard formula (Section 2.2)
+  โ”‚   Factor: source authority level, data freshness, corroboration
+  โ”‚
+  โ–ผ 4. Conflict detection
+  โ”‚   For each field: does an existing value exist from another provider?
+  โ”‚   Same value โ†’ corroboration (confidence increases, multiplier applied)
+  โ”‚   Different value โ†’ conflict record created
+  โ”‚   No existing value โ†’ new assertion (accept)
+  โ”‚
+  โ–ผ 5. Conflict resolution policy
+  โ”‚   Apply declared resolution strategy:
+  โ”‚   higher_authority_wins โ†’ use higher authority_level value
+  โ”‚   higher_confidence_wins โ†’ use higher confidence score
+  โ”‚   higher_priority_wins โ†’ use value from higher-priority provider
+  โ”‚   escalate โ†’ create conflict record; human resolves; existing value retained
+  โ”‚   merge โ†’ combine values (array/set fields only)
+  โ”‚
+  โ–ผ 6. Entity record update
+  โ”‚   Write accepted values with full field provenance
+  โ”‚
+  โ–ผ 7. Audit record โ€” INGEST action
+     All field changes, conflicts detected/resolved, confidence scores,
+     authority assertions recorded in Audit Store
+```
+
+### 4.2 The Conflict Record
+
+```yaml
+information_provider_conflict_record:
+  record_uuid: 
+  detected_at: 
+  field_path: owner_business_unit
+  subject_entity_uuid: 
+  conflicting_values:
+    - provider_uuid: 
+      authority_level: primary
+      priority: 900.0.0
+      value: "Payments Team"
+      confidence_score: 86
+      received_at: 
+    - provider_uuid: 
+      authority_level: secondary
+      priority: 500.0.0
+      value: "Infrastructure Team"
+      confidence_score: 72
+      received_at: 
+  resolution_strategy: higher_authority_wins
+  resolution_status: 
+  auto_resolved_value: "Payments Team"
+  auto_resolved_reason: "primary authority_level wins over secondary"
+  # If escalated:
+  resolved_by: 
+  resolved_value: "Payments Team"
+  resolution_reason: "CMDB is authoritative for business unit โ€” HR system has stale data"
+  resolution_audit_record_uuid: 
+```
+
+### 4.3 Authority Scope Conflicts at Registration
+
+When a new provider registers and declares authority over a field already claimed by an existing provider at the same or higher authority level, DCM detects the conflict at registration time and requires explicit resolution before the provider becomes active.
+
+```
+New provider registers authority_scope: [owner_business_unit, primary]
+  โ”‚
+  โ–ผ DCM checks: existing primary authority for owner_business_unit?
+  โ”‚
+  โ”œโ”€โ”€ No existing primary โ†’ register; no conflict
+  โ”‚
+  โ””โ”€โ”€ Existing primary provider found:
+        Create: authority_scope_conflict_record
+        Action required:
+          - Demote new provider to secondary, or
+          - Demote existing provider to secondary, or
+          - Declare explicit resolution strategy for this field
+        Provider registration blocked until resolved
+```
+
+---
+
+## 5. Write-Back Capability (Q63)
+
+Information Providers may optionally support write-back โ€” DCM updating external records when entity state changes. Write-back is triggered by policy, never automatic.
+
+```yaml
+information_provider_registration:
+  capabilities:
+    read: true                       # always required
+    write_back: true                 # optional
+    write_back_operations:
+      - operation: create
+        resource_types: [Compute.VirtualMachine]
+        fields: [hostname, ip_address, owner_business_unit, lifecycle_state]
+      - operation: update
+        resource_types: [Compute.VirtualMachine]
+      - operation: delete
+        resource_types: [Compute.VirtualMachine]
+    write_back_authentication:
+      mode: api_key
+      key_ref:
+        credential_provider_uuid: 
+        secret_path: "dcm/providers/info/cmdb/write-key"
+```
+
+**Policy triggers write-back:**
+
+```yaml
+policy:
+  type: transformation
+  placement_phase: post
+  rule: >
+    If action IN [CREATE, STATE_TRANSITION, DELETE]
+    AND resource_type == Compute.VirtualMachine
+    THEN trigger_write_back:
+      provider_uuid: 
+      operation: update
+      fields: [hostname, ip_address, lifecycle_state, owner_business_unit]
+```
+
+Write-back produces an audit record with `ENRICH` action and `source_type: information_provider_write_back`.
+
+---
+
+## 6. Extended Schema Versioning (Q64)
+
+Information Provider extended schemas follow semver semantics โ€” the same model as Resource Type Specifications.
+
+```yaml
+information_provider_registration:
+  extended_schema:
+    version: "2.1.0"
+    fields:
+      - name: cmdb_id
+        type: string
+        required: false
+      - name: cmdb_ci_class
+        type: string
+        required: false
+      - name: cmdb_location
+        type: object
+        required: false
+    changelog:
+      "2.0.0": "Removed deprecated cmdb_legacy_id field (major โ€” breaking)"
+      "2.1.0": "Added cmdb_location optional field (minor โ€” compatible)"
+    migration_plan:              # required for major version bumps
+      from_version: "1.x"
+      migration_script_ref: "git://cmdb-provider/migrations/v1-to-v2.yaml"
+      migration_window: P30D
+```
+
+**Semver semantics for extended schemas:**
+
+| Change | Version Bump | Reason |
+|--------|-------------|--------|
+| Field removed | **Major** | Breaking โ€” consumers may depend on it |
+| Field type changed | **Major** | Breaking โ€” consumers must update |
+| New optional field added | **Minor** | Compatible โ€” additive |
+| Description or constraint changed | **Revision** | Compatible โ€” no structural change |
+
+DCM validates incoming push data against the declared schema version. Major version bumps require a declared migration plan before the new schema version activates.
+
+---
+
+## 7. Well-Known Information Provider Registry (Q65)
+
+DCM maintains a three-tier Information Provider Registry following the same governance model as the Resource Type Registry.
+
+| Tier | Name | Contains | Examples |
+|------|------|---------|---------|
+| 1 | DCM Core | Universal integration patterns | Generic CMDB, Generic IPAM, Generic DNS |
+| 2 | Verified Community | Specific platform integrations | ServiceNow, Infoblox, NetBox, FreeIPA, AD, HashiCorp Vault |
+| 3 | Organization | Internal/proprietary | Acme ERP, Corp Asset Database |
+
+Well-known provider registrations include:
+- Pre-configured authority scope declarations
+- Pre-built extended schema definitions
+- Pre-configured write-back operation mappings
+- Connection templates with documented credential requirements
+- Example Policy Group activations for common use cases
+- Health check endpoint patterns
+
+The Information Provider Registry is **distinct from the Resource Type Registry** โ€” separate governance, separate GitOps repositories โ€” but shares the same infrastructure pattern: federated, PR-based proposals, automated validation, shadow validation period, and signed bundles for air-gapped import.
+
+---
+
+## 8. Air-Gapped Verification (Q66)
+
+Three modes for Information Provider verification in air-gapped environments:
+
+### 8.1 Mode 1 โ€” Pre-Verified Signed Bundle (Recommended)
+
+```yaml
+air_gapped_provider_bundle:
+  bundle_uuid: 
+  bundle_type: information_provider
+  signed_at: 
+  signing_key_ref: 
+  provider_registration: 
+  tls_certificate_chain: 
+  schema_definitions: 
+  verification_token: 
+  valid_until:         # bundles have expiry
+```
+
+### 8.2 Mode 2 โ€” Internal mTLS (Internal Providers)
+
+Providers that are themselves internal (internal CMDB, internal IPAM) register with `air_gap_mode: internal_only` and verify using internal mTLS certificates issued by the organization's internal CA (FreeIPA CA or equivalent).
+
+### 8.3 Mode 3 โ€” Periodic Online Re-Verification
+
+For environments air-gapped most of the time but with occasional connectivity windows:
+
+```yaml
+provider_verification:
+  mode: periodic_online
+  cache_ttl: P30D                # how long cached verification is valid
+  on_cache_expiry:
+    minimal: continue            # continue without re-verification
+    dev: alert                   # warn but continue
+    standard: alert              # warn but continue
+    prod: suspend                # suspend until re-verified
+    fsi: suspend
+    sovereign: suspend
+```
+
+---
+
+## 8a. Information Provider Trust Score Validation (Q15)
+
+### 8a.1 Dual-Trigger Model
+
+Information Provider trust scores are maintained per provider using the same dual-trigger pattern as conflict validation: event-triggered updates (primary) with scheduled re-verification (safety net).
+
+**Event-triggered updates:**
+- Provider push fails schema validation โ†’ `source_trust` degraded
+- Provider push conflicts with primary authority source โ†’ `source_trust` degraded
+- Health check fails โ†’ `source_trust` degraded
+- Sovereignty declaration change โ†’ trust re-evaluated against current Tenant requirements
+- Provider registration update (new credentials, endpoint change) โ†’ re-verification triggered
+
+**Scheduled re-verification:**
+- Daily: health check against all active Information Providers
+- Weekly: full re-verification (identity, sovereignty, certifications, schema compatibility)
+- `fsi` / `sovereign` profiles: daily full re-verification
+
+### 8a.2 Trust Score Structure
+
+```yaml
+information_provider_trust_score:
+  provider_uuid: 
+  score: 87                        # 0-100; contributes to source_trust field
+  scored_at: 
+  components:
+    identity_verified: true        # mTLS cert chain valid; re-checked weekly
+    endpoint_reachable: true       # health check; re-checked daily
+    schema_current: true           # schema version matches registered
+    sovereignty_compatible: true   # sovereignty matches Tenant requirements
+    certifications_current: true   # certifications not expired
+    push_error_rate:
+      rate: 0.02                   # 2% of pushes had schema/auth errors (7-day rolling)
+      weight: 0.15                 # contributes 15% to score degradation
+    conflict_rate:
+      rate: 0.05                   # 5% of pushes had value conflicts (7-day rolling)
+      weight: 0.10
+  decay_rate: per_7_days           # score decays if not re-verified
+  current_source_trust: verified   # verified | degraded | suspended
+  action_on_score_below:
+    threshold: 60
+    action: 
+    # suspend:       stop accepting pushes from this provider
+    # alert:         notify platform admin; continue with degraded trust
+    # reduce_weight: automatically degrade source_trust to degraded
+```
+
+### 8a.3 Trust Score to source_trust Mapping
+
+| Trust Score | source_trust | Effect on Confidence |
+|------------|-------------|---------------------|
+| โ‰ฅ 80 | `verified` | Full confidence multiplier (1.00) |
+| 60-79 | `degraded` | Reduced confidence multiplier (0.75) |
+| < 60 | `suspended` | No new data accepted; score = 0 |
+
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `INF-001` | Information Providers declare authority_level (primary, secondary, advisory) and authority_scope (resource types and fields). Conflicting authority scope declarations are detected at registration time. Conflicting field values from different providers at ingestion time are resolved per declared strategy. All conflicts produce audit records. |
+| `INF-002` | Information Providers may declare write_back capability with specific operations (create, update, delete) and resource types. Write-back is triggered by policy only โ€” never automatic. Write-back produces audit records with ENRICH action. Credentials reference a registered Credential Provider. |
+| `INF-003` | Information Provider extended schemas are versioned using semver. Removing a field or changing a field type is a major (breaking) version bump requiring a declared migration plan. Adding an optional field is a minor bump. DCM validates incoming push data against the declared schema version. |
+| `INF-004` | DCM maintains a three-tier Information Provider Registry (Core, Verified Community, Organization) following the same governance model as the Resource Type Registry. |
+| `INF-009` | Information Provider trust scores (0-100) are maintained per provider with event-triggered updates (push failure, schema mismatch, conflict, health check) and scheduled weekly re-verification. Trust score degradation transitions source_trust to degraded (reduced confidence multiplier). Suspension stops accepting pushes. Policy governs thresholds and actions per provider. |
+| `INF-005` | In air-gapped environments, Information Providers verify via pre-verified signed bundles, internal mTLS, or periodic online re-verification with cached tokens. Profile governs cache expiry behavior. |
+| `INF-006` | Information Provider field values carry a confidence score (0-100) computed from: source authority level, data freshness, and corroboration. DCM computes scores โ€” providers do not self-declare confidence. Scores decay with data age. |
+| `INF-007` | Authority scope and priority for Information Providers are declared in platform or system domain layers. Policy acts on confidence scores and bands โ€” gating, filtering, and escalating based on threshold declarations. |
+| `INF-008` | Conflict detection occurs at ingestion time. Policy governs automated resolution strategy. All conflicts โ€” detected, auto-resolved, and escalated โ€” produce INGEST audit records with the full conflict detail. |
+
+---
+
+## 10. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | Should confidence score decay be linear or exponential? | Scoring model | โœ… Resolved โ€” freshness is discrete threshold bands not decay curve; stale multiplier 0.50 provides the cliff effect |
+| 2 | Should DCM expose confidence score aggregation APIs โ€” e.g., "average confidence across all fields for this entity"? | Consumer experience | โœ… Resolved โ€” per-entity confidence aggregation endpoint; overall band = lowest field band; computed on demand; identifies contested and stale fields (INF-010) |
+| 3 | Should conflicting providers receive notification when their value is overridden? | Provider relationship | โœ… Resolved โ€” provider opt-in override notifications; webhook or Message Bus; payload policy-governed; overriding value may be redacted (INF-011) |
+
+
+## 11. Information Provider Advanced Gap Resolutions
+
+### 11.1 Confidence Score Aggregation API (Q2)
+
+DCM exposes a per-entity confidence aggregation endpoint. The overall band reflects the lowest (most conservative) field band โ€” preventing high-scoring fields from masking problematic ones. Aggregations are computed on demand โ€” never stored (freshness changes continuously).
+
+```yaml
+confidence_aggregation_api:
+  endpoint: GET /api/v1/entities/{uuid}/confidence
+  response:
+    entity_uuid: 
+    overall_band: high              # lowest band across all fields (conservative)
+    field_summaries:
+      - field: owner_business_unit
+        band: high
+        score: 86
+        authority_level: primary
+        last_updated_at: 
+      - field: cost_center
+        band: medium
+        score: 54
+        corroboration: contested    # two providers disagree
+    lowest_confidence_fields:
+      - field: cost_center
+        reason: contested
+      - field: asset_tag
+        reason: stale               # data age > 7 days
+    computed_at: 
+```
+
+### 11.2 Conflicting Provider Override Notifications (Q3)
+
+Information Providers may opt in to override notifications. Not a universal default โ€” read-only public registries have no use for notifications; internal CMDBs may want to investigate discrepancies.
+
+```yaml
+information_provider_registration:
+  conflict_notification:
+    enabled: true                   # provider opts in
+    notification_channel: webhook   # or: message_bus
+    notification_endpoint: https://cmdb.corp.example.com/dcm-notifications
+    notify_on: [value_overridden, value_contested, authority_superseded]
+    notification_payload:
+      field_path: true
+      overriding_provider_uuid: true
+      overriding_value: false       # may be redacted for confidentiality
+      conflict_record_uuid: true
+```
+
+**Privacy:** The overriding value may be confidential (from a classified source). Policy governs what is included in the notification payload. GateKeeper can redact the overriding value if classified.
+
+### 11.3 System Policies โ€” Information Provider Advanced Gaps
+
+| Policy | Rule |
+|--------|------|
+| `INF-010` | DCM exposes a confidence aggregation endpoint per entity (GET /api/v1/entities/{uuid}/confidence). The overall confidence band reflects the lowest (most conservative) field band. Aggregations are computed on demand โ€” never stored. The response identifies contested and stale fields requiring attention. |
+| `INF-011` | Information Providers may opt in to override notifications by declaring conflict_notification in their registration. Notifications sent via webhook or Message Bus. The notification payload is policy-governed โ€” the overriding value may be redacted for confidentiality reasons. |
+
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/22-dcm-federation.md b/content/docs/data-model/22-dcm-federation.md
new file mode 100644
index 0000000..89c0a10
--- /dev/null
+++ b/content/docs/data-model/22-dcm-federation.md
@@ -0,0 +1,597 @@
+# DCM Data Model โ€” DCM Federation, Peering, and Cross-Instance Coordination
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete  
+**Related Documents:** [Federated Contribution Model](28-federated-contribution-model.md) | [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md)
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: PROVIDER + POLICY**
+>
+> Provider: Peer DCM as typed Provider. Policy: federation governance rules
+
+
+
+---
+
+> **Federated Contribution:** Federation contribution follows the [Federated Contribution Model](28-federated-contribution-model.md) โ€” peer DCMs are contributors to each other's artifact stores, scoped by their federation trust posture.
+
+## 1. Purpose
+
+DCM instances do not operate in isolation. Organizations with multiple data centers, regions, or organizational boundaries may run multiple DCM instances that need to coordinate, share resources, and maintain consistent governance. This document defines how DCM instances relate to each other โ€” as peers, in parent-child hierarchies, or as hub-and-spoke configurations โ€” and the mechanisms for cross-instance resource sharing, data export/import, and provider federation eligibility.
+
+---
+
+## 2. DCM-to-DCM Relationship Types
+
+### 2.1 Three Relationship Types
+
+**Peer DCM** โ€” two DCM instances at the same organizational level that share resources or information. Regional DCM instances sharing VLAN allocations. Campus DCM instances sharing compute capacity across departments.
+
+**Parent-Child DCM** โ€” hierarchical relationship where the parent has governance overlay authority over child instances. Corporate DCM with regional children. Service provider DCM with customer children. Parent does not own child resources โ€” it has governance visibility and policy overlay authority (same model as nested Tenants in the Universal Group Model).
+
+**Hub DCM** โ€” specialized Parent DCM acting as a clearinghouse for resource allocation across multiple children. The Hub holds the master resource inventory; children request allocations from the Hub.
+
+### 2.2 Relationship Mapping to Universal Group Model
+
+DCM-to-DCM relationships use the existing Universal Group Model:
+
+```yaml
+# Peer relationship โ€” federation group
+dcm_group:
+  group_class: federation
+  name: "EU-US Regional Federation"
+  members:
+    - member_uuid: 
+      member_type: dcm_peer
+      member_role: eu_region_dcm
+    - member_uuid: 
+      member_type: dcm_peer
+      member_role: us_region_dcm
+  federation_config:
+    shared_policy_inheritance: opt_in
+    cross_member_visibility: audit_only
+
+# Parent-Child relationship โ€” nesting
+dcm_group:
+  group_class: tenant_boundary
+  name: "Corporate DCM"
+  child_groups:
+    - 
+    - 
+  policy_inheritance: opt_out      # parent policies cascade unless child excludes
+```
+
+---
+
+## 3. Provider Federation Eligibility
+
+### 3.1 Concept
+
+Every provider registration carries a `federation_eligibility` declaration โ€” whether the provider can participate in cross-DCM federation, with whom, and under what conditions. This is **layer-defined** (static organizational knowledge) and **policy-enforced** (runtime governance).
+
+### 3.2 Federation Eligibility on Provider Registration
+
+```yaml
+provider_registration:
+  handle: "providers/service/eu-compute-primary"
+
+  federation_eligibility:
+    mode: 
+    # none:      Provider cannot participate in any DCM federation
+    #            (sovereign, classified, or compliance-restricted providers)
+    # selective: Federation permitted only with explicitly declared partners
+    # open:      Federation permitted with any trusted DCM peer
+    #            (sovereignty checks always apply regardless)
+
+    permitted_partners:
+      - partner_type: 
+        dcm_instance_uuids: []          # specific instances
+        dcm_instance_tags: [region-eu, internal]  # tag-based matching
+        dcm_certification_required: [ISO-27001, GDPR-compliant]
+        relationship_requires_approval: true   # bilateral approval required
+
+    federation_scope:
+      permitted_resource_types:
+        - resource_type: Compute.VirtualMachine
+          operations: [allocate, query_capacity]
+          # NOT: decommission โ€” remote DCMs cannot decommission local resources
+        - resource_type: Network.VLAN
+          operations: [allocate, query_capacity, release]
+      data_sharing:
+        capacity_data: true
+        realized_state: false           # do not share realized state details
+        pricing_data: true
+        sovereignty_declaration: true   # always share โ€” required for federation
+      max_allocations_per_partner: 100
+      max_concurrent_allocations: 500
+
+    override_reason: null               # populated when overriding layer default
+```
+
+### 3.3 Layer-Defined Federation Defaults
+
+Federation eligibility defaults live in a `platform` domain layer โ€” static organizational knowledge inherited by all providers unless overridden. Individual provider registrations may be **more restrictive** than the layer default (always permitted); **less restrictive** requires GateKeeper policy approval.
+
+```yaml
+layer:
+  handle: "platform/federation/provider-federation-defaults"
+  domain: platform
+  priority: 600.0.0
+  fields:
+    provider_federation_defaults:
+      compute_providers:
+        federation_eligibility:
+          mode: selective
+          permitted_partners:
+            - partner_type: dcm_peer
+              dcm_instance_tags: [internal, eu-region]
+              dcm_certification_required: [ISO-27001]
+          federation_scope:
+            permitted_resource_types:
+              - resource_type: Compute.VirtualMachine
+                operations: [allocate, query_capacity]
+      storage_providers:
+        federation_eligibility:
+          mode: none                    # storage never federated โ€” data sovereignty
+      network_providers:
+        federation_eligibility:
+          mode: selective
+          permitted_partners:
+            - partner_type: dcm_peer
+              dcm_instance_tags: [internal]
+      information_providers:
+        federation_eligibility:
+          mode: selective
+          data_sharing_restrictions:
+            max_classification: internal   # never share confidential/restricted
+```
+
+### 3.4 Policy Enforcement on Federation
+
+Policies act on federation eligibility at three enforcement points:
+
+**At tunnel establishment:**
+```yaml
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_establishment
+  rule: >
+    If provider.federation_eligibility.mode == none
+    THEN gatekeep: "Provider is not eligible for federation"
+
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_establishment
+  rule: >
+    If remote_dcm.sovereignty_zone NOT IN permitted_sovereignty_zones
+    THEN gatekeep: "Remote DCM sovereignty zone incompatible with local requirements"
+
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_establishment
+  rule: >
+    If remote_dcm.certifications NOT CONTAINS
+       provider.federation_eligibility.permitted_partners.dcm_certification_required
+    THEN gatekeep: "Remote DCM does not hold required certifications"
+```
+
+**At allocation time:**
+```yaml
+policy:
+  type: gatekeeper
+  target: cross_dcm_allocation
+  rule: >
+    If resource_type NOT IN provider.federation_eligibility.federation_scope.permitted_resource_types
+    THEN gatekeep: "Resource type not permitted through this federation tunnel"
+
+policy:
+  type: gatekeeper
+  target: cross_dcm_allocation
+  rule: >
+    If cross_dcm_allocations_active > provider.federation_eligibility.max_concurrent_allocations
+    THEN gatekeep: "Maximum concurrent federation allocations exceeded"
+```
+
+**At data egress:**
+```yaml
+policy:
+  type: gatekeeper
+  target: dcm_tunnel_data_egress
+  rule: >
+    If data.classification > remote_dcm.max_data_classification_receivable
+    THEN gatekeep: "Data classification exceeds remote DCM authorization"
+```
+
+---
+
+## 4. The DCM Provider โ€” Cross-Instance Tunneling
+
+### 4.1 Concept
+
+A **DCM Provider** is a ninth provider type that wraps another DCM instance's API, enabling one DCM to consume resources managed by another DCM as if they were local providers.
+
+| # | Type | Purpose |
+|---|------|---------|
+| 1 | Service Provider | Realizes resources |
+| 2 | Information Provider | Serves authoritative external data |
+| 3 | Meta Provider | Composes multiple providers |
+| 4 | Storage Provider | Persists DCM state |
+| 5 | Policy Provider | Supplies and evaluates policies |
+| 6 | Message Bus Provider | Bridges internal/external event streams |
+| 7 | Credential Provider | Resolves secrets |
+| 8 | Auth Provider | Authenticates identities |
+| 9 | **DCM Provider** | Wraps another DCM instance's API |
+
+### 4.2 DCM Provider Registration
+
+```yaml
+dcm_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "providers/dcm/region-eu-dcm"
+    status: active
+
+  provider_type: dcm_provider
+  relationship_type: 
+
+  remote_dcm:
+    instance_uuid: 
+    endpoint: https://dcm.region-eu.corp.example.com
+    dcm_version: "2.1.0"          # minimum compatible version
+    sovereignty_declaration_ref:   # verified at registration time
+
+  # Authentication โ€” always mTLS for DCM-to-DCM
+  auth:
+    mode: mtls
+    client_cert_ref:
+      credential_provider_uuid: 
+      path: "dcm/dcm-providers/region-eu/client-cert"
+    server_ca_ref:
+      credential_provider_uuid: 
+      path: "dcm/dcm-providers/region-eu/server-ca"
+
+  # Tunnel configuration
+  tunnel_config:
+    encrypted: true                # always โ€” not configurable
+    sovereignty_boundary_check: true  # always โ€” not configurable
+    permitted_resource_types: [Compute.VirtualMachine, Network.VLAN]
+    max_allocation_per_request: 10
+    audit_forwarding: true         # forward audit records to local Audit Store
+    observability_forwarding: true
+
+  # Sovereignty โ€” must be compatible with local requirements
+  sovereignty_declaration:
+    remote_jurisdiction: eu-west
+    data_residency_guarantee: true
+    certifications: [ISO-27001, GDPR-compliant]
+
+  # Health check
+  health_check:
+    endpoint: /api/v1/health
+    interval_seconds: 60
+    on_unhealthy: suspend_allocations
+```
+
+### 4.3 Primary Concerns on All DCM Tunnels
+
+These are non-negotiable on every DCM-to-DCM connection:
+
+| Concern | Enforcement |
+|---------|-----------|
+| **Sovereignty** | Sovereignty_declaration verified before tunnel establishment; data classification checked before every egress |
+| **Authentication** | Always mTLS โ€” no API key, no bearer token โ€” mutual certificate authentication between DCM instances |
+| **Authorization** | Local DCM policies govern ALL resources obtained through a tunnel; remote DCM's policies do not override local |
+| **Audit** | All cross-DCM operations produce audit records in BOTH DCM instances; shared `correlation_id` links the two trails |
+| **Observability** | Cross-DCM resource allocation visible in both instances' observability stores |
+| **Governance** | Local GateKeeper policies apply to all resources from any tunnel source |
+
+---
+
+## 5. Cross-DCM Confidence Scoring
+
+Resources obtained through a DCM tunnel carry compound confidence scores โ€” the resource's confidence in its source DCM, degraded by the tunnel trust score:
+
+```
+cross_dcm_confidence = source_resource_confidence ร— (tunnel_trust_score / 100)
+```
+
+A resource with confidence 90 in source DCM, through a tunnel with trust score 85: `90 ร— 85/100 = 76.5` โ†’ **77**
+
+### 5.1 Federation Trust Score
+
+```yaml
+dcm_federation_trust_score:
+  remote_dcm_uuid: 
+  score: 84                        # 0-100
+  scored_at: 
+  decay_rate: per_30_days
+  factors:
+    identity_verified: true        # mTLS certificate chain verified
+    sovereignty_compatible: true
+    certifications_current: true
+    audit_trail_integrity: true    # audit hash chains verified on sample
+    uptime_score: 0.98
+    compliance_score: 0.90         # policy compliance in recent operations
+    data_completeness: 0.92
+  action_on_score_below:
+    threshold: 60
+    action: 
+```
+
+---
+
+## 6. DCM Export and Import
+
+### 6.1 Export Package
+
+DCM state is fully exportable as a signed package โ€” for disaster recovery, migration, cross-DCM sharing, and Hub DCM onboarding.
+
+```yaml
+dcm_export_package:
+  package_uuid: 
+  exported_at: 
+  exported_by: 
+  dcm_version: 
+  signed_by: 
+
+  scope:
+    tenants: [, ...]
+    resource_types: all
+    layers: [platform, tenant]
+    policies: [platform, tenant]
+    providers: registrations_only   # not credentials โ€” never export credentials
+    entities: [intent_state, requested_state]   # not realized (that's provider state)
+    groups: all
+    audit_records:
+      date_range: [, ]
+      include_hash_chain: true      # for audit trail verification on import
+
+  sovereignty:
+    classification: internal
+    permitted_import_jurisdictions: [eu-west]
+    signed: true
+    encryption: aes256_gcm
+```
+
+### 6.2 Import Trust Score
+
+When importing from another DCM instance, each imported resource carries a trust score:
+
+```yaml
+import_trust_score:
+  score: 78                        # 0-100
+  factors:
+    source_dcm_verified: true      # source DCM identity verified
+    sovereignty_compatible: true
+    data_completeness: 0.92
+    schema_compatibility: 1.00     # source schema matches current version
+    audit_trail_complete: true     # audit records included and hash chain valid
+    certifications_current: true
+  action_on_low_score: 
+  threshold: 70                    # reject if below
+```
+
+### 6.3 Scoring in Resource Definition and Allocation
+
+Resources imported from or allocated through peer DCMs carry their compound confidence score throughout their lifecycle in the importing DCM. The score is visible to the placement engine, Cost Analysis, and the Policy Engine โ€” enabling policies that prefer locally-managed resources over federated resources when scores are comparable.
+
+---
+
+## 7. Profile-Appropriate Federation Policy Groups
+
+DCM ships built-in federation policy groups activated by default per profile:
+
+| Group | Profile | Behavior |
+|-------|---------|---------|
+| `system/group/federation-minimal` | minimal | No federation โ€” single instance only |
+| `system/group/federation-dev` | dev | Peer federation permitted; advisory only |
+| `system/group/federation-standard` | standard | Peer federation with certification requirements |
+| `system/group/federation-prod` | prod | Selective federation; bilateral approval; audit forwarding |
+| `system/group/federation-fsi` | fsi | Strict federation; within-jurisdiction only; full audit; no storage federation |
+| `system/group/federation-sovereign` | sovereign | No external federation; internal peer federation within sovereignty boundary only |
+
+---
+
+## 8. DCM System Policies โ€” Federation
+
+| Policy | Rule |
+|--------|------|
+| `DCM-001` | DCM instances may establish peer, parent-child, or hub relationships using the Universal Group Model federation and nesting constructs. |
+| `DCM-002` | All DCM-to-DCM communication uses mTLS. No API key or bearer token. Sovereignty checks are mandatory before tunnel establishment. These requirements are non-configurable. |
+| `DCM-003` | Local DCM policies govern all resources obtained through DCM tunnels. Cross-DCM operations produce audit records in both DCM instances with a shared correlation_id. |
+| `DCM-004` | DCM state is exportable as a signed package. Imported packages carry a trust score (0-100) computed from source verification, sovereignty compatibility, data completeness, schema compatibility, and audit trail integrity. |
+| `DCM-005` | Resources obtained through DCM tunnels carry compound confidence scores: source_resource_confidence ร— (tunnel_trust_score / 100). |
+| `DCM-006` | Every provider registration must declare federation_eligibility (mode: none, selective, or open). Federation eligibility defaults are declared in platform domain layers. Individual provider registrations may be more restrictive โ€” never more permissive without GateKeeper policy approval. |
+| `DCM-007` | Provider federation scope declares: permitted resource types, permitted operations per type, data sharing permissions, and allocation limits. Remote DCMs cannot decommission local resources through a federation tunnel. |
+| `DCM-008` | Storage providers default to federation_eligibility.mode: none. Data sovereignty constraints prohibit storage federation unless explicitly authorized by sovereign policy with full justification. |
+
+---
+
+## 9. Open Questions
+
+| # | Question | Impact | Status |
+|---|----------|--------|--------|
+| 1 | How are DCM-to-DCM certificate rotation and renewal handled โ€” coordinated or independent? | Operations | โœ… Resolved โ€” independent rotation with P30D overlap; peer notification 60 days before expiry via Message Bus; auto-renewal at 90 days (DCM-009) |
+| 2 | Should Hub DCM relationships support automatic load balancing across child DCMs? | Architecture | โœ… Resolved โ€” full placement engine logic at DCM instance level; sovereignty as hard pre-filter; tie-breaking hierarchy same as provider selection; sub-regional routing recursive (DCM-010) |
+| 3 | How does drift detection work for resources allocated from a peer DCM โ€” who is responsible for discovery? | Operational | โœ… Resolved โ€” provider-side DCM discovers; consumer-side DCM compares; events via federation Message Bus; peer unavailable = alert-and-hold (DCM-011) |
+| 4 | Should cross-DCM audit records be synchronized โ€” so each DCM has the other's audit records? | Compliance | โœ… Resolved โ€” correlation_id reference model; no full sync; on-demand pull with platform admin auth + sovereignty check (DCM-012) |
+| 5 | What is the maximum supported federation depth (peer of peer of peer)? | Architecture | โœ… Resolved โ€” profile-governed max depth: minimal/dev=5, standard/prod=3, fsi/sovereign=2; measured as hops from deepest to Hub (DCM-013) |
+
+
+## 11. Federation Gap Resolutions
+
+### 11.1 Certificate Rotation and Renewal (Q1)
+
+DCM-to-DCM mTLS certificates rotate independently per instance with a coordinated notification model. Coordinated simultaneous rotation would create a single point of failure.
+
+```yaml
+dcm_federation_cert_rotation:
+  rotation_model: independent_with_overlap
+  overlap_period: P30D              # old cert valid for 30 days after new cert issued
+  notification:
+    notify_peers_at: P60D_before_expiry
+    notification_channel: message_bus
+    notification_payload:
+      new_cert_public_key: 
+      new_cert_valid_from: 
+      old_cert_valid_until: 
+  automatic_renewal:
+    trigger_at: P90D_before_expiry
+    requires_approval: false          # renewal is automatic; elevation requires approval
+```
+
+The P30D overlap allows peers to update their trust stores at their own pace without service interruption.
+
+### 11.2 Federation Routing โ€” Placement Engine at the DCM Level (Q2)
+
+**Hub DCM federation routing follows the same placement engine logic as provider selection.** Regional DCMs are treated as DCM Provider instances in the placement engine.
+
+**Sovereignty is a hard pre-filter โ€” not a preference:**
+
+```
+Before placement loop:
+  Filter eligible Regional DCMs where:
+    sovereignty_declaration satisfies request constraints
+    operating_jurisdictions includes required jurisdictions
+    sovereignty_zone matches tenant.sovereignty_zone
+  
+  If no eligible Regional DCMs โ†’ Reject with clear error
+  Only eligible DCMs enter the placement loop
+```
+
+**The full federation routing flow:**
+
+```
+Request arrives at Hub DCM
+  โ”‚
+  โ–ผ Steps 1-5: Standard nine-step assembly (layers, policies, placement constraints)
+  โ”‚  Pre-placement policies may declare federation routing constraints:
+  โ”‚    "This resource must be in a Regional DCM with EU sovereignty"
+  โ”‚    "This Tenant's resources must stay in Regional DCM-EU-West"
+  โ”‚
+  โ–ผ Step 6: Placement loop โ€” at the DCM instance level
+  โ”‚  Reserve query to eligible Regional DCMs:
+  โ”‚    capacity available? sovereignty compatible? trust score adequate?
+  โ”‚
+  โ–ผ Tie-breaking (same hierarchy as provider selection):
+  โ”‚  1. Policy preference (policy declares preferred Regional DCM)
+  โ”‚  2. Federation priority (numeric priority on DCM Provider registration)
+  โ”‚  3. Tenant affinity (Tenant's resources prefer a specific Regional DCM)
+  โ”‚  4. Sovereignty match quality (exact match over partial match)
+  โ”‚  5. Geographic affinity (closest regional to consumer)
+  โ”‚  6. Least loaded (capacity utilization across instances)
+  โ”‚  7. Consistent hash (deterministic tiebreaker)
+  โ”‚
+  โ–ผ Selected Regional DCM receives assembled request payload
+  โ”‚  Runs its own local assembly and placement (regional layers, regional providers)
+  โ”‚  Returns realization result to Hub DCM โ†’ forwarded to consumer
+  โ”‚
+  โ–ผ Sub-regional routing: Regional DCM acts as Hub for its children
+    Same logic applies recursively within federation depth limit (DCM-013)
+```
+
+**Load balancing is the least-loaded step in the hierarchy** โ€” not a primary strategy. Sovereignty, policy, and tenant affinity all take precedence. Optional `hub_dcm_load_balancing` configuration:
+
+```yaml
+hub_dcm_load_balancing:
+  enabled: true                     # default: true
+  sovereignty_override: true        # always โ€” sovereignty is a hard pre-filter
+  fallback_on_regional_unavailable: route_to_next_eligible
+```
+
+### 11.3 Federated Drift Detection Ownership (Q3)
+
+Provider-side DCM is responsible for discovery; consumer-side DCM is responsible for drift comparison.
+
+```yaml
+federated_drift_detection:
+  discovery_responsibility: provider_side_dcm
+  comparison_responsibility: consumer_side_dcm
+  mechanism:
+    provider_dcm:
+      - Run standard discovery against its providers
+      - Publish Discovered State events to federation Message Bus
+      - Tagged with: entity_uuid + consumer_dcm_uuid + correlation_id
+    consumer_dcm:
+      - Subscribe to Discovered State events for its federated entities
+      - Compare against its Requested State
+      - Trigger drift response policy if drift detected
+  on_peer_dcm_unavailable:
+    action: alert_and_hold            # not assumed drift
+    max_hold_period: PT24H
+    on_hold_exceeded: escalate_to_platform_admin
+```
+
+### 11.4 Cross-DCM Audit Record Correlation (Q4)
+
+No full synchronization. Each DCM keeps its own authoritative audit trail. Cross-DCM correlation uses correlation_id references and on-demand pull.
+
+```yaml
+cross_dcm_audit_correlation:
+  model: correlation_id_reference
+  local_audit_record:
+    action: ALLOCATE_FROM_PEER
+    correlation_id: 
+    peer_dcm_uuid: 
+    peer_audit_record_uuid:     # reference โ€” not a copy
+  on_demand_pull:
+    endpoint: GET /api/v1/audit/cross-dcm/{correlation_id}
+    requires: platform_admin + peer_dcm_authorization + sovereignty_check
+```
+
+Full synchronization is not required โ€” auditors follow correlation_id to the peer DCM on demand.
+
+### 11.5 Maximum Federation Depth (Q5)
+
+```yaml
+federation_depth_policy:
+  max_depth: 3                        # profile-governed
+  on_max_exceeded: reject_federation_establishment
+  profile_defaults:
+    minimal: 5
+    dev: 5
+    standard: 3
+    prod: 3
+    fsi: 2
+    sovereign: 2
+```
+
+Depth is measured as hops from the deepest instance to the Hub DCM. Depth 3 covers Hub โ†’ Regional โ†’ Sub-Regional โ†’ Edge โ€” sufficient for most real-world architectures.
+
+### 11.6 System Policies โ€” Federation Gaps
+
+| Policy | Rule |
+|--------|------|
+| `DCM-009` | DCM-to-DCM mTLS certificates rotate independently per instance with a P30D overlap period. Peers are notified 60 days before expiry via Message Bus. Automatic renewal triggers 90 days before expiry. The overlap period allows peers to update trust stores without coordinated downtime. |
+| `DCM-010` | Hub DCM federation routing follows the same placement engine logic as provider selection. Sovereignty is a hard pre-filter โ€” only Regional DCMs satisfying all sovereignty constraints enter the placement loop. The tie-breaking hierarchy applies at the DCM instance level: policy preference โ†’ federation priority โ†’ tenant affinity โ†’ sovereignty match quality โ†’ geographic affinity โ†’ least loaded โ†’ consistent hash. Regional DCMs are treated as DCM Provider instances. Sub-regional routing applies the same logic recursively within the federation depth limit. |
+| `DCM-011` | For resources allocated from peer DCMs, the provider-side DCM is responsible for discovery. The consumer-side DCM is responsible for drift comparison. Discovered State events are published via federation Message Bus with correlation_id. Peer DCM unavailability triggers alert-and-hold โ€” not assumed drift. |
+| `DCM-012` | Cross-DCM audit records are referenced via correlation_id โ€” not fully synchronized. Each DCM keeps its own authoritative audit trail. Cross-DCM correlation uses on-demand pull with platform admin authorization and sovereignty check. |
+| `DCM-013` | Federation depth is limited to a profile-governed maximum (default: 3 for standard/prod; 2 for fsi/sovereign; 5 for minimal/dev). Requests to establish federation beyond the maximum depth are rejected. Depth is measured as hops from the deepest instance to the Hub DCM. |
+
+
+---
+
+## 10. Related Concepts
+
+- **Universal Group Model** (doc 15) โ€” federation and nesting group classes
+- **Storage Providers** (doc 11) โ€” storage never federated by default
+- **Auth Providers** (doc 19) โ€” mTLS for DCM-to-DCM authentication
+- **Universal Audit Model** (doc 16) โ€” audit records in both DCM instances; correlation_id
+- **Registry Governance** (doc 20) โ€” signed bundles for air-gapped registry updates
+- **Information Providers Advanced** (doc 21) โ€” confidence scoring used in cross-DCM context
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/23-notification-model.md b/content/docs/data-model/23-notification-model.md
new file mode 100644
index 0000000..c15a907
--- /dev/null
+++ b/content/docs/data-model/23-notification-model.md
@@ -0,0 +1,606 @@
+# DCM Data Model โ€” Notification Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: PROVIDER + POLICY**
+>
+> Provider: Notification Provider. Policy: audience resolution and subscription rules
+
+
+**Related Documents:** [Webhooks, Messaging, and External Integration](18-webhooks-messaging.md) | [Entity Relationships](09-entity-relationships.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Auth Providers](19-auth-providers.md) | [Universal Audit](16-universal-audit.md)
+
+---
+
+
+> **See [Event Catalog](33-event-catalog.md)** โ€” authoritative source for all DCM event types and payload schemas.
+
+## 1. Purpose
+
+The DCM Notification Model defines a **unified, configurable notification pipeline** that routes event notifications to all parties with a stake in a changed resource โ€” not just the original requestor. The audience for any notification is derived from the **entity relationship graph**, not from who submitted the original request.
+
+This document defines:
+- The Notification Provider โ€” the ninth DCM provider type
+- The event taxonomy โ€” a closed vocabulary of notification-worthy events
+- The audience resolution model โ€” how the relationship graph determines who gets notified
+- The subscription model โ€” how actors declare their notification preferences
+- The notification payload structure โ€” the unified envelope all Notification Providers receive
+- The delivery pipeline โ€” from event trigger through audience resolution through provider delivery
+
+This model supersedes the standalone outbound webhook model in doc 18. Outbound webhooks are now one delivery channel of the Notification Provider, not a parallel mechanism.
+
+---
+
+## 2. Design Principles
+
+**Relationship graph determines audience.** When a resource changes, DCM traverses the entity relationship graph to find all stakeholders. A VLAN decommission notifies every VM attached to that VLAN, regardless of which Tenant owns each VM. The graph is the source of truth for notification scope.
+
+**Delivery mechanism is configurable, not prescribed.** DCM generates and routes notifications. How they are delivered โ€” email, Slack, PagerDuty, ServiceNow, webhook, SMS โ€” is the concern of a Notification Provider. Organizations register the Notification Provider(s) that fit their operations.
+
+**Three notification tiers.** Some notifications are mandatory and non-suppressable (security, sovereignty violations, audit chain breaks). Some are Tenant-default (all resource lifecycle events in a Tenant). Some are actor-subscription (specific events on specific resources). All three compose without conflict.
+
+**Audience role shapes the notification.** The same event produces different notifications for an owner ("your resource changed") versus a stakeholder ("a resource you depend on changed") versus an approver ("your approval is required"). The audience role is part of the notification envelope.
+
+**Delivery is audited.** Every notification dispatch is an audit record. Delivery failures are tracked and escalated per policy.
+
+---
+
+## 3. The Notification Provider
+
+The Notification Provider is the ninth formal DCM provider type. It handles the translation from DCM's unified notification envelope to the delivery channel's native format, and it handles delivery, retry, and delivery confirmation.
+
+### 3.1 Provider Types Table Update
+
+| # | Type | Purpose |
+|---|------|---------|
+| 1 | Service Provider | Realizes resources |
+| 2 | Information Provider | Serves authoritative external data |
+| 3 | Meta Provider | Composes multiple providers |
+| 4 | Storage Provider | Persists DCM state |
+| 5 | Message Bus Provider | Event streaming and messaging |
+| 6 | Policy Provider | External policy logic |
+| 7 | Credential Provider | Resolves secrets |
+| 8 | Auth Provider | Authenticates identities |
+| **9** | **Notification Provider** | **Delivers notifications via configured channels** |
+
+### 3.2 Notification Provider Registration
+
+```yaml
+notification_provider_registration:
+  artifact_metadata:
+    uuid: 
+    handle: "org/notifications/slack-provider"
+    version: "1.0.0"
+    status: active
+    owned_by: { display_name: "Platform Engineering" }
+
+  provider_type: notification
+  display_name: "Slack Notification Provider"
+  description: "Delivers DCM notifications to configured Slack channels"
+
+  # Delivery channels this provider supports
+  delivery_channels:
+    - channel_type: slack
+      config_schema_ref:           # JSON Schema for channel config
+      supports_threading: true
+      supports_urgency_routing: true     # different channels per urgency level
+    - channel_type: webhook
+      config_schema_ref: 
+
+  # Sovereignty declaration โ€” same model as all providers
+  sovereignty_declaration:
+    data_residency_guarantee: EU
+    operating_jurisdictions: [DE, FR, NL]
+
+  # Delivery guarantees this provider offers
+  delivery_guarantees:
+    at_least_once: true
+    idempotency_key: notification_uuid
+    max_delivery_latency: PT30S        # for critical urgency
+    retry_policy:
+      max_attempts: 7
+      backoff: exponential
+      initial_interval: PT5S
+      max_interval: PT1H
+      on_exhaustion: dead_letter
+
+  # Health check endpoint (on the provider)
+  health_endpoint: https://notif-provider.corp.example.com/health
+
+  # Callback endpoint (DCM calls this to submit notifications)
+  delivery_endpoint: https://notif-provider.corp.example.com/deliver
+```
+
+### 3.3 Multiple Notification Providers
+
+Organizations may register multiple Notification Providers โ€” one for Slack, one for PagerDuty, one for ServiceNow tickets. Notification subscriptions declare which provider to use for delivery. The Notification Router in DCM routes each notification to the correct provider based on the subscription's `notification_provider_uuid`.
+
+---
+
+## 4. The Event Taxonomy
+
+The notification event taxonomy is a **closed vocabulary** โ€” a finite, versioned set of event types that DCM can generate notifications for. Events are grouped by category. Subscriptions reference these event types by name.
+
+### 4.1 Request Lifecycle Events
+
+| Event Type | Trigger | Default Audience |
+|-----------|---------|-----------------|
+| `request.acknowledged` | Request received, Intent State created | Owner |
+| `request.requires_approval` | Policy requires human review before dispatch | Owner, Approvers |
+| `request.approved` | Intent State PR merged; proceeding to assembly | Owner |
+| `request.dispatched` | Requested State committed; dispatched to provider | Owner |
+| `request.completed` | Provider confirmed realization; Realized State written | Owner |
+| `request.failed` | Request failed at any stage | Owner |
+| `request.cancelled` | Consumer cancelled; request terminated | Owner |
+| `request.gatekeeper_rejected` | GateKeeper policy rejected the request | Owner, Policy Owner |
+
+### 4.2 Resource Lifecycle Events
+
+| Event Type | Trigger | Default Audience |
+|-----------|---------|-----------------|
+| `entity.realized` | Entity first realized by provider | Owner, Stakeholders (depth 1) |
+| `entity.state_changed` | Entity lifecycle state transition | Owner, Stakeholders (required) |
+| `entity.ttl_warning` | TTL expires within declared warning window | Owner |
+| `entity.ttl_expired` | TTL reached; expiry action triggered | Owner, Stakeholders (required) |
+| `entity.suspended` | Entity entered SUSPENDED state | Owner, Stakeholders (required) |
+| `entity.resumed` | Entity exited SUSPENDED state | Owner, Stakeholders (required) |
+| `entity.decommissioning` | Decommission initiated | Owner, Stakeholders (all) |
+| `entity.decommissioned` | Entity fully decommissioned | Owner, Stakeholders (all) |
+| `entity.decommission_deferred` | Decommission blocked by active stakes | Owner, Stakeholders (required) |
+| `entity.ownership_transferred` | Ownership moved to a different Tenant | Previous Owner, New Owner |
+| `entity.pending_review` | Entity entered PENDING_REVIEW state | Owner, Platform Admin |
+
+### 4.3 Drift and Discovery Events
+
+| Event Type | Trigger | Default Audience |
+|-----------|---------|-----------------|
+| `drift.detected` | Discovered State differs from Realized State | Owner |
+| `drift.severity_escalated` | Drift severity increased | Owner, Platform Admin |
+| `drift.resolved` | Drift resolved (REVERT or UPDATE_DEFINITION) | Owner |
+| `drift.escalated` | Drift escalated to human review | Owner, Platform Admin, SRE |
+| `unsanctioned_change.detected` | Change detected with no corresponding Requested State record | Owner, Security Team, Platform Admin |
+
+### 4.4 Provider Update Events
+
+| Event Type | Trigger | Default Audience |
+|-----------|---------|-----------------|
+| `provider_update.submitted` | Provider submitted an update notification | Owner |
+| `provider_update.requires_approval` | Provider update requires consumer approval | Owner (approval required) |
+| `provider_update.approved` | Provider update approved; Realized State updated | Owner |
+| `provider_update.rejected` | Provider update rejected; becomes drift | Owner, Provider Team |
+| `provider_update.auto_approved` | Provider update auto-approved by pre-authorization policy | Owner (informational) |
+
+### 4.5 Dependency and Relationship Events
+
+| Event Type | Trigger | Default Audience |
+|-----------|---------|-----------------|
+| `dependency.state_changed` | A required dependency's state changed | Owner of dependent entity |
+| `stakeholder.resource_decommissioning` | A shared resource the actor stakes is decommissioning | All stakeholders |
+| `allocation.pool_capacity_low` | Allocation pool capacity below threshold | Pool Owner, Platform Admin |
+| `allocation.released` | Allocation decommissioned; capacity returned to pool | Pool Owner |
+| `cross_tenant_auth.expiring` | Cross-tenant authorization expiring | Both Tenant Admins |
+| `cross_tenant_auth.revoked` | Cross-tenant authorization revoked while allocation active | Both Tenant Admins, Affected Resource Owners |
+
+### 4.6 Governance Events
+
+| Event Type | Trigger | Default Audience |
+|-----------|---------|-----------------|
+| `policy.activated` | Policy moved to active status | Platform Admin, Policy Owner |
+| `policy.deactivated` | Policy deactivated | Platform Admin, Policy Owner |
+| `policy_provider.trust_elevated` | Policy Provider mode level elevated | Platform Admin, Security Team |
+| `profile.changed` | Active deployment profile changed | Platform Admin, All Tenant Admins |
+| `catalog_item.deprecated` | Catalog item deprecated | All consumers with active resources of that type |
+
+### 4.7 Security and System Events (Mandatory โ€” Non-Suppressable)
+
+| Event Type | Trigger | Audience |
+|-----------|---------|---------|
+| `audit.chain_integrity_alert` | Hash chain verification failure detected | Security Team, Platform Admin |
+| `sovereignty.violation` | Resource in violation of sovereignty constraints | Platform Admin, Security Team, Resource Owner |
+| `sovereignty.migration_required` | Provider sovereignty change requires entity migration | Platform Admin, Resource Owner |
+| `federation.tunnel_degraded` | DCM-to-DCM federation tunnel health degraded | Platform Admin, SRE |
+| `auth.provider_failover` | Auth Provider failed over to secondary | Platform Admin |
+| `rehydration.blocked` | Concurrent rehydration attempt rejected | Requesting Actor, Platform Admin |
+| `security.unsanctioned_provider_write` | Attempted write to Realized Store without Requested State ref | Security Team, Platform Admin |
+
+---
+
+## 5. Audience Resolution โ€” The Relationship Graph Model
+
+### 5.1 The Fundamental Rule
+
+**The audience for a notification is every entity with a stake in the changed resource, resolved by traversing the relationship graph from the changed entity.**
+
+The notification system does not maintain a separate subscriber list per entity. It derives the audience at event time by traversing the relationship graph. This means the audience is always current โ€” adding a new VM attachment to a VLAN automatically includes that VM's owner in future VLAN notifications, without any subscription update required.
+
+### 5.2 Audience Resolution Algorithm
+
+```
+Event fires on entity E (e.g., VLAN-100 decommissioning)
+  โ”‚
+  โ–ผ Step 1: Resolve direct owner
+  โ”‚   entity.owned_by_tenant_uuid โ†’ Tenant Admin and resource owner actors
+  โ”‚   Audience role: owner
+  โ”‚
+  โ–ผ Step 2: Traverse relationship graph
+  โ”‚   For each relationship on entity E:
+  โ”‚     Check: is this relationship type notification-relevant for this event type?
+  โ”‚     Check: does the relationship's stake_strength meet the minimum for this event?
+  โ”‚     If yes: resolve the related entity's owner โ†’ add to audience
+  โ”‚     Audience role: stakeholder
+  โ”‚
+  โ–ผ Step 3: Check for approval requirements
+  โ”‚   Does this event require approval from a specific actor?
+  โ”‚   If yes: add approver to audience
+  โ”‚   Audience role: approver
+  โ”‚
+  โ–ผ Step 4: Apply mandatory system audiences
+  โ”‚   Security events: always include Security Team and Platform Admin
+  โ”‚   Governance events: always include Policy Owner and Platform Admin
+  โ”‚   (These cannot be filtered out by subscription preferences)
+  โ”‚
+  โ–ผ Step 5: Apply actor subscription overrides
+  โ”‚   Actors with explicit subscriptions to this event type โ†’ include/exclude per subscription
+  โ”‚   (Subscriptions can add additional audience; they cannot remove mandatory audiences)
+  โ”‚
+  โ–ผ Step 6: Deduplicate and resolve contact details
+  โ”‚   Same actor via multiple paths โ†’ one notification with all audience_roles listed
+  โ”‚   Resolve each actor to their configured notification channels
+  โ”‚
+  โ–ผ Step 7: Route to Notification Provider(s)
+      One notification per actor per configured channel
+      Notification envelope includes audience_role
+```
+
+### 5.3 Relationship Notification Relevance
+
+The Resource Type Specification declares which relationship types are notification-relevant and for which events:
+
+```yaml
+resource_type_spec:
+  fqn: Network.VLAN
+  notification_rules:
+    - event_type: entity.decommissioning
+      notify_relationships:
+        - relationship_type: attached_to     # source direction (VMs attached to this VLAN)
+          min_stake_strength: required        # only required stakes get notified
+          traversal_depth: 1                  # direct relationships only
+          audience_role: stakeholder
+        - relationship_type: attached_to
+          min_stake_strength: optional        # optional stakes get informational notice
+          traversal_depth: 1
+          audience_role: observer
+
+    - event_type: entity.state_changed
+      notify_relationships:
+        - relationship_type: attached_to
+          min_stake_strength: required
+          traversal_depth: 1
+          audience_role: stakeholder
+```
+
+**Traversal depth:** `1` means direct relationships only. `2` means relationships of related entities. In most cases `1` is correct โ€” deeper traversal is reserved for critical security events that affect the entire graph.
+
+### 5.4 Cross-Tenant Notification
+
+Notification traversal follows relationship graphs across Tenant boundaries. If a VM in AppTeam Tenant has a `required` stake in a VLAN owned by NetworkOps Tenant, and the VLAN is decommissioned, AppTeam receives a stakeholder notification โ€” even though the VLAN belongs to a different Tenant.
+
+Cross-tenant notifications are governed by the same sovereignty rules as cross-tenant data access:
+- Notification content is limited to what the receiving Tenant is authorized to know
+- The notification identifies the changed resource but does not expose the owning Tenant's configuration details
+- Sovereignty checks apply to notification delivery (a notification about an EU-sovereign resource cannot be delivered to a US-based endpoint)
+
+```yaml
+notification_sovereignty_check:
+  # Before delivering cross-tenant notification:
+  check:
+    - receiver_tenant_sovereignty_compatible: true
+    - notification_content_authorized_for_receiver: true
+    - delivery_endpoint_jurisdiction_compatible: true
+  on_failure: redact_and_deliver   # or: suppress_with_audit | block_with_alert
+```
+
+---
+
+## 6. Notification Subscriptions
+
+### 6.1 Three Subscription Tiers
+
+**Tier 1 โ€” Mandatory System Notifications (non-suppressable):**
+Security events, sovereignty violations, audit chain breaks. Always delivered to the declared system audiences (Security Team, Platform Admin) regardless of any subscription configuration. No actor or policy can suppress these.
+
+**Tier 2 โ€” Tenant Default Notifications:**
+Configured by Tenant admins for all resources in their Tenant. Establishes the baseline notification behavior โ€” which events trigger notifications, which channels to use, and which urgency mapping to apply.
+
+```yaml
+tenant_notification_defaults:
+  tenant_uuid: 
+  notification_provider_uuid: 
+
+  default_channel_config:
+    channel_type: slack
+    workspace: "corp"
+    urgency_routing:
+      critical: "#platform-incidents"
+      high: "#platform-alerts"
+      medium: "#platform-notifications"
+      low: "#platform-digest"        # batched hourly
+
+  # Which event categories are enabled by default for all resources in this Tenant
+  enabled_event_categories:
+    request_lifecycle: [request.completed, request.failed, request.gatekeeper_rejected]
+    resource_lifecycle: [entity.state_changed, entity.ttl_warning, entity.decommissioning]
+    drift: [drift.detected, unsanctioned_change.detected]
+    provider_update: [provider_update.requires_approval, provider_update.rejected]
+    dependency: [stakeholder.resource_decommissioning, cross_tenant_auth.revoked]
+
+  # Urgency defaults per event type
+  urgency_overrides:
+    unsanctioned_change.detected: critical
+    drift.detected: high
+    entity.ttl_warning: medium
+    request.completed: low
+```
+
+**Tier 3 โ€” Actor-Level Subscriptions:**
+Individual actors subscribe to specific events on specific resources or resource types. Most useful for service accounts (CI/CD pipelines, monitoring tools) that need targeted event feeds.
+
+```yaml
+actor_notification_subscription:
+  subscription_uuid: 
+  actor_uuid: 
+  notification_provider_uuid: 
+
+  channel_config:
+    channel_type: pagerduty
+    service_id: "payments-api-on-call"
+    escalation_policy_id: "payments-prod"
+
+  subscriptions:
+    # Subscribe to all drift events on VMs in AppTeam Tenant
+    - scope:
+        tenant_uuid: 
+        resource_type: Compute.VirtualMachine
+      events: [drift.detected, unsanctioned_change.detected]
+      urgency_override: high
+
+    # Subscribe to decommission of a specific VLAN I depend on
+    - scope:
+        entity_uuid: 
+      events: [entity.decommissioning, entity.decommissioned]
+      urgency_override: critical
+```
+
+### 6.2 Subscription Composition Rules
+
+When multiple subscription tiers match for the same actor and event:
+- Mandatory system notifications always fire (cannot be suppressed)
+- Tenant defaults fire unless the actor's subscription explicitly opts out for that event type
+- Actor subscriptions can add additional channels or override urgency โ€” they do not suppress Tenant defaults unless the subscription explicitly declares `suppress_tenant_default: true`
+- Deduplication: if the same notification would be delivered to the same actor via two channels from two subscription matches, deliver once per channel (not once per subscription match)
+
+---
+
+## 7. Notification Payload โ€” The Unified Envelope
+
+Every notification delivered to a Notification Provider uses this unified envelope. The Notification Provider translates it to the delivery channel's native format.
+
+```yaml
+notification:
+  # Identity
+  notification_uuid:               # idempotency key
+  correlation_id:                  # links to the audit record for the triggering event
+  generated_at: 
+
+  # The event
+  event_type: entity.decommissioning     # from the closed taxonomy
+  event_uuid:                      # the triggering event's UUID
+  urgency: 
+
+  # The subject entity
+  entity:
+    uuid: 
+    handle: 
+    resource_type: Network.VLAN
+    display_name: "VLAN-100 (EU-WEST Production)"
+    tenant_uuid: 
+    tenant_display_name: "NetworkOps"
+
+  # Audience context
+  audience:
+    actor_uuid: 
+    actor_display_name: "Jane Smith"
+    audience_role: 
+    # stakeholder: explains WHY this actor is in the audience
+    stakeholder_reason:
+      via_entity_uuid:        # "because your VM-A is attached to this VLAN"
+      via_relationship_type: attached_to
+      via_entity_display_name: "VM-A (payments-api-server-01)"
+
+  # What changed
+  context:
+    previous_state: OPERATIONAL
+    new_state: DECOMMISSIONING
+    change_summary: "VLAN-100 decommission initiated by NetworkOps team"
+    changed_fields: []
+    changed_by:
+      actor_uuid: 
+      actor_display_name: "Bob Jones (NetworkOps)"
+    effective_at: 
+
+  # Action required (if any)
+  requires_action: false
+  action:
+    type: null                           # approve | acknowledge | migrate | release_stake
+    description: null
+    action_url: null
+    deadline: null
+
+  # Deep links
+  links:
+    entity_url: "https://dcm.corp.example.com/resources/"
+    event_url: "https://dcm.corp.example.com/audit/"
+    related_entities:
+      - uuid: 
+        display_name: "VM-A (payments-api-server-01)"
+        url: "https://dcm.corp.example.com/resources/"
+```
+
+### 7.1 Urgency Mapping
+
+| Urgency | Meaning | Typical delivery target |
+|---------|---------|------------------------|
+| `critical` | Immediate action required; outage or security risk imminent | On-call pager, incident channel |
+| `high` | Action required; significant impact if not addressed | Alert channel, SRE queue |
+| `medium` | Action recommended; non-urgent but should not be ignored | Notification channel, daily digest |
+| `low` | Informational; no action required | Digest, async channel |
+
+Default urgency per event type is declared in the event taxonomy. Tenant defaults and actor subscriptions may override upward or downward.
+
+---
+
+## 8. The Delivery Pipeline
+
+```
+Event fires (e.g., VLAN-100 enters DECOMMISSIONING state)
+  โ”‚
+  โ–ผ Stage 1: Audit record written (Stage 1 Commit Log โ€” synchronous)
+  โ”‚   ENTITY_STATE_CHANGED audit record committed
+  โ”‚   event_uuid assigned
+  โ”‚
+  โ–ผ Stage 2: Notification Router evaluates
+  โ”‚   Load entity relationship graph for VLAN-100
+  โ”‚   Run audience resolution algorithm (Section 5.2)
+  โ”‚   Result: [AppTeam, DevTeam, OpsTeam] as stakeholders; [NetworkOps] as owner
+  โ”‚
+  โ–ผ Stage 3: Subscription resolution
+  โ”‚   For each audience member:
+  โ”‚     Resolve Tier 1 mandatory notifications
+  โ”‚     Apply Tier 2 Tenant defaults
+  โ”‚     Apply Tier 3 actor subscriptions
+  โ”‚     Determine: which Notification Provider(s); which channel config; urgency
+  โ”‚
+  โ–ผ Stage 4: Notification envelope generation
+  โ”‚   One envelope per audience member per delivery
+  โ”‚   Audience role set correctly (owner / stakeholder / approver / observer)
+  โ”‚   Stakeholder reason populated for non-owners
+  โ”‚
+  โ–ผ Stage 5: Route to Notification Provider(s)
+  โ”‚   POST to provider delivery endpoint with notification envelope
+  โ”‚   Provider translates to delivery channel (Slack, PagerDuty, email, etc.)
+  โ”‚   Provider returns delivery_uuid and status
+  โ”‚
+  โ–ผ Stage 6: Delivery confirmation
+  โ”‚   Provider reports: delivered | failed | queued
+  โ”‚   Delivery record written to Notification Delivery Store
+  โ”‚   NOTIFICATION_DISPATCHED audit record written (async)
+  โ”‚
+  โ–ผ Stage 7: Failure handling
+      On provider delivery failure:
+        Retry per provider's declared retry policy
+        On exhaustion: dead_letter to platform admin
+        On critical urgency exhaustion: escalate immediately
+        NOTIFICATION_DELIVERY_FAILED audit record written
+```
+
+### 8.1 Notification Delivery Store
+
+A lightweight store (not the Audit Store) tracking delivery status per notification:
+
+```yaml
+notification_delivery_record:
+  delivery_uuid: 
+  notification_uuid: 
+  actor_uuid: 
+  notification_provider_uuid: 
+  channel_type: slack
+  status: 
+  dispatched_at: 
+  delivered_at: 
+  failure_reason: 
+  retry_count: 2
+```
+
+---
+
+## 9. Provider Update Notification Integration
+
+Provider Update Notifications (doc 06, Section 7a) integrate with the notification model at two points:
+
+**When provider submits update notification:**
+- `provider_update.submitted` fires โ†’ Owner notified (informational)
+
+**When provider update requires consumer approval:**
+- `provider_update.requires_approval` fires โ†’ Owner notified (action required)
+- `action.type: approve`
+- `action.action_url` points to `/api/v1/resources/{uuid}/provider-notifications/{uuid}/approve`
+- `action.deadline` set per policy (default PT24H โ€” if no response, escalate)
+
+**On resolution:**
+- Approved โ†’ `provider_update.approved` fires โ†’ Owner notified; Stakeholders notified of state change via `entity.state_changed`
+- Rejected โ†’ `provider_update.rejected` fires โ†’ Owner notified; becomes drift event โ†’ `drift.detected` fires
+
+---
+
+## 10. Relationship to Webhooks and Message Bus
+
+### 10.1 Webhooks as a Notification Channel
+
+Outbound webhooks (doc 18) are now **one delivery channel type within the Notification Provider model** rather than a parallel mechanism. A Notification Provider with `channel_type: webhook` delivers notifications to configured HTTP endpoints using the unified notification envelope.
+
+The webhook registration model (doc 18, Section 3.2) is superseded for new implementations by actor-level subscriptions (Section 6.1, Tier 3) with a webhook-type Notification Provider. Existing webhook registrations remain supported via a compatibility layer.
+
+### 10.2 Message Bus as Notification Infrastructure
+
+The Message Bus Provider (doc 18, Section 5) is the **internal transport** for the notification pipeline. The Notification Router publishes notification events to the Message Bus. Notification Providers subscribe to their assigned topics. This decouples event generation from delivery and enables high-throughput notification processing.
+
+```
+DCM Event โ†’ Notification Router โ†’ Message Bus โ†’ Notification Provider subscription
+```
+
+The Message Bus is infrastructure โ€” not a notification channel. Consumers do not subscribe to the Message Bus directly for notifications; they use the subscription model (Section 6.1).
+
+---
+
+## 11. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `NOT-001` | The audience for every notification is derived from the entity relationship graph at event time. DCM does not maintain static subscriber lists per entity. |
+| `NOT-002` | Mandatory system notifications (Tier 1: security, sovereignty, audit chain) are never suppressable by any subscription configuration or policy. |
+| `NOT-003` | Cross-tenant notifications carry only information the receiving Tenant is authorized to see. Sovereignty checks apply to notification delivery endpoints. |
+| `NOT-004` | Every notification dispatch is an audit record. Delivery failures are tracked. Critical urgency delivery exhaustion triggers immediate escalation to Platform Admin. |
+| `NOT-005` | Provider Update Notifications that require consumer approval carry `action.type: approve` and `action.deadline`. If the deadline passes without resolution, the notification escalates per policy. |
+| `NOT-006` | Notification traversal depth is bounded. Resource Type Specifications declare the maximum traversal depth for each event type. Default: depth 1 (direct relationships only). |
+| `NOT-007` | A Notification Provider must be registered and active before notifications can be delivered. DCM does not have a built-in delivery channel โ€” at minimum a webhook-type Notification Provider must be configured for external delivery. |
+| `NOT-008` | The notification event taxonomy is a closed vocabulary. Custom event types are not supported. New event types require a DCM registry proposal following standard governance. |
+
+---
+
+## 12. Related Concepts
+
+- **Notification Provider** โ€” the ninth DCM provider type; handles translation and delivery
+- **Notification Router** โ€” DCM control plane component that resolves audiences and routes to providers
+- **Audience Resolution** โ€” deriving notification recipients from the entity relationship graph
+- **Notification Subscription** โ€” actor or Tenant declaration of notification preferences
+- **Notification Delivery Store** โ€” lightweight store tracking delivery status
+- **Provider Update Notification** โ€” formal provider mechanism for reporting authorized state changes (see doc 06, Section 7a)
+- **Outbound Webhook** โ€” one delivery channel type within the Notification Provider model
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/24-operational-models.md b/content/docs/data-model/24-operational-models.md
new file mode 100644
index 0000000..153f21c
--- /dev/null
+++ b/content/docs/data-model/24-operational-models.md
@@ -0,0 +1,864 @@
+# DCM Data Model โ€” Operational Models
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: POLICY**
+>
+> The Policy abstraction โ€” Recovery Policy types, trigger vocabulary, action vocabulary
+
+
+**Related Documents:** [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md) | [Policy Profiles](14-policy-profiles.md) | [Notification Model](23-notification-model.md)
+
+---
+
+## 1. Purpose
+
+This document defines the operational models that govern DCM behavior at the edges of the normal provisioning lifecycle โ€” when things go wrong, take too long, or produce ambiguous outcomes. Four operational models are defined:
+
+1. **Timeout Model** โ€” assembly, dispatch, and reserve-query timeouts
+2. **Cancellation Propagation Model** โ€” consumer-initiated cancellation at any lifecycle stage
+3. **Discovery Scheduling Model** โ€” what triggers discovery cycles and how they are managed
+4. **Recovery Policy Model** โ€” the unified, policy-governed response to all failure and ambiguity scenarios
+
+The Recovery Policy Model is the foundational concept. Timeouts, cancellation outcomes, partial realization, and compensation failures all produce trigger conditions that Recovery Policies handle. Organizations declare their recovery posture via profile-bound Policy Groups โ€” not via ad-hoc per-incident decisions.
+
+---
+
+## 2. Timeout Model
+
+### 2.1 Three Timeout Scopes
+
+There are three distinct timeout concerns in the DCM pipeline. Each is independently configurable and independently audited.
+
+```yaml
+timeout_declarations:
+  assembly_timeout:
+    description: "Maximum time for the Request Payload Processor to complete nine-step assembly"
+    profile_defaults:
+      minimal: PT5M
+      dev: PT5M
+      standard: PT3M
+      prod: PT2M
+      fsi: PT2M
+      sovereign: PT2M
+    on_timeout: trigger ASSEMBLY_TIMEOUT recovery policy
+    includes: layer_resolution, policy_evaluation, placement_engine_loop
+    # Assembly timeout fires if the total assembly pipeline exceeds this duration
+    # Individual sub-steps also have per-step timeouts (see below)
+
+  dispatch_timeout:
+    description: "Maximum time to wait for provider realization after dispatch"
+    profile_defaults:
+      minimal: PT2H
+      dev: PT1H
+      standard: PT1H
+      prod: PT30M
+      fsi: PT30M
+      sovereign: PT30M
+    resource_type_overrides:
+      # Some resource types legitimately take longer to provision
+      Compute.BareMetalServer: PT4H
+      Storage.LargeVolume: PT2H
+    on_timeout: trigger DISPATCH_TIMEOUT recovery policy
+
+  reserve_query_timeout:
+    description: "Maximum time for a single provider to respond to a reserve query"
+    profile_defaults:
+      minimal: PT30S
+      dev: PT30S
+      standard: PT10S
+      prod: PT5S
+      fsi: PT5S
+      sovereign: PT10S
+    on_timeout: skip this provider; continue placement loop with remaining candidates
+    # Reserve query timeout does not trigger RESERVE_QUERY_TIMEOUT recovery policy
+    # unless ALL candidates have timed out or been exhausted
+```
+
+### 2.2 Per-Step Assembly Sub-Timeouts
+
+The nine-step assembly has per-step sub-timeouts. These are not independently configurable โ€” they are proportional fractions of the assembly_timeout:
+
+| Step | Fraction of assembly_timeout |
+|------|----------------------------|
+| Layer Resolution | 20% |
+| Layer Merge | 10% |
+| Policy Evaluation (each policy) | 15% total, 5% per Mode 1/2, 30s per Mode 3, PT2M per Mode 4 |
+| Placement Engine Loop | 40% |
+| Requested State Persistence | 10% |
+
+A Mode 4 Policy Provider that takes longer than PT2M per query causes an ASSEMBLY_TIMEOUT regardless of the overall assembly_timeout remaining. This prevents a single slow Policy Provider from consuming the entire assembly budget.
+
+### 2.3 Timeout Audit Records
+
+Every timeout produces an audit record:
+
+```yaml
+audit_record:
+  action: ASSEMBLY_TIMEOUT | DISPATCH_TIMEOUT | RESERVE_QUERY_TIMEOUT
+  actor:
+    type: system
+    system_actor:
+      component: request_payload_processor | provider_dispatch | placement_engine
+      trigger: timeout
+  entity_uuid: 
+  details:
+    timeout_duration: 
+    actual_elapsed: 
+    step_at_timeout: 
+    recovery_policy_triggered: 
+```
+
+---
+
+## 3. Cancellation Propagation Model
+
+### 3.1 Three Cancellation Scenarios
+
+Cancellation behavior depends on the entity's lifecycle state at the time the consumer submits a cancellation request.
+
+**Scenario 1 โ€” Cancel before dispatch (ACKNOWLEDGED โ†’ ASSEMBLING โ†’ AWAITING_APPROVAL):**
+
+```
+Consumer submits DELETE /api/v1/requests/{uuid}
+  โ”‚
+  โ–ผ Entity state: pre-DISPATCHED
+  โ”‚   Assembly halted immediately
+  โ”‚   No provider interaction required
+  โ”‚   Intent State record marked CANCELLED
+  โ”‚   Entity enters CANCELLED state (terminal)
+  โ”‚   Audit: REQUEST_CANCELLED
+  โ”‚   Recovery Policy: not triggered (clean cancel)
+  โ”‚
+  โ””โ”€โ”€ Response: 200 OK { "status": "CANCELLED" }
+```
+
+**Scenario 2 โ€” Cancel after dispatch, provider not yet started (DISPATCHED):**
+
+```
+Consumer submits DELETE /api/v1/requests/{uuid}
+  โ”‚
+  โ–ผ Entity state: DISPATCHED (provider received payload but has not started)
+  โ”‚   DCM sends cancellation payload to provider cancel endpoint
+  โ”‚   Provider acknowledges: "not started, cancellation clean"
+  โ”‚   Entity enters CANCELLED state (terminal)
+  โ”‚   Recovery Policy: not triggered (clean cancel)
+  โ”‚
+  โ””โ”€โ”€ Response: 202 Accepted { "status": "CANCELLING" }
+      โ†’ status polling shows CANCELLED when provider confirms
+```
+
+**Scenario 3 โ€” Cancel while provider is executing (PROVISIONING):**
+
+```
+Consumer submits DELETE /api/v1/requests/{uuid}
+  โ”‚
+  โ–ผ Entity state: PROVISIONING
+  โ”‚   DCM checks provider.supports_cancellation
+  โ”‚
+  โ”œโ”€โ”€ Provider supports cancellation:
+  โ”‚   DCM sends cancellation payload
+  โ”‚   Provider attempts rollback
+  โ”‚   โ”œโ”€โ”€ Rollback clean: entity โ†’ CANCELLED (terminal)
+  โ”‚   โ”œโ”€โ”€ Rollback partial: trigger CANCELLATION_FAILED recovery policy
+  โ”‚   โ””โ”€โ”€ No response: trigger CANCELLATION_FAILED recovery policy
+  โ”‚
+  โ””โ”€โ”€ Provider does not support cancellation:
+      Entity enters CANCEL_PENDING state
+      DCM waits for provider to complete
+      On provider REALIZED response:
+        Recovery Policy LATE_RESPONSE_RECEIVED fires
+        (configured action: typically DISCARD_AND_REQUEUE for cancellation context)
+      On provider FAILED response:
+        Entity โ†’ FAILED (terminal) โ€” no compensation needed
+```
+
+### 3.2 Provider Cancellation Capability Declaration
+
+Providers declare cancellation support in their registration:
+
+```yaml
+provider_cancellation_capabilities:
+  supports_cancellation: true
+  cancellation_supported_during: [DISPATCHED, PROVISIONING]
+  # DISPATCHED: can cancel before work begins
+  # PROVISIONING: can cancel and roll back mid-execution
+  cancellation_endpoint: POST /api/v1/provider/entities/{entity_uuid}/cancel
+  cancellation_response_time: PT30S    # SLA for cancellation response
+  partial_rollback_possible: true
+  # true: cancellation may leave partial resources โ†’ CANCELLATION_FAILED path
+  # false: cancellation is all-or-nothing (rare)
+```
+
+### 3.3 Cancellation Payload
+
+```json
+{
+  "cancellation_uuid": "",
+  "entity_uuid": "",
+  "requested_state_uuid": "",
+  "reason": "consumer_requested | timeout | policy_triggered",
+  "requested_at": "",
+  "best_effort": true
+}
+```
+
+`best_effort: true` is always set โ€” DCM never guarantees cancellation success. The provider makes a best-effort attempt; outcomes flow through the Recovery Policy model.
+
+---
+
+## 4. Discovery Scheduling Model
+
+### 4.1 The Discovery Scheduler Component
+
+The **Discovery Scheduler** is a DCM control plane component responsible for triggering discovery cycles. It maintains a priority queue of pending discovery requests and dispatches them to the appropriate Service Provider's discovery endpoint.
+
+The Discovery Scheduler is distinct from drift detection. The Discovery Scheduler triggers discovery and writes Discovered State. Drift Detection reads Discovered State and compares it to Realized State. These are separate, independent components.
+
+### 4.2 Three Discovery Trigger Types
+
+**Trigger Type 1 โ€” Scheduled (cron-based):**
+
+Discovery schedules are declared in the Resource Type Specification and in provider registrations. The Discovery Scheduler runs these on the declared cadence.
+
+```yaml
+resource_type_spec:
+  fqn: Compute.VirtualMachine
+  discovery_schedule:
+    default_interval: PT15M      # discover VMs every 15 minutes
+    # Override by profile:
+    profile_overrides:
+      minimal: PT4H              # less frequent in home lab
+      fsi: PT5M                  # more frequent in regulated environments
+      sovereign: PT5M
+
+  # Per-provider discovery endpoint
+  discovery_endpoint_path: /api/v1/provider/discover
+  discovery_method: api_query    # api_query | passive_event | hybrid
+```
+
+```yaml
+provider_registration:
+  discovery_capabilities:
+    supports_discovery: true
+    discovery_endpoint: POST /api/v1/provider/entities/discover
+    max_entities_per_discovery_batch: 1000
+    discovery_latency_p95: PT10S    # how long discovery typically takes
+    supports_incremental_discovery: true
+    # incremental: only entities changed since last_discovery_timestamp
+    # full: all entities every time
+```
+
+**Trigger Type 2 โ€” Event-triggered:**
+
+Specific DCM events automatically schedule an out-of-cycle discovery pass:
+
+```yaml
+event_triggered_discovery:
+  triggers:
+    - event: entity.realized
+      discovery_delay: PT30S           # allow provider to stabilize
+      scope: this_entity
+      reason: "Confirm realization matches Requested State"
+
+    - event: drift.resolved
+      discovery_delay: PT60S
+      scope: this_entity
+      reason: "Confirm remediation took effect"
+
+    - event: provider_update.approved
+      discovery_delay: PT30S
+      scope: this_entity
+      reason: "Confirm provider update is reflected in infrastructure"
+
+    - event: provider.degraded
+      discovery_delay: PT0S           # immediate
+      scope: all_entities_on_provider
+      reason: "Assess impact of provider degradation"
+
+    - event: TIMEOUT_PENDING          # recovery trigger
+      discovery_delay: PT5M
+      scope: this_entity
+      reason: "Orphan detection after timeout"
+
+    - event: COMPENSATION_FAILED
+      discovery_delay: PT0S           # immediate
+      scope: this_entity_and_dependents
+      reason: "Find orphaned resources after compensation failure"
+```
+
+**Trigger Type 3 โ€” On-demand:**
+
+Platform admins and SREs can trigger discovery manually:
+
+```
+POST /api/v1/admin/discovery/trigger
+
+{
+  "scope": "entity | resource_type | provider | tenant",
+  "entity_uuid": "",          # if scope: entity
+  "resource_type": "",         # if scope: resource_type
+  "provider_uuid": "",         # if scope: provider
+  "tenant_uuid": "",           # if scope: tenant
+  "reason": "incident investigation",
+  "priority": "high"
+}
+```
+
+On-demand discovery is also used by:
+- The CI/CD pipeline pre-validation step (confirm current state before assembly)
+- The brownfield ingestion pipeline (initial discovery of existing infrastructure)
+- The orphan detection pipeline (targeted search for potentially-orphaned resources)
+
+### 4.3 Discovery Queue Management
+
+The Discovery Scheduler manages a priority queue. Priority order:
+
+1. **Critical** โ€” COMPENSATION_FAILED orphan detection, sovereignty violation assessment
+2. **High** โ€” on-demand from platform admin, event-triggered (provider.degraded)
+3. **Standard** โ€” event-triggered (entity.realized, drift.resolved)
+4. **Background** โ€” scheduled discovery passes
+
+Queue depth is bounded per profile. When the queue is full, new Background-priority items are dropped (with a log entry). Standard and above are never dropped โ€” they wait.
+
+### 4.4 Discovery Audit
+
+Every discovery cycle produces an audit record:
+
+```yaml
+audit_record:
+  action: DISCOVERY_CYCLE_COMPLETED | DISCOVERY_CYCLE_FAILED
+  actor:
+    type: system
+    system_actor:
+      component: discovery_scheduler
+      trigger: scheduled | event_triggered | on_demand
+      trigger_event_uuid: 
+  entity_uuid:           # null for batch discovery
+  details:
+    entities_discovered: 47
+    new_entities_found: 2           # brownfield candidates
+    duration: PT8S
+```
+
+---
+
+## 5. Recovery Policy Model
+
+### 5.1 Recovery Policy as a Policy Type
+
+Recovery Policies are a formal DCM policy type alongside GateKeeper, Validation, and Transformation. They use the same authoring model, the same GitOps store, the same shadow mode validation, the same activation workflow, and the same audit trail.
+
+```yaml
+recovery_policy:
+  artifact_metadata:
+    uuid: 
+    handle: "system/recovery/discard-on-timeout"
+    version: "1.0.0"
+    status: active
+    owned_by: { display_name: "DCM Core Team" }
+
+  policy_type: recovery
+  trigger: DISPATCH_TIMEOUT         # the trigger condition this policy handles
+  action: DISCARD_AND_REQUEUE       # the action to take
+
+  # Optional additional conditions
+  conditions:
+    - field: entity.resource_type
+      operator: in
+      value: [Compute.VirtualMachine, Container.Pod]
+    - field: entity.owned_by_tenant.profile
+      operator: equals
+      value: prod
+
+  # Action parameters (depend on action type)
+  action_parameters:
+    requeue_delay: PT0S             # immediate requeue
+    notify_before_action: true
+    notification_urgency: high
+
+  # Deadline for NOTIFY_AND_WAIT actions
+  deadline: null                    # not applicable for DISCARD_AND_REQUEUE
+  on_deadline_exceeded: null
+```
+
+### 5.2 Trigger Vocabulary (Closed)
+
+| Trigger | Description |
+|---------|-------------|
+| `ASSEMBLY_TIMEOUT` | Assembly pipeline exceeded configured timeout |
+| `DISPATCH_TIMEOUT` | Provider did not respond within dispatch_timeout |
+| `RESERVE_QUERY_ALL_EXHAUSTED` | All placement candidates timed out or rejected |
+| `LATE_RESPONSE_RECEIVED` | Provider responded after DCM declared timeout |
+| `CANCELLATION_SENT` | DCM sent cancellation to provider |
+| `CANCELLATION_CONFIRMED` | Provider confirmed clean cancellation |
+| `CANCELLATION_FAILED` | Provider could not cancel; partial state possible |
+| `PARTIAL_REALIZATION` | Compound service partially realized |
+| `COMPENSATION_IN_PROGRESS` | Rollback of partial components underway |
+| `COMPENSATION_FAILED` | Rollback itself failed; orphaned resources possible |
+
+### 5.3 Action Vocabulary (Closed)
+
+| Action | Description |
+|--------|-------------|
+| `DRIFT_RECONCILE` | Schedule discovery pass; let drift detection resolve actual state via configured drift response policy |
+| `DISCARD_AND_REQUEUE` | Best-effort cleanup sent to provider; new request cycle created immediately from Intent State |
+| `DISCARD_NO_REQUEUE` | Best-effort cleanup sent to provider; entity FAILED; no automatic requeue |
+| `ACCEPT_LATE_REALIZATION` | Accept late provider response; write Realized State; entity proceeds to OPERATIONAL |
+| `COMPENSATE_AND_FAIL` | Execute compensation rollback for compound service; entity FAILED when complete |
+| `NOTIFY_AND_WAIT` | Fire notification to configured audience; wait for human decision up to deadline |
+| `ESCALATE` | Notify platform admin immediately; no automatic action |
+| `RETRY` | Retry the failed operation with configured backoff |
+
+### 5.4 The Four Built-in Recovery Profile Groups
+
+#### recovery-automated-reconciliation
+
+"Let the system converge on correct state โ€” trust drift detection and policy."
+
+Appropriate for: standard and dev environments where operational continuity takes priority over strict consistency.
+
+```yaml
+recovery_policy_group:
+  handle: "system/group/recovery-automated-reconciliation"
+  concern_type: recovery_posture
+  policies:
+    - trigger: ASSEMBLY_TIMEOUT
+      action: RETRY
+      max_attempts: 3
+      backoff: exponential
+      initial_interval: PT30S
+      on_exhaustion: ESCALATE
+
+    - trigger: DISPATCH_TIMEOUT
+      action: DRIFT_RECONCILE
+      # Discovery finds what actually exists; drift response policy handles it
+
+    - trigger: LATE_RESPONSE_RECEIVED
+      action: ACCEPT_LATE_REALIZATION
+      # Provider did the work; accept it
+
+    - trigger: CANCELLATION_FAILED
+      action: DRIFT_RECONCILE
+      # Cannot confirm cleanup; discovery finds orphans
+
+    - trigger: PARTIAL_REALIZATION
+      action: DRIFT_RECONCILE
+      # Discover what's there; drift policy handles component gaps
+
+    - trigger: COMPENSATION_FAILED
+      action: ESCALATE
+      # Human needed when cleanup itself fails
+```
+
+#### recovery-discard-and-requeue
+
+"On any ambiguity, clean up and start fresh โ€” prioritize consistency over continuity."
+
+Appropriate for: environments where reproducibility is paramount, resources are cheap to reprovision, untracked resources are a compliance concern.
+
+```yaml
+recovery_policy_group:
+  handle: "system/group/recovery-discard-and-requeue"
+  concern_type: recovery_posture
+  policies:
+    - trigger: ASSEMBLY_TIMEOUT
+      action: RETRY
+      max_attempts: 2
+      on_exhaustion: DISCARD_NO_REQUEUE
+
+    - trigger: DISPATCH_TIMEOUT
+      action: DISCARD_AND_REQUEUE
+      # Best-effort cleanup; new request cycle immediately
+
+    - trigger: LATE_RESPONSE_RECEIVED
+      action: DISCARD_AND_REQUEUE
+      # Provider completed after DCM moved on; discard that work
+      # (requeue already happened on timeout; this prevents duplicate resources)
+
+    - trigger: CANCELLATION_FAILED
+      action: DISCARD_NO_REQUEUE
+      # Cannot clean up; FAILED; human reviews orphans before requeue
+
+    - trigger: PARTIAL_REALIZATION
+      action: COMPENSATE_AND_FAIL
+      # Roll back everything; start fresh
+
+    - trigger: COMPENSATION_FAILED
+      action: ESCALATE
+      # Cannot even roll back; human needed
+```
+
+#### recovery-notify-and-wait
+
+"Never act automatically โ€” always notify a human and wait for explicit authorization."
+
+Appropriate for: FSI and sovereign environments where automated resource creation or deletion has regulatory implications, where change control processes must be honored.
+
+```yaml
+recovery_policy_group:
+  handle: "system/group/recovery-notify-and-wait"
+  concern_type: recovery_posture
+  policies:
+    - trigger: ASSEMBLY_TIMEOUT
+      action: NOTIFY_AND_WAIT
+      deadline: PT2H
+      notification_urgency: high
+      on_deadline_exceeded: ESCALATE
+
+    - trigger: DISPATCH_TIMEOUT
+      action: NOTIFY_AND_WAIT
+      deadline: PT4H
+      notification_urgency: high
+      on_deadline_exceeded: ESCALATE
+
+    - trigger: LATE_RESPONSE_RECEIVED
+      action: NOTIFY_AND_WAIT
+      deadline: PT4H
+      notification_urgency: medium
+      on_deadline_exceeded: DISCARD_NO_REQUEUE
+
+    - trigger: CANCELLATION_FAILED
+      action: NOTIFY_AND_WAIT
+      deadline: PT8H
+      notification_urgency: high
+      on_deadline_exceeded: ESCALATE
+
+    - trigger: PARTIAL_REALIZATION
+      action: NOTIFY_AND_WAIT
+      deadline: PT8H
+      notification_urgency: high
+      on_deadline_exceeded: COMPENSATE_AND_FAIL
+
+    - trigger: COMPENSATION_FAILED
+      action: ESCALATE
+      # Always escalate compensation failures โ€” no deadline
+```
+
+#### recovery-aggressive-retry
+
+"Retry everything before giving up โ€” maximize first-time success rate."
+
+Appropriate for: environments with transient provider issues, where retries are cheap and manual intervention capacity is limited.
+
+```yaml
+recovery_policy_group:
+  handle: "system/group/recovery-aggressive-retry"
+  concern_type: recovery_posture
+  policies:
+    - trigger: ASSEMBLY_TIMEOUT
+      action: RETRY
+      max_attempts: 5
+      backoff: exponential
+      initial_interval: PT15S
+      max_interval: PT5M
+      on_exhaustion: NOTIFY_AND_WAIT
+      deadline: PT2H
+
+    - trigger: DISPATCH_TIMEOUT
+      action: RETRY
+      max_attempts: 3
+      backoff: linear
+      interval: PT5M
+      on_exhaustion: DRIFT_RECONCILE
+
+    - trigger: RESERVE_QUERY_ALL_EXHAUSTED
+      action: RETRY
+      max_attempts: 3
+      backoff: exponential
+      initial_interval: PT1M
+      on_exhaustion: ESCALATE
+
+    - trigger: PARTIAL_REALIZATION
+      action: RETRY
+      retry_scope: failed_components_only   # preserve succeeded components
+      max_attempts: 3
+      interval: PT15M
+      on_exhaustion: COMPENSATE_AND_FAIL
+
+    - trigger: CANCELLATION_FAILED
+      action: DRIFT_RECONCILE
+
+    - trigger: COMPENSATION_FAILED
+      action: ESCALATE
+```
+
+### 5.5 Profile Binding
+
+Recovery profile groups bind to deployment profiles as defaults, with override at Tenant and resource-type levels:
+
+```yaml
+profile_recovery_defaults:
+  minimal:    recovery-automated-reconciliation
+  dev:        recovery-automated-reconciliation
+  standard:   recovery-automated-reconciliation
+  prod:       recovery-notify-and-wait
+  fsi:        recovery-notify-and-wait
+  sovereign:  recovery-notify-and-wait
+
+# Tenant-level override
+tenant_config:
+  tenant_uuid: 
+  recovery_profile_override: recovery-discard-and-requeue
+
+# Resource-type-level override (most specific; wins over Tenant and profile)
+resource_type_recovery_override:
+  resource_type: Compute.VirtualMachine
+  recovery_profile: recovery-aggressive-retry
+  # VMs use aggressive retry; other types use Tenant/profile default
+```
+
+### 5.6 NOTIFY_AND_WAIT Consumer Interface
+
+When a recovery policy fires `NOTIFY_AND_WAIT`, a notification is sent to the entity owner with a time-bounded decision interface:
+
+```
+GET /api/v1/resources/{entity_uuid}/recovery-decisions
+
+Response:
+{
+  "recovery_decision_uuid": "",
+  "trigger": "DISPATCH_TIMEOUT",
+  "entity_uuid": "",
+  "deadline": "",
+  "available_actions": [
+    {
+      "action": "DRIFT_RECONCILE",
+      "description": "Let discovery determine actual state and reconcile automatically"
+    },
+    {
+      "action": "DISCARD_AND_REQUEUE",
+      "description": "Best-effort cleanup, then requeue as a new request"
+    },
+    {
+      "action": "DISCARD_NO_REQUEUE",
+      "description": "Best-effort cleanup only; no automatic requeue"
+    }
+  ]
+}
+
+POST /api/v1/resources/{entity_uuid}/recovery-decisions/{recovery_decision_uuid}
+{
+  "action": "DISCARD_AND_REQUEUE",
+  "reason": "Provider was known to be degraded at time of timeout"
+}
+```
+
+Platform admins may also use the Admin API to resolve pending recovery decisions for any entity regardless of Tenant.
+
+### 5.7 Recovery Policy Evaluation Precedence
+
+The Policy Engine evaluates recovery policies in domain precedence order, same as all other policies:
+
+```
+1. Resource-type-level override (most specific)
+2. Tenant-level override
+3. Active profile's recovery posture group
+4. System default (automated-reconciliation)
+
+First matching policy for the trigger condition wins.
+Multiple recovery policies for the same trigger at the same domain level
+โ†’ policy conflict; CONFLICT_ERROR at ingestion; platform admin notified.
+```
+
+---
+
+## 6. Compound Service Compensation Model
+
+### 6.1 Compensation Declaration in Service Dependencies
+
+Each component in a compound service declares its compensation behavior:
+
+```yaml
+compound_service_spec:
+  service_type: ApplicationStack.WebApp
+  components:
+    - id: vm
+      resource_type: Compute.VirtualMachine
+      required_for_delivery: atomic        # must succeed; failure triggers compensation
+      compensation_on_failure: decommission_immediately
+      compensation_order: 3                # decommissioned last (highest number = last)
+
+    - id: ip
+      resource_type: Network.IPAddress
+      required_for_delivery: atomic
+      compensation_on_failure: release_allocation
+      compensation_order: 1                # decommissioned first
+      depends_on: []
+
+    - id: dns
+      resource_type: DNS.Record
+      required_for_delivery: partial       # failure โ†’ DEGRADED, not FAILED
+      compensation_on_failure: skip        # DNS failure doesn't trigger VM decommission
+      depends_on: [vm, ip]
+
+    - id: loadbalancer
+      resource_type: Network.LoadBalancer
+      required_for_delivery: partial
+      compensation_on_failure: skip
+      depends_on: [vm, ip]
+
+  partial_delivery_policy:
+    min_required_components: [vm, ip]     # compound DEGRADED if only these succeed
+    degraded_is_acceptable: true          # DEGRADED entity is delivered; not FAILED
+    auto_retry_optional_components:
+      enabled: true
+      max_attempts: 3
+      interval: PT15M
+      on_exhaustion: notify_owner
+```
+
+### 6.2 Compensation Execution Order
+
+Compensation always runs in reverse dependency order โ€” last-provisioned is first-decommissioned:
+
+```
+Successful so far: vm โœ“, ip โœ“
+Failed: dns โœ— (atomic)
+Compensation triggered:
+  Step 1: decommission vm (compensation_order: 3 โ†’ runs first in reverse)
+  Step 2: release ip allocation (compensation_order: 1 โ†’ runs second in reverse)
+  Compound entity โ†’ FAILED (terminal for this request cycle)
+```
+
+### 6.3 Compensation Failure
+
+If a compensation step fails (the VM decommission itself fails):
+
+```
+Compensation of vm FAILED
+  Entity enters COMPENSATION_FAILED state
+  COMPENSATION_FAILED recovery policy fires:
+    default: ESCALATE to platform admin
+  Orphan detection triggered immediately:
+    Scoped to provider + entity characteristics
+    Finds the VM that couldn't be decommissioned
+    Creates ORPHAN_CANDIDATE record
+  Platform admin reviews:
+    Manually decommission at provider
+    OR adopt into DCM lifecycle as a new entity
+```
+
+---
+
+## 7. Orphan Detection Pipeline
+
+When cleanup cannot be guaranteed, DCM runs an orphan detection pass to find resources that may have been provisioned but have no corresponding Realized State record.
+
+### 7.1 Orphan Detection Triggers
+
+- Dispatch timeout with cancellation sent
+- Cancellation failed
+- Compensation failed
+- DISCARD_NO_REQUEUE action taken
+- Manual platform admin trigger
+
+### 7.2 Orphan Detection Query
+
+```yaml
+orphan_detection_query:
+  provider_uuid: 
+  time_window:
+    from: 
+    to: 
+  match_criteria:
+    resource_type: 
+    characteristics:             # key fields from the Requested State
+      name_pattern: 
+      size_class: 
+      tags: 
+  exclude:
+    known_realized_state_uuids: [, ...]   # entities DCM knows about
+```
+
+### 7.3 Orphan Candidate Lifecycle
+
+```yaml
+orphan_candidate:
+  orphan_candidate_uuid: 
+  suspected_request_uuid:      # the request that may have created this
+  provider_entity_id:        # what the provider calls it
+  provider_uuid: 
+  discovered_at: 
+  characteristics: { ... }
+  status: 
+  resolution:
+    action: 
+    resolved_by: 
+    resolved_at: 
+```
+
+Orphan candidates are surfaced in the Platform Admin dashboard and generate a NOTIFICATION (audience: Platform Admin) with urgency: high.
+
+---
+
+## 8. New Lifecycle States
+
+Five new states are added to the Infrastructure Resource Entity lifecycle:
+
+| State | Meaning | Recovery Policy Trigger |
+|-------|---------|------------------------|
+| `TIMEOUT_PENDING` | Dispatch timeout fired; cancellation sent; awaiting outcome | `DISPATCH_TIMEOUT` |
+| `LATE_REALIZATION_PENDING` | Provider responded after timeout; NOTIFY_AND_WAIT active | `LATE_RESPONSE_RECEIVED` |
+| `INDETERMINATE_REALIZATION` | State is ambiguous; drift detection resolving | โ€” (drift detection runs) |
+| `COMPENSATION_IN_PROGRESS` | Compound service rollback underway | โ€” |
+| `COMPENSATION_FAILED` | Rollback itself failed; orphaned resources possible | `COMPENSATION_FAILED` |
+
+Updated state machine (additions to doc 02 and doc 06):
+
+```
+Normal flow:
+REQUESTED โ†’ PENDING โ†’ PROVISIONING โ†’ REALIZED โ†’ OPERATIONAL โ†’ DECOMMISSIONED
+
+Recovery states:
+PROVISIONING โ†’ [timeout] โ†’ TIMEOUT_PENDING
+  TIMEOUT_PENDING โ†’ [late response + NOTIFY_AND_WAIT] โ†’ LATE_REALIZATION_PENDING
+  TIMEOUT_PENDING โ†’ [DRIFT_RECONCILE] โ†’ INDETERMINATE_REALIZATION
+  TIMEOUT_PENDING โ†’ [DISCARD_AND_REQUEUE] โ†’ FAILED + new REQUESTED (new cycle)
+
+PROVISIONING โ†’ [partial failure] โ†’ COMPENSATION_IN_PROGRESS
+  COMPENSATION_IN_PROGRESS โ†’ [all compensated] โ†’ FAILED
+  COMPENSATION_IN_PROGRESS โ†’ [compensation fails] โ†’ COMPENSATION_FAILED
+
+LATE_REALIZATION_PENDING โ†’ [human accepts / ACCEPT_LATE] โ†’ REALIZED โ†’ OPERATIONAL
+LATE_REALIZATION_PENDING โ†’ [human discards / DISCARD] โ†’ FAILED
+
+INDETERMINATE_REALIZATION โ†’ [drift reconciles] โ†’ REALIZED or FAILED
+COMPENSATION_FAILED โ†’ [human resolves] โ†’ FAILED (after manual cleanup)
+```
+
+---
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `OPS-010` | Assembly timeout, dispatch timeout, and reserve-query timeout are independently configurable. All are profile-governed with resource-type overrides permitted for types with legitimately long provisioning times. |
+| `OPS-011` | Cancellation is always best-effort. DCM never guarantees cancellation success. All cancellation outcomes flow through the Recovery Policy model. |
+| `OPS-012` | Provider cancellation capability is declared at registration. Providers that do not support cancellation use the CANCEL_PENDING โ†’ LATE_RESPONSE_RECEIVED path when a cancel is requested during PROVISIONING. |
+| `OPS-013` | Discovery is triggered by three independent mechanisms: scheduled (cron), event-triggered, and on-demand. All three write to the Discovered Store independently. |
+| `OPS-014` | Recovery Policies are a formal DCM policy type. They use the same authoring, activation, shadow mode, and audit model as GateKeeper, Validation, and Transformation policies. |
+| `OPS-015` | Four built-in recovery profile groups are provided: recovery-automated-reconciliation, recovery-discard-and-requeue, recovery-notify-and-wait, recovery-aggressive-retry. |
+| `OPS-016` | Recovery profile defaults are bound to deployment profiles. Organizations may override at Tenant or resource-type level. Resource-type override wins over Tenant override wins over profile default. |
+| `OPS-017` | Compound service compensation runs in reverse dependency order. Compensation failure triggers COMPENSATION_FAILED state and immediate orphan detection. |
+| `OPS-018` | Orphan detection triggers on any path where cleanup cannot be guaranteed. Orphan candidates are surfaced to platform admin with urgency: high. |
+| `OPS-019` | NOTIFY_AND_WAIT recovery actions carry a deadline. If the deadline passes without human resolution, the configured on_deadline_exceeded action fires automatically. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/25-control-plane-components.md b/content/docs/data-model/25-control-plane-components.md
new file mode 100644
index 0000000..1ca67e8
--- /dev/null
+++ b/content/docs/data-model/25-control-plane-components.md
@@ -0,0 +1,826 @@
+# DCM Data Model โ€” Control Plane Components
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: RUNTIME**
+>
+> Runtime implementations of the three abstractions โ€” not a fourth abstraction
+
+
+**Related Documents:** [Internal Component Authentication](36-internal-component-auth.md) | [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Operational Models](24-operational-models.md) | [Policy Profiles](14-policy-profiles.md)
+
+---
+
+
+The DCM Control Plane consists of **nine components** that implement the three foundational abstractions at runtime.
+
+## 1. Purpose
+
+> **Internal component authentication:** See [Internal Component Authentication](36-internal-component-auth.md) for the mTLS and interaction credential model governing all component-to-component calls within the DCM control plane.
+
+
+This document formally defines the DCM control plane components that are referenced throughout the data model documents but not previously specified in detail. Two components are defined here:
+
+1. **The Request Orchestrator** โ€” the event bus and coordinator of the request lifecycle pipeline
+2. **The Cost Analysis Component** โ€” the internal DCM component that provides cost signals for placement, catalog, and attribution
+
+---
+
+## 2. The Request Orchestrator
+
+### 2.1 Role
+
+The Request Orchestrator is the **event bus and pipeline coordinator** for all DCM request lifecycle operations. It does not perform any pipeline work itself โ€” it listens for events, evaluates which components need to act on them, and routes work to the appropriate components.
+
+The Request Orchestrator embodies DCM's **data-driven, policy-triggered orchestration model**: the pipeline is not a fixed procedural sequence. It is a cascade of event-condition-action responses, where policies define what happens when specific payload states are observed.
+
+### 2.2 Data-Driven Orchestration Principle
+
+**Policies ARE the orchestration.** The Request Orchestrator does not contain hardcoded pipeline logic. It publishes events to the Policy Engine; policies match on payload type and state; policy actions produce new payload states; those new states trigger further policy evaluations.
+
+This means:
+- Adding a new pipeline step = writing a new policy (no code change)
+- Removing a step = deactivating a policy
+- Changing when a step fires = changing a policy condition
+- A static workflow (e.g., always require human approval for prod VMs) = a policy that always matches for those conditions
+- A dynamic workflow (e.g., route to different approval processes based on cost) = a policy with conditional logic
+
+Static and dynamic flows compose naturally โ€” a static policy defines a guaranteed step; a dynamic policy defines a conditional step. Both are expressed as policies, evaluated by the same engine, producing deterministic outcomes.
+
+**Determinism guarantee:** Dynamic execution remains deterministic because:
+- The payload type vocabulary is a closed set
+- Policy evaluation order within a domain level is deterministic (domain precedence)
+- The payload mutation model is immutable (each policy produces a new payload version)
+- The same input state always produces the same output state
+
+### 2.3 The Payload Type Vocabulary
+
+Every event in DCM carries a payload with a declared type. Policies pattern-match on these types. The payload type vocabulary is the foundational contract of the orchestration model.
+
+```yaml
+payload_types:
+  # Request lifecycle
+  request.initiated:          # consumer submitted a request
+  request.intent_captured:    # Intent State written
+  request.layers_assembled:   # layer assembly complete
+  request.policies_evaluated: # all active policies evaluated
+  request.placement_complete: # provider selected
+  request.dispatched:         # sent to provider
+  request.realized:           # provider confirmed realization
+  request.failed:             # terminal failure
+  request.cancelled:          # cancelled
+
+  # Provider update
+  provider_update.received:   # provider submitted update notification
+  provider_update.evaluated:  # policy evaluation complete
+  provider_update.accepted:   # accepted; Realized State updating
+  provider_update.rejected:   # rejected; becomes drift
+
+  # Drift and discovery
+  discovery.cycle_complete:
+  drift.detected:
+  drift.resolved:
+
+  # Recovery
+  recovery.timeout_fired:
+  recovery.late_response:
+  recovery.compensation_triggered:
+
+  # Governance
+  policy.activated:
+  layer.updated:
+  profile.changed:
+```
+
+### 2.4 Event Routing Model
+
+```
+Event published: { type: "request.initiated", payload: {...}, entity_uuid: X }
+  โ”‚
+  โ–ผ Request Orchestrator receives event
+  โ”‚   Routes to Policy Engine: "evaluate all policies matching request.initiated"
+  โ”‚
+  โ–ผ Policy Engine evaluates in domain precedence order
+  โ”‚   Matching policies fire; payload mutations accumulated
+  โ”‚   New payload state produced: { type: "request.layers_assembled", ... }
+  โ”‚
+  โ–ผ Request Orchestrator receives new event
+  โ”‚   Routes to Policy Engine for next evaluation cycle
+  โ”‚   (parallel if no data dependencies between active policies)
+  โ”‚
+  โ–ผ Continues until terminal state (request.realized or request.failed)
+```
+
+**Parallel execution:** Policies that have no data dependencies on each other evaluate concurrently. The Request Orchestrator tracks dependency declarations between policies and executes in parallel where safe.
+
+### 2.5 Static Flow Support
+
+Organizations that require guaranteed sequential flows express them as ordered policy sets:
+
+```yaml
+static_flow_policy_group:
+  handle: "org/flows/prod-vm-approval-flow"
+  concern_type: orchestration_flow
+  ordered: true                # policies execute in declared sequence, not parallel
+  policies:
+    - step: 1
+      handle: "org/policies/cost-check"
+      condition: "request.initiated AND resource_type=Compute.VirtualMachine AND tenant.profile=prod"
+      on_fail: halt
+    - step: 2
+      handle: "org/policies/manager-approval"
+      condition: "request.cost_estimated > 500"
+      on_fail: halt
+    - step: 3
+      handle: "org/policies/security-review"
+      condition: "always"
+      on_fail: halt
+```
+
+A static flow is a Policy Group with `concern_type: orchestration_flow` and `ordered: true`. The Request Orchestrator respects the declared order. Static flows integrate with dynamic policies โ€” a dynamic policy can fire alongside the static flow steps.
+
+### 2.5a Named Workflows vs Dynamic Policies โ€” How They Compose
+
+The Request Orchestrator does not distinguish between named workflows and dynamic policies โ€” both arrive as events and are routed to the Policy Engine. The distinction is in *how they are declared*:
+
+**Named Workflow Artifacts** (Orchestration Flow Policies with `ordered: true`) declare an explicit step sequence. An operator reading the workflow can see every step in order. Steps reference payload types from the closed vocabulary. Named workflows are the *explicit, visible skeleton* of a process.
+
+**Dynamic Policies** (GateKeeper, Transformation, Recovery) fire when their match conditions are satisfied, regardless of workflow position. They are not declared in the workflow artifact. They are the *conditional behavior* that fills in the skeleton.
+
+**Example โ€” request lifecycle:**
+```
+Named workflow "system/workflows/request-lifecycle" declares:
+  Step 1: request.initiated    โ†’ capture intent
+  Step 2: request.intent_captured โ†’ run layer assembly
+  Step 3: request.layers_assembled โ†’ run placement
+  Step 4: request.placement_complete โ†’ dispatch
+
+Dynamic policies also fire:
+  GateKeeper "vm-size-limits" fires on request.layers_assembled
+    if cpu_count > 32 โ†’ deny
+  Transformation "inject-monitoring" fires on request.layers_assembled
+    โ†’ adds monitoring_endpoint field
+  Recovery "notify-on-timeout" fires on recovery.timeout_fired
+    โ†’ NOTIFY_AND_WAIT action
+```
+
+The named workflow and the dynamic policies are independent artifacts. Adding a new GateKeeper does not modify the workflow. Modifying the workflow does not affect dynamic policies. They compose through the same Policy Engine evaluation on the same events.
+
+### 2.6 Request Orchestrator Responsibilities
+
+| Responsibility | Description |
+|----------------|-------------|
+| Event routing | Receive all request lifecycle events; route to appropriate components |
+| Pipeline coordination | Sequence component interactions per data dependencies |
+| Timeout monitoring | Track dispatch_timeout and assembly_timeout; fire recovery triggers |
+| Dependency resolution | For compound services, sequence component provisioning per dependency graph |
+| Status tracking | Maintain current status of all in-flight requests; respond to status queries |
+| Recovery coordination | On timeout/failure, invoke Recovery Policy evaluation |
+
+---
+
+## 3. The Cost Analysis Component
+
+### 3.1 Role
+
+The Cost Analysis Component is an **internal DCM control plane component** that provides cost signals to other components. It is not a billing system and not a provider type. It does not manage financial transactions, produce invoices, or serve as the authoritative financial record. It provides cost *signals* that DCM uses for placement decisions, pre-request estimation, and ongoing attribution.
+
+The authoritative billing record lives in the organization's financial system. A billing system can register as an Information Provider to push authoritative cost data back into DCM for attribution records.
+
+### 3.2 Three Cost Functions
+
+**Function 1 โ€” Pre-request cost estimation:**
+Given a catalog item and assembled field values, compute the estimated lifecycle cost. Used by:
+- Service Catalog describe endpoint (consumer sees cost before requesting)
+- CI pipeline pre-validation (cost estimate in PR comment)
+- Placement engine tie-breaker step 4 (cheapest eligible provider)
+
+**Function 2 โ€” Placement cost input:**
+During Step 6 placement, provide current cost data per eligible provider for the requested resource type. If Cost Analysis data is unavailable, the placement engine falls back to static declared costs per REG-011.
+
+**Function 3 โ€” Ongoing cost attribution:**
+For realized entities, track ongoing consumption and attribute costs to the owning Tenant. Consumed by OBS-005 (consumer cost view) and the resource describe endpoint (`estimated_cost_per_hour` field).
+
+### 3.3 Cost Data Sources
+
+The Cost Analysis Component ingests cost data from two sources, following the REG-011 hybrid model:
+
+```yaml
+cost_data_sources:
+  static:
+    source: provider_registration     # declared at provider registration time
+    update_frequency: manual          # updated when rates change
+    fields: [capex_per_unit, opex_per_unit_per_hour, currency]
+
+  dynamic:
+    source: external_cost_api         # external billing API or cloud pricing API
+    registered_as: information_provider
+    query_interval: PT1H
+    fallback: static                  # use static if dynamic unavailable
+    fallback_max_age: PT24H
+```
+
+### 3.4 Cost Estimation Model
+
+```yaml
+cost_estimation_request:
+  catalog_item_uuid: 
+  assembled_fields:
+    cpu_count: 4
+    memory_gb: 8
+    storage_gb: 100
+  tenant_uuid: 
+  requested_duration: P30D           # optional; lifecycle estimate
+
+cost_estimation_response:
+  estimated_cost:
+    per_hour: 0.32
+    per_month: 230.40
+    lifecycle_estimate: 691.20       # if requested_duration provided
+    currency: USD
+    confidence: high                 # high: current Cost Analysis data
+                                     # medium: data > PT1H old
+                                     # low: static fallback
+    breakdown:
+      - component: compute
+        per_hour: 0.28
+      - component: ip_allocation
+        per_hour: 0.04
+  cost_data_timestamp: 
+```
+
+### 3.5 Cost Attribution for Realized Entities
+
+```yaml
+entity_cost_attribution:
+  entity_uuid: 
+  tenant_uuid: 
+  billing_state: billable            # billable | non_billable | reduced_rate
+  current_rate:
+    per_hour: 0.32
+    currency: USD
+    rate_effective_since: 
+  monthly_accrual: 230.40
+  cost_data_source: cost_analysis    # cost_analysis | static | unknown
+```
+
+### 3.6 Integration with Placement Engine
+
+The placement engine queries Cost Analysis at step 4 of the tie-breaking hierarchy:
+
+```
+Step 4 โ€” Cost Analysis (if available and determinable):
+  Query Cost Analysis for each eligible provider
+  Cost Analysis returns: estimated cost per unit per provider
+  Placement engine prefers lowest cost among equally-ranked candidates
+  If Cost Analysis unavailable: skip step 4; proceed to step 5
+  # Cost Analysis unavailability never blocks placement
+```
+
+---
+
+## 4. Related Policies
+
+| Policy | Rule |
+|--------|------|
+| `CTL-001` | The Request Orchestrator is the single event bus for all request lifecycle events. No component communicates directly with another component outside of events published to the Request Orchestrator. |
+| `CTL-002` | Policies ARE the orchestration. The Request Orchestrator does not contain hardcoded pipeline logic. Pipeline behavior is modified by adding, removing, or changing policies โ€” not by changing the orchestrator. |
+| `CTL-003` | Dynamic and static flows compose naturally. Static flows are Policy Groups with concern_type: orchestration_flow and ordered: true. Both types are evaluated by the same Policy Engine. |
+| `CTL-004` | Cost Analysis is not a billing system. It provides cost signals for placement and attribution. The authoritative billing record lives in the organization's financial system, which may register as an Information Provider. |
+| `CTL-005` | Cost Analysis unavailability never blocks placement. The placement engine falls back to static declared costs per REG-011 and skips the Cost Analysis tie-breaking step. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+---
+
+## 4. The Placement Engine
+
+### 4.1 Role
+
+The Placement Engine selects the specific Service Provider that will fulfill a resource request. It runs as step 6 of the Request Payload Processor assembly pipeline and is invoked by the Request Orchestrator after all policies have been evaluated and the assembled payload is ready for dispatch.
+
+The Placement Engine does not make business decisions โ€” those are made by policies (which inject constraints and preferences). The Placement Engine applies those constraints to find eligible providers, then deterministically resolves ties using a declared hierarchy.
+
+### 4.2 Input and Output
+
+**Input:**
+- Assembled payload with full field-level provenance
+- Sovereignty constraints (from compliance domain profile and any policy-injected constraints)
+- Accreditation requirements (from Data/Capability Authorization Matrix)
+- Preference scores or preferred provider UUIDs (if injected by Transformation policy)
+- Tenant affinity declarations
+
+**Output:**
+- Selected provider UUID
+- Placement reason (why this provider was selected)
+- Sovereignty satisfaction record (which constraints were checked and passed)
+- Reserve confirmation (provider confirmed it has capacity for this specific request)
+
+This output is written to `placement.yaml` in the Requested Store directory.
+
+### 4.3 Placement Algorithm โ€” Six Steps
+
+```
+Step 1: Sovereignty Pre-Filter
+  Eliminate any provider whose sovereignty_declaration does not satisfy
+  the request's sovereignty constraints.
+  โ†’ Providers that fail this step are never contacted.
+  โ†’ If zero providers remain: RESERVE_QUERY_ALL_EXHAUSTED recovery trigger fires.
+
+Step 2: Accreditation Filter
+  Eliminate any provider that does not hold the required accreditations
+  for the data classifications present in the assembled payload.
+  โ†’ Checked against the active Data/Capability Authorization Matrix.
+  โ†’ Providers with accreditation gaps are excluded.
+
+Step 3: Capability Filter
+  Eliminate any provider that does not declare support for the
+  requested resource type and all required capabilities.
+  โ†’ Based on provider registration catalog item declarations.
+
+Step 4: Reserve Query
+  Send a reserve query to each remaining candidate provider in parallel.
+  โ†’ Providers have reserve_query_timeout to respond (profile-governed: PT5โ€“30S).
+  โ†’ Providers that do not respond within timeout: excluded from this placement cycle.
+  โ†’ Providers that respond with INSUFFICIENT_CAPACITY: excluded; DCM updates
+    internal capacity rating for that provider.
+  โ†’ Providers that confirm capacity: advance to tie-breaking.
+
+Step 5: Tie-Breaking (deterministic hierarchy)
+  Applied when multiple providers confirmed capacity in Step 4:
+
+  Priority 1: Policy preference
+    A Transformation policy injected a preference_score or preferred_provider_uuid.
+    Highest preference_score wins. preferred_provider_uuid is absolute โ€” skips Steps 2-6.
+
+  Priority 2: Provider declared priority
+    Providers declare a numeric priority at registration (default: 50).
+    Higher value = preferred when all else equal.
+
+  Priority 3: Tenant affinity
+    Tenant's Policy Group declares preferred providers for specific resource types.
+    Affinity preference is a soft preference โ€” does not override accreditation or sovereignty.
+
+  Priority 4: Cost Analysis
+    Cost Analysis component provides current cost per unit per candidate provider.
+    Prefer lower total cost (CapEx + OpEx + licensing).
+    Skip if: Cost Analysis unavailable, data stale > PT1H, or cost difference < 5%.
+
+  Priority 5: Least loaded
+    Prefer provider with lower current capacity utilization from reserve_query response.
+    Skip if: utilization difference < 10%, or utilization data not returned.
+
+  Priority 6: Consistent hash (final tiebreaker โ€” always resolves)
+    SHA-256(request_uuid + resource_type + sorted_candidate_uuids)
+    Deterministic โ€” same request always resolves to the same provider in a stable cluster.
+    Never round-robin.
+
+Step 6: Reserve Confirmation
+  Notify the selected provider that its reservation is confirmed.
+  Other providers that responded to the reserve query receive a reservation release.
+  โ†’ Prevents capacity holds from accumulating across providers for the same request.
+```
+
+### 4.4 Reserve Query Protocol
+
+```yaml
+reserve_query:
+  query_uuid:                # idempotency key
+  entity_uuid: 
+  resource_type: Compute.VirtualMachine
+  resource_type_spec_version: "2.1.0"
+  requested_fields:
+    cpu_count: 4
+    memory_gb: 8
+    storage_gb: 100
+  sovereignty_requirements:
+    data_residency: EU
+  reservation_hold_ttl: PT5M      # provider holds capacity for this duration
+                                   # released when: confirmed, rejected, or TTL expires
+```
+
+```yaml
+reserve_query_response:
+  query_uuid: 
+  provider_uuid: 
+  status: confirmed | insufficient_capacity | capability_not_supported
+  capacity_held_until:    # if confirmed
+  utilization_pct: 42               # current load; used for Step 5 tiebreaking
+  cost_per_hour: 0.32               # if Cost Analysis integration enabled
+  currency: USD
+```
+
+### 4.5 Placement Configuration
+
+```yaml
+placement_engine_config:
+  reserve_query_timeout: PT10S       # profile-governed default
+  parallel_reserve_queries: true     # always true; all candidates queried simultaneously
+  max_candidates_per_placement: 10   # cap on parallel reserve queries
+  cost_freshness_max: PT1H
+  cost_difference_threshold: 0.05   # 5% โ€” skip cost step if within this band
+  utilization_difference_threshold: 0.10  # 10% โ€” skip utilization step if within this band
+  reservation_hold_ttl: PT5M
+```
+
+### 4.6 Placement Failure and Recovery
+
+When placement cannot find an eligible provider:
+
+| Failure Reason | Recovery Trigger |
+|---------------|-----------------|
+| All providers fail sovereignty filter | `RESERVE_QUERY_ALL_EXHAUSTED` |
+| All providers fail accreditation filter | `RESERVE_QUERY_ALL_EXHAUSTED` |
+| All providers respond INSUFFICIENT_CAPACITY | `RESERVE_QUERY_ALL_EXHAUSTED` |
+| All reserve queries time out | `RESERVE_QUERY_ALL_EXHAUSTED` |
+
+The `RESERVE_QUERY_ALL_EXHAUSTED` trigger fires the active Recovery Policy. Default action per profile: standard/prod โ†’ `NOTIFY_AND_WAIT`; dev โ†’ `RETRY` with exponential backoff.
+
+### 4.7 Placement System Policies
+
+| Policy | Rule |
+|--------|------|
+| `PLC-001` | Sovereignty pre-filter runs before any provider is contacted. Providers that fail sovereignty constraints never receive reserve queries. |
+| `PLC-002` | Accreditation filter runs before reserve queries. Providers without required accreditations for the payload's data classifications are excluded. |
+| `PLC-003` | Reserve queries are sent in parallel to all eligible candidates. Sequential querying is not permitted โ€” it introduces latency and prevents fair capacity comparison. |
+| `PLC-004` | The consistent hash tiebreaker is always the final tiebreaker. It ensures deterministic provider selection for identical inputs without round-robin non-determinism. |
+| `PLC-005` | A confirmed reservation hold must be released when not used โ€” either by confirmation dispatch or by explicit release on timeout. Capacity holds must not accumulate silently. |
+| `PLC-006` | The Placement Engine never selects a provider based solely on network position or co-location. Every selection is based on declared constraints, policies, and the tie-breaking hierarchy. |
+
+---
+
+## 5. The Lifecycle Constraint Enforcer
+
+### 5.1 Role
+
+The Lifecycle Constraint Enforcer is a DCM control plane component that monitors all realized entities against their declared lifecycle constraints and fires expiry actions when constraints are reached. It is the authoritative enforcer of TTL, expiry date, and maximum execution time declarations.
+
+Lifecycle constraint enforcement is a DCM concern โ€” not a provider concern. The provider does not need to know about or implement any TTL logic.
+
+### 5.2 What It Monitors
+
+The Lifecycle Constraint Enforcer monitors three categories of constraint:
+
+**Category 1 โ€” Entity TTL:**
+Duration-based: entity expires T duration after a reference point (realization, creation, last modification).
+
+**Category 2 โ€” Entity Expiry Date:**
+Calendar-based: entity expires at an absolute timestamp.
+
+**Category 3 โ€” Process Resource Maximum Execution Time:**
+Process Resources must declare `max_execution_time`. The Enforcer monitors all executing Process Resources and fires `on_max_exceeded` when the limit is reached.
+
+### 5.3 Monitoring Loop
+
+```
+Lifecycle Constraint Enforcer runs continuously:
+
+  Every cycle (interval: PT1M for standard/prod; PT5M for minimal/dev):
+
+    Query Realized Store for entities with:
+      lifecycle_state IN [OPERATIONAL, SUSPENDED, EXECUTING]
+      AND lifecycle_constraints declared
+      AND NOT already in terminal state
+
+    For each entity:
+      Compute time_remaining = constraint_expiry - now()
+
+      If time_remaining <= warn_before_expiry:
+        If warn_not_yet_sent:
+          Emit: entity.ttl_warning notification
+          Record: WARNING_EMITTED in entity provenance
+
+      If time_remaining <= 0:
+        Execute on_expiry action (see Section 5.4)
+```
+
+### 5.4 Expiry Action Execution
+
+When a lifecycle constraint fires, the Enforcer executes the declared `on_expiry` action:
+
+| Action | Behavior |
+|--------|---------|
+| `decommission` | Submit a decommission request through the standard pipeline โ€” produces Requested State, dispatches to provider, full audit trail |
+| `suspend` | Submit a suspend request through the standard pipeline |
+| `notify` | Fire `entity.ttl_expired` notification to entity owner; no automated action |
+| `review` | Entity enters PENDING_EXPIRY_ACTION state; Platform Admin and owner notified |
+| `escalate` | Immediately escalate to Platform Admin; entity enters PENDING_EXPIRY_ACTION state |
+
+**Grace period:** Expiry actions are not immediate. The Enforcer respects the declared `grace_period` (default PT1H) โ€” the action fires `grace_period` after the constraint expires, giving human operators a window to intervene.
+
+**Action failure:** If the expiry action fails to execute (provider unreachable, dependency conflict), the entity enters `PENDING_EXPIRY_ACTION` state (LTC-005). The Enforcer retries per the active Recovery Policy. Platform Admin is notified with urgency: high.
+
+### 5.5 Expiry Audit Records
+
+Every expiry-related event produces an audit record:
+
+```yaml
+audit_record:
+  action: EXPIRY_WARNING | EXPIRY_ACTION_FIRED | EXPIRY_ACTION_FAILED |
+          PENDING_EXPIRY_ACTION_ENTERED
+  actor:
+    type: system
+    system_actor:
+      component: lifecycle_constraint_enforcer
+      trigger: ttl_reached | expires_at_reached | max_execution_time_reached
+  entity_uuid: 
+  details:
+    constraint_type: ttl | expires_at | max_execution_time
+    constraint_value: 
+    action_taken: decommission | suspend | notify | review | escalate
+    grace_period_remaining: 
+```
+
+### 5.6 Process Resource Enforcement
+
+Process Resources require `max_execution_time` (mandatory). The Enforcer monitors all EXECUTING Process Resources:
+
+```
+Process Resource enters EXECUTING state
+  โ”‚
+  โ–ผ Enforcer records: execution_started_at; computes execution_timeout_at
+  โ”‚
+  โ–ผ On every monitoring cycle:
+  โ”‚   If now() >= execution_timeout_at:
+  โ”‚     Emit: PROCESS_TIMEOUT event
+  โ”‚     Entity state โ†’ FAILED
+  โ”‚     Recovery Policy: COMPENSATION_FAILED trigger if resources were modified
+  โ”‚     Notification: entity owner + Platform Admin (urgency: high)
+```
+
+### 5.7 Lifecycle Constraint Enforcer Policies
+
+| Policy | Rule |
+|--------|------|
+| `LCE-001` | The Lifecycle Constraint Enforcer runs as a continuous monitor. It does not rely on provider callbacks or event triggers for expiry detection โ€” it polls based on declared constraints. |
+| `LCE-002` | Expiry actions are submitted through the standard DCM request pipeline. Decommission-on-expiry produces a Requested State record with `actor: system/lifecycle-constraint-enforcer`. |
+| `LCE-003` | The Enforcer respects the declared grace_period before firing expiry actions. Grace period gives human operators a window to intervene before automated action. |
+| `LCE-004` | Process Resource max_execution_time enforcement fires immediately on breach โ€” no grace period. Hung processes are failed immediately to prevent resource leaks. |
+| `LCE-005` | Expiry action failures enter PENDING_EXPIRY_ACTION state. The Enforcer retries per the active Recovery Policy. Indefinite retry without escalation is not permitted. |
+
+---
+
+## 6. The Search Index
+
+### 6.1 Role
+
+The Search Index is a **non-authoritative, queryable projection** of the GitOps stores (Intent Store and Requested Store). It enables millisecond-latency queries against stored entities without traversing Git history, while the GitOps stores remain the authoritative source of truth.
+
+The Search Index is a Storage Provider sub-type. It has its own registration, health check, and sovereignty declaration. It is never the source of truth โ€” if the Search Index and the GitOps store disagree, the GitOps store wins unconditionally.
+
+### 6.2 What It Indexes
+
+The Search Index maintains a projection of key fields from Intent State and Requested State records, enabling queries without retrieving full payloads from Git:
+
+```yaml
+search_index_record:
+  entity_uuid: 
+  entity_handle: 
+  resource_type: Compute.VirtualMachine
+  resource_type_category: Compute
+  tenant_uuid: 
+  lifecycle_state: OPERATIONAL
+  drift_status: clean
+  provider_uuid: 
+  deployment_posture: prod
+  compliance_domains: [hipaa]
+  data_classifications: [restricted]      # highest classification in entity
+  created_at: 
+  updated_at: 
+  cost_per_hour: 0.32
+  currency: USD
+  git_path: intent-store/tenant-uuid/Compute/VirtualMachine/entity-uuid/intent.yaml
+  # git_path is the pointer back to the authoritative record
+  tags: { environment: production, team: payments }
+```
+
+### 6.3 Required Query Operations
+
+| Operation | Description |
+|-----------|-------------|
+| `find_by_uuid(entity_uuid)` | Return index record for a single entity |
+| `find_by_tenant(tenant_uuid, filters)` | Return all entities for a Tenant with optional field filters |
+| `find_by_resource_type(fqn, filters)` | Return all entities of a resource type |
+| `find_by_provider(provider_uuid, filters)` | Return all entities hosted at a provider |
+| `find_by_lifecycle_state(state, tenant_uuid)` | Return entities in a given lifecycle state |
+| `find_by_drift_status(status, tenant_uuid)` | Return drifted or clean entities |
+| `find_by_data_classification(classification)` | Return entities containing data of a given classification |
+| `full_text_search(query, tenant_uuid)` | Full-text search across handle, display_name, tags |
+
+All queries return the `git_path` โ€” consumers fetch the full payload from Git if needed.
+
+### 6.4 Consistency Model
+
+The Search Index is **eventually consistent** with the GitOps stores. There is a defined maximum staleness:
+
+```yaml
+search_index_consistency:
+  max_staleness: PT5M         # index must be within 5 minutes of GitOps store
+  profile_overrides:
+    prod: PT2M
+    fsi: PT1M
+    sovereign: PT1M
+  on_staleness_exceeded:
+    action: degrade_with_warning    # serve results with staleness warning
+    alert: platform_admin           # alert on staleness exceeding 2ร— max
+  rebuild_on_recovery: true         # full index rebuild from Git history on failure
+  rebuild_max_duration: PT4H        # must complete within 4 hours for standard+
+```
+
+### 6.5 Unavailability Behavior
+
+If the Search Index is unavailable:
+- DCM degrades search operations gracefully: returns a `503 Service Degraded` response with a reference to the authoritative Git store
+- Writes are not affected โ€” GitOps stores are written directly; the index is updated asynchronously
+- On recovery: the Search Index rebuilds from Git history
+- No data is lost if the index is lost โ€” it is always reconstructable from Git
+
+### 6.6 Search Index Policies
+
+| Policy | Rule |
+|--------|------|
+| `SIX-001` | The Search Index is non-authoritative. GitOps stores win on any disagreement. Consumers must be prepared to receive a git_path and fetch from the authoritative store. |
+| `SIX-002` | The Search Index must be rebuildable from Git history at any time. Implementations that cannot perform a full index rebuild are non-conformant. |
+| `SIX-003` | Search Index staleness beyond the profile-governed maximum triggers a platform admin alert. Staleness is surfaced in query responses โ€” consumers are never served stale data silently. |
+| `SIX-004` | Search Index unavailability degrades queries without impacting writes. Write operations proceed directly to the authoritative GitOps stores regardless of Search Index availability. |
+
+---
+
+
+---
+
+## 7. The Drift Reconciliation Component
+
+### 7.1 Role
+
+The Drift Reconciliation Component compares the Discovered State of entities against their Realized State to detect, classify, and respond to drift. It is the consumer of Discovered Store data and the producer of drift records that feed into the Policy Engine for response evaluation.
+
+Drift Reconciliation is purely a read-and-compare component โ€” it never writes to the Realized Store. It reads Discovered State, reads Realized State, computes differences, classifies severity, and fires events into the Request Orchestrator. The Policy Engine and Recovery Policies determine what happens next.
+
+### 7.2 Inputs and Outputs
+
+**Inputs:**
+- Discovered State snapshots (from Discovered Store, written by Discovery Scheduler)
+- Realized State snapshots (from Realized Store)
+- Resource Type Specifications (for field criticality declarations used in severity classification)
+- Active governance profile (for magnitude thresholds used in severity classification)
+
+**Outputs:**
+- Drift records (written to Drift Record Store โ€” a lightweight operational store)
+- Drift events published to the Request Orchestrator: `drift.detected`, `drift.resolved`, `drift.severity_escalated`
+- Unsanctioned change events: `unsanctioned_change.detected`
+
+### 7.3 Comparison Algorithm
+
+```
+Discovery cycle completes โ†’ Discovered State snapshot written
+  โ”‚
+  โ–ผ Drift Reconciliation Component receives discovery.cycle_complete event
+  โ”‚
+  โ–ผ For each entity UUID in the discovery snapshot:
+  โ”‚
+  โ”‚   Load: latest Realized State snapshot for entity UUID
+  โ”‚   Load: Discovered State snapshot (just written)
+  โ”‚   Load: Resource Type Specification (field criticality per field)
+  โ”‚
+  โ–ผ Field-by-field comparison:
+  โ”‚   For each field in Realized State:
+  โ”‚     Does Discovered State contain this field?
+  โ”‚     If yes: are the values equal?
+  โ”‚     If no: field is absent โ€” severity based on field criticality
+  โ”‚   For each field in Discovered State not in Realized State:
+  โ”‚     New field appeared โ€” severity based on field criticality
+  โ”‚
+  โ–ผ Severity classification (per field):
+  โ”‚   Field criticality (from Resource Type Spec) ร— Change magnitude (profile-governed)
+  โ”‚   โ†’ severity matrix โ†’ minor | significant | critical
+  โ”‚   Unsanctioned? โ†’ elevate one level
+  โ”‚   Multiple drifted fields? โ†’ overall = highest individual severity
+  โ”‚
+  โ–ผ Unsanctioned check:
+  โ”‚   Is there a Requested State record that explains this change?
+  โ”‚   If yes: sanctioned change (may still be drift if realization didn't match)
+  โ”‚   If no: unsanctioned_change.detected event fired (in addition to drift.detected)
+  โ”‚
+  โ”œโ”€โ”€ No drift detected:
+  โ”‚     Update entity.last_discovered_at
+  โ”‚     Update entity.drift_status = clean
+  โ”‚     No drift record created
+  โ”‚
+  โ””โ”€โ”€ Drift detected:
+        Create drift record
+        Publish drift.detected to Request Orchestrator
+        Policy Engine evaluates โ†’ response action
+```
+
+### 7.4 Drift Record Structure
+
+```yaml
+drift_record:
+  uuid: 
+  entity_uuid: 
+  detected_at: 
+  discovery_snapshot_uuid:         # the Discovered State snapshot that triggered this
+  realized_state_uuid:             # the Realized State snapshot compared against
+
+  overall_severity: minor | significant | critical
+  unsanctioned: true | false             # true if no corresponding Requested State record
+
+  drifted_fields:
+    - field_path: "fields.memory_gb"
+      realized_value: 8
+      discovered_value: 16
+      field_criticality: medium           # from Resource Type Spec
+      change_magnitude: significant       # 100% increase, threshold: standard 10-50%
+      field_severity: significant
+      elevated_for_unsanctioned: true     # elevated from significant โ†’ critical
+
+  status: open | acknowledged | resolved | escalated
+  resolution:
+    resolved_at: 
+    resolution_type: reverted | updated_definition | accepted | escalated | null
+    resolved_by_requested_state_uuid: 
+```
+
+### 7.5 Drift Resolution Tracking
+
+Drift records are not resolved by the Drift Reconciliation Component โ€” they are resolved by the Policy Engine's response actions. The Drift Reconciliation Component monitors for resolution:
+
+```
+REVERT action taken:
+  New Requested State submitted โ†’ provider reverts โ†’ new Realized State written
+  Next discovery cycle: Discovered State matches new Realized State
+  Drift Reconciliation: no drift detected โ†’ drift_record.status = resolved
+  drift.resolved event published
+
+UPDATE_DEFINITION action taken:
+  Consumer submits UPDATE_DEFINITION โ†’ new Realized State written with discovered values
+  Next discovery cycle: Discovered State matches new Realized State
+  Drift record.status = resolved with resolution_type: updated_definition
+
+Entity decommissioned:
+  Drift record.status = resolved with resolution_type: decommissioned
+```
+
+### 7.6 Governance Matrix Integration
+
+Before classifying a discovered change as drift, the Drift Reconciliation Component evaluates the governance matrix to determine if the change is expected:
+
+```
+Field value in Discovered State differs from Realized State
+  โ”‚
+  โ–ผ Check: Is there a governance matrix rule that permits this provider
+  โ”‚        to make this type of change to this field?
+  โ”‚
+  โ”œโ”€โ”€ Yes โ†’ This may be a Provider Update Notification that wasn't submitted
+  โ”‚   DCM logs a warning: "Provider changed field without submitting update notification"
+  โ”‚   Still treated as drift โ€” provider should have submitted update notification
+  โ”‚
+  โ””โ”€โ”€ No โ†’ Standard drift detection; severity classification runs
+```
+
+### 7.7 Drift Reconciliation Policies
+
+| Policy | Rule |
+|--------|------|
+| `DRC-001` | The Drift Reconciliation Component never writes to the Realized Store. It produces drift records and events only. |
+| `DRC-002` | Drift detection runs after every discovery cycle. An entity with no corresponding Realized State record is an orphan candidate โ€” not a drift event. |
+| `DRC-003` | Unsanctioned changes are always elevated one severity level above the matrix classification. An unsanctioned significant drift is reported as critical. |
+| `DRC-004` | Drift records are retained until the entity is decommissioned plus the configured audit retention period. They are not deleted on resolution โ€” resolution is recorded within the record. |
+| `DRC-005` | Drift detection produces events into the Request Orchestrator. The Policy Engine determines the response action. The Drift Reconciliation Component does not initiate remediation directly. |
+
+
+## 8. Related Policies โ€” Full Component Set
+
+| Policy | Rule |
+|--------|------|
+| `CTL-001` | The Request Orchestrator is the single event bus for all request lifecycle events. No component communicates directly with another component outside of events published to the Request Orchestrator. |
+| `CTL-002` | Policies ARE the orchestration. The Request Orchestrator does not contain hardcoded pipeline logic. |
+| `CTL-003` | Dynamic and static flows compose naturally. Static flows are Policy Groups with concern_type: orchestration_flow and ordered: true. |
+| `CTL-004` | Cost Analysis is not a billing system. It provides cost signals for placement and attribution. |
+| `CTL-005` | Cost Analysis unavailability never blocks placement. |
+| `PLC-001` through `PLC-006` | Placement Engine policies (see Section 4.7) |
+| `LCE-001` through `LCE-005` | Lifecycle Constraint Enforcer policies (see Section 5.7) |
+| `SIX-001` through `SIX-004` | Search Index policies (see Section 6.6) |
+| `DRC-001` through `DRC-005` | Drift Reconciliation policies (see Section 7.7) |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/26-accreditation-and-authorization-matrix.md b/content/docs/data-model/26-accreditation-and-authorization-matrix.md
new file mode 100644
index 0000000..eb0fb8d
--- /dev/null
+++ b/content/docs/data-model/26-accreditation-and-authorization-matrix.md
@@ -0,0 +1,634 @@
+# DCM Data Model โ€” Accreditation, Data Authorization Matrix, and Zero Trust
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: DATA + POLICY**
+>
+> Data: Accreditation artifacts. Policy: Zero Trust posture as policy concern type
+
+
+**Related Documents:** [Policy Profiles](14-policy-profiles.md) | [Resource/Service Entities](06-resource-service-entities.md) | [DCM Federation](22-dcm-federation.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Operational Models](24-operational-models.md)
+
+---
+
+> **Authentication Assurance Levels:** See [Standards and Compliance Catalog](40-standards-catalog.md) Section 7 for the NIST SP 800-63B AAL mapping per profile.
+
+## 1. Purpose
+
+This document defines three interconnected models that together govern how DCM handles trust, data handling obligations, and compliance verification across all interaction boundaries:
+
+1. **Accreditation Model** โ€” how DCM records, verifies, and enforces third-party compliance certifications for providers, policy engines, and DCM deployments themselves
+2. **Data/Capability Authorization Matrix** โ€” what data and capabilities are permitted across any DCM boundary given a component's accreditation level and the data's classification
+3. **Zero Trust Interaction Model** โ€” the authentication, authorization, and verification requirements for every interaction in DCM, regardless of network position
+
+These three models compose: Zero Trust verifies identity and authorization on every call. Accreditation verifies compliance certification status. The Authorization Matrix declares what is permitted given that certification status. Together they ensure that no interaction in DCM is implicitly trusted โ€” every boundary crossing is verified against all three models.
+
+---
+
+
+## 1b. Accreditation and the Scoring Model
+
+DCM distinguishes two distinct accreditation functions:
+
+**Required Accreditation (boolean gate):** Whether a provider holds a specific accreditation required for a particular request. PHI data requires an active BAA. This is a Governance Matrix enforcement โ€” always boolean, never scored. A provider without the required accreditation is ineligible for that request regardless of any other score.
+
+**Accreditation Richness (placement score):** The breadth and depth of a provider's accreditation portfolio. A provider with ISO 27001 + SOC2 Type II + FedRAMP Moderate + HIPAA BAA is preferable for placement over one with only self-declaration, all else equal. This is a continuous scoring signal โ€” it does not gate eligibility, it influences preference among eligible providers.
+
+Accreditation richness score contributes to:
+1. Placement tie-breaking (a richer portfolio is preferred)
+2. Request risk score Signal 5 (inversely โ€” higher richness reduces provider risk contribution)
+
+See [Scoring Model](29-scoring-model.md) Section 4.5 for the richness score weights and normalization.
+
+
+## 2. Data Classification
+
+Data classification is a **first-class field-level metadata property** in the DCM data model. Every field in every payload carries a `data_classification` value. This classification is the primary axis of the authorization matrix and is the key input to sovereignty and compliance enforcement.
+
+### 2.1 Classification Levels
+
+| Level | Description | Examples |
+|-------|-------------|---------|
+| `public` | No restrictions; freely shareable | Resource display names, catalog item descriptions |
+| `internal` | Organization-internal; not for external disclosure | Configuration details, operational metadata |
+| `confidential` | Sensitive business data; restricted access | Cost data, business unit assignments |
+| `restricted` | Highly sensitive; regulated or contractually protected | Security group IDs, network topology details |
+| `phi` | Protected Health Information under HIPAA/HITECH | Patient IDs, diagnosis codes, treatment plans |
+| `pci` | Payment Card Industry data under PCI-DSS | Cardholder data, authentication data |
+| `sovereign` | Nationally classified or sovereignty-restricted data | Data subject to national security law |
+| `classified` | Government-classified information | Classified defense or intelligence data |
+
+### 2.2 Classification as Field Metadata
+
+Every field in a DCM payload carries data classification as part of its field metadata:
+
+```yaml
+field_definition:
+  field_name: patient_record_id
+  value: "PAT-00421"
+  data_classification: phi
+  classification_basis: "Contains patient identifier โ€” HIPAA 45 CFR 164.514"
+  metadata:
+    override: immutable           # classification cannot be changed by policy
+    locked_by: system/compliance/hipaa-field-classifier
+```
+
+**Classification is declared in three places:**
+- **Resource Type Specification** โ€” default classification per field for all instances of that type
+- **Data Layer** โ€” classification applied across a domain (e.g., an org layer that marks all cost_center fields as `confidential`)
+- **Field-level override** โ€” explicit classification on a specific field instance (highest precedence, immutable once set for `phi`, `sovereign`, `classified`)
+
+### 2.3 Classification Immutability
+
+Fields classified as `phi`, `sovereign`, or `classified` cannot be downgraded by any layer or policy โ€” their classification is immutable once set. A GateKeeper policy attempting to downgrade a PHI field is rejected with a classification violation audit record.
+
+---
+
+## 3. Accreditation Model
+
+### 3.1 What Accreditation Is
+
+An **Accreditation** is a formal, versioned, time-bounded attestation that a DCM component โ€” a Service Provider, a Policy Provider, a Storage Provider, a Notification Provider, or a DCM deployment itself โ€” satisfies the requirements of a specific compliance framework. Accreditations are issued by an **Accreditor** and registered with DCM as first-class artifacts.
+
+Accreditation answers: **"Is this component certified to handle this type of data?"**
+
+### 3.2 Accreditation Types and Trust Levels
+
+| Type | Issued By | Trust Level | Examples |
+|------|-----------|-------------|---------|
+| `self_declared` | Component itself | Lowest | Dev/homelab; provider asserts own compliance |
+| `first_party` | DCM organization's own audit team | Low-Medium | Internal compliance review |
+| `third_party` | Independent certifying body | High | ISO 27001, SOC 2 Type II |
+| `qsa_assessment` | Qualified Security Assessor | High | PCI-DSS QSA report |
+| `baa` | Legal BAA with covered entity | High | HIPAA Business Associate Agreement |
+| `regulatory_certification` | Government regulatory body | Highest | FedRAMP P-ATO, DoD Provisional Authorization |
+| `sovereign_authorization` | National sovereignty authority | Highest | National cloud authorization |
+
+### 3.3 Accreditation Record Structure
+
+```yaml
+accreditation:
+  # Standard artifact metadata
+  artifact_metadata:
+    uuid: 
+    handle: "accreditations/providers/eu-west-prod-1/fedramp-high"
+    version: "1.0.0"
+    status: active
+    owned_by: { display_name: "Compliance Team" }
+
+  subject_uuid:           # what is being accredited
+  subject_type: service_provider | policy_provider | storage_provider |
+                notification_provider | dcm_deployment
+
+  accreditation_type: 
+  framework: fedramp_high | fedramp_moderate | hipaa | pci_dss_v4 |
+             iso_27001 | soc2_type2 | dod_il4 | dod_il5 | dod_il6 |
+             sovereign | classified | 
+
+  accreditor:
+    uuid: 
+    name: "DISA" | "HHS OIG" | "PCI SSC" | "BSI" | 
+    type: government | regulatory_body | qsa | certification_body | internal | self
+    contact_url: 
+
+  # Validity
+  issued_at: 
+  valid_until:             # null = perpetual until revoked
+  renewal_warning_before: P90D
+  last_verified_at:             # when DCM last confirmed still active
+
+  # What the accreditation covers
+  scope:
+    data_classifications: [phi, restricted]   # which classifications this covers
+    capabilities: [data_at_rest, data_in_transit, access_control, audit_logging]
+    geographic_scope: [US, EU-WEST]
+    exclusions: []
+
+  # Evidence
+  certificate_ref: 
+  audit_report_ref: 
+  external_registry_id: "FR2024-0042"    # e.g., FedRAMP Marketplace ID
+
+  # Status
+  status: active | suspended | revoked | expired | pending_renewal
+  revocation_reason: 
+  revoked_at: 
+```
+
+### 3.4 Accreditation Lifecycle
+
+```
+Accreditation submitted (via API or GitOps PR)
+  โ”‚
+  โ–ผ DCM validates structure and accreditor registration
+  โ”‚
+  โ–ผ status: proposed
+  โ”‚   Shadow mode: compliance policies use this accreditation in shadow evaluation
+  โ”‚   Platform admin reviews certificate_ref and audit_report_ref
+  โ”‚
+  โ–ผ Platform admin approves โ†’ status: active
+  โ”‚   Accreditation now enforced in compliance checks
+  โ”‚   All affected providers/deployments re-evaluated against new accreditation
+  โ”‚
+  โ–ผ Expiry monitoring:
+  โ”‚   At valid_until - renewal_warning_before:
+  โ”‚     notification.accreditation_expiring โ†’ Compliance Team, Platform Admin
+  โ”‚   At valid_until:
+  โ”‚     status โ†’ expired
+  โ”‚     Providers relying on this accreditation flagged: ACCREDITATION_GAP
+  โ”‚
+  โ–ผ Revocation:
+      Accreditor or Platform Admin revokes
+      status โ†’ revoked
+      All active provider interactions using this accreditation suspended
+      notification.accreditation_revoked โ†’ Platform Admin (urgency: critical)
+```
+
+### 3.5 Accreditation Gap
+
+When a required accreditation is missing, expired, or revoked, DCM enters an **Accreditation Gap** state for the affected provider:
+
+```yaml
+accreditation_gap_record:
+  uuid: 
+  provider_uuid: 
+  required_framework: hipaa
+  required_for: [phi data fields in active requests]
+  gap_type: missing | expired | revoked | suspended
+  detected_at: 
+  severity: critical                    # accreditation gaps are always high or critical
+  affected_entity_uuids: [, ...]  # entities currently hosted at this provider
+  policy_response: 
+  # Default: NOTIFY_AND_WAIT for fsi/sovereign; ESCALATE for standard/prod
+```
+
+### 3.6 DCM Deployment Accreditation
+
+DCM deployments themselves can carry accreditations โ€” a FedRAMP-authorized DCM deployment, for example. This enables cross-organization trust: a consuming organization's DCM can verify the providing organization's DCM deployment holds the required accreditation before federating with it.
+
+```yaml
+deployment_accreditation:
+  subject_type: dcm_deployment
+  subject_uuid: 
+  framework: fedramp_high
+  # The DCM deployment itself is accredited, not just the providers it manages
+```
+
+---
+
+
+> **Architecture Update:** Section 4 of this document (Data/Capability Authorization Matrix) has been superseded by the **Unified Governance Matrix** ([doc 27](27-governance-matrix.md)). The governance matrix provides a more powerful, unified model that replaces the standalone matrix described here. The accreditation model (Sections 2-3) and zero trust interaction model (Section 5) remain current and are consumed by the governance matrix as inputs.
+>
+> New implementations should reference doc 27 for data and capability boundary enforcement.
+
+## 4. Data/Capability Authorization Matrix
+
+### 4.1 Purpose
+
+The Data/Capability Authorization Matrix declares what data fields and provider capabilities are permitted across any DCM interaction boundary given the data's classification and the receiving component's accreditation level. It is the enforcement model that sits between compliance domain policies and the actual provider interaction.
+
+### 4.2 Matrix as a Policy Artifact
+
+The authorization matrix is a **Policy Group artifact** with `concern_type: data_authorization_boundary`. It is activated as part of the compliance domain group โ€” enabling the HIPAA compliance domain automatically activates the HIPAA boundary matrix. Organizations extend or restrict matrices via their own policy groups at the Tenant level.
+
+```yaml
+data_authorization_matrix:
+  artifact_metadata:
+    uuid: 
+    handle: "system/matrix/hipaa-provider-boundary"
+    version: "1.0.0"
+    status: active
+  
+  concern_type: data_authorization_boundary
+  applicable_compliance_domains: [hipaa]
+
+  # OUTBOUND: what DCM may send to a provider
+  outbound_data_permissions:
+    - data_classification: phi
+      required_accreditation_type: baa
+      required_accreditation_framework: hipaa
+      on_missing_accreditation: DENY_REQUEST
+      # DENY_REQUEST: block the entire request (PHI is required; cannot strip)
+      # STRIP_FIELD: remove field and proceed (for optional PHI fields)
+      # WARN_AND_ALLOW: allow but audit (dev profile only)
+
+    - data_classification: restricted
+      required_accreditation_type: third_party
+      on_missing_accreditation: STRIP_FIELD
+
+    - data_classification: internal
+      required_accreditation_type: self_declared
+      on_missing_accreditation: WARN_AND_ALLOW   # always has self_declared minimum
+
+    - data_classification: [public, internal]
+      required_accreditation_type: self_declared
+      on_missing_accreditation: ALLOW
+
+  # CAPABILITY: what operations the provider may perform on classified data
+  capability_permissions:
+    - capability: STORE_AT_REST
+      data_classification: phi
+      required_accreditation_type: baa
+      required_scope: [data_at_rest]
+      on_missing_accreditation: DENY_CAPABILITY
+
+    - capability: REPLICATE_CROSS_REGION
+      data_classification: phi
+      required_accreditation_type: baa
+      additional_requirement: replication_target_has_baa
+      on_missing_accreditation: DENY_CAPABILITY
+
+    - capability: EXPORT_TO_EXTERNAL_SYSTEM
+      data_classification: [phi, restricted, sovereign]
+      required_accreditation_type: regulatory_certification
+      on_missing_accreditation: DENY_CAPABILITY
+
+    - capability: PROVIDER_UPDATE_NOTIFICATION
+      data_classification: phi
+      required_accreditation_type: baa
+      # Provider may only notify DCM of changes to PHI-containing resources
+      # if it holds a valid BAA
+      on_missing_accreditation: DENY_CAPABILITY
+
+  # INBOUND: what the provider may return to DCM
+  inbound_data_permissions:
+    - data_classification: phi
+      provider_must_strip_before_return: false
+      # DCM receives PHI in Realized State but access-controls it
+      consumer_visibility_requires_accreditation: baa
+      stored_in_partition: realized_store_phi
+      # PHI partition has additional encryption and access control
+```
+
+### 4.3 Federation Boundary Matrix
+
+A dedicated matrix governs what crosses DCM-to-DCM federation boundaries:
+
+```yaml
+federation_boundary_matrix:
+  artifact_metadata:
+    handle: "system/matrix/federation-boundary"
+  concern_type: data_authorization_boundary
+  applicable_to: federation_tunnel
+
+  outbound_data_permissions:
+    - data_classification: sovereign
+      on_missing_accreditation: DENY_REQUEST
+      # Sovereign data NEVER crosses a federation boundary
+      # This is a hard system constraint, not a configurable policy
+      hard_constraint: true
+
+    - data_classification: classified
+      on_missing_accreditation: DENY_REQUEST
+      hard_constraint: true
+
+    - data_classification: phi
+      required_accreditation_type: baa
+      on_missing_accreditation: DENY_REQUEST
+
+    - data_classification: restricted
+      required_accreditation_type: third_party
+      additional_requirement: remote_dcm_holds_equivalent_accreditation
+      on_missing_accreditation: STRIP_FIELD
+
+    - data_classification: [public, internal]
+      required_accreditation_type: self_declared
+      on_missing_accreditation: ALLOW
+```
+
+### 4.4 Matrix Enforcement Pipeline
+
+The authorization matrix check is a distinct pipeline step executed at every interaction boundary:
+
+```
+Outbound interaction assembled (DCM โ†’ Provider OR DCM โ†’ DCM)
+  โ”‚
+  โ–ผ Data Classification Inventory:
+  โ”‚   For every field in the payload:
+  โ”‚     Resolve data_classification (field metadata โ†’ layer โ†’ resource type spec default)
+  โ”‚     Record classification โ†’ field mapping
+  โ”‚
+  โ–ผ Accreditation Resolution:
+  โ”‚   Load active accreditations for the target component
+  โ”‚   For each required classification level in the payload:
+  โ”‚     Does the target hold an active, in-scope accreditation?
+  โ”‚     Is the accreditation within its valid_until date?
+  โ”‚
+  โ–ผ Matrix Evaluation (per field):
+  โ”‚   Look up data_classification ร— accreditation_level in active matrix
+  โ”‚   Determine: ALLOW | STRIP_FIELD | DENY_REQUEST | DENY_CAPABILITY | WARN_AND_ALLOW
+  โ”‚
+  โ”œโ”€โ”€ All ALLOW โ†’ proceed
+  โ”‚
+  โ”œโ”€โ”€ STRIP_FIELD โ†’ remove field from payload; write FIELD_STRIPPED audit record
+  โ”‚     If stripped field is required for service โ†’ escalate to DENY_REQUEST
+  โ”‚
+  โ”œโ”€โ”€ DENY_REQUEST โ†’ block interaction; entity enters PENDING_REVIEW
+  โ”‚     notification.accreditation_gap dispatched to owner + platform admin
+  โ”‚
+  โ””โ”€โ”€ WARN_AND_ALLOW โ†’ proceed but write ACCREDITATION_ADVISORY audit record
+                       (dev profile only; blocked in standard+)
+```
+
+---
+
+> **Internal component authentication:** See [Internal Component Authentication](36-internal-component-auth.md) for the complete internal auth model including component identity, Internal CA, bootstrap tokens, and ICOM-001โ€“ICOM-009 policies.
+
+## 5. Zero Trust Interaction Model
+
+### 5.1 Principle
+
+**Network position grants zero trust.** A component inside the DCM control plane has no more implicit trust than one outside it. Every interaction โ€” internal or external, synchronous or asynchronous โ€” is authenticated, authorized, and verified as if the caller were an untrusted external party.
+
+Zero trust in DCM is not a network topology โ€” it is a **per-interaction verification discipline** applied at every call, every event, every tunnel message.
+
+### 5.2 The Five-Check Boundary Model
+
+Every DCM interaction boundary applies five checks in sequence. All five must pass:
+
+```
+Interaction attempt
+  โ”‚
+  โ–ผ Check 1: Identity Verification
+  โ”‚   mTLS certificate verification (mutual โ€” both sides present certificates)
+  โ”‚   Certificate chain validation against registered trust anchor
+  โ”‚   Certificate not in revocation list
+  โ”‚   Hardware attestation (fsi/sovereign profiles with hardware_attested posture)
+  โ”‚   โ†’ FAIL: connection refused; IDENTITY_VERIFICATION_FAILED audit record
+  โ”‚
+  โ–ผ Check 2: Authorization Verification
+  โ”‚   Does this identity have explicit permission for this operation type?
+  โ”‚   Is the presented credential scoped to this operation?
+  โ”‚   Has this credential been revoked or expired?
+  โ”‚   Does the scope match the minimum necessary for this call?
+  โ”‚   โ†’ FAIL: 403 Forbidden; AUTHORIZATION_DENIED audit record
+  โ”‚
+  โ–ผ Check 3: Accreditation Check
+  โ”‚   Does the target hold the required accreditation for the data classifications present?
+  โ”‚   Is the accreditation current and not suspended?
+  โ”‚   โ†’ FAIL: ACCREDITATION_GAP; recovery policy evaluates response
+  โ”‚
+  โ–ผ Check 4: Data/Capability Matrix Check
+  โ”‚   Is each field permitted to cross this boundary?
+  โ”‚   Is each capability permitted for this data classification?
+  โ”‚   โ†’ FAIL: FIELD_STRIPPED or DENY_REQUEST per matrix declaration
+  โ”‚
+  โ–ผ Check 5: Sovereignty Check
+  โ”‚   Is the target endpoint within the sovereignty boundary?
+  โ”‚   Does the interaction violate any sovereignty constraints?
+  โ”‚   BBQ-001 evaluation for Mode 4 endpoints
+  โ”‚   โ†’ FAIL: SOVEREIGNTY_VIOLATION; platform admin notified
+  โ”‚
+  โ–ผ All checks pass โ†’ interaction proceeds
+  โ”‚
+  โ””โ”€โ”€ Audit record written regardless of outcome:
+        INTERACTION_AUTHORIZED or INTERACTION_DENIED_{CHECK}
+        All five check results recorded
+        Credential UUID, interaction UUID for correlation
+```
+
+### 5.3 Credential Model โ€” Scoped, Short-Lived, Non-Transferable
+
+Zero trust requires that credentials are scoped to the minimum necessary operation and expire quickly:
+
+```yaml
+dcm_interaction_credential:
+  credential_uuid: 
+  issued_to: 
+  issued_at: 
+  valid_until:              # short-lived; typically PT15M to PT1H
+  operation_scope:
+    operation_type: dispatch | discovery | cancel | query | notify
+    entity_uuid:                # scoped to specific entity
+    provider_uuid:              # scoped to specific provider
+  non_transferable: true              # cannot be delegated or relayed
+  bound_to_ip:               # optional IP binding for fsi/sovereign
+```
+
+**Credential lifetimes by profile:**
+
+| Profile | Max credential lifetime | Renewal model |
+|---------|------------------------|---------------|
+| minimal | PT8H | Manual or long-lived |
+| dev | PT4H | Automatic refresh |
+| standard | PT1H | Automatic refresh |
+| prod | PT30M | Automatic refresh |
+| fsi | PT15M | Automatic refresh; dual approval for elevation |
+| sovereign | PT15M + hardware attestation | Hardware-bound; HSM-required |
+
+### 5.4 Zero Trust Posture as a Policy Group Concern Type
+
+`zero_trust_posture` is the sixth Policy Group concern type. Four posture levels:
+
+| Posture | Description | Profile Default |
+|---------|-------------|----------------|
+| `none` | No zero trust enforcement; perimeter model acceptable | minimal |
+| `boundary` | Zero trust at external boundaries (consumerโ†’DCM, DCMโ†’provider); internal components trust service mesh | dev, standard |
+| `full` | Zero trust everywhere including internal component communication; every call authenticated and authorized | prod, fsi |
+| `hardware_attested` | Full zero trust plus hardware attestation (TPM/HSM); component identity backed by hardware | sovereign |
+
+```yaml
+zero_trust_policy_group:
+  handle: "system/group/zt-full"
+  concern_type: zero_trust_posture
+  posture: full
+  policies:
+    - all_component_communication: mtls_required
+    - credential_lifetime: PT30M
+    - revocation_check: every_call          # not just at credential issuance
+    - session_continuation: re_verify_PT15M # re-verify identity during long operations
+    - failed_verification_response: terminate_and_alert
+```
+
+---
+
+## 6. Federation Zero Trust โ€” The Tunnel Model
+
+### 6.1 Federation Tunnel as a Zero Trust Boundary
+
+A federation tunnel between DCM instances is a **mutually authenticated, encrypted, scoped channel** where both sides verify each other on every interaction. It is not a VPN โ€” it does not establish perimeter trust. Every message crossing the tunnel is authenticated, authorized, and subject to the five-check model.
+
+**"Zero trust to any outside DCM/provider"** is implemented by: the remote DCM instance has no implicit access to local resources. Every cross-instance operation requires a scoped federation credential. The tunnel establishes secure transport โ€” it does not establish trust.
+
+### 6.2 Federation Tunnel Structure
+
+```yaml
+federation_tunnel:
+  uuid: 
+  local_dcm_uuid: 
+  remote_dcm_uuid: 
+  tunnel_type: peer | parent_child | hub_spoke
+  trust_model: zero_trust               # always; non-negotiable
+
+  # Mutual authentication
+  authentication:
+    protocol: mtls
+    local_certificate_ref: 
+    remote_certificate_pin:   # pinned; not just chain-valid
+    trust_anchor:                # common or cross-signed CA
+    certificate_rotation_interval: P90D
+    revocation_check: ocsp_stapling       # real-time revocation check
+
+  # Per-message signing
+  message_integrity:
+    signing_algorithm: ed25519
+    local_signing_key_ref: 
+    remote_verification_key_ref: 
+    replay_protection: true               # nonce + timestamp window PT5M
+
+  # What the remote DCM may request from this DCM (inbound)
+  inbound_authorization:
+    - operation: catalog_query
+      permitted_resource_types: [Compute.VirtualMachine, Network.VLAN]
+      requires_cross_tenant_authorization: true
+    - operation: allocation_request
+      permitted_resource_types: [Network.IPAddress]
+      max_allocations_per_request: 10
+      requires_cross_tenant_authorization: true
+
+  # What this DCM may request from the remote (outbound)
+  outbound_authorization:
+    - operation: placement_query
+      permitted_resource_types: [Compute.VirtualMachine]
+    - operation: realized_state_query
+      permitted_entity_uuids: []   # scoped to specific entities
+
+  # Data classification boundary (hard constraints)
+  data_boundary:
+    max_outbound_classification: restricted  # never send sovereign/classified
+    max_inbound_classification: restricted
+    # sovereign profile: max_*_classification: internal
+    # classified profile: no federation permitted
+
+  # Sovereignty scope
+  sovereignty_scope:
+    local_jurisdiction: EU
+    remote_jurisdiction: EU
+    cross_jurisdiction_permitted: false   # fsi/sovereign: always false
+```
+
+### 6.3 Federation Credential Scoping
+
+Federation credentials are scoped to the specific operations declared in the tunnel authorization. A federation credential issued for `catalog_query` cannot be used for `allocation_request`:
+
+```yaml
+federation_credential:
+  credential_uuid: 
+  issued_by_dcm_uuid: 
+  issued_to_dcm_uuid: 
+  valid_until:              # PT15M for fsi/sovereign
+  operation_scope: catalog_query
+  scoped_resource_types: [Compute.VirtualMachine]
+  non_transferable: true
+  tunnel_uuid:                  # bound to specific tunnel
+```
+
+### 6.4 Zero Trust in Hub-Spoke Federation
+
+In hub-spoke federation, the Hub DCM coordinates Regional DCMs. Zero trust means:
+- The Hub DCM does not have root-level access to Regional DCMs โ€” it has explicitly scoped federation credentials
+- A Regional DCM cannot impersonate the Hub DCM to another Regional DCM
+- Cross-Regional-DCM operations route through the Hub with the Hub's authorization, not the originating Regional DCM's authorization
+- The Hub DCM's accreditation is visible to Regional DCMs โ€” Regional DCMs can verify the Hub before accepting federation messages
+
+```
+RegionalDCM-A โ†’ HubDCM:  authenticated; scoped to allocation_request
+HubDCM โ†’ RegionalDCM-B:  authenticated; scoped to realization_request
+                          Hub presents its own credential to RegionalDCM-B
+                          Not RegionalDCM-A's credential
+RegionalDCM-B verifies:  Hub certificate; Hub accreditation; data classification boundary
+```
+
+---
+
+## 7. Profile-Governed Zero Trust Enforcement
+
+Zero trust enforcement levels are bound to deployment profiles. The profile determines which zero trust posture group is active:
+
+| Profile | Zero Trust Posture | Data Boundary | Federation |
+|---------|-------------------|---------------|-----------|
+| `minimal` | none | public/internal only | Not recommended |
+| `dev` | boundary | up to confidential | Permitted with warnings |
+| `standard` | boundary | up to restricted (with third-party accreditation) | Permitted |
+| `prod` | full | up to restricted | Permitted with accreditation |
+| `fsi` | full | up to restricted (with regulatory cert) | Restricted to same jurisdiction |
+| `sovereign` | hardware_attested | sovereign stays sovereign (no crossing) | Zero crossing of sovereign data |
+
+The `sovereign` profile enforces the hardest constraint: **sovereign-classified data never crosses any boundary** โ€” not to providers, not to federation tunnels, not to Notification Providers with external endpoints. The enforcement is at the Data/Capability Matrix level as a `hard_constraint: true` rule that cannot be overridden by any policy.
+
+---
+
+## 8. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ZT-001` | Network position grants zero trust. Every interaction is subject to the five-check model regardless of the caller's network location. |
+| `ZT-002` | All DCM interaction credentials are scoped, short-lived, and non-transferable. Credential lifetime is profile-governed. |
+| `ZT-003` | Data classified as `sovereign` or `classified` never crosses any DCM interaction boundary (provider dispatch, federation tunnel, notification delivery). This is a hard constraint enforced by the Data/Capability Matrix, not a configurable policy. |
+| `ZT-004` | Federation tunnels use mutual TLS with certificate pinning and per-message signing. A tunnel establishes secure transport, not implicit trust. |
+| `ZT-005` | Every interaction boundary check produces an audit record regardless of outcome. A denied interaction is audited as rigorously as a permitted one. |
+| `ACC-001` | Accreditations are first-class DCM artifacts. They follow the standard lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired) and are subject to GitOps governance. |
+| `ACC-002` | Accreditation gaps (missing, expired, or revoked accreditations required for active interactions) are always high or critical severity. The Recovery Policy governs the response. |
+| `ACC-003` | PHI, sovereign, and classified field classifications are immutable once set. No policy may downgrade these classifications. |
+| `ACC-004` | The Data/Capability Authorization Matrix is enforced at every outbound interaction boundary before dispatch. Fields failing the matrix check are stripped (STRIP_FIELD) or the request is blocked (DENY_REQUEST) per the matrix declaration. |
+| `ACC-005` | DCM deployments themselves carry accreditations. A federation peer DCM can verify the remote DCM deployment's accreditation before accepting federation messages. |
+| `ACC-006` | `zero_trust_posture` is the sixth Policy Group concern type. Profile defaults are: minimal=none, dev/standard=boundary, prod/fsi=full, sovereign=hardware_attested. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/27-governance-matrix.md b/content/docs/data-model/27-governance-matrix.md
new file mode 100644
index 0000000..0ae06c7
--- /dev/null
+++ b/content/docs/data-model/27-governance-matrix.md
@@ -0,0 +1,840 @@
+# DCM Data Model โ€” Unified Governance Matrix
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+
+> **Foundation Document Reference**
+>
+> This document is a detailed reference for a specific domain of the DCM architecture.
+> The three foundational abstractions โ€” Data, Provider, and Policy โ€” are defined in
+> [00-foundations.md](00-foundations.md). All concepts in this document map to one or
+> more of those three abstractions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+>
+> **This document maps to: POLICY**
+>
+> The Policy abstraction โ€” Governance Matrix Rule output schema for boundary control
+
+
+**Related Documents:** [Federated Contribution Model](28-federated-contribution-model.md) | [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) | [DCM Federation](22-dcm-federation.md) | [Policy Profiles](14-policy-profiles.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Control Plane Components](25-control-plane-components.md)
+
+---
+
+> **Federated Contribution:** The Governance Matrix enforces contributor permission boundaries at artifact submission time. See the [Federated Contribution Model](28-federated-contribution-model.md) for the complete contributor permission table and the hard DENY rules applied to out-of-scope contributions.
+
+## 1. Purpose
+
+The Unified Governance Matrix is the **single, declarative, multi-dimensional control surface** that governs every cross-boundary interaction in DCM. It answers one question at every interaction point:
+
+> **Given this subject, this data, this target, and this context โ€” is this interaction permitted, and under what conditions?**
+
+Previous DCM documents established several overlapping control mechanisms: the Data/Capability Authorization Matrix (doc 26), sovereignty constraints in federation tunnels (doc 22), BBQ-001 sovereignty checks (doc 14), and profile-governed data boundaries. The Governance Matrix unifies all of these into a single model with a single evaluation algorithm and a single enforcement point.
+
+**This document supersedes** Section 4 of doc 26 (Data/Capability Authorization Matrix) for structural purposes. The accreditation model (Sections 2-3 of doc 26) and the zero trust interaction model (Section 5 of doc 26) remain current โ€” the Governance Matrix consumes them as inputs.
+
+**Key properties of the Governance Matrix:**
+
+- **Fine-grained to broad** โ€” rules can target a single field path on a specific entity, or broadly govern all data of a given classification. Both are first-class citizens of the same model.
+- **Profile-bound defaults** โ€” every deployment profile ships with sensible default rules that are immediately operative. Operators configure overrides rather than building from scratch.
+- **Hard and soft enforcement** โ€” hard rules cannot be relaxed by any downstream rule. Soft rules establish defaults that can be tightened but never relaxed.
+- **Single evaluation algorithm** โ€” every interaction boundary runs the same algorithm against the same rule set. No parallel enforcement paths.
+- **Audited always** โ€” every evaluation produces an audit record regardless of outcome.
+
+---
+
+
+## 1b. Governance Matrix and the Scoring Model
+
+The Governance Matrix is **always boolean**. This is not a design limitation โ€” it is an explicit architectural decision.
+
+Governance Matrix decisions (ALLOW, DENY, ALLOW_WITH_CONDITIONS, STRIP_FIELD, REDACT, AUDIT_ONLY) govern whether data may cross a boundary. These are regulatory and legal facts โ€” PHI either crosses a compliant boundary or it doesn't. "Mostly compliant" is not a legal defense. No Governance Matrix Rule may declare `scoring_weight` or `enforcement_class`.
+
+**The Governance Matrix fires before the Scoring Model evaluates.** If a Governance Matrix Rule produces DENY, the request is halted and no risk score is calculated. The score pipeline only runs for requests that have already passed all Governance Matrix checks.
+
+This ensures that scoring cannot be used to route around data sovereignty or regulatory boundaries. See [Scoring Model](29-scoring-model.md) Section 8 for the full pipeline sequence and SMX-004.
+
+
+## 2. The Four Matrix Axes
+
+Every governance matrix rule is expressed as a match across four axes. A rule fires when all declared axis conditions are satisfied.
+
+### 2.1 Axis 1 โ€” Subject (Who)
+
+The subject is the entity initiating or involved in the interaction.
+
+```yaml
+subject:
+  type: 
+  # Subject types:
+  # actor                โ€” human or service account making a request
+  # service_provider     โ€” Service Provider sending/receiving data
+  # dcm_peer             โ€” federated DCM instance
+  # policy_provider      โ€” Policy Provider receiving payload data for evaluation
+  # storage_provider     โ€” Storage Provider receiving/returning state data
+  # notification_provider โ€” Notification Provider receiving notification envelopes
+  # information_provider โ€” Information Provider returning external data
+  # system               โ€” DCM internal component (Request Orchestrator, etc.)
+
+  identity:
+    provider_uuid:               # specific provider instance
+    dcm_peer_uuid:               # specific federated DCM instance
+    trust_posture:   # for dcm_peer subjects
+    accreditation_level:         # accreditation type the subject holds
+    actor_role:                  # for actor subjects
+
+  tenant:
+    uuid:                        # specific Tenant
+    match: any_tenant | cross_tenant | system_tenant
+```
+
+### 2.2 Axis 2 โ€” Data (What)
+
+The data axis declares what is being accessed, sent, or operated on. This is where field-level granularity lives.
+
+```yaml
+data:
+  # Broad controls โ€” classification level
+  classification:
+    match:  | in: [] | minimum:  | maximum: 
+    # minimum: restricted means restricted and above (phi, sovereign, classified)
+    # maximum: internal means internal and below (public, internal)
+
+  # Resource-type scoping
+  resource_type:
+    match:  | category:  | any
+
+  # Fine-grained controls โ€” specific field paths
+  field_paths:
+    mode: allowlist | blocklist | any
+    # allowlist: only these fields are permitted to cross the boundary
+    # blocklist: these fields are explicitly prohibited
+    # any: no field-level restriction (default)
+    paths:
+      - "fields.patient_id"
+      - "fields.diagnosis_code"
+      - "fields.treatment_plan"
+    # Supports wildcards: "fields.phi_*" matches all fields prefixed phi_
+
+  # Capability being exercised
+  capability:
+    match:  | in: [] | any
+    # Capabilities: read | write | store | replicate | export | notify |
+    #               execute | discover | query | federate
+```
+
+### 2.3 Axis 3 โ€” Target (Where)
+
+The target is where the data is going โ€” provider, peer DCM, storage, notification endpoint.
+
+```yaml
+target:
+  type: 
+  # service_provider | dcm_peer | storage_provider | notification_provider |
+  # information_provider | policy_provider | external_endpoint
+
+  # Identity
+  provider_uuid:                   # specific provider
+  dcm_peer_uuid:                   # specific peer
+
+  # Sovereignty
+  sovereignty_zone:
+    match:  | in: [] | same_as_source | any
+    not_in: []                     # exclusion list
+
+  jurisdiction:
+    includes: []
+    excludes: []
+    intersects: []        # target jurisdiction overlaps with list
+
+  # Trust and accreditation
+  trust_posture:
+    match:  | minimum: 
+    # minimum: vouched means vouched or verified (not provisional)
+
+  accreditation_held:
+    includes: []              # target MUST hold these accreditations
+    not_includes: []          # target must NOT hold (exclusion pattern)
+    minimum_type:    # minimum trust level of accreditation
+```
+
+### 2.4 Axis 4 โ€” Context (Under What Conditions)
+
+Context captures the operational conditions at the time of the interaction.
+
+```yaml
+context:
+  # Active deployment governance
+  profile:
+    deployment_posture:  | in: []
+    compliance_domains:
+      includes: []
+      not_includes: []
+
+  # Security posture
+  zero_trust_posture:
+    minimum:                      # none | boundary | full | hardware_attested
+  tls_mutual: 
+  hardware_attestation: 
+
+  # Interaction characteristics
+  federated: 
+  cross_jurisdiction: 
+  cross_tenant: 
+
+  # Time-based conditions
+  time_of_day:                    # for regulated maintenance windows
+  request_age_max:    # reject stale requests
+```
+
+---
+
+## 3. Rule Structure
+
+### 3.1 The Governance Matrix Rule
+
+```yaml
+governance_matrix_rule:
+  # Artifact metadata (standard DCM artifact)
+  artifact_metadata:
+    uuid: 
+    handle: "system/matrix/phi-federation-boundary"
+    version: "1.0.0"
+    status: active                       # developing | proposed | active | deprecated | retired
+    owned_by: { display_name: "Platform Security" }
+    tier: system | platform | tenant | resource_type | entity
+
+  description: "PHI must not cross to federated peers without HIPAA accreditation"
+  rationale: "HIPAA 45 CFR 164.502 โ€” minimum necessary standard for PHI disclosure"
+
+  # Match conditions (all declared axes must match for rule to fire)
+  match:
+    subject: { ... }
+    data: { ... }
+    target: { ... }
+    context: { ... }
+
+  # Decision
+  decision: ALLOW | DENY | ALLOW_WITH_CONDITIONS | STRIP_FIELD | REDACT | AUDIT_ONLY
+
+  # Enforcement level
+  enforcement: hard | soft
+  # hard: cannot be relaxed by any downstream rule; ever
+  # soft: downstream rules at same or higher domain can tighten further
+
+  # Conditions that must be met for ALLOW_WITH_CONDITIONS
+  conditions:
+    - field: target.trust_posture
+      operator: minimum
+      value: verified
+    - field: context.tls_mutual
+      operator: equals
+      value: required
+    - field: target.accreditation_held
+      operator: includes
+      value: hipaa
+
+  # Field permission model (for ALLOW and ALLOW_WITH_CONDITIONS)
+  field_permissions:
+    mode: allowlist | blocklist | passthrough
+    paths: []
+    on_blocked_field: STRIP_FIELD | DENY_REQUEST | REDACT
+    # STRIP_FIELD: remove field and proceed (if field is optional)
+    # DENY_REQUEST: block entire interaction (if field is required)
+    # REDACT: replace field value with  in payload
+
+  # Audit and notification
+  audit_on: [ALLOW, DENY, STRIP_FIELD, REDACT]
+  notification_on: [DENY]
+  notification_urgency: low | medium | high | critical
+
+  # Metadata
+  applicable_profiles: [standard, prod, fsi, sovereign]    # which profiles activate this rule
+  compliance_basis: "HIPAA 45 CFR 164.502"                 # regulatory basis
+  review_required_before: "2027-01-01"                     # when rule should be reviewed
+```
+
+### 3.2 Decision Vocabulary
+
+| Decision | Meaning | Field behavior |
+|----------|---------|----------------|
+| `ALLOW` | Interaction permitted unconditionally (within field_permissions) | Fields per field_permissions |
+| `ALLOW_WITH_CONDITIONS` | Permitted only if all conditions are satisfied; DENY if conditions fail | Fields per field_permissions |
+| `DENY` | Interaction blocked; interaction does not proceed | N/A โ€” entire interaction stopped |
+| `STRIP_FIELD` | Specific fields are removed from the payload; interaction proceeds with remaining fields | Named fields stripped |
+| `REDACT` | Specific field values replaced with ``; field presence is preserved | Named fields redacted |
+| `AUDIT_ONLY` | Interaction proceeds but is flagged in the audit trail; no blocking | All fields pass |
+
+### 3.3 Hard vs Soft Enforcement
+
+**Hard enforcement (`enforcement: hard`):**
+- The rule decision cannot be relaxed by any more-specific or higher-domain rule
+- A hard DENY is absolute โ€” no Tenant-level, entity-level, or operator override can permit the interaction
+- Hard rules are reserved for: sovereign/classified data classification boundaries, regulatory hard requirements (HIPAA BAA requirement for PHI), and explicit security policies
+- Hard ALLOW is rare โ€” it means this interaction is always permitted regardless of other rules (use with extreme caution)
+
+**Soft enforcement (`enforcement: soft`):**
+- The rule establishes a default that can be tightened by more-specific downstream rules
+- A soft ALLOW can be restricted to DENY or STRIP_FIELD by a more-specific rule
+- A soft DENY cannot be relaxed to ALLOW by a downstream rule (DENY always wins at the same level)
+- Most profile-level defaults are soft โ€” they set sensible baselines that Tenants can restrict further
+
+---
+
+## 4. Evaluation Algorithm
+
+The governance matrix evaluates all matching rules and produces a single terminal decision.
+
+```
+Interaction attempt:
+  subject: { type: dcm_peer, trust_posture: verified, jurisdiction: [DE] }
+  data: { classification: phi, field_paths: [patient_id, diagnosis_code] }
+  target: { type: dcm_peer, accreditation_held: [], jurisdiction: [US] }
+  context: { federated: true, zero_trust_posture: full, tls_mutual: required }
+
+Step 1: Collect matching rules
+  Load all active governance matrix rules across all tiers
+  Evaluate match conditions for each rule against the four axes
+  Result: set of matching rules with their decisions and enforcement levels
+
+Step 2: Evaluate hard constraints first
+  For each hard DENY rule that matches:
+    โ†’ DENY immediately; record rule_uuid; no further evaluation
+  For each hard ALLOW rule that matches:
+    โ†’ Record as a hard allow candidate; still evaluate conditions
+  If hard DENY exists: terminal decision = DENY
+
+Step 3: Evaluate soft constraints by domain precedence
+  Sort matching soft rules: entity > resource_type > tenant > platform > system
+  For each precedence level, most restrictive decision wins:
+    DENY > STRIP_FIELD > REDACT > ALLOW_WITH_CONDITIONS > AUDIT_ONLY > ALLOW
+  If DENY at any level: terminal decision = DENY
+
+Step 4: Evaluate conditions for ALLOW_WITH_CONDITIONS
+  For each ALLOW_WITH_CONDITIONS rule that survived Steps 2-3:
+    Evaluate all declared conditions
+    If any condition fails: downgrade decision to DENY
+    If all conditions pass: decision remains ALLOW_WITH_CONDITIONS
+
+Step 5: Apply field permissions
+  If terminal decision is ALLOW or ALLOW_WITH_CONDITIONS:
+    Apply field_permissions from the governing rule:
+      allowlist mode: strip all fields not in the allowed list
+      blocklist mode: strip all fields in the blocked list
+      passthrough mode: all fields pass
+    For each stripped field:
+      If field is required: escalate to DENY_REQUEST
+      If field is optional: STRIP_FIELD (proceed without it)
+
+Step 6: Produce audit record
+  Record: interaction_uuid, all matching rules, terminal decision,
+          fields stripped or redacted, rule that governed the decision
+  Notification: if terminal decision is in rule's notification_on list
+
+Step 7: Enforce decision
+  ALLOW / ALLOW_WITH_CONDITIONS: interaction proceeds with permitted fields
+  DENY: interaction blocked; 403 response with governance_matrix_rule_uuid
+  STRIP_FIELD: interaction proceeds with stripped payload
+  REDACT: interaction proceeds with redacted field values
+  AUDIT_ONLY: interaction proceeds; flagged audit record written
+```
+
+---
+
+## 5. Sovereignty Zones
+
+Sovereignty zones are registered DCM artifacts that define geopolitical and regulatory boundaries. They are a first-class input to the governance matrix โ€” rules reference zones, not raw country codes.
+
+```yaml
+sovereignty_zone:
+  artifact_metadata:
+    uuid: 
+    handle: "zones/eu-west-sovereign"
+    version: "1.0.0"
+    status: active
+    tier: system | platform
+
+  display_name: "EU Western Europe Sovereign Zone"
+  description: "GDPR-covered EU member states with NIS2 alignment"
+
+  jurisdictions: [DE, FR, NL, BE, AT, CH, LU]
+  excluded_jurisdictions: []             # explicit exclusions within the zone
+
+  data_residency_guarantee: EU           # GDPR Article 44 transfer basis
+  regulatory_frameworks: [GDPR, NIS2, eIDAS]
+
+  cross_zone_permitted: false            # data does not leave this zone by default
+  inter_zone_agreements:                 # zones this zone has data transfer agreements with
+    - zone_id: eu-north-sovereign
+      agreement_basis: "EU adequacy decision"
+      permitted_classifications: [public, internal, confidential]
+      # restricted, phi, sovereign: NOT included
+
+  # What accreditation providers must hold to operate in this zone
+  required_provider_accreditation: gdpr_adequacy | third_party
+  required_provider_accreditation_minimum_type: third_party
+```
+
+---
+
+## 6. Field-Level Controls โ€” Complete Model
+
+### 6.1 Field Path Syntax
+
+Field paths use dot-notation to address specific fields within a DCM payload:
+
+```
+fields.                      # top-level field
+fields..           # nested field
+fields.phi_*                             # wildcard: all fields matching prefix
+fields.*                                 # all fields
+metadata.                    # metadata fields
+provenance.                  # provenance fields (rarely restricted)
+```
+
+### 6.2 Broad to Fine-Grained Rule Examples
+
+**Broadest โ€” classification-level block:**
+```yaml
+# Block ALL phi fields from crossing to non-HIPAA peers
+match:
+  data.classification: phi
+  target.type: dcm_peer
+  target.accreditation_held.not_includes: hipaa
+decision: DENY
+enforcement: hard
+```
+
+**Mid-level โ€” resource type + classification:**
+```yaml
+# For VM resources: restricted fields to EU zones only
+match:
+  data.classification: restricted
+  data.resource_type: Compute.VirtualMachine
+  target.sovereignty_zone.not_in: [eu-west-sovereign, eu-north-sovereign]
+decision: STRIP_FIELD
+field_permissions:
+  mode: blocklist
+  paths: ["fields.security_group_ids", "fields.network_interface_ids"]
+  on_blocked_field: STRIP_FIELD
+```
+
+**Fine-grained โ€” specific fields:**
+```yaml
+# Allow federated phi-accredited peers to receive limited PHI fields only
+match:
+  data.classification: phi
+  target.type: dcm_peer
+  target.accreditation_held.includes: hipaa
+  target.trust_posture: verified
+decision: ALLOW_WITH_CONDITIONS
+conditions:
+  - field: context.tls_mutual
+    operator: equals
+    value: required
+  - field: context.zero_trust_posture
+    operator: minimum
+    value: full
+field_permissions:
+  mode: allowlist
+  paths:
+    - "fields.resource_type"
+    - "fields.lifecycle_state"
+    - "fields.provider_entity_id"
+    # PHI-containing fields explicitly NOT in allowlist:
+    # fields.patient_id, fields.diagnosis_code, fields.treatment_plan
+    # are stripped automatically
+  on_blocked_field: STRIP_FIELD
+```
+
+**Most specific โ€” entity-level rule:**
+```yaml
+# Provider A: explicit PHI block regardless of accreditation
+match:
+  target.provider_uuid: provider-a-uuid
+  data.classification: phi
+decision: DENY
+enforcement: hard
+reason: "Provider A has unresolved data handling concerns โ€” PHI explicitly prohibited"
+```
+
+### 6.3 Field-Level Redaction vs Stripping
+
+| Operation | Effect on payload | Use case |
+|-----------|------------------|----------|
+| `STRIP_FIELD` | Field entirely removed from payload | Field is optional; receiver has no need to know it exists |
+| `REDACT` | Field present with value `` | Receiver needs to know field exists but not its value (e.g., audit evidence that a field was present) |
+| `DENY_REQUEST` | Entire interaction blocked | Field is required for the operation to make sense; stripping would produce invalid state |
+
+---
+
+## 7. Profile-Bound Default Matrix Rules
+
+Every deployment profile activates a set of default governance matrix rules. These are soft rules (tightenable by Tenant/resource-type overrides) unless marked hard.
+
+### 7.1 minimal Profile Defaults
+
+```yaml
+profile_matrix_defaults:
+  profile: minimal
+  rules:
+    - handle: "system/matrix/minimal-sovereign-hard"
+      enforcement: hard
+      match:
+        data.classification: [sovereign, classified]
+        target.type: [dcm_peer, service_provider, notification_provider]
+      decision: DENY
+      reason: "Sovereign and classified data never crosses any boundary โ€” any profile"
+
+    - handle: "system/matrix/minimal-passthrough"
+      enforcement: soft
+      match:
+        data.classification: [public, internal]
+        target.type: any
+      decision: ALLOW
+      field_permissions:
+        mode: passthrough
+```
+
+### 7.2 dev Profile Defaults
+
+```yaml
+profile_matrix_defaults:
+  profile: dev
+  inherits: minimal
+  additional_rules:
+    - handle: "system/matrix/dev-confidential-allow"
+      enforcement: soft
+      match:
+        data.classification: confidential
+        target.type: [service_provider, dcm_peer]
+        target.trust_posture: [verified, vouched, provisional]
+      decision: ALLOW_WITH_CONDITIONS
+      conditions:
+        - field: context.tls_mutual
+          operator: equals
+          value: required
+      field_permissions:
+        mode: passthrough
+      # Dev allows confidential to flow broadly; standard+ tightens this
+```
+
+### 7.3 standard Profile Defaults
+
+```yaml
+profile_matrix_defaults:
+  profile: standard
+  inherits: minimal
+  additional_rules:
+    - handle: "system/matrix/standard-restricted-accreditation"
+      enforcement: soft
+      match:
+        data.classification: restricted
+        target.type: [service_provider, dcm_peer]
+      decision: ALLOW_WITH_CONDITIONS
+      conditions:
+        - field: target.accreditation_held
+          operator: minimum_type
+          value: third_party
+        - field: context.tls_mutual
+          operator: equals
+          value: required
+      field_permissions:
+        mode: passthrough
+
+    - handle: "system/matrix/standard-phi-deny-default"
+      enforcement: soft
+      match:
+        data.classification: phi
+        target.type: [service_provider, dcm_peer]
+      decision: DENY
+      # Tenants with HIPAA compliance domain active override this with their own rules
+```
+
+### 7.4 prod Profile Defaults
+
+```yaml
+profile_matrix_defaults:
+  profile: prod
+  inherits: standard
+  additional_rules:
+    - handle: "system/matrix/prod-federation-verified-only"
+      enforcement: soft
+      match:
+        data.classification: [confidential, restricted]
+        target.type: dcm_peer
+        target.trust_posture: [vouched, provisional]
+      decision: DENY
+      # prod: only verified peers receive confidential+ data
+
+    - handle: "system/matrix/prod-notification-restricted"
+      enforcement: soft
+      match:
+        data.classification: restricted
+        target.type: notification_provider
+      decision: STRIP_FIELD
+      field_permissions:
+        mode: blocklist
+        paths: ["fields.*"]      # strip all payload fields from notifications
+        # Notification envelope metadata (entity_uuid, event_type) passes through
+        # Actual field values do not appear in notification payloads for restricted data
+```
+
+### 7.5 fsi Profile Defaults
+
+```yaml
+profile_matrix_defaults:
+  profile: fsi
+  inherits: prod
+  additional_rules:
+    - handle: "system/matrix/fsi-cross-jurisdiction-deny"
+      enforcement: hard
+      match:
+        data.classification: [restricted, phi, pci, sovereign]
+        target.type: [service_provider, dcm_peer]
+        context.cross_jurisdiction: true
+      decision: DENY
+      reason: "FSI profile: regulated data does not cross jurisdictional boundaries"
+
+    - handle: "system/matrix/fsi-phi-baa-required"
+      enforcement: hard
+      match:
+        data.classification: phi
+        target.type: [service_provider, dcm_peer]
+      decision: ALLOW_WITH_CONDITIONS
+      conditions:
+        - field: target.accreditation_held
+          operator: includes
+          value: hipaa_baa
+        - field: target.trust_posture
+          operator: minimum
+          value: verified
+        - field: context.zero_trust_posture
+          operator: minimum
+          value: full
+      field_permissions:
+        mode: passthrough          # HIPAA-accredited verified peers get full PHI scope
+        # Tenant-level rules can further restrict to specific field paths
+
+    - handle: "system/matrix/fsi-pci-qsa-required"
+      enforcement: hard
+      match:
+        data.classification: pci
+        target.type: [service_provider, dcm_peer]
+      decision: ALLOW_WITH_CONDITIONS
+      conditions:
+        - field: target.accreditation_held
+          operator: includes
+          value: pci_dss_qsa
+        - field: context.zero_trust_posture
+          operator: minimum
+          value: full
+```
+
+### 7.6 sovereign Profile Defaults
+
+```yaml
+profile_matrix_defaults:
+  profile: sovereign
+  inherits: fsi
+  additional_rules:
+    - handle: "system/matrix/sovereign-no-federation-sensitive"
+      enforcement: hard
+      match:
+        data.classification: [restricted, phi, pci, sovereign, classified]
+        target.type: dcm_peer
+      decision: DENY
+      reason: "Sovereign profile: sensitive data never crosses DCM federation boundaries"
+
+    - handle: "system/matrix/sovereign-internal-only-federation"
+      enforcement: hard
+      match:
+        data.classification: [public, internal]
+        target.type: dcm_peer
+        context.zero_trust_posture:
+          not_minimum: hardware_attested
+      decision: DENY
+      reason: "Sovereign profile: federation requires hardware attestation"
+
+    - handle: "system/matrix/sovereign-provider-sovereign-zone-only"
+      enforcement: hard
+      match:
+        data.classification: [restricted, phi, pci, sovereign, classified]
+        target.type: service_provider
+        target.sovereignty_zone.not_in: []
+      decision: DENY
+      reason: "Sovereign profile: sensitive data only to providers in declared sovereignty zone"
+```
+
+---
+
+## 8. Compliance Domain Matrix Rules
+
+When a compliance domain is active, its matrix rules are automatically added to the active rule set.
+
+### 8.1 HIPAA Compliance Domain Matrix
+
+```yaml
+compliance_domain_matrix:
+  domain: hipaa
+  rules:
+    - handle: "system/matrix/hipaa-phi-minimum-necessary"
+      enforcement: hard
+      match:
+        data.classification: phi
+        target.type: any
+      decision: ALLOW_WITH_CONDITIONS
+      conditions:
+        - principle: minimum_necessary    # only fields required for the specific purpose
+      field_permissions:
+        mode: blocklist                   # default: all fields except explicitly blocked
+        paths: []                         # Tenant adds specific field blocks
+        on_blocked_field: STRIP_FIELD
+
+    - handle: "system/matrix/hipaa-phi-no-export"
+      enforcement: hard
+      match:
+        data.classification: phi
+        data.capability: export
+      decision: DENY
+      reason: "HIPAA: PHI export to external systems requires explicit BAA and regulatory review"
+
+    - handle: "system/matrix/hipaa-audit-all-phi"
+      enforcement: hard
+      match:
+        data.classification: phi
+        target.type: any
+      decision: AUDIT_ONLY           # added to all PHI interactions โ€” does not block
+      audit_on: [ALLOW, DENY, STRIP_FIELD]
+      # Every PHI interaction produces an audit record โ€” HIPAA requirement
+```
+
+### 8.2 GDPR Compliance Domain Matrix
+
+```yaml
+compliance_domain_matrix:
+  domain: gdpr
+  rules:
+    - handle: "system/matrix/gdpr-eu-residency"
+      enforcement: hard
+      match:
+        data.classification: [restricted, phi]
+        target.sovereignty_zone.not_in: 
+        context.compliance_domains.includes: gdpr
+      decision: DENY
+      reason: "GDPR Article 44: personal data transfer outside EU requires adequacy decision"
+
+    - handle: "system/matrix/gdpr-right-to-erasure-fields"
+      enforcement: hard
+      match:
+        data.field_paths.includes: ["fields.personal_identifier_*", "fields.contact_*"]
+        data.capability: [store, replicate]
+        target.accreditation_held.not_includes: gdpr_adequacy
+      decision: STRIP_FIELD
+      field_permissions:
+        mode: blocklist
+        paths: ["fields.personal_identifier_*", "fields.contact_*"]
+```
+
+---
+
+## 9. Tenant and Resource-Type Override Rules
+
+Tenants and resource-type specifications declare additional rules that compose with system and profile rules per the standard precedence model.
+
+### 9.1 Tenant Override Rule
+
+```yaml
+# Tenant payments-team: additional restriction on PHI fields
+governance_matrix_rule:
+  artifact_metadata:
+    tier: tenant
+    handle: "tenant/payments/phi-field-restriction"
+  
+  match:
+    subject.tenant.uuid: payments-tenant-uuid
+    data.classification: phi
+    target.type: dcm_peer
+    target.accreditation_held.includes: hipaa
+  
+  decision: ALLOW_WITH_CONDITIONS
+  conditions:
+    - field: target.trust_posture
+      operator: equals
+      value: verified                    # only verified (not vouched)
+  
+  field_permissions:
+    mode: allowlist                      # tighter than the fsi default (passthrough)
+    paths:
+      - "fields.resource_type"
+      - "fields.lifecycle_state"
+      # PHI-containing fields not listed โ†’ automatically stripped
+    on_blocked_field: STRIP_FIELD
+```
+
+### 9.2 Resource-Type Override Rule
+
+```yaml
+# For Patient Record resources: maximum restriction regardless of Tenant settings
+governance_matrix_rule:
+  artifact_metadata:
+    tier: resource_type
+    handle: "resource-type/patient-record/no-federation"
+  
+  match:
+    data.resource_type: Healthcare.PatientRecord
+    target.type: dcm_peer
+  
+  decision: DENY
+  enforcement: hard
+  reason: "Patient Record entities are never federated โ€” local only"
+```
+
+---
+
+## 10. Governance Matrix in the Registration Flow
+
+When a provider attempts to register with DCM, the governance matrix is evaluated before the registration is accepted. This answers: "Is a provider of this type, with these accreditations, in this sovereignty zone, permitted to register in this DCM deployment?"
+
+```
+Provider submits registration
+  โ”‚
+  โ–ผ Governance matrix evaluation:
+  โ”‚   subject: { type: , accreditation_held: [...], sovereignty_zone:  }
+  โ”‚   data: { capability: register }
+  โ”‚   target: { type: dcm_instance, sovereignty_zone:  }
+  โ”‚   context: { profile: , compliance_domains: [...] }
+  โ”‚
+  โ”œโ”€โ”€ DENY: registration rejected immediately
+  โ”‚   Provider type not permitted in this profile
+  โ”‚   Provider in excluded jurisdiction
+  โ”‚   Required accreditation not held
+  โ”‚
+  โ””โ”€โ”€ ALLOW / ALLOW_WITH_CONDITIONS: registration proceeds to validation pipeline
+```
+
+---
+
+## 11. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `GMX-001` | The Governance Matrix is the single enforcement point for all cross-boundary data and capability decisions. Parallel enforcement mechanisms (standalone sovereignty checks, standalone accreditation checks) are inputs to the matrix โ€” not independent enforcement paths. |
+| `GMX-002` | Hard rules cannot be relaxed by any downstream rule at any domain level. Hard DENY is absolute. |
+| `GMX-003` | Soft rules establish defaults that can only be tightened by downstream rules. Soft DENY cannot be relaxed to ALLOW by a more-specific rule. |
+| `GMX-004` | Sovereign and classified data classifications carry hard DENY rules for all federation and external provider interactions in all profiles including minimal. This is the one rule that cannot be changed by any configuration. |
+| `GMX-005` | Every governance matrix evaluation produces an audit record regardless of outcome. |
+| `GMX-006` | Field-level stripping (STRIP_FIELD) is always audited with the field path and the rule_uuid that governed the stripping. |
+| `GMX-007` | Profile default matrix rules are soft unless explicitly marked hard. Tenant and resource-type rules can tighten profile defaults but cannot relax hard rules. |
+| `GMX-008` | Compliance domain matrix rules are automatically added to the active rule set when the compliance domain is active. They compose with profile rules โ€” they do not replace them. |
+| `GMX-009` | The Governance Matrix is evaluated before provider dispatch, before federation tunnel data transmission, before notification delivery, and before any cross-boundary capability invocation. |
+| `GMX-010` | A STRIP_FIELD decision that removes a required field escalates to DENY_REQUEST automatically. Optional fields may be stripped without blocking the interaction. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/28-federated-contribution-model.md b/content/docs/data-model/28-federated-contribution-model.md
new file mode 100644
index 0000000..aadabc2
--- /dev/null
+++ b/content/docs/data-model/28-federated-contribution-model.md
@@ -0,0 +1,592 @@
+# DCM Data Model โ€” Federated Contribution Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Read This First for Multi-User Data Governance
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Layering and Versioning](03-layering-and-versioning.md) | [Policy Profiles](14-policy-profiles.md) | [Registry Governance](20-registry-governance.md) | [DCM Federation](22-dcm-federation.md) | [Governance Matrix](27-governance-matrix.md) | [Consumer API](../specifications/consumer-api-spec.md)
+
+> **This document maps to: DATA + POLICY + PROVIDER**
+>
+> The federated contribution model governs how Data artifacts are created and managed across all contributor types. It extends the Data abstraction with explicit contributor identity, applies Policies to govern contribution permissions and review requirements, and uses the Provider abstraction for cross-instance federation of contributions.
+
+---
+
+> **Authority Tier Reference:** Contribution approval tiers (`reviewed`, `verified`, `authorized`) are named positions in the [Authority Tier Model](32-authority-tier-model.md) ordered list. Organizations may add custom tiers between existing ones. Changes to the tier registry that affect contribution approval requirements trigger impact detection (ATM-009โ€“ATM-012).
+
+## 1. Purpose and Principle
+
+DCM is a multi-user, multi-contributor system. Platform admins are not the only actors who create data. Consumers define their own service configurations, resource groups, and policy overlays. Service Providers publish their own resource type specs and catalog items. Peer DCM instances contribute registry entries across federation boundaries. Organizations extend DCM with their own artifact types.
+
+**The federated contribution model** is the governing framework for how all of these actors create, review, activate, and lifecycle-manage DCM data artifacts. It extends the Data abstraction with one additional universal property:
+
+> **Every DCM data artifact has a contributor** โ€” an actor or system that authored it โ€” and that contributor's role determines what review is required before the artifact becomes active.
+
+This is not a special model for special cases. It is the same GitOps PR workflow, the same lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired), and the same domain precedence (system โ†’ platform โ†’ tenant โ†’ resource_type โ†’ entity) โ€” applied consistently across all contributor types.
+
+**The core principle:** DCM defaults to a federated model for data creation, import, usage, and lifecycle. Every authorized actor can contribute within the bounds their role permits. The Governance Matrix governs the boundaries. The GitOps PR flow provides the review mechanism. Profile-bound auto-approval policies determine what needs human review and what does not.
+
+---
+
+## 2. Contributor Types and Permissions
+
+### 2.1 The Four Contributor Types
+
+| Contributor | Examples | Default domain scope |
+|-------------|---------|---------------------|
+| **Platform Admin** | DCM operators, SRE team | system, platform โ€” all artifact types |
+| **Consumer / Tenant** | Application teams, developers, Tenant admins | tenant โ€” scoped to their Tenant |
+| **Service Provider** | Infrastructure teams, automation platforms | provider โ€” resource types they offer |
+| **Peer DCM** | Federated DCM instances, Hub DCM, community registry | federated โ€” governed by federation trust posture |
+
+### 2.2 What Each Contributor Can Contribute
+
+**Platform Admin:** All artifact types at all domain levels. No restrictions within the DCM deployment.
+
+**Consumer / Tenant:**
+- Tenant-domain policies (GateKeeper, Transformation, Recovery, Lifecycle, Orchestration Flow)
+- Resource groups and group memberships within their Tenant
+- Notification subscriptions for their Tenant
+- Webhook registrations for their Tenant
+- Custom catalog item definitions (within their Tenant's resource type scope)
+- Tenant-scoped data layers (Request Layer โ€” directly attached to their requests)
+- Cross-tenant authorization records (requires counterpart Tenant acceptance)
+
+**Service Provider:**
+- Resource Type Specifications for resource types they offer (Organization or Verified Community tier)
+- Provider Catalog Items for their registered resource types
+- Service Layers for their offered resource types
+- Provider-specific GateKeeper and Validation policies (provider domain)
+- Cost metadata updates
+- Sovereignty declaration updates
+
+**Peer DCM:**
+- Registry entries (Resource Type Specs, provider type definitions) contributed through federation channels
+- Policy bundles contributed through verified federation relationships
+- Layer contributions through Hub DCM governance
+- Accreditation vouching for providers registered with the contributing DCM
+
+### 2.3 What Each Contributor Cannot Contribute
+
+| Contributor | Cannot contribute |
+|-------------|-----------------|
+| Consumer | System or platform domain policies; core layers; resource type specs (unless granted elevated role); provider catalog items for other providers |
+| Service Provider | Policies outside their resource type domain; core layers; other providers' catalog items; tenant-domain policies for specific Tenants |
+| Peer DCM | Artifacts above the federation trust level granted; system-domain policies without authorized approval; sovereignty zones for jurisdictions not in their declared scope |
+
+---
+
+## 3. Contribution Artifact Types
+
+Every DCM data artifact type has a declared set of contributor permissions. The following table specifies who can contribute each type and at what domain level:
+
+| Artifact Type | Platform Admin | Consumer/Tenant | Service Provider | Peer DCM |
+|--------------|---------------|-----------------|-----------------|---------|
+| Resource Type Specification | All tiers | โŒ | Org + Community tiers | Community tier (via federation) |
+| Provider Catalog Item | All | โŒ | Their resource types only | โŒ |
+| Core Layer | โœ… | โŒ | โŒ | โŒ |
+| Service Layer | โœ… | โŒ | Their resource types only | โŒ |
+| Request Layer | โœ… | Their requests only | โŒ | โŒ |
+| GateKeeper Policy | All domains | Tenant domain only | Provider domain only | Via federation governance |
+| Transformation Policy | All domains | Tenant domain only | Provider domain only | Via federation governance |
+| Recovery Policy | All domains | Tenant domain only | Provider domain only | Via federation governance |
+| Orchestration Flow Policy | All domains | Tenant domain only | โŒ | โŒ |
+| Governance Matrix Rule | All domains | Tenant domain only | โŒ | โŒ |
+| Lifecycle Policy | All domains | Tenant domain (on their entities) | โŒ | โŒ |
+| Accreditation | All | โŒ | Their own accreditations | Vouching for their providers |
+| Sovereignty Zone | โœ… | โŒ | โŒ | โŒ |
+| DCMGroup / Resource Group | All | Tenant domain only | โŒ | โŒ |
+| Notification Subscription | All | Their Tenant only | โŒ | โŒ |
+| Webhook Registration | All | Their Tenant only | โŒ | โŒ |
+
+---
+
+## 4. The Contribution Flow
+
+All contributions โ€” regardless of contributor type โ€” flow through the same GitOps PR model. What varies is:
+- **The target store** (which GitOps repository receives the PR)
+- **The review requirement** (auto-approval vs human review vs dual approval)
+- **The shadow mode behavior** (policies enter shadow mode automatically; other artifacts enter proposed status)
+
+### 4.1 The Universal Contribution Pipeline
+
+```
+Contributor authors a data artifact
+  โ”‚
+  โ”‚ Via one of three contribution surfaces:
+  โ”œโ”€โ”€ Flow GUI Canvas / Policy Authoring Interface
+  โ”œโ”€โ”€ Direct API (POST /api/v1/contribute/{artifact_type})
+  โ””โ”€โ”€ Git PR directly to the target repository
+  โ”‚
+  โ–ผ Artifact submitted โ†’ status: developing (local only)
+  โ”‚
+  โ–ผ Contributor submits for review โ†’ status: proposed
+  โ”‚   For policies: shadow mode activates automatically
+  โ”‚   For other artifacts: staged in proposed state
+  โ”‚
+  โ–ผ Governance Matrix evaluates the contribution:
+  โ”‚   Is this contributor permitted to contribute this artifact type?
+  โ”‚   Is the artifact in the correct domain for this contributor?
+  โ”‚   Does the artifact pass structural validation?
+  โ”‚   DENY โ†’ rejected with reason; no further processing
+  โ”‚
+  โ–ผ Review flow (per profile + artifact type):
+  โ”‚   auto:          artifact activates immediately
+  โ”‚   reviewed:  one platform admin or designated reviewer approves
+  โ”‚   verified: two independent reviewers approve
+  โ”‚   authorized:   N members of declared authority group record decisions via Admin API
+  โ”‚
+  โ–ผ On approval โ†’ status: active
+  โ”‚   For policies: shadow mode results reviewed; full enforcement begins
+  โ”‚   For resource type specs: available in registry
+  โ”‚   For catalog items: visible in service catalog (per RBAC)
+  โ”‚
+  โ–ผ Lifecycle managed by contributor (deprecate, retire)
+      Subject to platform admin override at any time
+```
+
+### 4.2 Review Requirements by Contributor and Artifact Type
+
+Review requirements are profile-governed. The table below shows defaults:
+
+| Artifact Type | Platform Admin | Consumer/Tenant | Service Provider |
+|--------------|---------------|-----------------|-----------------|
+| Tenant-domain policy | auto | reviewed (standard+) | reviewed |
+| Resource Type Spec (Org tier) | auto | โŒ | reviewed |
+| Resource Type Spec (Community tier) | reviewed | โŒ | verified |
+| Provider Catalog Item | auto | โŒ | reviewed |
+| Service Layer | auto | โŒ | reviewed |
+| Governance Matrix Rule (tenant) | auto | verified | โŒ |
+| Governance Matrix Rule (platform) | reviewed | โŒ | โŒ |
+| Accreditation | reviewed | โŒ | reviewed |
+
+**Profile overrides:**
+- `dev`: most contributions auto-approved; shadow mode optional
+- `standard`: consumer policies require reviewed; provider specs require reviewed
+- `prod`: consumer governance matrix rules require verified; provider specs require verified
+- `fsi`: all contributions require verified; community registry entries require authorized
+- `sovereign`: all contributions require authorized approval
+
+---
+
+## 5. Consumer Contribution Model
+
+### 5.1 Consumer as Policy Author
+
+Consumers are not passive requesters. Tenant admins and designated Tenant members with `policy_author` role can define and maintain their own Tenant-domain policies directly.
+
+**What this enables:**
+- A Payments team defining their own cost ceiling GateKeeper: "Reject any VM request over $500/month"
+- An Operations team defining their own expiry Transformation: "All dev VMs get a 30-day TTL injected"
+- A Security team defining their own governance matrix rule: "Our Tenant never sends confidential data to unaccredited providers"
+
+**The scope constraint is enforced by DCM, not by convention.** When a consumer submits a policy with `domain: tenant`, DCM validates that the contributing actor belongs to that Tenant. Attempts to submit platform or system domain policies are rejected by the Governance Matrix at contribution time.
+
+### 5.2 Consumer Contribution API
+
+```
+POST /api/v1/contribute/policy
+
+Authorization: Bearer 
+X-DCM-Tenant: 
+
+{
+  "policy_type": "gatekeeper",
+  "handle": "tenant/payments/gatekeeper/cost-ceiling",
+  "domain": "tenant",
+  "concern_type": "operational",
+  "enforcement": "soft",
+  "match": {
+    "payload_type": "request.policies_evaluated",
+    "conditions": [
+      { "field": "payload.cost_estimate.per_month", "operator": "gt", "value": 500 }
+    ]
+  },
+  "output": {
+    "decision": "deny",
+    "reason": "Estimated monthly cost exceeds Tenant budget ceiling of $500"
+  },
+  "shadow_mode": true,           # start in shadow mode (proposed status)
+  "commit_message": "Add monthly cost ceiling GateKeeper for Payments Tenant"
+}
+
+Response 202 Accepted:
+{
+  "contribution_uuid": "",
+  "artifact_type": "policy",
+  "policy_handle": "tenant/payments/gatekeeper/cost-ceiling",
+  "status": "proposed",
+  "shadow_mode": true,
+  "review_required": true,
+  "review_type": "reviewed",
+  "reviewer_group": "platform-admins",
+  "pr_url": "https://git.corp.example.com/dcm-policies/pulls/145",
+  "shadow_results_url": "/flow/api/v1/shadow/"
+}
+```
+
+### 5.3 Consumer Resource Group and Service Definitions
+
+Consumers can define their own resource groups and service compositions within their Tenant:
+
+```
+POST /api/v1/contribute/resource-group
+
+{
+  "handle": "tenant/payments/groups/prod-vms",
+  "display_name": "Production VMs โ€” Payments",
+  "group_class": "resource_grouping",
+  "description": "All production VMs owned by the Payments team",
+  "membership_policy": {
+    "auto_include": {
+      "resource_type": "Compute.VirtualMachine",
+      "tags": { "team": "payments", "env": "production" }
+    }
+  }
+}
+```
+
+---
+
+## 6. Service Provider Contribution Model
+
+### 6.1 Provider as Resource Type Publisher
+
+Service Providers are not just execution targets โ€” they are first-class contributors of the resource type definitions that consumers request. A provider registering a new virtual machine offering publishes the Resource Type Specification, the Catalog Item, and the Service Layer that consumers use to interact with it.
+
+**What this enables:**
+- A storage team publishing a new `Storage.DistributedVolume` resource type with its full schema, constraints, and cost model
+- A networking team publishing provider-specific VLAN configurations as a Catalog Item with their own Service Layer injecting provider-specific defaults
+- A platform team publishing an updated `Compute.VirtualMachine` spec with new fields and deprecating old ones
+
+**Provider contributions flow through the same registry governance as all other registry entries** โ€” submitted as PRs to the organization registry, reviewed per profile requirements, activated when approved.
+
+### 6.2 Provider Contribution API
+
+```
+POST /api/v1/provider/contribute/resource-type-spec
+
+Authorization: mTLS + provider credential
+
+{
+  "resource_type_fqn": "Storage.DistributedVolume",
+  "tier": "organization",
+  "version": "1.0.0",
+  "schema": {
+    "fields": [
+      { "field_name": "capacity_gb", "type": "integer", "required": true },
+      { "field_name": "replication_factor", "type": "integer",
+        "default": 3, "constraint": { "min": 1, "max": 5 } },
+      { "field_name": "encryption_at_rest", "type": "boolean", "default": true }
+    ]
+  },
+  "portability_class": "provider_specific",
+  "commit_message": "Publish DistributedVolume resource type v1.0.0"
+}
+
+Response 202 Accepted:
+{
+  "contribution_uuid": "",
+  "resource_type_fqn": "Storage.DistributedVolume",
+  "status": "proposed",
+  "review_required": true,
+  "review_type": "reviewed",
+  "pr_url": "https://git.corp.example.com/dcm-registry/pulls/89"
+}
+```
+
+### 6.3 Provider Service Layer Contribution
+
+Providers contribute Service Layers that DCM applies during request assembly for their resource types:
+
+```
+POST /api/v1/provider/contribute/service-layer
+
+{
+  "resource_type_fqn": "Compute.VirtualMachine",
+  "layer_handle": "providers/eu-west-prod-1/layers/vm-defaults",
+  "layer_domain": "service",
+  "provider_uuid": "",
+  "version": "2.0.0",
+  "fields": {
+    "hypervisor": { "value": "KVM", "metadata": { "override": "immutable" } },
+    "network_segment": { "value": "prod-segment-01" },
+    "backup_enabled": { "value": true }
+  }
+}
+```
+
+---
+
+## 7. Federation Contribution Model
+
+### 7.1 Peer DCM as Contributor
+
+A federated peer DCM is a contributor to the receiving DCM's artifact stores, subject to the federation trust posture. This enables:
+
+- **Hub DCM contributing policy templates** to Regional DCMs โ€” standard compliance policies distributed from a central Hub
+- **Community DCM registry contributions** โ€” a community-maintained DCM instance publishing Verified Community resource type specs to subscribing organizations
+- **Provider contributions across DCM boundaries** โ€” a provider registered with DCM-A contributing its resource type specs to DCM-B through a verified federation relationship
+
+### 7.2 Federation Contribution Trust Model
+
+Federation contributions inherit the federation trust posture of the contributing peer:
+
+| Peer trust posture | Contribution review requirement | Artifact types permitted |
+|-------------------|--------------------------------|------------------------|
+| `verified` | reviewed (standard+); auto (dev) | Registry entries, policy templates, service layers |
+| `vouched` | reviewed always | Registry entries, service layers only |
+| `provisional` | `authorized` tier approval | Registry entries only (no policies) |
+
+**Hard rule:** A peer DCM cannot contribute artifacts at a higher domain level than its trust posture permits. A `vouched` peer cannot contribute system-domain policies. This is enforced by the Governance Matrix at the federation contribution boundary.
+
+### 7.3 Federation Contribution Flow
+
+```
+Peer DCM publishes a contribution bundle:
+  Content: resource type specs, policy templates, or layers
+  Transport: federation tunnel (mTLS, signed, scoped credential)
+  Metadata: contributing_dcm_uuid, trust_posture, artifact_list
+
+Receiving DCM evaluates:
+  1. Governance Matrix: is this peer permitted to contribute this artifact type?
+  2. Signature verification: bundle signed by peer's private key?
+  3. Structural validation: artifacts conform to DCM schemas?
+  4. Domain scope check: artifacts within peer's permitted domain?
+
+On validation pass:
+  Artifacts enter proposed status in receiving DCM's policy/registry store
+  Review flow per receiving DCM's profile + peer trust posture
+
+On approval:
+  Artifacts become active in receiving DCM
+  Source attribution: contributed_by.dcm_uuid, contributed_by.trust_posture
+```
+
+### 7.4 Hub DCM Policy Distribution
+
+In a Hub-Spoke federation, the Hub DCM is the authoritative source for platform-wide policy templates. Regional DCMs subscribe to the Hub's policy distribution feed:
+
+```yaml
+hub_policy_distribution:
+  hub_dcm_uuid: 
+  distribution_type: push          # Hub pushes on policy change
+  auto_approve_from_hub:           # profile-governed; security-first: prod+ always requires review
+    minimal: true
+    dev: true
+    standard: true
+    prod: false         # reviewed required even from verified Hub
+    fsi: false          # verified required
+    sovereign: false    # authorized approval required
+  policy_handles_subscribed:
+    - "system/compliance/hipaa/*"
+    - "system/governance/drift-remediation"
+  # Regional DCM always reviews before activating
+  # Hub cannot force-activate policies on Regional DCMs
+```
+
+---
+
+## 8. Artifact Lifecycle Across Contributors
+
+### 8.1 Contributor Ownership and Transfer
+
+Every artifact is owned by its contributor at creation. Ownership can be transferred:
+- Consumer-authored policies transfer to a new Tenant admin when the original actor departs
+- Provider-contributed catalog items remain owned by the provider registration
+- Federation-contributed artifacts are owned by the contributing peer DCM
+
+Ownership transfer requires the receiving owner's explicit acceptance (same model as entity ownership transfer in the Consumer API).
+
+### 8.2 Platform Admin Override
+
+Platform admins can override any contributor's artifact lifecycle at any time:
+- Suspend an active consumer-authored policy that is causing harm
+- Retire a provider-contributed resource type spec that is no longer safe
+- Reject a proposed federation contribution without providing a public reason (security discretion)
+
+Override actions are always audited with the overriding admin's actor UUID and reason.
+
+### 8.3 Deprecation and Sunset
+
+Contributors deprecate their own artifacts. When a Service Provider deprecates a resource type spec:
+1. All consumers using that type receive deprecation notifications
+2. A sunset period is declared (minimum: P30D for standard profile; P90D for prod/fsi/sovereign)
+3. During sunset: new requests using the deprecated spec are warned; existing resources unaffected
+4. After sunset: new requests using the deprecated spec are blocked
+5. Platform admin must confirm final retirement
+
+### 8.4 Orphaned Artifacts
+
+When a contributor's access is revoked (actor departs, provider deregisters, peer DCM federation ends):
+- Active artifacts remain active โ€” orphaned artifacts do not automatically deactivate
+- A platform admin is notified: "Artifact tenant/payments/gatekeeper/cost-ceiling has no active owner"
+- Platform admin assigns a new owner or explicitly retires the artifact
+- Auto-retire-on-orphan is configurable per profile (enabled in sovereign profile; disabled in standard)
+
+---
+
+## 9. The Contribution Store
+
+All contributed artifacts are stored in the GitOps store with contributor attribution. The directory structure reflects the contributor hierarchy:
+
+```
+dcm-policy-store/
+  system/                     # Platform admin authored; DCM built-in
+    compliance/
+    governance/
+    orchestration/
+  platform/                   # Platform admin authored; deployment-specific
+    security/
+    operations/
+  tenant/
+    /           # Consumer/Tenant authored
+      gatekeeper/
+      transformation/
+      groups/
+  provider/
+    /         # Provider authored
+      layers/
+      policies/
+  federated/
+    /           # Peer DCM contributed
+      registry/
+      policy-templates/
+
+dcm-registry/
+  core/                       # DCM project maintained
+  community/                  # Community contributed (via community DCM)
+    /
+  organization/               # Organization contributed
+    /
+```
+
+Every artifact in the store includes a `contributed_by` block in its artifact metadata:
+
+```yaml
+artifact_metadata:
+  uuid: 
+  handle: "tenant/payments/gatekeeper/cost-ceiling"
+  version: "1.0.0"
+  status: active
+  contributed_by:
+    contributor_type: consumer       # platform_admin | consumer | service_provider | peer_dcm
+    actor_uuid:                # for consumer/platform_admin contributions
+    tenant_uuid:               # for consumer contributions
+    provider_uuid:             # for provider contributions
+    peer_dcm_uuid:             # for federation contributions
+    contribution_method: api         # api | flow_gui | git_pr | federation_push
+    pr_url: "https://..."            # if submitted via PR
+    reviewed_by: []      # actors who approved
+    reviewed_at: 
+```
+
+---
+
+## 10. Profile-Governed Contribution Defaults
+
+Each deployment profile has a default contribution policy that governs auto-approval eligibility, required review, and shadow mode defaults:
+
+```yaml
+contribution_policy:
+  minimal:
+    consumer_policy_auto_approve: true             # ease of use: homelab auto-approves
+    provider_spec_auto_approve: true
+    federation_contribution_auto_approve: true    # homelab: federation auto-approved
+    shadow_mode_default: true                     # security: shadow always on even in minimal
+
+  dev:
+    consumer_policy_auto_approve: true
+    provider_spec_auto_approve: true
+    federation_contribution_auto_approve: false   # reviewed for federation
+    shadow_mode_default: true                     # shadow mode on by default
+
+  standard:
+    consumer_policy_auto_approve: false           # reviewed for all policies
+    provider_spec_auto_approve: false
+    federation_contribution_auto_approve: false
+    shadow_mode_default: true
+    shadow_review_period: P7D                     # 7 days of shadow before promotion
+
+  prod:
+    consumer_policy_auto_approve: false
+    consumer_governance_matrix_requires: verified
+    provider_spec_auto_approve: false
+    provider_spec_requires: reviewed
+    federation_contribution_requires: reviewed
+    shadow_mode_default: true
+    shadow_review_period: P14D
+
+  fsi:
+    consumer_policy_auto_approve: false
+    consumer_policy_requires: verified
+    consumer_governance_matrix_requires: verified
+    provider_spec_requires: verified
+    federation_contribution_requires: verified
+    shadow_mode_default: true
+    shadow_review_period: P30D
+    min_shadow_divergence_review: true            # must review all divergence cases
+
+  sovereign:
+    consumer_policy_requires: authorized
+    provider_spec_requires: authorized
+    federation_contribution_requires: authorized
+    shadow_mode_default: true
+    shadow_review_period: P30D
+    min_shadow_divergence_review: true
+    auto_retire_orphaned_artifacts: true          # orphaned artifacts retire automatically
+```
+
+---
+
+## 11. Governance Matrix Integration
+
+The Governance Matrix evaluates every contribution at submission time. This is the enforcement point for the contributor permission table in Section 2.3.
+
+**Contribution evaluation:**
+
+```yaml
+governance_matrix_rule:
+  handle: "system/matrix/consumer-policy-scope"
+  enforcement: hard
+  match:
+    subject.type: consumer
+    data.artifact_type: policy
+    data.domain: [system, platform]    # consumer attempting non-tenant domain
+  decision: DENY
+  reason: "Consumers may only contribute tenant-domain policies"
+
+governance_matrix_rule:
+  handle: "system/matrix/provider-spec-scope"
+  enforcement: hard
+  match:
+    subject.type: service_provider
+    data.artifact_type: resource_type_spec
+    data.resource_type_fqn:
+      not_in: subject.declared_resource_types   # provider contributing type they don't offer
+  decision: DENY
+  reason: "Providers may only contribute Resource Type Specs for resource types they offer"
+```
+
+---
+
+## 12. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `FCM-001` | Every DCM data artifact has a contributor. The contributor is recorded in artifact_metadata.contributed_by at creation and is immutable. |
+| `FCM-002` | Contributor permissions are enforced by the Governance Matrix at submission time. Domain scope violations are hard DENY โ€” they cannot be overridden by the contributor. |
+| `FCM-003` | All contributions flow through the GitOps PR model. No contributor can write directly to the authoritative artifact store without a PR review (unless the active profile grants auto-approval for that contributor type and artifact type combination). |
+| `FCM-004` | Policies submitted by any contributor enter proposed (shadow) status by default. Shadow mode results must be available before the active profile's shadow_review_period expires. |
+| `FCM-005` | Platform admins may override any contributor's artifact lifecycle at any time. Override actions are audited. |
+| `FCM-006` | Orphaned artifacts (contributor access revoked) do not automatically deactivate. A platform admin assigns a new owner or explicitly retires them. Exception: sovereign profile auto-retires orphaned artifacts. |
+| `FCM-007` | Federation contributions from peer DCMs are scoped by the peer's federation trust posture. Verified peers: reviewed (standard+). Vouched peers: reviewed always. Provisional peers: authorized approval. |
+| `FCM-008` | Contributor-tier scope limits are absolute. A consumer-authored policy in the tenant domain cannot affect the system or platform domain regardless of the policy's declared match conditions. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/29-scoring-model.md b/content/docs/data-model/29-scoring-model.md
new file mode 100644
index 0000000..ee2e93b
--- /dev/null
+++ b/content/docs/data-model/29-scoring-model.md
@@ -0,0 +1,516 @@
+# DCM Data Model โ€” Hybrid Scoring Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. This document specifies the hybrid scoring model โ€” a first-class architectural addition approved in the DCM architecture review. It extends the Policy Engine and Profile system without replacing any existing components.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Scoring Model Specification
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Policy Profiles](14-policy-profiles.md) | [Control Plane Components](25-control-plane-components.md) | [Governance Matrix](27-governance-matrix.md) | [Federated Contribution Model](28-federated-contribution-model.md)
+
+> **This document maps to: DATA + POLICY**
+>
+> The Scoring Model is an extension of the Policy abstraction. Scored signals are Data artifacts with lifecycle and provenance. Profile thresholds are Policy-governed configuration. The Governance Matrix remains a pure boolean gate โ€” scoring never applies to cross-boundary data decisions.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+> > **See also:** [Authority Tier Model](32-authority-tier-model.md) โ€” the ordered authority tier list, custom tier definition, dynamic threshold format, and ATM system policies.
+
+> **Design Priority:** The Scoring Model is the primary mechanism for Priority 2 (ease of use) in service of Priority 1 (security). The auto-approval threshold (SMX-008: โ‰ค 50) and compliance-class GateKeepers are non-negotiable security properties. Profile thresholds and signal weights are the ease-of-use scaling mechanism. See [Design Priorities](00-design-priorities.md).
+
+---
+
+## 1. Purpose and Governing Principle
+
+DCM uses a **hybrid scoring model**: some decisions are boolean gates (facts), others are scored signals (degrees). The governing principle is explicit:
+
+> **Questions of fact use boolean gates. Questions of degree use scoring.**
+
+A secondary test for any ambiguous decision:
+> **Can a regulator accept "the score was below threshold" as a complete explanation? If not, the decision must be boolean.**
+
+This document specifies the scoring half of the hybrid. For boolean decisions, see [Governance Matrix](27-governance-matrix.md) and the compliance enforcement model in [Policy Contract](B-policy-contract.md).
+
+### 1.1 What This Model Does
+
+The scoring model adds three capabilities to the existing architecture:
+
+1. **Operational GateKeeper policies** contribute a weighted risk score instead of producing a binary deny. The aggregate score drives approval routing.
+2. **Advisory Validation policies** produce a completeness score and warning list without blocking the request.
+3. **Five scoring signals** aggregate into a request risk score that determines approval routing tier โ€” replacing the current per-policy approval flag with a continuous, profile-governed threshold system.
+
+### 1.2 What This Model Does Not Do
+
+The scoring model does **not**:
+- Apply to Governance Matrix decisions โ€” these remain boolean always
+- Apply to compliance-class GateKeeper policies โ€” PHIโ†’BAA, sovereign dataโ†’sovereign provider remain hard gates
+- Apply to authentication, authorization, or five-check boundary enforcement
+- Apply to lifecycle state transitions
+- Replace the Policy Engine โ€” it is a function within it
+
+---
+
+## 2. GateKeeper Enforcement Classes
+
+Every GateKeeper policy declares an `enforcement_class`. This is a required field in the Policy base contract (added in this document).
+
+```yaml
+enforcement_class: compliance | operational
+```
+
+### 2.1 Compliance Class
+
+Behavior: **boolean gate**. A compliance-class GateKeeper that fires produces a `deny` decision. The request is halted immediately. No score is produced.
+
+**Use for:**
+- Data classification boundary rules (PHI requires BAA accreditation)
+- Sovereignty violations (classified data leaving declared zone)
+- Security hard requirements (unencrypted data, expired certificates)
+- Regulatory mandates with no legitimate override path
+- Any rule where "score-around" creates legal or compliance liability
+
+```yaml
+# Example compliance-class GateKeeper
+policy_type: gatekeeper
+enforcement_class: compliance
+handle: "system/compliance/phi-baa-required"
+match:
+  payload_type: request.layers_assembled
+  conditions:
+    - field: payload.data_classification
+      operator: contains
+      value: phi
+    - field: payload.provider.accreditations
+      operator: not_contains
+      value: baa_active
+output:
+  decision: deny
+  reason: "PHI data requires provider with active BAA. Provider has no active BAA."
+  audit_required: true
+  notify_on: [DENY]
+```
+
+### 2.2 Operational Class
+
+Behavior: **risk score contribution**. An operational-class GateKeeper that fires contributes a weighted score to the request risk score. The request is not immediately halted. Instead the aggregate score determines routing.
+
+**Use for:**
+- Cost ceiling policies (request cost exceeds Tenant recommendation)
+- Resource sizing policies (CPU/memory above recommended maximums)
+- Unusual timing or context (off-hours request, unusual field combinations)
+- Quota pressure (Tenant approaching quota limit)
+- Business rule preferences that should escalate review, not block
+
+```yaml
+# Example operational-class GateKeeper
+policy_type: gatekeeper
+enforcement_class: operational
+handle: "tenant/payments/gatekeeper/cost-ceiling"
+scoring_weight: 35           # contribution to request risk score when fired
+match:
+  payload_type: request.layers_assembled
+  conditions:
+    - field: payload.cost_estimate.per_month
+      operator: gt
+      value: 500
+output:
+  risk_score_contribution: 35
+  reason: "Estimated monthly cost ${{payload.cost_estimate.per_month}} exceeds Tenant ceiling $500"
+  label: "cost_ceiling_exceeded"
+  audit_required: true
+```
+
+### 2.3 Profile-Level Enforcement Class Override
+
+Profiles can override the enforcement class of individual policies. This is the mechanism for making the scoring system tunable without touching individual policies.
+
+```yaml
+# In a profile definition:
+policy_enforcement_overrides:
+  - policy_handle: "tenant/payments/gatekeeper/cost-ceiling"
+    override_enforcement_class: compliance   # escalate to hard gate in this profile
+    rationale: "FSI profile: all cost violations are hard gates"
+
+  - policy_handle: "system/security/off-hours-request"
+    override_enforcement_class: operational  # demote to soft score in dev profile
+    rationale: "Dev profile: off-hours requests are expected; score but don't block"
+```
+
+**Hard constraint:** A profile can **only** override `operational โ†’ compliance` or `compliance โ†’ operational` for explicitly non-regulatory policies. Policies with `regulatory_mandate: true` in their metadata cannot be demoted to operational by any profile.
+
+---
+
+## 3. Validation Output Classes
+
+Every Validation policy declares an `output_class`. This is a required field.
+
+```yaml
+output_class: structural | advisory
+```
+
+### 3.1 Structural Class
+
+Behavior: **boolean pass/fail**. A structural Validation that fails halts the request. No score is produced.
+
+**Use for:**
+- Required field presence (missing required fields)
+- Type correctness (wrong field type)
+- Referential integrity (UUID references that don't resolve)
+- Format validation (malformed handle, invalid semver)
+- Schema conformance
+
+### 3.2 Advisory Class
+
+Behavior: **completeness score contribution + warning list**. An advisory Validation that fires contributes to the completeness score and adds a warning to the advisory_warnings list. The request is not halted.
+
+**Use for:**
+- Recommended fields absent (cost_center not provided)
+- Unusual values (memory_gb at 1 for a database VM โ€” unusual but not invalid)
+- Low-confidence field values (field sourced from a provider with confidence < 0.5)
+- Naming convention violations (non-compliant resource name โ€” advisory only)
+
+```yaml
+policy_type: validation
+output_class: advisory
+handle: "platform/advisory/cost-center-recommended"
+scoring_weight: 10
+match:
+  payload_type: request.layers_assembled
+output:
+  completeness_contribution: 10
+  warning_code: "recommended_field_absent"
+  warning_message: "cost_center not provided โ€” cost attribution will use Tenant default"
+  field: "fields.cost_center"
+```
+
+---
+
+## 4. The Five Scoring Signals
+
+The request risk score is assembled from five independent signals. Each signal is normalized to 0โ€“100. The aggregate is a weighted sum, also normalized to 0โ€“100.
+
+### 4.1 Signal 1 โ€” Operational GateKeeper Score
+
+**Source:** All operational-class GateKeeper policies that fired during policy evaluation.
+**Composition:** Sum of `risk_score_contribution` values from all fired operational GateKeepers.
+**Normalization:** Capped at 100 before weighting. Multiple GateKeepers can fire; their contributions accumulate.
+**Default weight in aggregate:** 0.45
+
+```yaml
+operational_gatekeeper_score:
+  fired_policies:
+    - handle: "tenant/payments/gatekeeper/cost-ceiling"
+      contribution: 35
+      reason: "Cost $620/month exceeds ceiling $500"
+    - handle: "platform/gatekeeper/off-hours"
+      contribution: 15
+      reason: "Request submitted outside business hours"
+  raw_score: 50     # sum of contributions
+  normalized: 50    # already within 0-100
+```
+
+### 4.2 Signal 2 โ€” Policy Completeness Score
+
+**Source:** All advisory-class Validation policies that fired.
+**Composition:** Sum of `completeness_contribution` values from all fired advisory Validations.
+**Normalization:** Capped at 100. Score represents "how incomplete is this request" โ€” higher = more warnings.
+**Default weight in aggregate:** 0.15
+
+### 4.3 Signal 3 โ€” Actor Risk History Score
+
+**Source:** Decay-weighted history of the actor's previous request outcomes.
+**Composition:** Each historical event has a base score contribution and a time-decay multiplier.
+**Decay model:** `contribution ร— e^(-ฮปt)` where `t` is days since event, `ฮป` = 0.1 (half-life โ‰ˆ 7 days).
+**Normalization:** 0โ€“100. A clean history = 0. Recent consecutive failures approach 100.
+**Default weight in aggregate:** 0.20
+
+```yaml
+# Events that contribute to actor risk history score
+actor_risk_events:
+  - event: validation_failure        # base_contribution: 5
+  - event: gatekeeper_deny           # base_contribution: 10
+  - event: compliance_deny           # base_contribution: 20
+  - event: policy_override_requested # base_contribution: 8
+  - event: drift_caused              # base_contribution: 15
+  - event: decommission_forced       # base_contribution: 12
+  - event: request_abandoned         # base_contribution: 3
+```
+
+**Privacy constraint:** Actor risk history scores are never exposed in consumer-facing API responses beyond the actor's own history. They are available in the Admin API for platform admins and in the audit trail.
+
+### 4.4 Signal 4 โ€” Tenant Quota Pressure Score
+
+**Source:** Current quota utilization for the resource type being requested.
+**Composition:** `max(0, (utilization_pct - free_threshold) / (1 - free_threshold)) ร— 100`
+**Free threshold:** 0.75 (quota pressure score = 0 below 75% utilization).
+**At 100% utilization:** quota pressure = 100, but the hard quota gate also fires (blocking the request regardless of score).
+**Default weight in aggregate:** 0.10
+
+```yaml
+quota_pressure_score:
+  resource_type: "Compute.VirtualMachine"
+  current_usage: 87
+  limit: 100
+  utilization_pct: 0.87
+  free_threshold: 0.75
+  score: 48   # (0.87 - 0.75) / (1 - 0.75) ร— 100 = 48
+```
+
+### 4.5 Signal 5 โ€” Provider Accreditation Richness Score
+
+**Source:** Accreditation portfolio of the selected/candidate provider.
+**Composition:** Weighted sum of accreditation types held, normalized against the maximum possible portfolio.
+**Usage:** Used in placement tie-breaking (supplements existing tie-breaking algorithm). Also contributes inversely to request risk score โ€” a richly accredited provider reduces risk.
+**Default weight in aggregate:** 0.10 (inverse โ€” higher richness = lower risk contribution)
+
+```yaml
+accreditation_weights:
+  self_declared: 5
+  third_party_audit: 15
+  iso_27001: 20
+  soc2_type2: 20
+  fedramp_moderate: 30
+  fedramp_high: 40
+  hipaa_baa: 25
+  pci_dss: 25
+  sovereign_authorization: 50
+
+# richness_score = sum(weights for held accreditations) / max_possible ร— 100
+# risk_contribution = (1 - richness_score/100) ร— 10   [lower richness = higher risk]
+```
+
+### 4.6 Aggregate Request Risk Score
+
+```
+request_risk_score =
+  (operational_gatekeeper_score ร— 0.45) +
+  (completeness_score           ร— 0.15) +
+  (actor_risk_history_score     ร— 0.20) +
+  (quota_pressure_score         ร— 0.10) +
+  (provider_risk_contribution   ร— 0.10)
+
+# Normalized: 0โ€“100
+# 0  = clean request, no concerns
+# 100 = maximum risk signal across all dimensions
+```
+
+Signal weights are profile-governed and can be adjusted per deployment. The weights above are the `standard` profile defaults.
+
+---
+
+## 5. Profile-Governed Thresholds
+
+Every profile declares scoring thresholds that map the continuous risk score to a discrete approval routing decision.
+
+```yaml
+# Approval routing uses named tier thresholds โ€” see Authority Tier Model (doc 32)
+# Tier names are resolved from the ordered authority tier list; numeric weights are derived.
+scoring_thresholds:
+  approval_routing:
+    - tier: auto
+      max_score: 24          # score 0โ€“24: auto-approve (SMX-008: never exceed 50)
+    - tier: reviewed
+      max_score: 59          # score 25โ€“59: reviewed tier required
+    - tier: verified
+      max_score: 79          # score 60โ€“79: verified tier required
+    - tier: authorized
+      max_score: 100         # score 80โ€“100: authorized tier required
+  # Custom tiers (if defined) are inserted into this list; existing names unchanged.
+  # "authorized" means DCM holds the pipeline and notifies the declared DCMGroup;
+  # the review process and deliberation are the organization's responsibility.
+  # DCM records votes via Admin API; external systems (ServiceNow, Jira, Slack)
+  # may call the API on behalf of authorized group members. See [Design Priorities](00-design-priorities.md).
+  # Note: compliance-class GateKeeper deny always halts regardless of score
+```
+
+### 5.1 Per-Profile Threshold Defaults
+
+| Profile | auto_approve | reviewed | verified | authorized | signal_weights |
+|---------|-------------|-------------|--------------|-----------|----------------|
+| `minimal` | < 45 | 45โ€“74 | 75โ€“100 | โ€” | default |
+| `dev` | < 40 | 40โ€“69 | 70โ€“100 | โ€” | default |
+| `standard` | < 25 | 25โ€“59 | 60โ€“79 | 80โ€“100 | default |
+| `prod` | < 15 | 15โ€“49 | 50โ€“74 | 75โ€“100 | gatekeeper_weight: 0.50 |
+| `fsi` | < 10 | 10โ€“39 | 40โ€“69 | 70โ€“100 | gatekeeper_weight: 0.55, actor_weight: 0.25 |
+| `sovereign` | < 5 | 5โ€“29 | 30โ€“59 | 60โ€“100 | gatekeeper_weight: 0.60 |
+
+### 5.2 Resource-Type Threshold Overrides
+
+Profiles can declare tighter thresholds for specific resource types:
+
+```yaml
+resource_type_threshold_overrides:
+  - resource_type: "Compute.VirtualMachine"
+    auto_approve_below: 20    # tighter than profile default
+  - resource_type: "Network.VLAN"
+    auto_approve_below: 10    # VLANs require more scrutiny
+  - resource_type: "Storage.Volume"
+    verified_above: 40   # storage changes escalate earlier
+```
+
+### 5.3 Tenant Threshold Overrides
+
+Platform admins can declare Tenant-level scoring threshold adjustments:
+
+```yaml
+tenant_scoring_config:
+  tenant_uuid: 
+  threshold_overrides:
+    auto_approve_below: 15    # more conservative for this Tenant
+  signal_weight_overrides:
+    actor_risk_history_weight: 0.30   # higher actor scrutiny for this Tenant
+  trusted_actors:
+    - actor_uuid: 
+      actor_risk_history_score_override: 0   # zero out risk history for trusted automation
+```
+
+### 5.4 Switching Between Scoring and Boolean Per Policy
+
+A profile can declare that a specific operational-class policy should behave as boolean (compliance-class) in that profile's context:
+
+```yaml
+# In profile definition:
+policy_enforcement_overrides:
+  - policy_handle: "platform/gatekeeper/cpu-size-limit"
+    override_enforcement_class: compliance
+    rationale: "Prod profile: CPU limit is a hard constraint, not a risk signal"
+    applies_to_resource_types: ["Compute.VirtualMachine"]
+```
+
+And conversely, a compliance-class policy that is **not** a regulatory mandate can be demoted to operational in lower-trust profiles:
+
+```yaml
+  - policy_handle: "platform/gatekeeper/naming-convention"
+    override_enforcement_class: operational
+    scoring_weight_override: 20
+    rationale: "Dev profile: naming violations are warnings, not blocks"
+    requires_regulatory_mandate_false: true   # safety check
+```
+
+---
+
+## 6. Score Lifecycle and Audit Trail
+
+### 6.1 Score Record Structure
+
+Every scored evaluation produces a Score Record stored in the Audit Store alongside the standard audit record:
+
+```yaml
+score_record:
+  score_record_uuid: 
+  request_uuid: 
+  entity_uuid: 
+  evaluated_at: 
+  
+  request_risk_score: 47
+  routing_decision: reviewed
+  routing_threshold_applied: 25      # the threshold that triggered this tier
+  profile_uuid: 
+  
+  signal_breakdown:
+    operational_gatekeeper:
+      score: 50
+      weight: 0.45
+      weighted_contribution: 22.5
+      fired_policies:
+        - handle: "tenant/payments/gatekeeper/cost-ceiling"
+          contribution: 35
+        - handle: "platform/gatekeeper/off-hours"
+          contribution: 15
+    completeness:
+      score: 20
+      weight: 0.15
+      weighted_contribution: 3.0
+      advisory_warnings: 2
+    actor_risk_history:
+      score: 30
+      weight: 0.20
+      weighted_contribution: 6.0
+      recent_events: 2
+    quota_pressure:
+      score: 48
+      weight: 0.10
+      weighted_contribution: 4.8
+    provider_risk:
+      score: 15
+      weight: 0.10
+      weighted_contribution: 1.5
+  
+  compliance_gates_evaluated: 3
+  compliance_gates_fired: 0         # if any > 0: request halted regardless of risk score
+```
+
+### 6.2 Score Immutability
+
+Score Records are immutable once written. Threshold changes do not retroactively alter historical Score Records. If thresholds change, requests evaluated before the change retain their original routing decisions in the audit trail.
+
+### 6.3 Human Override of Score-Based Routing
+
+A platform admin or reviewer can override a score-based routing decision with a recorded justification. The override is audited, but the Score Record is never modified โ€” instead an Override Record is written referencing the original Score Record.
+
+---
+
+## 7. Score Exposure in APIs
+
+### 7.1 Consumer-Facing Score Exposure
+
+Consumers receive a simplified score view:
+- `risk_score` on request status (integer 0โ€“100)
+- `routing_decision` (auto_approved | pending_review | pending_verified | pending_authorized)
+- `advisory_warnings` list from advisory Validation
+- `score_drivers` โ€” human-readable list of the top 3 contributing factors (no raw weights)
+
+Consumers **do not** receive:
+- Actor risk history score breakdown (privacy)
+- Signal weights
+- Provider accreditation richness detail
+
+### 7.2 Platform Admin Score Exposure
+
+Platform admins receive full Score Record detail via the Admin API including all signal breakdowns, weights, and actor risk history detail.
+
+---
+
+## 8. Relationship to Existing Decision Model
+
+The scoring model slots into the existing pipeline without replacing any component:
+
+```
+Policy Engine evaluation run:
+  1. Evaluate all matching policies (existing behavior)
+  2. Compliance-class GateKeeper fires โ†’ HALT (existing deny behavior)
+  3. Structural Validation fails โ†’ HALT (existing fail behavior)
+  4. Governance Matrix DENY fires โ†’ HALT (existing behavior, unchanged)
+  5. NEW: Collect operational GateKeeper contributions โ†’ Signal 1
+  6. NEW: Collect advisory Validation contributions โ†’ Signal 2
+  7. NEW: Fetch actor risk history score โ†’ Signal 3
+  8. NEW: Calculate quota pressure score โ†’ Signal 4
+  9. NEW: Calculate provider accreditation richness โ†’ Signal 5
+  10. NEW: Aggregate โ†’ request_risk_score
+  11. NEW: Apply profile thresholds โ†’ routing_decision
+  12. NEW: Write Score Record to Audit Store
+  13. Route request: auto_approve | queue_for_review | queue_dual | queue_authorized
+```
+
+Step 2, 3, and 4 are unchanged from the existing model. Steps 5โ€“13 are additive.
+
+---
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `SMX-001` | Every GateKeeper policy must declare `enforcement_class: compliance` or `enforcement_class: operational`. Policies without a declared enforcement_class are treated as compliance-class. |
+| `SMX-002` | Every Validation policy must declare `output_class: structural` or `output_class: advisory`. Policies without a declared output_class are treated as structural. |
+| `SMX-003` | Compliance-class GateKeeper policies with `regulatory_mandate: true` cannot be overridden to operational by any profile. This flag is set by platform admins and is audited. |
+| `SMX-004` | The Governance Matrix is always boolean. No Governance Matrix Rule may declare a scoring weight or enforcement_class. |
+| `SMX-005` | Signal weights in a profile must sum to 1.00. Profiles with invalid weight sums fail validation at activation time. |
+| `SMX-006` | Score Records are immutable. Threshold changes do not retroactively alter historical Score Records. |
+| `SMX-007` | Actor risk history scores are not exposed to consumers beyond the actor's own history. Platform admins have full access. |
+| `SMX-008` | A profile's `auto_approve_below` threshold may not exceed 50. Auto-approving requests with risk scores above 50 is prohibited in all profiles. |
+| `SMX-009` | Operational-class GateKeeper `scoring_weight` values must be declared between 1 and 100. Weights above 100 are validation errors. The aggregate of all fired policies is capped at 100 before weighting. |
+| `SMX-010` | Score breakdown must be included in the audit trail for every request that receives a routing decision. A request with no Score Record is an audit integrity violation. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/30-meta-provider-model.md b/content/docs/data-model/30-meta-provider-model.md
new file mode 100644
index 0000000..039a4ed
--- /dev/null
+++ b/content/docs/data-model/30-meta-provider-model.md
@@ -0,0 +1,502 @@
+# DCM Data Model โ€” Meta Provider Composability Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. This document specifies the Meta Provider composability model โ€” what a Meta Provider is, what it declares to DCM, and how standard DCM machinery handles the rest.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Meta Provider Specification
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Service Dependencies](07-service-dependencies.md) | [Four States](02-four-states.md) | [Operational Models](24-operational-models.md) | [Scoring Model](29-scoring-model.md) | [Control Plane Components](25-control-plane-components.md)
+
+> **This document maps to: DATA + PROVIDER**
+>
+> A Meta Provider is a typed Provider that declares a compound service composition to DCM. The compound service it delivers is Data โ€” a Composite Entity across all four states. DCM's standard machinery (Placement Engine, Orchestration Flow Policy, Recovery Policy) handles everything beyond registration and constituent execution. The Meta Provider is not an orchestrator โ€” it is a compound service definition plus a set of standard Service Providers that happen to serve multiple resource types within the same system.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+
+---
+
+## 1. What a Meta Provider Is
+
+### 1.1 The Core Model
+
+A Meta Provider is a **compound Service Provider** that uses other providers in the DCM catalog to fulfill a higher-order service request. Its defining characteristic is that it registers a **compound service definition** โ€” a declaration of constituent resource types, their dependencies, and their delivery requirements โ€” so that DCM has enough information to:
+
+1. **Select appropriate constituent providers** via the standard Placement Engine
+2. **Determine execution order** from the dependency graph
+3. **Govern rehydration sequence** using the same dependency information
+
+Beyond providing that definition, a Meta Provider operates as a standard Service Provider for each constituent resource type it owns. DCM's standard machinery handles everything else: placement, sequencing, failure handling, compensation, and audit.
+
+**A Meta Provider is not an orchestrator.** It does not:
+- Select constituent providers โ€” the Placement Engine does this
+- Sequence execution rounds โ€” the dependency graph informs DCM's Orchestration Flow Policy
+- Manage parallel execution โ€” parallelism is derived from the dependency graph (resources with no unresolved dependencies execute simultaneously)
+- Run compensation โ€” DCM's Recovery Policy executes compensation using the dependency graph in reverse
+- Make routing decisions โ€” these are DCM policy decisions
+
+### 1.2 Why This Model Is Correct
+
+Every DCM design principle is preserved:
+- **Governance stays with DCM** โ€” constituent provider selection goes through the Placement Engine, including sovereignty filtering, accreditation checking, and trust scoring
+- **Policy stays with DCM** โ€” GateKeeper, Validation, and Transformation policies fire on the compound payload; the same policies govern each constituent sub-request
+- **Audit stays with DCM** โ€” each constituent request is a standard DCM request with its own audit trail; the compound audit is assembled from constituent audit records
+- **Recovery stays with DCM** โ€” the Recovery Policy handles constituent failures using the dependency graph; the Meta Provider does not make recovery decisions
+
+### 1.3 The Practical Meaning
+
+A Meta Provider registration tells DCM: "Here is a compound service called `ApplicationStack.WebApp`. To fulfill it, you will need a `Compute.VirtualMachine`, a `Network.IPAddress`, a `DNS.Record` (which depends on both), and a `Network.LoadBalancer` (which also depends on both). I can provide the DNS and LoadBalancer; you should place the VM and IP with appropriate compute and network providers."
+
+DCM then:
+- Creates a Composite Entity with one entity UUID
+- Runs the compound layer assembly to produce the full payload
+- Applies policies to the compound payload
+- Dispatches constituent sub-requests to the appropriate providers (compute provider for VM, network provider for IP, Meta Provider for DNS and LoadBalancer)
+- Sequences those sub-requests based on the declared dependency graph
+- Handles any constituent failures using Recovery Policy
+- Assembles the aggregate Realized State from all constituent realized states
+
+The Meta Provider's execution responsibility is limited to: naturalizing and realizing the constituent resource types it owns, then denaturalizing and returning the realized state โ€” exactly as a standard Service Provider does.
+
+---
+
+## 2. Compound Service Definition
+
+The compound service definition is the Meta Provider's primary contribution to DCM. It is declared at registration and stored in the Resource Type Registry as a compound Resource Type Specification.
+
+### 2.1 Constituent Declaration
+
+```yaml
+resource_types_composed:
+  - fqn: ApplicationStack.WebApp
+    version: "2.0.0"
+
+    constituents:
+      - component_id: vm-primary
+        resource_type: Compute.VirtualMachine
+        provided_by: external    # DCM places this with an appropriate compute provider
+        depends_on: []
+        required_for_delivery: required
+
+      - component_id: ip-primary
+        resource_type: Network.IPAddress
+        provided_by: external    # DCM places this with an appropriate network provider
+        depends_on: []
+        required_for_delivery: required
+
+      - component_id: dns-primary
+        resource_type: DNS.Record
+        provided_by: self        # This Meta Provider handles DNS
+        depends_on: [vm-primary, ip-primary]
+        required_for_delivery: partial
+
+      - component_id: lb-frontend
+        resource_type: Network.LoadBalancer
+        provided_by: self        # This Meta Provider handles LoadBalancer
+        depends_on: [vm-primary, ip-primary]
+        required_for_delivery: partial
+
+    composition_visibility: selective   # opaque | transparent | selective
+    dcm_visible_sub_resources:
+      - resource_type: Compute.VirtualMachine
+        role: application_host
+      - resource_type: Network.LoadBalancer
+        role: ingress_endpoint
+```
+
+### 2.2 provided_by Declaration
+
+`provided_by` is the key field that tells DCM who is responsible for each constituent:
+
+| Value | Meaning |
+|-------|---------|
+| `self` | This Meta Provider handles this constituent. DCM dispatches it to the Meta Provider. |
+| `external` | DCM places this constituent with the best available provider via the standard Placement Engine. |
+| `` | DCM dispatches this constituent to a specific named provider. |
+
+For `external` constituents, DCM runs a full placement cycle โ€” sovereignty filtering, accreditation checking, trust scoring, reserve query โ€” exactly as it would for any standalone request.
+
+For `self` constituents, DCM dispatches to the Meta Provider using the standard Services API. The Meta Provider receives a standard constituent payload and responds with a standard realized state โ€” no special handling required.
+
+### 2.3 Dependency Graph
+
+The `depends_on` list is the mechanism by which the Meta Provider informs DCM of execution ordering. DCM reads this graph and:
+
+- Identifies which constituents have no dependencies โ†’ dispatches these first (or simultaneously)
+- Identifies which constituents have dependencies on already-realized constituents โ†’ dispatches these when their dependencies are complete
+- Derives parallelism directly from the graph โ€” constituents with no shared unresolved dependencies execute concurrently within DCM's standard pipeline
+
+**The Meta Provider does not manage this sequencing.** It declares the graph. DCM executes it.
+
+```
+depends_on: []                          โ†’ eligible for immediate dispatch
+depends_on: [vm-primary]                โ†’ dispatched after vm-primary is REALIZED
+depends_on: [vm-primary, ip-primary]    โ†’ dispatched after BOTH are REALIZED
+```
+
+### 2.4 required_for_delivery Classification
+
+Each constituent declares how its success or failure affects the compound service:
+
+| Classification | Failure effect |
+|----------------|---------------|
+| `required` | DCM halts the compound request; triggers Recovery Policy; unrealized constituents are not dispatched |
+| `partial` | DCM notes the failure; compound service continues; final status may be `DEGRADED` |
+| `optional` | DCM notes the failure; compound service continues unaffected |
+
+The Recovery Policy governs what happens on `required` constituent failure โ€” the same Recovery Policy that governs any request failure. No special Meta Provider recovery logic exists.
+
+---
+
+## 3. Composite Entity โ€” Four-State Representation
+
+A compound service request produces a **Composite Entity** โ€” a single DCM entity that aggregates constituent sub-entities. The Composite Entity has one entity UUID that links it across all four states.
+
+### 3.1 Intent State
+
+The consumer submits one request against the compound catalog item. The intent payload contains consumer-declared fields for the compound service โ€” not individual constituent fields.
+
+```yaml
+entity_uuid:           # assigned at intent creation; stable across all states
+catalog_item_uuid:     # ApplicationStack.WebApp
+fields:
+  app_name: payments-api
+  environment: production
+  region: EU-WEST
+  cpu_count: 4
+  memory_gb: 16
+  dns_hostname: payments-api.internal.corp.example
+```
+
+DCM does not expand this into constituent requests at Intent State. The intent is stored as-is.
+
+### 3.2 Requested State
+
+The Request Payload Processor expands the compound intent into the full constituent payload. This is where the compound service definition from the Meta Provider registration is applied.
+
+```yaml
+entity_uuid: 
+composite_entity: true
+meta_provider_uuid: 
+
+top_level:
+  app_name: payments-api
+  environment: production
+  region: EU-WEST
+
+constituents:
+  - component_id: vm-primary
+    resource_type: Compute.VirtualMachine
+    provided_by: external
+    provider_uuid: null        # resolved by Placement Engine
+    fields:
+      cpu_count: 4
+      memory_gb: 16
+      os_family: rhel
+    depends_on: []
+    required_for_delivery: required
+
+  - component_id: ip-primary
+    resource_type: Network.IPAddress
+    provided_by: external
+    provider_uuid: null        # resolved by Placement Engine
+    fields:
+      ip_version: 4
+      allocation_pool: prod-EU-WEST
+    depends_on: []
+    required_for_delivery: required
+
+  - component_id: dns-primary
+    resource_type: DNS.Record
+    provided_by: self
+    provider_uuid: 
+    fields:
+      hostname: payments-api.internal.corp.example
+      record_type: A
+    depends_on: [vm-primary, ip-primary]
+    required_for_delivery: partial
+
+  - component_id: lb-frontend
+    resource_type: Network.LoadBalancer
+    provided_by: self
+    provider_uuid: 
+    fields:
+      backend_component: vm-primary
+    depends_on: [vm-primary, ip-primary]
+    required_for_delivery: partial
+```
+
+### 3.3 Realized State
+
+The Realized State is assembled by DCM from the constituent realized payloads returned by each dispatched provider. DCM writes it as a unified composite record.
+
+```yaml
+entity_uuid: 
+composite_entity: true
+composite_status: DEGRADED    # REALIZED | DEGRADED | FAILED
+
+composite_fields:             # synthesized consumer-facing view
+  primary_ip: 10.1.45.23
+  vm_id: vm-0a1b2c3d
+  lb_endpoint: lb-7f8e9d.eu-west.corp
+  dns_name: null              # absent โ€” dns constituent failed
+
+constituents_realized:
+  - component_id: vm-primary
+    status: REALIZED
+    provider_uuid: 
+    realized_fields:
+      vm_id: vm-0a1b2c3d
+      hypervisor_host: host-eu-w-04
+    required_for_delivery: required
+
+  - component_id: ip-primary
+    status: REALIZED
+    provider_uuid: 
+    realized_fields:
+      assigned_ip: 10.1.45.23
+    required_for_delivery: required
+
+  - component_id: dns-primary
+    status: FAILED
+    provider_uuid: 
+    failure_reason: "DNS service degraded โ€” record not created"
+    required_for_delivery: partial
+
+  - component_id: lb-frontend
+    status: REALIZED
+    provider_uuid: 
+    realized_fields:
+      lb_id: lb-7f8e9d
+      endpoint: lb-7f8e9d.eu-west.corp
+    required_for_delivery: partial
+
+degradation_summary:
+  - component_id: dns-primary
+    impact: "DNS name resolution unavailable โ€” access via IP only"
+    recovery_eligible: true
+```
+
+### 3.4 Discovered State
+
+Discovery for composite entities follows the composition visibility mode:
+
+- `opaque` โ€” DCM schedules one discovery call to the Meta Provider; it returns the compound discovered state as a single payload
+- `transparent` โ€” DCM schedules independent discovery calls to each constituent's provider; drift detection runs on each constituent independently
+- `selective` โ€” DCM schedules discovery calls for DCM-visible constituents; opaque discovery for the rest via Meta Provider
+
+---
+
+## 4. What DCM Does vs What the Meta Provider Does
+
+This table is the definitive statement of responsibility.
+
+| Concern | DCM | Meta Provider |
+|---------|-----|---------------|
+| Compound catalog item presentation | โœ… โ€” Service Catalog manages the compound item | Declares the compound service definition at registration |
+| Consumer-facing API | โœ… โ€” Consumer API handles compound requests identically to simple requests | โ€” |
+| Layer assembly | โœ… โ€” Request Payload Processor assembles constituent blocks from compound service definition | โ€” |
+| Policy evaluation on compound payload | โœ… โ€” Same Policy Engine, same policies, same scoring model | May contribute provider-domain policies for its own constituents |
+| External constituent placement | โœ… โ€” Placement Engine selects provider for each `provided_by: external` constituent | Declares what resource types each constituent needs so placement can filter appropriately |
+| Self constituent dispatch | โœ… โ€” API Gateway dispatches to Meta Provider using standard Services API | Receives constituent payload; naturalizes; executes; denaturalizes; returns realized state |
+| Execution ordering from dependency graph | โœ… โ€” Derived from `depends_on` declarations; DCM dispatches in order | Declares `depends_on` relationships for each constituent |
+| Parallelism | โœ… โ€” Constituents with no unresolved dependencies execute concurrently within DCM's pipeline | Emerges from the dependency graph declaration; Meta Provider does not manage this |
+| Constituent failure handling | โœ… โ€” Recovery Policy fires based on `required_for_delivery` classification | Declares `required_for_delivery` for each constituent |
+| Compensation (teardown of realized constituents on failure) | โœ… โ€” Recovery Policy executes compensation using dependency graph in reverse | Implements standard decommission handling for `self` constituents |
+| Composite status determination | โœ… โ€” Determined by DCM from constituent outcomes and `required_for_delivery` | โ€” |
+| Realized State assembly | โœ… โ€” DCM assembles composite Realized State from constituent realized payloads | Returns standard realized payload for `self` constituents |
+| Drift detection | โœ… โ€” Standard drift detection per composition visibility mode | Implements standard discovery endpoint for `self` constituents |
+| Lifecycle management of Composite Entity | โœ… โ€” Standard DCM entity lifecycle | Handles decommission of `self` constituents when decommission payload received |
+| Audit trail | โœ… โ€” Each constituent request has its own audit record; composite audit assembled by DCM | โ€” |
+
+### 4.1 The Meta Provider's Execution Scope Is Narrow
+
+For each `self` constituent, the Meta Provider:
+1. Receives a standard DCM constituent payload (already fully assembled by DCM)
+2. Naturalizes it to its native format
+3. Executes the realization
+4. Denaturalizes the result
+5. Returns a standard realized payload
+
+This is identical to what any Service Provider does. The Meta Provider is not special during execution โ€” it is simply a Service Provider that happens to be registered for multiple resource types within the same underlying system.
+
+---
+
+## 5. Composition Visibility
+
+Introduced in [07-service-dependencies.md](07-service-dependencies.md), this section specifies the operational implications.
+
+| Mode | Consumer sees | DCM manages independently | Drift detection |
+|------|--------------|--------------------------|-----------------|
+| `opaque` | Top-level entity only | Composite entity only | Via Meta Provider discovery endpoint |
+| `transparent` | All constituents as DCM entities | All constituent entities individually | Per-constituent via each provider |
+| `selective` | Declared visible constituents | Declared visible constituents | Per-constituent for visible; via Meta Provider for rest |
+
+### 5.1 Transparent Mode Entity UUIDs
+
+In transparent mode, constituent entities receive stable DCM entity UUIDs:
+```
+constituent_entity_uuid = deterministic_uuid(parent_entity_uuid + component_id)
+```
+
+These UUIDs are stable across rehydration โ€” the same compound entity always produces the same constituent UUIDs. This enables consistent audit trail linkage.
+
+### 5.2 Decommission Cascade
+
+When a composite entity is decommissioned:
+- DCM dispatches constituent decommission sub-requests in dependency-reverse order
+- Each constituent's provider (whether `self` or `external`) receives a standard decommission payload
+- The composite entity's lifecycle state transitions to DECOMMISSIONED only after all constituents confirm decommission
+
+---
+
+## 6. Rehydration
+
+Rehydration of a composite entity is the primary use case for the dependency graph declaration โ€” which the Meta Provider provides precisely for this purpose.
+
+### 6.1 Rehydration Sequence
+
+DCM reads the `depends_on` graph and rehydrates constituents in dependency order:
+
+```
+Round 1: components with depends_on: []          โ†’ rehydrate first
+Round 2: components whose depends_on are realized โ†’ rehydrate next
+Round N: continue until all constituents complete
+```
+
+The Meta Provider's dependency declarations give DCM exactly the information it needs to sequence rehydration correctly without requiring any special Meta Provider involvement beyond standard constituent execution.
+
+### 6.2 Rehydration Provider Selection
+
+`provided_by: external` constituents are re-placed by the Placement Engine during rehydration โ€” they may end up on a different provider than the original realization if the original provider is no longer eligible. `provided_by: self` constituents always return to the same Meta Provider.
+
+---
+
+## 7. Compound Request Pipeline
+
+A compound service request flows through DCM's standard pipeline with compound-specific extensions at the Request Payload Processor step.
+
+```
+Consumer submits compound request
+  โ”‚
+  โ–ผ Intent State captured (compound intent, no constituent expansion)
+  โ”‚
+  โ–ผ Request Payload Processor:
+  โ”‚   Layer assembly on compound payload
+  โ”‚   Expansion: compound service definition โ†’ constituent blocks
+  โ”‚     (resource_type, provided_by, fields, depends_on, required_for_delivery per constituent)
+  โ”‚   Policy evaluation on compound payload (GateKeeper, Validation, Transformation, scoring)
+  โ”‚   Placement: external constituents โ†’ Placement Engine
+  โ”‚             self constituents     โ†’ Meta Provider
+  โ”‚
+  โ–ผ Requested State written (full constituent specification with provider assignments)
+  โ”‚
+  โ–ผ Constituent dispatch โ€” dependency-ordered by DCM:
+  โ”‚   Round 1 (no dependencies): dispatch vm-primary, ip-primary in parallel
+  โ”‚     vm-primary โ†’ compute_provider (standard Services API)
+  โ”‚     ip-primary โ†’ network_provider (standard Services API)
+  โ”‚
+  โ”‚   Round 2 (vm+ip REALIZED): dispatch dns-primary, lb-frontend in parallel
+  โ”‚     dns-primary โ†’ meta_provider (standard Services API)
+  โ”‚     lb-frontend โ†’ meta_provider (standard Services API)
+  โ”‚
+  โ”‚   Any constituent FAILS:
+  โ”‚     required  โ†’ Recovery Policy fires; unstarted constituents cancelled
+  โ”‚     partial   โ†’ noted as degraded; execution continues
+  โ”‚     optional  โ†’ noted; execution continues
+  โ”‚
+  โ–ผ Constituent realized payloads collected by DCM
+  โ”‚
+  โ–ผ Composite status determined: REALIZED | DEGRADED | FAILED
+  โ”‚
+  โ–ผ Realized State written (composite record assembled by DCM)
+  โ”‚
+  โ–ผ Consumer notified of compound request outcome
+```
+
+---
+
+## 8. Nested Meta Providers
+
+A Meta Provider may declare a constituent with `provided_by: external` where the appropriate provider is itself another Meta Provider. DCM's Placement Engine handles this transparently โ€” it places the constituent with whichever registered provider best satisfies the constraints, whether that is a simple Service Provider or another Meta Provider.
+
+**Maximum nesting depth: 3** โ€” enforced by DCM at placement time by checking the compound service definition chain depth. Deeper nesting creates dependency graph complexity that exceeds DCM's governance model.
+
+**The nested Meta Provider has no special awareness** that it is being called as a constituent of an outer compound service. It receives a standard constituent payload and responds with a standard realized state. Nesting is a DCM-level concept, not a provider-level one.
+
+---
+
+## 9. Scoring Model Integration
+
+Compound service requests are scored using the standard five-signal model with two compound-specific behaviors:
+
+**Operational GateKeepers** fire on the compound payload assembled from the top-level fields and declared constituent types. They do not fire per-constituent (constituent-level policy evaluation happens in each constituent sub-request's own pipeline).
+
+**Provider accreditation richness (Signal 5)** for compound entities uses the lowest richness score among all `required_for_delivery: required` constituents across all their assigned providers. The compound service is only as well-accredited as its least-accredited required constituent.
+
+---
+
+## 10. Meta Provider Registration Contract
+
+```yaml
+meta_provider_capabilities:
+  # Resource types this Meta Provider handles as a self provider
+  resource_types_provided:
+    - DNS.Record
+    - Network.LoadBalancer
+
+  # Compound service definitions this Meta Provider offers
+  resource_types_composed:
+    - fqn: ApplicationStack.WebApp
+      version: "2.0.0"
+      constituents:
+        - component_id: vm-primary
+          resource_type: Compute.VirtualMachine
+          provided_by: external
+          depends_on: []
+          required_for_delivery: required
+        - component_id: ip-primary
+          resource_type: Network.IPAddress
+          provided_by: external
+          depends_on: []
+          required_for_delivery: required
+        - component_id: dns-primary
+          resource_type: DNS.Record
+          provided_by: self
+          depends_on: [vm-primary, ip-primary]
+          required_for_delivery: partial
+        - component_id: lb-frontend
+          resource_type: Network.LoadBalancer
+          provided_by: self
+          depends_on: [vm-primary, ip-primary]
+          required_for_delivery: partial
+      composition_visibility: selective
+      dcm_visible_sub_resources:
+        - resource_type: Compute.VirtualMachine
+          role: application_host
+        - resource_type: Network.LoadBalancer
+          role: ingress_endpoint
+
+  # Standard provider declarations apply as for any Service Provider
+  # (sovereignty, accreditations, capacity reporting, health check, etc.)
+```
+
+---
+
+## 11. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `MPX-001` | A Meta Provider's `self` constituents are dispatched using the standard Services API. The Meta Provider receives a standard constituent payload and returns a standard realized state. No special dispatch protocol exists for Meta Provider self-constituents. |
+| `MPX-002` | Constituent execution ordering is derived from the `depends_on` declaration by DCM. The Meta Provider does not sequence constituent dispatch. |
+| `MPX-003` | Parallelism in constituent execution is derived from the dependency graph. Constituents with no unresolved dependencies execute concurrently within DCM's standard pipeline. The Meta Provider does not manage this. |
+| `MPX-004` | Composite status determination (`REALIZED` / `DEGRADED` / `FAILED`) is performed by DCM based on constituent outcomes and `required_for_delivery` classifications. |
+| `MPX-005` | Recovery Policy governs all constituent failure handling and compensation. The Meta Provider does not make recovery decisions. It implements standard decommission handling for `self` constituents when a decommission payload arrives. |
+| `MPX-006` | `provided_by: external` constituents are placed by the Placement Engine using standard placement rules. The Meta Provider does not influence external constituent provider selection. |
+| `MPX-007` | In transparent composition visibility mode, constituent entity UUIDs are `deterministic_uuid(parent_entity_uuid + component_id)` โ€” stable across rehydration. |
+| `MPX-008` | Maximum Meta Provider nesting depth is 3, enforced by DCM at placement time by checking the compound service definition chain depth. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/31-credential-provider-model.md b/content/docs/data-model/31-credential-provider-model.md
new file mode 100644
index 0000000..199f9a1
--- /dev/null
+++ b/content/docs/data-model/31-credential-provider-model.md
@@ -0,0 +1,984 @@
+# DCM Data Model โ€” Credential Provider Model
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed. This document specifies the Credential Provider model โ€” the issuance contract, credential lifecycle, rotation model, revocation propagation, and how credentials flow through the DCM pipeline.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Credential Provider Specification
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Auth Providers](19-auth-providers.md) | [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Scoring Model](29-scoring-model.md) | [Federated Contribution Model](28-federated-contribution-model.md)
+
+> **This document maps to: DATA + PROVIDER**
+>
+> Credentials are Data artifacts with UUID, type, lifecycle state, and provenance. The Credential Provider is a typed Provider with a defined capability extension. The Zero Trust model in [doc 26](26-accreditation-and-authorization-matrix.md) governs credential scope and lifetime โ€” this document specifies how that model is implemented.
+> See also: [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md)
+
+---
+
+> **Cryptographic Standards:** See [Standards and Compliance Catalog](40-standards-catalog.md) for the complete list of permitted/forbidden algorithms, FIPS requirements per profile, and certificate protocol RFCs (RFC 7030, RFC 8555, RFC 8894, RFC 4210).
+
+## 1. Purpose and Scope
+
+### 1.1 What the Credential Provider Does
+
+The Credential Provider is a typed DCM Provider responsible for issuing, rotating, and revoking credentials used within the DCM ecosystem. There are two categories of credential it manages:
+
+**1. DCM interaction credentials** โ€” short-lived, scoped credentials that DCM components and providers use to authenticate interactions. These implement the Zero Trust credential model from [doc 26](26-accreditation-and-authorization-matrix.md) Section 5.3. They are issued by DCM's Credential Provider and consumed entirely within the DCM control plane and its providers.
+
+**2. Consumer-facing credentials** โ€” credentials delivered to consumers as part of a realized service (kubeconfigs, database passwords, API keys, SSH keys, service account tokens). These are issued by the Credential Provider on behalf of a realized resource entity and delivered via the Consumer API.
+
+### 1.2 What the Credential Provider Does Not Do
+
+- It does not manage actor session tokens โ€” that is the Auth Provider's responsibility
+- It does not store secrets in DCM's data model โ€” credential values are never written to the GitOps stores or Realized State Store; only credential metadata (UUID, type, scope, expiry, status) is stored
+- It does not make authorization decisions โ€” authorization is the Governance Matrix's responsibility; the Credential Provider only issues if DCM has already authorized the operation
+- It does not replace secrets management for consumers' own applications โ€” it manages credentials that DCM issues for DCM-managed resources
+
+---
+
+## 2. Credential Types
+
+| Credential Type | Use Case | Typical Lifetime | Rotation Trigger |
+|----------------|----------|-----------------|-----------------|
+| `dcm_interaction` | DCM-internal component-to-provider auth | PT15Mโ€“PT1H (profile-governed) | Automatic; pre-expiry |
+| `api_key` | Programmatic consumer access to a realized resource | PT24Hโ€“P30D (configurable) | Scheduled or event-triggered |
+| `x509_certificate` | mTLS identity for providers and DCM components | P30Dโ€“P365D | P14D before expiry |
+| `ssh_key` | SSH access to realized VMs or infrastructure | P30Dโ€“P90D (configurable) | Scheduled or on-demand |
+| `service_account_token` | Workload identity for automated processes | PT1Hโ€“PT24H | Automatic; pre-expiry |
+| `database_password` | Access credential for realized database resources | PT24Hโ€“P7D (configurable) | Scheduled or on-demand |
+| `kubeconfig` | Access to realized Kubernetes clusters | PT8Hโ€“P30D (configurable) | Scheduled or on-demand |
+| `hsm_backed_key` | Sovereign/FSI deployments requiring hardware attestation | P30Dโ€“P365D | P14D before expiry; HSM-managed |
+
+---
+
+## 3. Credential Data Model
+
+A credential is a DCM Data artifact. Credential metadata is stored in DCM; credential values are held only by the Credential Provider (never in DCM stores).
+
+```yaml
+credential_record:
+  credential_uuid: 
+  credential_type: api_key | x509_certificate | ssh_key | service_account_token |
+                   database_password | kubeconfig | hsm_backed_key | dcm_interaction
+
+  # Lifecycle
+  status: active | rotating | revoked | expired
+  issued_at: 
+  valid_until: 
+  last_rotated_at: 
+  revoked_at: 
+  revocation_reason: 
+
+  # Scope โ€” what this credential authorizes
+  issued_to:
+    actor_uuid:           # consumer credential: issued to an actor
+    entity_uuid:          # resource credential: scoped to an entity
+    component_uuid:       # interaction credential: issued to a DCM component
+    provider_uuid:        # interaction credential: scoped to a provider
+  scope:
+    operations: [dispatch, discover, query, read, write, admin]  # allowed operations
+    resource_types: [Compute.VirtualMachine]                     # scoped resource types
+    tenant_uuid:                                    # Tenant scope
+  non_transferable: true               # always true for DCM-issued credentials
+  bound_to_ip:              # optional; enforced in fsi/sovereign profiles
+
+  # Provenance
+  credential_provider_uuid: 
+  issuing_request_uuid:          # which DCM request triggered issuance
+  entity_uuid:            # the realized entity this credential accesses
+  rotation_of:   # parent credential UUID if this is a rotation
+
+  # Storage (values never in DCM)
+  value_held_by: 
+  value_retrieval_endpoint:       # how the authorized consumer retrieves the value
+  value_retrieval_auth: bearer_token | mtls | step_up_mfa
+
+  # Cryptographic metadata (Sections 13)
+  algorithm: Ed25519 | ECDSA-P-384 | RSA-4096 | HS256 | RS256 | random_256bit
+  key_usage: [authentication]   # authentication | signing | encryption; declared at issuance
+  retrieved_count_threshold: 48  # hours; idle alert fires if not retrieved within this window
+```
+
+### 3.1 Credential Value Separation
+
+Credential values are never stored in DCM's data model, GitOps stores, or Realized State Store. DCM stores only the credential metadata record. The credential value is held exclusively by the Credential Provider.
+
+Authorized consumers retrieve the credential value via `value_retrieval_endpoint` using `value_retrieval_auth`. This retrieval is itself authenticated โ€” typically with a short-lived bearer token or mTLS โ€” and is audited.
+
+---
+
+## 4. Issuance Flows
+
+### 4.1 Resource Credential Issuance (consumer-facing)
+
+Credentials issued as part of resource realization flow through the standard provider dispatch pipeline.
+
+```
+Consumer requests resource (e.g., Compute.VirtualMachine)
+  โ”‚
+  โ–ผ Layer assembly + policy evaluation
+  โ”‚   Transformation policy may inject credential requirements:
+  โ”‚     fields.credential_requirements:
+  โ”‚       - credential_type: ssh_key
+  โ”‚         issued_to: requesting_actor
+  โ”‚         scope: [ssh_access]
+  โ”‚
+  โ–ผ Placement selects Service Provider for the VM
+  โ”‚
+  โ–ผ After VM realization: Credential Provider dispatched
+  โ”‚   DCM issues sub-request to Credential Provider:
+  โ”‚     entity_uuid: 
+  โ”‚     credential_type: ssh_key
+  โ”‚     issued_to.actor_uuid: 
+  โ”‚     scope.operations: [ssh_access]
+  โ”‚     scope.resource_types: [Compute.VirtualMachine]
+  โ”‚     valid_until: 
+  โ”‚
+  โ–ผ Credential Provider issues credential; returns credential_record
+  โ”‚   (value held by provider; metadata returned to DCM)
+  โ”‚
+  โ–ผ DCM writes credential_record to Realized State
+  โ”‚   Links credential_uuid to entity_uuid
+  โ”‚
+  โ–ผ Consumer receives realized entity + credential_record metadata
+  โ”‚   Consumer calls value_retrieval_endpoint to get actual credential
+  โ”‚   (step-up MFA may be required per profile)
+```
+
+### 4.2 DCM Interaction Credential Issuance
+
+DCM interaction credentials are issued automatically before each provider interaction. They implement the Zero Trust scoped credential model (ZTS-002).
+
+```
+DCM prepares to dispatch to a provider
+  โ”‚
+  โ–ผ Request interaction credential from Credential Provider:
+  โ”‚   credential_type: dcm_interaction
+  โ”‚   issued_to.component_uuid: 
+  โ”‚   issued_to.provider_uuid: 
+  โ”‚   scope.operations: [dispatch]
+  โ”‚   scope.resource_types: [Compute.VirtualMachine]
+  โ”‚   entity_uuid: 
+  โ”‚   valid_until:   (max; profile-governed)
+  โ”‚
+  โ–ผ Credential Provider issues scoped interaction credential
+  โ”‚
+  โ–ผ DCM includes credential in provider dispatch
+  โ”‚   Provider validates credential scope before executing
+  โ”‚
+  โ–ผ Credential expires after PT15M regardless of use
+  โ”‚   (no renewal; new credential issued for next interaction)
+```
+
+### 4.3 Bootstrap Credential Issuance
+
+During bootstrap, before the Credential Provider is registered, DCM uses a bootstrap credential mechanism. See [Deployment and Redundancy](17-deployment-redundancy.md) BOOT-003 for the bootstrap credential model. After bootstrap, all credentials are issued through a registered Credential Provider.
+
+---
+
+## 5. Credential Rotation Model
+
+Rotation is the primary mechanism for maintaining credential hygiene. DCM distinguishes scheduled rotation, pre-expiry rotation, and event-triggered rotation.
+
+### 5.1 Rotation Triggers
+
+| Trigger | Description | Default behavior |
+|---------|-------------|-----------------|
+| `scheduled` | Regular rotation on a declared schedule | Most credential types; interval is credential-type specific |
+| `pre_expiry` | Rotation initiated before the current credential expires | x509: P14D before expiry; ssh_key: P7D; dcm_interaction: PT5M |
+| `provider_initiated` | Credential Provider notifies DCM of a rotation requirement | Handled via provider update notification model |
+| `security_event` | Rotation triggered by a security signal (compromise, anomaly, policy change) | Immediate; see Section 5.4 |
+| `actor_request` | Consumer requests rotation of their own credential | Subject to rate limiting and policy |
+
+### 5.2 Rotation Protocol
+
+Rotation uses a transition window to prevent downtime. The old credential remains valid during the transition window; the new credential is issued and delivered before the old one expires.
+
+```
+Rotation initiated (by any trigger):
+  โ”‚
+  โ–ผ DCM requests new credential from Credential Provider
+  โ”‚   rotation_of: 
+  โ”‚   same scope as original; new valid_until
+  โ”‚
+  โ–ผ Credential Provider issues new credential
+  โ”‚   Returns new credential_record
+  โ”‚   Old credential NOT yet revoked
+  โ”‚
+  โ–ผ New credential delivered to authorized consumer/component
+  โ”‚   (same delivery mechanism as initial issuance)
+  โ”‚
+  โ–ผ Transition window: both credentials valid
+  โ”‚   Window duration: P1D for consumer credentials (default)
+  โ”‚                    PT5M for dcm_interaction credentials
+  โ”‚                    P7D for x509_certificate credentials
+  โ”‚   Configurable per credential type in Credential Provider registration
+  โ”‚
+  โ–ผ Old credential revoked at end of transition window
+  โ”‚   Revocation propagated to all registered consumers
+  โ”‚
+  โ–ผ Rotation record written to audit trail
+      old_credential_uuid, new_credential_uuid, rotation_trigger, rotation_at
+```
+
+### 5.3 Rotation Notification
+
+Before the old credential is revoked, DCM sends a rotation notification to any entity or actor whose credential is rotating:
+
+```yaml
+rotation_notification:
+  event_type: credential.rotating
+  credential_uuid: 
+  new_credential_uuid: 
+  transition_window_ends: 
+  retrieval_url: 
+  action_required: "Retrieve new credential before transition window ends"
+```
+
+### 5.4 Emergency Rotation (Security Event)
+
+On detection of a compromise or security event, DCM triggers emergency rotation:
+
+- No transition window โ€” old credential revoked immediately
+- New credential issued and delivered via the fastest available Notification Provider channel
+- Security event record written to Audit Store with full context
+- Compliance-class GateKeeper firing for this entity type audited against the event
+- Platform admin notified regardless of profile
+
+```
+Triggers for emergency rotation:
+  security.credential_compromised      # DCM or provider reports compromise
+  security.anomalous_usage_detected    # unusual access pattern detected
+  actor.deprovisioned                  # actor removed; all their credentials revoked
+  provider.deregistered                # provider leaving; all its interaction creds revoked
+  accreditation.revoked                # provider accreditation revoked; creds reassessed
+```
+
+---
+
+## 6. Revocation Model
+
+Revocation makes a credential permanently invalid before its natural expiry. Unlike rotation (which maintains continuity), revocation is an immediate termination.
+
+### 6.1 Revocation Triggers
+
+| Trigger | Initiator | Behavior |
+|---------|-----------|----------|
+| `actor_deprovisioned` | SCIM / Auth Provider | All credentials issued to the actor revoked immediately |
+| `entity_decommissioned` | DCM lifecycle | All credentials scoped to the entity revoked |
+| `security_event` | Platform admin or security automation | Immediate; no transition window |
+| `provider_deregistered` | Platform admin | All interaction credentials for the provider revoked |
+| `actor_request` | Consumer | Consumer may revoke their own credentials |
+| `ttl_expired` | Lifecycle Constraint Enforcer | Credential expired; revocation recorded |
+
+### 6.2 Revocation Propagation
+
+DCM maintains a **Credential Revocation Registry** โ€” a fast-queryable store of revoked credential UUIDs. All components that receive DCM interaction credentials must check this registry at each use (not just at issuance time).
+
+```
+Credential revoked:
+  โ”‚
+  โ–ผ Credential record status: active โ†’ revoked
+  โ”‚   revoked_at, revocation_reason written
+  โ”‚
+  โ–ผ Revocation event published to Message Bus
+  โ”‚   event_type: credential.revoked
+  โ”‚   credential_uuid: 
+  โ”‚   effective_at: 
+  โ”‚
+  โ–ผ All subscribed components update local revocation cache
+  โ”‚   (cache TTL: PT1M standard; PT30S fsi/sovereign)
+  โ”‚
+  โ–ผ Credential Provider notified to invalidate stored value
+  โ”‚   Provider must honor revocation within declared SLA:
+  โ”‚     standard/prod: PT5M
+  โ”‚     fsi/sovereign: PT1M
+  โ”‚
+  โ–ผ Audit record written
+      credential_uuid, revocation_trigger, revoked_by_actor, entity_uuid
+```
+
+### 6.3 Revocation Check at Use
+
+Providers receiving DCM interaction credentials must validate the credential at use time, not only at receipt time:
+
+1. Verify credential signature (if signed)
+2. Check credential UUID against local revocation cache
+3. Verify credential has not expired (`valid_until`)
+4. Verify operation is within credential scope
+5. Verify IP binding if `bound_to_ip` is set
+
+A credential that passes issuance validation but fails use-time validation is rejected. The provider must return `403 Forbidden` with `credential_revoked` or `credential_expired` error code.
+
+---
+
+## 7. Consumer Credential Delivery
+
+### 7.1 How Consumers Retrieve Credentials
+
+After a resource is realized with an associated credential, the consumer receives the `credential_record` metadata in the realized entity response. The actual credential value is retrieved separately via `value_retrieval_endpoint`.
+
+```
+GET /api/v1/resources/{entity_uuid}/credentials
+
+Response 200:
+{
+  "credentials": [
+    {
+      "credential_uuid": "",
+      "credential_type": "ssh_key",
+      "status": "active",
+      "issued_at": "",
+      "valid_until": "",
+      "scope": {
+        "operations": ["ssh_access"],
+        "entity_uuid": ""
+      },
+      "retrieval": {
+        "endpoint": "/api/v1/credentials//value",
+        "auth_required": "step_up_mfa",    # none | bearer_token | step_up_mfa | mtls
+        "retrieval_count": 1,              # how many times value has been retrieved
+        "last_retrieved_at": ""
+      },
+      "rotation_schedule": {
+        "next_rotation_at": "",
+        "rotation_trigger": "scheduled",
+        "transition_window_days": 1
+      }
+    }
+  ]
+}
+```
+
+### 7.2 Credential Value Retrieval
+
+```
+GET /api/v1/credentials/{credential_uuid}/value
+Authorization: Bearer 
+X-DCM-StepUp-Token:   # if auth_required: step_up_mfa
+
+Response 200:
+{
+  "credential_uuid": "",
+  "credential_type": "ssh_key",
+  "value": {
+    "private_key": "-----BEGIN OPENSSH PRIVATE KEY-----\n...",
+    "public_key": "ssh-ed25519 AAAA... dcm-issued@entity-",
+    "username": "dcm-provisioned"
+  },
+  "valid_until": "",
+  "retrieval_uuid": ""   # idempotency key for this retrieval event; audited
+}
+
+Response 404:  credential_uuid not found or not associated with an entity the actor owns
+Response 403:  step_up_mfa required but not completed
+Response 410:  credential revoked or expired
+```
+
+Every value retrieval is audited: credential_uuid, actor_uuid, retrieved_at, retrieval_uuid.
+
+---
+
+## 8. Credential Provider API Contract
+
+The full endpoint contract that all Credential Providers must implement.
+
+### 8.1 Issue Credential
+
+```
+POST {issue_endpoint}
+
+Request:
+{
+  "credential_type": "ssh_key",
+  "issued_to": {
+    "actor_uuid": "",
+    "entity_uuid": "",
+    "component_uuid": "",
+    "provider_uuid": ""
+  },
+  "scope": {
+    "operations": ["ssh_access"],
+    "resource_types": ["Compute.VirtualMachine"],
+    "tenant_uuid": ""
+  },
+  "valid_until": "",
+  "non_transferable": true,
+  "bound_to_ip": "",
+  "rotation_of": "",
+  "issuing_request_uuid": "",
+  "entity_uuid": ""
+}
+
+Response 201 Created:
+{
+  "credential_uuid": "",
+  "credential_type": "ssh_key",
+  "issued_at": "",
+  "valid_until": "",
+  "value_retrieval_endpoint": "",
+  "value_retrieval_auth": "step_up_mfa",
+  "metadata": {}   # provider-specific additional metadata
+}
+
+Response 422:  unsupported credential type
+Response 403:  issued_to scope exceeds provider's declared authority
+```
+
+### 8.2 Rotate Credential
+
+```
+POST {rotate_endpoint}
+
+Request:
+{
+  "credential_uuid": "",           # credential being rotated
+  "rotation_trigger": "pre_expiry | scheduled | security_event | actor_request",
+  "transition_window": "P1D",            # how long old credential remains valid
+  "new_valid_until": ""
+}
+
+Response 200:
+{
+  "old_credential_uuid": "",
+  "new_credential_uuid": "",
+  "new_valid_until": "",
+  "old_credential_revokes_at": "",   # end of transition window
+  "new_value_retrieval_endpoint": ""
+}
+```
+
+### 8.3 Revoke Credential
+
+```
+DELETE {revoke_endpoint}/{credential_uuid}
+
+Request body:
+{
+  "revocation_trigger": "actor_deprovisioned | entity_decommissioned | security_event | ...",
+  "revocation_reason": "",
+  "effective_immediately": true          # false = honor transition window if rotating
+}
+
+Response 200:
+{
+  "credential_uuid": "",
+  "revoked_at": "",
+  "effective_immediately": true
+}
+
+Response 404: credential not found
+Response 409: credential already revoked
+```
+
+### 8.4 Validate Credential (Use-Time Check)
+
+```
+POST {validate_endpoint}
+
+Request:
+{
+  "credential_uuid": "",
+  "operation_type": "dispatch",
+  "entity_uuid": "",
+  "provider_uuid": ""
+}
+
+Response 200:
+{
+  "valid": true,
+  "expires_in_seconds": 423
+}
+
+Response 200 (invalid):
+{
+  "valid": false,
+  "reason": "revoked | expired | scope_mismatch | ip_binding_failed"
+}
+```
+
+### 8.5 List Credentials for Entity
+
+```
+GET {list_endpoint}?entity_uuid=&status=active
+
+Response 200:
+{
+  "credentials": [
+    {
+      "credential_uuid": "",
+      "credential_type": "ssh_key",
+      "status": "active",
+      "issued_to": {...},
+      "valid_until": ""
+    }
+  ]
+}
+```
+
+---
+
+## 9. Credential Provider Registration
+
+```yaml
+credential_provider_capabilities:
+  # Credential types this provider can issue
+  credential_types:
+    - api_key
+    - x509_certificate
+    - ssh_key
+    - service_account_token
+    - database_password
+    - kubeconfig
+    - hsm_backed_key
+    - dcm_interaction         # must declare if provider handles DCM interaction creds
+
+  # Secret engine backing (for audit and accreditation)
+  secret_engines:
+    - vault                   # HashiCorp Vault
+    - aws_secrets_manager
+    - azure_key_vault
+    - gcp_secret_manager
+    - local_hsm               # sovereign deployments
+
+  # Security properties
+  hsm_backed: false           # true if all keys are HSM-protected
+  fips_140_2_level: 0         # 0=none, 1, 2, or 3
+  dynamic_secrets: true       # can generate credentials on demand (not just store/retrieve)
+
+  # Rotation capabilities
+  rotation_support: true
+  min_transition_window: PT5M
+  max_transition_window: P7D
+  supported_rotation_triggers:
+    - pre_expiry
+    - scheduled
+    - security_event
+    - actor_request
+
+  # Revocation SLA (how quickly revocations take effect)
+  revocation_sla: PT5M        # standard; PT1M for fsi/sovereign
+
+  # Endpoints (all relative to provider base URL)
+  endpoints:
+    issue:    /v1/credentials
+    rotate:   /v1/credentials/rotate
+    revoke:   /v1/credentials/{uuid}
+    validate: /v1/credentials/validate
+    list:     /v1/credentials
+```
+
+---
+
+## 10. Credential Lifecycle State Machine
+
+```
+                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+        issuance    โ”‚              โ”‚    expiry / explicit
+  โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บโ”‚    ACTIVE    โ”‚โ”€โ”€โ”€โ”€revocationโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–บ  REVOKED / EXPIRED
+                    โ”‚              โ”‚
+                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+                           โ”‚ rotation initiated
+                           โ–ผ
+                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+                    โ”‚   ROTATING   โ”‚  both old and new valid
+                    โ”‚              โ”‚  during transition window
+                    โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+                           โ”‚ transition window ends
+                           โ”‚ or emergency revocation
+                           โ–ผ
+                       REVOKED
+```
+
+State transitions and their audit requirements:
+
+| Transition | Audited fields |
+|-----------|---------------|
+| issued โ†’ ACTIVE | credential_uuid, type, issued_to, scope, valid_until, issuing_request_uuid |
+| ACTIVE โ†’ ROTATING | rotation_trigger, old_uuid, new_uuid, transition_window |
+| ROTATING โ†’ ACTIVE (new) | new credential activated after old revoked |
+| ACTIVE/ROTATING โ†’ REVOKED | revocation_trigger, revoked_by, effective_at, reason |
+| ACTIVE โ†’ EXPIRED | expired_at (system record, no actor) |
+
+---
+
+---
+
+## 12. Profile-Governed Credential Configuration
+
+Every credential security dimension is controlled by the active profile. This is the single authoritative configuration point โ€” a homelab deployment requires minimal configuration and implementation effort; an FSI or sovereign deployment gets full enforcement without per-deployment policy authoring.
+
+### 12.1 Credential Profile Configuration Block
+
+```yaml
+credential_profile:
+
+  # --- Credential Type Restrictions ---
+  # Which credential types are permitted. Omitted types are rejected at issuance.
+  permitted_credential_types:
+    minimal:   [api_key, x509_certificate, ssh_key, service_account_token, database_password]
+    dev:       [api_key, x509_certificate, ssh_key, service_account_token, database_password, kubeconfig]
+    standard:  [api_key, x509_certificate, ssh_key, service_account_token, database_password, kubeconfig]
+    prod:      [api_key, x509_certificate, ssh_key, service_account_token, database_password, kubeconfig]
+    fsi:       [x509_certificate, ssh_key, service_account_token, database_password, kubeconfig, hsm_backed_key]
+    sovereign: [x509_certificate, hsm_backed_key]     # all credentials must be hardware-backed
+
+  # --- Lifetime Limits ---
+  # Maximum valid_until per credential type. Provider may issue shorter; never longer.
+  max_lifetime:
+    #               minimal   dev     standard  prod    fsi     sovereign
+    api_key:        [P365D,   P90D,   P90D,     P30D,   โ€”,      โ€”]
+    x509_certificate:[P365D,  P365D,  P365D,    P180D,  P90D,   P90D]
+    ssh_key:        [P365D,   P90D,   P90D,     P30D,   P30D,   P30D]
+    service_account_token: [PT24H, PT24H, PT24H, PT12H, PT4H,  PT1H]
+    database_password: [P365D, P90D,  P90D,     P30D,   P30D,   โ€”]
+    kubeconfig:     [P365D,   P30D,   P30D,     P14D,   P7D,    โ€”]
+    dcm_interaction:[PT1H,    PT30M,  PT1H,     PT30M,  PT15M,  PT15M]
+    hsm_backed_key: [โ€”,       โ€”,      โ€”,        P365D,  P180D,  P90D]
+
+  # --- Rotation ---
+  max_rotation_interval:        # PCI DSS req 8.3.9: 90-day maximum for regulated profiles
+    standard:   P365D           # no enforcement; provider may choose longer
+    prod:       P90D            # enforced; rotation older than P90D triggers alert
+    fsi:        P90D            # enforced; PCI DSS compliance
+    sovereign:  P90D            # enforced
+  scheduled_rotation_required:
+    # Security-first: rotation is architecturally required in ALL profiles.
+    # What varies is the maximum interval, automation level, and trigger mechanism.
+    minimal:    true    # required; manual trigger acceptable; P365D max interval
+    dev:        true    # required; manual trigger acceptable; P180D max interval
+    standard:   true    # required; automated pre-expiry rotation
+    prod:       true    # required; automated; strict interval enforcement
+    fsi:        true    # required; automated; P90D max (PCI DSS)
+    sovereign:  true    # required; automated; hardware-triggered rotation
+  min_transition_window:
+    minimal:    PT0S            # homelab: immediate cutover acceptable
+    dev:        PT1H
+    standard:   P1D
+    prod:       P1D
+    fsi:        P1D             # PT15M for dcm_interaction
+    sovereign:  P1D             # PT15M for dcm_interaction
+
+  # --- Value Retrieval Security ---
+  value_retrieval_auth_required:
+    minimal:    bearer_token    # session token sufficient for homelab
+    dev:        bearer_token
+    standard:   bearer_token    # step_up_mfa for sensitive types (ssh_key, database_password)
+    prod:       step_up_mfa     # all credential types require step-up
+    fsi:        step_up_mfa     # hardware token MFA required
+    sovereign:  mtls            # mutual TLS + hardware attestation
+  step_up_sensitive_types:      # standard profile: step_up_mfa for these types even without full profile enforcement
+    - ssh_key
+    - database_password
+    - kubeconfig
+    - hsm_backed_key
+
+  # --- Retrieval Audit ---
+  audit_every_retrieval:
+    # Security-first: FIRST retrieval is always audited in ALL profiles (CPX-005).
+    # audit_every_retrieval controls whether SUBSEQUENT retrievals are also audited.
+    # audit_first_retrieval is always true regardless of this setting.
+    minimal:    false           # subsequent retrievals silent; first always audited
+    dev:        false           # subsequent retrievals silent; first always audited
+    standard:   true            # every retrieval audited
+    prod:       true
+    fsi:        true
+    sovereign:  true
+  idle_detection_threshold:     # alert if credential not retrieved within N after issuance
+    # Security-first: idle detection is on in ALL profiles. Threshold varies.
+    # Alert is notification-only; never blocks. No operational burden.
+    minimal:    P30D            # generous; homelab credentials may sit unused longer
+    dev:        P14D
+    standard:   P7D
+    prod:       P3D
+    fsi:        P1D
+    sovereign:  PT12H
+
+  # --- Network Binding ---
+  ip_binding_required:
+    minimal:    false
+    dev:        false
+    standard:   false           # optional; recommended for prod
+    prod:       false           # optional; recommended
+    fsi:        true            # mandatory
+    sovereign:  true            # mandatory
+
+  # --- Cryptographic Requirements ---
+  fips_140_level_required:
+    minimal:    0               # no requirement
+    dev:        0
+    standard:   0
+    prod:       1               # Level 1: software-only acceptable
+    fsi:        2               # Level 2: role-based authentication required
+    sovereign:  3               # Level 3: physical tamper evidence + identity-based auth
+  approved_algorithms:
+    minimal:              # negative list: anything not forbidden is permitted
+      forbidden_algorithms: [MD5, SHA-1, DES, 3DES, RC4, RSA-1024, RSA-512, DSA-1024]
+      # No weak/broken algorithms even in homelab. Real attacks hit all deployments.
+    standard:
+      api_key:              [random_256bit]
+      x509_certificate:     [RSA-4096, ECDSA-P-384, Ed25519]
+      ssh_key:              [Ed25519, ECDSA-P-384]
+      service_account_token:[HS256, RS256, ES256]
+      database_password:    [random_128bit_printable]
+    prod:                   # same as standard; provider must declare algorithm in credential record
+      inherits: standard
+    fsi:
+      x509_certificate:     [RSA-4096, ECDSA-P-384]    # Ed25519 not FIPS-approved in 140-2
+      ssh_key:              [RSA-4096, ECDSA-P-384]
+      service_account_token:[RS256, ES256]
+      database_password:    [random_256bit]
+    sovereign:
+      inherits: fsi
+      all_types:            hsm_backed_only              # all keys generated and stored in HSM
+
+  # --- Revocation ---
+  revocation_check_frequency:   # how often components must refresh revocation cache
+    minimal:    PT5M            # lazy; acceptable for homelab
+    dev:        PT5M
+    standard:   PT1M
+    prod:       PT1M
+    fsi:        PT30S
+    sovereign:  PT15S
+  revocation_sla:               # how quickly Credential Provider must invalidate on revocation
+    minimal:    PT10M
+    dev:        PT5M
+    standard:   PT5M
+    prod:       PT2M
+    fsi:        PT1M
+    sovereign:  PT30S
+```
+
+### 12.2 Authenticator Assurance Levels (NIST 800-63B Mapping)
+
+DCM profile credential requirements map to NIST 800-63B Authenticator Assurance Levels:
+
+| Profile | AAL | What it means |
+|---------|-----|--------------|
+| `minimal` | AAL1 | Single-factor; bearer token sufficient for credential retrieval |
+| `dev` | AAL1 | Same as minimal; shorter lifetimes |
+| `standard` | AAL2 | MFA required for sensitive credential retrieval (ssh_key, database_password, kubeconfig) |
+| `prod` | AAL2 | MFA required for all credential retrieval |
+| `fsi` | AAL2+ | Hardware MFA token required; FIPS 140-2 Level 2 modules |
+| `sovereign` | AAL3 | Hardware-bound authenticator; FIPS 140-2 Level 3; physical tamper evidence |
+
+### 12.3 Compliance Domain Overlays
+
+When a compliance domain is active, its credential requirements are **additive** to the profile base:
+
+```yaml
+compliance_credential_overlays:
+  hipaa:
+    min_key_size_bits: 256
+    max_lifetime_override:
+      api_key: P90D           # HIPAA requires rotation at least annually; 90-day recommended
+    audit_every_retrieval: true   # all PHI-adjacent credential access audited
+    idle_detection_threshold: P7D
+
+  pci_dss:
+    max_rotation_interval: P90D   # PCI DSS req 8.3.9 โ€” mandatory
+    min_password_complexity:
+      database_password:
+        length: 12
+        character_classes: 4    # upper, lower, digit, special
+    idle_detection_threshold: P30D
+
+  fedramp_moderate:
+    fips_140_level_required: 1
+    approved_algorithms:
+      inherits: standard
+
+  fedramp_high:
+    fips_140_level_required: 2
+    approved_algorithms:
+      inherits: fsi
+    ip_binding_required: true
+
+  dod_il4:
+    fips_140_level_required: 2
+    ip_binding_required: true
+    max_lifetime_override:
+      dcm_interaction: PT10M
+      service_account_token: PT1H
+```
+
+### 12.4 Design Priority and Implementation Consistency Principle
+
+The DCM design priority order applies directly to credential management:
+
+1. **Security first:** Security properties โ€” value separation, rotation, audit, idle detection, algorithm baselines, revocation โ€” are architecturally present in ALL profiles. What profiles control is enforcement strictness, threshold values, and automation level. A `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security."
+
+2. **Ease of use second:** The secure path must be the easy path. Homelab deployments use the same API contract, same data model, and same provider interface as sovereign deployments. The profile system eliminates the need to choose between security and operational simplicity.
+
+3. **Extensibility third:** Compliance domain overlays, profile overrides, and algorithm configuration make the credential model adaptable without code changes.
+
+Profile variation applies only to **enforcement level and required features** โ€” never to the underlying protocol or data model. A credential issued under the `minimal` profile has the same data structure, the same API contract, the same revocation mechanism, and the same audit record format as one issued under the `sovereign` profile. What differs is what is required vs optional.
+
+This means:
+- A Credential Provider built for a homelab deployment is compatible with a production deployment โ€” it just needs to demonstrate it satisfies the production profile's requirements
+- Migration from `dev` to `prod` profile does not require replacing the Credential Provider or re-issuing credentials under a different protocol โ€” it triggers more conservative enforcement of the same model
+- Testing and tooling built against the `dev` profile works against `sovereign` profile with the same interfaces
+
+**CPX-001 (values never in DCM stores) is non-negotiable in every profile including `minimal`.** This is the one property that does not scale down. It is the security property that makes the entire model trustworthy regardless of deployment size.
+
+---
+
+## 13. Cryptographic Algorithm Requirements
+
+### 13.1 Algorithm Declaration in Credential Record
+
+The credential record is extended with two new fields:
+
+```yaml
+credential_record:
+  # ... existing fields ...
+  algorithm: Ed25519 | ECDSA-P-384 | RSA-4096 | HS256 | RS256 | random_256bit | ...
+  key_usage: [authentication, signing, encryption]   # declared at issuance; non-overlapping
+  retrieved_count_threshold: 48           # hours after issuance before idle alert fires
+```
+
+`key_usage` enforces the principle of algorithm agility and purpose separation. A credential issued for `authentication` cannot be used for `signing` even if the underlying algorithm supports both. The Credential Provider must validate key_usage at the validate endpoint.
+
+### 13.2 Approved Algorithm Defaults (Standard Profile)
+
+| Credential Type | Algorithm | Key Size |
+|----------------|-----------|----------|
+| `api_key` | Cryptographically random | 256 bits minimum |
+| `x509_certificate` | Ed25519 or ECDSA P-384 | Ed25519: 256-bit; P-384: 384-bit |
+| `ssh_key` | Ed25519 (preferred), ECDSA P-384 | Ed25519: 256-bit |
+| `service_account_token` | RS256 or ES256 | RSA: 4096-bit; EC: P-256 |
+| `database_password` | Cryptographically random | 128-bit printable minimum |
+| `kubeconfig` | As per cluster's auth configuration | โ€” |
+| `hsm_backed_key` | ECDSA P-384 or RSA-4096 | HSM-generated |
+| `dcm_interaction` | HS256 or ES256 | AES-256 or P-256 |
+
+### 13.3 Key Escrow Policy
+
+DCM does not implement key escrow by default. For `sovereign` profile deployments, key escrow (if required by regulation) is declared in the Credential Provider's capability registration and governed by the provider โ€” DCM's role is to audit that escrowed credentials are disclosed only via the standard revocation and access model:
+
+```yaml
+credential_provider_capabilities:
+  key_escrow:
+    supported: false           # default; no escrow
+    # If true:
+    escrow_model: m_of_n       # m-of-n key shares; Shamir's Secret Sharing
+    escrow_quorum: "3 of 5"
+    escrow_record_stored_by: hsm   # never by DCM
+    dcm_role: audit_only           # DCM audits escrow access; does not participate
+```
+
+---
+
+## 14. Idle Credential Detection
+
+A credential issued but never retrieved within the declared threshold is a security signal โ€” it may indicate a provisioning error, a failed delivery, or an abandoned resource.
+
+```yaml
+idle_credential_record:
+  credential_uuid: 
+  issued_at: 
+  threshold_hours: 48           # from profile credential_profile.idle_detection_threshold
+  last_checked_at: 
+  retrieval_count: 0
+  status: idle_alert_pending
+```
+
+When an idle alert fires:
+- Platform admin notified: "Credential {uuid} for entity {entity_uuid} has not been retrieved in {N} hours"
+- Consumer notified (if consumer exists): "Your credential for {resource_name} has not been accessed โ€” confirm delivery"
+- Credential is NOT automatically revoked โ€” it remains valid until its `valid_until`
+- If still idle after 2ร— the threshold: optional auto-revocation per profile configuration
+
+---
+
+
+## 11. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `CPX-001` | Credential values are never stored in DCM's data model, GitOps stores, Realized State Store, or Audit Store. Only credential metadata (UUID, type, scope, expiry, status) is stored in DCM. |
+| `CPX-002` | Every DCM interaction with a provider must present a scoped, short-lived `dcm_interaction` credential. A provider that receives an interaction without a valid scoped credential must reject it with `403 Forbidden`. |
+| `CPX-003` | Credential revocation must propagate to the Credential Revocation Registry within the declared `revocation_sla`. Components must refresh their revocation cache no less frequently than the profile-governed cache TTL (PT1M standard; PT30S fsi/sovereign). |
+| `CPX-004` | Emergency rotation (security_event trigger) has no transition window. The old credential is revoked immediately. The new credential is delivered via the fastest available Notification Provider channel. |
+| `CPX-005` | The first credential value retrieval is audited in ALL profiles (credential_uuid, actor_uuid, retrieved_at, retrieval_uuid). Subsequent retrievals are audited in standard+ profiles. Emergency retrievals (rotation, security event) are always audited regardless of profile. |
+| `CPX-006` | Actor deprovisioning (via SCIM or manual) triggers immediate revocation of all credentials issued to that actor. Revocation events are published to the Message Bus before the deprovisioning event is acknowledged. |
+| `CPX-007` | Entity decommissioning triggers revocation of all credentials scoped to that entity before the decommission is confirmed. A decommission that cannot revoke all credentials enters `COMPENSATION_IN_PROGRESS` state. |
+| `CPX-008` | Credentials issued for `fsi` and `sovereign` profiles must be IP-bound (`bound_to_ip`) or hardware-attested (`hsm_backed_key`). Unbound credentials are rejected by the Governance Matrix for these profiles. |
+| `CPX-009` | `algorithm` and `key_usage` must be declared on every credential record at issuance (standard+ profiles). The Credential Provider must validate `key_usage` at the validate endpoint โ€” a credential issued for `authentication` cannot be used for `signing`. |
+| `CPX-010` | Idle credential detection fires at the profile-governed threshold. Idle credentials are NOT automatically revoked โ€” they trigger notification only. Auto-revocation after 2ร— threshold is profile-configurable. |
+| `CPX-011` | Profile credential requirements are additive when compliance domains are active (HIPAA, PCI DSS, FedRAMP, DoD IL4). Compliance overlay requirements always tighten, never relax, the base profile. |
+| `CPX-012` | CPX-001 (values never in DCM stores) applies in ALL profiles including `minimal`. There is no profile that permits credential values to be stored in DCM. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+## External CA Integration
+
+DCM's Credential Provider model natively supports external Certificate Authorities as backends for the `x509_certificate` credential type. This is the correct place for enterprise PKI integration โ€” not the Auth Provider.
+
+### Supported Protocols
+
+| Protocol | RFC | Common Implementations | Use case |
+|----------|-----|------------------------|----------|
+| ACME | RFC 8555 | Let's Encrypt, cert-manager, Venafi, DigiCert | Public and enterprise CAs with ACME support |
+| EST | RFC 7030 | Cisco CA, Microsoft NDES, Venafi | Enterprise PKI, IoT, internal use |
+| SCEP | RFC 8894 | Microsoft NDES, Cisco iOS CA | Legacy enterprise PKI, network equipment |
+| CMP | RFC 4210 | EJBCA, OpenXPKI | High-assurance enterprise PKI |
+| Native API | โ€” | HashiCorp Vault PKI, AWS ACM PCA, Azure Key Vault | Cloud-native PKI |
+
+### External CA Registration
+
+```yaml
+credential_provider_registration:
+  provider_type: credential_provider
+  credential_types: [x509_certificate]
+  
+  external_ca_config:
+    ca_protocol: acme | est | scep | cmp | vault_pki | aws_acm_pca | azure_key_vault
+    ca_endpoint: 
+    
+    # Protocol-specific
+    acme_config:
+      directory_url: 
+      account_key_credential_uuid: 
+      preferred_challenge: dns-01 | http-01 | tls-alpn-01
+      
+    vault_pki_config:
+      vault_addr: 
+      mount_path: pki
+      role_name: dcm-internal
+      vault_token_credential_uuid: 
+      
+    # Common to all
+    ca_chain_pem:   # for trust store installation
+    issued_cert_lifetime: P90D              # profile-governed; may be overridden by CA
+    subject_template: "CN={{component_type}}-{{component_uuid}},O=dcm-internal"
+```
+
+### How DCM Uses External CA Credential Providers
+
+When an external CA Credential Provider is registered and configured as the trust anchor for internal component auth (doc 36), DCM's component certificate requests flow through the Credential Provider interface instead of the built-in Internal CA:
+
+```
+Component needs certificate
+  โ”‚
+  โ–ผ Request to Credential Provider Proxy
+  โ”‚   credential_type: x509_certificate
+  โ”‚   subject: CN=-,O=dcm-internal
+  โ”‚   san: [component_uuid, component_name, dns_name]
+  โ”‚
+  โ–ผ Credential Provider Proxy โ†’ External CA Credential Provider
+  โ”‚   Issues certificate request via configured protocol (ACME/EST/Vault/etc.)
+  โ”‚
+  โ–ผ CA issues certificate (signed by enterprise root)
+  โ”‚
+  โ–ผ Certificate returned to component
+  โ”‚   Component uses for mTLS โ€” same as built-in CA path
+  โ”‚   Certificate in enterprise PKI chain โ†’ auditable in enterprise tooling
+```
+
+This design means DCM's internal mTLS is fully auditable through existing enterprise PKI infrastructure when using an external CA โ€” a key requirement for fsi and sovereign profiles.
+
diff --git a/content/docs/data-model/32-authority-tier-model.md b/content/docs/data-model/32-authority-tier-model.md
new file mode 100644
index 0000000..e195758
--- /dev/null
+++ b/content/docs/data-model/32-authority-tier-model.md
@@ -0,0 +1,527 @@
+# DCM Data Model โ€” Authority Tier Model
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Authority Tier Specification
+**Related Documents:** [Design Priorities](00-design-priorities.md) | [Scoring Model](29-scoring-model.md) | [Federated Contribution Model](28-federated-contribution-model.md) | [Policy Profiles](14-policy-profiles.md) | [Registry Governance](20-registry-governance.md)
+
+> **This document maps to: DATA + POLICY**
+>
+> The authority tier list is Data โ€” a versioned, ordered registry entry. Tiers are referenced by name in Policies (scoring thresholds, contribution approval requirements, registration requirements). The ordered list resolves numeric weight at evaluation time.
+
+---
+
+## 1. The Core Model
+
+### 1.1 What an Authority Tier Is
+
+An authority tier declares the **required level of organizational decision gravity** for an action. It answers: "how consequential is this decision, and therefore how much authority must be engaged to approve it?"
+
+Tiers do **not** prescribe organizational mechanisms โ€” who satisfies a tier, how many people are involved, or what tools they use. That is entirely the organization's definition. Tiers provide the vocabulary and enforcement gate; organizations provide the substance.
+
+### 1.2 The Ordered List
+
+Authority tiers are defined as a **named, ordered list**. Position in the list determines numeric weight. Names are stable references used throughout the system. New tiers can be inserted anywhere without changing existing names or breaking existing references.
+
+```yaml
+authority_tier_registry:
+  version: "1.0.0"
+  tiers:
+    - name: auto
+      position: 1              # derived from list order; do not hardcode
+      decision_gravity: none
+      description: >
+        No human judgment required. System confidence โ€” scoring, validation,
+        governance matrix checks โ€” is sufficient to proceed. DCM activates
+        automatically on pass.
+      dcm_gate: All structural and governance validation checks pass
+      organization_provides: Nothing โ€” fully automated
+      dcmgroup_required: false
+
+    - name: reviewed
+      position: 2
+      decision_gravity: routine
+      description: >
+        Standard authority. A qualified reviewer in the relevant domain
+        must evaluate and record a decision. Routine operational decisions
+        that benefit from human oversight but do not require elevated authority.
+      dcm_gate: One actor with reviewer role records a decision via Admin API
+      organization_provides: >
+        Who constitutes a qualified reviewer for this action type;
+        the review process; recording via DCM Admin API or external system
+      dcmgroup_required: false
+      typical_use: Standard request approval; routine policy contributions; dev/standard provider registration
+
+    - name: verified
+      position: 3
+      decision_gravity: elevated
+      description: >
+        Elevated authority. Two independent, distinct reviewers must each
+        evaluate and record a decision. Enforces separation of duties โ€”
+        the same actor cannot satisfy both requirements. Used for decisions
+        with operational or security significance requiring independent confirmation.
+      dcm_gate: Two distinct actors with reviewer role each record a decision via Admin API
+      organization_provides: >
+        Who constitutes qualified reviewers; both review processes;
+        may use external workflow tools that call the DCM Admin API
+      dcmgroup_required: false
+      typical_use: High-risk provider registration; elevated-score requests; significant policy changes
+
+    - name: authorized
+      position: 4
+      decision_gravity: critical
+      description: >
+        Highest authority weight. Reserved for decisions with organizational,
+        regulatory, or security consequence requiring the highest level of
+        deliberate authorization. Who constitutes sufficient authority is
+        entirely the organization's definition โ€” a CTO, a CISO and legal
+        counsel, a change advisory board, a single person with delegated
+        authority. DCM enforces that the declared authority group engaged
+        and recorded their decision; it does not prescribe the group structure
+        or deliberation process.
+      dcm_gate: N members of a declared DCMGroup record decisions via Admin API (quorum threshold)
+      organization_provides: >
+        Authority group composition (declared as a DCMGroup);
+        quorum threshold (N of M); deliberation process; external tools
+        (ServiceNow, Jira, Slack bots may call Admin API on behalf of members)
+      dcmgroup_required: true
+      typical_use: Governance matrix changes; sovereign-profile actions; credential provider registration; federation policy
+```
+
+### 1.3 Numeric Weight Resolution
+
+The numeric weight of a tier is its **position in the ordered list**, resolved at evaluation time. It is never stored as a hardcoded number in configuration.
+
+```
+Given the default list: auto(1) โ†’ reviewed(2) โ†’ verified(3) โ†’ authorized(4)
+
+If an organization inserts a custom tier:
+  auto(1) โ†’ reviewed(2) โ†’ verified(3) โ†’ compliance_reviewed(4) โ†’ authorized(5)
+
+DCM resolves:
+  weight("reviewed")         = 2
+  weight("verified")         = 3
+  weight("compliance_reviewed") = 4
+  weight("authorized")       = 5
+
+All existing references to "authorized" continue to work.
+No configuration changes required for existing tiers.
+```
+
+### 1.4 decision_gravity Vocabulary
+
+`decision_gravity` is a stable, position-independent classification used by the scoring model and profile system to reason about tier severity without depending on tier names. It is declared on each tier and must be assigned when creating custom tiers.
+
+| Value | Meaning | Default DCM tiers |
+|-------|---------|------------------|
+| `none` | Automated; no human judgment | `auto` |
+| `routine` | Standard operational decision | `reviewed` |
+| `elevated` | Significant decision; separation of duties | `verified` |
+| `critical` | Highest consequence; maximum authority | `authorized` |
+
+Organizations creating custom tiers must assign one of these four gravity values. If a future need arises for a gravity level between `elevated` and `critical`, the vocabulary can be extended โ€” but this is a DCM-level change, not an organization-level one.
+
+---
+
+## 2. Custom Tier Definition
+
+### 2.1 How Organizations Add Tiers
+
+Organizations can extend the authority tier list by contributing custom tier definitions through the standard contribution pipeline. Custom tiers are contributed at the organization or tenant domain scope.
+
+```yaml
+custom_tier_contribution:
+  name: compliance_reviewed          # unique within the deployment
+  insert_after: verified             # position declaration โ€” inserts after this tier
+  decision_gravity: elevated         # must match or be consistent with position
+  description: >
+    Elevated authority with mandatory compliance officer sign-off.
+    Required for actions affecting regulated data domains (PII, PCI, HIPAA).
+    The compliance officer may be one person or a designated compliance team;
+    the organization defines who satisfies this role.
+  dcm_gate: One actor with compliance_officer role records a decision via Admin API
+  organization_provides: >
+    Who holds the compliance_officer role; compliance review process;
+    may be satisfied by external GRC system calling Admin API
+  dcmgroup_required: false           # single reviewer sufficient at this gravity
+  applicable_profiles: [standard, prod, fsi, sovereign]
+  contribution_requires: verified    # adding a custom tier requires verified-tier approval
+```
+
+### 2.2 Contribution Approval Requirements
+
+Custom tier contributions require `verified` tier approval (two independent reviewers) because they affect all pipeline decisions in the deployment. An organization cannot unilaterally add a tier that demotes an existing gravity level or bypasses the `authorized` tier for critical decisions.
+
+**Constraints on custom tiers:**
+- `decision_gravity` must be consistent with position (a tier inserted before `reviewed` cannot have `critical` gravity)
+- Custom tiers cannot be inserted before `auto` or after the highest `critical` gravity tier
+- A custom tier with `dcmgroup_required: true` must declare a valid DCMGroup at contribution time
+- Custom tiers cannot change the `dcm_gate` semantics of existing DCM tiers
+
+### 2.3 External Tier Registries
+
+For federation deployments, peer DCM instances may have different custom tier lists. When a federated request requires approval from a peer's tier, DCM resolves the equivalent gravity level from the local list:
+
+```yaml
+federation_tier_resolution:
+  strategy: gravity_match            # match by decision_gravity, not tier name
+  on_unknown_tier: escalate_to_gravity  # if peer tier unknown, use its declared gravity
+  fallback_tier: authorized          # if gravity unknown, apply highest local tier
+```
+
+---
+
+## 3. Profile Threshold Configuration
+
+### 3.1 Dynamic Threshold Format
+
+Profile approval thresholds are expressed as a named-tier list, not fixed column headers. This allows the threshold table to accommodate custom tiers without restructuring.
+
+```yaml
+# Standard profile โ€” default DCM tiers
+approval_routing:
+  thresholds:
+    - tier: auto
+      max_score: 24          # score 0โ€“24: auto-approve
+    - tier: reviewed
+      max_score: 59          # score 25โ€“59: reviewed tier required
+    - tier: verified
+      max_score: 79          # score 60โ€“79: verified tier required
+    - tier: authorized
+      max_score: 100         # score 80โ€“100: authorized tier required
+
+# Organization adds compliance_reviewed between verified and authorized
+approval_routing:
+  thresholds:
+    - tier: auto
+      max_score: 24
+    - tier: reviewed
+      max_score: 59
+    - tier: verified
+      max_score: 74          # adjusted to make room
+    - tier: compliance_reviewed
+      max_score: 84          # new tier occupies this range
+    - tier: authorized
+      max_score: 100         # unchanged name; adjusted range
+```
+
+### 3.2 Default Profile Thresholds
+
+```yaml
+profile_approval_thresholds:
+  minimal:
+    - { tier: auto,       max_score: 44 }
+    - { tier: reviewed,   max_score: 100 }
+
+  dev:
+    - { tier: auto,       max_score: 39 }
+    - { tier: reviewed,   max_score: 69 }
+    - { tier: verified,   max_score: 100 }
+
+  standard:
+    - { tier: auto,       max_score: 24 }
+    - { tier: reviewed,   max_score: 59 }
+    - { tier: verified,   max_score: 79 }
+    - { tier: authorized, max_score: 100 }
+
+  prod:
+    - { tier: auto,       max_score: 14 }
+    - { tier: reviewed,   max_score: 49 }
+    - { tier: verified,   max_score: 74 }
+    - { tier: authorized, max_score: 100 }
+
+  fsi:
+    - { tier: auto,       max_score: 9 }
+    - { tier: reviewed,   max_score: 39 }
+    - { tier: verified,   max_score: 69 }
+    - { tier: authorized, max_score: 100 }
+
+  sovereign:
+    - { tier: auto,       max_score: 4 }
+    - { tier: reviewed,   max_score: 29 }
+    - { tier: verified,   max_score: 59 }
+    - { tier: authorized, max_score: 100 }
+```
+
+### 3.3 SMX-008 in the Dynamic Model
+
+SMX-008 (auto_approve_below โ‰ค 50) remains a hard constraint. In the dynamic model: the `auto` tier's `max_score` may never exceed 50 in any profile, regardless of custom tier additions. This constraint applies to the `auto` tier specifically, not to named thresholds.
+
+---
+
+## 4. Tier Evaluation in the Pipeline
+
+### 4.1 How DCM Resolves the Required Tier
+
+At request evaluation time, DCM:
+
+1. Computes the request risk score (0โ€“100) from the scoring model
+2. Loads the active profile's threshold list
+3. Walks the list in order; the first tier whose `max_score โ‰ฅ risk_score` is the required tier
+4. Resolves the numeric weight of the required tier from the ordered tier list
+5. Creates an approval record with the required tier name and weight
+
+The tier name โ€” not the weight โ€” is what is stored in the approval record and what reviewers see. The weight is used for comparison operations (e.g., "is this action at least as significant as `verified`?").
+
+### 4.2 Approval Record
+
+```yaml
+approval_record:
+  approval_uuid: 
+  subject_uuid: 
+  subject_type: request | policy_contribution | provider_registration | federation_contribution
+  required_tier: verified           # tier name โ€” stable reference
+  required_tier_weight: 3           # resolved at creation; stored for point-in-time audit
+  required_tier_gravity: elevated
+  dcmgroup_uuid:       # non-null only for dcmgroup_required: true tiers
+  quorum_threshold: 
+  status: pending_reviewed | pending_verified | pending_authorized | pending_
+  created_at: 
+  window_expires_at: 
+  decisions: []
+```
+
+---
+
+## 5. DCMGroup Assignment for Authorized Tier
+
+When a decision requires the `authorized` tier (or any custom tier with `dcmgroup_required: true`), the required DCMGroup and quorum threshold must be declared. This is configured in the profile or per-action-type:
+
+```yaml
+authorized_tier_configuration:
+  default_dcmgroup_handle: platform/security-council   # default group for authorized decisions
+  quorum_threshold: "2 of 5"                           # N of M
+  
+  # Per-action-type overrides
+  action_type_overrides:
+    - subject_type: provider_registration
+      provider_type: credential_provider
+      dcmgroup_handle: platform/credential-governance
+      quorum_threshold: "3 of 5"
+    - subject_type: federation_contribution
+      dcmgroup_handle: platform/federation-council
+      quorum_threshold: "2 of 3"
+    - subject_type: policy_contribution
+      policy_domain: system
+      dcmgroup_handle: platform/policy-governance
+      quorum_threshold: "3 of 5"
+```
+
+Organizations define the DCMGroup membership. DCM enforces that the declared group and quorum were satisfied.
+
+---
+
+---
+
+## 7. Tier Registry Change Impact Detection
+
+When the authority tier registry is modified โ€” a new tier inserted, a tier removed, a tier's `decision_gravity` changed, or a tier's position changed โ€” DCM must evaluate the impact on all items that reference tier names before activating the change. This section specifies the detection model.
+
+> **Implementation note:** The impact detection pipeline described here is a required implementation component, not an optional audit feature. A tier registry change that creates security degradations must not activate until each degradation is explicitly acknowledged by a reviewer at `verified` tier or above. The detection mechanism itself is an implementation detail; this specification defines the required behavior and data model.
+
+### 7.1 Tier Impact Diff
+
+Before activating a tier registry change, DCM computes a **tier impact diff** by comparing the proposed ordered list to the current ordered list.
+
+```yaml
+tier_impact_diff:
+  registry_change_uuid: 
+  proposed_at: 
+  proposed_by: 
+  
+  tier_changes:
+    - tier_name: verified
+      change_type: POSITION_CHANGED    # NEW | REMOVED | POSITION_CHANGED | GRAVITY_CHANGED | UNCHANGED
+      old_position: 3
+      new_position: 4                  # something inserted before it
+      old_gravity: elevated
+      new_gravity: elevated            # gravity unchanged
+      net_effect: UPGRADED             # higher position = more weight = more scrutiny required
+      
+    - tier_name: compliance_reviewed   # newly inserted
+      change_type: NEW
+      old_position: null
+      new_position: 3
+      old_gravity: null
+      new_gravity: elevated
+      net_effect: NEW
+      
+    - tier_name: authorized
+      change_type: POSITION_CHANGED
+      old_position: 4
+      new_position: 5
+      old_gravity: critical
+      new_gravity: critical
+      net_effect: UPGRADED
+
+  security_degradations: []           # list of DEGRADED tier changes
+  profile_gaps: []                    # profiles whose threshold list is incomplete after change
+  broken_references: []               # tier names referenced in config that no longer exist
+```
+
+**Net effect classification:**
+
+| Net Effect | Condition | Risk |
+|-----------|-----------|------|
+| `UPGRADED` | Tier's position increased (higher weight) OR gravity increased | None โ€” more scrutiny required than before |
+| `DEGRADED` | Tier's position decreased (lower weight) OR gravity decreased | **Security risk** โ€” items referencing this tier now have lower effective authority requirement |
+| `NEW` | Tier inserted into registry | Low โ€” no existing references; profile gap detection applies |
+| `REMOVED` | Tier deleted from registry | **Broken references** โ€” any item referencing this tier name is now unresolvable |
+| `UNCHANGED` | Position and gravity identical | None |
+
+### 7.2 Affected Item Query
+
+After computing the tier impact diff, DCM queries for all items affected by each changed tier:
+
+```
+Affected item categories:
+
+PENDING APPROVAL RECORDS
+  Query: approval_records WHERE required_tier IN (changed_tier_names) AND status LIKE 'pending_%'
+  Impact: The tier name is stable; the weight at which the item was queued may differ from
+          the current weight. Compare stored_tier_weight (ATM-008) vs current_tier_weight.
+  
+PROFILE THRESHOLD CONFIGURATIONS
+  Query: all profiles WHERE tier_registry_version < new_registry_version
+  Impact: Profiles whose threshold list doesn't include newly added tiers have a gap โ€”
+          requests that score into the new tier's range will fall back to the adjacent tier.
+          Flag as PROFILE_GAP; notify platform admin to update threshold list.
+
+PROVIDER REGISTRATION REQUIREMENTS
+  Query: provider_type_registry WHERE default_approval_method IN (changed_tier_names)
+         AND profile_registration_policy WHERE min_approval_method IN (changed_tier_names)
+  Impact: If the referenced tier's gravity decreased, the minimum requirement is now lower.
+
+FCM CONTRIBUTION POLICY REQUIREMENTS  
+  Query: contribution_policy WHERE any tier reference IN (changed_tier_names)
+  Impact: Same as provider registration โ€” if gravity decreased, requirement is lower.
+
+ACTIVE POLICY SETS
+  Query: active_policies WHERE policy_content CONTAINS tier_name_reference
+  Impact: Policies that reason about tiers by name should be using dynamic resolution.
+          If a policy hardcodes a tier weight, it may now be stale.
+          Flag for policy owner review.
+```
+
+### 7.3 Impact Classification
+
+Each affected item receives one or more impact classifications:
+
+| Classification | Condition | Required Action |
+|---------------|-----------|----------------|
+| `SECURITY_DEGRADATION` | Item references a tier whose gravity decreased OR position decreased | **Blocks activation** โ€” must be reviewed and accepted |
+| `SECURITY_UPGRADE` | Item references a tier whose gravity or position increased | Informational โ€” logged and reported; does not block |
+| `BROKEN_REFERENCE` | Item references a tier name that no longer exists in registry | **Blocks activation** โ€” must be resolved (tier restored, item updated, or item cancelled) |
+| `PROFILE_GAP` | Profile threshold list incomplete after new tier insertion | **Warning** โ€” does not block activation; platform admin must update thresholds or acknowledge gap |
+| `STALE_WEIGHT` | Pending approval record's `stored_tier_weight` differs from current weight for same tier name | Informational โ€” logged; record remains valid since tier name is stable |
+
+### 7.4 Degradation Review Gate
+
+Security degradations block tier registry activation. The blocking gate requires:
+
+1. Each `SECURITY_DEGRADATION` item is presented to a reviewer at `verified` tier or above
+2. The reviewer records an explicit acceptance decision for each degradation via the Admin API
+3. The acceptance includes a reason and is written to the audit trail
+4. Only after all degradations are accepted does the tier registry change activate
+
+This is the same pattern as the standard approval pipeline โ€” DCM provides the gate; the organization provides the review process. The difference is that the required tier for the degradation review is always at least `verified`, regardless of the profile in use.
+
+```
+POST /api/v1/admin/tier-registry/{change_uuid}/accept-degradation
+
+{
+  "affected_item_uuid": "",
+  "affected_item_type": "provider_registration_requirement",
+  "degradation_classification": "SECURITY_DEGRADATION",
+  "acceptance_reason": "",
+  "accepted_by": ""       # must be verified-tier or above reviewer
+}
+```
+
+Broken references cannot be accepted โ€” they must be resolved. DCM will not activate a tier registry change that leaves unresolvable tier references.
+
+### 7.5 Impact Report
+
+Whether or not the change requires a degradation review gate, DCM generates a tier registry impact report at proposed time and again at activation time:
+
+```yaml
+tier_registry_impact_report:
+  registry_change_uuid: 
+  report_generated_at: 
+  stage: proposed | accepted | activated
+
+  summary:
+    degradations: 0
+    upgrades: 3
+    new_tiers: 1
+    broken_references: 0
+    profile_gaps: 2
+    stale_weight_records: 4
+
+  degradations: []
+
+  upgrades:
+    - affected_item_uuid: 
+      affected_item_type: provider_registration_requirement
+      tier_name: verified
+      old_weight: 3
+      new_weight: 4
+      old_gravity: elevated
+      new_gravity: elevated
+      impact: "Effective authority requirement is higher โ€” more scrutiny now required"
+
+  profile_gaps:
+    - profile: standard
+      missing_tiers: [compliance_reviewed]
+      gap_effect: >
+        Requests scoring between the verified and authorized thresholds will route
+        to verified tier until the profile threshold list is updated to include compliance_reviewed
+
+  notification_targets:
+    - platform_admin
+    - provider_owners       # for SECURITY_DEGRADATION items
+    - affected_actor_groups # DCMGroup members for authorized-tier items
+```
+
+The impact report is stored in the Audit Store and is linked to the tier registry version. Platform admins can query historical impact reports to understand what changed and when.
+
+### 7.6 Audit Trail Requirements
+
+Every tier registry change produces the following audit records, regardless of whether degradations exist:
+
+- Registry change proposal record (who proposed, what changed, when)
+- Tier impact diff record (all tier changes, all affected items, all classifications)
+- Per-degradation acceptance records (if any degradations exist)
+- Registry activation record (actual effective timestamp)
+- Per-affected-item notification records (who was notified, when)
+
+Historical approval records retain their `stored_tier_weight` from time of creation (ATM-008). The audit trail thus contains both the point-in-time weight (what authority level was required when the decision was made) and the current weight (what authority level the same tier name requires today), enabling auditors to identify decisions made under different governance regimes.
+
+### 7.7 System Policy Additions
+
+| Policy | Rule |
+|--------|------|
+| `ATM-009` | A tier registry change that produces one or more `SECURITY_DEGRADATION` items must not activate until each degradation is explicitly accepted by a reviewer at `verified` tier or above. |
+| `ATM-010` | A tier registry change that produces one or more `BROKEN_REFERENCE` items must not activate. Broken references must be resolved before the change can proceed. |
+| `ATM-011` | Every tier registry change must produce a tier impact report. The report is stored in the Audit Store and linked to the registry version. |
+| `ATM-012` | Profile threshold lists that become incomplete after a tier registry change (PROFILE_GAP) generate a warning notification to platform admins. The change may activate; platform admins must update threshold lists or explicitly acknowledge the gap within the profile's approval window. |
+
+## 6. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ATM-001` | Authority tiers are identified by name, not numeric weight. Numeric weight is resolved from list position at evaluation time and is never hardcoded in configuration. |
+| `ATM-002` | The `auto` tier's `max_score` threshold may never exceed 50 in any profile (SMX-008 translated to dynamic model). |
+| `ATM-003` | Custom tiers must declare `decision_gravity` consistent with their position in the ordered list. A tier with lower gravity may not be inserted after a tier with higher gravity. |
+| `ATM-004` | Custom tier contributions require `verified` tier approval. Organizations cannot add tiers unilaterally. |
+| `ATM-005` | Custom tiers cannot alter the `dcm_gate` semantics of existing DCM system tiers (`auto`, `reviewed`, `verified`, `authorized`). |
+| `ATM-006` | For tiers with `dcmgroup_required: true`, the DCMGroup and quorum threshold must be declared in the profile configuration before the tier can be used as a routing target. |
+| `ATM-007` | The four default `decision_gravity` values (`none`, `routine`, `elevated`, `critical`) are DCM system vocabulary. New gravity values require a DCM-level change, not an organization-level contribution. |
+| `ATM-008` | Approval records store the tier name and the resolved weight at creation time. If the tier list changes after an approval record is created, the stored weight reflects the state at creation (point-in-time audit). |
+| `ATM-009` | A tier registry change that produces one or more `SECURITY_DEGRADATION` items must not activate until each degradation is explicitly accepted by a reviewer at `verified` tier or above. |
+| `ATM-010` | A tier registry change that produces one or more `BROKEN_REFERENCE` items must not activate. Broken references must be resolved before the change can proceed. |
+| `ATM-011` | Every tier registry change must produce a tier impact report stored in the Audit Store and linked to the registry version. |
+| `ATM-012` | `PROFILE_GAP` conditions generate a warning notification to platform admins. The change may activate; admins must update threshold lists or explicitly acknowledge the gap within the profile's approval window. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/33-event-catalog.md b/content/docs/data-model/33-event-catalog.md
new file mode 100644
index 0000000..1c35033
--- /dev/null
+++ b/content/docs/data-model/33-event-catalog.md
@@ -0,0 +1,782 @@
+# DCM Data Model โ€” Event Catalog
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Authoritative Event Catalog
+**Related Documents:** [Notification Model](23-notification-model.md) | [Webhooks and Messaging](18-webhooks-messaging.md) | [Universal Audit](16-universal-audit.md) | [Credential Provider Model](31-credential-provider-model.md) | [Authority Tier Model](32-authority-tier-model.md) | [Control Plane Components](25-control-plane-components.md)
+
+> **This is the single authoritative source for all DCM event types.**
+>
+> The Notification Model (doc 23) defines delivery pipeline, audience resolution, and urgency routing. The Webhooks doc (doc 18) defines the Message Bus integration. This document defines **what events exist, when they fire, and what their payloads contain**. Any document referencing an event type is authoritative only if it agrees with this catalog. Conflicts resolve in favor of this document.
+
+> **Implementation note:** Consumers (webhook receivers, Notification Providers, Message Bus subscribers, audit tooling) must implement idempotency using `event_uuid`. Events are delivered at-least-once. Per-entity ordering is guaranteed; cross-entity ordering is not.
+
+---
+
+## 1. Base Envelope
+
+Every DCM event shares a common envelope. Event-specific fields are in the `payload` object.
+
+```yaml
+# DCM Event Envelope โ€” all events
+event_uuid:                   # idempotency key; stable across retries
+event_type:                 # fully qualified: domain.event_name
+event_schema_version: "1.0"         # increments on breaking payload changes
+timestamp:                # from Commit Log โ€” authoritative source of truth
+dcm_version:                # DCM instance version that generated the event
+dcm_instance_uuid:            # identifies the DCM instance (federation context)
+
+subject:
+  entity_uuid:         # primary entity this event concerns
+  entity_type:       # entity type FQN (e.g. Compute.VirtualMachine)
+  entity_handle:     # human-readable identifier
+  tenant_uuid:         # tenant scope; null for system-scope events
+  actor_uuid:          # actor who triggered the event; null for system events
+
+urgency: critical | high | medium | low | info   # governs notification routing
+
+payload: {}                         # event-specific fields โ€” see Section 3+
+
+links:
+  self:                        # DCM API URL for the subject entity or record
+  audit_record:                # DCM API URL for the audit record for this event
+```
+
+### 1.1 Urgency Levels
+
+| Urgency | Meaning | Delivery expectation |
+|---------|---------|---------------------|
+| `critical` | Security or compliance event requiring immediate action | Push notification; page if configured |
+| `high` | Significant operational event; action likely required | Push notification |
+| `medium` | Notable event; review recommended | Standard delivery |
+| `low` | Informational; action unlikely required | Standard delivery |
+| `info` | Observational; no action expected | Batch or webhook only |
+
+### 1.2 Schema Versioning
+
+`event_schema_version` increments when breaking changes occur to the `payload` schema for an event type. Consumers should validate against the declared version. Non-breaking additions (new optional fields) do not increment the version.
+
+---
+
+## 2. Event Domain Index
+
+| Domain | Events | Description |
+|--------|--------|-------------|
+| `request.*` | 14 | Request pipeline lifecycle |
+| `entity.*` | 13 | Resource entity lifecycle |
+| `drift.*` | 4 | Drift detection and resolution |
+| `provider.*` | 5 | Provider registration and health |
+| `provider_update.*` | 5 | Provider-initiated update lifecycle |
+| `rehydration.*` | 5 | Entity rehydration lifecycle |
+| `policy.*` | 4 | Policy contribution lifecycle |
+| `credential.*` | 4 | Credential lifecycle |
+| `approval.*` | 4 | Approval pipeline |
+| `tier_registry.*` | 4 | Authority tier registry changes |
+| `audit.*` | 3 | Audit chain integrity |
+| `dependency.*` | 2 | Entity dependency events |
+| `stakeholder.*` | 1 | Stakeholder notifications |
+| `allocation.*` | 2 | Resource allocation events |
+| `ingestion.*` | 3 | Brownfield ingestion lifecycle |
+| `governance.*` | 3 | Catalog and profile governance |
+| `security.*` | 2 | Security and sovereignty events |
+| `sovereignty.*` | 2 | Sovereignty constraint events |
+| `federation.*` | 1 | Federation tunnel events |
+| `auth.*` | 1 | Authentication provider events |
+| **Total** | **82** | |
+
+---
+
+## 3. Request Events (`request.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `request.submitted` | info | Consumer submitted a request via API or UI |
+| `request.intent_captured` | info | Intent State created; entity UUID assigned |
+| `request.layers_assembled` | info | Layer assembly complete; compound payload ready for policy evaluation |
+| `request.policies_evaluated` | info | Policy evaluation complete; score computed; routing tier determined |
+| `request.requires_approval` | medium | Score routed to `reviewed`, `verified`, or `authorized` tier; pipeline holds |
+| `request.approved` | info | Required tier approval recorded; pipeline resumes |
+| `request.placement_complete` | info | Provider placement complete; Requested State committed |
+| `request.dispatched` | info | Payload dispatched to provider(s) |
+| `request.compound_assembled` | info | Compound service payload assembled (Meta Provider compound request) |
+| `request.dependencies_resolved` | info | Constituent dependencies resolved (Meta Provider) |
+| `request.realized` | medium | Provider confirmed realization; Realized State written |
+| `request.failed` | high | Request failed at any stage |
+| `request.gatekeeper_rejected` | high | GateKeeper policy denied the request |
+| `request.cancelled` | low | Consumer cancelled; pipeline terminated |
+| `request.progress_updated` | info | Provider sent interim progress update; constituent_status updated |
+
+### 3.1 Payload Schemas
+
+#### `request.submitted` / `request.intent_captured`
+```yaml
+payload:
+  request_uuid: 
+  catalog_item_uuid: 
+  catalog_item_handle: 
+  resource_type:             # FQN e.g. Compute.VirtualMachine
+  submitted_fields: {}               # consumer-declared fields (may be partial)
+```
+
+#### `request.layers_assembled` / `request.policies_evaluated`
+```yaml
+payload:
+  request_uuid: 
+  risk_score: <0-100>                # present after policies_evaluated
+  routing_tier: auto | reviewed | verified | authorized | 
+  score_drivers:                     # top contributing signals
+    - signal: operational_gatekeeper
+      contribution: 12
+```
+
+#### `request.requires_approval`
+```yaml
+payload:
+  request_uuid: 
+  approval_uuid: 
+  required_tier: reviewed | verified | authorized | 
+  required_tier_gravity: routine | elevated | critical
+  risk_score: <0-100>
+  window_expires_at: 
+  dcmgroup_uuid:        # non-null for authorized tier
+  quorum_required: 
+```
+
+#### `request.realized` / `request.failed`
+```yaml
+payload:
+  request_uuid: 
+  provider_uuid: 
+  outcome: realized | failed | degraded
+  failure_reason: 
+  realized_fields: {}                # key provider-returned values (IP, VM ID, etc.)
+  composite_status:   # for compound requests
+```
+
+#### `request.gatekeeper_rejected`
+```yaml
+payload:
+  request_uuid: 
+  policy_handle: 
+  enforcement_class: compliance | operational
+  rejection_reason: 
+  risk_score: <0-100>
+```
+
+---
+
+## 4. Entity Lifecycle Events (`entity.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `entity.realized` | medium | Entity first realized; Realized State written |
+| `entity.state_changed` | medium | Entity lifecycle state transition |
+| `entity.modified` | info | Entity fields updated (Day-2 operation) |
+| `entity.ttl_warning` | medium | TTL expires within declared warning window |
+| `entity.ttl_expired` | high | TTL reached; expiry action triggered |
+| `entity.suspended` | high | Entity entered SUSPENDED state |
+| `entity.resumed` | medium | Entity exited SUSPENDED state |
+| `entity.decommissioning` | medium | Decommission pipeline initiated |
+| `entity.decommissioned` | low | Entity fully decommissioned; resources released |
+| `entity.decommission_deferred` | medium | Decommission blocked by active stakes |
+| `entity.ownership_transferred` | medium | Ownership moved to a different Tenant |
+| `entity.pending_review` | medium | Entity entered PENDING_REVIEW state |
+| `entity.expired` | high | Entity reached terminal expired state |
+
+### 4.1 Payload Schemas
+
+#### `entity.realized`
+```yaml
+payload:
+  request_uuid: 
+  provider_uuid: 
+  realized_fields: {}                # key fields returned by provider
+  composite_entity:            # true for Meta Provider compound services
+  composite_status: 
+```
+
+#### `entity.state_changed`
+```yaml
+payload:
+  previous_state: 
+  new_state: 
+  triggered_by: ttl | decommission | consumer | policy | provider | system
+  reason: 
+```
+
+#### `entity.ttl_warning` / `entity.ttl_expired`
+```yaml
+payload:
+  ttl_expires_at: 
+  expiry_action: decommission | suspend | notify_only
+  warning_window:   # e.g. P7D
+```
+
+#### `entity.decommissioning` / `entity.decommissioned`
+```yaml
+payload:
+  initiated_by: 
+  initiated_at: 
+  reason: 
+  stakes_resolved: 
+  credential_revocation_status: complete | partial | pending
+```
+
+#### `entity.decommission_deferred`
+```yaml
+payload:
+  blocking_stakes:
+    - stake_uuid: 
+      stake_type: required | management
+      stakeholder_tenant_uuid: 
+      stakeholder_entity_uuid: 
+  retry_after: 
+```
+
+#### `entity.ownership_transferred`
+```yaml
+payload:
+  previous_owner_tenant_uuid: 
+  new_owner_tenant_uuid: 
+  transfer_reason: 
+  transferred_by: 
+```
+
+---
+
+## 5. Drift Events (`drift.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `drift.detected` | high | Discovered State differs from Realized State |
+| `drift.severity_escalated` | high | Drift severity increased (e.g. minor โ†’ significant) |
+| `drift.resolved` | low | Drift resolved via REVERT or UPDATE_DEFINITION |
+| `drift.escalated` | high | Drift escalated to human review |
+
+### 5.1 Payload Schemas
+
+#### `drift.detected`
+```yaml
+payload:
+  drift_record_uuid: 
+  drift_severity: minor | moderate | significant | critical
+  drifted_fields:
+    - field:                 # field path e.g. "cpu_count"
+      realized_value: 
+      discovered_value: 
+  discovery_run_uuid: 
+  discovered_at: 
+```
+
+#### `drift.severity_escalated`
+```yaml
+payload:
+  drift_record_uuid: 
+  previous_severity: minor | moderate | significant | critical
+  new_severity: minor | moderate | significant | critical
+  escalation_trigger: time_elapsed | field_count | field_sensitivity
+```
+
+#### `drift.resolved`
+```yaml
+payload:
+  drift_record_uuid: 
+  resolution: REVERT | UPDATE_DEFINITION | MANUAL
+  resolved_by: 
+  resolved_at: 
+```
+
+---
+
+## 6. Provider Events (`provider.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `provider.registered` | info | Provider successfully registered and activated |
+| `provider.deregistered` | medium | Provider deregistered; active entities may be affected |
+| `provider.healthy` | info | Provider health check returned healthy after unhealthy period |
+| `provider.unhealthy` | high | Provider health check failed |
+| `provider.degraded` | high | Provider reporting degraded capacity |
+
+### 6.1 Payload Schemas
+
+#### `provider.registered` / `provider.deregistered`
+```yaml
+payload:
+  provider_uuid: 
+  provider_type: service_provider | meta_provider | credential_provider | auth_provider | ...
+  provider_handle: 
+  resource_types_affected: []  # on deregistered: types now unserviced
+  active_entity_count:            # on deregistered: entities at risk
+```
+
+#### `provider.unhealthy` / `provider.degraded`
+```yaml
+payload:
+  provider_uuid: 
+  health_check_uuid: 
+  failure_reason: 
+  consecutive_failures: 
+  last_healthy_at: 
+  affected_resource_types: []
+```
+
+---
+
+## 7. Provider Update Events (`provider_update.*`)
+
+Provider-initiated update notifications โ€” when a provider reports a change to an entity it manages.
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `provider_update.submitted` | medium | Provider submitted an update notification for a realized entity |
+| `provider_update.requires_approval` | medium | Provider update requires consumer approval before applying |
+| `provider_update.approved` | info | Consumer approved; Realized State updated |
+| `provider_update.rejected` | medium | Consumer rejected; update becomes tracked drift |
+| `provider_update.auto_approved` | info | Update auto-approved per policy |
+
+### 7.1 Payload Schema
+
+```yaml
+payload:
+  provider_update_uuid: 
+  provider_uuid: 
+  update_type: patch | deprecation | security_advisory | capacity_change
+  update_summary: 
+  proposed_field_changes: {}         # what the provider wants to change
+  approval_required: 
+  approval_uuid: 
+```
+
+---
+
+## 8. Rehydration Events (`rehydration.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `rehydration.started` | info | Rehydration pipeline initiated |
+| `rehydration.paused` | medium | Rehydration paused (e.g. waiting on dependent constituent) |
+| `rehydration.interrupted` | high | Rehydration interrupted by error or cancellation |
+| `rehydration.completed` | medium | All constituents rehydrated; entity OPERATIONAL |
+| `rehydration.blocked` | high | Rehydration blocked โ€” provider unavailable or policy prevents |
+
+### 8.1 Payload Schema
+
+```yaml
+payload:
+  rehydration_uuid: 
+  trigger: ttl_expiry | manual | drift_recovery | system
+  constituents_total: 
+  constituents_complete: 
+  block_reason:       # for rehydration.blocked
+```
+
+---
+
+## 9. Policy Events (`policy.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `policy.activated` | medium | Policy promoted from shadow to active |
+| `policy.deactivated` | medium | Policy deactivated |
+| `policy.evaluated` | info | Policy evaluated against a request payload (shadow or active) |
+| `policy.shadow_result` | info | Shadow evaluation diverged from expected outcome |
+
+### 9.1 Payload Schema
+
+#### `policy.activated` / `policy.deactivated`
+```yaml
+payload:
+  policy_uuid: 
+  policy_handle: 
+  policy_type: gatekeeper | validation | transformation | recovery | orchestration_flow
+  enforcement_class: compliance | operational    # for gatekeeper
+  shadow_period_days: 
+  approved_by: 
+```
+
+#### `policy.shadow_result`
+```yaml
+payload:
+  policy_uuid: 
+  request_uuid: 
+  shadow_decision: allow | deny | transform
+  active_decision: allow | deny | transform      # what active policies decided
+  diverged: 
+  divergence_detail: 
+```
+
+---
+
+## 10. Credential Events (`credential.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `credential.rotating` | medium | Rotation initiated; transition window open |
+| `credential.revoked` | high | Credential revoked; all holders must stop using |
+| `credential.idle` | medium | Credential not retrieved within profile threshold |
+| `credential.expired` | medium | Credential reached `valid_until`; no longer valid |
+
+### 10.1 Payload Schema
+
+#### `credential.rotating`
+```yaml
+payload:
+  credential_uuid:            # old credential
+  new_credential_uuid: 
+  rotation_trigger: pre_expiry | scheduled | security_event | actor_request
+  transition_window_ends: 
+  retrieval_url:               # where to retrieve new value
+```
+
+#### `credential.revoked`
+```yaml
+payload:
+  credential_uuid: 
+  revocation_trigger: actor_deprovisioned | entity_decommissioned | security_event | ...
+  revocation_reason: 
+  effective_at: 
+  entity_uuid: 
+```
+
+#### `credential.idle`
+```yaml
+payload:
+  credential_uuid: 
+  credential_type: 
+  issued_at: 
+  threshold_elapsed: 
+  retrieval_count: 0
+```
+
+---
+
+## 11. Approval Events (`approval.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `approval.decision_recorded` | info | A reviewer recorded an approve or reject decision |
+| `approval.quorum_reached` | medium | Authorized tier quorum satisfied; pipeline resuming |
+| `approval.window_expiring` | medium | Approval window approaching expiry (75% elapsed) |
+| `approval.expired` | high | Approval window expired without decision |
+
+### 11.1 Payload Schema
+
+#### `approval.decision_recorded`
+```yaml
+payload:
+  approval_uuid: 
+  subject_type: request | policy_contribution | provider_registration | federation_contribution
+  subject_uuid: 
+  required_tier: reviewed | verified | authorized | 
+  decision: approve | reject
+  voter_uuid: 
+  recorded_via: dcm_admin_ui | servicenow | jira | slack_bot | api_direct | other
+  votes_recorded: 
+  quorum_required: 
+  quorum_reached: 
+```
+
+#### `approval.expired`
+```yaml
+payload:
+  approval_uuid: 
+  subject_type: 
+  subject_uuid: 
+  required_tier: 
+  votes_recorded: 
+  quorum_required: 
+  expiry_action: reject | escalate
+```
+
+---
+
+## 12. Tier Registry Events (`tier_registry.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `tier_registry.proposed` | medium | Tier registry change proposed; impact assessment starting |
+| `tier_registry.impact_assessed` | medium | Tier impact diff complete; review may be required |
+| `tier_registry.degradation_detected` | high | SECURITY_DEGRADATION items found; activation blocked |
+| `tier_registry.activated` | medium | Tier registry change activated; new list in effect |
+
+### 12.1 Payload Schema
+
+#### `tier_registry.proposed`
+```yaml
+payload:
+  registry_change_uuid: 
+  proposed_by: 
+  tiers_added: []
+  tiers_removed: []
+  tiers_repositioned: []
+```
+
+#### `tier_registry.impact_assessed`
+```yaml
+payload:
+  registry_change_uuid: 
+  degradations: 
+  broken_references: 
+  profile_gaps: 
+  upgrades: 
+  activation_blocked: 
+```
+
+#### `tier_registry.degradation_detected`
+```yaml
+payload:
+  registry_change_uuid: 
+  affected_item_uuid: 
+  affected_item_type: 
+  tier_name: 
+  old_gravity: none | routine | elevated | critical
+  new_gravity: none | routine | elevated | critical
+  acceptance_required_by: 
+```
+
+---
+
+## 13. Audit Events (`audit.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `audit.chain_integrity_alert` | critical | Hash chain verification failed; audit trail may be compromised |
+| `audit.chain_break` | critical | Explicit break detected in audit hash chain |
+| `audit.forward_failed` | high | Audit record failed to forward to external audit sink |
+
+### 13.1 Payload Schema
+
+#### `audit.chain_integrity_alert`
+```yaml
+payload:
+  affected_record_uuid: 
+  expected_hash: 
+  actual_hash: 
+  chain_segment_start: 
+  chain_segment_end: 
+  records_in_segment: 
+```
+
+---
+
+## 14. Dependency and Stakeholder Events
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `dependency.state_changed` | medium | A dependency entity changed state; dependents may be affected |
+| `stakeholder.resource_decommissioning` | medium | Resource this actor has a stake in is being decommissioned |
+| `allocation.pool_capacity_low` | high | Allocation pool approaching capacity limit |
+| `allocation.released` | info | Allocation returned to pool |
+
+### 14.1 Payload Schemas
+
+#### `dependency.state_changed`
+```yaml
+payload:
+  dependency_entity_uuid: 
+  previous_state: 
+  new_state: 
+  dependent_entity_uuids: []
+  impact_assessment: degraded | blocked | unaffected
+```
+
+#### `stakeholder.resource_decommissioning`
+```yaml
+payload:
+  resource_entity_uuid: 
+  stake_type: required | management | informational
+  decommission_at: 
+  action_required:             # true for required stakes
+  action_url: 
+```
+
+---
+
+## 15. Ingestion Events (`ingestion.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `ingestion.transitional_created` | info | Brownfield entity created as Transitional entity |
+| `ingestion.enriched` | info | Transitional entity enriched with additional data |
+| `ingestion.promotion_approved` | medium | Transitional entity approved for promotion to full DCM entity |
+
+### 15.1 Payload Schema
+
+```yaml
+payload:
+  ingestion_record_uuid: 
+  source_system: 
+  entity_handle: 
+  confidence_level: high | medium | low
+  missing_fields: []         # for ingestion.enriched
+  promoted_entity_uuid:  # for ingestion.promotion_approved
+```
+
+---
+
+## 16. Governance Events (`governance.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `governance.catalog_item_deprecated` | medium | Service catalog item marked for deprecation |
+| `governance.profile_changed` | high | Active profile configuration changed |
+| `governance.policy_trust_elevated` | medium | Policy provider trust level elevated |
+
+### 16.1 Payload Schema
+
+#### `governance.profile_changed`
+```yaml
+payload:
+  previous_profile: 
+  new_profile: 
+  changed_by: 
+  effective_at: 
+  affected_threshold_tiers: []
+```
+
+---
+
+## 17. Security and Sovereignty Events
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `security.unsanctioned_provider_write` | critical | Provider wrote to an entity without a corresponding Requested State record |
+| `sovereignty.violation` | critical | Data or operation crossed a declared sovereignty boundary |
+| `sovereignty.migration_required` | high | Entity must migrate to comply with sovereignty constraints |
+| `federation.tunnel_degraded` | high | Federation tunnel to peer DCM degraded or unavailable |
+| `auth.provider_failover` | high | Auth Provider failed; failover to secondary |
+
+### 17.1 Payload Schemas
+
+#### `security.unsanctioned_provider_write`
+```yaml
+payload:
+  provider_uuid: 
+  entity_uuid: 
+  write_detected_at: 
+  changed_fields: []
+  discovery_run_uuid: 
+```
+
+#### `sovereignty.violation`
+```yaml
+payload:
+  violation_type: data_boundary | operation_boundary | residency_requirement
+  constraint_uuid: 
+  constraint_handle: 
+  triggering_operation: 
+  remediation_required: 
+```
+
+---
+
+## 18. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `EVT-001` | Every event must include the base envelope fields (`event_uuid`, `event_type`, `event_schema_version`, `timestamp`, `dcm_version`, `dcm_instance_uuid`, `urgency`). Events omitting required envelope fields are invalid and must not be published. |
+| `EVT-002` | `event_uuid` is the idempotency key. Consumers must treat duplicate `event_uuid` values as already-processed. DCM may re-deliver events on failure; this is not a bug. |
+| `EVT-003` | `timestamp` is sourced from the Commit Log Stage 1 write. It represents when the event was authoritatively recorded, not when it was delivered. |
+| `EVT-004` | `event_schema_version` must increment on any breaking change to a payload schema. Adding optional fields is not a breaking change. Removing fields, changing field types, or changing field semantics are breaking changes. |
+| `EVT-005` | Events with `urgency: critical` must be delivered via the push channel if the Notification Provider supports it, regardless of consumer subscription preferences. |
+| `EVT-006` | This catalog is the authoritative source for event type names. Any event type not in this catalog is non-standard. Non-standard events may be published by providers or extensions but must use a reverse-DNS prefix (e.g. `com.acme.custom_event`). |
+| `EVT-007` | The `audit.*` events with `urgency: critical` are non-suppressable. They are delivered regardless of audience subscription rules and cannot be filtered by consumer preference. |
+
+---
+
+## 21. ITSM Events (`itsm.*`)
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `itsm.record_created` | info | ITSM Provider successfully created a record in the external ITSM system |
+| `itsm.record_updated` | info | ITSM Provider successfully updated an existing ITSM record |
+| `itsm.record_failed` | medium | ITSM Provider failed to create/update a record; or `block_until_created` timeout reached |
+
+### 21.1 Payload Schema
+
+#### `itsm.record_created` / `itsm.record_updated`
+```yaml
+payload:
+  itsm_provider_uuid: 
+  itsm_system: servicenow | jira_service_management | ...
+  action: create_change_request | create_incident | ...
+  record_type: change_request | incident | cmdb_ci | service_request
+  record_id: ""    # e.g. CHG0012345, INC-4821
+  record_url: ""                  # deep link to record in ITSM system
+  policy_handle: ""            # which ITSM Policy triggered this
+  stored_on_entity: 
+
+```
+
+#### `itsm.record_failed`
+```yaml
+payload:
+  itsm_provider_uuid: 
+  action: 
+  failure_reason: 
+  timeout_expired:              # true if block_until_created timeout hit
+  policy_handle: 
+```
+
+---
+
+## 19. Event Type Quick Reference
+
+```
+request.submitted          request.intent_captured      request.layers_assembled
+request.policies_evaluated request.requires_approval    request.approved
+request.placement_complete request.dispatched            request.compound_assembled
+request.dependencies_resolved  request.realized          request.failed
+request.gatekeeper_rejected    request.cancelled
+
+entity.realized            entity.state_changed         entity.modified
+entity.ttl_warning         entity.ttl_expired           entity.suspended
+entity.resumed             entity.decommissioning       entity.decommissioned
+entity.decommission_deferred   entity.ownership_transferred  entity.pending_review
+entity.expired
+
+drift.detected             drift.severity_escalated     drift.resolved
+drift.escalated
+
+provider.registered        provider.deregistered        provider.healthy
+provider.unhealthy         provider.degraded
+
+provider_update.submitted  provider_update.requires_approval  provider_update.approved
+provider_update.rejected   provider_update.auto_approved
+
+rehydration.started        rehydration.paused           rehydration.interrupted
+rehydration.completed      rehydration.blocked
+
+policy.activated           policy.deactivated           policy.evaluated
+policy.shadow_result
+
+credential.rotating        credential.revoked           credential.idle
+credential.expired
+
+approval.decision_recorded approval.quorum_reached      approval.window_expiring
+approval.expired
+
+tier_registry.proposed     tier_registry.impact_assessed
+tier_registry.degradation_detected  tier_registry.activated
+
+audit.chain_integrity_alert  audit.chain_break          audit.forward_failed
+
+dependency.state_changed   stakeholder.resource_decommissioning
+allocation.pool_capacity_low  allocation.released
+
+ingestion.transitional_created  ingestion.enriched      ingestion.promotion_approved
+
+governance.catalog_item_deprecated  governance.profile_changed
+governance.policy_trust_elevated
+
+security.unsanctioned_provider_write
+sovereignty.violation      sovereignty.migration_required
+federation.tunnel_degraded
+auth.provider_failover
+```
+
+**Total: 85 event types across 21 domains**
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/34-api-versioning-strategy.md b/content/docs/data-model/34-api-versioning-strategy.md
new file mode 100644
index 0000000..34c66bc
--- /dev/null
+++ b/content/docs/data-model/34-api-versioning-strategy.md
@@ -0,0 +1,386 @@
+# DCM Data Model โ€” API Versioning Strategy
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” API Versioning and Lifecycle
+**Related Documents:** [Consumer API Specification](../specifications/consumer-api-spec.md) | [Admin API Specification](../specifications/dcm-admin-api-spec.md) | [Operator Interface Specification](../specifications/dcm-operator-interface-spec.md) | [Event Catalog](33-event-catalog.md) | [Registry Governance](20-registry-governance.md) | [Design Priorities](00-design-priorities.md)
+
+> **This document governs all DCM API surfaces.** Every public API endpoint โ€” Consumer, Admin, Operator Interface (Provider), Flow GUI โ€” follows this versioning strategy. The strategy is designed to make the secure, compatible path the easy path: clients that do nothing get the version they requested; breaking changes are announced with sufficient lead time; the newest version is always the supported version.
+
+---
+
+## 1. Versioning Model
+
+### 1.1 URL-Based Major Version
+
+DCM APIs use **URL path versioning** for major versions. The version is the first path segment after the API surface prefix:
+
+```
+Consumer API:      https://{dcm-instance}/api/v1/
+Admin API:         https://{dcm-instance}/api/v1/admin/
+Provider API (OIS):https://{dcm-instance}/provider/api/v1/
+Flow GUI API:      https://{dcm-instance}/flow/api/v1/
+```
+
+The version segment (`v1`, `v2`, etc.) represents a major version. It increments only on breaking changes. Multiple major versions may coexist during a transition window (see Section 4).
+
+### 1.2 Version Granularity โ€” Per-API, Not Per-Endpoint
+
+Versioning is **per-API surface**, not per-endpoint. When a breaking change occurs to any endpoint within an API surface, the entire surface increments to the next major version. This means:
+
+- `v2` of the Consumer API is a complete API surface, not a patchwork of versioned endpoints
+- All endpoints within a surface version are internally consistent
+- Clients target a single version for all their interactions with that surface
+
+Individual endpoints are not independently versioned. If a single endpoint needs a breaking change, the API surface version increments and all other endpoints continue unchanged under the new version.
+
+### 1.3 Minor and Revision Changes
+
+Non-breaking changes within a major version are documented in the API changelog but do not change the URL. Clients do not need to take any action for non-breaking changes.
+
+The changelog follows semantic versioning conventions:
+- **Minor change**: new optional fields, new endpoints, expanded enum values with backward-compatible defaults
+- **Revision**: documentation corrections, clarifications, non-functional specification updates
+
+---
+
+## 2. Breaking Change Definition
+
+A change is **breaking** if it requires any existing client to modify its code or configuration to continue working correctly. The following changes are always breaking:
+
+**Request changes:**
+- Removing a field that was previously accepted
+- Changing a field from optional to required
+- Changing a field's type (e.g. string โ†’ integer)
+- Removing an accepted enum value
+- Changing URL path structure (endpoint rename or restructure)
+- Changing HTTP method for an existing operation
+- Removing an endpoint
+
+**Response changes:**
+- Removing a field from any response
+- Changing a field's type in any response
+- Changing a field's name in any response
+- Removing a previously returned enum value
+- Changing HTTP status code semantics (e.g. 200 โ†’ 202, or changing when 4xx vs 5xx is returned)
+- Changing the response envelope structure
+
+**Behavior changes:**
+- Changing default values in ways that alter existing behavior
+- Changing idempotency semantics
+- Removing a previously supported authentication method
+- Tightening validation (rejecting previously accepted inputs)
+- Changing pagination behavior in ways that break existing cursor patterns
+
+**The following are NOT breaking changes:**
+- Adding new optional request fields (with sensible defaults)
+- Adding new response fields (existing clients safely ignore unknown fields)
+- Adding new endpoints
+- Expanding an enum with new values (clients must handle unknown enum values gracefully)
+- Relaxing validation (accepting previously rejected inputs)
+- Adding new error codes (clients that handle errors generically are unaffected)
+- Performance improvements, infrastructure changes, security patches
+- Documentation improvements
+
+---
+
+## 3. Version Discovery
+
+Clients can discover available API versions and their status without prior knowledge:
+
+### 3.1 Well-Known Discovery Endpoint
+
+```
+GET https://{dcm-instance}/.well-known/dcm-api-versions
+
+Response 200:
+{
+  "dcm_version": "1.2.0",
+  "api_surfaces": {
+    "consumer": {
+      "current": "v2",
+      "supported": ["v1", "v2"],
+      "versions": {
+        "v1": {
+          "status": "deprecated",
+          "sunset_date": "2027-06-01",
+          "deprecation_date": "2026-06-01",
+          "base_url": "/api/v1/",
+          "changelog_url": "/api/v1/changelog"
+        },
+        "v2": {
+          "status": "stable",
+          "released_date": "2026-06-01",
+          "base_url": "/api/v2/",
+          "changelog_url": "/api/v2/changelog"
+        }
+      }
+    },
+    "admin": {
+      "current": "v1",
+      "supported": ["v1"],
+      "versions": {
+        "v1": { "status": "stable", "base_url": "/api/v1/admin/" }
+      }
+    },
+    "provider": {
+      "current": "v1",
+      "supported": ["v1"],
+      "versions": {
+        "v1": { "status": "stable", "base_url": "/provider/api/v1/" }
+      }
+    }
+  }
+}
+```
+
+### 3.2 Per-Version Changelog
+
+```
+GET /api/v1/changelog
+
+Response 200:
+{
+  "version": "v1",
+  "changes": [
+    {
+      "date": "2026-01-15",
+      "type": "minor",
+      "description": "Added optional `score_drivers` field to request status response",
+      "affected_endpoints": ["GET /api/v1/requests/{uuid}/status"]
+    }
+  ]
+}
+```
+
+---
+
+## 4. Deprecation and Sunset Lifecycle
+
+### 4.1 Deprecation Timeline
+
+When a new major version is released, the previous version enters a **deprecation period**. The deprecation timeline is profile-governed โ€” production deployments require longer support windows than development environments:
+
+```yaml
+api_version_support_lifecycle:
+  minimal:
+    deprecation_notice_period: P90D    # 90 days notice before sunset
+    deprecated_version_support: P180D  # old version supported 180 days after deprecation
+    
+  dev:
+    deprecation_notice_period: P60D
+    deprecated_version_support: P90D
+
+  standard:
+    deprecation_notice_period: P180D
+    deprecated_version_support: P365D  # 1 year
+
+  prod:
+    deprecation_notice_period: P365D   # 1 year notice
+    deprecated_version_support: P730D  # 2 years support after deprecation
+
+  fsi:
+    deprecation_notice_period: P548D   # 18 months notice
+    deprecated_version_support: P1095D # 3 years support after deprecation
+
+  sovereign:
+    deprecation_notice_period: P730D   # 2 years notice
+    deprecated_version_support: P1460D # 4 years support after deprecation
+```
+
+**Deprecation โ‰  Sunset.** A deprecated version continues to function. Sunset is when it stops working. The deprecation period is the window between "we recommend you migrate" and "you must migrate."
+
+### 4.2 Deprecation Headers
+
+When a client calls a deprecated API version, the response includes standard deprecation headers (per [RFC 8594](https://datatracker.ietf.org/doc/html/rfc8594) and [RFC 9745](https://datatracker.ietf.org/doc/html/rfc9745)):
+
+```http
+HTTP/1.1 200 OK
+Deprecation: @1749340800          # Unix timestamp when this version was deprecated
+Sunset: @1781049600               # Unix timestamp when this version will stop working
+Link: ; rel="successor-version"
+Link: ; rel="deprecation"
+```
+
+### 4.3 Deprecation Events
+
+When a version is deprecated or sunsetted, DCM fires notification events:
+
+- `governance.api_version_deprecated` โ€” version entered deprecation; Sunset header begins appearing
+- `governance.api_version_sunset_warning` โ€” 30 days before sunset; high urgency
+- `governance.api_version_sunset` โ€” version has reached sunset date; calls now return 410 Gone
+
+Platform admins should configure notification routing for these events to ensure API consumers receive timely warning.
+
+### 4.4 Sunset Behavior
+
+After the sunset date, calls to the deprecated version return:
+
+```http
+HTTP/1.1 410 Gone
+Content-Type: application/json
+
+{
+  "error": "api_version_sunset",
+  "message": "API version v1 reached its sunset date on 2027-06-01. Migrate to v2.",
+  "successor_version": "v2",
+  "migration_guide_url": "/api/v2/migration-guide",
+  "sunset_date": "2027-06-01"
+}
+```
+
+---
+
+## 5. Version Negotiation
+
+### 5.1 How Clients Specify a Version
+
+The URL path is the primary versioning mechanism. No headers or query parameters are required โ€” the URL is authoritative:
+
+```
+GET /api/v1/resources        โ†’ Consumer API v1
+GET /api/v2/resources        โ†’ Consumer API v2 (when available)
+```
+
+### 5.2 Version Preference Header (Optional)
+
+For clients that need to pin to a specific version or test against a new version before migrating, an optional `DCM-API-Version` header is supported:
+
+```http
+GET /api/v1/resources
+DCM-API-Version: v1          # explicit pin; returns 406 if v1 is sunsetted
+```
+
+If the header specifies a sunsetted version, the response is `406 Not Acceptable` with a migration guide reference.
+
+### 5.3 Latest-Version Alias
+
+```
+GET /api/latest/resources    # always routes to current stable version
+```
+
+The `latest` alias is provided for development and testing. It is **not recommended for production** โ€” production clients should pin to a specific version to avoid inadvertent breaking changes when a new major version becomes `latest`.
+
+---
+
+## 6. Beta and Preview Endpoints
+
+New capabilities that are not yet stable may be released as **preview endpoints** within the current major version:
+
+```
+GET /api/v1/preview/new-feature
+```
+
+Preview endpoints:
+- Are not covered by the stability guarantees of the parent version
+- May change or be removed without a major version increment
+- Are marked in the API changelog and discovery endpoint as `status: preview`
+- Must not be used in production automation without explicit acknowledgment of instability
+
+```yaml
+# Discovery response for a preview endpoint
+"new-feature": {
+  "status": "preview",
+  "stability_commitment": "none",
+  "planned_graduation": "v2",
+  "feedback_url": "https://github.com/dcm-project/discussions"
+}
+```
+
+Preview endpoints graduate to stable when they are included in a new major version release.
+
+---
+
+## 7. Provider API (OIS) Versioning
+
+The Operator Interface Specification (OIS) governs how DCM calls providers. Provider implementations must support the version of the OIS they declare in their capability registration.
+
+### 7.1 OIS Version in Capability Registration
+
+```yaml
+provider_registration:
+  ois_version: "1.0"           # which OIS version this provider implements
+  ois_version_min: "1.0"       # minimum OIS version supported
+  ois_version_max: "1.x"       # maximum OIS version supported (x = any minor)
+```
+
+### 7.2 OIS Compatibility
+
+DCM maintains backward compatibility with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 during the deprecation window.
+
+When the OIS version is incremented:
+1. DCM announces the new OIS version via the event `governance.ois_version_released`
+2. Providers have the deprecation notice period to upgrade their implementation
+3. DCM dispatches using the appropriate OIS version per the provider's declared capability
+4. After sunset, providers still on deprecated OIS versions receive `410 Gone` on dispatch
+
+### 7.3 Provider-Initiated API Versioning
+
+Providers that expose their own management APIs (beyond the standard OIS surface) are responsible for their own versioning. DCM does not version-manage provider-internal APIs. Providers should follow the same breaking-change definition (Section 2) and announce breaking changes via `provider_update.submitted` events.
+
+---
+
+## 8. Client Migration Path
+
+### 8.1 Migration Guide Structure
+
+Each new major version publishes a migration guide accessible at:
+
+```
+GET /api/v{N}/migration-guide
+```
+
+The migration guide is machine-readable JSON listing all breaking changes from the previous version:
+
+```json
+{
+  "from_version": "v1",
+  "to_version": "v2",
+  "breaking_changes": [
+    {
+      "change_id": "BC-001",
+      "type": "field_removed",
+      "endpoint": "GET /api/v2/resources/{uuid}",
+      "description": "Field 'legacy_id' removed from response. Use 'entity_uuid' instead.",
+      "migration": "Replace references to 'legacy_id' with 'entity_uuid'",
+      "affected_since": "2026-06-01"
+    }
+  ],
+  "non_breaking_additions": [ ... ],
+  "sunset_date_of_previous_version": "2027-06-01"
+}
+```
+
+### 8.2 Parallel Operation
+
+During the deprecation window, clients may run v1 and v2 in parallel โ€” for example, migrating one service at a time. Both versions return consistent data from the same underlying DCM data stores. There are no data synchronization concerns between versions.
+
+---
+
+## 9. Internal API Versioning
+
+DCM internal component APIs (Control Plane components communicating with each other) follow a simpler model:
+
+- Internal APIs are not exposed externally and not subject to the external versioning lifecycle
+- Internal breaking changes require a coordinated deployment of all affected components
+- DCM release versions (e.g. `1.2.0`) cover the complete set of internal APIs for that release
+- Operators upgrading DCM must upgrade all components together per the release upgrade guide
+
+---
+
+## 10. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `VER-001` | All DCM public API surfaces use URL path versioning. The version path segment is the only authoritative version indicator. |
+| `VER-002` | A change is breaking if any existing client must modify code or configuration to continue working. When in doubt, treat a change as breaking. |
+| `VER-003` | Deprecated API versions must return `Deprecation`, `Sunset`, and `Link` headers on every response during the deprecation period (per RFC 8594 / RFC 9745). |
+| `VER-004` | Deprecated versions must remain fully functional until the sunset date. Bugs in deprecated versions are fixed; new features are not backported. |
+| `VER-005` | The deprecation notice period and deprecated version support window are profile-governed. Production deployments require longer windows than development. See Section 4.1. |
+| `VER-006` | The `latest` version alias is available but must not be recommended for production use. Production clients must pin to a specific version. |
+| `VER-007` | Preview endpoints are not stable. They may change or be removed without a major version increment. They are identified by the `/preview/` path segment. |
+| `VER-008` | Every new major version must publish a machine-readable migration guide at `/api/v{N}/migration-guide`. |
+| `VER-009` | DCM must maintain dispatch compatibility with providers registered on supported OIS versions during the OIS deprecation window. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/35-session-revocation.md b/content/docs/data-model/35-session-revocation.md
new file mode 100644
index 0000000..55c0c38
--- /dev/null
+++ b/content/docs/data-model/35-session-revocation.md
@@ -0,0 +1,351 @@
+# DCM Data Model โ€” Session Token Revocation
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Session Lifecycle and Revocation
+**Related Documents:** [Auth Providers](19-auth-providers.md) | [Credential Provider Model](31-credential-provider-model.md) | [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Event Catalog](33-event-catalog.md) | [Design Priorities](00-design-priorities.md)
+
+> **This document maps to: DATA + POLICY**
+>
+> A session is a Data artifact with a UUID, lifecycle state, and audit trail. Session revocation is a Policy concern โ€” it fires on triggers defined here and enforced by the Auth Provider and Ingress layer. This document extends the Auth Provider model (doc 19) with the explicit revocation lifecycle that was previously unspecified.
+>
+> **Relationship to credential revocation:** CPX-006 (doc 31) governs credential revocation โ€” when an actor is deprovisioned, all credentials issued to that actor are revoked. This document governs the complementary concern: active *session tokens* must also be invalidated on the same trigger. Credential revocation and session revocation are parallel processes that both fire on actor deprovisioning.
+
+---
+
+## 1. What a Session Is
+
+A DCM session represents an authenticated actor's active interaction context. It is created when an actor successfully authenticates through an Auth Provider and is destroyed (or expires) when the session ends.
+
+```yaml
+session_record:
+  session_uuid: 
+  actor_uuid: 
+  auth_provider_uuid:            # which provider issued the session
+  auth_method: oidc | ldap | api_key | mtls | built_in
+  mfa_verified:                  # whether per-session MFA was completed
+  step_up_verified_at:   # last step-up MFA completion
+  
+  created_at: 
+  last_active_at: 
+  expires_at:                # absolute expiry (from token_ttl)
+  
+  refresh_token_uuid:     # if refresh_enabled: true
+  refresh_expires_at: 
+  
+  status: active | refreshing | revoked | expired
+  revocation_reason: 
+  revoked_at: 
+  revoked_by: 
+  
+  # Provenance
+  client_ip: 
+  user_agent: 
+  tenant_uuid: 
+  
+  # Concurrent session position
+  session_sequence:               # 1 = oldest active session for this actor
+```
+
+### 1.1 Session Store
+
+Active sessions are maintained in a **Session Store** โ€” a fast-queryable, low-latency store separate from the Realized State Store. The Session Store is not GitOps-backed; it is operational state that does not need version history.
+
+```yaml
+session_store:
+  implementation: redis | postgres | in_memory   # profile-governed
+  ttl_enforcement: hard                          # sessions expire at expires_at regardless
+  revocation_index: true                         # fast lookup by session_uuid for revocation
+  actor_index: true                              # fast lookup by actor_uuid for bulk revocation
+```
+
+**Profile-governed defaults:**
+
+| Profile | Store | Session TTL | Refresh TTL | Max concurrent |
+|---------|-------|-------------|-------------|---------------|
+| `minimal` | in_memory or sqlite | PT8H | P7D | unlimited |
+| `dev` | redis or postgres | PT4H | P3D | 10 |
+| `standard` | redis or postgres | PT1H | P1D | 5 |
+| `prod` | redis or postgres | PT30M | PT8H | 3 |
+| `fsi` | redis or postgres | PT15M | PT1H | 2 |
+| `sovereign` | redis or postgres (HSM-backed) | PT15M | PT30M | 1 |
+
+---
+
+## 2. Revocation Triggers
+
+Session revocation invalidates a session immediately โ€” regardless of its remaining TTL. The following triggers cause revocation:
+
+| Trigger | Scope | Who initiates | Behavior |
+|---------|-------|--------------|---------|
+| `actor_logout` | Single session | Actor (self) | Immediate; that session only |
+| `actor_logout_all` | All sessions for actor | Actor (self) | Immediate; all active sessions for this actor |
+| `actor_deprovisioned` | All sessions for actor | SCIM / Platform admin | Immediate; fires before deprovisioning acknowledged |
+| `actor_suspended` | All sessions for actor | Platform admin | Immediate |
+| `security_event` | Specified sessions or all | Platform admin / security automation | Immediate; emergency channel notification |
+| `concurrent_limit_exceeded` | Oldest session(s) | System | Oldest session revoked when new session created beyond limit |
+| `auth_provider_deregistered` | All sessions from that provider | Platform admin | Immediate; actors must re-authenticate via another provider |
+| `credential_compromised` | All sessions for actor | Security automation | Immediate; correlates with CPX emergency rotation |
+| `admin_forced_logout` | Specified session(s) | Platform admin | Immediate |
+
+---
+
+## 3. Revocation Lifecycle
+
+### 3.1 Standard Revocation
+
+```
+Revocation trigger fires
+  โ”‚
+  โ–ผ Session record status โ†’ revoked
+  โ”‚   revoked_at, revocation_reason, revoked_by written
+  โ”‚
+  โ–ผ Refresh token invalidated (if exists)
+  โ”‚   Cannot be exchanged; refresh endpoint returns 401
+  โ”‚
+  โ–ผ Session UUID added to Session Revocation Registry
+  โ”‚   (fast-queryable; all DCM components check this on every request)
+  โ”‚
+  โ–ผ Revocation event published to Message Bus
+  โ”‚   event_type: auth.session_revoked
+  โ”‚   session_uuid, actor_uuid, revocation_trigger, revoked_at
+  โ”‚
+  โ–ผ Audit record written
+      session_uuid, actor_uuid, revocation_trigger, revoked_by, revoked_at
+```
+
+### 3.2 Actor Deprovisioning Revocation (parallel with CPX-006)
+
+Actor deprovisioning fires both credential revocation (CPX-006) and session revocation simultaneously. Neither blocks the other; both must complete before the deprovisioning is acknowledged.
+
+```
+Actor deprovisioning initiated
+  โ”‚
+  โ”œโ”€โ”€โ†’ Credential revocation (CPX-006)
+  โ”‚     All credentials issued to actor_uuid โ†’ revoked
+  โ”‚     Credential Revocation Registry updated
+  โ”‚
+  โ””โ”€โ”€โ†’ Session revocation (this document)
+        All active sessions for actor_uuid โ†’ revoked
+        Session Revocation Registry updated
+        auth.session_revoked events published per session
+  โ”‚
+  โ–ผ Both complete โ†’ deprovisioning acknowledged
+    actor_deprovisioned event published
+    Audit record for deprovisioning written
+```
+
+### 3.3 Emergency Revocation (Security Event)
+
+Security events bypass the standard pipeline. Revocation is immediate with no grace period.
+
+```
+Security event detected
+  โ”‚
+  โ–ผ Target sessions determined
+  โ”‚   (single session, all sessions for actor, or all sessions from a provider)
+  โ”‚
+  โ–ผ Sessions โ†’ revoked immediately
+  โ”‚   Session Revocation Registry updated within SLA:
+  โ”‚     standard/prod: PT30S
+  โ”‚     fsi: PT10S
+  โ”‚     sovereign: PT5S
+  โ”‚
+  โ–ผ auth.security_session_revoked event published (critical urgency)
+  โ”‚   Routed to security team via configured Notification Provider
+  โ”‚
+  โ–ผ Platform admin notified regardless of profile
+  โ”‚
+  โ–ผ All in-flight requests from these sessions โ†’ 401 Unauthorized
+```
+
+---
+
+## 4. Session Revocation Registry
+
+The Session Revocation Registry is the authoritative list of revoked-but-not-yet-expired session UUIDs. Every DCM component that accepts bearer tokens must check this registry on each request.
+
+```yaml
+session_revocation_registry:
+  # Session UUID โ†’ revocation record
+  # Fast in-memory cache with TTL equal to original session TTL
+  # After the original session TTL would have expired, the entry is
+  # removed (the session would have been invalid anyway)
+  
+  entry:
+    session_uuid: 
+    revoked_at: 
+    original_expires_at:     # entry removed after this time
+    revocation_trigger: 
+```
+
+**Cache refresh behavior by profile:**
+
+| Profile | Max cache age | Behavior on cache miss |
+|---------|--------------|----------------------|
+| `minimal` | PT5M | Check authoritative store; cache result |
+| `standard` | PT1M | Check authoritative store; cache result |
+| `prod` | PT30S | Check authoritative store; cache result |
+| `fsi` | PT10S | Check authoritative store; cache result |
+| `sovereign` | PT5S | No cache โ€” always check authoritative store |
+
+---
+
+## 5. Token Introspection
+
+DCM's Ingress layer exposes a token introspection endpoint for internal components and external systems that need to validate a token without maintaining their own cache:
+
+```
+POST /api/v1/auth/introspect
+
+Authorization: Bearer 
+Content-Type: application/json
+
+{
+  "token": ""
+}
+
+Response 200 (active session):
+{
+  "active": true,
+  "session_uuid": "",
+  "actor_uuid": "",
+  "expires_at": "",
+  "mfa_verified": true,
+  "tenant_uuid": "",
+  "roles": ["consumer"],
+  "scopes": ["read", "write"]
+}
+
+Response 200 (revoked or expired):
+{
+  "active": false,
+  "reason": "revoked | expired | not_found"
+}
+```
+
+Session tokens use JWT format (RFC 7519). This introspection endpoint follows [RFC 7662 (OAuth 2.0 Token Introspection)](https://datatracker.ietf.org/doc/html/rfc7662).
+
+---
+
+## 6. Consumer API โ€” Session Management Endpoints
+
+### 6.1 Logout (Single Session)
+
+```
+DELETE /api/v1/auth/session
+
+Response 204 No Content
+```
+
+Revokes the session corresponding to the bearer token in the `Authorization` header. No body required.
+
+### 6.2 Logout All Sessions
+
+```
+DELETE /api/v1/auth/sessions
+
+Response 204 No Content
+```
+
+Revokes all active sessions for the authenticated actor.
+
+### 6.3 List Active Sessions
+
+```
+GET /api/v1/auth/sessions
+
+Response 200:
+{
+  "items": [
+    {
+      "session_uuid": "",
+      "created_at": "",
+      "last_active_at": "",
+      "expires_at": "",
+      "auth_method": "oidc",
+      "client_ip": "",
+      "current": true    // true for the session making this request
+    }
+  ],
+  "total": 2
+}
+```
+
+### 6.4 Revoke Specific Session
+
+```
+DELETE /api/v1/auth/sessions/{session_uuid}
+
+Response 204 No Content
+Response 404: session not found or does not belong to this actor
+```
+
+### 6.5 Admin: Force Revoke Session(s)
+
+```
+POST /api/v1/admin/actors/{actor_uuid}/revoke-sessions
+
+{
+  "scope": "all | session",
+  "session_uuid": "",    // required if scope: session
+  "reason": ""               // required for audit trail
+}
+
+Response 204 No Content
+Response 404: actor not found
+```
+
+---
+
+## 7. Concurrent Session Enforcement
+
+When `concurrent_sessions: N` is declared and a new session would exceed the limit, the oldest active session is revoked automatically:
+
+```
+New authentication succeeds
+  โ”‚
+  โ–ผ Count active sessions for actor_uuid
+  โ”‚   If count >= concurrent_sessions limit:
+  โ”‚     Revoke oldest session (by created_at)
+  โ”‚     Trigger: concurrent_limit_exceeded
+  โ”‚
+  โ–ผ New session created
+```
+
+The evicted actor receives an `auth.session_revoked` notification if a Notification Provider is configured with the actor's notification preferences. The event does not block the new session creation.
+
+---
+
+## 8. Relationship to the Credential Revocation Model
+
+Session revocation (this document) and credential revocation (doc 31, CPX-001โ€“CPX-012) are parallel but distinct:
+
+| | Session Revocation | Credential Revocation |
+|--|---|---|
+| **What** | Bearer token / session cookie validity | API key, x509, SSH key, service account token |
+| **Store** | Session Revocation Registry | Credential Revocation Registry |
+| **Propagation** | Auth layer cache refresh | Message Bus โ†’ all components |
+| **Actor deprovision** | All sessions revoked | All credentials revoked |
+| **TTL** | Session TTL (minutes to hours) | Credential TTL (hours to years) |
+| **Emergency SLA** | PT5Sโ€“PT30S | PT30Sโ€“PT5M |
+| **Event** | `auth.session_revoked` | `credential.revoked` |
+
+**AUTH-016:** On actor deprovisioning, session revocation and credential revocation are parallel operations. The deprovisioning is not acknowledged until both are confirmed complete.
+
+---
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `AUTH-016` | On actor deprovisioning, session revocation and credential revocation (CPX-006) are parallel operations. Deprovisioning is not acknowledged until both complete. |
+| `AUTH-017` | Session revocation must propagate to the Session Revocation Registry within the profile-governed SLA: minimal PT5M, standard PT1M, prod PT30S, fsi PT10S, sovereign PT5S. |
+| `AUTH-018` | All DCM components that accept bearer tokens must check the Session Revocation Registry on each request. Cache age must not exceed the profile-governed maximum (sovereign: no cache). |
+| `AUTH-019` | Emergency session revocation (security_event trigger) fires immediately with no grace period. The `auth.security_session_revoked` event has `urgency: critical` and is non-suppressable. |
+| `AUTH-020` | The token introspection endpoint (`POST /api/v1/auth/introspect`) must be authenticated. Access requires an actor or service account with the `introspection` scope. |
+| `AUTH-021` | When concurrent session limits are enforced, the oldest session is revoked before the new session is created. The evicted actor is notified via Notification Provider if configured. |
+| `AUTH-022` | Refresh tokens are invalidated when their parent session is revoked. A revoked refresh token returns 401 on exchange; it cannot be used to create a new session. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/36-internal-component-auth.md b/content/docs/data-model/36-internal-component-auth.md
new file mode 100644
index 0000000..a82a9ac
--- /dev/null
+++ b/content/docs/data-model/36-internal-component-auth.md
@@ -0,0 +1,369 @@
+# DCM Data Model โ€” Internal Component Authentication
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Zero Trust Internal Auth
+**Related Documents:** [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Credential Provider Model](31-credential-provider-model.md) | [Auth Providers](19-auth-providers.md) | [Session Revocation](35-session-revocation.md) | [Design Priorities](00-design-priorities.md)
+
+> **This document maps to: DATA + POLICY**
+>
+> Internal component identities are Data โ€” each component has a UUID, certificate, and service account. Internal auth is Policy โ€” the same five-check boundary model from doc 26 applies at every internal call boundary, with no exceptions for "trusted internal network." This document specifies what was previously only mentioned: how DCM's control plane components authenticate to each other in a distributed deployment.
+
+---
+
+## 1. The Core Principle
+
+**Network position grants zero trust.** This is stated in doc 26 for external interactions. It applies equally to internal component communication. A call from the Policy Engine to the Placement Engine receives the same boundary checks as a call from an external consumer. The service mesh enforces this at the infrastructure level; DCM enforces it at the application level.
+
+**Two-layer enforcement:**
+1. **Mesh layer (infrastructure):** mTLS mutual authentication (RFC 8446 TLS 1.3), certificate validation (RFC 5280), traffic policies โ€” enforced by the service mesh (Istio or equivalent)
+2. **Application layer (DCM):** component identity verification, operation authorization, scoped interaction credentials โ€” enforced by DCM's Ingress and Auth subsystems
+
+Neither layer alone is sufficient. The mesh layer prevents impersonation at the transport level; the application layer enforces what each component is permitted to do.
+
+---
+
+## 2. Component Identity Model
+
+Every DCM control plane component has a **component identity** โ€” a stable, verifiable identity used for both mTLS and application-layer authorization.
+
+```yaml
+component_identity:
+  component_uuid:            # stable; assigned at deployment time
+  component_type: api_gateway | policy_engine | placement_engine | request_orchestrator |
+                  scoring_engine | drift_reconciler | lifecycle_enforcer | notification_router |
+                  audit_store | session_store | message_bus | credential_provider_proxy
+  component_name:          # human-readable; e.g. "policy-engine-eu-west-1"
+  deployment_uuid:           # identifies the DCM deployment instance
+  
+  # Certificate identity
+  mtls_certificate:
+    subject: "CN=-,O=dcm-internal"
+    san: [, , ]
+    issuer_ca: 
+    issued_at: 
+    valid_until: 
+    
+  # Service account (application layer)
+  service_account_uuid:      # DCM actor of type "component_service_account"
+  allowed_operations: []   # what this component may call
+  allowed_targets: []      # which components it may call
+```
+
+### 2.1 Component Types and Communication Graph
+
+Not every component may call every other. The allowed communication graph is declared and enforced:
+
+```
+Consumer/Admin/Provider โ†’ API Gateway
+API Gateway โ†’ Request Orchestrator
+API Gateway โ†’ Policy Engine (for direct policy evaluation)
+API Gateway โ†’ Session Store (token validation)
+
+Request Orchestrator โ†’ Policy Engine
+Request Orchestrator โ†’ Placement Engine  
+Request Orchestrator โ†’ Scoring Engine
+Request Orchestrator โ†’ Audit Store
+Request Orchestrator โ†’ Message Bus
+
+Policy Engine โ†’ Audit Store
+Policy Engine โ†’ Message Bus (policy evaluation events)
+
+Placement Engine โ†’ Audit Store
+Placement Engine โ†’ Message Bus
+
+Scoring Engine โ†’ Audit Store
+
+Drift Reconciler โ†’ API Gateway (discovery dispatch)
+Drift Reconciler โ†’ Audit Store
+Drift Reconciler โ†’ Message Bus
+
+Lifecycle Enforcer โ†’ API Gateway (decommission dispatch)
+Lifecycle Enforcer โ†’ Audit Store
+Lifecycle Enforcer โ†’ Message Bus
+
+Notification Router โ†’ Message Bus (subscribe)
+Notification Router โ†’ Credential Provider Proxy (notification channel credentials)
+
+All components โ†’ Session Store (revocation check)
+All components โ†’ Credential Provider Proxy (interaction credential requests)
+```
+
+**ICOM-004:** Components may only call components declared in their `allowed_targets` list. A call from an unexpected source component is rejected with `403 Forbidden` and an audit record.
+
+---
+
+## 3. Certificate Issuance and Internal CA
+
+### 3.1 Certificate Authority for Internal Components
+
+Each DCM deployment uses a **registered Certificate Authority (CA)** for issuing component mTLS certificates. This may be:
+
+**Option A โ€” Built-in Internal CA (default):** DCM operates its own CA per deployment. Simple to configure; no external dependencies; suitable for minimal through standard profiles.
+
+**Option B โ€” External CA via Credential Provider:** An enterprise CA registered as a Credential Provider (HashiCorp Vault PKI, Venafi TLS Protect, EJBCA, AWS ACM Private CA, Azure Key Vault). The external CA issues component certificates using the standard Credential Provider interface โ€” DCM requests certificates via the provider's API (ACME/EST/SCEP/CMP). See [Credential Provider Model](31-credential-provider-model.md) for registration. Recommended for fsi and sovereign profiles where the enterprise PKI chain must be maintained.
+
+Both options satisfy ICOM-001 (mTLS required). The distinction is who issues the certificates, not whether mTLS is used.
+
+**The registered CA's root certificate is installed in all component trust stores at deployment time.** For Option B, the Credential Provider's CA root (which may itself be a subordinate of an enterprise root) is the trust anchor.
+
+```yaml
+internal_ca:
+  ca_uuid: 
+  deployment_uuid: 
+  ca_type: built_in | external_credential_provider
+  credential_provider_uuid:   # if ca_type: external
+  external_ca_protocol: acme | est | scep | cmp | null  # if external
+  root_cert_fingerprint: 
+  certificate_lifetime: P90D           # profile-governed โ€” see table below
+  renewal_trigger: P14D
+  algorithm: ECDSA-P-384               # FIPS-compliant; all profiles
+  crl_endpoint: 
+  ocsp_endpoint: 
+```
+
+```yaml
+internal_ca:
+  ca_uuid: 
+  deployment_uuid: 
+  root_cert_fingerprint: 
+  certificate_lifetime: P90D           # all component certs valid 90 days
+  renewal_trigger: P14D                # renew 14 days before expiry
+  algorithm: ECDSA-P-384               # FIPS-compliant for all profiles
+  crl_endpoint:          # revocation list for component certs
+  ocsp_endpoint:         # online status check
+```
+
+### 3.2 Profile-Governed Certificate Configuration
+
+| Profile | Cert lifetime | Renewal trigger | Bootstrap token TTL | Min key algorithm |
+|---------|--------------|-----------------|--------------------|--------------------|
+| `minimal` | P180D | P30D | PT4H | RSA-2048 (min) |
+| `dev` | P90D | P14D | PT1H | RSA-2048 (min) |
+| `standard` | P90D | P14D | PT1H | ECDSA-P-256 (min) |
+| `prod` | P90D | P14D | PT1H | ECDSA-P-384 |
+| `fsi` | P30D | P7D | PT30M | ECDSA-P-384 |
+| `sovereign` | P14D | P3D | PT15M | ECDSA-P-384 (HSM-backed if hardware_attested) |
+
+> **sovereign profile:** Certificates must be HSM-backed if the deployment posture is `hardware_attested`. The external CA option (Option B) using an HSM-backed Vault PKI backend satisfies this requirement.
+
+### 3.3 Certificate Lifecycle
+
+```
+Component starts
+  โ”‚
+  โ–ผ Does component have a valid certificate?
+  โ”‚   YES โ†’ Use existing certificate
+  โ”‚   NO (first start or expired) โ†’ Request certificate from Internal CA
+  โ”‚
+  โ–ผ Certificate request to Internal CA:
+  โ”‚   component_uuid, component_type, deployment_uuid
+  โ”‚   CSR signed with bootstrap key (see Section 5)
+  โ”‚
+  โ–ผ Internal CA issues certificate
+  โ”‚   Subject: CN=-,O=dcm-internal
+  โ”‚   SAN: component_uuid, component_name, internal DNS name
+  โ”‚   Valid for: P90D (profile-governed)
+  โ”‚
+  โ–ผ Component stores certificate; begins accepting mTLS connections
+  โ”‚
+  โ–ผ 14 days before expiry: auto-renewal
+      Background thread requests new certificate
+      Transition: both old and new cert valid for PT1H
+      Old cert retired after transition
+```
+
+---
+
+## 4. Application-Layer Authorization
+
+mTLS verifies **who** is calling. Application-layer authorization verifies **what** the caller is permitted to do.
+
+### 4.1 Interaction Credential for Internal Calls
+
+Every internal component call follows the same ZTS-002 scoped interaction credential model used for external provider dispatch:
+
+```
+Component A prepares to call Component B
+  โ”‚
+  โ–ผ Request interaction credential from Credential Provider Proxy:
+  โ”‚   credential_type: dcm_interaction
+  โ”‚   issued_to.component_uuid: 
+  โ”‚   scope.operations: []
+  โ”‚   scope.target_component: 
+  โ”‚   valid_until: 
+  โ”‚
+  โ–ผ Call Component B with:
+  โ”‚   mTLS certificate (transport identity)
+  โ”‚   Interaction credential in Authorization header (operation authorization)
+  โ”‚   Correlation ID (tracing)
+  โ”‚
+  โ–ผ Component B validates:
+  โ”‚   1. mTLS cert from Component A's known CA โœ“
+  โ”‚   2. Interaction credential: not revoked, not expired, scoped to this operation โœ“
+  โ”‚   3. Component A is in allowed_sources for this endpoint โœ“
+  โ”‚   4. Operation matches declared scope โœ“
+  โ”‚   โ†’ All pass: process request
+  โ”‚   โ†’ Any fail: 403 + audit record ICOM_AUTH_FAILURE
+```
+
+### 4.2 Internal Endpoint Authorization
+
+Each internal component endpoint declares which source components are permitted to call it:
+
+```yaml
+internal_endpoint:
+  component: policy_engine
+  endpoint: POST /internal/evaluate
+  allowed_sources:
+    - api_gateway
+    - request_orchestrator
+  required_scope: policy.evaluate
+  audit_every_call: true           # all internal calls are audited
+```
+
+**ICOM-003:** Internal endpoints that receive calls from unauthorized source components return 403 and write an `ICOM_UNAUTHORIZED_SOURCE` audit record. This audit record has urgency: high โ€” unexpected internal call patterns are security signals.
+
+---
+
+## 5. Bootstrap โ€” First Certificate
+
+The bootstrap problem: a new component needs a certificate, but it has no certificate yet to authenticate its request. DCM solves this with a **bootstrap token** mechanism.
+
+### 5.1 Bootstrap Token
+
+At deployment time, the platform admin generates a one-time bootstrap token for each component:
+
+```
+POST /api/v1/admin/components/bootstrap-tokens
+
+{
+  "component_type": "policy_engine",
+  "component_uuid": "",
+  "deployment_uuid": "",
+  "expires_at": ""     // short-lived: PT1H maximum
+}
+
+Response 201:
+{
+  "bootstrap_token": "",    // one-time use; stored as env var or secret
+  "component_uuid": "",
+  "expires_at": ""
+}
+```
+
+### 5.2 First Certificate Acquisition
+
+```
+New component starts with bootstrap_token in environment
+  โ”‚
+  โ–ผ POST /internal/ca/issue-certificate
+  โ”‚   Authorization: Bootstrap 
+  โ”‚   Body: { component_uuid, component_type, deployment_uuid, csr_pem }
+  โ”‚
+  โ–ผ Internal CA validates:
+  โ”‚   Bootstrap token not expired
+  โ”‚   Bootstrap token not previously used (one-time)
+  โ”‚   component_uuid matches token's declared component_uuid
+  โ”‚
+  โ–ผ Certificate issued
+  โ”‚   Bootstrap token invalidated immediately after use
+  โ”‚
+  โ–ผ Component uses certificate for all subsequent communication
+    No further need for bootstrap token
+```
+
+**ICOM-007:** Bootstrap tokens are one-time-use and must expire within PT1H of creation. A bootstrap token that is not used within PT1H is automatically invalidated. Platform admins must generate new tokens if a component fails to start within the window.
+
+### 5.3 Kubernetes Deployment Integration
+
+In Kubernetes deployments, bootstrap tokens are injected as Kubernetes Secrets and mounted as environment variables. The component reads the bootstrap token on startup, acquires its certificate, then deletes the Kubernetes Secret. This ensures the bootstrap credential is not persisted beyond initial use.
+
+```yaml
+# Kubernetes Secret (deleted by component after first cert acquisition)
+apiVersion: v1
+kind: Secret
+metadata:
+  name: dcm-policy-engine-bootstrap
+type: Opaque
+stringData:
+  DCM_BOOTSTRAP_TOKEN: ""
+  DCM_COMPONENT_UUID: ""
+  DCM_INTERNAL_CA_ENDPOINT: "https://dcm-internal-ca.dcm-system.svc.cluster.local"
+```
+
+---
+
+## 6. Certificate Compromise Response
+
+If a component certificate is compromised, the response follows the same emergency pattern as credential compromise:
+
+```
+Certificate compromise detected
+  โ”‚
+  โ–ผ Compromised cert added to Internal CA CRL
+  โ”‚   CRL update propagated to all components within SLA:
+  โ”‚     standard/prod: PT1M
+  โ”‚     fsi/sovereign: PT15S
+  โ”‚
+  โ–ผ Component identity suspended in DCM
+  โ”‚   All active interaction credentials for this component โ†’ revoked
+  โ”‚   ICOM_CERT_COMPROMISED audit record written
+  โ”‚
+  โ–ผ Platform admin notified (urgency: critical)
+  โ”‚
+  โ–ผ New certificate issued for legitimate component instance
+  โ”‚   Previous certificate remains in CRL permanently
+  โ”‚
+  โ–ผ Component resumes with new certificate
+```
+
+**ICOM-008:** Compromised internal component certificates are added to the Internal CA CRL immediately. All other components refresh their CRL cache within the profile-governed SLA and reject connections presenting the revoked certificate.
+
+---
+
+## 7. Deployment Architecture Summary
+
+```
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚                    DCM Control Plane                             โ”‚
+โ”‚                                                                  โ”‚
+โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  mTLS+cred  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”                  โ”‚
+โ”‚  โ”‚API Gatewayโ”‚โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ†’โ”‚Request Orchestratorโ”‚                 โ”‚
+โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜             โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜                  โ”‚
+โ”‚                                    โ”‚ mTLS+cred (each call)       โ”‚
+โ”‚                    โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”            โ”‚
+โ”‚                    โ†“               โ†“               โ†“            โ”‚
+โ”‚             โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”      โ”‚
+โ”‚             โ”‚Policy Engineโ”‚ โ”‚Placement โ”‚ โ”‚Scoring Engineโ”‚      โ”‚
+โ”‚             โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚Engine    โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜      โ”‚
+โ”‚                             โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜                        โ”‚
+โ”‚                                                                  โ”‚
+โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”   โ”‚
+โ”‚  โ”‚Credential Providerโ”‚  โ”‚Session Storeโ”‚  โ”‚Internal CA       โ”‚   โ”‚
+โ”‚  โ”‚Proxy             โ”‚  โ”‚             โ”‚  โ”‚(cert authority)   โ”‚   โ”‚
+โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜   โ”‚
+โ”‚                                                                  โ”‚
+โ”‚  Service Mesh (Istio): mTLS enforcement at transport layer      โ”‚
+โ”‚  All calls: authenticated + authorized + audited                โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+```
+
+---
+
+## 8. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ICOM-001` | All internal component-to-component communication must use mTLS with certificates issued by the deployment's Internal CA. Plaintext internal communication is prohibited in all profiles. |
+| `ICOM-002` | Every internal call must present a scoped interaction credential (ZTS-002) in addition to the mTLS certificate. The mTLS certificate proves identity; the interaction credential proves authorization for the specific operation. |
+| `ICOM-003` | Internal endpoints reject calls from components not in their `allowed_sources` list with 403 and an `ICOM_UNAUTHORIZED_SOURCE` audit record (urgency: high). |
+| `ICOM-004` | Components may only call components declared in their `allowed_targets` list. Attempts to call unauthorized components are rejected at the mesh layer (traffic policy) and, if they reach the application layer, at the application layer. |
+| `ICOM-005` | All internal component calls are audited: source component, target component, operation, interaction credential UUID, outcome. Internal audit records are written to the same Audit Store as external interactions. |
+| `ICOM-006` | Component certificates are issued by the Internal CA with a maximum validity of P90D and renewed automatically P14D before expiry. Component certificates may not be issued by external CAs. |
+| `ICOM-007` | Bootstrap tokens are one-time-use and expire within PT1H. A bootstrap token that has been used is immediately invalidated. Unused tokens are invalidated at expiry. |
+| `ICOM-008` | Compromised internal component certificates are added to the Internal CA CRL immediately. All components refresh their CRL cache within the profile-governed SLA. |
+| `ICOM-009` | The trust anchor for internal component mTLS is a registered root or intermediate CA whose certificate is installed in all component trust stores at deployment time. The trust anchor may be the built-in Internal CA or an external CA registered as a Certificate Provider (e.g. HashiCorp Vault PKI, Venafi, EJBCA) โ€” see [Credential Provider Model](31-credential-provider-model.md) Section on External CAs. Components do not accept certificates from unregistered trust anchors. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/37-scheduled-requests.md b/content/docs/data-model/37-scheduled-requests.md
new file mode 100644
index 0000000..dad7660
--- /dev/null
+++ b/content/docs/data-model/37-scheduled-requests.md
@@ -0,0 +1,291 @@
+# DCM Data Model โ€” Scheduled and Deferred Requests
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Request Scheduling
+**Related Documents:** [Resource and Service Entities](06-resource-service-entities.md) | [Operational Models](24-operational-models.md) | [Request Dependency Graph](38-request-dependency-graph.md) | [Event Catalog](33-event-catalog.md) | [Consumer API Specification](../specifications/consumer-api-spec.md)
+
+> **This document maps to: DATA + PROVIDER**
+>
+> A scheduled request is still a request โ€” it goes through the same Intent โ†’ Requested โ†’ Realized pipeline. The only difference is when the pipeline's dispatch step fires. Scheduling is a field on the request, not a separate object type. The Request Orchestrator handles dispatch timing; the Policy Engine evaluates at declaration time (gatekeeping) and again at dispatch time (policy correctness at the moment of execution).
+
+---
+
+## 1. The Scheduling Model
+
+### 1.1 Core Concept
+
+Every DCM request has an implicit `schedule: immediate`. Scheduled requests make this explicit:
+
+```yaml
+# Standard immediate request (implicit)
+schedule:
+  dispatch: immediate
+
+# Deferred โ€” dispatch at a specific time
+schedule:
+  dispatch: at
+  not_before: "2026-04-01T02:00:00Z"   # UTC; dispatch begins at or after this time
+  not_after: "2026-04-01T04:00:00Z"    # optional deadline; cancel if missed
+
+# Maintenance window โ€” dispatch during the next matching window
+schedule:
+  dispatch: window
+  window_id:   # references a declared Maintenance Window
+  not_after: "2026-04-30T00:00:00Z"    # optional: cancel if no window occurs before this
+
+# Recurring โ€” for decommission, TTL extension, or rehydration operations
+schedule:
+  dispatch: recurring
+  cron: "0 2 * * 0"                    # cron expression (UTC)
+  max_occurrences: 4                   # optional limit
+  not_after: "2026-12-31T00:00:00Z"   # optional end date
+```
+
+### 1.2 What Can Be Scheduled
+
+Scheduling applies to any request operation that results in a dispatch to a provider. This includes:
+
+| Operation | Scheduling supported | Notes |
+|-----------|---------------------|-------|
+| Resource creation | โœ… | Full scheduling model |
+| Resource update (PATCH) | โœ… | Full scheduling model |
+| Suspend | โœ… | Full scheduling model |
+| Resume | โœ… | Full scheduling model |
+| Decommission | โœ… | Full scheduling model; `not_after` recommended |
+| Rehydration | โœ… | Full scheduling model |
+| TTL extension | โœ… | Full scheduling model |
+| Discovery trigger | โŒ | Handled by Discovery Scheduling Model (doc 24 ยง4) |
+
+---
+
+## 2. Request State During Deferral
+
+A scheduled request moves through the four states with one additional intermediate status:
+
+```
+Submit request with schedule.dispatch: at
+  โ”‚
+  โ–ผ ACKNOWLEDGED (Intent State created)
+  โ”‚   entity_uuid assigned
+  โ”‚   schedule stored in Intent State
+  โ”‚
+  โ–ผ Policy evaluation at declaration time
+  โ”‚   GateKeeper policies run immediately
+  โ”‚   If rejected: request fails before entering queue
+  โ”‚   If approved: request enters scheduled queue
+  โ”‚
+  โ–ผ SCHEDULED (new status within Intent State)
+  โ”‚   stored in Request Scheduler queue
+  โ”‚   visible via GET /api/v1/requests/{uuid}/status
+  โ”‚   cancellable: DELETE /api/v1/requests/{uuid}
+  โ”‚
+  โ–ผ [at not_before time] โ†’ Policy re-evaluation at dispatch
+  โ”‚   Transformation policies re-run (data may have changed)
+  โ”‚   GateKeeper re-evaluation with current data
+  โ”‚   If still approved: proceed to LAYERS_ASSEMBLED โ†’ dispatch
+  โ”‚   If rejected at dispatch time: FAILED with reason schedule_policy_rejection
+  โ”‚
+  โ–ผ DISPATCHED โ†’ REALIZED (normal pipeline)
+```
+
+### 2.1 Why Policy Runs Twice
+
+Policies are evaluated at declaration time to catch obvious rejections early (fail fast). They run again at dispatch time because data may have changed โ€” quota may be exhausted, a compliance policy may have been activated, the actor's role may have changed. The dispatch-time evaluation uses the current policy set, not the one in effect at declaration.
+
+**SCH-003:** Scheduled requests that fail dispatch-time policy re-evaluation enter FAILED state with `failure_reason: schedule_policy_rejection`. The consumer receives a `request.failed` event with the policy rejection detail.
+
+---
+
+## 3. Maintenance Windows
+
+A Maintenance Window is a reusable schedule artifact โ€” a named recurrence that scheduled requests can reference. This allows operations teams to declare approved change windows once and have requests automatically slot into them.
+
+```yaml
+maintenance_window:
+  window_uuid: 
+  window_handle: "weekly-sunday-0200-utc"
+  description: "Weekly maintenance window โ€” low traffic period"
+  
+  # Recurrence
+  cron: "0 2 * * 0"          # every Sunday at 02:00 UTC
+  duration: PT2H              # window is 2 hours long
+  
+  # Scope
+  tenant_uuid:   # null = platform-wide window
+  resource_types: []     # empty = all resource types
+  
+  # Approval
+  status: active | suspended
+  approved_by: 
+  effective_from: 
+  
+  # Metadata
+  created_at: 
+  created_by: 
+```
+
+### 3.1 Maintenance Window API
+
+```
+# Platform admin operations
+POST   /api/v1/admin/maintenance-windows
+GET    /api/v1/admin/maintenance-windows
+GET    /api/v1/admin/maintenance-windows/{window_uuid}
+PATCH  /api/v1/admin/maintenance-windows/{window_uuid}
+DELETE /api/v1/admin/maintenance-windows/{window_uuid}
+
+# Consumer operations
+GET    /api/v1/maintenance-windows          # list windows visible to consumer
+GET    /api/v1/maintenance-windows/{uuid}  # describe a specific window
+```
+
+---
+
+## 4. The Request Scheduler Component
+
+The Request Scheduler is a DCM control plane component responsible for managing the scheduled request queue and triggering dispatch at the appropriate time.
+
+```
+Request Scheduler responsibilities:
+  - Maintain a priority queue of SCHEDULED requests ordered by not_before
+  - Poll queue; dispatch requests when not_before is reached
+  - Check not_after deadlines; cancel expired requests with reason: schedule_deadline_missed
+  - Listen for maintenance_window events to trigger window-scheduled requests
+  - On dispatch: hand off to Request Orchestrator (same path as immediate requests)
+  - Write SCHEDULED status updates to Intent State
+  - Publish request.scheduled and request.schedule_cancelled events
+```
+
+### 4.1 Deadline Enforcement
+
+If a request has `not_after` set and the deadline passes before dispatch:
+
+```
+not_after reached without dispatch
+  โ”‚
+  โ–ผ Request status โ†’ FAILED
+  โ”‚   failure_reason: schedule_deadline_missed
+  โ”‚
+  โ–ผ request.failed event published (urgency: medium)
+  โ”‚   consumer notified
+  โ”‚
+  โ–ผ Intent State marked terminal โ€” no further retries
+```
+
+---
+
+## 5. Consumer API Additions
+
+### 5.1 Submit Scheduled Request
+
+Scheduling is an optional `schedule` field on the existing request submission body:
+
+```
+POST /api/v1/requests
+
+{
+  "catalog_item_uuid": "",
+  "fields": { ... },
+  "schedule": {
+    "dispatch": "at",
+    "not_before": "2026-04-01T02:00:00Z",
+    "not_after": "2026-04-01T06:00:00Z"
+  }
+}
+
+Response 202:
+{
+  "request_uuid": "",
+  "entity_uuid": "",
+  "status": "SCHEDULED",
+  "scheduled_dispatch_at": "2026-04-01T02:00:00Z",
+  "schedule_deadline": "2026-04-01T06:00:00Z"
+}
+```
+
+### 5.2 List Scheduled Requests
+
+```
+GET /api/v1/requests?status=SCHEDULED
+
+Response 200:
+{
+  "items": [
+    {
+      "request_uuid": "",
+      "entity_uuid": "",
+      "status": "SCHEDULED",
+      "catalog_item_handle": "compute.vm.standard",
+      "scheduled_dispatch_at": "2026-04-01T02:00:00Z",
+      "schedule_deadline": "2026-04-01T06:00:00Z",
+      "created_at": ""
+    }
+  ],
+  "total": 3
+}
+```
+
+### 5.3 Cancel Scheduled Request
+
+Cancellation uses the existing endpoint โ€” no new endpoint needed:
+
+```
+DELETE /api/v1/requests/{request_uuid}
+
+# Works on SCHEDULED requests; moves status to CANCELLED
+# Returns 409 if request is already dispatched (past SCHEDULED)
+
+Response 204 No Content
+```
+
+---
+
+## 6. New Events
+
+Two new event types for the Event Catalog (doc 33):
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `request.scheduled` | info | Request entered SCHEDULED queue |
+| `request.schedule_cancelled` | low | Scheduled request cancelled before dispatch |
+| `request.schedule_deadline_missed` | medium | not_after passed without dispatch |
+
+These add to the `request.*` domain. Updated domain total: 17 request events.
+
+---
+
+## 8. Profile-Governed Scheduling Configuration
+
+Scheduling constraints are profile-governed to reflect the operational risk tolerance of each deployment context:
+
+| Profile | Max scheduling horizon | Max concurrent scheduled/actor | Recurring max frequency | Maintenance window approval tier |
+|---------|----------------------|-------------------------------|------------------------|----------------------------------|
+| `minimal` | P365D | unlimited | PT1H | auto |
+| `dev` | P365D | 50 | PT1H | auto |
+| `standard` | P90D | 20 | PT4H | reviewed |
+| `prod` | P30D | 10 | PT12H | reviewed |
+| `fsi` | P14D | 5 | PT24H | verified |
+| `sovereign` | P7D | 3 | PT24H | authorized |
+
+**Max scheduling horizon:** How far in the future a `not_before` may be set. Requests with `not_before` beyond the profile limit are rejected (422) at submission.
+
+**Max concurrent scheduled/actor:** How many SCHEDULED (not yet dispatched) requests a single actor may have at one time. Exceeding this limit returns 429.
+
+**Recurring max frequency:** The minimum interval between recurring dispatches. A cron expression that would dispatch more frequently than this is rejected.
+
+**Maintenance window approval tier:** The authority tier required to create or modify a Maintenance Window (see ATM-001, doc 32).
+
+## 7. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `SCH-001` | Scheduled requests undergo GateKeeper policy evaluation at declaration time (to catch rejections early) and again at dispatch time (to validate against current state). Both evaluations must pass. |
+| `SCH-002` | The `not_before` field must be a future timestamp at submission time. DCM rejects scheduled requests with a past `not_before` (returns 422). |
+| `SCH-003` | Requests that fail dispatch-time policy re-evaluation enter FAILED state with `failure_reason: schedule_policy_rejection`. Consumers receive a `request.failed` event with the rejection detail. |
+| `SCH-004` | Scheduled requests are cancellable (DELETE /api/v1/requests/{uuid}) at any time before dispatch. Once the Request Orchestrator has accepted the handoff (status moves beyond SCHEDULED), cancellation follows the standard cancellation model. |
+| `SCH-005` | If `not_after` is set and passes without dispatch, the request enters FAILED state with `failure_reason: schedule_deadline_missed`. No retry is attempted. |
+| `SCH-006` | Maintenance Windows are platform-level or tenant-scoped artifacts requiring platform admin approval. Window schedules are versioned artifacts subject to standard DCM lifecycle. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/38-request-dependency-graph.md b/content/docs/data-model/38-request-dependency-graph.md
new file mode 100644
index 0000000..6b9f311
--- /dev/null
+++ b/content/docs/data-model/38-request-dependency-graph.md
@@ -0,0 +1,305 @@
+# DCM Data Model โ€” Consumer Request Dependency Graph
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Cross-Request Ordering
+**Related Documents:** [Service Dependencies](07-service-dependencies.md) | [Scheduled Requests](37-scheduled-requests.md) | [Meta Provider Composability](30-meta-provider-model.md) | [Operational Models](24-operational-models.md) | [Consumer API Specification](../specifications/consumer-api-spec.md)
+
+> **Events:** Dependency resolution events (`request.dependencies_resolved`, `dependency.state_changed`) are defined in the [Event Catalog](33-event-catalog.md).
+
+> **This document maps to: DATA + PROVIDER**
+>
+> **Distinction from existing dependency models:**
+> - Doc 07 (Service Dependencies): *type-level* dependencies โ€” DCM knows that a VM *type* requires an IP type. Resolved automatically during layer assembly.
+> - Doc 30 (Meta Provider): *compound service* dependencies โ€” a Meta Provider declares its own constituents and DCM sequences them. Consumer does not manage this.
+> - **This document**: *consumer-declared cross-request ordering* โ€” a consumer submitting multiple independent requests says "Request B may not dispatch until Request A is realized." These are requests for different resource types that have no type-level dependency; the consumer is expressing an ordering constraint for their specific deployment.
+
+---
+
+## 1. The Problem
+
+A consumer deploying a three-tier application submits three requests: a database VM, an application VM, and a load balancer. Without ordering, all three dispatch simultaneously. But the application VM's startup configuration needs the database's IP address, which only exists after the database is realized.
+
+This is not a type-level dependency (the VM type does not require a VM type). It is a *deployment-time ordering constraint* declared by the consumer for this specific deployment.
+
+Meta Provider composition handles this when a platform team has pre-defined the compound service. But consumers also need to express ad-hoc ordering for their own deployments without requiring a Meta Provider to exist.
+
+---
+
+## 2. The Request Dependency Graph
+
+A Request Dependency Group is a consumer-declared set of requests with ordering constraints between them.
+
+```yaml
+request_dependency_group:
+  group_uuid: 
+  group_handle: "three-tier-app-deploy"   # optional, consumer-defined
+  
+  requests:
+    - request_uuid:                # database VM
+      depends_on: []                     # no dependencies โ€” dispatches immediately
+      
+    - request_uuid:                # application VM
+      depends_on:
+        - request_uuid: 
+          wait_for: realized             # dispatch only after db is REALIZED
+          inject_fields:                 # optional: inject realized fields into this request
+            - from_field: "realized_fields.primary_ip"
+              to_field: "fields.db_host"
+              
+    - request_uuid:                # load balancer
+      depends_on:
+        - request_uuid: 
+          wait_for: realized
+          inject_fields:
+            - from_field: "realized_fields.primary_ip"
+              to_field: "fields.backend_hosts[0]"
+  
+  # Group-level options
+  on_failure: cancel_remaining | continue   # what to do if a request fails
+  timeout: PT2H                             # group-level deadline
+```
+
+### 2.1 wait_for Values
+
+| Value | Meaning |
+|-------|---------|
+| `acknowledged` | Dispatch as soon as dependency has an entity_uuid |
+| `approved` | Dispatch when dependency has passed approval |
+| `dispatched` | Dispatch when dependency has been sent to its provider |
+| `realized` | Dispatch only when dependency is fully realized (default, most common) |
+
+### 2.2 Field Injection
+
+The `inject_fields` mechanism passes realized output fields from a dependency directly into a dependent request's fields โ€” without the consumer having to poll and re-submit. The injection happens at dispatch time, after the dependency is realized.
+
+```
+Dependency realized โ†’ Realized State written
+  โ”‚
+  โ–ผ DCM reads inject_fields declarations for dependent requests
+  โ”‚   For each injection: extract from_field from Realized State
+  โ”‚   Inject into dependent request's field at to_field path
+  โ”‚
+  โ–ผ Dependent request proceeds to layer assembly with injected fields
+```
+
+Field injection is subject to the same transformation policies as any other field โ€” if a policy transforms `db_host`, the injection result passes through it.
+
+---
+
+## 3. Submitting a Dependency Group
+
+### 3.1 Declare and Submit in One Call
+
+```
+POST /api/v1/request-groups
+
+{
+  "group_handle": "three-tier-app-deploy",
+  "on_failure": "cancel_remaining",
+  "timeout": "PT2H",
+  "requests": [
+    {
+      "ref": "db",                          # local reference within this submission
+      "catalog_item_uuid": "",
+      "fields": { "cpu_count": 8, "memory_gb": 32, "role": "database" }
+    },
+    {
+      "ref": "app",
+      "catalog_item_uuid": "",
+      "fields": { "cpu_count": 4, "memory_gb": 16, "role": "application" },
+      "depends_on": [
+        {
+          "ref": "db",
+          "wait_for": "realized",
+          "inject_fields": [
+            { "from_field": "realized_fields.primary_ip", "to_field": "fields.db_host" }
+          ]
+        }
+      ]
+    },
+    {
+      "ref": "lb",
+      "catalog_item_uuid": "",
+      "fields": { "backend_port": 8080 },
+      "depends_on": [
+        { "ref": "app", "wait_for": "realized",
+          "inject_fields": [
+            { "from_field": "realized_fields.primary_ip", "to_field": "fields.backend_hosts[0]" }
+          ]
+        }
+      ]
+    }
+  ]
+}
+
+Response 202:
+{
+  "group_uuid": "",
+  "group_handle": "three-tier-app-deploy",
+  "requests": [
+    { "ref": "db",  "request_uuid": "", "entity_uuid": "", "status": "ACKNOWLEDGED" },
+    { "ref": "app", "request_uuid": "", "entity_uuid": "", "status": "PENDING_DEPENDENCY" },
+    { "ref": "lb",  "request_uuid": "", "entity_uuid": "", "status": "PENDING_DEPENDENCY" }
+  ],
+  "estimated_completion": ""
+}
+```
+
+### 3.2 Add an Existing Request to a Group
+
+```
+POST /api/v1/request-groups/{group_uuid}/members
+
+{
+  "request_uuid": "",
+  "depends_on": [ ... ]
+}
+```
+
+### 3.3 Query Group Status
+
+```
+GET /api/v1/request-groups/{group_uuid}
+
+Response 200:
+{
+  "group_uuid": "",
+  "group_handle": "three-tier-app-deploy",
+  "status": "in_progress | completed | failed | cancelled",
+  "requests": [
+    { "request_uuid": "", "ref": "db",  "status": "REALIZED" },
+    { "request_uuid": "", "ref": "app", "status": "DISPATCHED" },
+    { "request_uuid": "", "ref": "lb",  "status": "PENDING_DEPENDENCY" }
+  ],
+  "created_at": "",
+  "timeout_at": ""
+}
+```
+
+### 3.4 Cancel a Group
+
+```
+DELETE /api/v1/request-groups/{group_uuid}
+
+# Cancels all PENDING_DEPENDENCY and ACKNOWLEDGED requests in the group
+# Already-dispatched requests follow standard cancellation model
+Response 204
+```
+
+---
+
+## 4. PENDING_DEPENDENCY Status
+
+A request in a dependency group that is waiting for its dependency to reach `wait_for` state has status `PENDING_DEPENDENCY`. This is a new status in the Intent State lifecycle:
+
+```
+ACKNOWLEDGED โ†’ PENDING_DEPENDENCY โ†’ [dependency met] โ†’ LAYERS_ASSEMBLED โ†’ ... โ†’ REALIZED
+```
+
+`PENDING_DEPENDENCY` requests:
+- Are visible in `GET /api/v1/requests` with `status=PENDING_DEPENDENCY`
+- Can be cancelled: `DELETE /api/v1/requests/{uuid}`
+- Receive the `request.pending_dependency` event (new, info urgency)
+- Do not time out independently โ€” the group-level `timeout` governs
+
+---
+
+## 5. Failure Handling
+
+### 5.1 `on_failure: cancel_remaining`
+
+When a request in the group fails and `on_failure: cancel_remaining` is set:
+
+```
+Request fails
+  โ”‚
+  โ–ผ All PENDING_DEPENDENCY and ACKNOWLEDGED requests in group โ†’ CANCELLED
+  โ”‚   failure_reason: dependency_failed
+  โ”‚
+  โ–ผ request.failed event for the failing request
+  โ”‚   request.cancelled events for each cancelled dependent
+  โ”‚
+  โ–ผ Group status โ†’ failed
+```
+
+### 5.2 `on_failure: continue`
+
+Failed request is marked FAILED; dependents that depended on it are also marked FAILED with `dependency_failed`. Independent requests in the group continue unaffected.
+
+### 5.3 Group Timeout
+
+If the group `timeout` duration elapses without all requests reaching a terminal state:
+
+```
+Group timeout reached
+  โ”‚
+  โ–ผ All non-terminal requests โ†’ FAILED
+  โ”‚   failure_reason: group_timeout
+  โ”‚
+  โ–ผ request.failed events for each
+  โ”‚   Group status โ†’ failed
+```
+
+---
+
+## 6. Relationship to Meta Providers
+
+Request dependency groups and Meta Providers solve overlapping but distinct problems:
+
+| | Request Dependency Group | Meta Provider |
+|--|---|---|
+| **Who declares** | Consumer at request time | Platform team at catalog time |
+| **Reusable** | No โ€” ad hoc | Yes โ€” catalog item |
+| **Type constraints** | None โ€” any resources | Defined by Meta Provider spec |
+| **Policy governance** | Standard consumer request policies | Meta Provider policies (MPX-*) |
+| **Field injection** | Consumer-declared inject_fields | Meta Provider handles internally |
+| **Use case** | Ad-hoc deployment ordering | Standard compound service |
+
+When a standard compound service exists as a Meta Provider, consumers should use it. Request dependency groups are for deployments that don't fit a predefined compound service pattern.
+
+---
+
+## 7. New Events
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `request.pending_dependency` | info | Request entered PENDING_DEPENDENCY state |
+| `request.dependency_met` | info | Dependency reached wait_for state; request proceeding |
+| `request.group_completed` | medium | All requests in group reached terminal state |
+| `request.group_failed` | high | Group failed or timed out |
+
+---
+
+## 9. Profile-Governed Dependency Group Configuration
+
+| Profile | Max group size | Max group timeout | Field injection validation | Max nesting depth |
+|---------|---------------|-------------------|---------------------------|-------------------|
+| `minimal` | 100 | P30D | advisory (warn only) | 3 |
+| `dev` | 100 | P7D | advisory | 3 |
+| `standard` | 50 | P3D | enforced | 3 |
+| `prod` | 25 | P1D | enforced + audited | 3 |
+| `fsi` | 10 | PT8H | enforced + audited + policy gated | 2 |
+| `sovereign` | 5 | PT4H | enforced + audited + policy gated | 2 |
+
+**Max group size:** Maximum number of requests in a single dependency group. RDG-002 sets the absolute upper bound at 100; profiles may set lower limits.
+
+**Max group timeout:** Maximum value of the `timeout` field. Groups declaring a timeout beyond the profile limit are rejected (422).
+
+**Field injection validation:** `advisory` โ€” warns if injected fields fail schema validation but proceeds; `enforced` โ€” rejects dispatch if injected fields fail validation; `policy gated` โ€” field injection also passes through GateKeeper policy evaluation.
+
+**Max nesting depth:** Maximum depth of `depends_on` chains. Aโ†’Bโ†’C is depth 2. Exceeding this is rejected at submission (422).
+
+## 8. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `RDG-001` | Circular dependencies within a request group are rejected at submission time (422 Unprocessable Entity). DCM validates the dependency graph is a DAG before acknowledging the group. |
+| `RDG-002` | Maximum group size is 50 requests. Groups exceeding this must use Meta Provider composition or be split into multiple groups. |
+| `RDG-003` | Field injection (`inject_fields`) is subject to all active Transformation policies. Injected values are not exempt from policy evaluation. |
+| `RDG-004` | `PENDING_DEPENDENCY` requests count against the consumer's quota. Resources are reserved at group submission, not at dispatch time. |
+| `RDG-005` | Group-level `timeout` is measured from group submission. Individual requests do not have independent timeouts while in PENDING_DEPENDENCY status. |
+| `RDG-006` | A request may belong to at most one dependency group. Attempts to add a request to a second group return 409 Conflict. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/39-dcm-self-health.md b/content/docs/data-model/39-dcm-self-health.md
new file mode 100644
index 0000000..e3ba7ef
--- /dev/null
+++ b/content/docs/data-model/39-dcm-self-health.md
@@ -0,0 +1,370 @@
+# DCM Data Model โ€” DCM Self-Health Endpoints
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference โ€” Operational Health
+**Related Documents:** [Deployment and Redundancy](17-deployment-redundancy.md) | [Internal Component Authentication](36-internal-component-auth.md) | [Operator Interface Specification](../specifications/dcm-operator-interface-spec.md) | [Admin API Specification](../specifications/dcm-admin-api-spec.md)
+
+> **Events:** Health state change events fire as `provider.healthy` / `provider.unhealthy` for external systems, and `governance.profile_changed` when health thresholds are adjusted. See [Event Catalog](33-event-catalog.md).
+
+> **This document maps to: PROVIDER**
+>
+> DCM itself must expose the same health contract it requires of Service Providers (doc OIS ยง4). This document specifies DCM's own liveness, readiness, and component health endpoints โ€” required for Kubernetes operator deployment, load balancer health checking, and operational monitoring.
+
+---
+
+## 1. Three Health Endpoints
+
+DCM exposes three distinct health endpoints, following Kubernetes conventions:
+
+| Endpoint | Purpose | Failure action | Authentication |
+|----------|---------|---------------|---------------|
+| `GET /livez` | Is DCM alive? | Kubernetes restarts the pod | None |
+| `GET /readyz` | Is DCM ready to serve traffic? | Kubernetes removes from load balancer | None |
+| `GET /api/v1/admin/health` | Detailed component status | Informational โ€” no automatic action | Admin auth required |
+
+Liveness and readiness are unauthenticated because they must work before authentication infrastructure is operational (e.g. during startup).
+
+---
+
+## 2. Liveness โ€” `/livez`
+
+Liveness answers: **is this DCM process alive?**
+
+A liveness failure means the process is deadlocked, in an unrecoverable state, or otherwise unable to continue. Kubernetes responds by restarting the pod.
+
+```http
+GET /livez HTTP/1.1
+
+HTTP/1.1 200 OK
+Content-Type: application/health+json
+
+{
+  "status": "pass"
+}
+```
+
+**Liveness checks (minimal โ€” fast):**
+- Process is responding
+- No deadlock detected in core event loop
+- Internal CA is reachable (for deployments with component auth)
+
+**Liveness failure response:**
+```http
+HTTP/1.1 503 Service Unavailable
+Content-Type: application/health+json
+
+{
+  "status": "fail",
+  "failure_reason": "event_loop_deadlock | internal_ca_unreachable | oom_imminent"
+}
+```
+
+**Liveness SLA:** Must respond within PT5S. No external calls. No database reads.
+
+---
+
+## 3. Readiness โ€” `/readyz`
+
+Readiness answers: **is this DCM instance ready to serve requests?**
+
+A readiness failure removes the instance from the load balancer rotation without restarting it. This handles startup, migration, and graceful drain scenarios.
+
+```http
+GET /readyz HTTP/1.1
+
+HTTP/1.1 200 OK
+Content-Type: application/health+json
+
+{
+  "status": "pass",
+  "checks": {
+    "session_store": "pass",
+    "audit_store": "pass",
+    "policy_engine": "pass",
+    "message_bus": "pass",
+    "auth_provider": "pass"
+  }
+}
+```
+
+**Readiness checks:**
+- Session Store: can write and read a test record
+- Audit Store: reachable and writable
+- Policy Engine: responding to internal health ping
+- Message Bus: connected and subscribed
+- Auth Provider: at least one Auth Provider is responding
+- Schema version: database schema matches running code version
+
+**Readiness failure** (any check fails):
+```http
+HTTP/1.1 503 Service Unavailable
+Content-Type: application/health+json
+
+{
+  "status": "fail",
+  "checks": {
+    "session_store": "pass",
+    "audit_store": "fail",
+    "policy_engine": "pass",
+    "message_bus": "pass",
+    "auth_provider": "pass"
+  },
+  "failing_checks": ["audit_store"]
+}
+```
+
+**Readiness SLA:** Must respond within PT10S. Performs lightweight connectivity checks โ€” no heavy queries.
+
+### 3.1 Startup vs Operational Readiness
+
+During startup, DCM goes through a startup sequence before becoming ready:
+
+```
+Process starts
+  โ”‚
+  โ–ผ /livez โ†’ pass (process alive)
+  โ”‚   /readyz โ†’ fail (not ready yet)
+  โ”‚
+  โ–ผ Internal CA connects โ†’ component certs verified
+  โ–ผ Session Store connected โ†’ revocation registry loaded
+  โ–ผ Audit Store connected โ†’ schema version validated
+  โ–ผ Policy Engine ready โ†’ policies loaded and shadow mode initialized
+  โ–ผ Auth Providers connected โ†’ at least one responding
+  โ–ผ Message Bus connected โ†’ subscriptions established
+  โ”‚
+  โ–ผ /readyz โ†’ pass (ready to serve traffic)
+```
+
+`startupProbe` in Kubernetes uses `/readyz` with a longer `failureThreshold` to allow startup time before the liveness probe takes over.
+
+---
+
+## 4. Detailed Health โ€” `/api/v1/admin/health`
+
+The detailed health endpoint provides per-component status for operational monitoring. Requires admin authentication.
+
+```http
+GET /api/v1/admin/health HTTP/1.1
+Authorization: Bearer 
+
+HTTP/1.1 200 OK
+Content-Type: application/health+json
+
+{
+  "status": "pass | warn | fail",
+  "dcm_version": "1.2.0",
+  "dcm_instance_uuid": "",
+  "deployment_profile": "prod",
+  "uptime_seconds": 864023,
+  "checked_at": "",
+
+  "components": {
+    "api_gateway": {
+      "status": "pass",
+      "latency_p99_ms": 12,
+      "requests_per_minute": 340
+    },
+    "request_orchestrator": {
+      "status": "pass",
+      "queue_depth": 3,
+      "in_flight": 7
+    },
+    "policy_engine": {
+      "status": "pass",
+      "active_policies": 42,
+      "shadow_policies": 3,
+      "evaluations_per_minute": 280
+    },
+    "placement_engine": {
+      "status": "pass"
+    },
+    "scoring_engine": {
+      "status": "pass",
+      "evaluations_per_minute": 280
+    },
+    "request_scheduler": {
+      "status": "pass",
+      "scheduled_requests_queued": 5,
+      "next_dispatch_at": ""
+    },
+    "drift_reconciler": {
+      "status": "pass",
+      "last_cycle_completed_at": "",
+      "open_drift_records": 2
+    },
+    "lifecycle_enforcer": {
+      "status": "pass",
+      "entities_monitored": 1240,
+      "ttl_warnings_pending": 3
+    },
+    "discovery_scheduler": {
+      "status": "pass",
+      "pending_jobs": 1,
+      "last_completed_at": ""
+    },
+    "notification_router": {
+      "status": "pass",
+      "providers_active": 2,
+      "delivery_backlog": 0
+    },
+    "session_store": {
+      "status": "pass",
+      "active_sessions": 47,
+      "revocation_registry_size": 3
+    },
+    "audit_store": {
+      "status": "pass",
+      "records_last_hour": 1840,
+      "chain_integrity": "verified"
+    },
+    "message_bus": {
+      "status": "pass",
+      "lag_consumer_group_ms": 12
+    },
+    "internal_ca": {
+      "status": "pass",
+      "certificates_active": 12,
+      "next_expiry_at": ""
+    }
+  },
+
+  "providers": {
+    "registered": 4,
+    "healthy": 4,
+    "degraded": 0,
+    "unhealthy": 0
+  },
+
+  "auth_providers": {
+    "registered": 2,
+    "healthy": 2,
+    "unhealthy": 0
+  }
+}
+```
+
+### 4.1 Status Semantics
+
+| Status | Meaning |
+|--------|---------|
+| `pass` | Component fully operational |
+| `warn` | Operational but degraded (high latency, reduced capacity, elevated error rate) |
+| `fail` | Component not operational; DCM degraded |
+
+The top-level `status` is the worst status across all components:
+- Any `fail` โ†’ top-level `fail`
+- Any `warn`, no `fail` โ†’ top-level `warn`
+- All `pass` โ†’ top-level `pass`
+
+---
+
+## 5. Kubernetes Manifest
+
+```yaml
+# Standard Kubernetes probe configuration for DCM
+livenessProbe:
+  httpGet:
+    path: /livez
+    port: 8443
+    scheme: HTTPS
+  initialDelaySeconds: 10
+  periodSeconds: 10
+  failureThreshold: 3
+  timeoutSeconds: 5
+
+readinessProbe:
+  httpGet:
+    path: /readyz
+    port: 8443
+    scheme: HTTPS
+  initialDelaySeconds: 30
+  periodSeconds: 5
+  failureThreshold: 6
+  timeoutSeconds: 10
+
+startupProbe:
+  httpGet:
+    path: /readyz
+    port: 8443
+    scheme: HTTPS
+  initialDelaySeconds: 10
+  periodSeconds: 10
+  failureThreshold: 30    # allow up to 300s for startup
+  timeoutSeconds: 10
+```
+
+---
+
+## 6. Prometheus Metrics
+
+DCM exposes Prometheus-compatible metrics alongside health endpoints:
+
+```
+GET /metrics      # Prometheus scrape endpoint (unauthenticated in cluster; 
+                  # configurable for external exposure)
+```
+
+Key metric families:
+
+```
+# Request pipeline
+dcm_requests_total{status, resource_type, profile}
+dcm_request_duration_seconds{quantile, resource_type}
+dcm_requests_pending_dependency_total
+dcm_requests_scheduled_total
+
+# Policy engine
+dcm_policy_evaluations_total{outcome, enforcement_class}
+dcm_policy_shadow_divergences_total
+
+# Sessions
+dcm_sessions_active_total
+dcm_session_revocations_total{trigger}
+
+# Drift
+dcm_drift_open_records_total{severity}
+dcm_drift_detected_total
+
+# Providers
+dcm_providers_registered_total
+dcm_providers_healthy_total
+dcm_provider_dispatch_duration_seconds{provider_type, quantile}
+
+# Internal
+dcm_internal_ca_certificates_active
+dcm_internal_ca_days_until_next_expiry
+```
+
+---
+
+## 8. Profile-Governed Health Exposure
+
+| Profile | /livez | /readyz | /api/v1/admin/health | /metrics scraping |
+|---------|--------|---------|----------------------|-------------------|
+| `minimal` | Unauthenticated | Unauthenticated | Admin auth | Internal network only |
+| `dev` | Unauthenticated | Unauthenticated | Admin auth | Internal network only |
+| `standard` | Unauthenticated | Unauthenticated | Admin auth | mTLS client cert or auth token |
+| `prod` | Unauthenticated | Unauthenticated | Admin auth | mTLS client cert or auth token |
+| `fsi` | Unauthenticated | Unauthenticated | Admin auth (MFA required) | mTLS + authorized scraper registration |
+| `sovereign` | Unauthenticated within cluster | Unauthenticated within cluster | Admin auth (MFA + step-up) | Disabled externally; internal only |
+
+**Notes:**
+- `/livez` and `/readyz` are always unauthenticated *within the cluster* โ€” Kubernetes probes cannot present auth credentials. However, at the ingress boundary (external load balancer), these paths may be network-restricted.
+- For `fsi` and `sovereign` profiles, `/api/v1/admin/health` requires MFA-verified sessions (mfa_verified: true). Step-up MFA is required for sovereign.
+- The `sovereign` profile does not expose `/metrics` externally. Prometheus must scrape from within the cluster network only.
+- Component-level detail in `/api/v1/admin/health` may be redacted in fsi/sovereign profiles based on the requesting actor's role โ€” SRE sees full detail; read-only admin sees summary only.
+
+## 7. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `HLT-001` | DCM must expose `/livez` and `/readyz` endpoints on the same port as the API, unauthenticated, following RFC 8615 / IANA health+json format. |
+| `HLT-002` | `/livez` must respond within PT5S with no external calls or database reads. A non-response within PT5S is treated as liveness failure. |
+| `HLT-003` | `/readyz` returns `fail` if the Session Store, Audit Store, Policy Engine, Message Bus, or any Auth Provider is unreachable. It returns `warn` if any optional component is degraded. |
+| `HLT-004` | `GET /api/v1/admin/health` requires admin authentication and provides per-component status. It must include the DCM version, instance UUID, and deployment profile. |
+| `HLT-005` | DCM must expose Prometheus-compatible metrics at `GET /metrics`. Metrics must include request pipeline, policy engine, session, drift, and provider metrics at minimum. |
+| `HLT-006` | The startup sequence must be observable via `/readyz`. DCM must not report `pass` on `/readyz` until the Session Store, Audit Store, Policy Engine, Auth Provider, and Message Bus are all reachable. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/40-standards-catalog.md b/content/docs/data-model/40-standards-catalog.md
new file mode 100644
index 0000000..26c45aa
--- /dev/null
+++ b/content/docs/data-model/40-standards-catalog.md
@@ -0,0 +1,293 @@
+# DCM Data Model โ€” Standards and Compliance Catalog
+
+**Document Status:** ๐Ÿ”„ In Progress
+**Document Type:** Architecture Reference โ€” Normative Standards
+**Purpose:** Single authoritative source for all RFCs, protocols, specifications, and compliance frameworks referenced by the DCM architecture. For each standard: what it is, where DCM uses it, and what obligation it places on implementations.
+
+> **How to read this document:**
+> - **Normative** โ€” DCM implementations MUST comply with this standard in the specified context
+> - **Informative** โ€” DCM draws on this standard as guidance or reference without strict compliance
+> - **Optional** โ€” DCM supports this standard in applicable profiles or configurations
+
+---
+
+## 1. Internet Standards (IETF RFCs)
+
+### 1.1 Authentication and Authorization
+
+| RFC | Title | Use in DCM | Obligation |
+|-----|-------|-----------|-----------|
+| **RFC 7519** | JSON Web Token (JWT) | Bearer token format for session tokens and API key tokens; claims carry actor_uuid, roles, tenant_uuid, exp | Normative |
+| **RFC 7517** | JSON Web Key (JWK) | Public key format for Auth Provider OIDC verification keys; JWKS endpoint for key discovery | Normative |
+| **RFC 7662** | OAuth 2.0 Token Introspection | `POST /api/v1/auth/introspect` โ€” validates bearer tokens; response format `{active, session_uuid, actor_uuid, exp, roles}` | Normative |
+| **RFC 6749** | OAuth 2.0 Authorization Framework | Authorization flow for OIDC Auth Providers; client credentials flow for service account API keys | Informative |
+| **RFC 4511** | Lightweight Directory Access Protocol (LDAP) | LDAP/FreeIPA/Active Directory Auth Provider integration; bind operations, search filters for group membership | Normative |
+| **RFC 7643** | SCIM 2.0 Core Schema | Actor and group provisioning schema for enterprise IdP integration; SCIM deprovision triggers session + credential revocation | Normative |
+| **RFC 7644** | SCIM 2.0 Protocol | SCIM REST API for actor provisioning; DELETE triggers AUTH-016 (session) and CPX-006 (credential) revocation in parallel | Normative |
+
+### 1.2 Transport Security
+
+| RFC | Title | Use in DCM | Obligation |
+|-----|-------|-----------|-----------|
+| **RFC 8446** | TLS 1.3 | All external API communication; preferred TLS version; mandatory cipher suite compliance | Normative |
+| **RFC 5246** | TLS 1.2 | Permitted TLS version for compatibility; minimum acceptable version; TLS 1.0/1.1 prohibited | Normative |
+| **RFC 5280** | X.509 PKI Certificate and CRL Profile | All DCM certificates (component mTLS, Internal CA, Credential Provider certs); CRL format for revocation; certificate chain validation | Normative |
+| **RFC 6960** | Online Certificate Status Protocol (OCSP) | Internal CA OCSP endpoint for real-time certificate status; Internal CA CRL supplement | Normative |
+
+### 1.3 Certificate Enrollment
+
+| RFC | Title | Use in DCM | Obligation |
+|-----|-------|-----------|-----------|
+| **RFC 7030** | Enrollment over Secure Transport (EST) | Certificate enrollment for Internal CA (alternative to bootstrap token); preferred for automated cert lifecycle | Informative |
+| **RFC 8555** | ACME โ€” Automatic Certificate Management Environment | Automated certificate lifecycle for external-facing TLS certificates; provider certificates | Informative |
+| **RFC 8894** | Simple Certificate Enrolment Protocol (SCEP) | Legacy certificate enrollment for environments without EST/ACME support | Optional |
+| **RFC 4210** | Certificate Management Protocol (CMP) | X.509 PKI certificate management in enterprise PKI environments | Optional |
+
+### 1.4 API Lifecycle
+
+| RFC | Title | Use in DCM | Obligation |
+|-----|-------|-----------|-----------|
+| **RFC 8594** | The Sunset HTTP Header Field | Deprecated API version responses include `Sunset: ` header (VER-003); also `Deprecation` header | Normative |
+| **RFC 9745** | The Deprecation HTTP Header Field | Deprecated API version responses include `Deprecation: ` header paired with RFC 8594 Sunset | Normative |
+
+### 1.5 Service Discovery and Health
+
+| RFC | Title | Use in DCM | Obligation |
+|-----|-------|-----------|-----------|
+| **RFC 8615** | Well-Known Uniform Resource Identifiers | `GET /.well-known/dcm-api-versions` (version discovery); `/livez` and `/readyz` path conventions; IANA health+json media type | Normative |
+
+### 1.6 Data Formats
+
+| RFC | Title | Use in DCM | Obligation |
+|-----|-------|-----------|-----------|
+| **ISO 8601** | Date and Time Format | All timestamps in DCM: `created_at`, `expires_at`, `not_before`, `not_after`, event timestamps; durations as ISO 8601 periods (P90D, PT8H) | Normative |
+| **RFC 8259** | The JavaScript Object Notation (JSON) Data Interchange Format | All DCM API request/response bodies; all entity definitions in stores | Normative |
+
+---
+
+## 2. Identity and Access Protocols
+
+| Protocol | Specification | Use in DCM | Obligation |
+|----------|--------------|-----------|-----------|
+| **OIDC / OpenID Connect** | OpenID Foundation Core 1.0 | Primary enterprise Auth Provider type; ID token format; JWKS endpoint for key verification; userinfo endpoint for actor enrichment | Normative |
+| **SAML 2.0** | OASIS SAML 2.0 | Auth Provider type for organizations without OIDC; assertion format for role mapping | Optional |
+| **mTLS** | RFC 8446 + RFC 5280 | All internal component-to-component communication (ICOM-001); provider-to-DCM authentication in ZTS model | Normative |
+| **LDAP v3** | RFC 4511 | FreeIPA, Active Directory, OpenLDAP Auth Provider types; group membership queries for RBAC | Normative |
+| **SCIM 2.0** | RFC 7643 + RFC 7644 | Optional enterprise provisioning; actor creation, update, deprovision; deprovision triggers parallel session + credential revocation | Optional |
+
+---
+
+## 3. Cryptographic Standards
+
+| Standard | Use in DCM | Profiles | Obligation |
+|----------|-----------|---------|-----------|
+| **ECDSA P-384** | Internal CA certificates; component mTLS certs; preferred curve for all DCM-issued certificates | All profiles | Normative for Internal CA |
+| **ECDSA P-256** | Permitted for performance-constrained contexts where P-384 is not available | minimal, dev | Optional |
+| **RSA โ‰ฅ 2048** | Permitted for compatibility with legacy systems; RSA < 2048 prohibited | All profiles | Conditional |
+| **AES-256-GCM** | Credential encryption at rest; audit record encryption (sovereign); data classification-driven | standard+ | Normative |
+| **AES-128-GCM** | Permitted for minimal/dev profiles where performance matters | minimal, dev | Conditional |
+| **SHA-256** | Hash function for audit chain integrity; entity handle generation; minimum acceptable | All profiles | Normative |
+| **SHA-384 / SHA-512** | Preferred hash function for fsi/sovereign profiles | fsi, sovereign | Normative for fsi+ |
+| **FIPS 140-2 Level 1** | Minimum cryptographic module requirement for standard/prod | standard, prod | Normative |
+| **FIPS 140-2 Level 2** | Cryptographic module requirement for regulated environments | fsi, fedramp_moderate | Normative |
+| **FIPS 140-3 Level 3** | Cryptographic module requirement for sovereign deployments | sovereign, dod_il4 | Normative |
+| **TLS 1.3** | Preferred; mandatory cipher suites; forward secrecy required | All profiles | Normative (preferred) |
+| **TLS 1.2** | Minimum acceptable; TLS 1.0/1.1 strictly prohibited | All profiles | Normative (minimum) |
+
+### 3.1 Forbidden Algorithms
+
+DCM prohibits the following algorithms in all profiles:
+
+| Algorithm | Reason |
+|-----------|--------|
+| MD5 | Cryptographically broken |
+| SHA-1 | Deprecated; collision attacks demonstrated |
+| DES | 56-bit key; insecure |
+| 3DES / Triple-DES | Deprecated; Sweet32 attack |
+| RC4 | Cryptographically broken |
+| RSA < 2048 | Insufficient key length |
+| ECDSA curves weaker than P-256 | Insufficient security level |
+
+---
+
+## 4. Operational Standards and Protocols
+
+| Standard | Specification | Use in DCM | Obligation |
+|----------|--------------|-----------|-----------|
+| **Prometheus / OpenMetrics** | Prometheus exposition format; OpenMetrics spec | `GET /metrics` scrape endpoint; all DCM metric families (HLT-005); provider health metrics | Normative |
+| **OpenTelemetry (OTel)** | CNCF OpenTelemetry specification | Distributed tracing for request pipeline; X-DCM-Correlation-ID propagation; span context for audit provenance | Informative |
+| **Kubernetes API** | kubernetes.io API conventions | Resource type spec format mirrors k8s YAML; CRD-based DCM Operator integration; probe endpoints (/livez, /readyz) | Normative (k8s deployments) |
+| **GitOps / OpenGitOps** | OpenGitOps principles (v1.0) | All DCM data model artifacts stored in Git; PR-based contribution model; Git as source of truth for policy and layer definitions | Normative |
+| **Unix cron** | POSIX cron expression format | Recurring schedule expressions in scheduled requests (doc 37) and maintenance window definitions | Normative |
+| **IANA health+json** | RFC 8615 + IANA media type registry | Health response format for `/livez`, `/readyz`, `/api/v1/admin/health`, and OIS health endpoint | Normative |
+| **W3C Server-Sent Events (SSE)** | W3C Living Standard | `GET /api/v1/requests/{uuid}/stream` live request status stream; events: status_change, progress_updated, approval_required, approval_recorded, heartbeat; stream closes on terminal status; alternative to polling for browser/CLI consumers | Normative |
+| **OpenAPI 3.1** | OpenAPI Initiative 3.1 | REST API specification format for Consumer API, Admin API, and Operator Interface Specification; schema definitions for request/response bodies | Normative |
+| **SPIFFE** | CNCF SPIFFE Specification v1.0 | Workload identity framework that inspired DCM's internal component identity model (ICOM); each DCM component has a stable UUID and certificate analogous to a SPIFFE ID; Istio/Envoy enforce SPIFFE-compatible workload identity | Informative |
+| **Istio / Service Mesh** | Istio service mesh specification | Internal component mTLS enforcement; traffic policies; circuit breaking; observability; service-to-service authorization | Normative (distributed deployments) |
+
+| **HashiCorp Vault PKI** | HashiCorp Vault PKI Secrets Engine | External CA Credential Provider backend (optional); issues x509 component certificates via native API or EST/ACME; recommended for fsi/sovereign profiles where enterprise PKI chain is required; typically operates as a subordinate CA of the organization root | Optional |
+| **Venafi TLS Protect** | Venafi Platform | External CA Credential Provider backend (optional); enterprise certificate lifecycle management; ACME/EST/REST API integration | Optional |
+| **EJBCA** | Enterprise JavaBeans Certificate Authority | External CA Credential Provider backend (optional); ACME/CMP/SCEP integration | Optional |
+
+---
+
+## 5. Compliance Frameworks
+
+These frameworks drive specific DCM profiles, overlays, and policy constraints. DCM does not certify compliance โ€” it provides the architectural primitives that enable compliant implementations.
+
+### 5.1 US Federal and Defense
+
+| Framework | Full Name | DCM Profile/Overlay | Key DCM Requirements |
+|-----------|-----------|-------------------|---------------------|
+| **NIST SP 800-53** | Security and Privacy Controls for Information Systems | `fedramp_moderate`, `fedramp_high` | Policy control families mapped to DCM policy domains; access control, audit, configuration management |
+| **NIST SP 800-63B** | Digital Identity Guidelines | All profiles (AAL mapping) | AAL1 (minimal/dev), AAL2 (standard/prod), AAL2+ (fsi), AAL3 (sovereign); MFA requirements per level |
+| **FedRAMP Moderate** | Federal Risk and Authorization Management Program โ€” Moderate | `fedramp_moderate` overlay | NIST 800-53 Moderate baseline; FIPS 140-2 Level 1+; Federal data handling requirements |
+| **FedRAMP High** | Federal Risk and Authorization Management Program โ€” High | `fedramp_high` overlay | NIST 800-53 High baseline; FIPS 140-2 Level 2+; enhanced audit retention |
+| **DoD IL4** | Department of Defense Impact Level 4 | `dod_il4` overlay | Controlled Unclassified Information; FIPS 140-2 Level 2; hardware attestation; enhanced logging |
+| **FIPS 140-2/140-3** | Federal Information Processing Standard โ€” Cryptographic Modules | fsi+ profiles | Cryptographic module validation; forbidden algorithm enforcement; key management requirements |
+
+### 5.2 Industry Compliance
+
+| Framework | Full Name | DCM Profile/Overlay | Key DCM Requirements |
+|-----------|-----------|-------------------|---------------------|
+| **PCI DSS** | Payment Card Industry Data Security Standard | `pci_dss` overlay | Req 8.3.9: P90D maximum credential rotation; network segmentation via sovereignty constraints; cardholder data access logging; 12-month audit retention |
+| **HIPAA** | Health Insurance Portability and Accountability Act | `fsi` profile; `hipaa` overlay | PHI access logging; minimum necessary access (RBAC); audit controls; transmission security (TLS 1.2+); workforce authentication (MFA) |
+| **SOC 2** | Service Organization Control 2 | `standard`+ profiles | Type II audit trail requirements; availability, security, confidentiality trust service criteria; change management via GitOps |
+| **ISO 27001** | Information Security Management Systems | All profiles | Risk-based approach; asset management; access control; cryptography; operations security; incident management |
+
+### 5.3 Data Protection / Sovereignty
+
+| Framework | Full Name | DCM Feature | Key DCM Requirements |
+|-----------|-----------|------------|---------------------|
+| **GDPR** | General Data Protection Regulation (EU) | Sovereignty constraints; data classification | Data residency enforcement; right to erasure model (entity decommission + audit retention policy); data minimization via field-level classification; consent/purpose tracking via Governance Matrix |
+| **Schrems II** | CJEU ruling on EU-US data transfers | Sovereignty constraints; federation boundaries | Data transfer restrictions between DCM federation peers; sovereign profile enforcement |
+
+---
+
+## 6. CNCF Ecosystem
+
+DCM is designed for CNCF ecosystem compatibility. The following CNCF projects are referenced:
+
+| Project | CNCF Status | DCM Use |
+|---------|------------|---------|
+| **Kubernetes** | Graduated | Deployment target; CRD-based DCM Operator; resource model inspiration |
+| **Open Policy Agent (OPA)** | Graduated | Policy engine backend option; Rego policies for DCM GateKeeper and Validation policy types |
+| **Prometheus** | Graduated | Metrics exposition format; DCM scrape endpoint |
+| **OpenTelemetry** | Graduated | Distributed tracing; correlation ID propagation |
+| **Istio** | Graduated | Service mesh for internal mTLS; traffic policies |
+| **Argo CD / Flux** | Graduated | GitOps delivery for DCM layer definitions and policy artifacts |
+
+---
+
+## 7. Authentication Assurance Levels (NIST SP 800-63B)
+
+DCM maps profile security postures to NIST Authentication Assurance Levels:
+
+| Profile | AAL | Requirements |
+|---------|-----|-------------|
+| `minimal` | AAL1 | Single-factor authentication acceptable; password or API key |
+| `dev` | AAL1 | Single-factor authentication acceptable |
+| `standard` | AAL2 | MFA required for all actor sessions; phishing-resistant preferred |
+| `prod` | AAL2 | MFA required; TOTP, FIDO2, or hardware token |
+| `fsi` | AAL2+ | MFA required; phishing-resistant authenticator (FIDO2/hardware token) |
+| `sovereign` | AAL3 | Hardware-based authenticator required; verifier impersonation resistance; physical authenticator possession |
+
+---
+
+## 8. Standard Usage Map โ€” Where Each Standard Appears
+
+| Standard | Documents |
+|----------|----------|
+| RFC 7519 (JWT) | 19-auth-providers, 35-session-revocation, consumer-api-spec |
+| RFC 7517 (JWK) | 19-auth-providers |
+| RFC 7662 (Token Introspection) | 35-session-revocation, consumer-api-spec |
+| RFC 7643/7644 (SCIM 2.0) | 19-auth-providers |
+| RFC 8446 (TLS 1.3) | 14-policy-profiles, 26-accreditation, 31-credential-provider, 36-internal-component-auth |
+| RFC 5280 (X.509/CRL) | 31-credential-provider, 36-internal-component-auth |
+| RFC 6960 (OCSP) | 36-internal-component-auth |
+| RFC 7030 (EST) | 31-credential-provider |
+| RFC 8555 (ACME) | 31-credential-provider |
+| RFC 8894 (SCEP) | 31-credential-provider |
+| RFC 4210 (CMP) | 31-credential-provider |
+| RFC 8594 (Sunset) | 34-api-versioning-strategy, consumer-api-spec |
+| RFC 9745 (Deprecation) | 34-api-versioning-strategy, consumer-api-spec |
+| RFC 8615 (Well-Known URIs) | 34-api-versioning-strategy, 39-dcm-self-health, dcm-operator-interface-spec |
+| RFC 8259 (JSON) | All specifications |
+| ISO 8601 (timestamps) | All documents |
+| OIDC / OpenID Connect | 19-auth-providers, consumer-api-spec |
+| LDAP (RFC 4511) | 19-auth-providers |
+| FIPS 140 | 14-policy-profiles, 31-credential-provider, 36-internal-component-auth |
+| NIST SP 800-63B (AAL) | 31-credential-provider |
+| NIST SP 800-53 | 14-policy-profiles |
+| HIPAA | 14-policy-profiles, 31-credential-provider |
+| PCI DSS | 14-policy-profiles, 31-credential-provider |
+| FedRAMP | 14-policy-profiles |
+| GDPR | 08-resource-grouping, 22-dcm-federation |
+| ISO 27001 | 14-policy-profiles, 26-accreditation |
+| Kubernetes | 11-kubernetes-compatibility, dcm-operator-sdk-api |
+| OPA | dcm-opa-integration-spec |
+| Prometheus | 12-audit-provenance, 39-dcm-self-health |
+| OpenTelemetry | 12-audit-provenance |
+| Istio | 17-deployment-redundancy, 36-internal-component-auth |
+| GitOps/OpenGitOps | 00-context-and-purpose, 20-registry-governance, 28-federated-contribution |
+| W3C SSE (Server-Sent Events) | consumer-api-spec, 39-dcm-self-health |
+| OpenAPI 3.1 | dcm-operator-interface-spec, consumer-api-spec, dcm-admin-api-spec |
+| SPIFFE (conceptual) | 36-internal-component-auth |
+| HashiCorp Vault PKI | 31-credential-provider-model, 36-internal-component-auth |
+| RFC 7009 (Token Revocation) | 35-session-revocation |
+| NIST SP 800-63B (AAL) | 31-credential-provider-model, 14-policy-profiles |
+| SCH policies (scheduling) | 37-scheduled-requests |
+| RDG policies (dependency graph) | 38-request-dependency-graph |
+| HLT policies (self-health) | 39-dcm-self-health |
+| SES policies (session revocation) | 35-session-revocation |
+| ICOM policies (internal component auth) | 36-internal-component-auth |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+---
+
+## 10. ITSM Integration Standards
+
+These standards and protocols are used by ITSM Provider implementations:
+
+| Standard / Protocol | Use in DCM ITSM Integration | Obligation |
+|--------------------|-----------------------------|------------|
+| **ServiceNow REST Table API** | Primary integration for ServiceNow; create/read/update change_request, incident, cmdb_ci tables | Normative for ServiceNow provider |
+| **Jira REST API v3** | Primary integration for Jira Service Management; issue create/update/transition | Normative for Jira provider |
+| **BMC AR REST API v1** | Integration for BMC Remedy/Helix ITSM; form-based create/update | Normative for BMC provider |
+| **PagerDuty Events API v2** | Incident creation and update for alert-type integrations | Normative for PagerDuty provider |
+| **HMAC-SHA256** | Inbound webhook signature verification for all ITSM systems; shared secret via Credential Provider | Normative |
+| **ITIL v4 Change Management** | Conceptual framework for DCM change record lifecycle mapping (Normal, Standard, Emergency change types) | Informative |
+| **JSON:API** | Used by several ITSM REST APIs for response formatting | Informative |
+| **JSONPath** | Template expression resolution for `generic_rest` action payloads; response field extraction | Normative for generic_rest |
+
+## 9. Policy Family to Standard Mapping
+
+Each DCM system policy family maps to one or more industry standards. This table supports compliance traceability.
+
+| Policy Family | Standards Basis | Key Policies |
+|--------------|----------------|-------------|
+| **AUTH-001โ€“015** | RFC 6749, RFC 7519, OIDC Core, NIST SP 800-63B, RFC 7643/7644 | Auth Provider lifecycle; session TTL; MFA enforcement; SCIM provisioning |
+| **AUTH-016โ€“022** | RFC 7662, RFC 6749 spirit, OAuth 2.0 best practices | Session revocation; token introspection; refresh token invalidation |
+| **CPX-001โ€“012** | FIPS 140-2/3, RFC 5280, RFC 8555/7030/8894/4210, NIST SP 800-57 | Credential never stored; rotation; revocation; algorithm baseline |
+| **ATM-001โ€“012** | ISO 27001 change management; organizational governance practices | Authority tier ordering; security degradation gate; profile gap detection |
+| **EVT-001โ€“007** | OpenTelemetry, CNCF event-driven best practices | Event envelope; idempotency; non-suppressable audit events |
+| **VER-001โ€“009** | RFC 8594, RFC 9745, industry API lifecycle practices | Breaking change definition; deprecation headers; migration guides |
+| **SES-001โ€“005** | RFC 7662, RFC 7009, OAuth 2.0 security best practices | Session lifecycle; concurrent limits; emergency revocation |
+| **ICOM-001โ€“009** | RFC 8446, RFC 5280, SPIFFE conceptual model, FIPS 140 | mTLS; component identity; Internal CA; bootstrap; certificate revocation |
+| **SCH-001โ€“006** | Industry job scheduling practices; dual-evaluation pattern | Scheduled request dual policy evaluation; deadline enforcement |
+| **RDG-001โ€“006** | DAG-based workflow ordering; dependency injection patterns | Circular dependency rejection; quota at group submission; field injection |
+| **HLT-001โ€“006** | RFC 8615, Kubernetes probe conventions, Prometheus OpenMetrics | Liveness/readiness; unauthenticated probes; profile-governed metrics exposure |
+| **DPO-001โ€“006** | Design-by-contract; security-first architecture principles | Design priority order; security as Priority 1 |
+| **ZTS-001โ€“005** | Zero Trust Architecture (NIST SP 800-207); NIST SP 800-63B | Five-check boundary model; mTLS; scoped interaction credentials |
+| **MPX-001โ€“008** | Service mesh composition patterns; dependency graph execution | Meta Provider constituent orchestration; compensation |
+| **SMX-001โ€“010** | Risk scoring; NIST RMF; organizational risk tolerance | Hybrid scoring; approval routing; enforcement class |
+| **FCM-001โ€“008** | GitOps contribution model; CNCF governance practices | Federated policy contribution; shadow validation; trust levels |
+| **GMX-001โ€“006** | Governance Matrix; policy-as-code; organizational controls | Cross-domain policy enforcement; data classification |
+| **ITSM-001โ€“007 + ITSM-POL-001โ€“004** | ITIL v4, ServiceNow/Jira/Remedy REST APIs, HMAC-SHA256, ITIL change management | ITSM Provider registration; inbound webhook auth; ITSM Policy evaluation; blocking gate with timeout guarantee |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
\ No newline at end of file
diff --git a/content/docs/data-model/41-operational-reference.md b/content/docs/data-model/41-operational-reference.md
new file mode 100644
index 0000000..a666a09
--- /dev/null
+++ b/content/docs/data-model/41-operational-reference.md
@@ -0,0 +1,521 @@
+# DCM Data Model โ€” Operational Reference
+
+**Document Status:** ๐Ÿ”„ In Progress
+**Document Type:** SRE Reference โ€” GitOps Scale, Store Migration, Disaster Recovery
+**Related Documents:** [Storage Providers](11-storage-providers.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Four States Model](02-four-states.md) | [Internal Component Authentication](36-internal-component-auth.md) | [DCM Self-Health Endpoints](39-dcm-self-health.md)
+
+> **Audience:** Platform engineers and SREs operating DCM in production. This document covers three operational concerns that require guidance beyond the architectural specifications: GitOps store partitioning at large scale, migrating between store implementations, and recovering from failure scenarios.
+
+---
+
+## 1. GitOps Store Scale and Partitioning
+
+### 1.1 When a Repo Becomes Too Large
+
+The default DCM GitOps layout uses one repository per store type (Intent, Requested, Layer, Policy). For most deployments this is correct. At large scale โ€” tens of thousands of active entities or hundreds of active tenants โ€” a single repository can exhibit:
+
+- Git operation latency (clone, fetch, log) growing beyond SLA
+- CI/CD pipeline fan-out delays as every write triggers the full repository
+- Access control granularity limits (all tenants share one repo)
+- Search index sync lag from large diffs
+
+**Thresholds that suggest partitioning:**
+
+| Signal | Threshold | Recommended action |
+|--------|-----------|-------------------|
+| Entities in Intent/Requested store | > 50,000 active | Consider tenant-shard partitioning |
+| Git clone time | > PT30S | Add shallow-clone depth; consider partitioning |
+| PR merge latency | > PT5M | Partition or add write buffer |
+| Tenant count | > 500 | Consider per-tenant repositories |
+| Repository size on disk | > 10 GB | Partition |
+
+These are guidelines, not hard limits. Profile, hardware, and Git host performance all affect the actual inflection point.
+
+### 1.2 Partitioning Strategies
+
+DCM supports three partitioning strategies. All are compatible with the storage contract โ€” partitioning changes how stores are organized, not what the store contract requires.
+
+#### Strategy A โ€” Tenant Shard Partitioning (recommended for most)
+
+Split each store type into N shard repositories, with tenants assigned to shards by a deterministic hash of `tenant_uuid`:
+
+```
+shard = hash(tenant_uuid) % N
+
+dcm-intent-shard-0/     โ† tenants whose hash(uuid) % N == 0
+dcm-intent-shard-1/     โ† tenants whose hash(uuid) % N == 1
+  tenants/
+    {tenant-uuid}/
+      requests/
+        {request-uuid}/
+          intent.yaml
+```
+
+**DCM configuration:**
+```yaml
+gitops_store:
+  intent_store:
+    partitioning: tenant_shard
+    shard_count: 8
+    shard_routing: hash_mod    # deterministic; no routing table needed
+    repositories:
+      - shard: 0
+        url: https://git.corp/dcm/dcm-intent-shard-0
+      - shard: 1
+        url: https://git.corp/dcm/dcm-intent-shard-1
+      # ...
+```
+
+**Operational implications:** Adding shards requires re-hashing. Plan shard counts for 3โ€“5 years of expected growth; use a power of 2 to simplify future doubling.
+
+#### Strategy B โ€” Per-Tenant Repositories (for strict isolation)
+
+Each tenant has its own set of store repositories. Used when:
+- Tenants are separate organizations (MSP model)
+- Compliance requires complete data isolation per tenant
+- Different retention policies per tenant
+
+```
+dcm-intent-{tenant-uuid}/    โ† one repository per tenant
+  requests/
+    {request-uuid}/
+      intent.yaml
+```
+
+**Operational implications:** Repository count scales with tenant count. Requires automation for tenant onboarding (repository creation, access provisioning). Git host must support large numbers of repositories.
+
+#### Strategy C โ€” Time-Based Archiving (for retention management)
+
+Active entities stay in the primary repository. Entities past a declared age threshold are archived to read-only archive repositories:
+
+```
+dcm-intent-active/           โ† current entities (hot)
+dcm-intent-archive-2025/     โ† entities from 2025 (cold, read-only)
+dcm-intent-archive-2024/     โ† entities from 2024 (cold, read-only)
+```
+
+DCM's audit and search components are configured with both active and archive repository lists. Strategy C is typically combined with Strategy A or B.
+
+### 1.3 Large-Scale Layer Store Partitioning
+
+The Layer Store grows more slowly than the Intent/Requested stores (layers are reused across requests). Layer Store partitioning is by domain rather than by tenant:
+
+```
+dcm-layers-compute/        โ† Compute.* resource type layers
+dcm-layers-network/        โ† Network.* resource type layers
+dcm-layers-storage/        โ† Storage.* resource type layers
+dcm-layers-platform/       โ† Platform.* and cross-cutting layers
+dcm-policies-core/         โ† System and core policies
+dcm-policies-tenant/       โ† Tenant-contributed policies (per tenant or sharded)
+```
+
+Domain-partitioned Layer stores are configured in DCM's layer assembly engine:
+
+```yaml
+layer_store:
+  repositories:
+    - domain: Compute.*
+      url: https://git.corp/dcm/dcm-layers-compute
+      priority: provider_contribution  # provider contributions go here
+    - domain: Network.*
+      url: https://git.corp/dcm/dcm-layers-network
+    - domain: "*"                       # catch-all for uncategorized
+      url: https://git.corp/dcm/dcm-layers-platform
+```
+
+### 1.4 Shallow Clones and Read-Only Mirrors
+
+For read-heavy operations (audit, search index rebuild, drift reconciliation) that do not need full Git history:
+
+```yaml
+gitops_store:
+  read_operations:
+    clone_depth: 1           # shallow clone for read-only consumers
+    use_mirror: true         # read from read-only mirror; writes go to primary
+    mirror_url: https://git-mirror.corp/dcm/
+    mirror_sync_lag_max: PT5M  # alert if mirror is more than 5 minutes behind
+```
+
+---
+
+## 2. Store Migration
+
+### 2.1 Migration Principles
+
+DCM store migrations follow three invariants:
+
+1. **No data loss** โ€” every record in the source store must exist in the target store after migration
+2. **Audit chain continuity** โ€” the audit hash chain must be unbroken across the migration; audit records written before and after must chain correctly
+3. **Read availability during migration** โ€” DCM continues serving read requests throughout; write availability may be briefly paused during cutover
+
+### 2.2 Migration Playbook Structure
+
+Every store migration follows this pattern regardless of source or target implementation:
+
+```
+Phase 1 โ€” Prepare
+  โ”‚  Provision target store alongside source
+  โ”‚  Validate target store meets storage contract (health check, write test, read test)
+  โ”‚  Configure DCM to write to BOTH source and target (dual-write mode)
+  โ”‚
+Phase 2 โ€” Backfill
+  โ”‚  Export all existing records from source
+  โ”‚  Import records to target in chronological order (preserving provenance timestamps)
+  โ”‚  Verify record counts match; spot-check content hashes
+  โ”‚
+Phase 3 โ€” Validate
+  โ”‚  Run DCM's store validation suite against target
+  โ”‚  Verify audit chain integrity on target store
+  โ”‚  Verify search index can be rebuilt from target store
+  โ”‚
+Phase 4 โ€” Cutover
+  โ”‚  Brief write pause (PT30Sโ€“PT5M depending on profile)
+  โ”‚  Disable dual-write; switch DCM to target as primary
+  โ”‚  Verify /readyz returns healthy
+  โ”‚  Resume writes to target only
+  โ”‚
+Phase 5 โ€” Decommission source (after burn-in period)
+     Default burn-in: P30D (standard), P90D (fsi/sovereign)
+     Keep source in read-only mode during burn-in for rollback
+```
+
+### 2.3 Common Migration Paths
+
+#### SQLite โ†’ PostgreSQL (minimal/dev โ†’ standard)
+
+Typical trigger: scaling beyond single-node evaluation environment.
+
+```bash
+# Step 1: Export from SQLite
+dcm-admin store export \
+  --store realized \
+  --format jsonl \
+  --output realized-export.jsonl
+
+# Step 2: Import to PostgreSQL
+dcm-admin store import \
+  --store realized \
+  --source realized-export.jsonl \
+  --target postgres://pg-host:5432/dcm_realized \
+  --validate-chain
+
+# Step 3: Enable dual-write
+dcm-admin store dual-write enable \
+  --store realized \
+  --primary sqlite://dcm-realized.db \
+  --secondary postgres://pg-host:5432/dcm_realized
+
+# Step 4: Validate
+dcm-admin store validate \
+  --store realized \
+  --target postgres://pg-host:5432/dcm_realized \
+  --check-count --check-chain --check-spot-sample 0.05
+
+# Step 5: Cutover
+dcm-admin store cutover \
+  --store realized \
+  --target postgres://pg-host:5432/dcm_realized
+```
+
+#### PostgreSQL single-instance โ†’ CockroachDB / PostgreSQL HA
+
+Typical trigger: HA requirement for production; multi-region deployment.
+
+**Key difference from SQLite โ†’ PostgreSQL:** CockroachDB uses serializable isolation and distributed transactions. Test write throughput under realistic load before cutover โ€” CockroachDB's latency profile differs from single-node PostgreSQL.
+
+```yaml
+# Pre-migration checklist
+migration_checklist:
+  - Load test target under realistic DCM write volume (PT4H minimum)
+  - Verify CockroachDB schema compatibility (DCM uses standard PostgreSQL wire protocol)
+  - Configure connection pooler (PgBouncer or similar) โ€” CockroachDB default connection count
+  - Verify time synchronization (CockroachDB requires NTP within PT500MS across nodes)
+  - Test audit chain write under partition scenario
+```
+
+#### GitOps Store โ€” Repo Restructuring
+
+Restructuring a GitOps repository (e.g. monorepo to sharded) requires special handling because Git history must be preserved.
+
+```
+Step 1: Enable write buffer โ€” all new writes queue while migration proceeds
+Step 2: git filter-repo or git subtree to extract tenant directories to shard repos
+Step 3: Validate file counts and content hashes in each shard
+Step 4: Update DCM gitops_store configuration to point to shards
+Step 5: Drain write buffer โ€” queued writes replay to new shard repos
+Step 6: Verify search index rebuild from shards
+Step 7: Archive or delete monorepo after burn-in period
+```
+
+### 2.4 Rollback Procedure
+
+If migration fails before cutover: disable dual-write, discard target, no impact to production.
+
+If migration fails after cutover (during burn-in):
+
+```
+1. Alert: /readyz reports degraded or source store discrepancy detected
+2. dcm-admin store rollback --store  --to source
+   (requires source still in read-only mode โ€” NOT decommissioned)
+3. DCM restarts reads/writes from source
+4. Export any writes that reached target but not source (if any, during dual-write gap)
+5. Import gap records to source
+6. Re-enable source as primary
+```
+
+**This is why burn-in period exists.** Do not decommission source stores until burn-in completes.
+
+### 2.5 Profile-Governed Migration Constraints
+
+| Profile | Min dual-write duration | Max cutover pause | Burn-in period |
+|---------|------------------------|-------------------|----------------|
+| `minimal` | P1D | PT5M | P7D |
+| `dev` | P3D | PT5M | P14D |
+| `standard` | P7D | PT2M | P30D |
+| `prod` | P14D | PT1M | P30D |
+| `fsi` | P30D | PT30S | P90D |
+| `sovereign` | P60D | PT30S | P90D |
+
+---
+
+## 3. Disaster Recovery Runbook
+
+### 3.1 DCM Recovery Architecture
+
+DCM's recovery model is built on a key property: **all durable state is in the stores, not in the control plane.** Control plane components (Policy Engine, Request Orchestrator, etc.) are stateless and can be restarted without data loss. Recovery from most failures is component restart, not data restoration.
+
+The five DCM stores and their recovery characteristics:
+
+| Store | Implementation | Data durability | Recovery method |
+|-------|---------------|----------------|-----------------|
+| Intent Store | GitOps (Git) | Git replication + remote | Re-clone from remote |
+| Requested Store | GitOps or write-once | Git replication / DB replication | Re-clone or DB restore |
+| Layer Store | GitOps (Git) | Git replication + remote | Re-clone from remote |
+| Realized Store | Write-once (PostgreSQL/CockroachDB) | DB replication / WAL | DB failover or restore |
+| Audit Store | Append-only (Kafka/PostgreSQL) | Replication / WAL | Kafka failover or restore |
+
+### 3.2 Recovery Scenarios and Procedures
+
+#### Scenario 1: Single Component Failure (Most Common)
+
+**Symptoms:** One DCM component (e.g. Policy Engine) is unhealthy. `/readyz` shows degraded. Requests may be delayed but not lost.
+
+**RTO:** PT5M  
+**RPO:** 0 (no data loss โ€” components are stateless)
+
+```
+1. Identify failing component via GET /api/v1/admin/health
+2. Check component logs for panic/OOM/deadlock
+3. Kubernetes: pod restart is automatic (liveness probe)
+   Manual: kubectl rollout restart deployment/dcm-policy-engine
+4. Monitor /readyz โ€” should recover within PT2M of pod restart
+5. If component repeatedly fails: check Internal CA cert expiry (ICOM-006)
+   dcm-admin component cert-status --component policy-engine
+6. Write post-incident note to DCM audit store
+```
+
+#### Scenario 2: Store Failure (Database / Kafka)
+
+**Symptoms:** `/readyz` fails specific store check. Requests queue or fail depending on which store.
+
+**Realized Store failure (highest severity โ€” blocks realization):**
+
+```
+RTO target: PT30M (standard), PT15M (prod), PT5M (fsi/sovereign)
+RPO: 0 for PostgreSQL HA (synchronous replication); near-zero for async
+
+1. Confirm store failure: GET /api/v1/admin/health โ†’ realized_store: fail
+2. If HA: check if automatic failover occurred
+   kubectl get pods -n dcm-stores | grep postgres
+   Check PostgreSQL replication lag / CockroachDB node status
+3. Manual failover if automatic did not trigger:
+   dcm-admin store failover --store realized --target replica-2
+4. Verify replication caught up: dcm-admin store lag --store realized
+5. Verify /readyz recovers
+6. Root cause analysis: WAL lag, disk full, network partition
+```
+
+**Audit Store failure:**
+
+```
+RTO: PT1H acceptable (audit trail can tolerate temporary buffering)
+RPO: profile-governed โ€” see Audit Store write buffer policy
+
+1. DCM buffers audit records locally (Commit Log) during store outage
+   Write buffer capacity: profile-governed (PT1H standard, PT15M sovereign)
+2. Restore Kafka cluster from replica or snapshot
+3. DCM drains buffer to restored store automatically on reconnection
+4. Verify chain integrity: dcm-admin audit chain-verify --since 
+```
+
+**GitOps Store failure (Intent/Requested/Layer):**
+
+```
+RTO: PT30M (stores are remountable from Git remote)
+RPO: 0 (all writes go to Git remote; loss only if remote is also lost)
+
+1. Git remote unreachable: check network connectivity
+2. If Git host is down: DCM switches to cached/buffered mode
+   New requests queue in write buffer; reads served from local clone
+3. Write buffer capacity: PT4H (standard) โ€” configure per deployment
+4. When Git host recovers: buffer drains automatically
+5. Force drain: dcm-admin store drain-buffer --store intent
+```
+
+#### Scenario 3: Full Control Plane Loss
+
+**Symptoms:** All DCM pods down. Stores intact. Users cannot submit requests.
+
+**RTO:** PT15M (kubernetes deployment restart)  
+**RPO:** 0 (stores are external โ€” no data in pods)
+
+```
+1. Verify stores are healthy (connect directly):
+   dcm-admin store health-check --all --direct
+
+2. Verify Internal CA is available:
+   curl -k https://dcm-internal-ca.dcm-system.svc.cluster.local/health
+
+3. Restart DCM deployment (Kubernetes):
+   kubectl rollout restart deployment -n dcm-system
+
+4. Monitor /readyz โ€” startup sequence should complete within PT3M:
+   watch -n 5 kubectl get pods -n dcm-system
+
+5. Verify session store recovers:
+   GET /api/v1/admin/health โ†’ session_store: pass
+
+6. Alert consumers: any in-flight requests at time of failure
+   are in ACKNOWLEDGED/DISPATCHED state and may need status check
+   dcm-admin requests find --status in-flight --since 
+```
+
+#### Scenario 4: Partial Region Loss (Multi-Region Deployments)
+
+**Symptoms:** One region's DCM instance degraded. Other regions operational.
+
+```
+1. DCM federation routes requests away from degraded region (automatic)
+   Verify: GET /api/v1/admin/health โ†’ federation peer status
+
+2. If region is sovereign-scoped (data must not leave): 
+   Alert: sovereignty.migration_required event fires
+   Consumers in that region may be blocked until region recovers
+
+3. For non-sovereign regions: traffic reroutes automatically
+   Monitor: dcm_requests_total{region} for traffic shift
+
+4. Region recovery: standard Scenario 3 procedure
+   After recovery: drift detection validates recovered state
+```
+
+#### Scenario 5: Complete Loss (Repave)
+
+The nuclear scenario: entire DCM installation destroyed. Git remote intact.
+
+**RTO:** PT4Hโ€“PT24H (depends on infrastructure provisioning speed)  
+**RPO:** 0 for GitOps stores; near-zero for Realized/Audit stores
+
+```
+1. Provision new Kubernetes cluster (or equivalent)
+
+2. Deploy DCM bootstrap installer:
+   helm install dcm-bootstrap dcm/dcm-bootstrap \
+     --set gitops.manifest_url=https://git.corp/dcm/dcm-deployment \
+     --set gitops.manifest_ref=
+
+3. DCM bootstrap reads dcm_deployment manifest from Git
+   Provisions itself: control plane, Internal CA, stores
+
+4. Restore Realized Store from backup:
+   dcm-admin store restore --store realized \
+     --from s3://dcm-backups/realized/latest \
+     --validate-chain
+
+5. Restore Audit Store from backup or Kafka snapshot:
+   dcm-admin store restore --store audit \
+     --from s3://dcm-backups/audit/latest \
+     --chain-verify
+
+6. Intent/Requested/Layer stores: re-clone from Git remote (already current)
+
+7. DCM rehydrates managed resources in dependency order:
+   dcm-admin rehydrate --all-tenants --dry-run  # verify plan first
+   dcm-admin rehydrate --all-tenants
+
+8. Drift detection validates recovered state matches declared state:
+   dcm-admin drift scan --all --post-recovery
+
+9. Re-issue Internal CA certificates for all components:
+   (handled automatically by bootstrap โ€” components acquire new certs)
+
+10. Notify consumers: recovery complete; request status available
+```
+
+### 3.3 Recovery Time Objectives by Profile
+
+| Profile | Scenario 1 (component) | Scenario 2 (store) | Scenario 3 (full CP) | Scenario 5 (repave) |
+|---------|------------------------|--------------------|-----------------------|---------------------|
+| `minimal` | PT15M | PT2H | PT30M | PT24H |
+| `standard` | PT5M | PT30M | PT15M | PT8H |
+| `prod` | PT2M | PT15M | PT10M | PT4H |
+| `fsi` | PT2M | PT5M | PT5M | PT2H |
+| `sovereign` | PT1M | PT5M | PT5M | PT2H |
+
+### 3.4 Recovery Point Objectives
+
+| Store | Standard RPO | fsi/sovereign RPO | Notes |
+|-------|-------------|------------------|-------|
+| Intent Store | 0 | 0 | Git remote is source of truth |
+| Requested Store | 0 | 0 | Write-once; replicated |
+| Layer Store | 0 | 0 | Git remote is source of truth |
+| Realized Store | PT5M | PT1M | Async replication lag |
+| Audit Store | PT15M | PT1M | Kafka replication + write buffer |
+
+### 3.5 Backup Schedule
+
+DCM does not manage backups of infrastructure stores directly โ€” that responsibility belongs to the storage platform. Recommended schedules by store:
+
+| Store | Backup method | Frequency | Retention |
+|-------|--------------|-----------|-----------|
+| Intent / Requested / Layer | Git push to offsite remote | Continuous | Per Git host policy |
+| Realized Store | PostgreSQL PITR + daily snapshot | Continuous WAL + P1D snapshot | P90D (standard), P365D (fsi/sovereign) |
+| Audit Store | Kafka topic snapshot | P4H | P365D (all profiles โ€” regulatory minimum) |
+| Internal CA | Key material backup to HSM/Vault | On change | P7Y (key material outlives certs) |
+
+### 3.6 Post-Recovery Validation Checklist
+
+Run after any Scenario 3+ recovery:
+
+```
+โ–ก /livez returns pass on all control plane pods
+โ–ก /readyz returns pass (all 5 core dependencies green)
+โ–ก GET /api/v1/admin/health shows all components pass
+โ–ก dcm-admin audit chain-verify --full returns no broken links
+โ–ก dcm-admin store validate --all returns no discrepancies
+โ–ก dcm-admin drift scan --all returns no unexpected drift
+โ–ก Internal CA certificates valid for all components (ICOM-006)
+โ–ก At least one Auth Provider healthy (GET /api/v1/admin/health โ†’ auth_providers)
+โ–ก Search index rebuild complete (if search index store was affected)
+โ–ก Session Store empty (expected โ€” all sessions expired during outage; users re-authenticate)
+โ–ก Write post-incident note to audit store with recovery timeline
+โ–ก Notify consumers of recovery completion
+```
+
+---
+
+## 4. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `OPS-001` | GitOps store partitioning strategy must be declared in the DCM deployment manifest. Changes to partitioning strategy require dual-write migration procedure (Section 2). |
+| `OPS-002` | Store migrations must maintain audit chain continuity across cutover. Audit records written to the source store before cutover and to the target store after cutover must form an unbroken chain. |
+| `OPS-003` | Source stores must remain accessible in read-only mode for the profile-governed burn-in period after cutover. Source stores must not be decommissioned until the burn-in period completes and rollback is confirmed unnecessary. |
+| `OPS-004` | Recovery from Scenario 3 (full control plane loss) must complete within the profile-governed RTO. If RTO cannot be met, the incident must be escalated and root cause must address the recovery path. |
+| `OPS-005` | The post-recovery validation checklist (Section 3.6) must be completed and its results written to the audit store before declaring an incident resolved. |
+| `OPS-006` | Audit Store backups must be retained for a minimum of P365D in all profiles, regardless of other data retention policies, to satisfy regulatory audit trail requirements. |
+| `OPS-007` | Git remote repositories serving as GitOps stores must be configured with push access from at least two geographically separated locations to prevent single-point-of-failure data loss. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/42-itsm-integration.md b/content/docs/data-model/42-itsm-integration.md
new file mode 100644
index 0000000..5dfc4f4
--- /dev/null
+++ b/content/docs/data-model/42-itsm-integration.md
@@ -0,0 +1,666 @@
+# DCM Data Model โ€” ITSM Integration
+
+**Document Status:** ๐Ÿ”„ In Progress
+**Document Type:** Architecture Reference โ€” ITSM Provider Type and ITSM Policy Type
+**Related Documents:** [Provider Contract](A-provider-contract.md) | [Policy Contract](B-policy-contract.md) | [Notification Model](23-notification-model.md) | [Event Catalog](33-event-catalog.md) | [Authority Tier Model](32-authority-tier-model.md) | [Consumer API Specification](../specifications/consumer-api-spec.md)
+
+> **Design principle:** DCM is built to *replace* the infrastructure ticket as the primary provisioning mechanism. ITSM integration is additive โ€” it enriches DCM entities with ITSM metadata, enables ITSM-initiated requests, and provides bidirectional lifecycle traceability for organizations that need it for compliance. **DCM never requires an ITSM system to function.**
+>
+> Two new additions to the DCM architecture:
+> 1. **ITSM Provider** โ€” a new Provider type (12th) that speaks ITSM system APIs bidirectionally
+> 2. **ITSM Policy** โ€” a new Policy output type (8th) that triggers ITSM actions as a side-effect of DCM pipeline events
+
+---
+
+## 1. ITSM Provider
+
+### 1.1 What an ITSM Provider Is
+
+An ITSM Provider is a DCM Provider that connects DCM to an external IT Service Management system. It handles:
+
+- **Outbound**: DCM lifecycle events โ†’ ITSM records (create change requests, update CMDB CIs, close incidents, link tickets to entities)
+- **Inbound**: ITSM approvals and decisions โ†’ DCM (change approval recorded via approval vote API, request initiation from ITSM workflow)
+- **Sync**: ITSM record references stored on DCM entities as business data (bidirectional link)
+
+The ITSM Provider is **not** a Service Provider (it doesn't realize resources), **not** a Notification Provider (though it may create notification-like records), and **not** a Policy Provider (though ITSM approval status may inform DCM policies). It is its own type because it has a bidirectional contract, manages external record lifecycle, and requires specific capability declarations around ITSM system connectivity.
+
+### 1.2 Data Flow
+
+```
+DCM lifecycle event fires (e.g. request.dispatched)
+  โ”‚
+  โ–ผ ITSM Policy evaluates (see Section 3)
+  โ”‚   Determines: should an ITSM action fire? Which action?
+  โ”‚
+  โ–ผ ITSM Provider receives action request
+  โ”‚   Translates to target system's API format
+  โ”‚   Calls ITSM system (ServiceNow, Jira, etc.)
+  โ”‚
+  โ–ผ ITSM system creates/updates record
+  โ”‚   Returns record ID (CHG0012345, INC-4821, etc.)
+  โ”‚
+  โ–ผ ITSM Provider stores reference on DCM entity
+  โ”‚   entity.business_data.itsm_references[] updated
+  โ”‚
+  โ–ผ ITSM Provider reports back to DCM
+      itsm_reference_created event published
+      External record ID in audit record
+
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+ITSM system approves a change record
+  โ”‚
+  โ–ผ ITSM system calls DCM API (via webhook or polling)
+  โ”‚   POST /api/v1/admin/approvals/{uuid}/vote
+  โ”‚   { decision: "approve", recorded_via: "servicenow",
+  โ”‚     external_reference: "CHG0012345" }
+  โ”‚
+  โ–ผ DCM records approval vote
+  โ”‚   approval.decision_recorded event
+  โ”‚
+  โ–ผ Pipeline resumes if quorum/tier satisfied
+```
+
+### 1.3 Capability Declaration
+
+```yaml
+itsm_provider_capabilities:
+  itsm_system: servicenow | jira_service_management | bmc_remedy | bmc_helix |
+               freshservice | zendesk | pagerduty | opsgenie | manageengine |
+               cherwell | topdesk | generic_rest
+  
+  # What this provider can do
+  supported_actions:
+    - create_change_request      # create a change record for DCM provisioning events
+    - update_change_request      # update change record on state transitions
+    - close_change_request       # close change record on realization/failure
+    - create_incident            # create incident for failures, drift, security events
+    - update_incident            # update incident on resolution
+    - close_incident             # close incident on recovery
+    - update_cmdb_ci             # update CMDB configuration item record
+    - create_cmdb_ci             # create new CMDB CI for realized entities
+    - retire_cmdb_ci             # retire CMDB CI on decommission
+    - create_service_request     # create service request record
+    - link_parent_record         # link DCM entity to existing ITSM record
+    - inbound_approval           # accept approval decisions from ITSM system
+    - inbound_request_initiation # allow ITSM workflows to submit DCM requests
+  
+  # System connectivity
+  endpoint_url:             # ITSM system API base URL
+  api_version:           # system-specific API version
+  auth_credential_uuid:    # references Credential Provider
+  
+  # Bidirectional webhook (for inbound)
+  inbound_webhook:
+    enabled: 
+    secret_credential_uuid:    # HMAC secret for webhook verification
+    
+  # Field mappings (system-specific)
+  field_mapping_ref:   # path to field mapping YAML in Layer Store
+  
+  # CMDB CI type mapping
+  cmdb_ci_type_map:
+    - dcm_resource_type: Compute.VirtualMachine
+      itsm_ci_type: cmdb_ci_server        # ServiceNow CI class
+    - dcm_resource_type: Network.VLAN
+      itsm_ci_type: cmdb_ci_network_gear
+    - dcm_resource_type: Storage.Volume
+      itsm_ci_type: cmdb_ci_storage_device
+```
+
+### 1.4 Required API Endpoints (ITSM Provider implements)
+
+```
+POST {provider_base}/actions              # DCM submits action requests
+GET  {provider_base}/actions/{action_id}  # DCM checks action status
+GET  {provider_base}/records/{record_id}  # DCM retrieves record status
+POST {provider_base}/inbound             # ITSM system sends inbound events
+GET  /health                             # standard OIS health check
+```
+
+### 1.5 DCM Entity ITSM References
+
+Realized entities gain an `itsm_references` block in business data:
+
+```yaml
+itsm_references:
+  - system: servicenow
+    provider_uuid: 
+    record_type: change_request
+    record_id: "CHG0012345"
+    record_url: "https://corp.service-now.com/nav_to.do?uri=change_request.do?sys_id=..."
+    created_at: 
+    status: approved        # DCM's view of the record status
+    last_synced_at: 
+    
+  - system: jira_service_management
+    provider_uuid: 
+    record_type: incident
+    record_id: "INC-4821"
+    record_url: "https://corp.atlassian.net/browse/INC-4821"
+    created_at: 
+    status: open
+    last_synced_at: 
+```
+
+---
+
+## 2. Supported ITSM Systems
+
+### 2.1 ServiceNow
+
+**API:** REST Table API (`/api/now/table/`), Business Rule webhooks, Flow Designer
+
+```yaml
+# ServiceNow ITSM Provider registration
+itsm_provider_registration:
+  provider_handle: "servicenow-prod"
+  itsm_system: servicenow
+  endpoint_url: "https://corp.service-now.com"
+  api_version: "v2"
+  auth_credential_uuid:     # api_key or oauth2 credential
+  
+  supported_actions:
+    - create_change_request        # โ†’ change_request table
+    - update_change_request
+    - close_change_request
+    - create_incident              # โ†’ incident table
+    - update_cmdb_ci               # โ†’ cmdb_ci_server (or mapped class)
+    - create_cmdb_ci
+    - retire_cmdb_ci
+    - inbound_approval             # Change Advisory Board approval โ†’ DCM vote
+    
+  # ServiceNow-specific field mapping
+  change_request_template:
+    assignment_group: "Infrastructure Automation"
+    category: "Software"
+    risk: "2"                      # Low
+    impact: "3"                    # Low
+    # DCM fields injected at runtime:
+    short_description: "DCM: Provision {resource_type} '{entity_handle}'"
+    description: "Requested by: {actor_handle}\nTenant: {tenant_handle}\nDCM Request: {request_uuid}"
+    
+  # CAB approval โ†’ DCM vote mapping
+  inbound_approval:
+    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote"
+    trigger_on: "change_request.state โ†’ 'Approved'"
+    decision_field: "state"
+    decision_map:
+      "Approved": "approve"
+      "Rejected": "reject"
+      "Cancelled": "reject"
+    external_reference_field: "number"    # โ†’ CHG0012345
+    
+  cmdb_ci_type_map:
+    - dcm_resource_type: Compute.VirtualMachine
+      itsm_ci_type: cmdb_ci_server
+    - dcm_resource_type: Network.VLAN
+      itsm_ci_type: cmdb_ci_netgear
+    - dcm_resource_type: Storage.Volume
+      itsm_ci_type: cmdb_ci_disk
+    - dcm_resource_type: Kubernetes.Cluster
+      itsm_ci_type: cmdb_ci_kubernetes_cluster
+```
+
+**Inbound: CAB approval flow**
+
+```
+ServiceNow Change Advisory Board approves CHG0012345
+  โ”‚
+  โ–ผ ServiceNow Business Rule fires on state change โ†’ "Approved"
+  โ”‚   Calls DCM webhook: POST /api/v1/admin/approvals/{uuid}/vote
+  โ”‚   Headers: X-ServiceNow-Signature: 
+  โ”‚   Body: { decision: "approve", recorded_via: "servicenow",
+  โ”‚           external_reference: "CHG0012345" }
+  โ”‚
+  โ–ผ DCM verifies HMAC signature against secret_credential_uuid
+  โ”‚   Records approval vote
+  โ”‚   Pipeline resumes if tier satisfied
+```
+
+### 2.2 Jira Service Management (Atlassian)
+
+**API:** REST API v3, Atlassian Connect webhooks, Automation rules
+
+```yaml
+itsm_provider_registration:
+  provider_handle: "jira-service-mgmt-prod"
+  itsm_system: jira_service_management
+  endpoint_url: "https://corp.atlassian.net"
+  api_version: "3"
+  auth_credential_uuid:     # API token or OAuth2
+  
+  supported_actions:
+    - create_change_request        # โ†’ Jira issue (Change type)
+    - update_change_request
+    - close_change_request
+    - create_incident              # โ†’ Jira issue (Incident type)
+    - create_service_request       # โ†’ Jira issue (Service Request type)
+    - inbound_approval             # Jira Change approval โ†’ DCM vote
+    
+  change_request_template:
+    project_key: "OPS"
+    issue_type: "Change"
+    summary: "DCM: {resource_type} '{entity_handle}'"
+    description: |
+      *Requested by:* {actor_handle}
+      *Tenant:* {tenant_handle}
+      *DCM Request UUID:* {request_uuid}
+      *Catalog Item:* {catalog_item_handle}
+    priority: "Medium"
+    labels: ["dcm-automated", "{tenant_handle}"]
+    
+  inbound_approval:
+    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote"
+    trigger_on: "issue.status โ†’ 'Approved'"
+    decision_map:
+      "Approved": "approve"
+      "Declined": "reject"
+    external_reference_field: "key"    # โ†’ OPS-4821
+```
+
+### 2.3 BMC Remedy / Helix ITSM
+
+**API:** REST API (Remedy AR System REST), webhook callbacks
+
+```yaml
+itsm_provider_registration:
+  provider_handle: "bmc-helix-prod"
+  itsm_system: bmc_helix
+  endpoint_url: "https://remedy.corp.example.com/api/arsys/v1"
+  api_version: "v1"
+  
+  supported_actions:
+    - create_change_request        # โ†’ CHG:Infrastructure Change
+    - update_change_request
+    - close_change_request
+    - create_incident              # โ†’ HPD:Help Desk
+    - update_cmdb_ci               # โ†’ AST:Config Item
+    - inbound_approval
+    
+  change_request_template:
+    form: "CHG:Infrastructure Change"
+    Location_Company: "{tenant_handle}"
+    Summary: "DCM: Provision {resource_type} '{entity_handle}'"
+    Categorization_Tier_1: "Infrastructure"
+    Categorization_Tier_2: "Provisioning"
+    Change_Type: "Normal"
+```
+
+### 2.4 Freshservice
+
+```yaml
+itsm_provider_registration:
+  provider_handle: "freshservice-prod"
+  itsm_system: freshservice
+  endpoint_url: "https://corp.freshservice.com/api/v2"
+  
+  supported_actions:
+    - create_change_request
+    - update_change_request
+    - close_change_request
+    - create_incident
+    - create_service_request
+    
+  change_request_template:
+    type: "Normal"
+    risk: "Low"
+    impact: "Low"
+    subject: "DCM: {resource_type} '{entity_handle}'"
+    description: "Tenant: {tenant_handle} | Actor: {actor_handle} | Request: {request_uuid}"
+    group_id: 
+```
+
+### 2.5 PagerDuty (Incident Management)
+
+```yaml
+itsm_provider_registration:
+  provider_handle: "pagerduty-prod"
+  itsm_system: pagerduty
+  endpoint_url: "https://api.pagerduty.com"
+  
+  supported_actions:
+    - create_incident              # for DCM failures, drift, security events
+    - update_incident
+    - close_incident
+    
+  # PagerDuty Events API v2
+  incident_template:
+    service_id: 
+    escalation_policy_id: 
+    payload:
+      summary: "DCM {event_type}: {entity_handle}"
+      severity: "{{ drift_severity | map: criticalโ†’critical, significantโ†’error, moderateโ†’warning, minorโ†’info }}"
+      source: "dcm"
+      custom_details:
+        entity_uuid: "{entity_uuid}"
+        tenant: "{tenant_handle}"
+        dcm_event: "{event_type}"
+```
+
+### 2.6 Generic REST (Custom ITSM)
+
+For ITSM systems not natively supported, the `generic_rest` type allows template-based HTTP calls:
+
+```yaml
+itsm_provider_registration:
+  provider_handle: "custom-itsm-prod"
+  itsm_system: generic_rest
+  endpoint_url: "https://itsm.corp.example.com/api"
+  
+  action_templates:
+    - action: create_change_request
+      method: POST
+      path: "/changes"
+      headers:
+        Content-Type: "application/json"
+        X-API-Key: "{{ credential_value }}"
+      body_template: |
+        {
+          "title": "DCM: {{ resource_type }} '{{ entity_handle }}'",
+          "requested_by": "{{ actor_handle }}",
+          "category": "Infrastructure",
+          "external_id": "{{ request_uuid }}"
+        }
+      response_id_path: "$.id"    # JSONPath to extract record ID from response
+      
+    - action: inbound_approval
+      inbound_field: "status"
+      decision_map:
+        "approved": "approve"
+        "rejected": "reject"
+```
+
+---
+
+## 3. ITSM Policy Type
+
+### 3.1 What an ITSM Policy Is
+
+An **ITSM Policy** is a new DCM Policy output type (8th, alongside GateKeeper, Validation, Transformation, Recovery, Orchestration Flow, Governance Matrix Rule, and Lifecycle Policy).
+
+It fires as a **side-effect policy** โ€” it does not block pipeline execution (it is not a GateKeeper) and does not transform the payload. It fires on a DCM event and triggers an ITSM action via a registered ITSM Provider. The pipeline continues whether or not the ITSM action succeeds; ITSM failures are logged and alerted but do not block DCM operations.
+
+**Key distinction:** An ITSM Policy is about *record-keeping and integration* with external governance systems. A GateKeeper Policy is about *allowing or blocking* operations. These are complementary, not competing.
+
+### 3.2 Output Schema
+
+```yaml
+# ITSM Policy output schema
+itsm_policy_output:
+  type: itsm_action            # new output type identifier
+  
+  # Required
+  itsm_provider_uuid:    # which ITSM Provider to call
+  action: create_change_request | update_change_request | close_change_request |
+          create_incident | update_incident | close_incident |
+          update_cmdb_ci | create_cmdb_ci | retire_cmdb_ci |
+          create_service_request | link_parent_record
+          
+  # Payload โ€” fields to pass to ITSM Provider
+  # Supports template variables from the triggering event payload
+  action_payload:
+    : 
+    
+  # How to handle ITSM failure
+  on_failure: log_and_continue | alert_and_continue | alert_only
+  
+  # Store the ITSM record reference on the DCM entity (optional)
+  store_reference_on_entity: 
+  reference_label:     # human-readable label for the reference
+  
+  # Require ITSM record creation before dispatch (optional โ€” see note)
+  block_until_created:   # default: false
+  block_timeout:   # max wait if block_until_created: true
+```
+
+> **`block_until_created`:** When `true`, the ITSM Policy behaves like a pre-dispatch gate โ€” DCM waits for the ITSM record to be created before dispatching to the Service Provider. This is used when organizational policy requires a change record to exist before any provisioning begins. When `false` (default), the ITSM record is created in parallel with or after dispatch โ€” suitable for notification-only use cases.
+
+### 3.3 Example Policies
+
+#### Policy 1: Create Change Request on Dispatch (ServiceNow)
+
+```yaml
+policy_handle: "create-change-on-dispatch"
+policy_type: itsm_action
+enforcement_level: soft
+status: active
+
+match:
+  payload_type: request.dispatched
+  conditions:
+    - field: resource_type
+      operator: in
+      value: [Compute.VirtualMachine, Storage.Volume, Network.VLAN]
+
+output:
+  type: itsm_action
+  itsm_provider_uuid: 
+  action: create_change_request
+  action_payload:
+    short_description: "DCM: Provision {{ resource_type }} '{{ entity_handle }}'"
+    description: |
+      Automated provisioning via DCM.
+      Request UUID: {{ request_uuid }}
+      Actor: {{ actor_handle }}
+      Tenant: {{ tenant_handle }}
+      Catalog Item: {{ catalog_item_handle }}
+    risk: "{{ risk_score | map: <25โ†’'Low', <60โ†’'Medium', elseโ†’'High' }}"
+  store_reference_on_entity: true
+  reference_label: "Change Request"
+  on_failure: alert_and_continue
+```
+
+#### Policy 2: Block Dispatch Until Change Record Exists (Compliance Gate)
+
+```yaml
+policy_handle: "require-change-record-before-dispatch"
+policy_type: itsm_action
+enforcement_level: hard
+status: active
+
+match:
+  payload_type: request.layers_assembled
+  conditions:
+    - field: tenant_handle
+      operator: in
+      value: [payments-team, pci-scope-team]
+
+output:
+  type: itsm_action
+  itsm_provider_uuid: 
+  action: create_change_request
+  action_payload:
+    short_description: "DCM: {{ resource_type }} provision โ€” {{ tenant_handle }}"
+    change_type: "Normal"
+    assignment_group: "Change Advisory Board"
+  store_reference_on_entity: true
+  reference_label: "Change Request (PCI Scope)"
+  block_until_created: true
+  block_timeout: PT30M
+  on_failure: alert_and_continue
+```
+
+#### Policy 3: Update CMDB on Realization
+
+```yaml
+policy_handle: "sync-cmdb-on-realization"
+policy_type: itsm_action
+status: active
+
+match:
+  payload_type: entity.realized
+  conditions:
+    - field: resource_type
+      operator: in
+      value: [Compute.VirtualMachine, Compute.BareMetalServer]
+
+output:
+  type: itsm_action
+  itsm_provider_uuid: 
+  action: create_cmdb_ci
+  action_payload:
+    name: "{{ entity_handle }}"
+    ip_address: "{{ realized_fields.primary_ip }}"
+    os: "{{ realized_fields.os_family }}"
+    managed_by: "DCM"
+    environment: "{{ tenant_handle }}"
+    correlation_id: "{{ entity_uuid }}"
+  store_reference_on_entity: true
+  reference_label: "CMDB CI"
+  on_failure: alert_and_continue
+```
+
+#### Policy 4: Create Incident on Drift (Jira)
+
+```yaml
+policy_handle: "create-incident-on-critical-drift"
+policy_type: itsm_action
+status: active
+
+match:
+  payload_type: drift.detected
+  conditions:
+    - field: drift_severity
+      operator: in
+      value: [significant, critical]
+
+output:
+  type: itsm_action
+  itsm_provider_uuid: 
+  action: create_incident
+  action_payload:
+    summary: "DCM Drift: {{ entity_handle }} โ€” {{ drift_severity }}"
+    description: |
+      DCM has detected significant configuration drift.
+      Entity: {{ entity_handle }} ({{ entity_uuid }})
+      Severity: {{ drift_severity }}
+      Drifted fields: {{ drifted_fields | count }} fields
+      Detected at: {{ discovered_at }}
+      View in DCM: https://dcm.corp/resources/{{ entity_uuid }}/drift
+    priority: "{{ drift_severity | map: criticalโ†’'Highest', significantโ†’'High' }}"
+    labels: ["dcm-drift", "{{ resource_type | slugify }}"]
+  store_reference_on_entity: true
+  reference_label: "Drift Incident"
+  on_failure: log_and_continue
+```
+
+#### Policy 5: Retire CMDB CI on Decommission
+
+```yaml
+policy_handle: "retire-cmdb-on-decommission"
+policy_type: itsm_action
+status: active
+
+match:
+  payload_type: entity.decommissioned
+
+output:
+  type: itsm_action
+  itsm_provider_uuid: 
+  action: retire_cmdb_ci
+  action_payload:
+    correlation_id: "{{ entity_uuid }}"    # find CI by DCM entity UUID
+    install_status: "7"                    # ServiceNow: Retired
+    retired_at: "{{ event_timestamp }}"
+    decommission_reason: "DCM decommission โ€” {{ actor_handle }}"
+  on_failure: alert_and_continue
+```
+
+#### Policy 6: Close Change Record on Completion
+
+```yaml
+policy_handle: "close-change-on-completion"
+policy_type: itsm_action
+status: active
+
+match:
+  payload_type: request.realized
+  conditions:
+    - field: entity.itsm_references[?(@.record_type=='change_request')].record_id
+      operator: exists
+
+output:
+  type: itsm_action
+  itsm_provider_uuid: 
+  action: close_change_request
+  action_payload:
+    state: "3"                              # ServiceNow: Closed
+    close_code: "Successful"
+    close_notes: "Provisioning completed successfully by DCM. Entity: {{ entity_uuid }}"
+  on_failure: log_and_continue
+```
+
+---
+
+## 4. ITSM Provider System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ITSM-001` | ITSM Providers implement the base Provider contract (PRV-001) including registration, health check, sovereignty declaration, and zero trust authentication. ITSM system connectivity credentials must reference a registered Credential Provider โ€” no plaintext credentials in provider registration. |
+| `ITSM-002` | DCM does not require ITSM integration to function. ITSM Policies with `on_failure: alert_and_continue` (the default) never block DCM pipeline execution. Organizations must explicitly set `block_until_created: true` to gate pipeline on ITSM record creation. |
+| `ITSM-003` | Inbound events from ITSM systems must be authenticated. ITSM Providers must verify HMAC signatures or OAuth tokens on all inbound webhooks before forwarding to DCM. Unauthenticated inbound events are rejected and logged. |
+| `ITSM-004` | ITSM record references stored on DCM entities follow entity lifecycle โ€” they are included in the Realized State record, preserved through updates, and retained in the decommissioned entity record for audit purposes. |
+| `ITSM-005` | ITSM Policies that use `block_until_created: true` must declare a `block_timeout`. If the ITSM system does not confirm record creation within the timeout, the policy fires `on_failure` behavior and the block is released โ€” the pipeline continues. A blocked pipeline is never permanently stalled by ITSM unavailability. |
+| `ITSM-006` | Field mappings between DCM entities and ITSM CI types must be declared in the ITSM Provider capability registration. Unmapped resource types are silently skipped for CMDB sync actions. |
+| `ITSM-007` | Template expressions in ITSM Policy `action_payload` fields must resolve using values from the triggering event payload. Template expressions that reference unavailable fields produce a warning in the audit record and substitute an empty string. They do not block ITSM action execution. |
+
+---
+
+## 5. ITSM Policy System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ITSM-POL-001` | ITSM Policies follow the full Policy base contract (B-policy-contract.md): lifecycle (developing โ†’ proposed โ†’ active), shadow mode validation, audit obligation on every evaluation, domain precedence. |
+| `ITSM-POL-002` | ITSM Policies are side-effect policies โ€” they do not produce pipeline decisions (allow/deny/transform). They may not be used as GateKeeper substitutes except through the explicit `block_until_created: true` mechanism, which has its own timeout guarantee (ITSM-005). |
+| `ITSM-POL-003` | ITSM Policy evaluation is recorded in the audit trail. The audit record includes: policy handle, matched event, ITSM provider UUID, action requested, ITSM record ID returned, and outcome (success/failure/timeout). |
+| `ITSM-POL-004` | Multiple ITSM Policies may fire on the same event. All fire independently โ€” one policy's failure does not prevent other ITSM Policies from executing. |
+
+---
+
+## 6. Additions to the Foundations Document
+
+The foundations document provider type table gains a 12th row:
+
+| Provider Type | Capability | Data direction |
+|--------------|-----------|----------------|
+| **ITSM Provider** | Bidirectional integration with ITSM systems; creates/updates ITSM records from DCM events; routes ITSM approvals back to DCM | DCM โ†’ ITSM (outbound) / ITSM โ†’ DCM (inbound) |
+
+The foundations document policy type table gains an 8th entry:
+
+| Policy Type | Output | Pipeline role |
+|------------|--------|---------------|
+| **ITSM Action** | Triggers action in connected ITSM system; optionally stores record reference on entity; optionally gates pipeline on record creation | Side-effect (non-blocking by default) |
+
+---
+
+## 7. Event Catalog Additions
+
+Two new events for the Event Catalog (doc 33):
+
+| Event Type | Urgency | Trigger |
+|-----------|---------|---------|
+| `itsm.record_created` | info | ITSM Provider successfully created a record in external system |
+| `itsm.record_failed` | medium | ITSM Provider failed to create/update record; `block_until_created` timeout reached |
+
+These extend the existing event catalog with a new `itsm.*` domain prefix.
+
+---
+
+## 8. Standards Catalog Addition
+
+ITSM integration standards and protocols used:
+
+| Standard | Use in DCM ITSM |
+|----------|----------------|
+| ServiceNow REST Table API | Primary integration for ServiceNow create/update/query |
+| Jira REST API v3 | Primary integration for Atlassian Jira Service Management |
+| BMC AR REST API v1 | Primary integration for BMC Remedy/Helix |
+| PagerDuty Events API v2 | Incident creation for alert-type ITSM integrations |
+| ITIL v4 Change Management | Conceptual framework for DCM change record lifecycle mapping |
+| JSON:API | Standard used by several ITSM REST APIs |
+| HMAC-SHA256 | Inbound webhook signature verification for all ITSM systems |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/A-provider-contract.md b/content/docs/data-model/A-provider-contract.md
new file mode 100644
index 0000000..1020355
--- /dev/null
+++ b/content/docs/data-model/A-provider-contract.md
@@ -0,0 +1,560 @@
+# DCM โ€” Unified Provider Contract
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Foundation
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Policy Contract](B-policy-contract.md) | [Registration Specification](../specifications/dcm-registration-spec.md) | [Governance Matrix](27-governance-matrix.md) | [Accreditation](26-accreditation-and-authorization-matrix.md)
+
+---
+
+> > **Design Priority:** Provider types implement all four design priorities simultaneously. Security properties (mTLS, scoped credentials, sovereignty declarations, accreditation) are present in all provider registrations. The capability extension model (Priority 3) enables new provider types without changing the base contract. See [Design Priorities](00-design-priorities.md).
+
+## 1. The Unified Provider Contract
+
+Every Provider in DCM โ€” regardless of type โ€” implements a single base contract. What varies between provider types is the **capability extension**: the specific operations exposed, the data that flows in each direction, and the typed schemas for that exchange.
+
+```
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚                BASE PROVIDER CONTRACT                    โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Registration ยท Health ยท Sovereignty ยท Accreditation    โ”‚
+โ”‚  Governance Matrix ยท Zero Trust ยท Lifecycle              โ”‚
+โ”‚                                                          โ”‚
+โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”   โ”‚
+โ”‚  โ”‚           CAPABILITY EXTENSION                   โ”‚   โ”‚
+โ”‚  โ”‚                                                  โ”‚   โ”‚
+โ”‚  โ”‚  What operations this provider type exposes.     โ”‚   โ”‚
+โ”‚  โ”‚  What data flows in which direction.             โ”‚   โ”‚
+โ”‚  โ”‚  What schemas govern the exchange.               โ”‚   โ”‚
+โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜   โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+```
+
+**Adding a new provider type** = implement the base contract + define a capability extension. No changes to the core required.
+
+---
+
+## 2. Base Contract โ€” Registration
+
+All providers register through the same pipeline. See [Registration Specification](../specifications/dcm-registration-spec.md) for the complete flow.
+
+```yaml
+provider_base_registration:
+  # Standard artifact metadata
+  artifact_metadata:
+    uuid: 
+    handle: "//"    # e.g., "org/compute/eu-west-prod-1"
+    version: "1.0.0"
+    status: submitted                      # submitted โ†’ validating โ†’ active
+    owned_by: { display_name: "" }
+
+  provider_type_id:                # from Provider Type Registry
+  display_name: ""
+  description: ""
+
+  # All providers declare these
+  sovereignty_declaration:
+    operating_jurisdictions: []
+    data_residency_zones: []
+    sub_processors: []                   # third parties with data access
+
+  accreditations:
+    - accreditation_uuid:          # reference to registered accreditation
+      framework: 
+      status: active
+
+  # Endpoints (which endpoints are required varies by type โ€” see extensions)
+  health_endpoint: "https:///health"
+
+  # Zero trust identity
+  certificate:
+    pem: 
+    ca_chain: 
+    rotation_interval: P90D
+```
+
+**Registration lifecycle states:**
+```
+SUBMITTED โ†’ VALIDATING โ†’ PENDING_APPROVAL โ†’ ACTIVE
+                       โ†˜ REJECTED
+ACTIVE โ†’ SUSPENDED | DEREGISTERING โ†’ DEREGISTERED | FORCED_DEREGISTERED
+```
+
+---
+
+## 3. Base Contract โ€” Health Check
+
+Every provider implements a health endpoint. DCM calls it on the declared interval.
+
+```
+GET {health_endpoint}
+
+Response 200:
+{
+  "status": "healthy | degraded | unhealthy",
+  "version": "",
+  "capabilities_available": [""],
+  "details": { }    # provider-specific; DCM treats as opaque
+}
+```
+
+**DCM response to health states:**
+- `healthy` โ†’ normal operations; next poll scheduled
+- `degraded` โ†’ reduced routing preference; platform admin notified (medium urgency)
+- `unhealthy` / no response โ†’ after `failure_threshold`: status โ†’ DEGRADED; new requests not routed
+- After 2ร— `failure_threshold`: status โ†’ UNAVAILABLE; drift detection triggered on all hosted entities
+
+---
+
+## 4. Base Contract โ€” Governance Matrix Enforcement
+
+Every interaction with every provider is evaluated against the Governance Matrix before data crosses the boundary. This is not optional and not configurable per provider โ€” it is a base contract requirement.
+
+```
+Outbound interaction (DCM โ†’ Provider):
+  1. Classify all fields in the payload by data_classification
+  2. Resolve provider's active accreditations
+  3. Evaluate Governance Matrix: permitted | strip_field | deny | redact
+  4. Apply field permissions
+  5. Audit record written (regardless of outcome)
+  6. If DENY: interaction blocked; entity enters PENDING_REVIEW if appropriate
+
+Inbound interaction (Provider โ†’ DCM):
+  1. Authenticate provider identity (mTLS)
+  2. Verify credential scope matches the operation
+  3. Accept payload; apply data_classification tags
+  4. Store in appropriate store per data_classification
+```
+
+---
+
+## 5. Base Contract โ€” Zero Trust
+
+All provider interactions operate under the active zero trust posture. Minimum requirement for all providers at all profiles:
+
+- Mutual TLS authentication on every call (both sides present certificates)
+- Scoped, short-lived interaction credentials (not long-lived API keys)
+- Every call authenticated; no implicit trust from network position or prior calls
+- Certificate rotation on declared interval
+
+Higher profiles add: certificate pinning, per-message signing, hardware attestation.
+
+---
+
+## 6. Base Contract โ€” Provider Lifecycle Events
+
+Providers must report state changes via lifecycle events. This is a base contract obligation โ€” not optional:
+
+```json
+POST {dcm_lifecycle_endpoint}
+{
+  "event_uuid": "",
+  "event_type": "",
+  "provider_uuid": "",
+  "affected_entity_uuids": [""],
+  "event_timestamp": "",
+  "severity": "INFO | WARNING | CRITICAL"
+}
+```
+
+---
+
+## 7. Capability Extensions โ€” All Eleven Types
+
+### 7.1 Service Provider
+
+**What it does:** Realizes infrastructure resources. Receives assembled payloads, provisions the resource, returns realized state.
+
+**Additional endpoints:**
+```
+POST {dispatch_endpoint}         # receive and execute dispatch payload
+POST {cancel_endpoint}           # receive cancellation request (if supported)
+POST {discover_endpoint}         # receive discovery request; return discovered state
+```
+
+**Capability declaration extension:**
+```yaml
+service_provider_capabilities:
+  resource_types:
+    - fqn: Compute.VirtualMachine
+      spec_version: "2.1.0"
+      catalog_item_uuid: 
+  cancellation:
+    supports_cancellation: true
+    cancellation_supported_during: [DISPATCHED, PROVISIONING]
+  discovery:
+    supports_discovery: true
+    discovery_method: api_query | passive_event | hybrid
+  naturalization:
+    target_format: openstack_nova | vmware_vsphere | custom
+  cost_metadata:
+    opex_per_unit_per_hour: 0.28
+    currency: USD
+```
+
+**Data direction:** DCM sends assembled Requested State โ†’ Provider naturalizes โ†’ executes โ†’ denaturalizes โ†’ returns Realized State. DCM writes Realized State to Snapshot Store.
+
+---
+
+### 7.2 Information Provider
+
+**What it does:** Serves authoritative external data to enrich DCM's understanding of resources and business context.
+
+**Additional endpoints:**
+```
+POST {query_endpoint}            # receive query; return data in DCM unified format
+POST {write_back_endpoint}       # optional; receive DCM updates to push to source system
+```
+
+**Capability declaration extension:**
+```yaml
+information_provider_capabilities:
+  data_domains:
+    - domain: business_data
+      data_types: [business_unit, cost_center, product_owner]
+      authority_level: primary | secondary | supplementary
+  query_capacity:
+    max_queries_per_second: 100
+  confidence_model:
+    data_freshness_sla: PT1H
+  write_back_supported: false
+```
+
+**Data direction:** DCM sends lookup query โ†’ Provider returns data in DCM format โ†’ DCM enriches entity fields.
+
+---
+
+### 7.3 Storage Provider
+
+**What it does:** Persists DCM state data. Implements one or more store contracts (GitOps, write-once snapshot, event stream, search index, audit).
+
+**Additional endpoints:**
+```
+POST {write_endpoint}            # receive and persist data
+GET  {read_endpoint}             # return stored data
+POST {query_endpoint}            # execute indexed query (search index sub-type)
+GET  {health_endpoint}           # includes store-specific metrics
+```
+
+**Capability declaration extension:**
+```yaml
+storage_provider_capabilities:
+  store_types:
+    - gitops               # Intent and Requested stores
+    - write_once_snapshot  # Realized store
+    - event_stream         # Discovered store
+    - search_index         # Query projection
+    - audit                # Audit store
+  consistency: strong | eventual | bounded_staleness
+  geo_replicated: true
+  encryption_at_rest: AES-256
+  hsm_backed: false
+```
+
+**Data direction:** Bidirectional. DCM writes state; DCM reads state. Provider never initiates.
+
+---
+
+### 7.4 Meta Provider
+
+**What it does:** Composes multiple child providers to deliver a compound service as a single catalog item. The Meta Provider declares a compound service definition โ€” constituent resource types, dependencies, and delivery requirements โ€” so DCM can place, sequence, and govern the constituents. For its own resource types (`provided_by: self`), the Meta Provider executes as a standard Service Provider. All orchestration, placement, sequencing, failure handling, and compensation is performed by DCM using the declared dependency graph.
+
+> **Full specification:** See [Meta Provider Composability Model](30-meta-provider-model.md) for the complete orchestration contract, four-state model, failure propagation, compensation, and system policies (MPX-001โ€“MPX-008).
+
+**Capability declaration extension (summary โ€” full schema in doc 30):**
+```yaml
+meta_provider_capabilities:
+  constituent_provider_types: [service_provider, information_provider, meta_provider]
+  composition_model:
+    execution: dependency_ordered    # sequential | parallel | dependency_ordered
+    max_concurrent_realizations: 10
+    max_constituent_count: 20
+    max_nesting_depth: 3
+  partial_delivery_supported: true
+  compensation_supported: true      # required if partial_delivery_supported: true (MPX-001)
+  compensation_timeout: PT30M
+  idempotency_guaranteed: true
+  status_reporting:
+    supported: true
+    interval: PT30S
+  resource_types_composed:
+    - fqn: ApplicationStack.WebApp
+      version: "2.0.0"
+      constituents:
+        - resource_type: Compute.VirtualMachine
+          required_for_delivery: required
+        - resource_type: Network.IPAddress
+          required_for_delivery: required
+        - resource_type: DNS.Record
+          required_for_delivery: partial
+      composition_visibility: selective   # opaque | transparent | selective
+```
+
+**Composite status determination:**
+- `REALIZED` โ€” all required constituents succeeded
+- `DEGRADED` โ€” required constituents succeeded; one or more partial constituents failed (accepted if `partial_delivery_supported: true`)
+- `FAILED` โ€” one or more required constituents failed โ†’ compensation executes
+
+**Data direction:** DCM sends fully assembled compound payload โ†’ Meta Provider orchestrates constituents in dependency order โ†’ aggregates realized states โ†’ returns compound realized state to DCM.
+
+---
+
+### 7.5 Policy Provider
+
+**What it does:** Evaluates policies externally. Receives a DCM payload, evaluates Rego or custom logic, returns a typed policy decision.
+
+**Additional endpoints:**
+```
+POST {evaluate_endpoint}         # receive payload; return policy decision
+POST {test_endpoint}             # receive test case; return evaluation result (shadow mode)
+```
+
+**Capability declaration extension:**
+```yaml
+policy_provider_capabilities:
+  mode: 1 | 2 | 3 | 4
+  policy_types: [gatekeeper, validation, transformation, recovery]
+  framework: opa | cedar | custom
+  shadow_mode_supported: true
+  endpoint_sovereignty_zone:    # required for Mode 4
+```
+
+**Data direction:** DCM sends payload + active policy bundle โ†’ Provider evaluates โ†’ returns typed decision (allow/deny, mutations, action).
+
+---
+
+### 7.6 Credential Provider
+
+**What it does:** Issues, rotates, and revokes credentials used within the DCM ecosystem โ€” both DCM interaction credentials (short-lived, scoped, used for provider dispatch under the Zero Trust model) and consumer-facing resource credentials (SSH keys, API keys, kubeconfigs, service account tokens, database passwords, x509 certificates).
+
+> **Full specification:** See [Credential Provider Model](31-credential-provider-model.md) for the complete issuance contract, rotation protocol, revocation propagation, consumer delivery, and system policies (CPX-001โ€“CPX-008).
+
+**Credential values are never stored in DCM** โ€” only credential metadata (UUID, type, scope, expiry, status) is stored. Values are held by the Credential Provider and retrieved by authorized consumers via a declared `value_retrieval_endpoint`.
+
+**Additional endpoints:**
+```
+POST   {issue_endpoint}              # issue credential; return metadata + retrieval URL
+POST   {rotate_endpoint}             # rotate; return old/new UUIDs + transition window
+DELETE {revoke_endpoint}/{uuid}      # revoke immediately or at transition window end
+POST   {validate_endpoint}           # use-time validity check (scope, revocation, expiry)
+GET    {list_endpoint}               # list credentials by entity_uuid or issued_to
+```
+
+**Capability declaration extension (summary โ€” full schema in doc 31):**
+```yaml
+credential_provider_capabilities:
+  credential_types:
+    - api_key
+    - x509_certificate
+    - ssh_key
+    - service_account_token
+    - database_password
+    - kubeconfig
+    - hsm_backed_key
+    - dcm_interaction          # required if handling DCM interaction credentials
+  hsm_backed: false
+  fips_140_2_level: 0 | 1 | 2 | 3   # enforced per profile (Section 12)
+  dynamic_secrets: true
+  rotation_support: true
+  revocation_sla: PT5M         # profile-governed; PT30S for sovereign
+  approved_algorithms:         # declare which algorithms the provider supports
+    ssh_key: [Ed25519, ECDSA-P-384, RSA-4096]
+    x509_certificate: [Ed25519, ECDSA-P-384, RSA-4096]
+    service_account_token: [RS256, ES256, HS256]
+    # ... per credential type
+  key_escrow:
+    supported: false           # true only for regulated sovereign deployments
+```
+
+**Data direction:** DCM requests credential โ†’ Provider issues scoped credential + returns metadata โ†’ DCM stores metadata, includes retrieval URL in realized entity โ†’ Consumer retrieves value via authenticated endpoint. Revocation: DCM requests revocation โ†’ Provider invalidates โ†’ DCM publishes revocation event to Message Bus โ†’ all components refresh revocation cache within profile-governed TTL.
+
+---
+
+### 7.7 Auth Provider
+
+**What it does:** Authenticates actor identities and resolves their roles and group memberships.
+
+**Additional endpoints:**
+```
+POST {authenticate_endpoint}     # receive credentials; return auth token + claims
+POST {authorize_endpoint}        # receive token + operation; return allow/deny
+GET  {identity_endpoint}         # return actor claims for a token
+```
+
+**Capability declaration extension:**
+```yaml
+auth_provider_capabilities:
+  authentication_modes: [oidc, ldap, saml, mtls, hardware_token]
+  mfa_methods: [totp, push_notification, hardware_token]
+  rbac_model: flat | hierarchical | abac
+  step_up_supported: true
+  token_lifetime:
+    default: PT1H
+    max: PT8H
+```
+
+**Data direction:** Consumer sends credentials โ†’ Auth Provider validates โ†’ returns token + claims โ†’ DCM extracts actor identity.
+
+---
+
+### 7.8 Notification Provider
+
+**What it does:** Receives unified notification envelopes from DCM and delivers them via configured channels.
+
+**Additional endpoints:**
+```
+POST {delivery_endpoint}         # receive notification envelope; deliver to channel
+POST {delivery_status_endpoint}  # callback: report delivery status to DCM
+```
+
+**Capability declaration extension:**
+```yaml
+notification_provider_capabilities:
+  delivery_channels:
+    - channel_type: slack | pagerduty | email | webhook | sms | servicenow
+      supports_urgency_routing: true
+      config_schema_ref: 
+  delivery_guarantees:
+    at_least_once: true
+    idempotency_key: notification_uuid
+    max_latency_seconds: 30
+  sovereignty_aware_delivery: true
+```
+
+**Data direction:** DCM sends notification envelope โ†’ Provider translates to channel format โ†’ delivers โ†’ reports status.
+
+---
+
+### 7.9 Message Bus Provider
+
+**What it does:** Provides persistent, high-throughput asynchronous event streaming between DCM components and external systems.
+
+**Capability declaration extension:**
+```yaml
+message_bus_capabilities:
+  protocols: [kafka, amqp, mqtt]
+  persistence: true
+  durability: at_least_once | exactly_once
+  external_endpoints: false          # true only if messages leave sovereignty boundary
+  topics:
+    - name: dcm.events
+      retention: P7D
+```
+
+**Data direction:** Bidirectional publish/subscribe. DCM publishes events; components and external systems subscribe.
+
+---
+
+### 7.10 Registry Provider
+
+**What it does:** Serves the Resource Type Registry โ€” the authoritative catalog of resource types available to DCM deployments.
+
+**Additional endpoints:**
+```
+GET  {registry_endpoint}         # serve registry entries (full or incremental)
+GET  {bundle_endpoint}           # serve signed registry bundle (air-gapped mode)
+```
+
+**Capability declaration extension:**
+```yaml
+registry_provider_capabilities:
+  serves_tiers: [core, verified_community, organization]
+  incremental_sync: true
+  signed_bundles: true           # for air-gapped deployments
+  bundle_signing_key_ref: 
+```
+
+**Data direction:** DCM pulls registry entries โ†’ Provider returns signed bundle or live entries.
+
+---
+
+### 7.11 Peer DCM (Federation)
+
+**What it does:** Another DCM instance participating in federation. Treated as a typed Provider with a federation tunnel as the communication channel.
+
+**Capability declaration extension:**
+```yaml
+peer_dcm_capabilities:
+  dcm_version: "1.0.0"
+  tunnel_type: peer | parent_child | hub_spoke
+  deployment_accreditations: []
+  inbound_authorization:          # what this peer may request from local DCM
+    - operation: catalog_query
+      resource_types: [Compute.VirtualMachine]
+  outbound_authorization:         # what local DCM may request from this peer
+    - operation: placement_query
+      resource_types: [Compute.VirtualMachine]
+  data_boundary:
+    max_classification: restricted
+  trust_posture: verified | vouched | provisional
+```
+
+**Data direction:** Bidirectional within declared authorization scope. Federation tunnel with mTLS, certificate pinning, per-message signing.
+
+---
+
+## 8. Provider Type Registry
+
+The Provider Type Registry is the authoritative list of provider types that a DCM deployment accepts registrations for. It follows the three-tier registry model (Core / Verified Community / Organization).
+
+```yaml
+provider_type_registry_entry:
+  provider_type_id: service_provider
+  tier: core
+  default_approval_method: reviewed   # auto | reviewed | verified | authorized
+  enabled_in_profiles: [minimal, dev, standard, prod, fsi, sovereign]
+  capability_extension_schema_ref: 
+```
+
+Profile-governed approval methods override provider type defaults. See [Registration Specification](../specifications/dcm-registration-spec.md) Section 3 for the complete approval method resolution model.
+
+---
+
+## 9. Related Policies
+
+| Policy | Rule |
+|--------|------|
+| `PRV-001` | All providers implement the base contract. No provider is exempt from registration, health check, sovereignty declaration, governance matrix enforcement, or zero trust authentication. |
+| `PRV-002` | Governance Matrix evaluation occurs before every provider interaction. It is not configurable per provider and cannot be bypassed. |
+| `PRV-003` | Provider capability declarations are verified at registration. Capabilities not declared at registration cannot be invoked after activation. |
+| `PRV-004` | Peer DCM instances are treated as typed providers. Federation is the Provider abstraction applied across DCM instances โ€” not a separate abstraction. |
+| `PRV-005` | Adding a new provider type requires implementing the base contract and defining a capability extension. No changes to DCM core are required. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+## ITSM Provider
+
+**What it does:** Provides bidirectional integration with external ITSM systems. Outbound: receives DCM lifecycle event data and creates/updates records in the ITSM system (change requests, incidents, CMDB CIs). Inbound: routes ITSM approval decisions back to DCM as approval votes, and accepts ITSM-initiated request submissions.
+
+**Additional endpoints (ITSM Provider implements):**
+```
+POST {provider_base}/actions              # receive action request from DCM
+GET  {provider_base}/actions/{action_id}  # DCM checks action completion status
+GET  {provider_base}/records/{record_id}  # DCM retrieves record status from ITSM
+POST {provider_base}/inbound             # ITSM system sends inbound approval/event
+```
+
+**Capability declaration extension:**
+```yaml
+itsm_provider_capabilities:
+  itsm_system: servicenow | jira_service_management | bmc_remedy | bmc_helix |
+               freshservice | zendesk | pagerduty | opsgenie | manageengine |
+               cherwell | topdesk | generic_rest
+  supported_actions: []
+  endpoint_url: 
+  auth_credential_uuid: 
+  inbound_webhook:
+    enabled: 
+    secret_credential_uuid: 
+  field_mapping_ref: 
+  cmdb_ci_type_map: []
+```
+
+> **See [ITSM Integration](42-itsm-integration.md)** for complete capability declaration schemas, supported systems, and system policies ITSM-001โ€“007.
diff --git a/content/docs/data-model/B-policy-contract.md b/content/docs/data-model/B-policy-contract.md
new file mode 100644
index 0000000..a2f964f
--- /dev/null
+++ b/content/docs/data-model/B-policy-contract.md
@@ -0,0 +1,430 @@
+# DCM โ€” Unified Policy Contract
+
+> **โš ๏ธ Active Development Notice**
+>
+> The DCM data model and architecture documentation are actively being developed.
+>
+> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Foundation
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Profiles](14-policy-profiles.md) | [Governance Matrix](27-governance-matrix.md) | [OPA Integration](../specifications/dcm-opa-integration-spec.md)
+
+---
+
+## 1. The Unified Policy Contract
+
+Every Policy in DCM โ€” regardless of type โ€” implements a single base contract. What varies between policy types is the **output schema**: what the Policy produces when its match conditions are satisfied.
+
+```
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚                 BASE POLICY CONTRACT                     โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Match Conditions ยท Enforcement Level ยท Domain          โ”‚
+โ”‚  Lifecycle ยท Audit ยท Shadow Mode                         โ”‚
+โ”‚                                                          โ”‚
+โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”   โ”‚
+โ”‚  โ”‚              OUTPUT SCHEMA                       โ”‚   โ”‚
+โ”‚  โ”‚                                                  โ”‚   โ”‚
+โ”‚  โ”‚  What this policy type produces when it fires.   โ”‚   โ”‚
+โ”‚  โ”‚  Seven typed output schemas.                     โ”‚   โ”‚
+โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜   โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+```
+
+**Adding a new policy type** = define a new output schema. The base contract, evaluation algorithm, lifecycle, and audit obligations are inherited.
+
+---
+
+## 2. Base Contract โ€” Match Conditions
+
+All policies declare when they fire using one or both of two match condition models:
+
+**Model A โ€” Payload type + field conditions** (for pipeline policies: GateKeeper, Validation, Transformation, Recovery, Orchestration Flow):
+
+```yaml
+match:
+  payload_type: request.initiated | request.layers_assembled | ...   # from closed vocabulary
+  conditions:
+    - field:                 # dot-notation path into the payload
+      operator: equals | in | minimum | maximum | contains | matches
+      value: 
+    - field: 
+      operator: not_equals
+      value: 
+  condition_logic: all | any             # default: all
+```
+
+**Model B โ€” Four-axis boundary conditions** (for boundary policies: Governance Matrix Rules):
+
+```yaml
+match:
+  subject:
+    type: 
+    identity: { ... }
+    tenant: { ... }
+  data:
+    classification: 
+    resource_type: 
+    field_paths: { mode: allowlist | blocklist, paths: [...] }
+    capability: 
+  target:
+    type: 
+    sovereignty_zone: { match:  }
+    accreditation_held: { includes: [...] }
+    trust_posture: 
+  context:
+    profile: { deployment_posture:  }
+    zero_trust_posture: { minimum:  }
+    federated: true | false
+```
+
+Policies may declare match conditions using either model. Orchestration Flow policies and Lifecycle Policies may also use relationship event conditions (see Section 9).
+
+---
+
+## 3. Base Contract โ€” Enforcement Level
+
+```yaml
+enforcement: hard | soft
+
+# hard: cannot be relaxed by any downstream rule at any domain level
+#        A hard DENY cannot be overridden by any Tenant, entity, or operator override
+#        Reserved for: sovereign/classified data boundaries, regulatory hard requirements
+
+# soft: establishes a default that downstream rules can tighten
+#        A soft ALLOW can be restricted to DENY by a more-specific rule
+#        A soft DENY cannot be relaxed to ALLOW by a downstream rule
+```
+
+Most policies are soft. Hard enforcement is reserved for absolute security constraints.
+
+---
+
+## 4. Base Contract โ€” Domain Precedence
+
+Policies operate within a domain hierarchy. More-specific domains win within the same concern type:
+
+```
+system (most trusted โ€” DCM built-in)
+  โ””โ”€โ”€ platform (platform admin declared)
+        โ””โ”€โ”€ tenant (Tenant admin declared)
+              โ””โ”€โ”€ resource_type (per resource type spec)
+                    โ””โ”€โ”€ entity (per specific entity โ€” most specific)
+```
+
+Within the same domain level, DENY wins over ALLOW. More-specific domain wins over less-specific.
+
+---
+
+## 5. Base Contract โ€” Artifact Structure
+
+All policies are first-class DCM Data artifacts. They share the standard artifact metadata and lifecycle:
+
+```yaml
+policy_artifact:
+  # Standard DCM artifact metadata (all artifacts carry this)
+  artifact_metadata:
+    uuid: 
+    handle: "//"
+    version: "1.0.0"
+    status: developing | proposed | active | deprecated | retired
+    owned_by: { display_name: "", email: "" }
+    created_by: { display_name: "" }
+    created_via: pr | api | migration | system
+
+  # Policy classification
+  policy_type:                     # gatekeeper | validation | transformation |
+                                         # recovery | orchestration_flow |
+                                         # governance_matrix_rule | lifecycle
+  concern_type:                 # security | compliance | operational |
+                                         # recovery_posture | zero_trust_posture |
+                                         # data_authorization_boundary | orchestration_flow
+
+  domain: system | platform | tenant | resource_type | entity
+
+  # Match conditions (Model A or B โ€” see Section 2)
+  match: { ... }
+
+  # Enforcement
+  enforcement: hard | soft
+
+  # Output schema (varies by policy_type โ€” see Sections 8-14)
+  output: { ... }
+
+  # Audit
+  audit_on: [ALLOW, DENY, STRIP_FIELD]   # which decisions produce audit records
+  notification_on: [DENY]               # which decisions trigger notifications
+  notification_urgency: low | medium | high | critical
+
+  # Compliance reference
+  compliance_basis: ""
+  review_required_before: ""
+```
+
+---
+
+## 6. Base Contract โ€” Lifecycle
+
+All policies follow the five-status lifecycle:
+
+| Status | Behavior |
+|--------|---------|
+| `developing` | Dev mode only. Not applied in any environment. |
+| `proposed` | Shadow mode: executes against real traffic; output captured but never applied. Used for safe validation. |
+| `active` | Applied to all matching requests. |
+| `deprecated` | Still active; replacement available; warning on evaluation. |
+| `retired` | Terminal; cannot be used. |
+
+**Shadow mode (proposed status):** The policy evaluates against real traffic. Its output is captured in the Validation Store. Platform admins review shadow results before promoting to active. This is the primary mechanism for safe policy change management.
+
+---
+
+## 7. Base Contract โ€” Evaluation and Audit
+
+**Evaluation order:** Within a domain level, policies are evaluated in declared priority order. Across domain levels, more-specific domains evaluate after (and can override) less-specific domains.
+
+**Parallel evaluation:** Policies with no data dependencies on each other evaluate concurrently. The Policy Engine tracks dependency declarations.
+
+**Audit:** Every policy evaluation produces an audit record regardless of outcome. The record includes: policy_uuid, policy_version, match_result, output, enforcement_level, actor, timestamp. No evaluation is silent.
+
+---
+
+## 8. Output Schema โ€” GateKeeper
+
+**Fires on:** Request payload at assembly time.
+**Produces:** An allow or deny decision for the request.
+
+```yaml
+gatekeeper_output:
+  decision: allow | deny
+  reason: ""
+  field_locks:                           # optional: lock specific fields as immutable
+    - field: 
+      lock_type: immutable | constrained
+      constraint_schema:    # if constrained
+  warnings: [""]
+```
+
+**Policy Engine behavior:**
+- `allow` โ†’ request proceeds; field_locks applied to payload
+- `deny` โ†’ request blocked; `reason` included in consumer error response
+- Any active GateKeeper producing `deny` โ†’ request blocked (all must allow)
+
+---
+
+## 9. Output Schema โ€” Validation
+
+**Fires on:** Request payload; validates correctness of field values.
+**Produces:** Pass or fail with field-level detail.
+
+```yaml
+validation_output:
+  result: pass | fail
+  field_results:
+    - field: 
+      result: valid | invalid
+      message: ""
+      suggested_value:            # optional
+  advisory: [""]
+```
+
+**Policy Engine behavior:**
+- `pass` โ†’ request proceeds
+- `fail` โ†’ request blocked; `field_results` included in consumer error response
+
+---
+
+## 10. Output Schema โ€” Transformation
+
+**Fires on:** Request payload; enriches, modifies, or injects field values.
+**Produces:** A set of field mutations to apply to the payload.
+
+```yaml
+transformation_output:
+  mutations:
+    - field: 
+      operation: set | append | delete | lock
+      value:                  # for set/append
+      reason: ""
+      source_type: enrichment | injection | normalization | correction
+```
+
+**Policy Engine behavior:** All mutations from all active Transformation policies are collected and applied to the payload. Each mutation is recorded in field-level provenance with the policy_uuid as source.
+
+---
+
+## 11. Output Schema โ€” Recovery
+
+**Fires on:** A failure or ambiguity trigger condition (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, CANCELLATION_FAILED, etc.).
+**Produces:** A recovery action and parameters.
+
+```yaml
+recovery_output:
+  action: DRIFT_RECONCILE | DISCARD_AND_REQUEUE | DISCARD_NO_REQUEUE |
+          ACCEPT_LATE_REALIZATION | COMPENSATE_AND_FAIL |
+          NOTIFY_AND_WAIT | ESCALATE | RETRY
+  action_parameters:
+    requeue_delay: PT0S                  # for DISCARD_AND_REQUEUE
+    max_attempts: 3                      # for RETRY
+    backoff: exponential                 # for RETRY
+    deadline: PT4H                       # for NOTIFY_AND_WAIT
+    on_deadline_exceeded: ESCALATE       # for NOTIFY_AND_WAIT
+  notify_before_action: true
+  notification_urgency: high
+```
+
+**Policy Engine behavior:** The first matching Recovery policy's action is executed. Recovery policies follow the same domain precedence โ€” resource_type override wins over tenant override wins over profile default.
+
+---
+
+## 12. Output Schema โ€” Orchestration Flow
+
+**The two-level orchestration model:**
+
+Orchestration in DCM operates at two levels that compose through the same Policy Engine:
+
+- **Level 1 โ€” Named Workflow Artifacts:** Orchestration Flow Policies with `ordered: true` declare an explicit, visible, auditable sequence of steps. Each step references a payload type from the closed vocabulary. This is what operators see and reason about. Adding a step = adding to a workflow Policy.
+- **Level 2 โ€” Dynamic Policies:** GateKeeper, Transformation, Recovery, and Governance Matrix Policies fire when their conditions match, within or alongside workflow steps, without being declared in the workflow. Adding conditional behavior = writing a dynamic policy.
+
+The Request Orchestrator (event bus) routes all payload type events through the Policy Engine. Both named workflow steps and dynamic policies evaluate against the same events. The workflow provides the skeleton; dynamic policies fill in conditional behavior.
+
+**Fires on:** Pipeline payload type events.
+**Produces:** A flow directive governing step ordering.
+
+```yaml
+orchestration_flow_output:
+  ordered: true | false
+  steps:
+    - step: 1
+      policy_handle: ""
+      condition: ""
+      on_fail: halt | skip | escalate
+  parallel_groups:                       # steps that may execute in parallel
+    - [step_1_id, step_2_id]
+```
+
+**Step vocabulary** โ€” steps reference payload types from the closed vocabulary, mapping to control plane operations:
+
+| Payload type | Maps to |
+|-------------|---------|
+| `request.initiated` | Start of request pipeline |
+| `request.layers_assembled` | Layer assembly complete |
+| `request.policies_evaluated` | All policies evaluated |
+| `request.placement_complete` | Provider selected |
+| `request.dispatched` | Sent to provider |
+| `discovery.cycle_complete` | Discovery cycle done |
+| `drift.detected` | Drift found |
+| `recovery.timeout_fired` | Dispatch timeout |
+| `provider_update.received` | Provider update notification |
+
+Custom steps extend this vocabulary by publishing new payload types.
+
+**Policy Engine behavior:** When `ordered: true`, steps execute in declared sequence. When `ordered: false`, the Policy Engine executes steps in parallel where no data dependencies exist. Orchestration Flow policies compose with standard GateKeeper and Transformation policies โ€” both types evaluate in the same pipeline.
+
+---
+
+## 13. Output Schema โ€” Governance Matrix Rule
+
+**Fires on:** Any cross-boundary interaction (DCM โ†’ Provider, DCM โ†’ Peer DCM, Provider โ†’ DCM).
+**Produces:** A boundary control decision with optional field permissions.
+
+```yaml
+governance_matrix_output:
+  decision: ALLOW | DENY | ALLOW_WITH_CONDITIONS | STRIP_FIELD | REDACT | AUDIT_ONLY
+  conditions:                            # for ALLOW_WITH_CONDITIONS
+    - field: 
+      operator: 
+      value: 
+  field_permissions:
+    mode: allowlist | blocklist | passthrough
+    paths: ["", ...]
+    on_blocked_field: STRIP_FIELD | DENY_REQUEST | REDACT
+  audit_on: [ALLOW, DENY, STRIP_FIELD]
+  notification_on: [DENY]
+  notification_urgency: critical
+```
+
+**Policy Engine behavior:** Hard DENY evaluated first โ€” any hard DENY is terminal. Soft decisions evaluated by domain precedence; DENY wins over ALLOW at the same level. Field permissions applied after decision determined. Audit record always written.
+
+---
+
+## 14. Output Schema โ€” Lifecycle Policy
+
+**Fires on:** Relationship events (related entity state changes, relationship creation/release).
+**Produces:** A lifecycle action to apply to related entities.
+
+```yaml
+lifecycle_policy_output:
+  on_related_destroy: cascade | protect | detach | notify
+  on_related_suspend: cascade | ignore | notify
+  on_last_relationship_released: destroy | retain | notify
+  propagation_depth: 1 | 2 | N          # how many relationship hops to propagate
+  action_delay: PT0S                     # grace period before executing action
+```
+
+**Policy Engine behavior:** When a relationship event occurs, all matching Lifecycle policies on both related entities are evaluated. The most restrictive action wins (save beats destroy). Conflicts between policies at the same domain level produce a CONFLICT_ERROR at policy ingestion time.
+
+---
+
+## 15. Output Schema โ€” ITSM Action
+
+The ITSM Action policy type triggers actions in connected ITSM systems as a side-effect of DCM pipeline events.
+
+```yaml
+itsm_action_output:
+  type: itsm_action
+  itsm_provider_uuid:        # registered ITSM Provider UUID
+  action: create_change_request | update_change_request | close_change_request |
+          create_incident | update_incident | close_incident |
+          update_cmdb_ci | create_cmdb_ci | retire_cmdb_ci |
+          create_service_request | link_parent_record
+  action_payload:
+    : 
+  store_reference_on_entity:    # default: false
+  reference_label: 
+  block_until_created:          # default: false โ€” see ITSM-005
+  block_timeout:   # required if block_until_created: true
+  on_failure: log_and_continue | alert_and_continue | alert_only
+```
+
+> **See [ITSM Integration](42-itsm-integration.md)** for full ITSM Provider registration, capability declarations, supported ITSM systems (ServiceNow, Jira, Remedy, Freshservice, PagerDuty, generic REST), policy examples, and system policies (ITSM-001โ€“007, ITSM-POL-001โ€“004).
+
+**Key constraints:**
+- ITSM Action policies are side-effect only โ€” they do not produce allow/deny decisions
+- `block_until_created: true` creates a pipeline gate with mandatory timeout (ITSM-005)
+- Multiple ITSM Action policies on the same event fire independently (ITSM-POL-004)
+- Full audit record produced on every evaluation (ITSM-POL-003)
+
+## 16. Policy Composition
+
+Policies compose naturally through the domain precedence model:
+
+```
+System policy (GateKeeper: cpu_count max 64)
+  โ””โ”€โ”€ Platform policy (GateKeeper: prod VMs require manager approval)
+        โ””โ”€โ”€ Tenant policy (GateKeeper: payments team max cpu_count 32)
+              โ””โ”€โ”€ Resource-type policy (Transformation: inject monitoring)
+```
+
+For a single request, all active matching policies at all domain levels evaluate. GateKeepers at all levels must allow (any deny blocks). Transformations from all levels are collected and applied. Recovery policies use the most-specific matching policy.
+
+**Policy Groups** are Data artifacts that group related policies by concern_type. Profiles activate Policy Groups. This is how "apply the HIPAA profile" works โ€” it activates the HIPAA compliance domain's Policy Group, which contains all the GateKeeper, Validation, Transformation, and Governance Matrix policies required for HIPAA compliance.
+
+---
+
+## 17. Related Policies
+
+| Policy | Rule |
+|--------|------|
+| `POL-001` | All DCM policy types implement the unified base contract. The output schema is the only thing that varies. |
+| `POL-002` | Every policy evaluation produces an audit record. No evaluation is silent. |
+| `POL-003` | Hard enforcement policies cannot be relaxed by any downstream rule at any domain level. |
+| `POL-004` | Policies in `proposed` status execute in shadow mode โ€” output is captured and never applied. Shadow mode is the primary mechanism for safe policy change management. |
+| `POL-005` | The Policy Engine is the sole evaluator of all policies. No component bypasses the Policy Engine to enforce rules directly. |
+| `POL-006` | Adding a new policy type requires defining a new output schema. The base contract, evaluation algorithm, lifecycle, and audit obligations are inherited. |
+| `POL-007` | Policies ARE the orchestration. Pipeline steps are Policies firing on payload type events. Static flows are Orchestration Flow Policies with `ordered: true`. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/schemas/README.md b/content/docs/schemas/README.md
new file mode 100644
index 0000000..d0af7e9
--- /dev/null
+++ b/content/docs/schemas/README.md
@@ -0,0 +1,176 @@
+# DCM Normative Data Specifications
+
+**Status:** Draft โ€” Ready for implementation feedback
+**Version:** 1.0.0
+**GitHub:** https://github.com/dcm-project
+
+These are the machine-readable normative specifications for the DCM data model. They are the
+code-generation layer โ€” SDK authors, server stub generators, and validation middleware should
+use these files, not the narrative documentation in `data-model/`.
+
+The narrative documentation remains authoritative for intent and rationale. These schemas are
+authoritative for field names, types, constraints, and API shapes.
+
+---
+
+## Files
+
+### JSON Schema (entity, policy, provider, and event data shapes)
+
+| File | Describes | Types | Source docs |
+|------|-----------|-------|-------------|
+| `dcm-common.json` | Shared primitive types reused across all schemas | 17 | `00-foundations.md`, `16-universal-audit.md` |
+| `entities/dcm-entities.json` | All three entity types (Infrastructure Resource, Composite Resource, Process Resource) | 18 | `01-entity-types.md` |
+| `policies/dcm-policies.json` | All seven policy types and their output schemas | 31 | `B-policy-contract.md`, `14-policy-profiles.md` |
+| `providers/dcm-providers.json` | Provider base contract and all 11 capability extensions | 19 | `A-provider-contract.md`, `dcm-registration-spec.md` |
+| `events/dcm-events.json` | Base event envelope + all 82 event-specific payload schemas | 75 | `33-event-catalog.md` |
+| `resource-types/resource-type-spec-template.json` | Resource Type Spec schema + authoring guide + Compute.VirtualMachine example | 3 | `05-resource-type-hierarchy.md`, `20-registry-governance.md` |
+
+### OpenAPI 3.1 (API shapes)
+
+| File | Describes | Paths | Schemas | Source docs |
+|------|-----------|-------|---------|-------------|
+| `openapi/dcm-consumer-api.yaml` | Consumer API โ€” application teams and Tenant owners | 59 | 31 | `consumer-api-spec.md` |
+| `openapi/dcm-admin-api.yaml` | Admin API โ€” platform engineers and SREs | 41 | 19 | `dcm-admin-api-spec.md` |
+| `openapi/dcm-operator-api.yaml` | Operator Interface Services API โ€” provider-facing (DCM is client) | 5 | 12 | `dcm-operator-interface-spec.md` |
+| `openapi/dcm-provider-callback-api.yaml` | Provider Callback API โ€” endpoints DCM exposes for operators to call (registration, realized state push, interim status, update notifications, lifecycle events) | 7 | 11 | `dcm-operator-interface-spec.md` |
+
+## AEP Alignment
+
+The OpenAPI specifications follow [AEP (API Enhancement Proposals)](https://aep.dev) conventions
+in three areas:
+
+**1. Custom methods (AEP-136)** โ€” Actions on resources use colon syntax rather than slash-separated
+verb paths. For example: `POST /resources/{name}:suspend` rather than `POST /resources/{name}/suspend`.
+This applies to all state-transition and action operations across the Consumer and Admin APIs.
+
+**2. Long-Running Operations (AEP-151)** โ€” Async operations that produce a trackable result return
+an `Operation` resource with a stable `name` field (poll URL), a `done` boolean, and either a
+`response` or `error` field when complete. Operations that are genuinely fire-and-forget (capacity
+reports, interim status, lifecycle events) retain `202 Accepted` without an Operation body.
+
+**3. Pagination (AEP-158)** โ€” List endpoints use `page_size` and `page_token` query parameters.
+Responses include a `next_page_token` field (empty string when no further pages exist).
+
+**What was deliberately not aligned:**
+
+- **Resource names** โ€” DCM retains UUIDs as the primary identifier. AEP's hierarchical
+  `name` strings (`tenants/{t}/resources/{r}`) are not used because DCM's UUID immutability
+  guarantee across ownership transfers and provider migrations is more architecturally
+  significant than AEP naming convention compliance.
+- **Timestamp field names** โ€” `created_at`/`updated_at` are retained in API responses rather
+  than `create_time`/`update_time`. Renaming these would require a data model change (the fields
+  exist in the entity schemas and Realized State records) with no functional benefit.
+
+
+---
+
+## Schema Design Principles
+
+**1. `$ref` over duplication.** Types defined in `dcm-common.json` are referenced by `$ref` from
+all other schemas. The common types โ€” uuid, handle, semver, iso8601_datetime, artifact_metadata,
+sovereignty_declaration โ€” are defined once.
+
+**2. Discriminated unions via `entity_type` / `policy_type`.** The `dcm_entity` and `dcm_policy`
+union types use the `entity_type` and `policy_type` fields as discriminators. Validation tools
+that support OpenAPI 3.1 discriminators can route to the correct sub-schema automatically.
+
+**3. `additionalProperties: false` on all closed shapes.** Entities and provider registrations
+use `additionalProperties: false` where the shape is fully known. Resource-type-specific fields
+(which vary by resource type) use `additionalProperties: true` on the `fields` object only.
+
+**4. Field-level provenance as a pattern.** The `field_provenance` type in `dcm-common.json`
+documents the provenance metadata that any data field may carry as a sibling `_provenance` key.
+This is a pattern, not enforced by schema (because JSON Schema cannot express "every field may
+have a sibling `_fieldname_provenance` key" without enumeration).
+
+**5. ISO 8601 throughout.** All datetime fields use `format: date-time`. All duration fields use
+`format: iso8601_duration` with the pattern `P...`. All country codes use `pattern: [A-Z]{2}`.
+
+**6. Closed vocabularies as enums.** All closed vocabularies from the narrative documentation
+are expressed as `enum` arrays. The closed vocabulary for payload_type (policy match conditions),
+lifecycle states, ownership models, provider types, and credential types are all enumerated.
+
+---
+
+## Usage
+
+### Validation (Python)
+
+```python
+import json
+import jsonschema
+from pathlib import Path
+
+schema_dir = Path("dcm-docs/schemas")
+
+# Load schemas
+common = json.loads((schema_dir / "dcm-common.json").read_text())
+entities = json.loads((schema_dir / "entities/dcm-entities.json").read_text())
+
+# Build a resolver that handles $ref across files
+store = {
+    "https://dcm-project.io/schemas/common/v1": common,
+    "https://dcm-project.io/schemas/entities/v1": entities,
+}
+resolver = jsonschema.RefResolver.from_schema(entities, store=store)
+
+# Validate an entity
+entity = { ... }
+jsonschema.validate(entity, entities, resolver=resolver)
+```
+
+### Validation (Go)
+
+```go
+import (
+    "github.com/santhosh-tekuri/jsonschema/v5"
+    _ "github.com/santhosh-tekuri/jsonschema/v5/httploader"
+)
+
+compiler := jsonschema.NewCompiler()
+compiler.AddResource("dcm-common.json", openFile("schemas/dcm-common.json"))
+compiler.AddResource("entities/dcm-entities.json", openFile("schemas/entities/dcm-entities.json"))
+
+schema, err := compiler.Compile("entities/dcm-entities.json")
+if err != nil { panic(err) }
+
+var entity interface{}
+json.Unmarshal(data, &entity)
+if err := schema.Validate(entity); err != nil {
+    fmt.Println(err)
+}
+```
+
+### OpenAPI Code Generation
+
+```bash
+# Generate Go server stubs from Consumer API
+oapi-codegen -package api -generate server,types \
+  schemas/openapi/dcm-consumer-api.yaml > pkg/api/consumer.gen.go
+
+# Generate Python client from Admin API
+openapi-python-client generate \
+  --path schemas/openapi/dcm-admin-api.yaml \
+  --output dcm-admin-client/
+
+# Generate TypeScript types
+openapi-typescript schemas/openapi/dcm-consumer-api.yaml \
+  --output src/types/dcm-consumer-api.ts
+```
+
+---
+
+## What Is Not Yet Here
+
+### Resource Type Field Schemas (per resource type)
+
+The `resource-types/resource-type-spec-template.json` defines the schema for the Resource Type Spec envelope and provides a worked `Compute.VirtualMachine` example. Individual resource type field schemas are owned by Service Providers and published to the Resource Type Registry โ€” DCM does not define them centrally. Service Providers should use the template and authoring guide to produce their own schemas.
+
+### Resource Type Field Schemas (per resource type)
+
+The `resource-types/resource-type-spec-template.json` defines the schema for the Resource Type Spec envelope and includes a complete `Compute.VirtualMachine` worked example. Individual resource type field schemas are owned by Service Providers and published to the Resource Type Registry per the template authoring guide.
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/taxonomy.md b/content/docs/taxonomy.md
index 12bc7cf..22e8827 100644
--- a/content/docs/taxonomy.md
+++ b/content/docs/taxonomy.md
@@ -129,6 +129,74 @@ The DCM taxonomy defines the precise vocabulary used throughout the architecture
 | **constituent_status** | Array of named component statuses in a compound/Meta Provider request (e.g. `[{ref: "vm", status: "REALIZED"}, {ref: "dns", status: "PROVISIONING"}]`). Surfaced in SSE stream and polling response so consumers can track multi-part operations. |
 
 
+
+
+
+### RHDH and Backstage Integration Terms
+
+| Term | Definition |
+|------|-----------|
+| **Red Hat Developer Hub (RHDH)** | Red Hat's enterprise distribution of Backstage. Primary deployment target for DCM consumer GUI. Provides pre-built auth (RHSSO/Keycloak), RBAC plugin, Dynamic Plugins, ArgoCD/Tekton/AAP integrations, and the PatternFly design system. |
+| **Dynamic Plugin** | An RHDH/Backstage plugin loaded at runtime without rebuilding the RHDH image. DCM ships all its plugins as Dynamic Plugins โ€” new versions deploy by updating a tag. |
+| **DCMService** | Backstage custom entity kind representing a DCM service catalog item. Auto-generated by `@dcm/backstage-plugin-catalog-backend`. Appears in RHDH Software Catalog and drives Software Template generation. |
+| **DCMResource** | Backstage custom entity kind representing a realized DCM resource (entity in REALIZED or later lifecycle state). Auto-synced from DCM Realized State every PT5M. Namespace = DCM tenant UUID. |
+| **Software Template** | Backstage Scaffolder construct that provides a wizard-based form for creating resources. DCM auto-generates one Software Template per catalog item from the item's field schema. The Scaffolder IS the DCM request form in RHDH mode. |
+| **Scaffolder Action** | A Backstage backend function executable from a Software Template. DCM provides: `dcm:request:submit`, `dcm:request:wait` (with live log streaming), `dcm:request:group`, `dcm:request:estimate`, `dcm:catalog:refresh`. |
+| **Entity Provider** | A Backstage catalog backend component that emits entity mutations. `@dcm/backstage-plugin-catalog-backend` implements an entity provider that syncs DCM catalog items and realized resources to the RHDH catalog. |
+| **PatternFly** | Red Hat's open-source design system and React component library. Used for all DCM GUI surfaces. Key components: Nav (sidebar), NavGroup (non-clickable headers), NavItem with NotificationBadge (approvals count), Table, Toolbar, Gallery (catalog cards), Drawer (live status), Modal (step-up MFA). |
+| **GUI-011โ€“GUI-013** | RHDH-specific capabilities: Dynamic Plugin loading (GUI-011), Scaffolder Template auto-generation (GUI-012), DCM Entity Provider for catalog sync (GUI-013). |
+
+
+
+### ITSM Integration Terms
+
+| Term | Definition |
+|------|-----------|
+| **ITSM Bridge** | DCM's bidirectional integration with ITSM systems (ServiceNow, Jira Service Management, Remedy). DCM is the system of record; ITSM is a consumer of DCM events and a source of approval votes. No ITSM dependency for DCM core operation. |
+| **ITSM Reference** | Optional metadata on a DCM entity linking it to one or more ITSM records (Change Request, Incident, CMDB CI). Stored as business data fields; visible on resource entity Overview tab; included in audit records. |
+| **CMDB Sync** | One-way sync from DCM to CMDB. DCM entities are the system of record; CMDB CI state is updated via Notification Provider subscription to `entity.*` events. Field mapping is provider-configured, not hardcoded. |
+| **GUI-014** | ITSM Integration Bridge capability โ€” ITSM references on entity pages, change records linked to requests, ITSM-sourced approval votes in request status, CMDB sync via Notification Provider. |
+
+
+
+### ITSM Integration Terms
+
+| Term | Definition |
+|------|-----------|
+| **ITSM Provider** | The 12th DCM Provider type. Provides bidirectional integration with IT Service Management systems (ServiceNow, Jira Service Management, BMC Remedy/Helix, Freshservice, PagerDuty, etc.). Outbound: DCM lifecycle events โ†’ ITSM records. Inbound: ITSM approvals โ†’ DCM approval votes. Implements full base Provider contract. |
+| **ITSM Action Policy** | The 8th DCM Policy output type. Side-effect policy that triggers an action in a connected ITSM system when a DCM event matches. Non-blocking by default (`on_failure: log_and_continue`). May gate the pipeline via `block_until_created: true` with mandatory timeout (ITSM-005). |
+| **ITSM Reference** | Metadata stored on a DCM entity linking it to an ITSM record: system, record_type (change_request/incident/cmdb_ci), record_id, record_url, status, last_synced_at. Preserved through entity lifecycle; included in audit records. |
+| **block_until_created** | Optional ITSM Policy flag. When `true`, DCM waits for ITSM record creation confirmation before dispatching to Service Provider. Requires `block_timeout`. Timeout expiry triggers `on_failure` behavior โ€” the pipeline never permanently stalls due to ITSM unavailability (ITSM-005). |
+| **CMDB CI Mapping** | ITSM Provider configuration mapping DCM resource type FQNs to ITSM CI class names (e.g. `Compute.VirtualMachine โ†’ cmdb_ci_server` in ServiceNow). Used for `create_cmdb_ci`, `update_cmdb_ci`, and `retire_cmdb_ci` actions. |
+| **recorded_via** | Field on DCM approval vote records identifying the system that submitted the vote (dcm_admin_ui / servicenow / jira / slack_bot / api_direct / other). Used by ITSM Provider inbound approval routing and in audit records for compliance traceability. |
+| **ITSM-001โ€“007** | ITSM Provider system policies. Key: ITSM-002 (DCM never requires ITSM โ€” non-blocking default), ITSM-003 (inbound webhooks must be authenticated), ITSM-005 (block_until_created must have timeout โ€” pipeline never permanently stalled). |
+| **ITSM-POL-001โ€“004** | ITSM Policy system policies. Key: ITSM-POL-002 (ITSM Policies are side-effect only โ€” not GateKeeper substitutes), ITSM-POL-003 (full audit record per evaluation), ITSM-POL-004 (multiple ITSM Policies on same event fire independently). |
+
+
+### Web Interface Terms
+
+| Term | Definition |
+|------|-----------|
+| **Consumer Portal** | The self-service web interface for application developers, owners, and tenant admins. Wraps the Consumer API completely. Bounded by tenancy (X-DCM-Tenant context). Features: catalog browse, request submission with scheduling and dependency groups, live SSE status stream with constituent tracking, resource management, approvals, cost and quota, notifications, sessions. |
+| **Admin Panel** | The platform operations console for Platform Admins, SREs, Policy Owners, Security teams, and Auditors. Wraps the Admin API. Features: platform health dashboard, tenant management, provider registration approval, accreditation, quota, scoring configuration, approval queue, tier registry editor, audit and compliance, session management. |
+| **Provider Management GUI** | The management interface for provider owner teams. One common shell (overview, health, config, audit) for all 11 provider types, with type-specific extension tabs. Service Provider extends with capacity, managed entities, naturalization; Credential Provider with inventory, rotation, revocation, external CA config; Auth Provider with session stats and SCIM sync; etc. |
+| **Unified Shell** | A single DCM web application with role-gated surfaces: Consumer Portal (all actors), Admin Panel (platform-level roles), Provider Management (provider_owner role), Flow GUI (policy_owner/sre). One login, one session. Navigation adapts to actor's highest privilege level. |
+| **GUI-001โ€“GUI-010** | Web Interface capabilities. Key: GUI-002 (SSE live status stream for consumer โ€” status_change, progress_updated, approval events), GUI-006 (tier registry drag-and-drop with hard-stop at auto_approve_below โ‰ค 50), GUI-010 (unified shell โ€” one application, role-gated surfaces). |
+
+
+### Operational Reference Terms
+
+| Term | Definition |
+|------|-----------|
+| **GitOps Store Partitioning** | Splitting a DCM GitOps repository into multiple repositories to manage scale. Three strategies: tenant-shard (hash of tenant_uuid), per-tenant (one repo per tenant), and time-based archiving (active vs cold). Declared in deployment manifest; changes require dual-write migration. |
+| **Dual-Write Mode** | Migration technique where DCM writes to both source and target store simultaneously. Required before store cutover. Duration is profile-governed (P1D minimal โ†’ P60D sovereign). |
+| **Burn-In Period** | Post-cutover period during which the source store remains accessible in read-only mode for rollback. Profile-governed: P7D minimal โ†’ P90D fsi/sovereign. Source must NOT be decommissioned before burn-in completes. |
+| **Repave** | The complete recovery scenario: all DCM infrastructure lost but Git remotes intact. DCM bootstraps from Git, restores operational stores from backup, and rehydrates managed resources. OPS-005 requires post-recovery validation checklist completion. |
+| **RTO (Recovery Time Objective)** | Maximum acceptable time to restore DCM service after failure. Profile-governed: PT1Mโ€“PT5M component failure (sovereign/standard), PT5Mโ€“PT30M full control plane loss. |
+| **RPO (Recovery Point Objective)** | Maximum acceptable data loss window. GitOps stores: 0 (Git remote is source of truth). Realized Store: PT1M (fsi/sovereign) to PT5M (standard). Audit Store: PT1M (fsi/sovereign) to PT15M (standard). |
+| **OPS-001โ€“007** | Operational Reference system policies. Key: OPS-002 (audit chain continuity across migration), OPS-003 (source read-only during burn-in; do not decommission early), OPS-006 (Audit Store retention minimum P365D all profiles). |
+
+
 ### Scheduling and Dependency Terms
 
 | Term | Definition |
@@ -327,6 +395,9 @@ Terms to avoid because they introduce ambiguity. Use the precise alternatives in
 | SCH | Scheduled Requests |
 | RDG | Request Dependency Graph |
 | HLT | DCM Self-Health |
+| OPS | Operational Reference |
+| GUI | Web Interfaces |
+| ITSM | ITSM Integration |
 
 ---
 

From f1e1cf694f83fe664c11c4d7c30dd48d66314c27 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Tue, 31 Mar 2026 09:30:33 -0400
Subject: [PATCH 47/49] Updated docs, consolidated updates into core documents.

Signed-off-by: Chris Roadfeldt 
---
 content/docs/DISCUSSION-TOPICS.md             |  34 ++
 content/docs/architecture/ai-prompt.md        | 124 +++-
 .../data-model/A-provider-contract.md         | 555 ++++++++++++++++++
 .../data-model/B-policy-contract.md           | 425 ++++++++++++++
 .../accreditation-and-authorization-matrix.md |  40 +-
 .../data-model/accreditation-monitor.md       | 496 ++++++++++++++++
 .../data-model/api-versioning-strategy.md     |  32 +-
 .../audit-provenance-observability.md         |  11 +-
 .../architecture/data-model/auth-providers.md |  15 +-
 .../data-model/authority-tier-model.md        |   8 +-
 .../data-model/consistency-review.md          | 257 ++++++++
 .../data-model/context-and-purpose.md         |  11 +-
 .../data-model/control-plane-components.md    |  11 +-
 .../data-model/credential-provider-model.md   |  13 +-
 .../architecture/data-model/dcm-federation.md |  11 +-
 .../data-model/dcm-self-health.md             |  85 ++-
 .../data-model/deployment-redundancy.md       |  15 +-
 .../data-model/design-priorities.md           |   8 +-
 .../data-model/entity-relationships.md        |  15 +-
 .../architecture/data-model/entity-types.md   |  11 +-
 .../architecture/data-model/event-catalog.md  |  78 ++-
 .../docs/architecture/data-model/examples.md  |  11 +-
 .../federated-contribution-model.md           |  11 +-
 .../architecture/data-model/foundations.md    |  11 +-
 .../architecture/data-model/four-states.md    |  11 +-
 .../data-model/governance-matrix.md           |  13 +-
 .../information-providers-advanced.md         |  11 +-
 .../data-model/information-providers.md       |  11 +-
 .../data-model/ingestion-model.md             |  89 +--
 .../data-model/internal-component-auth.md     |   6 +-
 .../data-model/itsm-integration.md            |  14 +-
 .../kessel-integration-evaluation.md          | 470 +++++++++++++++
 .../data-model/layering-and-versioning.md     |  11 +-
 .../data-model/meta-provider-model.md         |  11 +-
 .../data-model/notification-model.md          |  15 +-
 .../data-model/operational-models.md          |  13 +-
 .../data-model/operational-reference.md       |   6 +-
 .../ownership-sharing-allocation.md           |  11 +-
 .../data-model/policy-profiles.md             |  13 +-
 .../data-model/provider-callback-auth.md      | 421 +++++++++++++
 .../data-model/registry-governance.md         |  11 +-
 .../data-model/request-dependency-graph.md    |   6 +-
 .../data-model/resource-grouping.md           |  13 +-
 .../data-model/resource-service-entities.md   |  14 +-
 .../data-model/resource-type-hierarchy.md     |  35 +-
 .../data-model/scheduled-requests.md          |   6 +-
 .../architecture/data-model/scoring-model.md  |  33 +-
 .../data-model/service-dependencies.md        |  13 +-
 .../data-model/session-revocation.md          |  14 +-
 .../data-model/standards-catalog.md           |   8 +-
 .../data-model/storage-providers.md           |  13 +-
 .../data-model/universal-audit.md             |  13 +-
 .../data-model/universal-groups.md            |  29 +-
 .../data-model/webhooks-messaging.md          |  17 +-
 .../data-model/workload-analysis.md           | 253 ++++++++
 .../11-kubernetes-compatibility.md            |  31 +-
 .../specifications/cncf-strategy.md           |   5 -
 .../specifications/consumer-api-spec.md       | 162 +++--
 .../specifications/dcm-admin-api-spec.md      |  37 +-
 .../specifications/dcm-admin-gui-spec.md      |  19 +-
 .../specifications/dcm-consumer-gui-spec.md   |   5 +-
 .../specifications/dcm-examples.md            | 196 ++++++-
 .../specifications/dcm-flow-gui-spec.md       | 127 +++-
 .../dcm-opa-integration-spec.md               |   3 +
 .../dcm-operator-interface-spec.md            |   7 +-
 .../specifications/dcm-operator-sdk-api.md    |  59 ++
 .../specifications/dcm-provider-gui-spec.md   |   3 +
 .../specifications/dcm-registration-spec.md   |  73 ++-
 .../dcm-rhdh-integration-spec.md              |  40 +-
 content/docs/capabilities-matrix.md           | 100 +++-
 .../docs/data-model/00-context-and-purpose.md |   5 -
 .../docs/data-model/00-design-priorities.md   |   2 +-
 content/docs/data-model/00-foundations.md     |   5 -
 content/docs/data-model/01-entity-types.md    |   5 -
 content/docs/data-model/02-four-states.md     |   5 -
 .../data-model/03-layering-and-versioning.md  |   5 -
 content/docs/data-model/04-examples.md        |   5 -
 .../04b-ownership-sharing-allocation.md       |   5 -
 .../data-model/05-resource-type-hierarchy.md  |  29 +-
 .../06-resource-service-entities.md           |   8 +-
 .../data-model/07-service-dependencies.md     |   7 +-
 .../docs/data-model/08-resource-grouping.md   |   7 +-
 .../data-model/09-entity-relationships.md     |   9 +-
 .../data-model/10-information-providers.md    |   5 -
 .../docs/data-model/11-storage-providers.md   |   5 -
 .../12-audit-provenance-observability.md      |   5 -
 content/docs/data-model/13-ingestion-model.md |  83 +--
 content/docs/data-model/14-policy-profiles.md |   5 -
 .../docs/data-model/15-universal-groups.md    |  23 +-
 content/docs/data-model/16-universal-audit.md |   7 +-
 .../data-model/17-deployment-redundancy.md    |   5 -
 .../docs/data-model/18-webhooks-messaging.md  |  11 +-
 content/docs/data-model/19-auth-providers.md  |   5 -
 .../docs/data-model/20-registry-governance.md |   5 -
 .../21-information-providers-advanced.md      |   5 -
 content/docs/data-model/22-dcm-federation.md  |   5 -
 .../docs/data-model/23-notification-model.md  |   9 +-
 .../docs/data-model/24-operational-models.md  |   7 +-
 .../data-model/25-control-plane-components.md |   5 -
 ...-accreditation-and-authorization-matrix.md |  32 +-
 .../docs/data-model/27-governance-matrix.md   |   7 +-
 .../28-federated-contribution-model.md        |   5 -
 content/docs/data-model/29-scoring-model.md   |  27 +-
 .../docs/data-model/30-meta-provider-model.md |   5 -
 .../31-credential-provider-model.md           |   5 -
 .../data-model/32-authority-tier-model.md     |   2 +-
 content/docs/data-model/33-event-catalog.md   |  72 +++
 .../data-model/34-api-versioning-strategy.md  |  26 +-
 .../docs/data-model/35-session-revocation.md  |   6 +-
 content/docs/data-model/39-dcm-self-health.md |  79 +++
 .../docs/data-model/40-standards-catalog.md   |   2 +-
 .../docs/data-model/42-itsm-integration.md    |   8 +-
 .../data-model/43-provider-callback-auth.md   | 421 +++++++++++++
 .../44-kessel-integration-evaluation.md       | 470 +++++++++++++++
 .../docs/data-model/45-consistency-review.md  | 257 ++++++++
 .../docs/data-model/46-workload-analysis.md   | 253 ++++++++
 .../data-model/47-accreditation-monitor.md    | 496 ++++++++++++++++
 .../docs/data-model/A-provider-contract.md    |   5 -
 content/docs/data-model/B-policy-contract.md  |   5 -
 119 files changed, 6243 insertions(+), 986 deletions(-)
 create mode 100644 content/docs/architecture/data-model/A-provider-contract.md
 create mode 100644 content/docs/architecture/data-model/B-policy-contract.md
 create mode 100644 content/docs/architecture/data-model/accreditation-monitor.md
 create mode 100644 content/docs/architecture/data-model/consistency-review.md
 create mode 100644 content/docs/architecture/data-model/kessel-integration-evaluation.md
 create mode 100644 content/docs/architecture/data-model/provider-callback-auth.md
 create mode 100644 content/docs/architecture/data-model/workload-analysis.md
 create mode 100644 content/docs/data-model/43-provider-callback-auth.md
 create mode 100644 content/docs/data-model/44-kessel-integration-evaluation.md
 create mode 100644 content/docs/data-model/45-consistency-review.md
 create mode 100644 content/docs/data-model/46-workload-analysis.md
 create mode 100644 content/docs/data-model/47-accreditation-monitor.md

diff --git a/content/docs/DISCUSSION-TOPICS.md b/content/docs/DISCUSSION-TOPICS.md
index d9901e4..0126f96 100644
--- a/content/docs/DISCUSSION-TOPICS.md
+++ b/content/docs/DISCUSSION-TOPICS.md
@@ -27,6 +27,15 @@
 - Multi-cloud federation model โ€” extending DCM federation to public cloud providers
 - GitOps PR UX improvements โ€” better tooling for policy review workflow
 
+
+### 5. Kessel Integration (pre-implementation evaluation)
+- Evaluation document written: `44-kessel-integration-evaluation.md`
+- **Action required:** Discussion with Kessel development team to validate assumptions before any implementation work begins
+- Key questions: API stability, sovereign/air-gapped deployment model, SpiceDB schema extensibility, resource type registry extensibility, HA/DR patterns
+- **Do not implement until alignment confirmed** โ€” document is for discussion only
+- Two integration paths evaluated: Kessel Relations as Auth Provider (checks 1-2 of five-check model), Kessel Inventory as Discovered State Storage Provider
+- 10 blocking items identified in doc 44 Section 10
+
 ### 4. Governance Questions
 - Community governance model โ€” how will the DCM project make decisions once public?
 - Certified Profile Program โ€” self-certified vs project-reviewed for compliance profiles
@@ -541,3 +550,28 @@ Copy the following template and fill in the fields:
 ---
 
 *This document is maintained by the DCM Project team. Add topics freely โ€” no topic is too small if it needs a decision.*
+
+---
+
+### 6. Universal Lightspeed Interface for Operations (future concept)
+
+**Note captured for future exploration.**
+
+A universal operations interface concept โ€” working title "Lightspeed" โ€” for DCM. Intent is a unified, high-velocity operational surface for all DCM actions regardless of the underlying provider, resource type, or lifecycle stage. Think of it as the operational equivalent of what the unified data model is to data: a single consistent interaction model for humans and automation alike across the full DCM estate.
+
+**Initial thoughts to explore:**
+- Single interface for any operation on any resource managed by DCM โ€” no provider-specific tooling, no context switching
+- "Lightspeed" implies minimal friction: operations that currently take multiple steps, approvals, and tool handoffs should be expressible and executable in a single interaction
+- Applicable to both human operators (Web UI / CLI) and automation (agentic workflows, AIOps)
+- Likely surfaces DCM's existing policy engine, lifecycle model, and unified data model as the execution layer โ€” the interface is the innovation, not new backend capability
+- Could be the primary surface for DCM's "AI Ready" design principle (README): agentic workflows operating over DCM's control plane via a natural-language-capable interface that still enforces all policy, sovereignty, and accreditation constraints
+
+**Questions to answer when this gets scoped:**
+- Is this a new GUI surface, a CLI, an AI agent interface, or all three?
+- How does it relate to the existing Web UI spec and Flow GUI spec?
+- What does "lightspeed" mean operationally โ€” sub-second execution, zero-confirmation for pre-approved patterns, predictive pre-staging?
+- How does it interact with the Authority Tier model โ€” can it auto-route approval gates without interrupting the operator's flow?
+- Is this the primary interface for the AIOps layer referenced in the README?
+
+**Status:** Concept note โ€” no design work started. Capture for roadmap planning.
+
diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index e7f59c9..4a44aac 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -4924,7 +4924,129 @@ Full consistency review completed 2026-03. Key findings and fixes:
 **Three implementation decisions still needed:** (1) resource_type field: accept FQN string or require UUID at dispatch? (2) Operation polling: same endpoint as request status or separate? (3) API Gateway must map resource_id โ†’ entity_uuid at callback boundary.
 
 
-## SECTION 80 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 80 โ€” IMPLEMENTATION DECISIONS RESOLVED (doc updates 2026-03)
+
+Three open implementation decisions have been resolved:
+
+**DECISION 1 โ€” resource_type field: accept both FQN and UUID**
+- Consumers may supply resource_type as either FQN string (`Compute.VirtualMachine`) or Registry UUID
+- DCM resolves either form to the canonical (resource_type_uuid, resource_type_name) pair at request assembly time in the Request Payload Processor
+- FQN recommended for consumer use (stable across deployments; returned by service catalog)
+- UUID accepted for programmatic use where UUID was obtained from catalog API
+- Unresolvable references rejected at validation time: 422 + code RESOURCE_TYPE_NOT_FOUND
+- Dispatch payloads (CreateRequest/UpdateRequest) to operators ALWAYS carry both: resource_type_uuid + resource_type_name
+- Added to: dcm-common.json (resource_type_ref oneOf type), CatalogItem schema (oneOf in consumer YAML), doc 05, consumer spec, operator spec
+
+**DECISION 2 โ€” operation.name โ†’ /api/v1/operations/{uuid} (separate AEP endpoint)**
+- operation_uuid == request_uuid โ€” the same UUID serves both endpoints
+- GET /api/v1/operations/{uuid} โ†’ Operation schema (AEP-standard: done, metadata, response/error)
+- GET /api/v1/requests/{uuid}/status โ†’ RequestStatus schema (DCM-native: pipeline_stage, full history)
+- Both endpoints reflect the same underlying operation state
+- Operation.metadata includes request_uuid field so AEP clients can navigate to the rich view if needed
+- POST /api/v1/requests returns Operation with name = /api/v1/operations/{request_uuid}
+- Added to: consumer YAML (new GET /api/v1/operations/{operation_uuid} path, 61 paths total), Operation schema (request_uuid in metadata, dual-endpoint description), consumer spec (Operations polling section with dual-endpoint table)
+
+**DECISION 3 โ€” resource_id โ†’ entity_uuid mapping (RESOLVED by existing data model)**
+- Not an open decision โ€” already solved by the schema
+- DCM sends dcm_entity_uuid in every CreateRequest dispatch; operator echoes it in every response and callback
+- DCM uses entity_uuid for all routing; resource_id is operator's own correlation handle stored opaquely
+- API Gateway validates dcm_entity_uuid in each callback matches the entity under the calling provider's credential
+- Documented explicitly in operator interface spec
+
+
+## SECTION 81 โ€” SPEC COMPLETION: K8S + OPERATOR SDK (2026-03)
+
+**11-kubernetes-compatibility.md** โ€” Completed:
+- Document status header and related docs cross-references added
+- Section 2 intro: explains the superset relationship explicitly
+- Section 4 summary table: 7 DCM capabilities vs Kubernetes gaps, side-by-side
+- Comment lines cleaned from section 3a (code comment artifacts removed)
+- AEP alignment note added
+- All 5 open questions were already resolved; Resolution Notes complete
+- Status: โœ… Complete
+
+**dcm-operator-sdk-api.md** โ€” Completed:
+- Document status header, related docs, AEP alignment note added
+- New Section 11: Callback Credential Management โ€” automatic rotation, entity_uuid vs resource_id contract, DCM validation rule
+- resource_type_name field annotated: "FQN โ€” always present alongside resource_type_uuid"
+- All 5 open questions were already resolved; Q10 header annotated
+- Status: โœ… Complete
+
+**All 14 specifications now have:** AEP alignment notes, document status headers, no stale slash-verb paths, consistent field naming.
+
+
+## SECTION 82 โ€” PDF VALIDATION + THREE NEW ADDITIONS (2026-03)
+
+**PDF (Miro board) was validated against current architecture.** Core finding: PDF = original design intent; current docs = evolved, more detailed implementation. Structurally aligned. Three gaps identified and filled:
+
+**GAP 1 โ€” Static Replace use case (dcm-examples.md sections 1.9 + 1.10):**
+- Static Replace = re-provision using existing Requested State verbatim, no layer enrichment, no policy re-evaluation. Deterministic rebuild.
+- Distinct from Rehydration (mode: intent) which replays original Intent State through current policies and layers.
+- `POST /api/v1/resources/{uuid}:rehydrate` with `mode: static` (same endpoint, different mode).
+- Precondition: application data on separate partition (VM OS/config is what gets rebuilt).
+- Orchestration Flow Policy specified (4-step: validate โ†’ decommission โ†’ dispatch original requested state โ†’ restore operational).
+- In-Place Upgrade (Leapp/IPU) also documented as Section 1.10 โ€” upgrades OS in-place, preserves entity UUID, creates new Realized State delta record.
+
+**GAP 2 โ€” Workload Analysis (new doc 46-workload-analysis.md):**
+- New capability: actively classifies discovered resources by operational characteristics.
+- Answers: "what is this resource?", "is it migratable?", "what lifecycle model applies?"
+- WorkloadProfile = process_resource_entity of type Analysis.WorkloadProfile.
+- Fires automatically as part of brownfield ingestion pipeline (DRC โ†’ WLA โ†’ Ingestion).
+- MTA (Migration Toolkit for Applications) is the reference Information Provider implementation.
+- 6 system policies WLA-001 through WLA-006.
+- API: GET /api/v1/resources/{uuid}/workload-profile, POST /api/v1/resources/{uuid}/workload-profile:analyze
+- Added to Capabilities Matrix as Domain 36 (6 capabilities: WLA-001 to WLA-006).
+
+**GAP 3 โ€” Per-provider monitoring contract (registration spec + self-health doc):**
+- 7 Provider Readiness Gates (GATE-SP-01 through GATE-SP-07) added to registration spec Section 7.2.
+- Gates 1-3 required for all profiles; Gates 4-7 required for standard+.
+- GATE-SP-01: OpenAPI spec declared and reachable.
+- GATE-SP-02: Healthy API at activation.
+- GATE-SP-03: State Management callback implemented.
+- GATE-SP-04: Tenant Metadata endpoint (usage by tenant, quota consumed).
+- GATE-SP-05: Prometheus metrics (4 required families: dispatches_total, dispatch_duration_seconds, realizations_total, health_status).
+- GATE-SP-06: AEP.DEV linting passes (no errors in provider OpenAPI spec).
+- GATE-SP-07: Multi-tenant dispatch (accepts tenant_uuid).
+- Self-health doc (39-dcm-self-health.md) Section 7: Per-Provider Metrics Contract added.
+- Added to Capabilities Matrix as PRR-001 through PRR-007 in Provider Contract domain.
+- Matrix now: 37 domains / 287 capabilities.
+
+**PDF naming differences documented (not errors โ€” terminology evolution):**
+Widget โ†’ Resource/Entity; Requested/Realized/Discovered Widget Store โ†’ State Stores;
+Widget Discovery โ†’ Discovery Scheduler; Interoperability API โ†’ Operator Interface;
+Provisioned Store โ†’ Realized State Store; Job Queue โ†’ Message Bus; Rules Engine โ†’ Policy Engine.
+
+
+## SECTION 83 โ€” ACCREDITATION MONITOR (doc 47 โ€” 47-accreditation-monitor.md)
+
+**Purpose:** Continuously verifies registered accreditations against authoritative external sources. Answers: "Is this accreditation still valid according to the issuing authority โ€” not just the expiry date we were told?"
+
+**Four verification tiers (by automation depth):**
+- **Tier 1 โ€” External Registry API (full automation):** FedRAMP (marketplace.fedramp.gov/api), CMMC 2.0 (cyberab.org/catalog), StateRAMP, ISO 27001 (iaf.nu CertSearch). Queries by external_registry_id; detects status changes including mid-cycle revocations.
+- **Tier 2 โ€” Document Currency (partial automation):** SOC 2, PCI DSS AoC. Fetches document from certificate_ref/audit_report_ref, extracts date via PDF metadata or header parsing, validates against max_age threshold (default P365D).
+- **Tier 3 โ€” Contract Webhook (event-driven):** HIPAA BAA, DoD IL. Inbound webhook from DocuSign/Ironclad/etc fires when BAA is signed, amended, or terminated. DCM processes: signedโ†’active, amendedโ†’pending_review, terminatedโ†’revoked.
+- **Tier 4 โ€” Expiry-Only (no external check):** Self-declared, internal, HIPAA BAA without contract system. Monitors declared valid_until only.
+
+**Key flows:**
+- Tier 1 poll detects Revoked status โ†’ immediate accreditation revocation โ†’ Accreditation Gap triggered (no admin confirmation required for revocations)
+- Tier 1 poll detects status change to non-revoked (e.g., Authorized โ†’ In Process) โ†’ status: pending_review โ†’ admin confirms
+- Registry unreachable โ†’ increment failure_count; no status change; fires verification_stale at threshold
+- stale_after exceeded โ†’ stale_action: warn/suspend/escalate (profile-governed: warn for dev/standard, suspend for prod, escalate for fsi/sovereign)
+
+**Accreditation record additions (doc 26 Section 3.3):** `verification` block with tier, registry_api/document_check/contract_webhook sub-blocks, stale_after, stale_action, verification_failure_count. `status` gains `pending_review` state. `gap_type` gains `verification_stale`.
+
+**Scoring Model addition (doc 29 Signal 5):** `verification_multipliers` โ€” accreditation weight discounted based on verification currency: external_registry verified today = 1.0, expiry_only = 0.7, stale = 0.4, failed threshold = 0.1.
+
+**New events (doc 33 Section 20):** accreditation.verified, accreditation.status_changed, accreditation.registry_mismatch, accreditation.verification_stale, accreditation.document_expired, accreditation.contract_event, accreditation.expiry_approaching.
+
+**8 system policies: ACM-001 through ACM-008.** Key: ACM-002 (status change โ†’ pending_review except Revoked which is immediate), ACM-003 (registry unreachable does NOT revoke), ACM-004 (fsi/sovereign must use tier โ‰ฅ document_currency), ACM-007 (all verifications produce audit records โ€” no silent checks).
+
+**Air-gapped mode:** Tiers fall back to expiry_only for unreachable registries. Retries after air_gapped_retry_interval (default P30D). Manual update of last_verified_at permitted with required justification.
+
+**Matrix:** Domain 37, ACM-001 through ACM-007. Total: 37 domains / 287 capabilities.
+
+
+## SECTION 84 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure):
 
diff --git a/content/docs/architecture/data-model/A-provider-contract.md b/content/docs/architecture/data-model/A-provider-contract.md
new file mode 100644
index 0000000..b781576
--- /dev/null
+++ b/content/docs/architecture/data-model/A-provider-contract.md
@@ -0,0 +1,555 @@
+# DCM โ€” Unified Provider Contract
+
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Foundation
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Policy Contract](B-policy-contract.md) | [Registration Specification](../specifications/dcm-registration-spec.md) | [Governance Matrix](27-governance-matrix.md) | [Accreditation](26-accreditation-and-authorization-matrix.md)
+
+---
+
+> > **Design Priority:** Provider types implement all four design priorities simultaneously. Security properties (mTLS, scoped credentials, sovereignty declarations, accreditation) are present in all provider registrations. The capability extension model (Priority 3) enables new provider types without changing the base contract. See [Design Priorities](00-design-priorities.md).
+
+## 1. The Unified Provider Contract
+
+Every Provider in DCM โ€” regardless of type โ€” implements a single base contract. What varies between provider types is the **capability extension**: the specific operations exposed, the data that flows in each direction, and the typed schemas for that exchange.
+
+```
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚                BASE PROVIDER CONTRACT                    โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Registration ยท Health ยท Sovereignty ยท Accreditation    โ”‚
+โ”‚  Governance Matrix ยท Zero Trust ยท Lifecycle              โ”‚
+โ”‚                                                          โ”‚
+โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”   โ”‚
+โ”‚  โ”‚           CAPABILITY EXTENSION                   โ”‚   โ”‚
+โ”‚  โ”‚                                                  โ”‚   โ”‚
+โ”‚  โ”‚  What operations this provider type exposes.     โ”‚   โ”‚
+โ”‚  โ”‚  What data flows in which direction.             โ”‚   โ”‚
+โ”‚  โ”‚  What schemas govern the exchange.               โ”‚   โ”‚
+โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜   โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+```
+
+**Adding a new provider type** = implement the base contract + define a capability extension. No changes to the core required.
+
+---
+
+## 2. Base Contract โ€” Registration
+
+All providers register through the same pipeline. See [Registration Specification](../specifications/dcm-registration-spec.md) for the complete flow.
+
+```yaml
+provider_base_registration:
+  # Standard artifact metadata
+  artifact_metadata:
+    uuid: 
+    handle: "//"    # e.g., "org/compute/eu-west-prod-1"
+    version: "1.0.0"
+    status: submitted                      # submitted โ†’ validating โ†’ active
+    owned_by: { display_name: "" }
+
+  provider_type_id:                # from Provider Type Registry
+  display_name: ""
+  description: ""
+
+  # All providers declare these
+  sovereignty_declaration:
+    operating_jurisdictions: []
+    data_residency_zones: []
+    sub_processors: []                   # third parties with data access
+
+  accreditations:
+    - accreditation_uuid:          # reference to registered accreditation
+      framework: 
+      status: active
+
+  # Endpoints (which endpoints are required varies by type โ€” see extensions)
+  health_endpoint: "https:///health"
+
+  # Zero trust identity
+  certificate:
+    pem: 
+    ca_chain: 
+    rotation_interval: P90D
+```
+
+**Registration lifecycle states:**
+```
+SUBMITTED โ†’ VALIDATING โ†’ PENDING_APPROVAL โ†’ ACTIVE
+                       โ†˜ REJECTED
+ACTIVE โ†’ SUSPENDED | DEREGISTERING โ†’ DEREGISTERED | FORCED_DEREGISTERED
+```
+
+---
+
+## 3. Base Contract โ€” Health Check
+
+Every provider implements a health endpoint. DCM calls it on the declared interval.
+
+```
+GET {health_endpoint}
+
+Response 200:
+{
+  "status": "healthy | degraded | unhealthy",
+  "version": "",
+  "capabilities_available": [""],
+  "details": { }    # provider-specific; DCM treats as opaque
+}
+```
+
+**DCM response to health states:**
+- `healthy` โ†’ normal operations; next poll scheduled
+- `degraded` โ†’ reduced routing preference; platform admin notified (medium urgency)
+- `unhealthy` / no response โ†’ after `failure_threshold`: status โ†’ DEGRADED; new requests not routed
+- After 2ร— `failure_threshold`: status โ†’ UNAVAILABLE; drift detection triggered on all hosted entities
+
+---
+
+## 4. Base Contract โ€” Governance Matrix Enforcement
+
+Every interaction with every provider is evaluated against the Governance Matrix before data crosses the boundary. This is not optional and not configurable per provider โ€” it is a base contract requirement.
+
+```
+Outbound interaction (DCM โ†’ Provider):
+  1. Classify all fields in the payload by data_classification
+  2. Resolve provider's active accreditations
+  3. Evaluate Governance Matrix: permitted | strip_field | deny | redact
+  4. Apply field permissions
+  5. Audit record written (regardless of outcome)
+  6. If DENY: interaction blocked; entity enters PENDING_REVIEW if appropriate
+
+Inbound interaction (Provider โ†’ DCM):
+  1. Authenticate provider identity (mTLS)
+  2. Verify credential scope matches the operation
+  3. Accept payload; apply data_classification tags
+  4. Store in appropriate store per data_classification
+```
+
+---
+
+## 5. Base Contract โ€” Zero Trust
+
+All provider interactions operate under the active zero trust posture. Minimum requirement for all providers at all profiles:
+
+- Mutual TLS authentication on every call (both sides present certificates)
+- Scoped, short-lived interaction credentials (not long-lived API keys)
+- Every call authenticated; no implicit trust from network position or prior calls
+- Certificate rotation on declared interval
+
+Higher profiles add: certificate pinning, per-message signing, hardware attestation.
+
+---
+
+## 6. Base Contract โ€” Provider Lifecycle Events
+
+Providers must report state changes via lifecycle events. This is a base contract obligation โ€” not optional:
+
+```json
+POST {dcm_lifecycle_endpoint}
+{
+  "event_uuid": "",
+  "event_type": "",
+  "provider_uuid": "",
+  "affected_entity_uuids": [""],
+  "event_timestamp": "",
+  "severity": "INFO | WARNING | CRITICAL"
+}
+```
+
+---
+
+## 7. Capability Extensions โ€” All Eleven Types
+
+### 7.1 Service Provider
+
+**What it does:** Realizes infrastructure resources. Receives assembled payloads, provisions the resource, returns realized state.
+
+**Additional endpoints:**
+```
+POST {dispatch_endpoint}         # receive and execute dispatch payload
+POST {cancel_endpoint}           # receive cancellation request (if supported)
+POST {discover_endpoint}         # receive discovery request; return discovered state
+```
+
+**Capability declaration extension:**
+```yaml
+service_provider_capabilities:
+  resource_types:
+    - fqn: Compute.VirtualMachine
+      spec_version: "2.1.0"
+      catalog_item_uuid: 
+  cancellation:
+    supports_cancellation: true
+    cancellation_supported_during: [DISPATCHED, PROVISIONING]
+  discovery:
+    supports_discovery: true
+    discovery_method: api_query | passive_event | hybrid
+  naturalization:
+    target_format: openstack_nova | vmware_vsphere | custom
+  cost_metadata:
+    opex_per_unit_per_hour: 0.28
+    currency: USD
+```
+
+**Data direction:** DCM sends assembled Requested State โ†’ Provider naturalizes โ†’ executes โ†’ denaturalizes โ†’ returns Realized State. DCM writes Realized State to Snapshot Store.
+
+---
+
+### 7.2 Information Provider
+
+**What it does:** Serves authoritative external data to enrich DCM's understanding of resources and business context.
+
+**Additional endpoints:**
+```
+POST {query_endpoint}            # receive query; return data in DCM unified format
+POST {write_back_endpoint}       # optional; receive DCM updates to push to source system
+```
+
+**Capability declaration extension:**
+```yaml
+information_provider_capabilities:
+  data_domains:
+    - domain: business_data
+      data_types: [business_unit, cost_center, product_owner]
+      authority_level: primary | secondary | supplementary
+  query_capacity:
+    max_queries_per_second: 100
+  confidence_model:
+    data_freshness_sla: PT1H
+  write_back_supported: false
+```
+
+**Data direction:** DCM sends lookup query โ†’ Provider returns data in DCM format โ†’ DCM enriches entity fields.
+
+---
+
+### 7.3 Storage Provider
+
+**What it does:** Persists DCM state data. Implements one or more store contracts (GitOps, write-once snapshot, event stream, search index, audit).
+
+**Additional endpoints:**
+```
+POST {write_endpoint}            # receive and persist data
+GET  {read_endpoint}             # return stored data
+POST {query_endpoint}            # execute indexed query (search index sub-type)
+GET  {health_endpoint}           # includes store-specific metrics
+```
+
+**Capability declaration extension:**
+```yaml
+storage_provider_capabilities:
+  store_types:
+    - gitops               # Intent and Requested stores
+    - write_once_snapshot  # Realized store
+    - event_stream         # Discovered store
+    - search_index         # Query projection
+    - audit                # Audit store
+  consistency: strong | eventual | bounded_staleness
+  geo_replicated: true
+  encryption_at_rest: AES-256
+  hsm_backed: false
+```
+
+**Data direction:** Bidirectional. DCM writes state; DCM reads state. Provider never initiates.
+
+---
+
+### 7.4 Meta Provider
+
+**What it does:** Composes multiple child providers to deliver a compound service as a single catalog item. The Meta Provider declares a compound service definition โ€” constituent resource types, dependencies, and delivery requirements โ€” so DCM can place, sequence, and govern the constituents. For its own resource types (`provided_by: self`), the Meta Provider executes as a standard Service Provider. All orchestration, placement, sequencing, failure handling, and compensation is performed by DCM using the declared dependency graph.
+
+> **Full specification:** See [Meta Provider Composability Model](30-meta-provider-model.md) for the complete orchestration contract, four-state model, failure propagation, compensation, and system policies (MPX-001โ€“MPX-008).
+
+**Capability declaration extension (summary โ€” full schema in doc 30):**
+```yaml
+meta_provider_capabilities:
+  constituent_provider_types: [service_provider, information_provider, meta_provider]
+  composition_model:
+    execution: dependency_ordered    # sequential | parallel | dependency_ordered
+    max_concurrent_realizations: 10
+    max_constituent_count: 20
+    max_nesting_depth: 3
+  partial_delivery_supported: true
+  compensation_supported: true      # required if partial_delivery_supported: true (MPX-001)
+  compensation_timeout: PT30M
+  idempotency_guaranteed: true
+  status_reporting:
+    supported: true
+    interval: PT30S
+  resource_types_composed:
+    - fqn: ApplicationStack.WebApp
+      version: "2.0.0"
+      constituents:
+        - resource_type: Compute.VirtualMachine
+          required_for_delivery: required
+        - resource_type: Network.IPAddress
+          required_for_delivery: required
+        - resource_type: DNS.Record
+          required_for_delivery: partial
+      composition_visibility: selective   # opaque | transparent | selective
+```
+
+**Composite status determination:**
+- `REALIZED` โ€” all required constituents succeeded
+- `DEGRADED` โ€” required constituents succeeded; one or more partial constituents failed (accepted if `partial_delivery_supported: true`)
+- `FAILED` โ€” one or more required constituents failed โ†’ compensation executes
+
+**Data direction:** DCM sends fully assembled compound payload โ†’ Meta Provider orchestrates constituents in dependency order โ†’ aggregates realized states โ†’ returns compound realized state to DCM.
+
+---
+
+### 7.5 Policy Provider
+
+**What it does:** Evaluates policies externally. Receives a DCM payload, evaluates Rego or custom logic, returns a typed policy decision.
+
+**Additional endpoints:**
+```
+POST {evaluate_endpoint}         # receive payload; return policy decision
+POST {test_endpoint}             # receive test case; return evaluation result (shadow mode)
+```
+
+**Capability declaration extension:**
+```yaml
+policy_provider_capabilities:
+  mode: 1 | 2 | 3 | 4
+  policy_types: [gatekeeper, validation, transformation, recovery]
+  framework: opa | cedar | custom
+  shadow_mode_supported: true
+  endpoint_sovereignty_zone:    # required for Mode 4
+```
+
+**Data direction:** DCM sends payload + active policy bundle โ†’ Provider evaluates โ†’ returns typed decision (allow/deny, mutations, action).
+
+---
+
+### 7.6 Credential Provider
+
+**What it does:** Issues, rotates, and revokes credentials used within the DCM ecosystem โ€” both DCM interaction credentials (short-lived, scoped, used for provider dispatch under the Zero Trust model) and consumer-facing resource credentials (SSH keys, API keys, kubeconfigs, service account tokens, database passwords, x509 certificates).
+
+> **Full specification:** See [Credential Provider Model](31-credential-provider-model.md) for the complete issuance contract, rotation protocol, revocation propagation, consumer delivery, and system policies (CPX-001โ€“CPX-008).
+
+**Credential values are never stored in DCM** โ€” only credential metadata (UUID, type, scope, expiry, status) is stored. Values are held by the Credential Provider and retrieved by authorized consumers via a declared `value_retrieval_endpoint`.
+
+**Additional endpoints:**
+```
+POST   {issue_endpoint}              # issue credential; return metadata + retrieval URL
+POST   {rotate_endpoint}             # rotate; return old/new UUIDs + transition window
+DELETE {revoke_endpoint}/{uuid}      # revoke immediately or at transition window end
+POST   {validate_endpoint}           # use-time validity check (scope, revocation, expiry)
+GET    {list_endpoint}               # list credentials by entity_uuid or issued_to
+```
+
+**Capability declaration extension (summary โ€” full schema in doc 31):**
+```yaml
+credential_provider_capabilities:
+  credential_types:
+    - api_key
+    - x509_certificate
+    - ssh_key
+    - service_account_token
+    - database_password
+    - kubeconfig
+    - hsm_backed_key
+    - dcm_interaction          # required if handling DCM interaction credentials
+  hsm_backed: false
+  fips_140_2_level: 0 | 1 | 2 | 3   # enforced per profile (Section 12)
+  dynamic_secrets: true
+  rotation_support: true
+  revocation_sla: PT5M         # profile-governed; PT30S for sovereign
+  approved_algorithms:         # declare which algorithms the provider supports
+    ssh_key: [Ed25519, ECDSA-P-384, RSA-4096]
+    x509_certificate: [Ed25519, ECDSA-P-384, RSA-4096]
+    service_account_token: [RS256, ES256, HS256]
+    # ... per credential type
+  key_escrow:
+    supported: false           # true only for regulated sovereign deployments
+```
+
+**Data direction:** DCM requests credential โ†’ Provider issues scoped credential + returns metadata โ†’ DCM stores metadata, includes retrieval URL in realized entity โ†’ Consumer retrieves value via authenticated endpoint. Revocation: DCM requests revocation โ†’ Provider invalidates โ†’ DCM publishes revocation event to Message Bus โ†’ all components refresh revocation cache within profile-governed TTL.
+
+---
+
+### 7.7 Auth Provider
+
+**What it does:** Authenticates actor identities and resolves their roles and group memberships.
+
+**Additional endpoints:**
+```
+POST {authenticate_endpoint}     # receive credentials; return auth token + claims
+POST {authorize_endpoint}        # receive token + operation; return allow/deny
+GET  {identity_endpoint}         # return actor claims for a token
+```
+
+**Capability declaration extension:**
+```yaml
+auth_provider_capabilities:
+  authentication_modes: [oidc, ldap, saml, mtls, hardware_token]
+  mfa_methods: [totp, push_notification, hardware_token]
+  rbac_model: flat | hierarchical | abac
+  step_up_supported: true
+  token_lifetime:
+    default: PT1H
+    max: PT8H
+```
+
+**Data direction:** Consumer sends credentials โ†’ Auth Provider validates โ†’ returns token + claims โ†’ DCM extracts actor identity.
+
+---
+
+### 7.8 Notification Provider
+
+**What it does:** Receives unified notification envelopes from DCM and delivers them via configured channels.
+
+**Additional endpoints:**
+```
+POST {delivery_endpoint}         # receive notification envelope; deliver to channel
+POST {delivery_status_endpoint}  # callback: report delivery status to DCM
+```
+
+**Capability declaration extension:**
+```yaml
+notification_provider_capabilities:
+  delivery_channels:
+    - channel_type: slack | pagerduty | email | webhook | sms | servicenow
+      supports_urgency_routing: true
+      config_schema_ref: 
+  delivery_guarantees:
+    at_least_once: true
+    idempotency_key: notification_uuid
+    max_latency_seconds: 30
+  sovereignty_aware_delivery: true
+```
+
+**Data direction:** DCM sends notification envelope โ†’ Provider translates to channel format โ†’ delivers โ†’ reports status.
+
+---
+
+### 7.9 Message Bus Provider
+
+**What it does:** Provides persistent, high-throughput asynchronous event streaming between DCM components and external systems.
+
+**Capability declaration extension:**
+```yaml
+message_bus_capabilities:
+  protocols: [kafka, amqp, mqtt]
+  persistence: true
+  durability: at_least_once | exactly_once
+  external_endpoints: false          # true only if messages leave sovereignty boundary
+  topics:
+    - name: dcm.events
+      retention: P7D
+```
+
+**Data direction:** Bidirectional publish/subscribe. DCM publishes events; components and external systems subscribe.
+
+---
+
+### 7.10 Registry Provider
+
+**What it does:** Serves the Resource Type Registry โ€” the authoritative catalog of resource types available to DCM deployments.
+
+**Additional endpoints:**
+```
+GET  {registry_endpoint}         # serve registry entries (full or incremental)
+GET  {bundle_endpoint}           # serve signed registry bundle (air-gapped mode)
+```
+
+**Capability declaration extension:**
+```yaml
+registry_provider_capabilities:
+  serves_tiers: [core, verified_community, organization]
+  incremental_sync: true
+  signed_bundles: true           # for air-gapped deployments
+  bundle_signing_key_ref: 
+```
+
+**Data direction:** DCM pulls registry entries โ†’ Provider returns signed bundle or live entries.
+
+---
+
+### 7.11 Peer DCM (Federation)
+
+**What it does:** Another DCM instance participating in federation. Treated as a typed Provider with a federation tunnel as the communication channel.
+
+**Capability declaration extension:**
+```yaml
+peer_dcm_capabilities:
+  dcm_version: "1.0.0"
+  tunnel_type: peer | parent_child | hub_spoke
+  deployment_accreditations: []
+  inbound_authorization:          # what this peer may request from local DCM
+    - operation: catalog_query
+      resource_types: [Compute.VirtualMachine]
+  outbound_authorization:         # what local DCM may request from this peer
+    - operation: placement_query
+      resource_types: [Compute.VirtualMachine]
+  data_boundary:
+    max_classification: restricted
+  trust_posture: verified | vouched | provisional
+```
+
+**Data direction:** Bidirectional within declared authorization scope. Federation tunnel with mTLS, certificate pinning, per-message signing.
+
+---
+
+## 8. Provider Type Registry
+
+The Provider Type Registry is the authoritative list of provider types that a DCM deployment accepts registrations for. It follows the three-tier registry model (Core / Verified Community / Organization).
+
+```yaml
+provider_type_registry_entry:
+  provider_type_id: service_provider
+  tier: core
+  default_approval_method: reviewed   # auto | reviewed | verified | authorized
+  enabled_in_profiles: [minimal, dev, standard, prod, fsi, sovereign]
+  capability_extension_schema_ref: 
+```
+
+Profile-governed approval methods override provider type defaults. See [Registration Specification](../specifications/dcm-registration-spec.md) Section 3 for the complete approval method resolution model.
+
+---
+
+## 9. Related Policies
+
+| Policy | Rule |
+|--------|------|
+| `PRV-001` | All providers implement the base contract. No provider is exempt from registration, health check, sovereignty declaration, governance matrix enforcement, or zero trust authentication. |
+| `PRV-002` | Governance Matrix evaluation occurs before every provider interaction. It is not configurable per provider and cannot be bypassed. |
+| `PRV-003` | Provider capability declarations are verified at registration. Capabilities not declared at registration cannot be invoked after activation. |
+| `PRV-004` | Peer DCM instances are treated as typed providers. Federation is the Provider abstraction applied across DCM instances โ€” not a separate abstraction. |
+| `PRV-005` | Adding a new provider type requires implementing the base contract and defining a capability extension. No changes to DCM core are required. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+## ITSM Provider
+
+**What it does:** Provides bidirectional integration with external ITSM systems. Outbound: receives DCM lifecycle event data and creates/updates records in the ITSM system (change requests, incidents, CMDB CIs). Inbound: routes ITSM approval decisions back to DCM as approval votes, and accepts ITSM-initiated request submissions.
+
+**Additional endpoints (ITSM Provider implements):**
+```
+POST {provider_base}/actions              # receive action request from DCM
+GET  {provider_base}/actions/{action_id}  # DCM checks action completion status
+GET  {provider_base}/records/{record_id}  # DCM retrieves record status from ITSM
+POST {provider_base}/inbound             # ITSM system sends inbound approval/event
+```
+
+**Capability declaration extension:**
+```yaml
+itsm_provider_capabilities:
+  itsm_system: servicenow | jira_service_management | bmc_remedy | bmc_helix |
+               freshservice | zendesk | pagerduty | opsgenie | manageengine |
+               cherwell | topdesk | generic_rest
+  supported_actions: []
+  endpoint_url: 
+  auth_credential_uuid: 
+  inbound_webhook:
+    enabled: 
+    secret_credential_uuid: 
+  field_mapping_ref: 
+  cmdb_ci_type_map: []
+```
+
+> **See [ITSM Integration](42-itsm-integration.md)** for complete capability declaration schemas, supported systems, and system policies ITSM-001โ€“007.
diff --git a/content/docs/architecture/data-model/B-policy-contract.md b/content/docs/architecture/data-model/B-policy-contract.md
new file mode 100644
index 0000000..83276b7
--- /dev/null
+++ b/content/docs/architecture/data-model/B-policy-contract.md
@@ -0,0 +1,425 @@
+# DCM โ€” Unified Policy Contract
+
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Foundation
+**Related Documents:** [Foundational Abstractions](00-foundations.md) | [Provider Contract](A-provider-contract.md) | [Policy Profiles](14-policy-profiles.md) | [Governance Matrix](27-governance-matrix.md) | [OPA Integration](../specifications/dcm-opa-integration-spec.md)
+
+---
+
+## 1. The Unified Policy Contract
+
+Every Policy in DCM โ€” regardless of type โ€” implements a single base contract. What varies between policy types is the **output schema**: what the Policy produces when its match conditions are satisfied.
+
+```
+โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
+โ”‚                 BASE POLICY CONTRACT                     โ”‚
+โ”‚                                                          โ”‚
+โ”‚  Match Conditions ยท Enforcement Level ยท Domain          โ”‚
+โ”‚  Lifecycle ยท Audit ยท Shadow Mode                         โ”‚
+โ”‚                                                          โ”‚
+โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”   โ”‚
+โ”‚  โ”‚              OUTPUT SCHEMA                       โ”‚   โ”‚
+โ”‚  โ”‚                                                  โ”‚   โ”‚
+โ”‚  โ”‚  What this policy type produces when it fires.   โ”‚   โ”‚
+โ”‚  โ”‚  Seven typed output schemas.                     โ”‚   โ”‚
+โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜   โ”‚
+โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
+```
+
+**Adding a new policy type** = define a new output schema. The base contract, evaluation algorithm, lifecycle, and audit obligations are inherited.
+
+---
+
+## 2. Base Contract โ€” Match Conditions
+
+All policies declare when they fire using one or both of two match condition models:
+
+**Model A โ€” Payload type + field conditions** (for pipeline policies: GateKeeper, Validation, Transformation, Recovery, Orchestration Flow):
+
+```yaml
+match:
+  payload_type: request.initiated | request.layers_assembled | ...   # from closed vocabulary
+  conditions:
+    - field:                 # dot-notation path into the payload
+      operator: equals | in | minimum | maximum | contains | matches
+      value: 
+    - field: 
+      operator: not_equals
+      value: 
+  condition_logic: all | any             # default: all
+```
+
+**Model B โ€” Four-axis boundary conditions** (for boundary policies: Governance Matrix Rules):
+
+```yaml
+match:
+  subject:
+    type: 
+    identity: { ... }
+    tenant: { ... }
+  data:
+    classification: 
+    resource_type: 
+    field_paths: { mode: allowlist | blocklist, paths: [...] }
+    capability: 
+  target:
+    type: 
+    sovereignty_zone: { match:  }
+    accreditation_held: { includes: [...] }
+    trust_posture: 
+  context:
+    profile: { deployment_posture:  }
+    zero_trust_posture: { minimum:  }
+    federated: true | false
+```
+
+Policies may declare match conditions using either model. Orchestration Flow policies and Lifecycle Policies may also use relationship event conditions (see Section 9).
+
+---
+
+## 3. Base Contract โ€” Enforcement Level
+
+```yaml
+enforcement: hard | soft
+
+# hard: cannot be relaxed by any downstream rule at any domain level
+#        A hard DENY cannot be overridden by any Tenant, entity, or operator override
+#        Reserved for: sovereign/classified data boundaries, regulatory hard requirements
+
+# soft: establishes a default that downstream rules can tighten
+#        A soft ALLOW can be restricted to DENY by a more-specific rule
+#        A soft DENY cannot be relaxed to ALLOW by a downstream rule
+```
+
+Most policies are soft. Hard enforcement is reserved for absolute security constraints.
+
+---
+
+## 4. Base Contract โ€” Domain Precedence
+
+Policies operate within a domain hierarchy. More-specific domains win within the same concern type:
+
+```
+system (most trusted โ€” DCM built-in)
+  โ””โ”€โ”€ platform (platform admin declared)
+        โ””โ”€โ”€ tenant (Tenant admin declared)
+              โ””โ”€โ”€ resource_type (per resource type spec)
+                    โ””โ”€โ”€ entity (per specific entity โ€” most specific)
+```
+
+Within the same domain level, DENY wins over ALLOW. More-specific domain wins over less-specific.
+
+---
+
+## 5. Base Contract โ€” Artifact Structure
+
+All policies are first-class DCM Data artifacts. They share the standard artifact metadata and lifecycle:
+
+```yaml
+policy_artifact:
+  # Standard DCM artifact metadata (all artifacts carry this)
+  artifact_metadata:
+    uuid: 
+    handle: "//"
+    version: "1.0.0"
+    status: developing | proposed | active | deprecated | retired
+    owned_by: { display_name: "", email: "" }
+    created_by: { display_name: "" }
+    created_via: pr | api | migration | system
+
+  # Policy classification
+  policy_type:                     # gatekeeper | validation | transformation |
+                                         # recovery | orchestration_flow |
+                                         # governance_matrix_rule | lifecycle
+  concern_type:                 # security | compliance | operational |
+                                         # recovery_posture | zero_trust_posture |
+                                         # data_authorization_boundary | orchestration_flow
+
+  domain: system | platform | tenant | resource_type | entity
+
+  # Match conditions (Model A or B โ€” see Section 2)
+  match: { ... }
+
+  # Enforcement
+  enforcement: hard | soft
+
+  # Output schema (varies by policy_type โ€” see Sections 8-14)
+  output: { ... }
+
+  # Audit
+  audit_on: [ALLOW, DENY, STRIP_FIELD]   # which decisions produce audit records
+  notification_on: [DENY]               # which decisions trigger notifications
+  notification_urgency: low | medium | high | critical
+
+  # Compliance reference
+  compliance_basis: ""
+  review_required_before: ""
+```
+
+---
+
+## 6. Base Contract โ€” Lifecycle
+
+All policies follow the five-status lifecycle:
+
+| Status | Behavior |
+|--------|---------|
+| `developing` | Dev mode only. Not applied in any environment. |
+| `proposed` | Shadow mode: executes against real traffic; output captured but never applied. Used for safe validation. |
+| `active` | Applied to all matching requests. |
+| `deprecated` | Still active; replacement available; warning on evaluation. |
+| `retired` | Terminal; cannot be used. |
+
+**Shadow mode (proposed status):** The policy evaluates against real traffic. Its output is captured in the Validation Store. Platform admins review shadow results before promoting to active. This is the primary mechanism for safe policy change management.
+
+---
+
+## 7. Base Contract โ€” Evaluation and Audit
+
+**Evaluation order:** Within a domain level, policies are evaluated in declared priority order. Across domain levels, more-specific domains evaluate after (and can override) less-specific domains.
+
+**Parallel evaluation:** Policies with no data dependencies on each other evaluate concurrently. The Policy Engine tracks dependency declarations.
+
+**Audit:** Every policy evaluation produces an audit record regardless of outcome. The record includes: policy_uuid, policy_version, match_result, output, enforcement_level, actor, timestamp. No evaluation is silent.
+
+---
+
+## 8. Output Schema โ€” GateKeeper
+
+**Fires on:** Request payload at assembly time.
+**Produces:** An allow or deny decision for the request.
+
+```yaml
+gatekeeper_output:
+  decision: allow | deny
+  reason: ""
+  field_locks:                           # optional: lock specific fields as immutable
+    - field: 
+      lock_type: immutable | constrained
+      constraint_schema:    # if constrained
+  warnings: [""]
+```
+
+**Policy Engine behavior:**
+- `allow` โ†’ request proceeds; field_locks applied to payload
+- `deny` โ†’ request blocked; `reason` included in consumer error response
+- Any active GateKeeper producing `deny` โ†’ request blocked (all must allow)
+
+---
+
+## 9. Output Schema โ€” Validation
+
+**Fires on:** Request payload; validates correctness of field values.
+**Produces:** Pass or fail with field-level detail.
+
+```yaml
+validation_output:
+  result: pass | fail
+  field_results:
+    - field: 
+      result: valid | invalid
+      message: ""
+      suggested_value:            # optional
+  advisory: [""]
+```
+
+**Policy Engine behavior:**
+- `pass` โ†’ request proceeds
+- `fail` โ†’ request blocked; `field_results` included in consumer error response
+
+---
+
+## 10. Output Schema โ€” Transformation
+
+**Fires on:** Request payload; enriches, modifies, or injects field values.
+**Produces:** A set of field mutations to apply to the payload.
+
+```yaml
+transformation_output:
+  mutations:
+    - field: 
+      operation: set | append | delete | lock
+      value:                  # for set/append
+      reason: ""
+      source_type: enrichment | injection | normalization | correction
+```
+
+**Policy Engine behavior:** All mutations from all active Transformation policies are collected and applied to the payload. Each mutation is recorded in field-level provenance with the policy_uuid as source.
+
+---
+
+## 11. Output Schema โ€” Recovery
+
+**Fires on:** A failure or ambiguity trigger condition (DISPATCH_TIMEOUT, PARTIAL_REALIZATION, CANCELLATION_FAILED, etc.).
+**Produces:** A recovery action and parameters.
+
+```yaml
+recovery_output:
+  action: DRIFT_RECONCILE | DISCARD_AND_REQUEUE | DISCARD_NO_REQUEUE |
+          ACCEPT_LATE_REALIZATION | COMPENSATE_AND_FAIL |
+          NOTIFY_AND_WAIT | ESCALATE | RETRY
+  action_parameters:
+    requeue_delay: PT0S                  # for DISCARD_AND_REQUEUE
+    max_attempts: 3                      # for RETRY
+    backoff: exponential                 # for RETRY
+    deadline: PT4H                       # for NOTIFY_AND_WAIT
+    on_deadline_exceeded: ESCALATE       # for NOTIFY_AND_WAIT
+  notify_before_action: true
+  notification_urgency: high
+```
+
+**Policy Engine behavior:** The first matching Recovery policy's action is executed. Recovery policies follow the same domain precedence โ€” resource_type override wins over tenant override wins over profile default.
+
+---
+
+## 12. Output Schema โ€” Orchestration Flow
+
+**The two-level orchestration model:**
+
+Orchestration in DCM operates at two levels that compose through the same Policy Engine:
+
+- **Level 1 โ€” Named Workflow Artifacts:** Orchestration Flow Policies with `ordered: true` declare an explicit, visible, auditable sequence of steps. Each step references a payload type from the closed vocabulary. This is what operators see and reason about. Adding a step = adding to a workflow Policy.
+- **Level 2 โ€” Dynamic Policies:** GateKeeper, Transformation, Recovery, and Governance Matrix Policies fire when their conditions match, within or alongside workflow steps, without being declared in the workflow. Adding conditional behavior = writing a dynamic policy.
+
+The Request Orchestrator (event bus) routes all payload type events through the Policy Engine. Both named workflow steps and dynamic policies evaluate against the same events. The workflow provides the skeleton; dynamic policies fill in conditional behavior.
+
+**Fires on:** Pipeline payload type events.
+**Produces:** A flow directive governing step ordering.
+
+```yaml
+orchestration_flow_output:
+  ordered: true | false
+  steps:
+    - step: 1
+      policy_handle: ""
+      condition: ""
+      on_fail: halt | skip | escalate
+  parallel_groups:                       # steps that may execute in parallel
+    - [step_1_id, step_2_id]
+```
+
+**Step vocabulary** โ€” steps reference payload types from the closed vocabulary, mapping to control plane operations:
+
+| Payload type | Maps to |
+|-------------|---------|
+| `request.initiated` | Start of request pipeline |
+| `request.layers_assembled` | Layer assembly complete |
+| `request.policies_evaluated` | All policies evaluated |
+| `request.placement_complete` | Provider selected |
+| `request.dispatched` | Sent to provider |
+| `discovery.cycle_complete` | Discovery cycle done |
+| `drift.detected` | Drift found |
+| `recovery.timeout_fired` | Dispatch timeout |
+| `provider_update.received` | Provider update notification |
+
+Custom steps extend this vocabulary by publishing new payload types.
+
+**Policy Engine behavior:** When `ordered: true`, steps execute in declared sequence. When `ordered: false`, the Policy Engine executes steps in parallel where no data dependencies exist. Orchestration Flow policies compose with standard GateKeeper and Transformation policies โ€” both types evaluate in the same pipeline.
+
+---
+
+## 13. Output Schema โ€” Governance Matrix Rule
+
+**Fires on:** Any cross-boundary interaction (DCM โ†’ Provider, DCM โ†’ Peer DCM, Provider โ†’ DCM).
+**Produces:** A boundary control decision with optional field permissions.
+
+```yaml
+governance_matrix_output:
+  decision: ALLOW | DENY | ALLOW_WITH_CONDITIONS | STRIP_FIELD | REDACT | AUDIT_ONLY
+  conditions:                            # for ALLOW_WITH_CONDITIONS
+    - field: 
+      operator: 
+      value: 
+  field_permissions:
+    mode: allowlist | blocklist | passthrough
+    paths: ["", ...]
+    on_blocked_field: STRIP_FIELD | DENY_REQUEST | REDACT
+  audit_on: [ALLOW, DENY, STRIP_FIELD]
+  notification_on: [DENY]
+  notification_urgency: critical
+```
+
+**Policy Engine behavior:** Hard DENY evaluated first โ€” any hard DENY is terminal. Soft decisions evaluated by domain precedence; DENY wins over ALLOW at the same level. Field permissions applied after decision determined. Audit record always written.
+
+---
+
+## 14. Output Schema โ€” Lifecycle Policy
+
+**Fires on:** Relationship events (related entity state changes, relationship creation/release).
+**Produces:** A lifecycle action to apply to related entities.
+
+```yaml
+lifecycle_policy_output:
+  on_related_destroy: cascade | protect | detach | notify
+  on_related_suspend: cascade | ignore | notify
+  on_last_relationship_released: destroy | retain | notify
+  propagation_depth: 1 | 2 | N          # how many relationship hops to propagate
+  action_delay: PT0S                     # grace period before executing action
+```
+
+**Policy Engine behavior:** When a relationship event occurs, all matching Lifecycle policies on both related entities are evaluated. The most restrictive action wins (save beats destroy). Conflicts between policies at the same domain level produce a CONFLICT_ERROR at policy ingestion time.
+
+---
+
+## 15. Output Schema โ€” ITSM Action
+
+The ITSM Action policy type triggers actions in connected ITSM systems as a side-effect of DCM pipeline events.
+
+```yaml
+itsm_action_output:
+  type: itsm_action
+  itsm_provider_uuid:        # registered ITSM Provider UUID
+  action: create_change_request | update_change_request | close_change_request |
+          create_incident | update_incident | close_incident |
+          update_cmdb_ci | create_cmdb_ci | retire_cmdb_ci |
+          create_service_request | link_parent_record
+  action_payload:
+    : 
+  store_reference_on_entity:    # default: false
+  reference_label: 
+  block_until_created:          # default: false โ€” see ITSM-005
+  block_timeout:   # required if block_until_created: true
+  on_failure: log_and_continue | alert_and_continue | alert_only
+```
+
+> **See [ITSM Integration](42-itsm-integration.md)** for full ITSM Provider registration, capability declarations, supported ITSM systems (ServiceNow, Jira, Remedy, Freshservice, PagerDuty, generic REST), policy examples, and system policies (ITSM-001โ€“007, ITSM-POL-001โ€“004).
+
+**Key constraints:**
+- ITSM Action policies are side-effect only โ€” they do not produce allow/deny decisions
+- `block_until_created: true` creates a pipeline gate with mandatory timeout (ITSM-005)
+- Multiple ITSM Action policies on the same event fire independently (ITSM-POL-004)
+- Full audit record produced on every evaluation (ITSM-POL-003)
+
+## 16. Policy Composition
+
+Policies compose naturally through the domain precedence model:
+
+```
+System policy (GateKeeper: cpu_count max 64)
+  โ””โ”€โ”€ Platform policy (GateKeeper: prod VMs require manager approval)
+        โ””โ”€โ”€ Tenant policy (GateKeeper: payments team max cpu_count 32)
+              โ””โ”€โ”€ Resource-type policy (Transformation: inject monitoring)
+```
+
+For a single request, all active matching policies at all domain levels evaluate. GateKeepers at all levels must allow (any deny blocks). Transformations from all levels are collected and applied. Recovery policies use the most-specific matching policy.
+
+**Policy Groups** are Data artifacts that group related policies by concern_type. Profiles activate Policy Groups. This is how "apply the HIPAA profile" works โ€” it activates the HIPAA compliance domain's Policy Group, which contains all the GateKeeper, Validation, Transformation, and Governance Matrix policies required for HIPAA compliance.
+
+---
+
+## 17. Related Policies
+
+| Policy | Rule |
+|--------|------|
+| `POL-001` | All DCM policy types implement the unified base contract. The output schema is the only thing that varies. |
+| `POL-002` | Every policy evaluation produces an audit record. No evaluation is silent. |
+| `POL-003` | Hard enforcement policies cannot be relaxed by any downstream rule at any domain level. |
+| `POL-004` | Policies in `proposed` status execute in shadow mode โ€” output is captured and never applied. Shadow mode is the primary mechanism for safe policy change management. |
+| `POL-005` | The Policy Engine is the sole evaluator of all policies. No component bypasses the Policy Engine to enforce rules directly. |
+| `POL-006` | Adding a new policy type requires defining a new output schema. The base contract, evaluation algorithm, lifecycle, and audit obligations are inherited. |
+| `POL-007` | Policies ARE the orchestration. Pipeline steps are Policies firing on payload type events. Static flows are Orchestration Flow Policies with `ordered: true`. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md
index 8c3c3a5..7184cb5 100644
--- a/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md
+++ b/content/docs/architecture/data-model/accreditation-and-authorization-matrix.md
@@ -1,14 +1,5 @@
----
-title: "Accreditation, Data Authorization Matrix, and Zero Trust"
-type: docs
-weight: 26
----
+# DCM Data Model โ€” Accreditation, Data Authorization Matrix, and Zero Trust
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -30,6 +21,8 @@ weight: 26
 
 ---
 
+> **Authentication Assurance Levels:** See [Standards and Compliance Catalog](40-standards-catalog.md) Section 7 for the NIST SP 800-63B AAL mapping per profile.
+
 ## 1. Purpose
 
 This document defines three interconnected models that together govern how DCM handles trust, data handling obligations, and compliance verification across all interaction boundaries:
@@ -167,11 +160,25 @@ accreditation:
   external_registry_id: "FR2024-0042"    # e.g., FedRAMP Marketplace ID
 
   # Status
-  status: active | suspended | revoked | expired | pending_renewal
+  status: active | suspended | revoked | expired | pending_renewal | pending_review
   revocation_reason: 
   revoked_at: 
+
+  # Automated verification (see doc 47 โ€” Accreditation Monitor)
+  verification:
+    tier: external_registry | document_currency | contract_webhook | expiry_only
+    stale_after: P7D                # max gap between verifications before stale_action fires
+    stale_action: warn | suspend | escalate   # profile-governed default: warn/suspend/escalate
+    verification_failure_count: 0
+    # tier-specific fields: see doc 47 Section 3 for full schema
 ```
 
+> **Accreditation Monitor:** The `last_verified_at` field is maintained by the
+> Accreditation Monitor (doc 47), which continuously verifies accreditation status
+> against external registries, document currency checks, or contract system webhooks
+> depending on the `verification.tier`. See doc 47 for the complete monitoring
+> specification and framework-by-framework automation coverage.
+
 ### 3.4 Accreditation Lifecycle
 
 ```
@@ -194,6 +201,11 @@ Accreditation submitted (via API or GitOps PR)
   โ”‚     status โ†’ expired
   โ”‚     Providers relying on this accreditation flagged: ACCREDITATION_GAP
   โ”‚
+  โ–ผ External status change detected by Accreditation Monitor:
+  โ”‚   status โ†’ pending_review
+  โ”‚   Platform Admin notified (urgency: high)
+  โ”‚   Exception: external status = Revoked โ†’ immediate revocation (no review)
+  โ”‚
   โ–ผ Revocation:
       Accreditor or Platform Admin revokes
       status โ†’ revoked
@@ -211,7 +223,7 @@ accreditation_gap_record:
   provider_uuid: 
   required_framework: hipaa
   required_for: [phi data fields in active requests]
-  gap_type: missing | expired | revoked | suspended
+  gap_type: missing | expired | revoked | suspended | verification_stale
   detected_at: 
   severity: critical                    # accreditation gaps are always high or critical
   affected_entity_uuids: [, ...]  # entities currently hosted at this provider
@@ -234,9 +246,7 @@ deployment_accreditation:
 ---
 
 
-> **Architecture Update:** Section 4 of this document (Data/Capability Authorization Matrix) has been superseded by the **Unified Governance Matrix** ([doc 27](27-governance-matrix.md)). The governance matrix provides a more powerful, unified model that replaces the standalone matrix described here. The accreditation model (Sections 2-3) and zero trust interaction model (Section 5) remain current and are consumed by the governance matrix as inputs.
->
-> New implementations should reference doc 27 for data and capability boundary enforcement.
+> **Scope:** This document covers the accreditation model (Sections 2-3) and zero trust interaction model (Section 5). Data and capability boundary enforcement is specified in the [Unified Governance Matrix](27-governance-matrix.md) (doc 27), which consumes the accreditation and classification models defined here as inputs.
 
 ## 4. Data/Capability Authorization Matrix
 
diff --git a/content/docs/architecture/data-model/accreditation-monitor.md b/content/docs/architecture/data-model/accreditation-monitor.md
new file mode 100644
index 0000000..cee708e
--- /dev/null
+++ b/content/docs/architecture/data-model/accreditation-monitor.md
@@ -0,0 +1,496 @@
+# DCM Data Model โ€” Accreditation Monitor
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Capability Specification
+**Related Documents:** [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) | [Information Providers](10-information-providers.md) | [Advanced Information Providers](21-information-providers-advanced.md) | [Standards Catalog](40-standards-catalog.md) | [Scoring Model](29-scoring-model.md) | [Event Catalog](33-event-catalog.md) | [Governance Matrix](27-governance-matrix.md)
+
+> **AEP Alignment:** API endpoints follow [AEP](https://aep.dev) conventions.
+> Custom methods use colon syntax. Async operations return `Operation` resources.
+> See `schemas/openapi/dcm-admin-api.yaml` for the normative admin API specification.
+
+---
+
+## 1. Purpose
+
+The Accreditation Monitor is a DCM **Information Provider** that continuously verifies the status of registered accreditations against authoritative external sources โ€” compliance registries, certificate authority portals, and contract management systems. It answers the question DCM's existing accreditation lifecycle model cannot answer alone:
+
+> *"Is this accreditation still valid according to the issuing authority โ€” not just according to the expiry date we were told?"*
+
+An accreditation can become invalid before its declared `valid_until` date: FedRAMP authorizations can be revoked mid-cycle, ISO 27001 certificates can be suspended by the certification body, CMMC certifications can be downgraded. Without external monitoring, DCM would continue treating a provider as accredited until the date it was told to check โ€” which may be 12 months away.
+
+The Accreditation Monitor closes this gap by polling authoritative external registries on a configurable schedule and surfacing discrepancies to DCM's existing accreditation lifecycle model.
+
+---
+
+## 2. What Can Be Automated โ€” Framework by Framework
+
+Automation depth varies significantly by framework. The Accreditation Monitor implements four verification tiers:
+
+### Tier 1 โ€” Full External Verification (public registry API)
+
+DCM can query the authoritative registry directly. Status changes are detected without any manual intervention.
+
+| Framework | Authoritative Source | What Is Verified |
+|-----------|---------------------|-----------------|
+| **FedRAMP Moderate/High/LI-SaaS** | [marketplace.fedramp.gov/api](https://marketplace.fedramp.gov) | Authorization status (Authorized / In Process / Revoked), impact level, authorization date, annual assessment currency |
+| **StateRAMP** | [stateramp.org](https://stateramp.org) | Authorization status, status changes |
+| **CMMC 2.0** | [cyberab.org/catalog](https://cyberab.org/catalog) | Certification level (1/2/3), certification status, expiry date |
+| **IAF / ISO 27001** | [iaf.nu CertSearch](https://iaf.nu) | Certificate status (valid/suspended/withdrawn), expiry date, scope, issuing CB identity |
+
+**Required field at registration:** `external_registry_id` โ€” the provider's identifier in the external registry (FedRAMP ID, CMMC certification number, ISO certificate number).
+
+### Tier 2 โ€” Document Currency Verification
+
+DCM cannot query a central registry, but it can verify that the stored evidence document is current relative to the framework's required renewal cycle.
+
+| Framework | Verification Method | Currency Check |
+|-----------|---------------------|---------------|
+| **SOC 2 Type I / Type II** | Document metadata extraction from `audit_report_ref` | Report period end date must be within 12 months (configurable); examines PDF metadata or report header |
+| **PCI DSS** | QSA authorization check + AoC date | Verifies the issuing QSA firm is currently listed as authorized at PCI SSC; verifies stored AoC covers current period |
+| **ISO 27001** (when CB portal is unavailable) | Certificate document date + P3Y renewal cycle | Falls back to document-based verification if IAF CertSearch is unreachable |
+
+### Tier 3 โ€” Contract Lifecycle Integration
+
+No external registry exists. Verification is driven by events from contract management systems.
+
+| Framework | Integration Mechanism | What Triggers Verification |
+|-----------|----------------------|--------------------------|
+| **HIPAA BAA** | Contract management webhook (DocuSign, Ironclad, Agiloft, custom) | BAA signed, amended, terminated, or approaching renewal |
+| **DoD IL2/IL4/IL5/IL6** | Manual submission + webhook on DISA action letter | Authorization letter received, amended, or revoked |
+| **Custom / Sovereign** | Manual submission + optional webhook | Platform admin triggers; or custom webhook from issuing authority |
+
+### Tier 4 โ€” Expiry-Only Monitoring (no external verification available)
+
+No external API, no document currency check, no contract integration. DCM monitors only the declared `valid_until` date.
+
+| Framework | What Is Monitored |
+|-----------|-----------------|
+| **HIPAA BAA** (if no contract system) | Declared BAA expiry date |
+| **Internal / first_party accreditations** | Declared validity period |
+| **Self-declared** | Declared validity period |
+
+---
+
+## 3. Accreditation Record Additions
+
+The existing accreditation record (doc 26 Section 3.3) is extended with three new fields to support automated monitoring:
+
+```yaml
+accreditation:
+  # ... existing fields unchanged ...
+
+  # NEW โ€” Verification model
+  verification:
+    tier: external_registry | document_currency | contract_webhook | expiry_only
+    # โ”€โ”€ Tier 1 specific โ”€โ”€
+    registry_api:
+      provider: fedramp | stateramp | cmmc_ab | iaf_certsearch | custom
+      lookup_key:     # e.g., FedRAMP ID "FR2024-0042"
+      poll_interval: P1D                          # how often to check
+      last_checked_at: 
+      last_result: confirmed_active | status_changed | registry_unavailable | not_found
+    
+    # โ”€โ”€ Tier 2 specific โ”€โ”€
+    document_check:
+      document_url: 
+      max_age: P365D                              # how old the document can be
+      date_extraction_method: pdf_metadata | report_header_parse | manual
+      last_checked_at: 
+      last_document_date: 
+    
+    # โ”€โ”€ Tier 3 specific โ”€โ”€
+    contract_webhook:
+      contract_system: docusign | ironclad | agiloft | custom
+      contract_id: 
+      webhook_url: 
+      last_event_at: 
+    
+    # โ”€โ”€ Shared โ”€โ”€
+    stale_after: P7D                              # how long before last_checked_at = stale
+    stale_action: warn | suspend | escalate       # what to do when stale
+    verification_failure_count: 0                 # consecutive failures; triggers escalation
+    verification_failure_threshold: 3
+```
+
+The `last_verified_at` field on the existing accreditation record (doc 26) is updated by the Accreditation Monitor on each successful verification. It remains the canonical "last confirmed active" timestamp used by the Governance Matrix and Scoring Model.
+
+---
+
+## 4. Accreditation Monitor as an Information Provider
+
+The Accreditation Monitor registers with DCM as an Information Provider with `information_type: accreditation_verification`. It is a separately deployable component โ€” it does not require changes to the DCM control plane and can be upgraded independently.
+
+```yaml
+accreditation_monitor_registration:
+  provider_type: information_provider
+  information_type: accreditation_verification
+  display_name: "DCM Accreditation Monitor"
+  version: "1.0.0"
+  
+  # What it monitors
+  supported_tiers:
+    - external_registry
+    - document_currency
+    - contract_webhook
+    - expiry_only
+  
+  supported_registries:
+    - fedramp
+    - stateramp
+    - cmmc_ab
+    - iaf_certsearch
+    - pci_ssc_qsa      # QSA verification only
+  
+  supported_contract_systems:
+    - docusign
+    - ironclad
+    - custom_webhook
+  
+  # How it communicates results back to DCM
+  push_events: true
+  event_types:
+    - accreditation.verified           # periodic confirmation: still active
+    - accreditation.status_changed     # external registry shows different status
+    - accreditation.registry_mismatch  # external status != DCM recorded status
+    - accreditation.verification_stale # last_checked_at exceeds stale_after threshold
+    - accreditation.expiry_approaching # approaching valid_until (supplement to existing)
+    - accreditation.document_expired   # document_check: document older than max_age
+    - accreditation.contract_event     # contract_webhook: BAA signed/amended/terminated
+
+  health_check:
+    endpoint: /health
+    interval: PT5M
+```
+
+---
+
+## 5. Verification Flows
+
+### 5.1 Tier 1 โ€” FedRAMP External Registry Verification
+
+```
+Accreditation Monitor poll cycle (default: P1D):
+  โ”‚
+  โ–ผ For each active Tier 1 accreditation:
+  โ”‚   Load accreditation record
+  โ”‚   Extract: framework, external_registry_id, last known status
+  โ”‚
+  โ–ผ Query external registry:
+  โ”‚   FedRAMP: GET marketplace.fedramp.gov/api/products?id={external_registry_id}
+  โ”‚   CMMC:    GET cyberab.org/api/certifications?cert_number={external_registry_id}
+  โ”‚   IAF:     GET iaf.nu/certsearch?cert={external_registry_id}
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: status = Authorized, impact_level = High
+  โ”‚     Matches DCM record โ†’ no action
+  โ”‚     Update: last_checked_at, last_result: confirmed_active
+  โ”‚     Fire: accreditation.verified (urgency: low)
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: status = In Process (was Authorized)
+  โ”‚     Status changed โ†’ MISMATCH
+  โ”‚     Fire: accreditation.status_changed (urgency: high)
+  โ”‚     Payload: {from: authorized, to: in_process, external_source: fedramp_marketplace}
+  โ”‚     DCM action: accreditation status โ†’ pending_review
+  โ”‚     Platform Admin notified โ€” human must review and decide: suspend or retain
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: status = Revoked
+  โ”‚     Fire: accreditation.status_changed (urgency: critical)
+  โ”‚     DCM action: accreditation status โ†’ revoked immediately
+  โ”‚     Accreditation Gap triggered for all affected providers
+  โ”‚     Recovery Policy evaluated
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: 404 / not_found
+  โ”‚     May indicate ID change or deregistration
+  โ”‚     Fire: accreditation.registry_mismatch (urgency: high)
+  โ”‚     Increment verification_failure_count
+  โ”‚     Platform Admin notified to verify external_registry_id is correct
+  โ”‚
+  โ””โ”€โ”€ Registry unreachable (timeout, 5xx)
+        Update: last_result: registry_unavailable
+        Increment verification_failure_count
+        If count >= verification_failure_threshold:
+          Fire: accreditation.verification_stale (urgency: medium)
+        Do NOT change accreditation status on registry failure alone
+        (conservative: prefer false negative over false positive revocation)
+```
+
+### 5.2 Tier 2 โ€” Document Currency Verification (SOC 2, PCI DSS AoC)
+
+```
+Verification cycle (default: P7D):
+  โ”‚
+  โ–ผ Fetch document from certificate_ref or audit_report_ref URL
+  โ”‚
+  โ”œโ”€โ”€ PDF: extract creation_date from PDF metadata
+  โ”‚         or parse report header for "Report Date: YYYY-MM-DD"
+  โ”‚
+  โ”œโ”€โ”€ HTML report: parse structured date field
+  โ”‚
+  โ””โ”€โ”€ Fallback: flag for manual review if date cannot be extracted
+  
+  โ–ผ Compare document date to max_age threshold (default P365D):
+  โ”‚
+  โ”œโ”€โ”€ Within threshold โ†’ update last_checked_at, last_document_date
+  โ”‚                      Fire: accreditation.verified (urgency: low)
+  โ”‚
+  โ””โ”€โ”€ Beyond threshold โ†’ Fire: accreditation.document_expired (urgency: high)
+                          Platform Admin notified: new report needed
+                          Accreditation status โ†’ pending_renewal
+```
+
+### 5.3 Tier 3 โ€” Contract Webhook (HIPAA BAA, DoD IL)
+
+```
+Contract management system fires webhook to DCM:
+  POST /api/v1/admin/accreditations/{uuid}/contract-event
+  
+  Payload:
+  {
+    "contract_id": "",
+    "event_type": "signed | amended | terminated | renewal_due | renewed",
+    "effective_date": "",
+    "details": { ... contract-system-specific fields ... }
+  }
+  
+  DCM processes:
+  โ”œโ”€โ”€ signed   โ†’ accreditation status: active (if was pending)
+  โ”œโ”€โ”€ amended  โ†’ accreditation status: pending_review; Platform Admin notified
+  โ”œโ”€โ”€ terminated โ†’ accreditation status: revoked; Accreditation Gap triggered
+  โ”œโ”€โ”€ renewal_due โ†’ notification to Compliance Team (urgency: medium)
+  โ””โ”€โ”€ renewed  โ†’ accreditation status: active; valid_until updated; last_verified_at updated
+```
+
+### 5.4 Stale Verification Handling
+
+Regardless of tier, when `last_checked_at` is older than `stale_after`:
+
+```
+stale_action: warn     โ†’ Fire: accreditation.verification_stale (urgency: low)
+                          No change to accreditation status
+                          
+stale_action: suspend  โ†’ Fire: accreditation.verification_stale (urgency: high)
+                          Accreditation status โ†’ suspended
+                          Accreditation Gap triggered (gap_type: suspended)
+                          Platform Admin must manually verify and reactivate
+                          
+stale_action: escalate โ†’ Fire: accreditation.verification_stale (urgency: critical)
+                          Escalation chain notified (Compliance Team + Platform Admin)
+                          No automatic status change
+                          If not resolved within escalation_window: โ†’ suspend
+```
+
+`stale_action` defaults by profile: `warn` for dev/standard, `suspend` for prod, `escalate` for fsi/sovereign.
+
+---
+
+## 6. New Event Types (additions to doc 33)
+
+These events are added to the event catalog as domain `accreditation.*`:
+
+| Event Type | Urgency | Description | Key Payload Fields |
+|-----------|---------|-------------|-------------------|
+| `accreditation.verified` | low | Periodic external confirmation โ€” accreditation still active | accreditation_uuid, framework, registry, checked_at |
+| `accreditation.status_changed` | high or critical | External registry shows a different status than DCM records | accreditation_uuid, framework, from_status, to_status, external_source |
+| `accreditation.registry_mismatch` | high | External registry cannot find the accreditation by its external_registry_id | accreditation_uuid, external_registry_id, registry, failure_detail |
+| `accreditation.verification_stale` | varies | last_checked_at exceeds stale_after threshold | accreditation_uuid, last_checked_at, stale_after, stale_action_taken |
+| `accreditation.document_expired` | high | Evidence document older than max_age threshold | accreditation_uuid, framework, document_url, document_date, max_age |
+| `accreditation.contract_event` | varies | Contract management webhook received | accreditation_uuid, contract_event_type, contract_id, effective_date |
+| `accreditation.expiry_approaching` | medium | Approaching valid_until within renewal_warning_before (supplement to existing TTL-based check) | accreditation_uuid, valid_until, days_remaining |
+
+---
+
+## 7. Accreditation Record Additions to doc 26
+
+The following fields are added to the accreditation record structure in doc 26 Section 3.3.
+These are non-breaking additions โ€” existing records without these fields default to `tier: expiry_only`.
+
+```yaml
+# Additions to existing accreditation record (doc 26 Section 3.3)
+
+  verification:
+    tier: external_registry | document_currency | contract_webhook | expiry_only
+    
+    # For external_registry tier:
+    registry_api:
+      provider: fedramp | stateramp | cmmc_ab | iaf_certsearch | pci_ssc_qsa | custom
+      lookup_key:            # value to use as query key in registry
+      poll_interval: P1D             # ISO 8601 duration; how often Monitor checks
+      last_checked_at: 
+      last_result: confirmed_active | status_changed | registry_unavailable | not_found | pending
+    
+    # For document_currency tier:
+    document_check:
+      document_url:             # usually same as certificate_ref or audit_report_ref
+      max_age: P365D                 # maximum acceptable document age
+      date_extraction_method: pdf_metadata | report_header_parse | manual
+      last_checked_at: 
+      last_document_date: 
+    
+    # For contract_webhook tier:
+    contract_webhook:
+      contract_system: docusign | ironclad | agiloft | custom
+      contract_id:           # ID in the contract management system
+      webhook_configured: true | false
+      last_event_at: 
+    
+    # Shared across all tiers:
+    stale_after: P7D                 # max acceptable gap between verifications
+    stale_action: warn | suspend | escalate
+    verification_failure_count: 0
+    verification_failure_threshold: 3
+```
+
+---
+
+## 8. Impact on Scoring Model (doc 29)
+
+The Scoring Model's Signal 5 (Provider Accreditation Richness, doc 29 Section 4.5) is
+enhanced with a verification currency dimension. An accreditation that has been externally
+verified recently is worth more than one that has only ever been manually submitted.
+
+```yaml
+# Addition to accreditation_weights in doc 29:
+verification_multipliers:
+  # Applied to each accreditation's weight based on verification currency
+  external_registry_verified_within_P1D:  1.0    # full weight
+  external_registry_verified_within_P7D:  0.9    # slight discount
+  document_verified_within_P30D:          0.85
+  contract_webhook_active:                0.9
+  expiry_only_no_external_check:          0.7    # meaningful discount
+  verification_stale:                     0.4    # significant discount
+  verification_failed_threshold_reached:  0.1    # near-zero weight
+```
+
+This means a provider with a FedRAMP High accreditation that was externally verified
+yesterday scores higher in placement tie-breaking than a provider with the same
+accreditation whose verification check has been stale for 30 days.
+
+---
+
+## 9. Admin API Additions
+
+```
+# List all accreditations with their current verification status
+GET /api/v1/admin/accreditations
+  ?verification_status=stale|failed|confirmed|pending
+  &framework=fedramp_high|iso_27001|...
+  &subject_uuid={provider_uuid}
+  page_size=50&page_token=...
+
+# Trigger immediate re-verification of a specific accreditation
+POST /api/v1/admin/accreditations/{accreditation_uuid}:verify
+
+Response 200 โ€” returns Operation:
+  {
+    "name": "/api/v1/operations/{uuid}",
+    "done": false,
+    "metadata": {
+      "stage": "VERIFICATION_INITIATED",
+      "accreditation_uuid": "{uuid}",
+      "verification_tier": "external_registry"
+    }
+  }
+
+# Register a contract webhook endpoint for a BAA or DoD IL accreditation
+POST /api/v1/admin/accreditations/{accreditation_uuid}:configure-webhook
+  {
+    "contract_system": "docusign",
+    "contract_id": "abc-def-123",
+    "webhook_secret": ""
+  }
+
+# Inbound webhook endpoint (called by contract management systems)
+POST /api/v1/admin/accreditations/{accreditation_uuid}/contract-event
+  Authorization: Bearer 
+  {
+    "contract_event_type": "terminated",
+    "effective_date": "2026-04-01",
+    "contract_id": "abc-def-123",
+    "details": {}
+  }
+```
+
+---
+
+## 10. Deployment and Configuration
+
+The Accreditation Monitor is deployed as a standalone container alongside the DCM control plane. It requires:
+
+- Network access to external registries (FedRAMP, CMMC AB, IAF CertSearch, PCI SSC)
+- Network access to the DCM API Gateway (to push accreditation events)
+- Access to `certificate_ref` and `audit_report_ref` document URLs (for Tier 2 checks)
+- Inbound webhook endpoint (for Tier 3 contract system integrations)
+
+**Air-gapped / sovereign deployments:** For deployments without external internet access, the Accreditation Monitor operates in Tier 4 (expiry-only) mode by default for all frameworks. Tier 2 checks can still work if documents are stored on internal object storage (certificate_ref points to an internal URL). Tier 3 contract webhooks work if the contract management system is internal. Tier 1 registry checks are disabled โ€” a manual verification workflow applies instead, with platform admins periodically updating `last_verified_at` after out-of-band confirmation.
+
+```yaml
+accreditation_monitor_config:
+  # Per-registry enable/disable
+  registries:
+    fedramp:
+      enabled: true
+      poll_interval: P1D
+      timeout: PT30S
+    cmmc_ab:
+      enabled: true
+      poll_interval: P7D
+    iaf_certsearch:
+      enabled: true
+      poll_interval: P7D
+    stateramp:
+      enabled: false               # enable if state/local gov providers present
+  
+  # Document currency checks
+  document_checks:
+    enabled: true
+    default_max_age: P365D
+    extraction_timeout: PT60S
+  
+  # Air-gapped mode
+  air_gapped_mode: false
+  air_gapped_fallback: expiry_only | manual_workflow
+  
+  # Global escalation
+  global_stale_after: P7D
+  global_failure_threshold: 3
+  escalation_contact:
+    notification_provider_uuid: 
+    urgency: critical
+```
+
+---
+
+## 11. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ACM-001` | The Accreditation Monitor is the authoritative source for `last_verified_at` on accreditation records. Platform admins may update it manually only when the Monitor is unavailable or in air-gapped mode โ€” all manual updates require a justification reason stored in the audit trail. |
+| `ACM-002` | An accreditation status change detected by the Monitor (external registry reports different status than DCM) does not automatically revoke the accreditation. It fires `accreditation.status_changed` and sets status to `pending_review`. A platform admin must confirm the change. The exception: if the external status is `Revoked` or `Terminated`, DCM immediately sets accreditation status to `revoked` without waiting for admin confirmation. |
+| `ACM-003` | Verification failure (registry unreachable, document inaccessible) does not revoke an accreditation. The Monitor increments `verification_failure_count`. At `verification_failure_threshold`, it fires `accreditation.verification_stale` and applies `stale_action`. Failure itself does not constitute a gap โ€” only confirmed negative status does. |
+| `ACM-004` | Accreditations in `sovereign` and `fsi` profiles must have `verification.tier` declared at a level of `document_currency` or above. `expiry_only` is not permitted for sovereign/fsi accreditations unless `air_gapped_mode: true` is explicitly configured. |
+| `ACM-005` | The `verification_multipliers` in the Scoring Model (Signal 5) apply to all accreditations, including those submitted before the Accreditation Monitor was deployed. Legacy accreditations with no `last_checked_at` are treated as `verification_stale` and weighted at the stale multiplier (0.4) until the Monitor performs its first check. |
+| `ACM-006` | Inbound contract webhooks (Tier 3) must authenticate using a provider callback credential issued at accreditation configuration time. Unauthenticated webhook calls are rejected with `401 Unauthorized` and generate an audit record. |
+| `ACM-007` | All verification events (`accreditation.*`) are written to the Audit Store regardless of outcome. There are no silent verifications โ€” every check, success or failure, has an audit record. |
+| `ACM-008` | In air-gapped mode, the Monitor operates in Tier 4 for registries it cannot reach. It does not repeatedly attempt unreachable external registries. After `air_gapped_retry_interval` (default P30D), it retries once to detect if network access has been restored. |
+
+---
+
+## 12. Relationship to Existing Architecture
+
+### doc 26 โ€” Accreditation and Authorization Matrix
+The Accreditation Monitor extends but does not replace the accreditation lifecycle model in doc 26. The existing `proposed โ†’ active โ†’ expired/revoked` lifecycle is preserved. The Monitor adds automated transitions into `pending_review` and `pending_renewal` states, and provides the data that drives the existing `ACCREDITATION_GAP` logic.
+
+### doc 27 โ€” Governance Matrix
+The Governance Matrix already evaluates accreditation status as part of Check 3 of the five-check boundary model. The Monitor improves the quality of that check: instead of relying solely on the declared `valid_until` date, the Governance Matrix now has access to externally verified current status via `last_verified_at` and `last_result`.
+
+### doc 29 โ€” Scoring Model
+The `verification_multipliers` addition to Signal 5 (Provider Accreditation Richness) means placement decisions can prefer providers whose accreditations have been recently externally verified over those relying on self-declared or stale verifications. This is a conservative, progressive enhancement โ€” it does not block placement, only refines tie-breaking.
+
+### doc 33 โ€” Event Catalog
+Seven new `accreditation.*` events are added (Section 6 of this document). These follow the same envelope and urgency model as all other DCM events.
+
+### doc 40 โ€” Standards Catalog
+The Accreditation Monitor is the operational implementation of the standards catalog's compliance framework entries. The standards catalog says *what* DCM recognizes; the Accreditation Monitor says *how* DCM verifies that recognition is still current.
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/api-versioning-strategy.md b/content/docs/architecture/data-model/api-versioning-strategy.md
index 22faf0f..b906171 100644
--- a/content/docs/architecture/data-model/api-versioning-strategy.md
+++ b/content/docs/architecture/data-model/api-versioning-strategy.md
@@ -1,8 +1,4 @@
----
-title: "API Versioning Strategy"
-type: docs
-weight: 34
----
+# DCM Data Model โ€” API Versioning Strategy
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” API Versioning and Lifecycle
@@ -42,7 +38,7 @@ Individual endpoints are not independently versioned. If a single endpoint needs
 Non-breaking changes within a major version are documented in the API changelog but do not change the URL. Clients do not need to take any action for non-breaking changes.
 
 The changelog follows semantic versioning conventions:
-- **Minor change**: new optional fields, new endpoints, expanded enum values with backward-compatible defaults
+- **Minor change**: new optional fields, new endpoints, expanded enum values with version-compatible defaults
 - **Revision**: documentation corrections, clarifications, non-functional specification updates
 
 ---
@@ -309,7 +305,7 @@ provider_registration:
 
 ### 7.2 OIS Compatibility
 
-DCM maintains backward compatibility with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 during the deprecation window.
+DCM maintains version-compatible with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 until the version is sunset.
 
 When the OIS version is incremented:
 1. DCM announces the new OIS version via the event `governance.ois_version_released`
@@ -323,17 +319,15 @@ Providers that expose their own management APIs (beyond the standard OIS surface
 
 ---
 
-## 8. Client Migration Path
-
-### 8.1 Migration Guide Structure
+## 8. Version Upgrade Path
 
-Each new major version publishes a migration guide accessible at:
+When a new major API version is published, a machine-readable change log is available at:
 
 ```
 GET /api/v{N}/migration-guide
 ```
 
-The migration guide is machine-readable JSON listing all breaking changes from the previous version:
+This endpoint returns all breaking changes from the previous major version:
 
 ```json
 {
@@ -344,22 +338,18 @@ The migration guide is machine-readable JSON listing all breaking changes from t
       "change_id": "BC-001",
       "type": "field_removed",
       "endpoint": "GET /api/v2/resources/{uuid}",
-      "description": "Field 'legacy_id' removed from response. Use 'entity_uuid' instead.",
-      "migration": "Replace references to 'legacy_id' with 'entity_uuid'",
-      "affected_since": "2026-06-01"
+      "description": "Field 'legacy_id' removed โ€” use 'entity_uuid' instead"
     }
   ],
-  "non_breaking_additions": [ ... ],
-  "sunset_date_of_previous_version": "2027-06-01"
+  "new_capabilities": []
 }
 ```
 
-### 8.2 Parallel Operation
-
-During the deprecation window, clients may run v1 and v2 in parallel โ€” for example, migrating one service at a time. Both versions return consistent data from the same underlying DCM data stores. There are no data synchronization concerns between versions.
+Clients declare the API version they target via the `Accept-Version` header or URL prefix. DCM supports all non-sunset major versions simultaneously. When a version reaches sunset, responses include `Deprecation` and `Sunset` headers (RFC 8594) before support is withdrawn.
 
 ---
 
+
 ## 9. Internal API Versioning
 
 DCM internal component APIs (Control Plane components communicating with each other) follow a simpler model:
@@ -383,7 +373,7 @@ DCM internal component APIs (Control Plane components communicating with each ot
 | `VER-006` | The `latest` version alias is available but must not be recommended for production use. Production clients must pin to a specific version. |
 | `VER-007` | Preview endpoints are not stable. They may change or be removed without a major version increment. They are identified by the `/preview/` path segment. |
 | `VER-008` | Every new major version must publish a machine-readable migration guide at `/api/v{N}/migration-guide`. |
-| `VER-009` | DCM must maintain dispatch compatibility with providers registered on supported OIS versions during the OIS deprecation window. |
+| `VER-009` | DCM must maintain dispatch compatibility with providers registered on supported OIS versions until the OIS version is sunset. |
 
 ---
 
diff --git a/content/docs/architecture/data-model/audit-provenance-observability.md b/content/docs/architecture/data-model/audit-provenance-observability.md
index c1c8e01..b981323 100644
--- a/content/docs/architecture/data-model/audit-provenance-observability.md
+++ b/content/docs/architecture/data-model/audit-provenance-observability.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Audit, Provenance, and Observability"
-type: docs
-weight: 12
----
+# DCM Data Model โ€” Audit, Provenance, and Observability
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/architecture/data-model/auth-providers.md b/content/docs/architecture/data-model/auth-providers.md
index e1c1c40..db57f10 100644
--- a/content/docs/architecture/data-model/auth-providers.md
+++ b/content/docs/architecture/data-model/auth-providers.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Authentication, Authorization, and Auth Providers"
-type: docs
-weight: 19
----
+# DCM Data Model โ€” Authentication, Authorization, and Auth Providers
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).*
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Webhooks and Messaging](18-webhooks-messaging.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md)
@@ -199,7 +190,7 @@ auth_provider:
           dcm_role: platform_admin
 ```
 
-### 4.3 LDAP / FreeIPA
+### 4.3 LDAP / FreeIPA (RFC 4511)
 
 ```yaml
 auth_provider:
@@ -514,7 +505,7 @@ dcm auth configure \
 
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
-| 1 | Should DCM support SCIM for automated user provisioning from enterprise IdPs? | Enterprise integration | โœ… Resolved โ€” SCIM 2.0 optional capability; provisions actors and group memberships; roles not SCIM-provisioned; suspend on deprovision default (AUTH-012) |
+| 1 | Should DCM support SCIM for automated user provisioning from enterprise IdPs? | Enterprise integration | โœ… Resolved โ€” SCIM 2.0 (RFC 7643 / RFC 7644) optional capability; provisions actors and group memberships; roles not SCIM-provisioned; suspend on deprovision default (AUTH-012) |
 | 2 | How does Auth Provider failover interact with in-flight requests during the transition? | Reliability | โœ… Resolved โ€” in-flight requests complete on cached tokens; failover chain for new auth; session TTL respected during outage; all providers unavailable โ†’ reject (AUTH-013) |
 | 3 | Should MFA enforcement be per-operation (step-up MFA) or per-session? | Security UX | โœ… Resolved โ€” two-tier MFA: per-session + step-up; policy declares which operations require step-up; PT10M step-up token TTL; profile-governed defaults (AUTH-014) |
 | 4 | Should the built-in Auth Provider's local user store be backed by a pluggable database? | Architecture | โœ… Resolved โ€” pluggable storage backend; SQLite for minimal/dev; PostgreSQL for standard+; encryption required for fsi/sovereign; local store for bootstrap/service accounts only (AUTH-015) |
diff --git a/content/docs/architecture/data-model/authority-tier-model.md b/content/docs/architecture/data-model/authority-tier-model.md
index bebf69f..9b01821 100644
--- a/content/docs/architecture/data-model/authority-tier-model.md
+++ b/content/docs/architecture/data-model/authority-tier-model.md
@@ -1,8 +1,4 @@
----
-title: "Authority Tier Model"
-type: docs
-weight: 32
----
+# DCM Data Model โ€” Authority Tier Model
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Authority Tier Specification
@@ -430,7 +426,7 @@ Security degradations block tier registry activation. The blocking gate requires
 This is the same pattern as the standard approval pipeline โ€” DCM provides the gate; the organization provides the review process. The difference is that the required tier for the degradation review is always at least `verified`, regardless of the profile in use.
 
 ```
-POST /api/v1/admin/tier-registry/{change_uuid}/accept-degradation
+POST /api/v1/admin/tier-registry/{change_uuid}:accept-degradation
 
 {
   "affected_item_uuid": "",
diff --git a/content/docs/architecture/data-model/consistency-review.md b/content/docs/architecture/data-model/consistency-review.md
new file mode 100644
index 0000000..86894a9
--- /dev/null
+++ b/content/docs/architecture/data-model/consistency-review.md
@@ -0,0 +1,257 @@
+# DCM โ€” Consistency Review Findings
+
+**Document Status:** โœ… Complete
+**Document Type:** Review Record
+**Date:** 2026-03
+**Scope:** Full review of all 50 data model documents, 14 specifications, 11 schema files, and 4 OpenAPI specs for naming inconsistencies, field conflicts, API surface misalignments, and terminology drift.
+
+---
+
+## Summary
+
+| Category | Issues Found | Fixed | Notes |
+|----------|-------------|-------|-------|
+| API path format (AEP colon syntax) | 13 + 4 | โœ… All | admin-api-spec.md and consumer-api-spec.md had stale slash-verb paths |
+| Stale entity_type values | 2 | โœ… All | `allocated_resource` and `resource_entity` replaced with canonical values |
+| Stale threshold key format | 5 | โœ… All | `auto_approve_below`/`verified_above` in scoring doc examples replaced with named-tier format |
+| provider_id vs provider_uuid | 6 | โœ… All | Resolved to `provider_uuid` in all DCM API paths |
+| Four store naming | Variant | โš ๏ธ Noted | Multiple names in use โ€” canonical list documented below |
+| Operation (LRO) shape completeness | 1 | โœ… | Polling section added to consumer spec |
+| lifecycle_state casing | Mixed | โš ๏ธ Noted | UPPERCASE in YAML examples, lowercase in prose โ€” by design |
+| Resource Type naming | Mixed | โš ๏ธ Noted | `resource_type` (field) vs `resource_type_fqn` (format ref) โ€” not a conflict |
+| Provider type count references | 1 | โœ… Already correct | All references say "eleven" or "11" |
+| Anti-vocabulary: 'widget' | 0 | N/A | Clean โ€” eliminated in prior sessions |
+
+---
+
+## 1. API Path Format (AEP Colon Syntax)
+
+### 1.1 Findings
+
+The AEP colon syntax was applied to the OpenAPI YAML files in a prior session but **not propagated** to the narrative specification documents. This meant `dcm-admin-api-spec.md` and `dcm-consumer-api-spec.md` still used slash-verb paths while the normative OpenAPI specs used colon-verb paths.
+
+**Affected specs:**
+- `dcm-admin-api-spec.md` โ€” 13 stale slash-verb paths
+- `dcm-consumer-api-spec.md` โ€” 4 stale slash-verb paths (partially fixed but not fully)
+- `dcm-flow-gui-spec.md` โ€” 2 stale paths (`:promote`, `:run` fixed in prior session)
+
+### 1.2 Resolution
+
+Applied the full set of colon conversions to both narrative specs. Both specs now include AEP alignment notes referencing the normative OpenAPI YAML files. Specific conversions:
+
+**Consumer spec:** `:suspend`, `:resume`, `:rehydrate`, `:rotate`, `:extend-ttl`, `:transfer`, `:bulk-decommission`, `:acknowledge`, `:revert`, `:accept`, `:reject`, `:approve`, `:read-all`
+
+**Admin spec:** `:approve`, `:reject`, `:suspend`, `:reinstate`, `:revoke-sessions`, `:reset`, `:vote`, `:rotate-credential`, `:trigger`, `:rebuild`, `:accept-degradation`, `:activate`
+
+### 1.3 Canonical Rule
+
+> Custom method paths use colon syntax: `POST /resources/{name}:verb`.
+> This applies to all narrative specs, OpenAPI YAML, and code examples.
+> Sub-resources (`.../status`, `.../stream`, `.../pending` as filtered list) keep slash notation.
+
+---
+
+## 2. Entity Type Values
+
+### 2.1 Findings
+
+Two documents used stale or incorrect values for the `entity_type` field:
+
+| Document | Stale Value | Correct Value |
+|----------|-------------|---------------|
+| `09-entity-relationships.md` | `allocated_resource` | `infrastructure_resource` (with `ownership_model: allocation`) |
+| `18-webhooks-messaging.md` | `resource_entity` | `infrastructure_resource` |
+
+The `related_entity_type: internal` and `related_entity_type: external` in doc 09 are **not** entity_type values โ€” they are relationship scope descriptors and are correct as-is. They describe whether the related entity is managed within DCM or is an external reference.
+
+### 2.2 Resolution
+
+Both corrected in place. The `allocated_resource` correction includes a comment: `# ownership_model: allocation` to preserve the semantic intent of the original example.
+
+### 2.3 Canonical Values
+
+The three valid `entity_type` values are:
+- `infrastructure_resource` โ€” persistent physical or virtual resource
+- `composite_resource` โ€” Meta Provider-orchestrated aggregate
+- `process_resource` โ€” ephemeral execution (automation job, playbook)
+
+Pool resources and shared resources are `infrastructure_resource` entities with `ownership_model: whole_allocation` or `ownership_model: shareable`. There is no separate pool or shared entity type.
+
+---
+
+## 3. Scoring Model Threshold Keys
+
+### 3.1 Findings
+
+`29-scoring-model.md` contained 5 examples using the old fixed-column threshold key format (`auto_approve_below`, `verified_above`) in per-service-type override examples. The authority tier model (doc 32) replaced these with a named-tier list format in a prior session, but the scoring doc examples were not updated.
+
+**Old format (stale):**
+```yaml
+auto_approve_below: 20
+verified_above: 40
+```
+
+**Current format:**
+```yaml
+thresholds:
+  - { tier: auto,     max_score: 20 }
+  - { tier: verified, max_score: 40 }
+```
+
+### 3.2 Resolution
+
+Replaced stale threshold keys with comments pointing to the named-tier format. The SMX-008 policy row was verified as already using current terminology.
+
+---
+
+## 4. provider_id vs provider_uuid
+
+### 4.1 Findings
+
+The operator interface spec (`dcm-operator-interface-spec.md`) and provider callback auth doc (`43-provider-callback-auth.md`) used `provider_id` in some places where `provider_uuid` is the correct DCM term.
+
+The distinction is important:
+- **`provider_uuid`** โ€” DCM-assigned UUID for the provider record. Used in all DCM API paths and payloads.
+- **`resource_id`** โ€” Operator-assigned identifier for a specific resource instance. Used in operator-to-DCM callbacks to identify the resource being reported on.
+
+### 4.2 Resolution
+
+DCM API endpoint paths updated to use `provider_uuid` consistently:
+- `POST /api/v1/providers/{provider_uuid}/capacity`
+- Registration response field `provider_id` โ†’ `provider_uuid`
+
+`resource_id` in callback APIs is **intentionally different** from `entity_uuid`. It is the operator's own identifier for the resource (returned in the `CreateResponse`). DCM maps it to `entity_uuid` internally. This distinction is correct and remains unchanged.
+
+---
+
+## 5. Four Store Naming โ€” Canonical Reference
+
+Multiple naming variants found across documents. The canonical names are:
+
+| Store | Canonical Name | Also Used (acceptable) | Do Not Use |
+|-------|---------------|----------------------|------------|
+| Intent State storage | **Intent Store** | GitOps Store (when emphasizing the implementation) | Intent State Store |
+| Requested State storage | **Requested Store** | โ€” | Requested State Store |
+| Realized State storage | **Realized Store** | Realized State Store | Realization Store |
+| Discovered State storage | **Discovered Store** | โ€” | Discovered State Store |
+
+No bulk renaming was performed โ€” both "Intent Store" and "GitOps Store" are used accurately in different contexts (the former emphasizes the state model, the latter the implementation). The variation is acceptable context-dependent usage, not an error.
+
+---
+
+## 6. Operation (LRO) Shape
+
+### 6.1 Finding
+
+The `consumer-api-spec.md` applied LRO `Operation` responses to async endpoints but did not include a dedicated section explaining the `GET /api/v1/operations/{uuid}` polling endpoint or the complete Operation shape.
+
+### 6.2 Resolution
+
+Added an "Operations โ€” Polling Long-Running Requests" section to `consumer-api-spec.md` covering:
+- The polling endpoint shape with in-progress, success, and failure states
+- Polling backoff guidance (1s โ†’ 2s โ†’ 5s โ†’ 10s โ†’ 30s)
+- Alternatives: webhook subscription (`request.progress_updated`), SSE stream
+
+---
+
+## 7. Lifecycle State Casing
+
+### 7.1 Finding
+
+Lifecycle state values appear in UPPERCASE in YAML examples (e.g., `lifecycle_state: OPERATIONAL`) and in lowercase in prose text (e.g., "the resource enters the operational state"). This is **by design**, not an inconsistency:
+
+- UPPERCASE in YAML/JSON โ€” machine-readable, matches enum values in schemas
+- Lowercase in prose โ€” natural language, matches how engineers write documentation
+
+No changes made.
+
+---
+
+## 8. resource_type vs resource_type_fqn
+
+### 8.1 Finding
+
+The JSON entity schema uses `$ref: resource_type_fqn` as a type reference (meaning the field value must be a Fully Qualified Name like `Compute.VirtualMachine`). Narrative YAML examples use `resource_type:` as the field name. This appears inconsistent but is not.
+
+`resource_type` is the **field name**. The value it holds must conform to the **FQN format** (`Category.TypeName`). The schema type reference is just documenting the format constraint.
+
+No changes made.
+
+---
+
+## 9. What Remains Acceptable (Not Fixed)
+
+These were identified but are not bugs โ€” they are intentional or context-appropriate variation:
+
+| Item | Why It's Acceptable |
+|------|---------------------|
+| `Service Provider` (title case) vs `service_provider` (snake_case) | Title case in prose, snake_case in code/YAML โ€” correct by context |
+| `Auth Provider` vs `auth_provider` | Same as above |
+| `Resource Type Spec` vs `Resource Type Specification` | Shortened form acceptable in prose; full form in formal definitions |
+| `Realized State` vs `realized state` | Title case for the formal concept, lowercase in general prose |
+| `related_entity_type: internal/external` in doc 09 | Not entity_type values โ€” relationship scope descriptors, correct as-is |
+| Provider type count varies ("nine", "eleven", "11") | All refer to the same 11 types; "nine" may be a historical reference pre-two additions |
+
+---
+
+## 10. Remaining Items Requiring Action by Implementers
+
+These are not documentation issues but implementation decisions that need to be made explicit when building DCM:
+
+| Item | Decision Needed |
+|------|----------------|
+| `resource_type` field in API payloads accepts short names (`Compute.VirtualMachine`) or requires registry UUID? | Current spec allows both `resource_type` (FQN string) and `resource_type_uuid` โ€” need to decide if UUID is required at dispatch or optional |
+| Operation polling endpoint `GET /api/v1/operations/{uuid}` โ€” is operation status part of request status, or a separate Operation resource? | Currently: request status via `GET /api/v1/requests/{uuid}/status`; Operation via `GET operation.name`. Need to clarify if these are the same or different responses |
+| `provider_id` in operator-assigned resource IDs vs DCM `provider_uuid` โ€” implementers must ensure they're not conflated at the API Gateway level | Code-level concern โ€” the API Gateway maps `resource_id` (operator-assigned) to `entity_uuid` (DCM-assigned) |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+---
+
+## Second Review Pass โ€” 2026-03
+
+A second comprehensive scan identified additional issues and confirmed no new architectural conflicts.
+
+### Additional Fixes Applied
+
+| Fix | Files Affected | Description |
+|-----|---------------|-------------|
+| Stale slash-verb paths | 14 files | Data model docs 00, 23, 24, 32, 35, 40, 42, 43 and specs dcm-admin-gui, dcm-consumer-gui, dcm-examples, dcm-registration, dcm-admin-api | All remaining colon-less custom method paths converted |
+| DELETE /api/v1/auth/session | dcm-consumer-api.yaml | Missing endpoint added to OpenAPI YAML (was in narrative spec only) |
+| GET /api/v1/operations/{uuid} | dcm-consumer-api.yaml | Operation polling endpoint added to OpenAPI YAML |
+| Missing event types | 33-event-catalog.md | 6 event types added: entity.deleted, entity.state_transition, group.deleted, group.member_added, group.member_removed, authorization.granted |
+| Prompt section numbering | DCM-AI-PROMPT.md | Duplicate section 78 resolved; sections 0โ€“80 now sequential with no duplicates |
+
+### Confirmed Clean (No Issues)
+
+- **OpenAPI schemas**: All 4 specs (consumer/admin/operator/callback) โ€” 0 schema conflicts
+- **Lifecycle state casing**: UPPERCASE in YAML/JSON examples, lowercase in prose โ€” intentional
+- **`Ingress API` usage**: Two occurrences are contextually correct (infrastructure layer explanation)
+- **`resource_type` vs `resource_type_fqn`**: Field name vs format description โ€” not a conflict
+- **`name:` vs `display_name:`**: Context-appropriate โ€” `name:` is a property name, `display_name:` is a human label
+- **Policy type casing**: Uppercase in section headers, lowercase in code/YAML โ€” by convention
+- **Provider type count**: All references to "eleven" or "11" provider types โ€” consistent
+- **Realized State write authority**: Consistent across all docs โ€” providers never write directly; DCM API Gateway is sole writer
+
+### Remaining Acceptable Variation
+
+These are not errors โ€” they are deliberate context-dependent usage:
+
+| Pattern | Both Forms Correct | Reason |
+|---------|--------------------|--------|
+| `Service Provider` / `service_provider` | Title case in prose, snake_case in code | Convention |
+| `Auth Provider` / `auth_provider` | Same | Convention |
+| `Policy Engine` / `policy_engine` | Same | Convention |
+| Intent Store / GitOps Store | Both describe the same store | Different emphasis (state model vs implementation) |
+| `entity.created` / `resource.provisioned` | Different semantic levels | Entity creation vs provisioning completion are distinct events |
+
+### Implementation Decisions Still Outstanding
+
+These three items were identified in the prior review and remain open โ€” they require implementation choices, not documentation changes:
+
+1. **`resource_type` at dispatch**: Accept FQN string (`Compute.VirtualMachine`) or require UUID? Current specs allow both; implementation must pick one canonical form.
+2. **Operation polling endpoint**: Is `GET /api/v1/operations/{uuid}` the same resource as `GET /api/v1/requests/{uuid}/status`, or a separate resource? Added to OpenAPI YAML as separate endpoint.
+3. **`resource_id` โ†’ `entity_uuid` mapping**: The API Gateway must map operator-assigned `resource_id` to DCM `entity_uuid` at the callback boundary. This is a code-level concern with no documentation gap.
+
diff --git a/content/docs/architecture/data-model/context-and-purpose.md b/content/docs/architecture/data-model/context-and-purpose.md
index 016e4fc..4bac891 100644
--- a/content/docs/architecture/data-model/context-and-purpose.md
+++ b/content/docs/architecture/data-model/context-and-purpose.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Context and Purpose"
-type: docs
-weight: 0
----
+# DCM Data Model โ€” Context and Purpose
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/architecture/data-model/control-plane-components.md b/content/docs/architecture/data-model/control-plane-components.md
index a33555e..dbe5448 100644
--- a/content/docs/architecture/data-model/control-plane-components.md
+++ b/content/docs/architecture/data-model/control-plane-components.md
@@ -1,14 +1,5 @@
----
-title: "Control Plane Components"
-type: docs
-weight: 25
----
+# DCM Data Model โ€” Control Plane Components
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
diff --git a/content/docs/architecture/data-model/credential-provider-model.md b/content/docs/architecture/data-model/credential-provider-model.md
index 625d5d4..67288ab 100644
--- a/content/docs/architecture/data-model/credential-provider-model.md
+++ b/content/docs/architecture/data-model/credential-provider-model.md
@@ -1,14 +1,5 @@
----
-title: "Credential Provider Model"
-type: docs
-weight: 31
----
+# DCM Data Model โ€” Credential Provider Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. This document specifies the Credential Provider model โ€” the issuance contract, credential lifecycle, rotation model, revocation propagation, and how credentials flow through the DCM pipeline.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Credential Provider Specification
@@ -21,6 +12,8 @@ weight: 31
 
 ---
 
+> **Cryptographic Standards:** See [Standards and Compliance Catalog](40-standards-catalog.md) for the complete list of permitted/forbidden algorithms, FIPS requirements per profile, and certificate protocol RFCs (RFC 7030, RFC 8555, RFC 8894, RFC 4210).
+
 ## 1. Purpose and Scope
 
 ### 1.1 What the Credential Provider Does
diff --git a/content/docs/architecture/data-model/dcm-federation.md b/content/docs/architecture/data-model/dcm-federation.md
index ac25d33..e7a07d2 100644
--- a/content/docs/architecture/data-model/dcm-federation.md
+++ b/content/docs/architecture/data-model/dcm-federation.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” DCM Federation, Peering, and Cross-Instance Coordination"
-type: docs
-weight: 22
----
+# DCM Data Model โ€” DCM Federation, Peering, and Cross-Instance Coordination
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Federated Contribution Model](28-federated-contribution-model.md) | [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md)
diff --git a/content/docs/architecture/data-model/dcm-self-health.md b/content/docs/architecture/data-model/dcm-self-health.md
index 004490a..ce116e8 100644
--- a/content/docs/architecture/data-model/dcm-self-health.md
+++ b/content/docs/architecture/data-model/dcm-self-health.md
@@ -1,8 +1,4 @@
----
-title: "DCM Self-Health Endpoints"
-type: docs
-weight: 39
----
+# DCM Data Model โ€” DCM Self-Health Endpoints
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Operational Health
@@ -341,6 +337,85 @@ dcm_internal_ca_days_until_next_expiry
 
 ---
 
+## 7. Per-Provider Metrics Contract
+
+In addition to DCM control plane metrics, each registered Service Provider must
+expose a Prometheus-compatible `/metrics` endpoint meeting the contract defined in
+the Registration Specification (GATE-SP-05).
+
+### 7.1 Required Provider Metric Families
+
+```
+# Dispatch metrics โ€” how many requests DCM sent to this provider
+dcm_provider_dispatches_total{resource_type="Compute.VirtualMachine", outcome="success|failed|timeout"}
+dcm_provider_dispatch_duration_seconds{resource_type="Compute.VirtualMachine", quantile="0.5|0.95|0.99"}
+
+# Realization metrics โ€” outcomes of provisioning
+dcm_provider_realizations_total{resource_type="Compute.VirtualMachine", status="OPERATIONAL|FAILED"}
+
+# Health signal
+dcm_provider_health_status  # gauge: 1=healthy, 0.5=degraded, 0=unhealthy
+```
+
+### 7.2 Recommended Provider Metric Families
+
+```
+# Capacity
+dcm_provider_capacity_remaining{resource_type="Compute.VirtualMachine"}  # gauge
+dcm_provider_capacity_total{resource_type="Compute.VirtualMachine"}      # gauge
+
+# Queue depth (for async-only providers)
+dcm_provider_queue_depth{resource_type="Compute.VirtualMachine"}  # gauge
+
+# Tenant usage
+dcm_provider_active_resources_total{tenant_uuid="...", resource_type="..."}
+```
+
+### 7.3 DCM Control Plane Aggregated Provider Metrics
+
+The DCM control plane exposes aggregated provider metrics at its own `/metrics`
+endpoint alongside the control plane metrics from Section 6:
+
+```
+# Already in Section 6 โ€” shown here for cross-reference
+dcm_providers_registered_total               # count of registered providers
+dcm_providers_healthy_total                  # count currently healthy
+dcm_provider_dispatch_duration_seconds{...}  # aggregated across all providers
+```
+
+### 7.4 Tenant Metadata Endpoint
+
+Service Providers in `standard` and above profiles (GATE-SP-04) must implement
+a tenant metadata endpoint that DCM calls to retrieve per-tenant usage summaries:
+
+```
+GET /api/v1/tenants/{tenant_uuid}/metadata
+Authorization: Bearer 
+
+Response 200:
+{
+  "tenant_uuid": "",
+  "active_resources": {
+    "Compute.VirtualMachine": 12,
+    "Storage.Block": 8
+  },
+  "capacity_consumed": {
+    "Compute.VirtualMachine": {
+      "cpu_cores": 96,
+      "ram_gb": 384
+    }
+  },
+  "quota_consumed_pct": {
+    "Compute.VirtualMachine": 48.0
+  }
+}
+```
+
+This data is used by DCM's Cost Analysis component and multi-tenant quota
+enforcement. It is also exposed to tenant administrators via the consumer API.
+
+---
+
 ## 8. Profile-Governed Health Exposure
 
 | Profile | /livez | /readyz | /api/v1/admin/health | /metrics scraping |
diff --git a/content/docs/architecture/data-model/deployment-redundancy.md b/content/docs/architecture/data-model/deployment-redundancy.md
index ec36394..f2224c1 100644
--- a/content/docs/architecture/data-model/deployment-redundancy.md
+++ b/content/docs/architecture/data-model/deployment-redundancy.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Deployment and Redundancy Model"
-type: docs
-weight: 17
----
+# DCM Data Model โ€” Deployment and Redundancy Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit Model](16-universal-audit.md) | [Policy Organization](14-policy-profiles.md)
@@ -29,6 +20,8 @@ weight: 17
 
 ---
 
+> **Operational guidance:** GitOps the disaster recovery runbook and RTO/RPO tables are in [Operational Reference](41-operational-reference.md) Section 3.
+
 ## 1. Purpose
 
 Every DCM component, every data store, and every capability is designed for redundancy by default. Redundancy is not an add-on or an advanced configuration โ€” it is the baseline operational posture for all profiles above `minimal`.
@@ -460,7 +453,7 @@ Read from any surviving Commit Log replica
 ### 8.1 Service Mesh
 
 All DCM component-to-component communication uses a service mesh (Istio or equivalent):
-- mTLS everywhere โ€” no plaintext internal communication
+- mTLS everywhere (RFC 8446 TLS 1.3 + RFC 5280 X.509) โ€” no plaintext internal communication
 - Traffic policies enforced at mesh level
 - Observability: traces, metrics, logs for all inter-component calls
 - Circuit breaking: prevent cascade failures
diff --git a/content/docs/architecture/data-model/design-priorities.md b/content/docs/architecture/data-model/design-priorities.md
index f1a3add..43dfb27 100644
--- a/content/docs/architecture/data-model/design-priorities.md
+++ b/content/docs/architecture/data-model/design-priorities.md
@@ -1,8 +1,4 @@
----
-title: "DCM Design Priorities"
-type: docs
-weight: -11
----
+# DCM Design Priorities
 
 **Document Status:** โœ… Stable โ€” Foundational reference
 **Document Type:** Architecture Reference โ€” Design Philosophy
@@ -227,7 +223,7 @@ External systems (ServiceNow, Jira, email workflows, Slack bots) connect to DCM
 The Admin API approval endpoint is designed to be called by external systems, not only by humans in a DCM UI:
 
 ```
-POST /api/v1/admin/approvals/{approval_uuid}/vote
+POST /api/v1/admin/approvals/{approval_uuid}:vote
 Authorization: Bearer      # any actor who is a member of the required DCMGroup
 
 {
diff --git a/content/docs/architecture/data-model/entity-relationships.md b/content/docs/architecture/data-model/entity-relationships.md
index 6a1d28f..abe418a 100644
--- a/content/docs/architecture/data-model/entity-relationships.md
+++ b/content/docs/architecture/data-model/entity-relationships.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Entity Relationships"
-type: docs
-weight: 9
----
+# DCM Data Model โ€” Entity Relationships
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -36,7 +27,7 @@ The DCM Entity Relationship model is the **universal mechanism for expressing re
 
 A single relationship model is used everywhere. There is no separate binding mechanism for storage, no separate dependency graph structure, no separate business data association mechanism. One model serves all relationship types across the full lifecycle โ€” from pre-realization planning through to post-realization management, drift detection, cost rollup, and rehydration.
 
-This document supersedes the dependency graph concept from the Service Dependencies document for data structure purposes. The Service Dependencies document retains content on rehydration ordering and failure handling, which operate on the relationship graph defined here.
+This document defines the Entity Relationship Graph, which is the data structure underlying service dependency declaration (doc 07) and rehydration ordering. The Service Dependencies document retains content on rehydration ordering and failure handling, which operate on the relationship graph defined here.
 
 ---
 
@@ -315,7 +306,7 @@ When a consuming Tenant claims an available allocation, DCM creates a first-clas
 ```yaml
 allocated_entity:
   uuid: 
-  entity_type: allocated_resource
+  entity_type: infrastructure_resource  # ownership_model: allocation
   resource_type_uuid: 
   tenant_uuid:   # Belongs to the consuming Tenant
 
diff --git a/content/docs/architecture/data-model/entity-types.md b/content/docs/architecture/data-model/entity-types.md
index 5a5b420..403700f 100644
--- a/content/docs/architecture/data-model/entity-types.md
+++ b/content/docs/architecture/data-model/entity-types.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Entity Types"
-type: docs
-weight: 1
----
+# DCM Data Model โ€” Entity Types
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
diff --git a/content/docs/architecture/data-model/event-catalog.md b/content/docs/architecture/data-model/event-catalog.md
index 7283dd7..34ebcd2 100644
--- a/content/docs/architecture/data-model/event-catalog.md
+++ b/content/docs/architecture/data-model/event-catalog.md
@@ -1,8 +1,4 @@
----
-title: "DCM Event Catalog"
-type: docs
-weight: 33
----
+# DCM Data Model โ€” Event Catalog
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Authoritative Event Catalog
@@ -724,6 +720,67 @@ payload:
 
 ---
 
+## 20. Accreditation Events (`accreditation.*`)
+
+Fired by the Accreditation Monitor (doc 47) when external verification
+of a registered accreditation produces a result or requires attention.
+
+| Event Type | Urgency | Description |
+|-----------|---------|-------------|
+| `accreditation.verified` | low | Periodic external confirmation โ€” accreditation still active in external registry |
+| `accreditation.status_changed` | high or critical | External registry reports a different status than DCM records โ€” requires platform admin review |
+| `accreditation.registry_mismatch` | high | External registry cannot find the accreditation by its `external_registry_id` โ€” ID may need correction |
+| `accreditation.verification_stale` | varies | `last_checked_at` exceeds `stale_after` threshold โ€” stale_action applied per configuration |
+| `accreditation.document_expired` | high | Evidence document (SOC 2 report, AoC) is older than `max_age` threshold โ€” new document required |
+| `accreditation.contract_event` | varies | Contract management webhook received (BAA signed, amended, or terminated) |
+| `accreditation.expiry_approaching` | medium | Approaching `valid_until` within `renewal_warning_before` window (automated complement to TTL-based check) |
+
+### 20.1 Payload Schemas
+
+```yaml
+# accreditation.status_changed โ€” the most critical event
+accreditation.status_changed:
+  accreditation_uuid: 
+  subject_uuid: 
+  framework: fedramp_high | iso_27001 | cmmc_2 | ...
+  from_status: authorized | active | certified
+  to_status: in_process | revoked | suspended | withdrawn
+  external_source: fedramp_marketplace | cmmc_ab | iaf_certsearch | contract_webhook
+  detected_at: 
+  action_taken: pending_review | immediate_revocation
+  # immediate_revocation when to_status is 'revoked' or 'terminated'
+
+# accreditation.verification_stale
+accreditation.verification_stale:
+  accreditation_uuid: 
+  subject_uuid: 
+  framework: 
+  last_checked_at: 
+  stale_after: P7D
+  stale_action_taken: warn | suspended | escalated
+  consecutive_failures: 
+
+# accreditation.contract_event
+accreditation.contract_event:
+  accreditation_uuid: 
+  subject_uuid: 
+  framework: hipaa | dod_il4 | 
+  contract_event_type: signed | amended | terminated | renewal_due | renewed
+  contract_id: 
+  effective_date: 
+  dcm_action_taken: activated | pending_review | revoked | none
+```
+
+### 20.2 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `EVT-ACM-001` | `accreditation.status_changed` events with `action_taken: immediate_revocation` are non-suppressable โ€” they are delivered to Compliance Team and Platform Admin regardless of notification preferences. |
+| `EVT-ACM-002` | `accreditation.verification_stale` urgency is profile-governed: `low` for dev/standard; `medium` for prod; `high` for fsi/sovereign. |
+
+---
+
+
 ## 19. Event Type Quick Reference
 
 ```
@@ -784,3 +841,14 @@ auth.provider_failover
 ---
 
 *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+### Additional Event Types
+
+| Event Type | Description | Key Fields | Consumers |
+|------------|-------------|-----------|----------|
+| `entity.deleted` | An entity has been fully decommissioned and removed from inventory | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `entity.state_transition` | An entity lifecycle state has changed (e.g., OPERATIONAL โ†’ SUSPENDED) | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `group.deleted` | A DCMGroup has been deleted | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `group.member_added` | A member (actor or entity) has been added to a DCMGroup | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `group.member_removed` | A member (actor or entity) has been removed from a DCMGroup | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `authorization.granted` | A cross-tenant authorization has been granted | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
diff --git a/content/docs/architecture/data-model/examples.md b/content/docs/architecture/data-model/examples.md
index 4ac0888..7b365de 100644
--- a/content/docs/architecture/data-model/examples.md
+++ b/content/docs/architecture/data-model/examples.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Worked Examples"
-type: docs
-weight: 4
----
+# DCM Data Model โ€” Worked Examples
 
-> **โš ๏ธ Active Development Notice**
->
-> Examples in this document are illustrative of the intended architecture. YAML structures, field names, and sequences represent the design intent and will be refined as implementation proceeds.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Reference Examples
diff --git a/content/docs/architecture/data-model/federated-contribution-model.md b/content/docs/architecture/data-model/federated-contribution-model.md
index 3d8454c..1ae4c2b 100644
--- a/content/docs/architecture/data-model/federated-contribution-model.md
+++ b/content/docs/architecture/data-model/federated-contribution-model.md
@@ -1,14 +1,5 @@
----
-title: "Federated Contribution Model"
-type: docs
-weight: 28
----
+# DCM Data Model โ€” Federated Contribution Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Read This First for Multi-User Data Governance
diff --git a/content/docs/architecture/data-model/foundations.md b/content/docs/architecture/data-model/foundations.md
index d543339..07eec14 100644
--- a/content/docs/architecture/data-model/foundations.md
+++ b/content/docs/architecture/data-model/foundations.md
@@ -1,14 +1,5 @@
----
-title: "DCM Foundational Abstractions"
-type: docs
-weight: 0
----
+# DCM โ€” Foundational Abstractions
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Foundation โ€” Read This First
diff --git a/content/docs/architecture/data-model/four-states.md b/content/docs/architecture/data-model/four-states.md
index 1ba93be..1dfe700 100644
--- a/content/docs/architecture/data-model/four-states.md
+++ b/content/docs/architecture/data-model/four-states.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” The Four States"
-type: docs
-weight: 2
----
+# DCM Data Model โ€” The Four States
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/architecture/data-model/governance-matrix.md b/content/docs/architecture/data-model/governance-matrix.md
index d5c7f79..43f300c 100644
--- a/content/docs/architecture/data-model/governance-matrix.md
+++ b/content/docs/architecture/data-model/governance-matrix.md
@@ -1,14 +1,5 @@
----
-title: "Unified Governance Matrix"
-type: docs
-weight: 27
----
+# DCM Data Model โ€” Unified Governance Matrix
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -40,7 +31,7 @@ The Unified Governance Matrix is the **single, declarative, multi-dimensional co
 
 Previous DCM documents established several overlapping control mechanisms: the Data/Capability Authorization Matrix (doc 26), sovereignty constraints in federation tunnels (doc 22), BBQ-001 sovereignty checks (doc 14), and profile-governed data boundaries. The Governance Matrix unifies all of these into a single model with a single evaluation algorithm and a single enforcement point.
 
-**This document supersedes** Section 4 of doc 26 (Data/Capability Authorization Matrix) for structural purposes. The accreditation model (Sections 2-3 of doc 26) and the zero trust interaction model (Section 5 of doc 26) remain current โ€” the Governance Matrix consumes them as inputs.
+The accreditation model and data classification model from doc 26 are inputs to the evaluation algorithm defined here. The accreditation model (Sections 2-3 of doc 26) and the zero trust interaction model (Section 5 of doc 26) remain current โ€” the Governance Matrix consumes them as inputs.
 
 **Key properties of the Governance Matrix:**
 
diff --git a/content/docs/architecture/data-model/information-providers-advanced.md b/content/docs/architecture/data-model/information-providers-advanced.md
index 191eea5..52c359f 100644
--- a/content/docs/architecture/data-model/information-providers-advanced.md
+++ b/content/docs/architecture/data-model/information-providers-advanced.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Information Providers: Confidence Scoring, Authority, and Conflict Resolution"
-type: docs
-weight: 21
----
+# DCM Data Model โ€” Information Providers: Confidence Scoring, Authority, and Conflict Resolution
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Information Providers](10-information-providers.md) | [Policy Organization](14-policy-profiles.md) | [Universal Audit Model](16-universal-audit.md) | [DCM Federation](22-dcm-federation.md)
diff --git a/content/docs/architecture/data-model/information-providers.md b/content/docs/architecture/data-model/information-providers.md
index ca66ee0..f5d052d 100644
--- a/content/docs/architecture/data-model/information-providers.md
+++ b/content/docs/architecture/data-model/information-providers.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Information Providers"
-type: docs
-weight: 10
----
+# DCM Data Model โ€” Information Providers
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/architecture/data-model/ingestion-model.md b/content/docs/architecture/data-model/ingestion-model.md
index 0f85688..7ca22af 100644
--- a/content/docs/architecture/data-model/ingestion-model.md
+++ b/content/docs/architecture/data-model/ingestion-model.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Ingestion Model"
-type: docs
-weight: 13
----
+# DCM Data Model โ€” Ingestion Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md)
@@ -33,7 +24,6 @@ weight: 13
 
 The DCM Ingestion Model is the **unified mechanism for bringing entities that exist outside DCM's lifecycle control into DCM's governance model**. It applies to three distinct sources:
 
-- **V1 Migration** โ€” entities from a DCM V1 deployment that predate the mandatory Tenant model
 - **Brownfield Discovery** โ€” entities discovered by a Service Provider that already exist in the infrastructure but are unknown to DCM
 - **Manual Import** โ€” entities imported from external systems (CMDBs, spreadsheets, legacy records) during onboarding
 
@@ -51,7 +41,7 @@ All three sources follow the same pattern: ingest, enrich, and promote. The same
 
 ## 2. Design Principles
 
-**Unified model โ€” minimum variance.** V1 migration and brownfield ingestion are the same fundamental operation. One model, one audit record structure, one set of governance policies.
+**Unified model โ€” minimum variance.** brownfield ingestion and brownfield ingestion are the same fundamental operation. One model, one audit record structure, one set of governance policies.
 
 **Non-blocking.** Entities that cannot be immediately assigned a Tenant do not block migration or discovery. They land in the `__transitional__` Tenant and are resolved progressively. Migration does not require every entity to be assigned before any entity can proceed.
 
@@ -145,11 +135,11 @@ ingestion_record:
   resource_entity_uuid: 
   ingestion_timestamp: 
 
-  ingestion_source: 
+  ingestion_source: 
 
-  # V1 migration fields (when ingestion_source: v1_migration)
-  v1_identifier: 
-  v1_metadata_snapshot: 
+  # brownfield ingestion fields (when ingestion_source: legacy_import)
+  legacy_identifier: 
+  legacy_metadata_snapshot: 
 
   # Brownfield discovery fields (when ingestion_source: brownfield_discovery)
   discovered_state_uuid: 
@@ -202,7 +192,7 @@ When DCM ingests an entity, it attempts auto-assignment to a real Tenant using t
 |--------|-----------|-------------|
 | Explicit ownership metadata | High | Business unit, cost center, or team tag on the resource maps unambiguously to a Tenant |
 | Resource group membership | High | Resource belongs to a group that maps to a known Tenant |
-| Request history | High | V1 request record identifies the requesting team, which maps to a Tenant |
+| Request history | High | Legacy record identifies the requesting team, which maps to a Tenant |
 | Network / location context | Medium | Resource's location, VLAN, or network segment maps to a Tenant by convention |
 | Naming convention | Medium | Resource name matches a known Tenant naming pattern |
 | Provider context | Medium | Resource was provisioned by a known provider associated with a Tenant |
@@ -212,68 +202,6 @@ Multiple signals can be combined. If signals conflict, the higher-confidence sig
 
 ---
 
-## 7. V1 Migration
-
-### 7.1 Overview
-
-V1 resources have no `tenant_uuid`. V2 requires one (`TEN-001`). The V1 migration process uses the ingestion model to assign every V1 resource a Tenant before it can participate in V2 operations.
-
-### 7.2 Resource Categories
-
-| Category | Description | Assignment Path |
-|----------|-------------|----------------|
-| **Auto-assignable** | Clear ownership signals โ€” resource group, business unit, request history | Auto-assigned during migration analysis pass |
-| **Manually assignable** | Ambiguous signals โ€” multiple possible owners, or medium-confidence signals only | Surfaced in admin assignment queue |
-| **Orphaned** | No signals โ€” no ownership data available | Assigned to `__transitional__` |
-
-### 7.3 Migration Flow
-
-```
-V1 estate
-  โ”‚
-  โ–ผ  Step 1 โ€” Pre-migration analysis pass
-  โ”‚  Inventory all V1 resources
-  โ”‚  Attempt auto-assignment via signals (Section 6)
-  โ”‚  Classify each resource: auto_assignable | manually_assignable | orphaned
-  โ”‚  Produce migration readiness report
-  โ”‚
-  โ–ผ  Step 2 โ€” Auto-assignment
-  โ”‚  Create or map to existing V2 Tenants
-  โ”‚  Assign auto_assignable resources in bulk
-  โ”‚  Create ingestion_record per resource (ingestion_source: v1_migration)
-  โ”‚  State: INGESTED โ†’ ENRICHING (for auto-assigned)
-  โ”‚
-  โ–ผ  Step 3 โ€” Manual assignment queue
-  โ”‚  manually_assignable resources surfaced in admin UI
-  โ”‚  Administrators review and assign Tenants
-  โ”‚  Each assignment recorded in enrichment_history
-  โ”‚
-  โ–ผ  Step 4 โ€” Transitional fallback
-  โ”‚  orphaned resources โ†’ __transitional__ Tenant
-  โ”‚  ingestion_record.assignment_method: transitional
-  โ”‚  ingestion_record.ingestion_confidence: low
-  โ”‚  Governance timer starts
-  โ”‚
-  โ–ผ  Step 5 โ€” Enrichment and promotion
-  โ”‚  Relationships established, missing fields populated
-  โ”‚  Each entity reviewed and promoted when complete
-  โ”‚  State: ENRICHING โ†’ PROMOTED โ†’ OPERATIONAL
-  โ”‚
-  โ–ผ  Migration complete when __transitional__ Tenant is empty
-```
-
-### 7.4 Migration System Policies
-
-| Policy | Rule |
-|--------|------|
-| `ING-001` | Every entity ingested into V2 from V1 must be assigned to exactly one Tenant โ€” either a real Tenant or `__transitional__` โ€” before it is eligible for new V2 requests |
-| `ING-002` | Entities in `INGESTED` or `ENRICHING` state may not be the parent resource for a new allocated resource claim |
-| `ING-003` | The `__transitional__` Tenant is system-managed and cannot be deleted, renamed, or used for new resource provisioning |
-| `ING-004` | Every ingested entity must carry an `ingestion_record` in its provenance chain |
-| `ING-005` | Entities in `__transitional__` for longer than `max_residency_days` must trigger the configured escalation action |
-
----
-
 ## 8. Brownfield Ingestion
 
 ### 8.1 Overview
@@ -351,7 +279,7 @@ Ingestion interacts with the Four States model as follows:
 
 | Ingestion Source | States Involved | Flow |
 |-----------------|----------------|------|
-| V1 Migration | Intent โ†’ Requested โ†’ (no Realized yet) | V1 records treated as incomplete Requested State; migration creates minimal Realized State |
+| brownfield ingestion | Intent โ†’ Requested โ†’ (no Realized yet) | Legacy records treated as incomplete Requested State; migration creates minimal Realized State |
 | Brownfield Discovery | Discovered โ†’ Realized | Discovered State is promoted to Realized State at promotion |
 | Manual Import | None initially | Entity stub created; no prior state records; Realized State created at promotion from import data |
 
@@ -391,7 +319,6 @@ In all cases: once an entity reaches `PROMOTED`, it has a Realized State record
 - **Four States** โ€” Discovered State is the entry point for brownfield ingestion; Realized State is the output of promotion
 - **Brownfield** โ€” existing infrastructure not yet under DCM lifecycle management
 - **Drift Detection** โ€” begins for brownfield entities at the moment of promotion
-- **V1 Migration** โ€” migration of pre-Tenant DCM V1 entities to V2 using the ingestion model
 - **Greening the Brownfield** โ€” the progressive process of bringing unmanaged infrastructure under DCM lifecycle control
 
 
diff --git a/content/docs/architecture/data-model/internal-component-auth.md b/content/docs/architecture/data-model/internal-component-auth.md
index cbd1252..a82a9ac 100644
--- a/content/docs/architecture/data-model/internal-component-auth.md
+++ b/content/docs/architecture/data-model/internal-component-auth.md
@@ -1,8 +1,4 @@
----
-title: "Internal Component Authentication"
-type: docs
-weight: 36
----
+# DCM Data Model โ€” Internal Component Authentication
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Zero Trust Internal Auth
diff --git a/content/docs/architecture/data-model/itsm-integration.md b/content/docs/architecture/data-model/itsm-integration.md
index d7f1419..df15199 100644
--- a/content/docs/architecture/data-model/itsm-integration.md
+++ b/content/docs/architecture/data-model/itsm-integration.md
@@ -1,8 +1,4 @@
----
-title: "ITSM Integration"
-type: docs
-weight: 42
----
+# DCM Data Model โ€” ITSM Integration
 
 **Document Status:** ๐Ÿ”„ In Progress
 **Document Type:** Architecture Reference โ€” ITSM Provider Type and ITSM Policy Type
@@ -55,7 +51,7 @@ DCM lifecycle event fires (e.g. request.dispatched)
 ITSM system approves a change record
   โ”‚
   โ–ผ ITSM system calls DCM API (via webhook or polling)
-  โ”‚   POST /api/v1/admin/approvals/{uuid}/vote
+  โ”‚   POST /api/v1/admin/approvals/{uuid}:vote
   โ”‚   { decision: "approve", recorded_via: "servicenow",
   โ”‚     external_reference: "CHG0012345" }
   โ”‚
@@ -186,7 +182,7 @@ itsm_provider_registration:
     
   # CAB approval โ†’ DCM vote mapping
   inbound_approval:
-    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote"
+    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}:vote"
     trigger_on: "change_request.state โ†’ 'Approved'"
     decision_field: "state"
     decision_map:
@@ -212,7 +208,7 @@ itsm_provider_registration:
 ServiceNow Change Advisory Board approves CHG0012345
   โ”‚
   โ–ผ ServiceNow Business Rule fires on state change โ†’ "Approved"
-  โ”‚   Calls DCM webhook: POST /api/v1/admin/approvals/{uuid}/vote
+  โ”‚   Calls DCM webhook: POST /api/v1/admin/approvals/{uuid}:vote
   โ”‚   Headers: X-ServiceNow-Signature: 
   โ”‚   Body: { decision: "approve", recorded_via: "servicenow",
   โ”‚           external_reference: "CHG0012345" }
@@ -255,7 +251,7 @@ itsm_provider_registration:
     labels: ["dcm-automated", "{tenant_handle}"]
     
   inbound_approval:
-    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote"
+    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}:vote"
     trigger_on: "issue.status โ†’ 'Approved'"
     decision_map:
       "Approved": "approve"
diff --git a/content/docs/architecture/data-model/kessel-integration-evaluation.md b/content/docs/architecture/data-model/kessel-integration-evaluation.md
new file mode 100644
index 0000000..da4a86c
--- /dev/null
+++ b/content/docs/architecture/data-model/kessel-integration-evaluation.md
@@ -0,0 +1,470 @@
+# DCM โ€” Kessel Integration Evaluation
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” For Discussion
+**Document Type:** Integration Evaluation โ€” Pre-Implementation
+**Purpose:** This document evaluates the potential integration of DCM with the [Kessel project](https://github.com/project-kessel) for identity/access management and resource inventory. It is intended as a basis for discussion with the Kessel development team. **No architectural changes should be made to DCM based on this document until alignment with the Kessel team is confirmed.**
+
+**Related Documents:** [Auth Providers](19-auth-providers.md) | [Universal Group Model](15-universal-groups.md) | [Entity Relationships](09-entity-relationships.md) | [Four States](02-four-states.md) | [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Control Plane Components](25-control-plane-components.md) | [Provider Callback Authentication](43-provider-callback-auth.md)
+
+**Related Projects:** [project-kessel](https://github.com/project-kessel) | [SpiceDB](https://github.com/authzed/spicedb) | [Google Zanzibar](https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/)
+
+---
+
+## 1. Executive Summary
+
+Kessel is a Red Hat project providing two capabilities: **Kessel Relations** (Relationship-Based Access Control built on SpiceDB, a Google Zanzibar implementation) and **Kessel Asset Inventory** (a hybrid cloud resource state tracking service with a common Protobuf/gRPC API).
+
+DCM has architecturally similar needs in both areas. The evaluation concludes:
+
+- **Kessel Relations** has strong alignment with DCM's access control requirements. The permission model maps cleanly, and the operational benefits โ€” Zanzibar-style consistency, scalable graph traversal, shared source of truth across Red Hat products โ€” are meaningful. Integration path exists via DCM's Auth Provider abstraction.
+
+- **Kessel Inventory** has partial alignment with DCM's Discovered State store. The fit is real but narrower than it might appear: Kessel Inventory is a current-state snapshot system; DCM's inventory is a four-state lifecycle model with field-level provenance, drift detection, and append-only audit. Integration path exists via DCM's Storage Provider abstraction.
+
+**Recommended next step:** Discussion with the Kessel development team to validate assumptions, confirm schema extensibility for DCM-specific resource types, and understand the Kessel Relations API stability and sovereign/air-gapped deployment model.
+
+---
+
+## 2. What Kessel Provides
+
+### 2.1 Kessel Relations
+
+Kessel Relations is an authorization service built on [SpiceDB](https://github.com/authzed/spicedb), which implements the [Google Zanzibar](https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/) consistent global authorization model.
+
+**Core model โ€” Relationship-Based Access Control (ReBAC):**
+- Resources and subjects are defined in a typed schema
+- Relationships between subjects and resources are stored as tuples: `subject:X relation:Y object:Z`
+- Permissions are computed by evaluating the relationship graph: "does user X have permission `submit_request` on tenant T?" traverses all paths from X to T through groups, roles, and other relationships
+- Transitive relationships are handled natively: if user X is a member of group G, and group G has `admin` on tenant T, X inherits `admin` on T
+
+**Zanzibar consistency model:**
+- Snapshot reads: consistent reads at a point in time
+- Zookie tokens: causality tokens that guarantee "read your own writes" without requiring full global linearizability โ€” after writing a relationship tuple, the response includes a zookie; subsequent reads with that zookie are guaranteed to observe the write
+
+**gRPC API surface (from the Kessel project):**
+- `CheckPermission(subject, permission, resource)` โ†’ allow/deny
+- `LookupResources(subject, permission, resource_type)` โ†’ list of resources subject has permission on
+- `LookupSubjects(resource, permission, subject_type)` โ†’ list of subjects that have permission on resource
+- `WriteRelationships(tuples)` โ†’ write relationship tuples
+- `DeleteRelationships(filter)` โ†’ remove relationship tuples
+
+### 2.2 Kessel Asset Inventory
+
+Kessel Asset Inventory is a resource tracking service designed to provide a unified inventory view across hybrid cloud infrastructure โ€” OpenShift clusters, RHEL systems, edge devices, and other Red Hat-managed resources.
+
+**Core model:**
+- Resources are described using a common Protobuf schema with a typed `ResourceType` and a `Spec` for type-specific fields
+- Current state is tracked as an upsertable snapshot โ€” last-write wins
+- gRPC streaming API for push (providers send state updates) and pull (consumers query current state)
+- Integration with Kessel Relations for auth-filtered inventory queries: "what resources of type X does subject Y have access to?"
+
+**Intended use case:** Giving tools like ACM (Advanced Cluster Management), Insights, and the Hybrid Cloud Console a single query surface for "what exists across my estate?"
+
+---
+
+## 3. DCM's Current Model โ€” What Needs to Be Understood
+
+Before evaluating integration, it is important to characterize what DCM already has in both areas.
+
+### 3.1 DCM's Authorization Model
+
+DCM's current authorization model has five components working together:
+
+**Auth Providers** (doc 19) โ€” DCM delegates authentication to registered Auth Providers (LDAP, OIDC, FreeIPA, Active Directory, mTLS). Auth Providers are registered through the standard Provider contract. Multiple Auth Providers can be active simultaneously. Auth Providers return: authenticated actor identity, group memberships, roles.
+
+**Universal Group Model** (doc 15) โ€” DCM groups (`DCMGroup`) are typed by `group_class`. The classes relevant to authorization:
+- `tenant_boundary` โ€” the ownership and isolation boundary; every resource entity belongs to exactly one tenant
+- `cross_tenant_authorization` โ€” the formal mechanism for Tenant A to grant Tenant B access to a specific resource
+- `policy_collection` โ€” groups that activate policy sets
+- DCMGroup membership is the basis for role resolution and policy application
+
+**RBAC via role mapping** โ€” Auth Providers map external groups to DCM roles (`consumer`, `platform_admin`, `sre`, etc.). The Policy Engine uses roles + group membership to evaluate access.
+
+**Five-check boundary model** (doc 26) โ€” Every interaction crosses five checks in sequence: identity verification โ†’ authorization โ†’ accreditation โ†’ data/capability matrix โ†’ sovereignty. Checks 1 and 2 are RBAC. Checks 3โ€“5 are DCM-specific and involve accreditation records, data classification, and sovereignty zones.
+
+**Cross-tenant authorization records** โ€” When Tenant A grants Tenant B access to a resource, a `cross_tenant_authorization` DCMGroup is created. The Policy Engine checks for the existence of this record when evaluating cross-tenant requests.
+
+**What DCM asks for in authorization decisions:**
+1. Does actor X have role Y within tenant T?
+2. What catalog items is actor X allowed to see? (RBAC-filtered list)
+3. Can actor X perform operation O on resource R? (role + tenant ownership)
+4. Does tenant T have a cross-tenant authorization to use resource R owned by tenant T2?
+5. Is actor X a member of DCMGroup G with the required quorum? (approval gates, `authorized` tier)
+
+### 3.2 DCM's Inventory Model
+
+DCM's inventory is the **Four States model** (doc 02). This is meaningfully different from a general-purpose resource inventory.
+
+**Intent State** โ€” The consumer's declared desired state. Stored as a GitOps artifact (PR-based workflow). Immutable after creation. Not a snapshot โ€” it is the authoritative record of what was requested and why.
+
+**Requested State** โ€” The assembled payload after layer enrichment, policy evaluation, and placement resolution. Write-once. Contains the full data model payload that was dispatched to the provider, including field-level provenance tracing every value back to its source.
+
+**Realized State** โ€” An append-only event stream of what the provider actually built. Every realization event is a new record โ€” not an upsert. Contains field-level provenance from the provider. The relationship between a Realized State record and its corresponding Requested State record is explicit and mandatory.
+
+**Discovered State** โ€” An ephemeral snapshot of what the provider currently reports as existing, obtained through active discovery polling. Used by the Drift Reconciliation Component to compare against Realized State.
+
+**What DCM asks for in inventory decisions:**
+1. What is the current lifecycle state of entity UUID X? (Realized State read)
+2. What resources does tenant T own? (indexed query over Realized State)
+3. What entities have relationship R to entity X? (Entity Relationship Graph, doc 09)
+4. What is the field-level provenance of field F on entity X? (Realized State metadata)
+5. What entities are currently drifted? (Drift Record Store, DRC component output)
+6. What did we discover vs what do we have as realized? (Drift comparison)
+7. What happened to entity X over its full lifecycle? (Audit Store, time-indexed)
+
+---
+
+## 4. Integration Analysis
+
+### 4.1 Kessel Relations โ€” Authorization Backend
+
+#### Mapping DCM's Permission Model to SpiceDB
+
+DCM's five authorization questions map to SpiceDB as follows:
+
+```
+# Proposed SpiceDB schema for DCM
+definition user {}
+
+definition group {
+  relation member:      user | group#member
+  relation parent_group: group
+  permission member     = member + parent_group->member
+}
+
+definition tenant {
+  relation member:      user | group#member
+  relation admin:       user | group#member
+  relation platform_admin: user | group#member
+  permission submit_request  = member + admin + platform_admin
+  permission manage_resources = admin + platform_admin
+  permission administer       = platform_admin
+}
+
+definition resource {
+  relation owner_tenant:    tenant
+  relation authorized_tenant: tenant    # cross-tenant authorization
+  relation viewer:          user | group#member
+  permission read   = owner_tenant->member + authorized_tenant->member + viewer
+  permission modify = owner_tenant->admin
+  permission decommission = owner_tenant->admin
+}
+
+definition dcm_group {
+  relation member:    user | group#member
+  relation quorum_threshold: integer  # NOTE: see Section 4.1.2
+}
+```
+
+**Question 1** (does actor X have role Y in tenant T?) โ†’ `CheckPermission(user:X, permission:submit_request, tenant:T)`
+
+**Question 2** (what catalog items can actor X see?) โ†’ `LookupResources(user:X, permission:read, resource_type:catalog_item)`
+
+**Question 3** (can actor X do operation O on resource R?) โ†’ `CheckPermission(user:X, permission:modify, resource:R)`
+
+**Question 4** (does tenant T have cross-tenant authorization on resource R?) โ†’ `CheckPermission(tenant:T#member, permission:read, resource:R)` โ€” satisfied if `authorized_tenant` relationship exists
+
+**Question 5** (approval gate quorum) โ€” **Does not map cleanly to SpiceDB.** See Section 4.1.2.
+
+#### 4.1.2 Approval Gate Quorum โ€” The Gap
+
+DCM's `authorized` tier approval requires N of M members of a declared DCMGroup to record decisions before an operation proceeds. SpiceDB is a membership and permission graph โ€” it answers "does this subject have this permission?" but it does not count decisions or track quorum state across time.
+
+**Resolution:** The approval gate workflow stays in DCM's Policy Engine regardless of Kessel integration. Kessel Relations handles who *can* approve (membership in the DCMGroup); DCM's Approval Store tracks who *has* approved and whether quorum is reached.
+
+This is a clean boundary: Kessel answers the structural question ("is this actor authorized to vote?"); DCM answers the state question ("how many valid votes have been recorded?").
+
+#### 4.1.3 DCM's Entity Relationship Graph is NOT an Authorization Graph
+
+This is a critical distinction. DCM's entity relationships โ€” `requires`, `constituent`, `shareable`, `allocated_from`, `peer` โ€” are **operational relationships between infrastructure resources**, not access control relationships. They express: "VM X requires Storage Y", "Composite C has constituent VM X."
+
+These must **not** be stored in Kessel Relations. They are:
+- Semantically different from access control (lifecycle implications, not permissions)
+- DCM-specific (not meaningful to any other system consuming Kessel)
+- Owned by DCM's entity lifecycle model
+
+DCM's Entity Relationship Graph (doc 09) remains entirely in DCM regardless of Kessel integration.
+
+#### 4.1.4 Checks 3โ€“5 of the Five-Check Boundary Model
+
+DCM's five-check boundary model (identity โ†’ authorization โ†’ accreditation โ†’ data matrix โ†’ sovereignty) maps to Kessel Relations only for checks 1 and 2. Checks 3โ€“5 are DCM-specific:
+
+- **Accreditation** (check 3): Does the target provider hold the required accreditation for the data classification present? This involves DCM's Accreditation Registry and is not a subject/permission/resource question.
+- **Data/Capability Matrix** (check 4): Is each field permitted to cross this boundary given its classification? This involves DCM's Governance Matrix policies.
+- **Sovereignty** (check 5): Is the target endpoint within the sovereignty boundary? This involves DCM's Sovereignty Zone declarations.
+
+None of checks 3โ€“5 can be delegated to Kessel Relations. They remain in DCM's Policy Engine.
+
+#### 4.1.5 Integration Path via Auth Provider Abstraction
+
+DCM's Auth Provider abstraction (doc 19) is the natural integration point. Kessel Relations would register as a DCM Auth Provider or Policy Provider:
+
+```yaml
+kessel_relations_auth_provider:
+  provider_type: auth_provider
+  auth_mode: kessel_rebac
+  endpoint: https://kessel-relations.internal:9000
+  schema_ref: 
+  
+  # What this provider handles:
+  handles:
+    - check_permission       # CheckPermission calls
+    - lookup_resources       # LookupResources calls
+    - lookup_subjects        # LookupSubjects calls
+  
+  # What stays in DCM's Policy Engine:
+  does_not_handle:
+    - accreditation_checks
+    - data_classification_matrix
+    - sovereignty_checks
+    - approval_gate_quorum
+```
+
+DCM's Policy Engine calls the Kessel Relations provider for authorization questions (checks 1 and 2) and evaluates checks 3โ€“5 internally. The five-check sequence is preserved; only the implementation of checks 1โ€“2 changes.
+
+**Zookie handling:** DCM's API Gateway must thread zookie tokens through the request lifecycle: when a relationship is written (e.g., a new cross-tenant authorization is created), the resulting zookie is stored and used for subsequent permission checks in the same request context, guaranteeing consistency.
+
+---
+
+### 4.2 Kessel Inventory โ€” Discovered State Store
+
+#### 4.2.1 The Fit
+
+Of DCM's four stores, **Discovered State** is the only one Kessel Inventory could plausibly replace. The reasons:
+
+- Discovered State is the most ephemeral store โ€” it is overwritten on each discovery cycle
+- Discovered State does not require immutability or append-only semantics โ€” it represents "what the provider reports right now"
+- Discovered State is the "current state of infrastructure" โ€” exactly what Kessel Inventory is designed to track
+- Other Red Hat tools consuming Kessel Inventory would benefit from seeing the same discovered state that DCM uses for drift detection
+
+The other three stores โ€” Intent, Requested, and Realized โ€” **cannot** be replaced by Kessel Inventory:
+- Intent and Requested State require GitOps semantics (PR workflow, immutability, version history)
+- Realized State requires append-only event stream semantics with field-level provenance and hash chain integrity
+- None of DCM's lifecycle or audit requirements are in scope for Kessel Inventory
+
+#### 4.2.2 The Schema Alignment Question
+
+DCM's Discovered State uses the same unified data model format as Realized State โ€” the DCM Resource Type Spec schema. Kessel Inventory uses a Protobuf-defined common resource schema.
+
+For standard resource types (Compute, Network, Storage that map to well-known infrastructure concepts), the alignment is likely achievable. For DCM-specific resource types (Automation.AnsiblePlaybook, Platform.KubernetesCluster, custom org-defined types), schema extension or mapping is required.
+
+**Open question for Kessel team:** How extensible is the Kessel Inventory resource type schema? Can DCM register custom resource types? Is there a type registry mechanism analogous to DCM's Resource Type Registry?
+
+#### 4.2.3 Drift Detection Logic Stays in DCM
+
+Kessel Inventory is a state store, not a drift detection system. Even if DCM uses Kessel Inventory as the Discovered State store, the Drift Reconciliation Component (doc 25, DRC domain) remains entirely in DCM:
+
+- DRC queries Kessel Inventory for current discovered state
+- DRC compares discovered state against DCM's Realized State
+- DRC classifies differences by field criticality and change magnitude
+- DRC produces Drift Records with SECURITY_DEGRADATION, BROKEN_REFERENCE, UNSANCTIONED_CHANGE classifications
+- DRC writes Drift Records to DCM's Drift Record Store
+
+Kessel Inventory's role is purely as the data source for the "what currently exists" side of the comparison. The intelligence stays in DCM.
+
+#### 4.2.4 Integration Path via Storage Provider Abstraction
+
+DCM's Storage Provider abstraction (doc 11) is the natural integration point. The Discovered Store would be implemented as a `storage_sub_type: snapshot_store` Storage Provider backed by Kessel Inventory:
+
+```yaml
+kessel_inventory_storage_provider:
+  provider_type: storage_provider
+  storage_sub_type: snapshot_store
+  backend: kessel_inventory
+  endpoint: https://kessel-inventory.internal:9001
+  
+  # DCM uses this provider for:
+  used_for: discovered_state
+  
+  # Write contract: provider calls POST /api/v1/instances/{id}/status
+  # which DCM translates to Kessel Inventory upsert
+  write_model: upsert_current_state
+  
+  # Read contract: DRC queries Kessel for discovered state
+  read_model: streaming_query_by_type_and_tenant
+```
+
+This means the Kessel Inventory integration requires **no changes to DCM's data model** โ€” only a new Storage Provider implementation. The Drift Reconciliation Component calls the same Discovered State Store interface; the underlying implementation happens to be Kessel Inventory.
+
+---
+
+## 5. Deployment and Sovereignty Considerations
+
+### 5.1 Air-Gapped and Sovereign Deployments
+
+DCM's `sovereign` profile requires air-gapped operation with no external dependencies. Any Kessel integration must support:
+
+- Local/on-premises Kessel deployment (not cloud-hosted)
+- Offline operation when Kessel is temporarily unavailable (cached authorization decisions for read-only operations)
+- mTLS between DCM and Kessel instances
+
+**Open question for Kessel team:** What is Kessel's deployment model for sovereign/air-gapped environments? Is there a supported on-premises deployment path? What is the operational footprint?
+
+### 5.2 Multi-Instance Federation
+
+DCM supports federation between multiple DCM instances (doc 22). A federated deployment may have multiple Kessel Relations instances (one per region or sovereignty zone) or a single shared instance.
+
+**Open question for Kessel team:** How does Kessel Relations handle multi-region replication? Can SpiceDB schema and relationship data be replicated across sovereignty boundaries? What are the consistency guarantees in a federated topology?
+
+### 5.3 Failure Mode Analysis
+
+If Kessel Relations is unavailable, DCM cannot evaluate authorization checks 1โ€“2 of the five-check model, which means DCM cannot process any requests. This is a critical dependency.
+
+**Required mitigation strategies:**
+- Read-through cache for CheckPermission results (short TTL, profile-governed)
+- Circuit breaker: if Kessel is unavailable for >N consecutive checks, DCM enters a safe-deny mode (no new requests accepted) rather than a fail-open mode
+- Kessel Relations HA deployment is a prerequisite, not optional
+
+**Open question for Kessel team:** What HA and disaster recovery patterns are recommended for production Kessel Relations deployments?
+
+---
+
+## 6. Questions for the Kessel Team
+
+The following questions should be addressed before any integration work begins:
+
+### 6.1 Kessel Relations
+
+| # | Question | Why It Matters |
+|---|----------|----------------|
+| 1 | What is the current API stability level of the Kessel Relations gRPC API? Are breaking changes expected? | DCM needs a stable contract to build against |
+| 2 | Does Kessel Relations support on-premises / air-gapped deployment? What is the operational footprint? | Required for DCM's `sovereign` profile |
+| 3 | How does the SpiceDB schema evolve? Is there a migration path when the DCM permission model changes? | Schema evolution is a production concern |
+| 4 | Can Kessel Relations store relationships at the scale DCM requires? How many relationship tuples per tenant at what query latency? | DCM may have thousands of cross-tenant authorization records per deployment |
+| 5 | How does Kessel handle the zookie (consistency token) lifecycle? Are zookies scoped to a namespace/tenant, or global? | Relevant to DCM's multi-tenant isolation model |
+| 6 | Is Kessel Relations multi-tenant natively, or does DCM need to namespace its SpiceDB schema? | Critical for DCM's tenant isolation requirements |
+| 7 | What is the intended integration pattern for other Red Hat products (ACM, Insights)? How would DCM's usage interoperate? | Kessel's value to DCM is partly the shared source of truth across RH products |
+| 8 | Does Kessel Relations have a concept equivalent to DCM's "cross-tenant authorization"? How are trust grants between tenants modeled? | Core to DCM's resource sharing model |
+
+### 6.2 Kessel Inventory
+
+| # | Question | Why It Matters |
+|---|----------|----------------|
+| 9 | How extensible is Kessel Inventory's resource type schema? Can DCM register custom resource types? | DCM has domain-specific resource types not in Kessel's default schema |
+| 10 | What is the write model? Last-write-wins upsert, or versioned? Does Kessel Inventory support the discovered state pattern (full overwrite on each discovery cycle)? | DCM's Discovered Store is a full-replacement snapshot per discovery cycle |
+| 11 | How does Kessel Inventory integrate with Kessel Relations for auth-filtered queries? Is the integration already built, or planned? | Core to the value of using Kessel Inventory |
+| 12 | What is the data retention model? Does Kessel Inventory keep history or only current state? | DCM needs "current state" only for Discovered State; history is in DCM's Audit Store |
+| 13 | What is the API stability level for Kessel Inventory? | Same concern as #1 for Relations |
+| 14 | Is there a reference implementation of a Kessel Inventory provider for a Kubernetes/OpenShift resource type? | DCM would follow this pattern for its Service Providers |
+
+### 6.3 Joint Architecture Questions
+
+| # | Question | Why It Matters |
+|---|----------|----------------|
+| 15 | Is the Kessel project open to DCM contributing Resource Type definitions and SpiceDB schema extensions to the upstream? | Reduces divergence risk; benefits broader community |
+| 16 | How does Kessel handle sovereign data โ€” data that must not cross jurisdictional boundaries? | Critical for DCM's sovereignty model |
+| 17 | What is the recommended pattern for bootstrapping the Kessel-DCM trust relationship? (mTLS? OIDC? Service account?) | Required for DCM's zero-trust model |
+| 18 | Does Kessel have a compatibility matrix for Red Hat platform versions (OpenShift, RHEL)? | DCM targets the same platforms |
+
+---
+
+## 7. Proposed Integration Architecture (Pending Kessel Alignment)
+
+This section describes the target architecture **conditional on positive answers to the questions in Section 6**. It should not be implemented until validated with the Kessel team.
+
+### 7.1 Kessel Relations as DCM Auth Provider
+
+```
+DCM Request Pipeline:
+  โ”‚
+  โ–ผ Auth Provider (Kessel Relations):
+  โ”‚   Check 1: identity verification via mTLS certificate
+  โ”‚   Check 2: CheckPermission(actor, operation, tenant/resource) via Kessel Relations gRPC
+  โ”‚            โ† returns allow/deny + zookie token
+  โ”‚
+  โ–ผ DCM Policy Engine (internal):
+  โ”‚   Check 3: Accreditation check (DCM Accreditation Registry)
+  โ”‚   Check 4: Data/Capability Matrix (DCM Governance Matrix)
+  โ”‚   Check 5: Sovereignty check (DCM Sovereignty Zone registry)
+  โ”‚
+  โ–ผ All five checks pass โ†’ request proceeds to layer assembly
+```
+
+**Impact on DCM architecture:**
+- Auth Provider registration: new `auth_mode: kessel_rebac` in doc 19
+- Cross-tenant authorization DCMGroup: writes to both DCM Group Registry AND Kessel Relations tuple store
+- RBAC evaluation: replaced by Kessel Relations CheckPermission call for checks 1โ€“2
+- Group membership sync: DCM Auth Providers (LDAP, OIDC) continue to manage authentication; group memberships are mirrored to Kessel Relations for use in permission evaluation
+
+### 7.2 Kessel Inventory as DCM Discovered State Store
+
+```
+Discovery Cycle:
+  โ”‚
+  โ–ผ Discovery Scheduler triggers provider discovery
+  โ”‚
+  โ–ผ Service Provider returns RealizedStatePayload stream
+  โ”‚   (current state in DCM Unified Data Model format)
+  โ”‚
+  โ–ผ Kessel Inventory Storage Provider:
+  โ”‚   Translates DCM format โ†’ Kessel Inventory Protobuf schema
+  โ”‚   Upserts to Kessel Inventory (replaces prior discovered state)
+  โ”‚
+  โ–ผ Drift Reconciliation Component (unchanged):
+  โ”‚   Queries Kessel Inventory for discovered state
+  โ”‚   Compares against DCM Realized State
+  โ”‚   Produces Drift Records (classification, severity, field detail)
+  โ”‚   Writes Drift Records to DCM Drift Record Store
+```
+
+**Impact on DCM architecture:**
+- Discovered State Store: implement as Storage Provider backed by Kessel Inventory
+- No changes to data model, drift detection logic, or Drift Reconciliation Component
+- Resource type mapping: DCM Resource Type Specs โ†’ Kessel Inventory resource types (new tooling required)
+
+---
+
+## 8. What Does Not Change Regardless of Integration
+
+The following DCM capabilities remain entirely in DCM regardless of how the Kessel integration develops:
+
+| Capability | Why it stays in DCM |
+|-----------|---------------------|
+| Intent State Store (GitOps) | GitOps semantics, PR workflow, immutability โ€” not in scope for Kessel |
+| Requested State Store (write-once) | Assembled payload with full provenance โ€” DCM-specific |
+| Realized State Store (append-only event stream) | Hash-chained, tamper-evident, field-level provenance โ€” DCM-specific |
+| Approval gate quorum tracking | State-tracking across time โ€” Kessel Relations answers membership, not quorum |
+| Five-check boundary model (checks 3โ€“5) | Accreditation, data classification, sovereignty โ€” DCM-specific |
+| Entity Relationship Graph | Operational relationships between resources โ€” not access control |
+| Field-level provenance | Source tracking per field โ€” not in scope for Kessel |
+| Drift detection logic and classification | DRC component โ€” Kessel Inventory is a data source, not a drift engine |
+| Audit trail (hash chain) | Tamper-evident audit โ€” DCM-specific requirement |
+| Resource lifecycle state machine | REQUESTED โ†’ OPERATIONAL โ†’ DECOMMISSIONED โ€” DCM-specific |
+| Policy Engine | GateKeeper, Transformation, Recovery, Orchestration Flow policies โ€” DCM-specific |
+| Authority Tier model | Approval routing โ€” DCM-specific governance model |
+
+---
+
+## 9. System Policies (Proposed โ€” Pending Validation)
+
+These policies should be reviewed and confirmed after Kessel team alignment:
+
+| Policy | Rule |
+|--------|------|
+| `KESSEL-001` | Kessel Relations, if registered as a DCM Auth Provider, handles authorization checks 1 and 2 of the five-check boundary model only. Checks 3โ€“5 remain in DCM's Policy Engine and cannot be delegated. |
+| `KESSEL-002` | DCM's entity relationship graph (operational relationships between infrastructure resources) must never be stored in Kessel Relations. Only access-control relationships (actorโ†’groupโ†’tenantโ†’resource permissions) are stored in Kessel Relations. |
+| `KESSEL-003` | Kessel Inventory, if registered as a DCM Storage Provider for Discovered State, holds only ephemeral current-state snapshots. Intent, Requested, and Realized State stores remain in DCM-managed Storage Providers. |
+| `KESSEL-004` | If Kessel Relations is unavailable, DCM enters safe-deny mode: no new requests are accepted. Fail-open behavior is not permitted under any profile. |
+| `KESSEL-005` | Zookie tokens from Kessel Relations CheckPermission responses must be threaded through the DCM request context to guarantee consistency across authorization checks within the same request. |
+| `KESSEL-006` | Cross-tenant authorization DCMGroups that are backed by Kessel Relations must be written atomically: the DCM Group Registry record and the Kessel Relations tuple must both succeed or both fail. Partial writes are treated as failures. |
+| `KESSEL-007` | DCM sovereign profile deployments require a locally-deployed Kessel instance. Cloud-hosted Kessel is not permitted for sovereign deployments. This requirement must be confirmed as feasible with the Kessel team. |
+
+---
+
+## 10. Open Items Before Integration Can Begin
+
+| # | Item | Owner | Blocking? |
+|---|------|-------|-----------|
+| 1 | Kessel team review of Section 6 questions | Kessel team | Yes |
+| 2 | Kessel Relations API stability confirmation | Kessel team | Yes |
+| 3 | Sovereign/air-gapped deployment validation | Kessel team | Yes (for sovereign profile) |
+| 4 | SpiceDB schema design review for DCM permission model | DCM + Kessel | Yes |
+| 5 | Kessel Inventory resource type extensibility confirmation | Kessel team | Yes (for inventory integration) |
+| 6 | HA/DR pattern review for production Kessel deployment | Kessel team | Yes |
+| 7 | DCM Auth Provider interface extension for `kessel_rebac` mode | DCM team | No (can design in parallel) |
+| 8 | DCM Storage Provider implementation for Kessel Inventory | DCM team | No (can design in parallel) |
+| 9 | Zookie lifecycle management design in DCM request pipeline | DCM team | No (can design in parallel) |
+| 10 | Resource type mapping: DCM Resource Type Specs โ†’ Kessel Inventory schema | DCM + Kessel | No (can design in parallel) |
+
+---
+
+*Document maintained by the DCM Project. For questions, contributions, or to schedule the Kessel alignment session see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/layering-and-versioning.md b/content/docs/architecture/data-model/layering-and-versioning.md
index ac85bc2..97fe0e0 100644
--- a/content/docs/architecture/data-model/layering-and-versioning.md
+++ b/content/docs/architecture/data-model/layering-and-versioning.md
@@ -1,14 +1,5 @@
----
-title: "Layering, Versioning, and Override Precedence"
-type: docs
-weight: 3
----
+# DCM Data Model โ€” Data Layers and the Assembly Process
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/architecture/data-model/meta-provider-model.md b/content/docs/architecture/data-model/meta-provider-model.md
index c74f8ac..0d69b51 100644
--- a/content/docs/architecture/data-model/meta-provider-model.md
+++ b/content/docs/architecture/data-model/meta-provider-model.md
@@ -1,14 +1,5 @@
----
-title: "Meta Provider Composability Model"
-type: docs
-weight: 30
----
+# DCM Data Model โ€” Meta Provider Composability Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. This document specifies the Meta Provider composability model โ€” what a Meta Provider is, what it declares to DCM, and how standard DCM machinery handles the rest.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Meta Provider Specification
diff --git a/content/docs/architecture/data-model/notification-model.md b/content/docs/architecture/data-model/notification-model.md
index 89d8bf2..2fa20a4 100644
--- a/content/docs/architecture/data-model/notification-model.md
+++ b/content/docs/architecture/data-model/notification-model.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Notification Model"
-type: docs
-weight: 23
----
+# DCM Data Model โ€” Notification Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -45,7 +36,7 @@ This document defines:
 - The notification payload structure โ€” the unified envelope all Notification Providers receive
 - The delivery pipeline โ€” from event trigger through audience resolution through provider delivery
 
-This model supersedes the standalone outbound webhook model in doc 18. Outbound webhooks are now one delivery channel of the Notification Provider, not a parallel mechanism.
+Outbound webhooks are one delivery channel within this model, implemented via the Notification Provider.
 
 ---
 
@@ -551,7 +542,7 @@ Provider Update Notifications (doc 06, Section 7a) integrate with the notificati
 **When provider update requires consumer approval:**
 - `provider_update.requires_approval` fires โ†’ Owner notified (action required)
 - `action.type: approve`
-- `action.action_url` points to `/api/v1/resources/{uuid}/provider-notifications/{uuid}/approve`
+- `action.action_url` points to `/api/v1/resources/{uuid}/provider-notifications/{uuid}:approve`
 - `action.deadline` set per policy (default PT24H โ€” if no response, escalate)
 
 **On resolution:**
diff --git a/content/docs/architecture/data-model/operational-models.md b/content/docs/architecture/data-model/operational-models.md
index 33250e8..cb63efd 100644
--- a/content/docs/architecture/data-model/operational-models.md
+++ b/content/docs/architecture/data-model/operational-models.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Operational Models"
-type: docs
-weight: 24
----
+# DCM Data Model โ€” Operational Models
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -311,7 +302,7 @@ event_triggered_discovery:
 Platform admins and SREs can trigger discovery manually:
 
 ```
-POST /api/v1/admin/discovery/trigger
+POST /api/v1/admin/discovery:trigger
 
 {
   "scope": "entity | resource_type | provider | tenant",
diff --git a/content/docs/architecture/data-model/operational-reference.md b/content/docs/architecture/data-model/operational-reference.md
index 41ec0d4..a666a09 100644
--- a/content/docs/architecture/data-model/operational-reference.md
+++ b/content/docs/architecture/data-model/operational-reference.md
@@ -1,8 +1,4 @@
----
-title: "Operational Reference: GitOps Scale, Store Migration, Disaster Recovery"
-type: docs
-weight: 41
----
+# DCM Data Model โ€” Operational Reference
 
 **Document Status:** ๐Ÿ”„ In Progress
 **Document Type:** SRE Reference โ€” GitOps Scale, Store Migration, Disaster Recovery
diff --git a/content/docs/architecture/data-model/ownership-sharing-allocation.md b/content/docs/architecture/data-model/ownership-sharing-allocation.md
index e2534dc..1e89ac1 100644
--- a/content/docs/architecture/data-model/ownership-sharing-allocation.md
+++ b/content/docs/architecture/data-model/ownership-sharing-allocation.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Ownership, Sharing, and Allocation"
-type: docs
-weight: 5
----
+# DCM Data Model โ€” Ownership, Sharing, and Allocation
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
diff --git a/content/docs/architecture/data-model/policy-profiles.md b/content/docs/architecture/data-model/policy-profiles.md
index d872844..f0772d3 100644
--- a/content/docs/architecture/data-model/policy-profiles.md
+++ b/content/docs/architecture/data-model/policy-profiles.md
@@ -1,14 +1,5 @@
----
-title: "Policy Organization: Groups, Profiles, and Policy Providers"
-type: docs
-weight: 14
----
+# DCM Data Model โ€” Policy Organization: Groups, Profiles, and Policy Providers
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 > **Universal Group Model:** Policy Groups (`group_class: policy_collection`) and Policy Profiles (`group_class: policy_profile`) are expressions of the [Universal Group Model](15-universal-groups.md). The structures defined in this document remain authoritative for policy-specific behavior; the universal model adds composability, cross-type membership, and the ability to include policy groups within composite groups.
 
@@ -31,6 +22,8 @@ weight: 14
 
 ---
 
+> **Standards and Compliance Reference:** See [Standards and Compliance Catalog](40-standards-catalog.md) for the complete mapping of compliance frameworks (HIPAA, PCI DSS, FedRAMP, NIST SP 800-53, GDPR, DoD IL4) to DCM profiles and system requirements.
+
 ## 1. Purpose
 
 DCM's Policy Engine is powerful โ€” but power without usability is a barrier to adoption. This document defines the **policy organization model**: the structures that make DCM easy to configure correctly for any use case, from a home lab evaluation to a sovereign financial services deployment.
diff --git a/content/docs/architecture/data-model/provider-callback-auth.md b/content/docs/architecture/data-model/provider-callback-auth.md
new file mode 100644
index 0000000..e14b964
--- /dev/null
+++ b/content/docs/architecture/data-model/provider-callback-auth.md
@@ -0,0 +1,421 @@
+# DCM Data Model โ€” Provider Callback Authentication
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+**Related Documents:** [Unified Provider Contract](A-provider-contract.md) | [Credential Provider Model](31-credential-provider-model.md) | [Accreditation, Auth Matrix, Zero Trust](26-accreditation-and-authorization-matrix.md) | [Internal Component Auth](36-internal-component-auth.md) | [Registration Specification](../specifications/dcm-registration-spec.md) | [Provider Callback API](../schemas/openapi/dcm-provider-callback-api.yaml)
+
+> **Foundation Document Reference**
+>
+> This document maps to: **PROVIDER** (authentication of the provider-to-DCM interaction boundary)
+> and **DATA** (the credential artifact that governs that boundary).
+>
+> See [00-foundations.md](00-foundations.md) ยท [A-provider-contract.md](A-provider-contract.md) ยท [B-policy-contract.md](B-policy-contract.md)
+
+---
+
+## 1. Purpose and Scope
+
+This document specifies how Service Providers authenticate inbound calls to the DCM control plane โ€” specifically, calls to the Provider Callback API endpoints:
+
+- `POST /api/v1/providers` โ€” Registration
+- `POST /api/v1/providers/{provider_uuid}/capacity` โ€” Capacity reporting
+- `PUT /api/v1/instances/{resource_id}/status` โ€” Realized state push
+- `POST /api/v1/provider/entities/{entity_uuid}/status` โ€” Interim progress
+- `POST /api/v1/provider/entities/{entity_uuid}/update-notification` โ€” Authorized state change notification
+- `GET /api/v1/provider/notifications/{notification_uuid}` โ€” Notification status poll
+- `POST /api/v1/instances/{resource_id}/events` โ€” Lifecycle event reporting
+
+The outbound model (DCM authenticating to providers) is specified in [doc 31 Section 4.2](31-credential-provider-model.md) and [doc 26 Section 5.3](26-accreditation-and-authorization-matrix.md). This document specifies the **inbound** model.
+
+---
+
+## 2. The Authentication Problem
+
+When DCM receives a callback at `PUT /api/v1/instances/{resource_id}/status`, it must verify:
+
+1. **Identity:** Is this call genuinely from the registered Service Provider for this resource?
+2. **Authorization:** Is this provider permitted to push realized state for this specific resource/entity?
+3. **Integrity:** Has the payload been tampered with in transit?
+4. **Freshness:** Is this a live call, not a replayed credential from a previous session?
+5. **Scope:** Is this credential permitted for this specific operation type?
+
+Network-level authentication alone (firewall rules, IP allowlisting) is insufficient under the DCM Zero Trust model โ€” it establishes perimeter trust, not per-call identity. Every provider call to DCM must carry a credential that answers all five questions independently of network position.
+
+---
+
+## 3. Two-Layer Authentication Model
+
+DCM uses a **two-layer** model for provider-to-DCM calls. Both layers must pass:
+
+```
+Provider calls DCM:
+  โ”‚
+  โ–ผ Layer 1: Transport Identity (mTLS)
+  โ”‚   Provider presents its registered certificate
+  โ”‚   DCM verifies the certificate chain against the provider's registered CA
+  โ”‚   Proves: this connection is from the registered provider
+  โ”‚   Does NOT prove: authorization for this specific operation
+  โ”‚
+  โ–ผ Layer 2: Operation Authorization (Provider Callback Credential)
+  โ”‚   Provider presents a scoped short-lived credential in the Authorization header
+  โ”‚   DCM validates: credential is active, scoped to this provider, scoped to this operation type
+  โ”‚   Proves: this specific call is authorized for this specific operation
+  โ”‚   Does NOT replace mTLS โ€” both layers are required
+  โ”‚
+  โ–ผ Both pass โ†’ five-check boundary model evaluates (doc 26 Section 5.2)
+  โ””โ”€โ”€ Audit record written regardless of outcome
+```
+
+**Why two layers?** mTLS proves the caller holds the private key for the registered certificate โ€” it proves identity at the transport level. The interaction credential proves the specific call is authorized for the specific operation type and scope. A compromised credential without the private key cannot establish the mTLS connection. A valid mTLS connection without a valid credential cannot perform operations. The layers are complementary, not redundant.
+
+---
+
+## 4. Layer 1 โ€” Transport Identity (mTLS)
+
+### 4.1 Provider Certificate Registration
+
+At registration, every provider declares a certificate:
+
+```yaml
+provider_registration:
+  certificate:
+    pem: 
+    ca_chain: 
+    rotation_interval: P90D
+```
+
+DCM validates:
+- Certificate chain is valid and trusted
+- Certificate is not in DCM's Credential Revocation Registry
+- Certificate `CN` or `SAN` matches the provider's declared `handle`
+- Certificate `valid_until` is not in the past
+
+DCM stores the certificate fingerprint. On every subsequent inbound connection, DCM validates the presented certificate against the stored fingerprint for this provider.
+
+### 4.2 Certificate Validation on Inbound Calls
+
+When a provider initiates a TLS connection to DCM:
+
+```
+Provider โ†’ DCM:
+  TLS ClientHello โ†’ ServerHello + DCM certificate
+  Provider verifies DCM certificate (validates DCM's identity)
+  Provider sends its certificate
+  DCM validates:
+    1. Certificate chain โ†’ registered CA for this provider
+    2. Certificate fingerprint โ†’ matches stored fingerprint for provider_uuid in path/payload
+    3. Certificate not in Credential Revocation Registry
+    4. Certificate valid_until not expired
+  If any check fails โ†’ TLS handshake rejected; connection refused
+```
+
+**Certificate rotation:** Providers must rotate certificates on the declared `rotation_interval`. DCM fires a `P14D` warning event when a certificate is approaching expiry. During the rotation transition window, DCM accepts both the current and new certificate simultaneously (P7D window). After the window, only the new certificate is accepted.
+
+### 4.3 Certificate Binding to Operations
+
+The mTLS certificate is **not sufficient alone** for operation authorization. Knowing a call came from Provider X does not mean Provider X is authorized to push realized state for entity Y owned by Tenant Z. The interaction credential (Layer 2) carries that authorization.
+
+---
+
+## 5. Layer 2 โ€” Operation Authorization (Provider Callback Credential)
+
+### 5.1 Provider Callback Credential
+
+The **Provider Callback Credential** is a `dcm_interaction` type credential issued to the provider by DCM's Credential Provider at registration activation time. It is the mechanism by which providers prove authorization for specific callback operations.
+
+```yaml
+provider_callback_credential:
+  credential_uuid: 
+  credential_type: dcm_interaction
+  issued_to:
+    provider_uuid:       # the specific registered provider
+    provider_handle:   # for human-readable audit records
+  issued_at: 
+  valid_until:       # profile-governed lifetime; see Section 5.3
+  operation_scope:
+    allowed_operations:
+      - realized_state_push
+      - capacity_report
+      - interim_status
+      - update_notification
+      - lifecycle_event
+      - notification_poll
+      # Note: registration uses a registration_token, not this credential
+    # Scope is bound to the provider_uuid โ€” cannot be used for other providers
+  non_transferable: true
+  bound_to_ip:        # fsi/sovereign profiles: IP-bound
+  revocation_check_url: 
+```
+
+**Presented as:** `Authorization: Bearer ` on all callback API calls.
+
+**Key property:** The credential is scoped to the `provider_uuid` โ€” not to specific entities or operations within that provider. Entity-level scope is enforced separately (Section 6). This means a provider holding the credential can call any callback endpoint, but DCM enforces entity-level ownership checks per call.
+
+### 5.2 Credential Issuance
+
+Provider callback credentials are issued through the following lifecycle:
+
+```
+Registration approved (provider status โ†’ ACTIVE):
+  โ”‚
+  โ–ผ DCM API Gateway requests credential from Credential Provider:
+  โ”‚   credential_type: dcm_interaction
+  โ”‚   issued_to.provider_uuid: 
+  โ”‚   allowed_operations: [realized_state_push, capacity_report, interim_status,
+  โ”‚                         update_notification, lifecycle_event, notification_poll]
+  โ”‚   valid_until: 
+  โ”‚
+  โ–ผ Credential Provider issues credential
+  โ”‚   Returns credential_value (the bearer token)
+  โ”‚   Stores credential_record in Credential Store
+  โ”‚
+  โ–ผ DCM delivers credential to provider via the activation response:
+  โ”‚   POST /api/v1/admin/providers/{uuid}:approve
+  โ”‚   Response includes: credential_ref (UUID for retrieval)
+  โ”‚
+  โ–ผ Provider retrieves credential value via Credential Provider endpoint:
+  โ”‚   GET {credential_provider_endpoint}/credentials/{credential_ref}/value
+  โ”‚   (Requires the registration token used at initial registration โ€” one-time bootstrap)
+  โ”‚
+  โ–ผ Provider stores credential securely and uses it for all callback API calls
+```
+
+**Bootstrap case:** During initial registration (before activation), the provider uses the registration token to authenticate. After activation, the provider callback credential replaces the registration token for all subsequent calls. The registration token is single-use and expires after the first successful registration response.
+
+### 5.3 Credential Lifetime by Profile
+
+| Profile | Lifetime | Rotation trigger | IP binding |
+|---------|----------|-----------------|------------|
+| minimal | PT8H | Pre-expiry P1H | No |
+| dev | PT4H | Pre-expiry P30M | No |
+| standard | PT1H | Pre-expiry PT10M | No |
+| prod | PT30M | Pre-expiry PT5M | Optional |
+| fsi | PT15M | Pre-expiry PT3M | Required |
+| sovereign | PT15M + hardware attestation | Pre-expiry PT3M | Required; HSM-bound |
+
+**Pre-expiry rotation:** DCM initiates rotation automatically before the current credential expires. The transition window is 50% of the credential lifetime โ€” the old credential remains valid during the window while the new one is delivered. Providers must implement credential refresh in their SDK or adapter.
+
+### 5.4 Credential Rotation Protocol
+
+```
+PT{rotation_trigger} before credential expiry:
+  โ”‚
+  โ–ผ DCM initiates rotation:
+  โ”‚   Requests new credential from Credential Provider
+  โ”‚   rotation_of: 
+  โ”‚   same allowed_operations scope; new valid_until
+  โ”‚
+  โ–ผ Credential Provider issues new credential
+  โ”‚   Old credential NOT yet revoked
+  โ”‚
+  โ–ผ DCM pushes rotation notification to provider:
+  โ”‚   POST {provider_health_endpoint}/credential-rotation (if provider supports it)
+  โ”‚   OR: credential.rotating event published to Message Bus
+  โ”‚   New credential_ref included; provider retrieves new value
+  โ”‚
+  โ–ผ Transition window opens:
+  โ”‚   DCM accepts BOTH old and new credential during transition window
+  โ”‚   Provider switches to new credential
+  โ”‚
+  โ–ผ Transition window closes:
+  โ”‚   Old credential revoked
+  โ”‚   Revocation event published โ†’ all components update revocation cache
+```
+
+If the provider fails to pick up the new credential before the transition window closes, the old credential is revoked and the provider's callback calls will return `403 Forbidden` with code `CREDENTIAL_EXPIRED`. The provider must re-register to obtain a new credential โ€” this is a recoverable state.
+
+### 5.5 DCM Validation on Inbound Calls
+
+When DCM receives a callback call, Layer 2 validation performs these checks in order:
+
+```
+1. Extract credential_value from Authorization: Bearer header
+   โ†’ Missing or malformed: 401 Unauthorized; MISSING_CREDENTIAL audit record
+
+2. Look up credential_record by credential_value hash
+   โ†’ Not found: 401 Unauthorized; CREDENTIAL_NOT_FOUND audit record
+
+3. Check credential_record.status is 'active'
+   โ†’ Revoked: 403 Forbidden; code: CREDENTIAL_REVOKED
+   โ†’ Expired: 403 Forbidden; code: CREDENTIAL_EXPIRED
+
+4. Check credential_record.valid_until > now
+   โ†’ Expired: 403 Forbidden; code: CREDENTIAL_EXPIRED
+
+5. Check credential_record.issued_to.provider_uuid matches:
+   a. The provider_uuid in the URL path (where applicable)
+   b. The mTLS certificate's registered provider (Layer 1 binding)
+   โ†’ Mismatch: 403 Forbidden; code: CREDENTIAL_SCOPE_VIOLATION
+
+6. Check that the operation_type for this endpoint is in allowed_operations
+   โ†’ Not in scope: 403 Forbidden; code: OPERATION_NOT_IN_SCOPE
+
+7. If bound_to_ip is set: verify client IP matches
+   โ†’ Mismatch: 403 Forbidden; code: IP_BINDING_VIOLATION
+```
+
+All failures write an audit record with the credential_uuid, provider_uuid, endpoint, and failure reason. After 5 consecutive `CREDENTIAL_SCOPE_VIOLATION` or `IP_BINDING_VIOLATION` failures from the same provider within PT1H, DCM fires a `security.unsanctioned_provider_write` event and notifies the platform admin (urgency: critical).
+
+---
+
+## 6. Entity-Level Authorization
+
+The provider callback credential proves the caller is the registered provider. It does not prove the provider is authorized to act on a specific entity. Entity-level authorization is a separate check that applies on each call.
+
+### 6.1 Resource Ownership Binding
+
+For `realized_state_push` and `interim_status` calls, DCM validates:
+
+```
+PUT /api/v1/instances/{resource_id}/status
+
+DCM checks:
+  1. Look up the Requested State record for resource_id
+  2. Verify the credential's provider_uuid matches the provider_uuid
+     in the Requested State record (i.e., this was the provider DCM dispatched to)
+  3. Verify the entity is in a lifecycle state that permits this push
+     (PROVISIONING, UPDATING, or DECOMMISSIONING โ€” not OPERATIONAL, not DECOMMISSIONED)
+  
+  โ†’ Mismatch on provider_uuid: 403 Forbidden; code: ENTITY_NOT_OWNED_BY_PROVIDER
+  โ†’ Wrong lifecycle state: 409 Conflict; code: INVALID_LIFECYCLE_STATE_FOR_PUSH
+```
+
+**Why this matters:** A provider that receives a `resource_id` (e.g., by observing network traffic or misconfiguration) cannot push realized state for an entity it was not dispatched to. The Requested State record binds the entity to the specific provider that received the dispatch.
+
+### 6.2 Update Notification Binding
+
+For `update_notification` calls, DCM validates:
+
+```
+POST /api/v1/provider/entities/{entity_uuid}/update-notification
+
+DCM checks:
+  1. Look up the Realized State record for entity_uuid
+  2. Verify the credential's provider_uuid matches the provider_uuid
+     in the most recent Realized State record
+  3. Verify the provider's registration includes the update_capability
+     declared in the notification_type field
+
+  โ†’ Provider not current owner: 403 Forbidden; code: ENTITY_NOT_OWNED_BY_PROVIDER
+  โ†’ Update type not declared at registration: 403 Forbidden;
+    code: UPDATE_TYPE_NOT_DECLARED
+    (provider must re-register to declare new update capability types)
+```
+
+### 6.3 Lifecycle Event Binding
+
+For `lifecycle_event` calls, DCM validates:
+
+```
+POST /api/v1/instances/{resource_id}/events
+
+DCM checks:
+  1. Verify the credential's provider_uuid matches the provider on record for resource_id
+  2. Verify the resource is in an operational state (not DECOMMISSIONED)
+  3. Verify the event_type is in the standard event catalog
+
+  โ†’ Provider not current owner: 403 Forbidden; code: ENTITY_NOT_OWNED_BY_PROVIDER
+  โ†’ Entity decommissioned: 409 Conflict; code: ENTITY_DECOMMISSIONED
+  โ†’ Unknown event_type: 400 Bad Request; code: UNKNOWN_EVENT_TYPE
+```
+
+---
+
+## 7. Registration Authentication (Special Case)
+
+The initial `POST /api/v1/providers` registration call cannot use the provider callback credential because no credential exists yet. Registration uses a different authentication mechanism:
+
+### 7.1 Registration Token
+
+The registration token is a short-lived, single-use credential issued by a platform admin before provider onboarding:
+
+```yaml
+registration_token:
+  token_uuid: 
+  token_value: 
+  issued_at: 
+  expires_at:    # typically PT72H
+  scope:
+    provider_type_id: service_provider
+    provider_handle_pattern: "eu-west-*"   # optional constraint
+    grants_auto_approval: true | false
+  used: false              # single-use; set to true after first successful use
+```
+
+The registration token is passed as `Authorization: Bearer ` on the initial `POST /api/v1/providers` call. After the first successful registration, the token is marked `used: true` and cannot be reused. If a provider needs to re-register (e.g., after a sovereignty declaration change), a new registration token is required.
+
+**mTLS still required for registration:** The mTLS layer (Layer 1) is enforced on the registration call. The provider must present the certificate declared in the registration payload. This ensures the entity performing the registration possesses the private key for the certificate it is claiming.
+
+### 7.2 Re-Registration
+
+For re-registration calls (same `name`, updating version or capabilities), the provider uses its active provider callback credential โ€” not a new registration token. Re-registration that changes the sovereignty declaration requires a new registration token from the platform admin (treated as a new registration requiring a new approval).
+
+---
+
+## 8. Credential Revocation and Emergency Response
+
+### 8.1 Revocation Triggers
+
+| Trigger | What happens |
+|---------|-------------|
+| Provider deregistered | All callback credentials for that provider revoked immediately |
+| Security event detected (5+ scope violations in PT1H) | Provider suspended; credential revoked; platform admin notified |
+| Provider certificate expiry without rotation | Credential revoked at certificate expiry |
+| Platform admin explicit revocation | Immediate revocation; provider must re-register |
+| Provider compromise suspected | Emergency revocation; Recovery Policy evaluates affected entities |
+
+### 8.2 Emergency Revocation Flow
+
+```
+Platform admin triggers emergency revocation:
+  โ”‚
+  โ–ผ POST /api/v1/admin/providers/{provider_uuid}/revoke-credential
+  โ”‚   reason: 
+  โ”‚   suspend_provider: true | false
+  โ”‚
+  โ–ผ DCM revokes credential immediately:
+  โ”‚   credential_record.status โ†’ revoked
+  โ”‚   Revocation event โ†’ Message Bus
+  โ”‚   All DCM components update revocation cache (within PT30S)
+  โ”‚
+  โ–ผ If suspend_provider: true:
+  โ”‚   Provider status โ†’ SUSPENDED
+  โ”‚   New requests not routed to this provider
+  โ”‚   Active realizations enter PENDING_REVIEW state
+  โ”‚
+  โ–ผ Recovery Policy evaluates affected entities:
+      Entities currently hosted at provider: notify Tenant owners
+      In-progress operations: depends on Recovery Policy profile
+```
+
+### 8.3 Revocation Cache
+
+DCM components that validate inbound credentials maintain a local **Credential Revocation Cache**:
+
+- Cache is populated from the Message Bus `credential.revoked` event stream
+- Cache TTL matches the maximum credential lifetime for the active profile
+- On cache miss: remote check against Credential Store (prevents stale cache from accepting revoked credentials)
+- Cache invalidation is immediate on `credential.revoked` event receipt (not TTL-based)
+
+The revocation cache ensures revocation propagates within PT30S even without a cache miss triggering a remote lookup.
+
+---
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `PCA-001` | All provider-to-DCM calls must present both a valid mTLS certificate (Layer 1) and a valid provider callback credential (Layer 2). Neither layer alone is sufficient. |
+| `PCA-002` | Provider callback credentials are scoped to the provider_uuid and cannot be used to act on entities hosted at other providers. |
+| `PCA-003` | Entity-level authorization is checked on every realized_state_push, update_notification, and lifecycle_event call, independent of credential validity. A valid credential does not grant access to entities the provider was not dispatched to. |
+| `PCA-004` | Five consecutive credential scope violations or IP binding violations from the same provider within PT1H triggers automatic provider suspension and platform admin notification. |
+| `PCA-005` | Provider callback credentials are issued by the Credential Provider, not directly by the DCM API Gateway. The Credential Provider is the authoritative source for all credential issuance, rotation, and revocation. |
+| `PCA-006` | Registration tokens are single-use. A registration token that has been used once is permanently invalidated regardless of its `expires_at` timestamp. |
+| `PCA-007` | Re-registration that changes the sovereignty declaration requires a new registration token and triggers a new approval pipeline. Version and capability updates do not require a new registration token. |
+| `PCA-008` | Provider callback credentials must be rotated before expiry. DCM initiates rotation automatically. If a credential expires without rotation, the provider enters a CREDENTIAL_EXPIRED state and must obtain a new credential via the platform admin. |
+| `PCA-009` | For fsi and sovereign profiles, provider callback credentials are IP-bound. A credential presented from an IP address that does not match the `bound_to_ip` field is rejected regardless of its validity. |
+| `PCA-010` | All inbound provider calls โ€” including rejected calls โ€” produce an audit record containing the credential_uuid, provider_uuid, endpoint, operation_type, outcome, and timestamp. There are no silent failures. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/data-model/registry-governance.md b/content/docs/architecture/data-model/registry-governance.md
index ea7c7cb..98567b6 100644
--- a/content/docs/architecture/data-model/registry-governance.md
+++ b/content/docs/architecture/data-model/registry-governance.md
@@ -1,14 +1,5 @@
----
-title: "Registry Governance"
-type: docs
-weight: 20
----
+# DCM Data Model โ€” Registry Governance
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Auth Providers](19-auth-providers.md)
diff --git a/content/docs/architecture/data-model/request-dependency-graph.md b/content/docs/architecture/data-model/request-dependency-graph.md
index 9077fb8..6b9f311 100644
--- a/content/docs/architecture/data-model/request-dependency-graph.md
+++ b/content/docs/architecture/data-model/request-dependency-graph.md
@@ -1,8 +1,4 @@
----
-title: "Consumer Request Dependency Graph"
-type: docs
-weight: 38
----
+# DCM Data Model โ€” Consumer Request Dependency Graph
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Cross-Request Ordering
diff --git a/content/docs/architecture/data-model/resource-grouping.md b/content/docs/architecture/data-model/resource-grouping.md
index c10e4ee..8d16b4a 100644
--- a/content/docs/architecture/data-model/resource-grouping.md
+++ b/content/docs/architecture/data-model/resource-grouping.md
@@ -1,17 +1,8 @@
----
-title: "DCM Data Model โ€” Resource Grouping"
-type: docs
-weight: 8
----
+# DCM Data Model โ€” Resource Grouping
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
-> **Universal Group Model:** The constructs defined in this document (DCM Tenant, Resource Groups) are superseded by the [Universal Group Model](15-universal-groups.md) for new implementations. Existing constructs map 1:1 to `group_class` values in the universal model โ€” UUIDs, handles, and API references are preserved. This document remains valid as a reference for pre-universal implementations and for understanding the migration path.
+> **Related:** See [Universal Group Model](15-universal-groups.md) for the canonical group model. The constructs here (Tenants, Resource Groups) map 1:1 to `group_class` values in that model.
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md)
diff --git a/content/docs/architecture/data-model/resource-service-entities.md b/content/docs/architecture/data-model/resource-service-entities.md
index 51c7eae..8814c7a 100644
--- a/content/docs/architecture/data-model/resource-service-entities.md
+++ b/content/docs/architecture/data-model/resource-service-entities.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Resource/Service Entities"
-type: docs
-weight: 6
----
+# DCM Data Model โ€” Resource/Service Entities
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -545,9 +536,8 @@ policy:
 
 This pre-authorization pattern allows providers to implement auto-scaling, auto-healing, and maintenance operations without requiring per-change manual approval, while keeping DCM's Realized Store accurate and traceable.
 
-### 7a.6 Updated Provider Lifecycle Events Table
+### 7a.6 Provider Lifecycle Events
 
-The following table supersedes the table in Section 7.2 with clearer DCM response categorization:
 
 | Event Type | Mechanism | DCM Response | Realized Store Updated? |
 |------------|-----------|-------------|------------------------|
diff --git a/content/docs/architecture/data-model/resource-type-hierarchy.md b/content/docs/architecture/data-model/resource-type-hierarchy.md
index 2f7c731..2ffc9cb 100644
--- a/content/docs/architecture/data-model/resource-type-hierarchy.md
+++ b/content/docs/architecture/data-model/resource-type-hierarchy.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Resource Type Hierarchy and Service Catalog"
-type: docs
-weight: 5
----
+# DCM Data Model โ€” Resource Type Hierarchy and Service Catalog
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -459,7 +450,7 @@ All definitions in the Resource Type Hierarchy follow the universal DCM versioni
 | Component | Trigger |
 |---|---|
 | **Major** | Breaking changes to the contract โ€” removing fields, changing field types, changing required/optional status of universal fields |
-| **Minor** | Additive changes, backward compatible โ€” adding new optional fields, adding new conditional fields, adding new extension points |
+| **Minor** | Additive changes โ€” adding new optional fields, adding new conditional fields, adding new extension points |
 | **Revision** | Data or configuration changes with no contract impact โ€” updating descriptions, updating constraints that don't break existing data, updating metadata |
 
 ### 9.2 Version Constraints in Requests
@@ -506,3 +497,25 @@ Once a version is published it is immutable. Any change โ€” even a documentation
 ---
 
 *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+---
+
+## Resource Type Reference โ€” Consumer vs Internal Format
+
+When consumers reference a resource type โ€” in API calls, policy conditions, or query filters โ€” DCM accepts two forms:
+
+| Form | Example | Notes |
+|------|---------|-------|
+| **FQN string** (recommended) | `Compute.VirtualMachine` | Stable across deployments; human-readable; returned by the service catalog |
+| **Registry UUID** | `a1b2c3d4-e5f6-...` | Deployment-specific; obtained from catalog API; suitable for programmatic use |
+
+DCM resolves either form to the canonical `(resource_type_uuid, resource_type_name)` pair during request assembly. The resolution happens in the **Request Payload Processor** before layer enrichment begins. Unresolvable references are rejected at validation time with a `422 Unprocessable Entity` response and code `RESOURCE_TYPE_NOT_FOUND`.
+
+**Internal representation:** All internal DCM data โ€” entity records, dispatch payloads, audit records โ€” always carry **both** `resource_type_uuid` and `resource_type_name` (FQN). The consumer-facing accept-both model is purely at the API boundary; internally DCM always uses the canonical pair.
+
+**Dispatch to operators:** The `CreateRequest` and `UpdateRequest` payloads sent to Service Providers always include both:
+- `resource_type_uuid` โ€” the Registry UUID
+- `resource_type_name` โ€” the FQN string
+
+Operators MUST NOT accept only one form; both will always be present.
+
diff --git a/content/docs/architecture/data-model/scheduled-requests.md b/content/docs/architecture/data-model/scheduled-requests.md
index 6a69ecd..dad7660 100644
--- a/content/docs/architecture/data-model/scheduled-requests.md
+++ b/content/docs/architecture/data-model/scheduled-requests.md
@@ -1,8 +1,4 @@
----
-title: "Scheduled and Deferred Requests"
-type: docs
-weight: 37
----
+# DCM Data Model โ€” Scheduled and Deferred Requests
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Request Scheduling
diff --git a/content/docs/architecture/data-model/scoring-model.md b/content/docs/architecture/data-model/scoring-model.md
index ce72412..c8c00ab 100644
--- a/content/docs/architecture/data-model/scoring-model.md
+++ b/content/docs/architecture/data-model/scoring-model.md
@@ -1,14 +1,5 @@
----
-title: "Hybrid Scoring Model"
-type: docs
-weight: 29
----
+# DCM Data Model โ€” Hybrid Scoring Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. This document specifies the hybrid scoring model โ€” a first-class architectural addition approved in the DCM architecture review. It extends the Policy Engine and Profile system without replacing any existing components.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Scoring Model Specification
@@ -279,7 +270,19 @@ accreditation_weights:
   pci_dss: 25
   sovereign_authorization: 50
 
-# richness_score = sum(weights for held accreditations) / max_possible ร— 100
+# richness_score = sum(weights for held accreditations) / max_possible ร— 1
+
+# Verification currency multipliers (applied per accreditation, see doc 47)
+# Multiplier reduces an accreditation's weight contribution based on how recently
+# it was externally verified by the Accreditation Monitor
+verification_multipliers:
+  external_registry_verified_within_P1D:  1.0   # full weight โ€” verified today
+  external_registry_verified_within_P7D:  0.9
+  document_verified_within_P30D:          0.85
+  contract_webhook_active:                0.9
+  expiry_only_no_external_check:          0.7   # never been externally verified
+  verification_stale:                     0.4   # check overdue
+  verification_failed_threshold_reached:  0.1   # Monitor cannot reach registry00
 # risk_contribution = (1 - richness_score/100) ร— 10   [lower richness = higher risk]
 ```
 
@@ -345,11 +348,11 @@ Profiles can declare tighter thresholds for specific resource types:
 ```yaml
 resource_type_threshold_overrides:
   - resource_type: "Compute.VirtualMachine"
-    auto_approve_below: 20    # tighter than profile default
+    # tier: auto, max_score: 20  # use named-tier threshold format    # tighter than profile default
   - resource_type: "Network.VLAN"
-    auto_approve_below: 10    # VLANs require more scrutiny
+    # tier: auto, max_score: 10  # use named-tier threshold format    # VLANs require more scrutiny
   - resource_type: "Storage.Volume"
-    verified_above: 40   # storage changes escalate earlier
+    # tier: verified, max_score: 40  # use named-tier threshold format   # storage changes escalate earlier
 ```
 
 ### 5.3 Tenant Threshold Overrides
@@ -360,7 +363,7 @@ Platform admins can declare Tenant-level scoring threshold adjustments:
 tenant_scoring_config:
   tenant_uuid: 
   threshold_overrides:
-    auto_approve_below: 15    # more conservative for this Tenant
+    # tier: auto, max_score: 15  # use named-tier threshold format    # more conservative for this Tenant
   signal_weight_overrides:
     actor_risk_history_weight: 0.30   # higher actor scrutiny for this Tenant
   trusted_actors:
diff --git a/content/docs/architecture/data-model/service-dependencies.md b/content/docs/architecture/data-model/service-dependencies.md
index e3d38aa..fc0d904 100644
--- a/content/docs/architecture/data-model/service-dependencies.md
+++ b/content/docs/architecture/data-model/service-dependencies.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Service Dependencies"
-type: docs
-weight: 7
----
+# DCM Data Model โ€” Service Dependencies
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -28,7 +19,7 @@ weight: 7
 
 
 
-> **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](09-entity-relationships.md). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document.
+> **Scope:** This document covers dependency declaration, rehydration ordering, and failure handling. The underlying data structure is the Entity Relationship Graph defined in [Entity Relationships](09-entity-relationships.md).
 
 ---
 
diff --git a/content/docs/architecture/data-model/session-revocation.md b/content/docs/architecture/data-model/session-revocation.md
index e1e78b9..6f431c1 100644
--- a/content/docs/architecture/data-model/session-revocation.md
+++ b/content/docs/architecture/data-model/session-revocation.md
@@ -1,8 +1,4 @@
----
-title: "Session Token Revocation"
-type: docs
-weight: 35
----
+# DCM Data Model โ€” Session Token Revocation
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Session Lifecycle and Revocation
@@ -199,7 +195,7 @@ session_revocation_registry:
 DCM's Ingress layer exposes a token introspection endpoint for internal components and external systems that need to validate a token without maintaining their own cache:
 
 ```
-POST /api/v1/auth/introspect
+POST /api/v1/auth:introspect
 
 Authorization: Bearer 
 Content-Type: application/json
@@ -227,7 +223,7 @@ Response 200 (revoked or expired):
 }
 ```
 
-This follows [RFC 7662 (OAuth 2.0 Token Introspection)](https://datatracker.ietf.org/doc/html/rfc7662).
+Session tokens use JWT format (RFC 7519). This introspection endpoint follows [RFC 7662 (OAuth 2.0 Token Introspection)](https://datatracker.ietf.org/doc/html/rfc7662).
 
 ---
 
@@ -287,7 +283,7 @@ Response 404: session not found or does not belong to this actor
 ### 6.5 Admin: Force Revoke Session(s)
 
 ```
-POST /api/v1/admin/actors/{actor_uuid}/revoke-sessions
+POST /api/v1/admin/actors/{actor_uuid}:revoke-sessions
 
 {
   "scope": "all | session",
@@ -346,7 +342,7 @@ Session revocation (this document) and credential revocation (doc 31, CPX-001โ€“
 | `AUTH-017` | Session revocation must propagate to the Session Revocation Registry within the profile-governed SLA: minimal PT5M, standard PT1M, prod PT30S, fsi PT10S, sovereign PT5S. |
 | `AUTH-018` | All DCM components that accept bearer tokens must check the Session Revocation Registry on each request. Cache age must not exceed the profile-governed maximum (sovereign: no cache). |
 | `AUTH-019` | Emergency session revocation (security_event trigger) fires immediately with no grace period. The `auth.security_session_revoked` event has `urgency: critical` and is non-suppressable. |
-| `AUTH-020` | The token introspection endpoint (`POST /api/v1/auth/introspect`) must be authenticated. Access requires an actor or service account with the `introspection` scope. |
+| `AUTH-020` | The token introspection endpoint (`POST /api/v1/auth:introspect`) must be authenticated. Access requires an actor or service account with the `introspection` scope. |
 | `AUTH-021` | When concurrent session limits are enforced, the oldest session is revoked before the new session is created. The evicted actor is notified via Notification Provider if configured. |
 | `AUTH-022` | Refresh tokens are invalidated when their parent session is revoked. A revoked refresh token returns 401 on exchange; it cannot be used to create a new session. |
 
diff --git a/content/docs/architecture/data-model/standards-catalog.md b/content/docs/architecture/data-model/standards-catalog.md
index 20f6cf6..f74b61e 100644
--- a/content/docs/architecture/data-model/standards-catalog.md
+++ b/content/docs/architecture/data-model/standards-catalog.md
@@ -1,8 +1,4 @@
----
-title: "Standards and Compliance Catalog"
-type: docs
-weight: 40
----
+# DCM Data Model โ€” Standards and Compliance Catalog
 
 **Document Status:** ๐Ÿ”„ In Progress
 **Document Type:** Architecture Reference โ€” Normative Standards
@@ -23,7 +19,7 @@ weight: 40
 |-----|-------|-----------|-----------|
 | **RFC 7519** | JSON Web Token (JWT) | Bearer token format for session tokens and API key tokens; claims carry actor_uuid, roles, tenant_uuid, exp | Normative |
 | **RFC 7517** | JSON Web Key (JWK) | Public key format for Auth Provider OIDC verification keys; JWKS endpoint for key discovery | Normative |
-| **RFC 7662** | OAuth 2.0 Token Introspection | `POST /api/v1/auth/introspect` โ€” validates bearer tokens; response format `{active, session_uuid, actor_uuid, exp, roles}` | Normative |
+| **RFC 7662** | OAuth 2.0 Token Introspection | `POST /api/v1/auth:introspect` โ€” validates bearer tokens; response format `{active, session_uuid, actor_uuid, exp, roles}` | Normative |
 | **RFC 6749** | OAuth 2.0 Authorization Framework | Authorization flow for OIDC Auth Providers; client credentials flow for service account API keys | Informative |
 | **RFC 4511** | Lightweight Directory Access Protocol (LDAP) | LDAP/FreeIPA/Active Directory Auth Provider integration; bind operations, search filters for group membership | Normative |
 | **RFC 7643** | SCIM 2.0 Core Schema | Actor and group provisioning schema for enterprise IdP integration; SCIM deprovision triggers session + credential revocation | Normative |
diff --git a/content/docs/architecture/data-model/storage-providers.md b/content/docs/architecture/data-model/storage-providers.md
index ccd5ed0..78465e9 100644
--- a/content/docs/architecture/data-model/storage-providers.md
+++ b/content/docs/architecture/data-model/storage-providers.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Storage Providers"
-type: docs
-weight: 11
----
+# DCM Data Model โ€” Storage Providers
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -30,6 +21,8 @@ weight: 11
 
 ---
 
+> **Operational guidance:** GitOps store-at-scale and migration are covered in [Operational Reference](41-operational-reference.md) Sections 1 and 2.
+
 ## 1. Purpose
 
 A **Storage Provider** is the fourth formal DCM provider type. It is the interface through which DCM persists, retrieves, and streams all state data. DCM defines the contract โ€” the characteristics, capabilities, and obligations each store must satisfy. The implementation technology is a deployment choice made by implementors.
diff --git a/content/docs/architecture/data-model/universal-audit.md b/content/docs/architecture/data-model/universal-audit.md
index 3c21575..30ad172 100644
--- a/content/docs/architecture/data-model/universal-audit.md
+++ b/content/docs/architecture/data-model/universal-audit.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Universal Audit Model"
-type: docs
-weight: 16
----
+# DCM Data Model โ€” Universal Audit Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Storage Providers](11-storage-providers.md) | [Universal Groups](15-universal-groups.md)
@@ -225,7 +216,7 @@ The `action` field uses a closed vocabulary. Free-text actions are invalid and r
 | `HOLD_RELEASE` | Resources | Resource hold released |
 | `DRIFT_DETECT` | Entities | Drift detected between Realized and Discovered |
 | `DRIFT_RESOLVE` | Entities | Drift resolved |
-| `INGEST` | Entities | Entity ingested (brownfield or V1 migration) |
+| `INGEST` | Entities | Entity ingested (brownfield or import) |
 | `PROMOTE` | Entities | Ingested entity promoted to full lifecycle |
 | `EXPIRE` | Entities | Lifecycle time constraint expiry action fired |
 | `REHYDRATE` | Entities | Rehydration requested |
diff --git a/content/docs/architecture/data-model/universal-groups.md b/content/docs/architecture/data-model/universal-groups.md
index 9b52cde..6103298 100644
--- a/content/docs/architecture/data-model/universal-groups.md
+++ b/content/docs/architecture/data-model/universal-groups.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Universal Group Model"
-type: docs
-weight: 15
----
+# DCM Data Model โ€” Universal Group Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Grouping](08-resource-grouping.md) | [Entity Relationships](09-entity-relationships.md) | [Policy Organization](14-policy-profiles.md)
@@ -395,7 +386,7 @@ All groups are stored in a single **Group Registry** โ€” a GitOps store followin
 
 ### 6.2 Class-Filtered API Views
 
-The universal registry exposes class-filtered views that preserve backward compatibility with existing API consumers:
+The universal registry exposes class-filtered views for convenience:
 
 | Endpoint | Equivalent Query |
 |----------|----------------|
@@ -409,22 +400,6 @@ Existing API references continue to work unchanged. New API consumers can use th
 
 ---
 
-## 7. Migration from Current Constructs
-
-Existing constructs migrate to the universal model with UUID preservation:
-
-| Current Construct | Migration | UUID Preserved? |
-|------------------|-----------|----------------|
-| Tenant entity | `group_class: tenant_boundary` | Yes |
-| DCM Default Resource Group | `group_class: resource_grouping` | Yes |
-| Custom Resource Group | `group_class: resource_grouping, group_subclass: ` | Yes |
-| Policy Group | `group_class: policy_collection` | Yes |
-| Policy Profile | `group_class: policy_profile` | Yes |
-
-Migration uses the standard ingestion model โ€” existing constructs are ingested as `ingestion_source: migration` with `ingestion_confidence: high` (UUID preservation, no ambiguity).
-
----
-
 ## 8. DCM System Policies
 
 | Policy | Rule |
diff --git a/content/docs/architecture/data-model/webhooks-messaging.md b/content/docs/architecture/data-model/webhooks-messaging.md
index 72d309c..840637f 100644
--- a/content/docs/architecture/data-model/webhooks-messaging.md
+++ b/content/docs/architecture/data-model/webhooks-messaging.md
@@ -1,14 +1,5 @@
----
-title: "DCM Data Model โ€” Webhooks, Messaging, and External Integration"
-type: docs
-weight: 18
----
+# DCM Data Model โ€” Webhooks, Messaging, and External Integration
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Related Documents (updated):** [Notification Model](23-notification-model.md) | [Entity Relationships](09-entity-relationships.md)  
@@ -261,7 +252,7 @@ quota_policy:
 
 > **โš ๏ธ Architecture Update โ€” Notification Model Supersedes Outbound Webhooks**
 >
-> The outbound webhook model described in Section 3 has been superseded by the **Unified Notification Model** (see [doc 23: Notification Model](23-notification-model.md)). Outbound webhooks are now one delivery channel type within the Notification Provider model rather than a parallel mechanism.
+> The outbound webhook model described in Section 3 has been one delivery channel within the Notification Model. Outbound webhooks are now one delivery channel type within the Notification Provider model rather than a parallel mechanism.
 >
 > **For new implementations:** Use the Notification Provider subscription model (doc 23, Section 6) with a webhook-type Notification Provider.
 >
@@ -353,7 +344,7 @@ webhook_registration:
 
 ### 3.3 Event Taxonomy
 
-> **See [doc 23: Notification Model](23-notification-model.md) Section 4 for the authoritative event taxonomy.** The table below is the legacy taxonomy for existing webhook registrations.
+> The table below is the event taxonomy for webhook subscriptions.
 
 The event taxonomy maps onto the Universal Audit action vocabulary. All are versioned registry entries:
 
@@ -385,7 +376,7 @@ webhook_payload:
   # Subject
   subject:
     entity_uuid: 
-    entity_type: resource_entity
+    entity_type: infrastructure_resource
     entity_handle: 
     tenant_uuid: 
 
diff --git a/content/docs/architecture/data-model/workload-analysis.md b/content/docs/architecture/data-model/workload-analysis.md
new file mode 100644
index 0000000..440fd90
--- /dev/null
+++ b/content/docs/architecture/data-model/workload-analysis.md
@@ -0,0 +1,253 @@
+# DCM โ€” Workload Analysis
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Specification in Progress
+**Document Type:** Capability Specification
+**Related Documents:** [Ingestion Model](13-ingestion-model.md) | [Information Providers](10-information-providers.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Discovery and Drift](25-control-plane-components.md) | [Kubernetes Compatibility](../specifications/11-kubernetes-compatibility.md)
+
+> **AEP Alignment:** API endpoints follow [AEP](https://aep.dev) conventions.
+> Workload analysis results are delivered as Information Provider payloads
+> and stored as `process_resource_entity` instances of type `Analysis.WorkloadProfile`.
+
+---
+
+## 1. Purpose
+
+Workload Analysis is the DCM capability that actively classifies discovered resources
+by their operational characteristics โ€” what they are, how they behave, what lifecycle
+model should apply to them, and which DCM Resource Type they best map to.
+
+It answers questions that passive discovery cannot:
+- *"This VM was discovered โ€” is it a web server, a database, a batch processor?"*
+- *"This workload can be migrated to containers โ€” what is its archetype?"*
+- *"This resource has no DCM UUID โ€” what is the minimum viable Resource Type we can
+   assign it for lifecycle management?"*
+
+Workload Analysis is the bridge between **Discovered State** (what exists) and
+**Intent State** (what should be managed). Without it, brownfield ingestion stalls
+at the enrichment phase because the Tenant, Resource Type, and lifecycle ownership
+cannot be automatically determined.
+
+---
+
+## 2. Relationship to Existing Capabilities
+
+```
+Discovery (DRC)          Workload Analysis       Ingestion (doc 13)
+      โ”‚                        โ”‚                        โ”‚
+      โ–ผ                        โ–ผ                        โ–ผ
+Discovered State     WorkloadProfile entity    INGESTED โ†’ ENRICHING
+(what physically      (what it IS and what      โ†’ PROMOTED โ†’ OPERATIONAL
+  exists today)        lifecycle applies)
+```
+
+**Workload Analysis is a DCM-managed process resource** (`Analysis.WorkloadProfile`)
+that fires as part of the brownfield ingestion pipeline. It is triggered by the
+Discovery Scheduler when a new resource enters Discovered State without a matching
+DCM entity in Realized State.
+
+It can also be triggered manually by a platform admin to re-classify a resource
+whose operational profile has changed (e.g., a VM that was a web server and is
+now a database).
+
+---
+
+## 3. WorkloadProfile Entity
+
+Workload Analysis produces a `process_resource_entity` of type `Analysis.WorkloadProfile`:
+
+```yaml
+workload_profile_entity:
+  entity_uuid: 
+  entity_type: process_resource
+  resource_type: Analysis.WorkloadProfile
+  lifecycle_state: OPERATIONAL       # while active; DECOMMISSIONED when superseded
+  
+  # Linked to the resource being analyzed
+  subject_entity_uuid:         # the VM, container, or other resource
+  subject_discovered_state_uuid: 
+  
+  classification:
+    resource_type_match:             # best-fit DCM Resource Type
+      primary:    Compute.VirtualMachine
+      confidence: high               # high | medium | low | undetermined
+      alternatives:
+        - resource_type: Platform.Container
+          confidence: medium
+          rationale: "Workload is containerizable per MTA assessment"
+    
+    workload_archetype:              # operational classification
+      type: web_server | database | batch_processor | message_broker |
+            api_gateway | cache | storage | monitoring | unknown
+      confidence: high
+      signals: [port_scan, process_list, resource_utilization_pattern]
+    
+    migration_readiness:             # if MTA integration is active
+      containerization_score: 7     # 1-10
+      blockers: []
+      suggested_target: Platform.KubernetesDeployment
+      mta_report_ref:           # link to MTA HTML report if available
+    
+    lifecycle_recommendation:
+      dcm_lifecycle_model: standard | stateful | ephemeral | infrastructure
+      rehydration_eligible: true
+      notes: "Application data on /data partition; OS on /; static replace eligible"
+    
+  analysis_metadata:
+    analyzed_at: 
+    analysis_version: "1.0.0"        # versioned analysis ruleset
+    information_providers_used:
+      - provider_uuid: 
+        provider_type: information_provider
+        data_types_used: [port_scan, process_list, os_metadata]
+    analyst_actor_uuid:        # null if automated; actor UUID if manual review
+```
+
+---
+
+## 4. Analysis Pipeline
+
+Workload Analysis is an Orchestration Flow Policy that fires when a discovered
+resource enters the enrichment phase:
+
+```
+discovery.new_entity_found
+  โ”‚
+  โ–ผ Orchestration Step 1: Create WorkloadProfile entity (INGESTED state)
+  โ”‚   Linked to discovered resource via 'operational' relationship
+  โ”‚
+  โ–ผ Orchestration Step 2: Gather signals from Information Providers
+  โ”‚   Port scan (network topology)
+  โ”‚   Process list (running services)
+  โ”‚   OS metadata (version, packages, mount points)
+  โ”‚   Resource utilization patterns (CPU/memory/disk I/O profile)
+  โ”‚   MTA assessment (if MTA Information Provider registered)
+  โ”‚
+  โ–ผ Orchestration Step 3: Apply classification ruleset (Policy Engine)
+  โ”‚   Transformation Policy: compute workload_archetype from signals
+  โ”‚   Transformation Policy: compute resource_type_match from archetype
+  โ”‚   Transformation Policy: compute migration_readiness from MTA signals
+  โ”‚   GateKeeper Policy: flag if confidence < medium for manual review
+  โ”‚
+  โ–ผ Orchestration Step 4: Write WorkloadProfile to Realized State
+  โ”‚   WorkloadProfile entity โ†’ OPERATIONAL
+  โ”‚
+  โ–ผ Orchestration Step 5: Trigger ingestion enrichment
+      WorkloadProfile classification informs:
+        - Tenant auto-assignment (if auto-assignment rules match)
+        - Resource Type assignment for the ingestion record
+        - Lifecycle model selection
+```
+
+---
+
+## 5. MTA (Migration Toolkit for Applications) Integration
+
+When the MTA Information Provider is registered, Workload Analysis invokes it
+as part of Step 2 above. MTA provides workload archetype classification and
+containerization readiness scores for discovered workloads.
+
+```yaml
+mta_information_provider_registration:
+  provider_type: information_provider
+  information_type: workload_analysis
+  display_name: "MTA โ€” Migration Toolkit for Applications"
+  endpoint: https://mta.internal:8080/api/v1
+  
+  capabilities:
+    workload_archetypes:
+      - web_server
+      - database
+      - batch_processor
+      - message_broker
+    provides_containerization_score: true
+    provides_migration_blockers: true
+    provides_target_recommendations: true
+  
+  query_interface:
+    # MTA receives discovered state payload and returns analysis
+    input: discovered_state_payload
+    output: mta_workload_report
+    async: true
+    callback_supported: true
+```
+
+The MTA integration is the primary implementation path for Workload Analysis in
+Red Hat environments. In non-MTA environments, a custom Information Provider
+implementing the same `workload_analysis` information type can be registered.
+
+---
+
+## 6. Consumer API โ€” Workload Analysis Endpoints
+
+```
+# Get the WorkloadProfile for a specific resource
+GET /api/v1/resources/{entity_uuid}/workload-profile
+
+Response 200:
+{
+  "workload_profile_uuid": "",
+  "subject_entity_uuid": "",
+  "classification": {
+    "resource_type_match": {
+      "primary": "Compute.VirtualMachine",
+      "confidence": "high"
+    },
+    "workload_archetype": {
+      "type": "web_server",
+      "confidence": "high"
+    },
+    "migration_readiness": {
+      "containerization_score": 7,
+      "blockers": [],
+      "suggested_target": "Platform.KubernetesDeployment"
+    },
+    "lifecycle_recommendation": {
+      "dcm_lifecycle_model": "standard",
+      "rehydration_eligible": true
+    }
+  },
+  "analyzed_at": ""
+}
+
+# Trigger a re-analysis of a resource
+POST /api/v1/resources/{entity_uuid}/workload-profile:analyze
+
+Request body:
+{
+  "reason": "Role change โ€” web server migrated to database role",
+  "include_mta": true
+}
+
+Response 200 โ€” returns Operation:
+{
+  "name": "/api/v1/operations/{request_uuid}",
+  "done": false,
+  "metadata": { "stage": "ANALYSIS_INITIATED", "resource_uuid": "{entity_uuid}" }
+}
+
+# List all resources with a given workload archetype (platform admin)
+GET /api/v1/admin/workload-analysis?archetype=web_server&confidence=high
+
+Response 200:
+{
+  "items": [ { "entity_uuid": "...", "resource_type": "...", "archetype": "..." } ],
+  "next_page_token": "..."
+}
+```
+
+---
+
+## 7. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `WLA-001` | Workload Analysis fires automatically for every entity entering Discovered State without a matching Realized State record. It is not optional โ€” it is part of the brownfield ingestion pipeline. |
+| `WLA-002` | WorkloadProfile entities are versioned. When re-analysis produces a different classification, the old WorkloadProfile enters DECOMMISSIONED state and a new one is created. The chain is preserved for audit. |
+| `WLA-003` | If classification confidence is `low` or `undetermined`, the WorkloadProfile GateKeeper policy fires and the entity is routed to manual review before ingestion can proceed to PROMOTED. |
+| `WLA-004` | The MTA Information Provider is the reference implementation for workload_analysis information type in Red Hat environments. Custom implementations must provide the same output schema. |
+| `WLA-005` | Workload Analysis results are stored in Realized State as `process_resource_entity` instances. They are immutable once written โ€” re-analysis creates a new entity, not an update. |
+| `WLA-006` | Migration readiness scores and archetype classifications are advisory โ€” they inform human decision-making and Orchestration Flow Policies but do not automatically trigger migrations. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/specifications/11-kubernetes-compatibility.md b/content/docs/architecture/specifications/11-kubernetes-compatibility.md
index 614e22c..99b8824 100644
--- a/content/docs/architecture/specifications/11-kubernetes-compatibility.md
+++ b/content/docs/architecture/specifications/11-kubernetes-compatibility.md
@@ -21,7 +21,7 @@
 
 
 
-**Document Status:** โœ… Complete  
+**Document Status:** โœ… Complete
 **Document Type:** Architecture Reference  
 **Related Documents:** [Foundational Abstractions](../data-model/00-foundations.md) | [Entity Relationships](../data-model/09-entity-relationships.md) | [Resource Type Hierarchy](../data-model/05-resource-type-hierarchy.md) | [Resource/Service Entities](../data-model/06-resource-service-entities.md) | [DCM Operator Interface Specification](dcm-operator-interface-spec.md)
 
@@ -29,6 +29,11 @@
 
 ## 1. Purpose
 
+> **AEP Alignment:** API endpoint references in this document follow [AEP](https://aep.dev) conventions
+> (custom methods use colon syntax). See `schemas/openapi/dcm-consumer-api.yaml` for the
+> normative OpenAPI specification.
+
+
 DCM is designed as a **superset of Kubernetes** โ€” extending Kubernetes' declarative, controller-based model upward to provide unified management across multiple clusters, infrastructure types, and organizational boundaries that Kubernetes alone cannot address.
 
 This document serves three purposes:
@@ -41,6 +46,10 @@ This document serves three purposes:
 
 ## 2. The Superset Relationship
 
+DCM is a superset of Kubernetes in the sense that it provides all the capabilities Kubernetes provides โ€” and more. An organization running Kubernetes exclusively is using a subset of what DCM can manage. DCM does not replace Kubernetes; it manages the lifecycle of Kubernetes clusters and the resources running on them.
+
+The superset relationship means DCM can manage Kubernetes-native resources (Deployments, Services, PersistentVolumes) through conformant operators, and it can manage the clusters themselves as catalog items. It also means DCM manages resources that have no Kubernetes equivalent โ€” bare metal, VMs, VLANs, IP allocations, and organizational data entities.
+
 ### 2.1 What Kubernetes Provides
 
 Kubernetes is a container orchestration platform that provides:
@@ -145,12 +154,10 @@ A Kubernetes cluster is a first-class catalog item in DCM. Any authorized Tenant
 **How it works:**
 
 ```yaml
-# Consumer requests a cluster via the catalog
 catalog_item: Platform.KubernetesCluster
 provider: CAPI-based Service Provider (or managed K8s Service Provider)
 tenant_uuid: 
 
-# The resulting entity:
 entity:
   resource_type: Platform.KubernetesCluster
   tenant_uuid:    # Tenant owns the cluster
@@ -183,6 +190,20 @@ This is a Meta Provider โ€” the cluster catalog item orchestrates all constituen
 
 ## 4. Where DCM Extends Beyond Kubernetes
 
+These are capabilities that exist in DCM but have no Kubernetes equivalent. None of these require Kubernetes to be present โ€” they operate across all provider types. For organizations running pure Kubernetes estates, these are the capabilities DCM brings that Kubernetes tooling alone cannot provide.
+
+**Summary of extensions:**
+
+| DCM Capability | Kubernetes Gap |
+|---------------|---------------|
+| Intent State | No concept of original consumer intent separate from desired state |
+| Field-Level Provenance | No field lineage โ€” a field is a field |
+| Data Layers and Assembly | No layering model โ€” manifests are flat declarations |
+| Policy Engine | Admission webhooks are cluster-scoped, admission-time only |
+| Cost Analysis | No native cost attribution in the request lifecycle |
+| Information Providers | No structured external organizational data relationships |
+| Cross-Cluster Lifecycle | Single-cluster scope โ€” multi-cluster requires external tooling |
+
 These are concepts that exist in DCM but have no Kubernetes equivalent. They are the capabilities DCM adds that justify the superset positioning.
 
 ### 4.1 Intent State
@@ -361,9 +382,9 @@ This is particularly significant: DCM managing the lifecycle of Kubernetes clust
 
 ---
 
-## 8. Migration Path โ€” Kubernetes-Native to DCM-Managed
+## 8. Incremental Adoption โ€” Kubernetes-Native to DCM-Managed
 
-Organizations running Kubernetes today can adopt DCM incrementally:
+Organizations running Kubernetes can adopt DCM incrementally across these phases:
 
 ### Phase 1 โ€” Observation (no operator changes)
 Deploy DCM with the Kubernetes Information Provider. DCM observes existing resources via the Kubernetes API and builds a Discovered State inventory. No changes to existing operators or workloads.
diff --git a/content/docs/architecture/specifications/cncf-strategy.md b/content/docs/architecture/specifications/cncf-strategy.md
index 9c1558e..f4bd3c3 100644
--- a/content/docs/architecture/specifications/cncf-strategy.md
+++ b/content/docs/architecture/specifications/cncf-strategy.md
@@ -1,10 +1,5 @@
 # DCM โ€” CNCF Strategy and Community Engagement Plan
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Version:** 0.1.0-draft  
diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md
index 94f8074..9b7826b 100644
--- a/content/docs/architecture/specifications/consumer-api-spec.md
+++ b/content/docs/architecture/specifications/consumer-api-spec.md
@@ -19,6 +19,12 @@ The Consumer API is the boundary between the Application domain and the Control
 
 ---
 
+> **AEP Alignment:** This specification follows [AEP](https://aep.dev) conventions.
+> Custom methods use colon syntax (`POST /resources/{name}:suspend`).
+> Async operations return an `Operation` resource (AEP-136 LRO) โ€” poll `operation.name` for completion.
+> List pagination uses `page_size` and `page_token` parameters.
+> See the normative OpenAPI specification: `schemas/openapi/dcm-consumer-api.yaml`
+
 ## 1. Introduction
 
 ### 1.1 Scope
@@ -94,14 +100,14 @@ If DCM receives two requests with the same `Idempotency-Key` from the same authe
 
 **Which endpoints support `Idempotency-Key`:**
 - `POST /api/v1/requests` โ€” resource request submission
-- `POST /api/v1/credentials/{uuid}/rotate` โ€” credential rotation request
-- `POST /api/v1/resources/{uuid}/rehydrate` โ€” rehydration trigger
+- `POST /api/v1/credentials/{uuid}:rotate` โ€” credential rotation request
+- `POST /api/v1/resources/{uuid}:rehydrate` โ€” rehydration trigger
 
 ### 1.6 Rate Limiting
 
 Rate limits are profile-governed and apply per authenticated actor:
 
-| Profile | Requests/minute | Burst allowance | Rate limit header |
+| Profile | Requests/minute | Burst allowance | Rate page_size header |
 |---------|----------------|-----------------|-------------------|
 | `minimal` | 60 | 20 | Yes |
 | `standard` | 300 | 100 | Yes |
@@ -120,7 +126,7 @@ X-RateLimit-Reset: 1749340800
 
 {
   "error": "rate_limit_exceeded",
-  "message": "Request rate limit exceeded. Retry after 12 seconds.",
+  "message": "Request rate page_size exceeded. Retry after 12 seconds.",
   "retry_after_seconds": 12
 }
 ```
@@ -146,7 +152,7 @@ Include `X-DCM-Request-ID` when contacting support. Use `X-DCM-Correlation-ID` t
   "items": [...],           // always "items" regardless of resource type
   "total": 142,             // total matching records (before pagination)
   "page_size": 25,
-  "next_cursor": "" // null if no more pages; use as ?cursor= on next request
+  "next_page_token": "" // null if no more pages; use as ?page_token= on next request
 }
 ```
 
@@ -286,7 +292,7 @@ Response 204 No Content
 
 ```http
 # Token introspection (RFC 7662) โ€” for internal components and trusted integrations
-POST /api/v1/auth/introspect
+POST /api/v1/auth:introspect
 Authorization: Bearer 
 
 { "token": "" }
@@ -477,7 +483,8 @@ Request body:
   }
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "request_uuid": "",
   "entity_uuid": "",            # the UUID the entity will have when realized
@@ -635,7 +642,8 @@ Cancellation is only available before the PROVISIONING state. Once a provider is
 ```
 DELETE /api/v1/requests/{request_uuid}
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "request_uuid": "",
   "status": "CANCELLING",
@@ -791,7 +799,7 @@ Response 200:
       "submitted_at": "",
       "status": "pending_approval",
       "changed_fields": ["memory_gb"],
-      "approval_url": "/api/v1/resources/{entity_uuid}/provider-notifications/{notification_uuid}/approve"
+      "approval_url": "/api/v1/resources/{entity_uuid}/provider-notifications/{notification_uuid}:approve"
     }
   ]
 }
@@ -812,7 +820,8 @@ Request body:
   "reason": "Renamed to align with new naming convention"
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "update_request_uuid": "",
   "entity_uuid": "",
@@ -832,7 +841,7 @@ Response 422 Unprocessable (if field is not editable):
 ### 5.4 Suspend Resource
 
 ```
-POST /api/v1/resources/{entity_uuid}/suspend
+POST /api/v1/resources/{entity_uuid}:suspend
 
 Request body:
 {
@@ -840,7 +849,8 @@ Request body:
   "auto_resume_at": "2026-03-16T06:00:00Z"    # optional
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "entity_uuid": "",
   "status": "SUSPENDING",
@@ -860,7 +870,8 @@ Request body:
                     # cannot force decommission if required stakes exist
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "entity_uuid": "",
   "status": "DECOMMISSIONING"
@@ -884,7 +895,7 @@ Response 409 Conflict (required stakes or dependencies active):
 ### 5.6 Trigger Rehydration
 
 ```
-POST /api/v1/resources/{entity_uuid}/rehydrate
+POST /api/v1/resources/{entity_uuid}:rehydrate
 
 Request body:
 {
@@ -898,7 +909,8 @@ Request body:
   "reason": "Provider migration โ€” EU-WEST-Prod-1 being decommissioned"
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "rehydration_request_uuid": "",
   "entity_uuid": "",
@@ -950,7 +962,7 @@ Response 200:
   ]
 }
 
-POST /api/v1/resources/{entity_uuid}/provider-notifications/{notification_uuid}/approve
+POST /api/v1/resources/{entity_uuid}/provider-notifications/{notification_uuid}:approve
 {
   "decision": "approve | reject",
   "reason": ""
@@ -1031,7 +1043,7 @@ Response 202 Accepted:
 Decommissions all resources matching a filter. Creates individual decommission requests for each resource. Useful for teardown of environments or project cleanup.
 
 ```
-POST /api/v1/resources/bulk-decommission
+POST /api/v1/resources:bulk-decommission
 
 Request body:
 {
@@ -1061,7 +1073,8 @@ Response 200 (dry_run=true):
   ]
 }
 
-Response 202 Accepted (dry_run=false):
+Response 200 OK โ€” returns `Operation` (dry_run=false):
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "bulk_decommission_uuid": "",
   "decommission_requests": [
@@ -1078,14 +1091,15 @@ Response 202 Accepted (dry_run=false):
 Resumes a suspended resource. The resource must be in SUSPENDED lifecycle state.
 
 ```
-POST /api/v1/resources/{entity_uuid}/resume
+POST /api/v1/resources/{entity_uuid}:resume
 
 Request body:
 {
   "reason": "Maintenance window complete"
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "entity_uuid": "",
   "status": "RESUMING"
@@ -1106,7 +1120,7 @@ Response 409 Conflict:
 Transfers ownership of a resource entity to a different Tenant. Both Tenants must have an active cross-tenant authorization record permitting the transfer. The receiving Tenant admin must confirm the transfer.
 
 ```
-POST /api/v1/resources/{entity_uuid}/transfer
+POST /api/v1/resources/{entity_uuid}:transfer
 
 Request body:
 {
@@ -1135,8 +1149,8 @@ Response 403 Forbidden:
 Target Tenant admin accepts or rejects:
 
 ```
-POST /api/v1/resources/transfers/{transfer_uuid}/accept
-POST /api/v1/resources/transfers/{transfer_uuid}/reject
+POST /api/v1/resources/transfers/{transfer_uuid}:accept
+POST /api/v1/resources/transfers/{transfer_uuid}:reject
 {
   "reason": ""
 }
@@ -1149,7 +1163,7 @@ POST /api/v1/resources/transfers/{transfer_uuid}/reject
 Extends the TTL of a resource entity that has a lifecycle time constraint declared. Extension is subject to policy โ€” a GateKeeper may reject or cap the extension.
 
 ```
-POST /api/v1/resources/{entity_uuid}/extend-ttl
+POST /api/v1/resources/{entity_uuid}:extend-ttl
 
 Request body:
 {
@@ -1205,7 +1219,7 @@ Response 200:
       "expires_at": "",
       "time_remaining": "P2DT4H",
       "on_expiry_action": "decommission",
-      "extend_url": "/api/v1/resources/{entity_uuid}/extend-ttl"
+      "extend_url": "/api/v1/resources/{entity_uuid}:extend-ttl"
     }
   ],
   "total": 3
@@ -1256,7 +1270,7 @@ Response 200:
 Marks a drift record as acknowledged. The entity remains drifted โ€” this signals the owner has reviewed it.
 
 ```
-POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}/acknowledge
+POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}:acknowledge
 {
   "reason": "Reviewing with provider before deciding on action"
 }
@@ -1274,14 +1288,15 @@ Response 200:
 Accepts the discovered state as the new authoritative desired state. Creates a new Requested State and Realized State snapshot reflecting the discovered values. Resolves the drift record.
 
 ```
-POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}/accept
+POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}:accept
 {
   "accept_all_fields": true,         # accept all drifted fields
   "accept_fields": ["fields.memory_gb"],   # or select specific fields
   "reason": "Auto-scale event was legitimate; accepting new memory configuration"
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "drift_uuid": "",
   "status": "resolved",
@@ -1295,12 +1310,13 @@ Response 202 Accepted:
 Submits a revert request โ€” dispatches a new request to restore the resource to its Realized State values.
 
 ```
-POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}/revert
+POST /api/v1/resources/{entity_uuid}/drift/{drift_uuid}:revert
 {
   "reason": "Unauthorized change โ€” reverting to declared state"
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "drift_uuid": "",
   "revert_request_uuid": "",
@@ -1653,7 +1669,7 @@ Response 200:
   "read_at": ""
 }
 
-POST /api/v1/notifications/read-all    # mark all unread as read
+POST /api/v1/notifications:read-all    # mark all unread as read
 
 Response 200:
 {
@@ -1832,7 +1848,7 @@ All error responses follow a consistent structure:
 | 422 | `policy_rejected` | GateKeeper policy rejected the request |
 | 422 | `constraint_violated` | Field value violates declared constraint |
 | 422 | `ttl_extension_rejected` | Policy rejected or capped the TTL extension request |
-| 429 | `rate_limit_exceeded` | Actor has exceeded request rate limit |
+| 429 | `rate_limit_exceeded` | Actor has exceeded request rate page_size |
 | 503 | `assembly_unavailable` | Request Payload Processor temporarily unavailable |
 | 503 | `search_index_degraded` | Search index unavailable; use authoritative_store_ref fallback |
 
@@ -2000,12 +2016,13 @@ Response 410 Gone:  { "error": "credential_revoked_or_expired" }
 ### 9b.3 Request Credential Rotation
 
 ```
-POST /api/v1/credentials/{credential_uuid}/rotate
+POST /api/v1/credentials/{credential_uuid}:rotate
 {
   "reason": "Scheduled rotation per security policy"
 }
 
-Response 202 Accepted:
+Response 200 OK โ€” returns `Operation`:
+> Returns `Operation` resource. Poll `operation.name` for completion.
 {
   "old_credential_uuid": "",
   "new_credential_uuid": "",
@@ -2028,3 +2045,80 @@ The Consumer API defines three conformance levels, mirroring the Operator Interf
 ---
 
 *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+
+## Operations โ€” Polling Long-Running Requests
+
+All async mutating operations return an `Operation` resource. The `operation.name` field
+is the stable polling URL: `GET /api/v1/operations/{operation_uuid}`.
+
+**Key relationship:** `operation_uuid == request_uuid`. The same UUID is used in both the
+AEP-standard Operation endpoint and the DCM-native Request Status endpoint. Two polling
+views are available โ€” use whichever fits your client:
+
+| Endpoint | Schema | Best for |
+|----------|--------|----------|
+| `GET /api/v1/operations/{uuid}` | `Operation` โ€” `done`, `metadata`, `response/error` | AEP-compatible clients, simple polling |
+| `GET /api/v1/requests/{uuid}/status` | `RequestStatus` โ€” `pipeline_stage`, full status history, `entity_uuid` | DCM-native clients, debugging, rich UI |
+
+Both endpoints reflect the same underlying state. When `done: true`, `operation.response`
+contains the realized entity (same as the resource returned by `GET /api/v1/resources/{entity_uuid}`).
+
+```
+POST /api/v1/requests
+
+Response 200 OK โ€” returns Operation:
+{
+  "name": "/api/v1/operations/{request_uuid}",
+  "done": false,
+  "metadata": {
+    "stage": "INITIATED",
+    "resource_uuid": "{entity_uuid}",   // set immediately on entity creation
+    "request_uuid": "{request_uuid}"    // == operation_uuid
+  }
+}
+```
+
+**Polling `GET /api/v1/operations/{operation_uuid}`:**
+
+```
+# While in progress:
+{
+  "name": "/api/v1/operations/{uuid}",
+  "done": false,
+  "metadata": {
+    "stage": "PROVISIONING",
+    "progress_pct": 45,
+    "resource_uuid": "{entity_uuid}",
+    "request_uuid": "{uuid}"
+  }
+}
+
+# On success:
+{
+  "name": "/api/v1/operations/{uuid}",
+  "done": true,
+  "metadata": { "stage": "OPERATIONAL", "resource_uuid": "{entity_uuid}", "request_uuid": "{uuid}" },
+  "response": { ... }   // the realized entity
+}
+
+# On failure:
+{
+  "name": "/api/v1/operations/{uuid}",
+  "done": true,
+  "metadata": { "stage": "FAILED", "request_uuid": "{uuid}" },
+  "error": {
+    "code": "PROVIDER_TIMEOUT",
+    "message": "Provider did not respond within the configured timeout",
+    "details": []
+  }
+}
+```
+
+**Polling guidance:** Use exponential backoff (1s โ†’ 2s โ†’ 5s โ†’ 10s โ†’ 30s).
+For push-based updates, subscribe to the `request.progress_updated` webhook event.
+For real-time browser monitoring, use the SSE stream: `GET /api/v1/requests/{uuid}/stream`.
+
+**Cancellation:** `DELETE /api/v1/requests/{uuid}` cancels an in-progress operation.
+The request enters CANCELLING state; cancellation success depends on provider support.
+
diff --git a/content/docs/architecture/specifications/dcm-admin-api-spec.md b/content/docs/architecture/specifications/dcm-admin-api-spec.md
index 7e684c7..a2169c7 100644
--- a/content/docs/architecture/specifications/dcm-admin-api-spec.md
+++ b/content/docs/architecture/specifications/dcm-admin-api-spec.md
@@ -19,6 +19,13 @@ The Admin API is the platform administration interface for DCM. It is served thr
 
 ---
 
+
+> **AEP Alignment:** This specification follows [AEP](https://aep.dev) conventions.
+> Custom methods use colon syntax (`POST /admin/providers/{uuid}:approve`).
+> Async operations return an `Operation` resource (AEP-136 LRO).
+> List pagination uses `page_size` and `page_token` parameters.
+> See the normative OpenAPI specification: `schemas/openapi/dcm-admin-api.yaml`
+
 ## 1. Authentication and Authorization
 
 All Admin API endpoints require Bearer token authentication (same as Consumer API). Role requirements are declared per endpoint:
@@ -110,8 +117,8 @@ Response 201 Created:
 ### 2.3 Suspend / Reinstate Tenant
 
 ```
-POST /api/v1/admin/tenants/{tenant_uuid}/suspend
-POST /api/v1/admin/tenants/{tenant_uuid}/reinstate
+POST /api/v1/admin/tenants/{tenant_uuid}:suspend
+POST /api/v1/admin/tenants/{tenant_uuid}:reinstate
 Role: platform_admin
 
 {
@@ -177,8 +184,8 @@ New provider registrations in `proposed` status require platform admin review:
 GET /api/v1/admin/providers/pending
 Role: platform_admin
 
-POST /api/v1/admin/providers/{provider_uuid}/approve
-POST /api/v1/admin/providers/{provider_uuid}/reject
+POST /api/v1/admin/providers/{provider_uuid}:approve
+POST /api/v1/admin/providers/{provider_uuid}:reject
 {
   "reason": ""
 }
@@ -187,7 +194,7 @@ POST /api/v1/admin/providers/{provider_uuid}/reject
 ### 3.3 Suspend Provider
 
 ```
-POST /api/v1/admin/providers/{provider_uuid}/suspend
+POST /api/v1/admin/providers/{provider_uuid}:suspend
 Role: platform_admin
 
 {
@@ -228,7 +235,7 @@ Response 200:
 ### 4.2 Approve Accreditation
 
 ```
-POST /api/v1/admin/accreditations/{accreditation_uuid}/approve
+POST /api/v1/admin/accreditations/{accreditation_uuid}:approve
 Role: platform_admin
 Requires: step-up MFA
 
@@ -258,7 +265,7 @@ Requires: step-up MFA
 ### 5.1 Trigger Discovery
 
 ```
-POST /api/v1/admin/discovery/trigger
+POST /api/v1/admin/discovery:trigger
 Role: platform_admin | tenant_admin
 
 {
@@ -411,7 +418,7 @@ Role: platform_admin
 ## 9. Search Index Management
 
 ```
-POST /api/v1/admin/search-index/rebuild
+POST /api/v1/admin/search-index:rebuild
 Role: platform_admin
 
 {
@@ -445,7 +452,7 @@ Response 200:
 ### 10.1 Rotate Bootstrap Admin Credential
 
 ```
-POST /api/v1/admin/bootstrap/rotate-credential
+POST /api/v1/admin/bootstrap:rotate-credential
 Role: platform_admin
 Requires: step-up MFA (hardware_token_mfa for fsi/sovereign)
 
@@ -547,7 +554,7 @@ Platform admins can force-revoke sessions for any actor โ€” used on actor compro
 
 ```http
 # Force-revoke all sessions for an actor
-POST /api/v1/admin/actors/{actor_uuid}/revoke-sessions
+POST /api/v1/admin/actors/{actor_uuid}:revoke-sessions
 Authorization: Bearer 
 
 {
@@ -711,7 +718,7 @@ Response 200:
   "score_half_life_days": 7
 }
 
-POST /api/v1/admin/actors/{actor_uuid}/risk-history/reset
+POST /api/v1/admin/actors/{actor_uuid}/risk-history:reset
 {
   "reason": "Actor confirmed as trusted automation account",
   "audit_note": "Reviewed and approved by platform admin"
@@ -786,7 +793,7 @@ Response 200:
 ### Record an Approval Decision
 
 ```
-POST /api/v1/admin/approvals/{approval_uuid}/vote
+POST /api/v1/admin/approvals/{approval_uuid}:vote
 
 {
   "decision": "approve | reject",
@@ -914,7 +921,7 @@ Response 200:
 ### Accept a Security Degradation
 
 ```
-POST /api/v1/admin/tier-registry/changes/{change_uuid}/accept-degradation
+POST /api/v1/admin/tier-registry/changes/{change_uuid}:accept-degradation
 
 {
   "affected_item_uuid": "",
@@ -938,7 +945,7 @@ Response 409: degradation already accepted
 ### Activate a Tier Registry Change
 
 ```
-POST /api/v1/admin/tier-registry/changes/{change_uuid}/activate
+POST /api/v1/admin/tier-registry/changes/{change_uuid}:activate
 
 Response 200:
 {
@@ -954,7 +961,7 @@ Response 409: change has unresolved blocking items (broken_references or unaccep
 ### List Historical Registry Changes
 
 ```
-GET /api/v1/admin/tier-registry/changes?status=activated&limit=20
+GET /api/v1/admin/tier-registry/changes?status=activated&page_size=20
 
 Response 200:
 {
diff --git a/content/docs/architecture/specifications/dcm-admin-gui-spec.md b/content/docs/architecture/specifications/dcm-admin-gui-spec.md
index f21c7a4..9f1fb06 100644
--- a/content/docs/architecture/specifications/dcm-admin-gui-spec.md
+++ b/content/docs/architecture/specifications/dcm-admin-gui-spec.md
@@ -1,5 +1,8 @@
 # DCM Admin Web GUI Specification
 
+> **AEP Alignment:** Admin API endpoints referenced in this spec follow [AEP](https://aep.dev) conventions โ€” custom methods use colon syntax (`POST /admin/providers/{uuid}:approve`). See `schemas/openapi/dcm-admin-api.yaml` for the normative specification.
+
+
 **Document Status:** ๐Ÿ”„ In Progress
 **Document Type:** Specification โ€” Platform Administration Interface
 **Related Documents:** [Admin API Specification](dcm-admin-api-spec.md) | [Consumer GUI Specification](dcm-consumer-gui-spec.md) | [Provider GUI Specification](dcm-provider-gui-spec.md) | [Flow GUI Specification](dcm-flow-gui-spec.md)
@@ -84,7 +87,7 @@ Dashboard widgets (configurable per role):
 
 ## 3. Tenant Management
 
-**API:** `GET /api/v1/admin/tenants`, `POST /api/v1/admin/tenants`, `POST /api/v1/admin/tenants/{uuid}/suspend`, `POST /api/v1/admin/tenants/{uuid}/reinstate`, `DELETE /api/v1/admin/tenants/{uuid}`
+**API:** `GET /api/v1/admin/tenants`, `POST /api/v1/admin/tenants`, `POST /api/v1/admin/tenants/{uuid}:suspend`, `POST /api/v1/admin/tenants/{uuid}:reinstate`, `DELETE /api/v1/admin/tenants/{uuid}`
 
 - Tenant list with status, member count, resource count, quota utilization
 - Create tenant form: name, description, initial quota set, initial admin member
@@ -96,7 +99,7 @@ Dashboard widgets (configurable per role):
 
 ## 4. Provider Management
 
-**API:** `GET /api/v1/admin/providers`, `GET /api/v1/admin/providers/pending`, `POST /api/v1/admin/providers/{uuid}/approve`, `POST /api/v1/admin/providers/{uuid}/reject`, `POST /api/v1/admin/providers/{uuid}/suspend`
+**API:** `GET /api/v1/admin/providers`, `GET /api/v1/admin/providers/pending`, `POST /api/v1/admin/providers/{uuid}:approve`, `POST /api/v1/admin/providers/{uuid}:reject`, `POST /api/v1/admin/providers/{uuid}:suspend`
 
 > **Full provider management** (configuration, capacity, entity lists, type-specific management) is in the **[Provider GUI](dcm-provider-gui-spec.md)**. This section covers the admin-level registration approval workflow.
 
@@ -110,7 +113,7 @@ Dashboard widgets (configurable per role):
 
 ## 5. Accreditation Management
 
-**API:** `GET /api/v1/admin/accreditations`, `POST /api/v1/admin/accreditations/{uuid}/approve`, `DELETE /api/v1/admin/accreditations/{uuid}`
+**API:** `GET /api/v1/admin/accreditations`, `POST /api/v1/admin/accreditations/{uuid}:approve`, `DELETE /api/v1/admin/accreditations/{uuid}`
 
 - Pending accreditations queue with submission detail
 - Approve / revoke with required comment
@@ -121,7 +124,7 @@ Dashboard widgets (configurable per role):
 
 ## 6. Discovery and Orphan Management
 
-**API:** `POST /api/v1/admin/discovery/trigger`, `GET /api/v1/admin/discovery/jobs/{uuid}`, `GET /api/v1/admin/orphans`, `POST /api/v1/admin/orphans/{uuid}/resolve`
+**API:** `POST /api/v1/admin/discovery:trigger`, `GET /api/v1/admin/discovery/jobs/{uuid}`, `GET /api/v1/admin/orphans`, `POST /api/v1/admin/orphans/{uuid}/resolve`
 
 ### 6.1 Discovery Console
 
@@ -175,7 +178,7 @@ Dashboard widgets (configurable per role):
 
 ## 9. Approval Management
 
-**API:** `GET /api/v1/admin/approvals/pending`, `POST /api/v1/admin/approvals/{uuid}/vote`, `GET /api/v1/admin/approvals/{uuid}`
+**API:** `GET /api/v1/admin/approvals/pending`, `POST /api/v1/admin/approvals/{uuid}:vote`, `GET /api/v1/admin/approvals/{uuid}`
 
 - All pending approvals across all tenants (Platform Admin view) vs own queue (approver view)
 - Filter by tier (reviewed / verified / authorized), resource type, tenant, age
@@ -187,7 +190,7 @@ Dashboard widgets (configurable per role):
 
 ## 10. Authority Tier Registry
 
-**API:** `POST /api/v1/admin/tier-registry/changes`, `GET /api/v1/admin/tier-registry/changes/{uuid}/impact`, `POST /api/v1/admin/tier-registry/changes/{uuid}/accept-degradation`, `POST /api/v1/admin/tier-registry/changes/{uuid}/activate`
+**API:** `POST /api/v1/admin/tier-registry/changes`, `GET /api/v1/admin/tier-registry/changes/{uuid}/impact`, `POST /api/v1/admin/tier-registry/changes/{uuid}:accept-degradation`, `POST /api/v1/admin/tier-registry/changes/{uuid}:activate`
 
 - Current tier registry: ordered list display (auto โ†’ reviewed โ†’ verified โ†’ authorized โ†’ [custom tiers])
 - Propose change: drag-and-drop reordering with add/remove custom tier
@@ -212,7 +215,7 @@ Visible to `auditor` and `platform_admin` roles.
 
 ## 12. Session and Security Management
 
-**API:** `GET /api/v1/admin/actors/{uuid}/...`, `POST /api/v1/admin/actors/{uuid}/revoke-sessions`
+**API:** `GET /api/v1/admin/actors/{uuid}/...`, `POST /api/v1/admin/actors/{uuid}:revoke-sessions`
 
 Visible to `security` and `platform_admin` roles.
 
@@ -225,7 +228,7 @@ Visible to `security` and `platform_admin` roles.
 
 ## 13. Health and Operations
 
-**API:** `GET /api/v1/admin/health`, `GET /api/v1/admin/discovery/trigger`, `POST /api/v1/admin/search-index/rebuild`
+**API:** `GET /api/v1/admin/health`, `GET /api/v1/admin/discovery:trigger`, `POST /api/v1/admin/search-index:rebuild`
 
 Visible to `sre` and `platform_admin` roles.
 
diff --git a/content/docs/architecture/specifications/dcm-consumer-gui-spec.md b/content/docs/architecture/specifications/dcm-consumer-gui-spec.md
index 1ce75b7..e97bd01 100644
--- a/content/docs/architecture/specifications/dcm-consumer-gui-spec.md
+++ b/content/docs/architecture/specifications/dcm-consumer-gui-spec.md
@@ -1,5 +1,8 @@
 # DCM Consumer Web GUI Specification
 
+> **AEP Alignment:** Consumer API endpoints referenced in this spec follow [AEP](https://aep.dev) conventions โ€” custom methods use colon syntax, async operations return `Operation` resources, and `operation_uuid == request_uuid`. See `schemas/openapi/dcm-consumer-api.yaml` for the normative specification.
+
+
 **Document Status:** ๐Ÿ”„ In Progress
 **Document Type:** Specification โ€” Consumer Web Interface
 **Related Documents:** [RHDH Integration Specification](dcm-rhdh-integration-spec.md) | [Consumer API Specification](consumer-api-spec.md) | [Admin GUI Specification](dcm-admin-gui-spec.md) | [Provider GUI Specification](dcm-provider-gui-spec.md) | [Flow GUI Specification](dcm-flow-gui-spec.md) | [Auth Providers](../data-model/19-auth-providers.md) | [Session Revocation](../data-model/35-session-revocation.md)
@@ -548,7 +551,7 @@ For organizations that require ITSM change board approval, DCM's `authorized` ti
   โ”‚
   โ–ผ Change Board reviews in ServiceNow (existing process unchanged)
   โ”‚   Approval decision โ†’ ServiceNow calls DCM Admin API:
-  โ”‚   POST /api/v1/admin/approvals/{uuid}/vote
+  โ”‚   POST /api/v1/admin/approvals/{uuid}:vote
   โ”‚   { "decision": "approve", "recorded_via": "servicenow", "voter_uuid": "..." }
   โ”‚
   โ–ผ DCM records vote; quorum tracked by DCM
diff --git a/content/docs/architecture/specifications/dcm-examples.md b/content/docs/architecture/specifications/dcm-examples.md
index 4a13734..5394f6b 100644
--- a/content/docs/architecture/specifications/dcm-examples.md
+++ b/content/docs/architecture/specifications/dcm-examples.md
@@ -166,11 +166,11 @@ After Step 3 (dynamic policies evaluate):
 โ†’ Notification dispatched:
     audience: manager (from actor's group membership via relationship graph)
     event_type: request.requires_approval
-    action_url: /api/v1/requests/req-001/approve
+    action_url: /api/v1/requests/req-001:approve
     action_deadline: PT24H
 
 Manager approves:
-POST /api/v1/requests/req-001/approve
+POST /api/v1/requests/req-001:approve
 { "approval_type": "manager_approval", "approver_uuid": "mgr-001" }
 
 โ†’ payload.approvals["manager_approval"] = { approved: true, by: "mgr-001" }
@@ -432,10 +432,10 @@ Step 2: Event: ingestion.transitional_created
 Step 3: Event: ingestion.enriched
 โ†’ Notification to Platform Admin:
   "Brownfield entity discovered. Review and assign to Tenant."
-  action_url: /api/v1/admin/ingestion/ing-001/promote
+  action_url: /api/v1/admin/ingestion/ing-001:promote
 
 Step 4: Operator approves:
-POST /api/v1/admin/ingestion/ing-001/promote
+POST /api/v1/admin/ingestion/ing-001:promote
 { "target_tenant_uuid": "payments-tenant-uuid",
   "compliance_overlay": "pci-dss" }
 
@@ -447,6 +447,190 @@ POST /api/v1/admin/ingestion/ing-001/promote
 
 ---
 
+
+## 1.9 VM Lifecycle โ€” Static Replace
+
+Re-provision a VM using its existing Requested State payload exactly as it was
+dispatched, without re-running layer enrichment or policy evaluation. The result
+is a functionally identical resource to the one being replaced.
+
+**When to use Static Replace vs Rehydration:**
+
+| | Static Replace | Rehydration |
+|-|---------------|-------------|
+| Uses | Original Requested State (the exact dispatch payload) | Original Intent State (what the consumer asked for) |
+| Policy re-evaluation | No โ€” payload is used as-is | Yes โ€” full layer enrichment + policy evaluation runs again |
+| Standards compliance | Reflects policies at time of original provisioning | Reflects current policies and current layers |
+| Use case | Known-good rebuild, emergency restore, hardware swap | Standards refresh, datacenter migration, DR in a new zone |
+| AEP endpoint | `POST /api/v1/resources/{entity_uuid}:rehydrate` with `mode: static` | `POST /api/v1/resources/{entity_uuid}:rehydrate` with `mode: intent` |
+
+**Preconditions:**
+- Entity exists in DCM with status `OPERATIONAL` or `SUSPENDED`
+- A Requested State record exists (all DCM-provisioned resources have one)
+- The Service Provider that originally provisioned the resource is still registered and healthy
+- The target resource type has not had a breaking schema change (VER-009)
+
+**Workflow:**
+
+```
+Consumer: POST /api/v1/resources/{entity_uuid}:rehydrate
+  {
+    "mode": "static",
+    "reason": "Hardware failure on host โ€” replacing on equivalent host in same zone",
+    "target_zone": null,             // null = same zone as original
+    "retain_entity_uuid": true,      // DCM entity UUID is preserved
+    "pre_rehydration_backup": true   // optional: snapshot before proceeding
+  }
+
+Response 200 OK โ€” returns Operation:
+  {
+    "name": "/api/v1/operations/{request_uuid}",
+    "done": false,
+    "metadata": {
+      "stage": "REHYDRATION_INITIATED",
+      "resource_uuid": "{entity_uuid}",
+      "rehydration_mode": "static",
+      "source_requested_state_uuid": "{original_requested_state_uuid}"
+    }
+  }
+```
+
+**What DCM does:**
+
+```
+1. Retrieve the most recent Requested State record for entity_uuid
+   (this is the exact payload that was dispatched at original provisioning time)
+
+2. Entity enters REHYDRATING lifecycle state
+   (incoming traffic should be shifted away at LTM/GTM level before initiating)
+
+3. Decommission the existing resource via the Service Provider
+   DELETE /{resource_id} โ†’ operator removes the resource
+   Decommission callback received โ†’ entity status: DECOMMISSIONED (transient)
+
+4. Re-dispatch the original Requested State payload to the Service Provider
+   POST / with the original CreateRequest body
+   No layer enrichment โ€” payload is used verbatim
+   No policy re-evaluation โ€” payload is used verbatim
+   NOTE: resource_type_uuid and resource_type_name are still validated
+         against the current Resource Type Registry (VER-009 compatibility check)
+
+5. Realization callback received โ†’ entity returns to OPERATIONAL
+   A new Realized State record is written (linked to the original Requested State)
+   The entity_uuid is preserved โ€” external references remain valid
+
+6. Operation reaches done: true
+   operation.response contains the realized entity
+```
+
+**The key distinction from Rehydration (intent mode):**
+Static Replace bypasses the entire layer assembly and policy evaluation pipeline.
+It takes the already-assembled, already-approved Requested State and re-executes it.
+This makes it deterministic โ€” the result is the same resource on equivalent hardware.
+If standards have changed since the original provisioning and you need the resource
+to comply with current standards, use `mode: intent` (Rehydration) instead.
+
+**Precaution โ€” IaC parity:**
+For Static Replace to be reliable, the application and its data must be on a separate
+partition or external storage. The VM's OS and configuration layers are what Static
+Replace rebuilds. Application data on the OS volume will be lost. This mirrors the
+assumption stated in the PDF architecture: *"Application install and data exist on
+separate partition."*
+
+**Orchestration Flow Policy (Static Replace):**
+
+```rego
+package dcm.orchestration.static_replace
+
+# Fired when consumer requests static rehydration
+steps := [
+    {"step": 1, "payload_type": "lifecycle.rehydration_requested",
+     "condition": "payload.mode == 'static'",
+     "policy_handle": "system/lifecycle/validate-static-replace-preconditions"},
+    {"step": 2, "payload_type": "lifecycle.static_replace_validated",
+     "policy_handle": "system/provider/decommission-for-replace"},
+    {"step": 3, "payload_type": "lifecycle.decommission_confirmed",
+     "policy_handle": "system/provider/dispatch-original-requested-state"},
+    {"step": 4, "payload_type": "realization.completed",
+     "policy_handle": "system/lifecycle/restore-operational-state"},
+]
+```
+
+**Related use cases:** See Section 1.8 (Brownfield Ingestion) for bringing existing
+resources under DCM management. See Section 1.5 (Drift Detection) for reconciling
+drift rather than replacing. See the Ingestion Model (doc 13) for the `mode: intent`
+Rehydration flow (replaying intent through current policies).
+
+---
+
+## 1.10 VM Lifecycle โ€” In-Place Upgrade (Leapp / IPU Pattern)
+
+Upgrade the OS of a running VM in-place, managed as a DCM lifecycle event.
+This preserves the VM entity UUID, Requested State, and Realized State chain โ€”
+the VM is the same DCM entity before and after the upgrade.
+
+**Preconditions:**
+- Entity is `OPERATIONAL`
+- The upgrade automation exists as a registered Process Resource Type
+  (e.g., `Process.LeappUpgrade`, `Process.OSUpgrade`)
+- A backup or snapshot policy is active for this entity
+
+**Workflow:**
+
+```
+Consumer: POST /api/v1/requests
+  {
+    "catalog_item_uuid": "{leapp-upgrade-catalog-item-uuid}",
+    "fields": {
+      "target_entity_uuid": "{vm-entity-uuid}",
+      "target_os_version": "RHEL 9.4",
+      "pre_upgrade_snapshot": true,
+      "maintenance_window_uuid": "{mw-uuid}"   // optional
+    }
+  }
+```
+
+**What DCM does:**
+
+```
+1. Policy Engine validates:
+   - Target entity is OPERATIONAL
+   - Target OS version is in the approved versions list (Core Policy)
+   - Maintenance window is active (if required by policy)
+   - Pre-upgrade snapshot capability exists on the provider
+
+2. Entity enters UPDATING lifecycle state
+   (DCM marks entity in maintenance โ€” routing at LTM layer should drain)
+
+3. Process Resource entity created (Process.LeappUpgrade)
+   UUID assigned; linked to the VM entity via 'operational' relationship
+   Dispatched to the Service Provider as a process execution request
+
+4. Service Provider executes upgrade automation
+   Interim status callbacks update the Process Resource entity status
+   VM entity remains UPDATING throughout
+
+5. Post-upgrade validation runs:
+   - Health checks pass
+   - OS version matches target_os_version in the Realized State
+
+6. New Realized State record written for the VM entity
+   delta_fields: {os_version: "RHEL 9.4", last_upgraded_at: }
+   The Process Resource entity moves to DECOMMISSIONED (process complete)
+
+7. Entity returns to OPERATIONAL
+   Maintenance mode released โ€” routing restored
+```
+
+**Key DCM properties preserved:**
+The VM entity UUID does not change. The Requested State (original intent) does not
+change. The upgrade is recorded as a new Realized State record with `delta_fields`
+carrying the changed values, linked to the prior Realized State. The full provenance
+chain is intact for audit.
+
+---
+
+
 # Section 2 โ€” Provider Interaction Examples
 
 ## 2.1 Service Provider โ€” Full Dispatch Cycle
@@ -714,7 +898,7 @@ Response: { "notifications": [{
 }] }
 
 # Consumer approves
-POST /api/v1/resources/ent-001/provider-notifications/notif-001/approve
+POST /api/v1/resources/ent-001/provider-notifications/notif-001:approve
 { "decision": "approve", "reason": "Legitimate auto-scale event" }
 
 Response 202: { "decision": "approve", "realized_state_uuid": "real-002" }
@@ -752,7 +936,7 @@ Response: { "registrations": [{
 }] }
 
 # Admin reviews and approves
-POST /api/v1/admin/registrations/reg-001/approve
+POST /api/v1/admin/registrations/reg-001:approve
 { "review_notes": "Certificate verified against corp CA. BAA reviewed and valid." }
 
 Response: { "registration_uuid": "reg-001", "status": "ACTIVE" }
diff --git a/content/docs/architecture/specifications/dcm-flow-gui-spec.md b/content/docs/architecture/specifications/dcm-flow-gui-spec.md
index 364c790..1b563f3 100644
--- a/content/docs/architecture/specifications/dcm-flow-gui-spec.md
+++ b/content/docs/architecture/specifications/dcm-flow-gui-spec.md
@@ -13,6 +13,14 @@
 
 ## Abstract
 
+> **AEP Alignment:** This specification follows [AEP](https://aep.dev) conventions.
+> Custom methods use colon syntax (e.g., `POST /flow/api/v1/shadow/{uuid}:promote`).
+> List endpoints use `page_size` and `page_token` parameters.
+> The Flow GUI API is a dedicated backend for the visual tooling โ€” it is separate from the
+> main Consumer and Admin APIs. See `schemas/openapi/dcm-consumer-api.yaml` for the
+> normative consumer-facing API specification.
+
+
 The DCM Flow GUI is the visual interface for platform engineers to compose, test, and manage DCM's data-driven orchestration. Because policies ARE the orchestration in DCM, the Flow GUI is fundamentally a **visual policy composer** โ€” it makes the active policy graph visible and editable without requiring direct YAML or Rego authoring.
 
 The Flow GUI is a **platform engineer tool**, not a consumer tool. It operates with platform admin or policy author role permissions. Consumers interact with DCM through the Consumer API and Web UI, not through the Flow GUI.
@@ -451,7 +459,7 @@ POST /flow/api/v1/policies/{policy_uuid}/tests/from-request
 }
 
 # Run all test cases
-POST /flow/api/v1/policies/{policy_uuid}/tests/run
+POST /flow/api/v1/policies/{policy_uuid}/tests:run
 
 Response 200:
 {
@@ -639,7 +647,7 @@ Response 200:
 ### 6.4 API โ€” Promote Shadow Policy to Active
 
 ```
-POST /flow/api/v1/shadow/{policy_uuid}/promote
+POST /flow/api/v1/shadow/{policy_uuid}:promote
 {
   "reason": "Shadow results reviewed โ€” divergence rate acceptable; promoting to active"
 }
@@ -721,6 +729,69 @@ Response 200:
 
 ---
 
+### 7.3 API โ€” Update Scoring Thresholds
+
+```
+PATCH /flow/api/v1/profile/scoring
+
+Authorization: Bearer 
+
+Request body:
+{
+  "thresholds": [
+    { "tier": "auto",       "max_score": 24 },
+    { "tier": "reviewed",   "max_score": 59 },
+    { "tier": "verified",   "max_score": 79 },
+    { "tier": "authorized", "max_score": 100 }
+  ],
+  "policy_overrides": [
+    {
+      "policy_uuid": "",
+      "enforcement_class_override": "compliance",
+      "reason": "Regulatory requirement"
+    }
+  ]
+}
+
+Response 200:
+{
+  "active_profile": "standard",
+  "thresholds": [ ... ],
+  "preview": {
+    "last_7d_auto_approve_pct": 61.2,
+    "last_7d_reviewed_pct": 31.5,
+    "last_7d_verified_pct": 7.3
+  }
+}
+```
+
+**Constraint:** `auto` tier `max_score` cannot exceed 50 (SMX-008 hard cap). The API
+returns `422 Unprocessable Entity` if this constraint is violated.
+
+### 7.4 API โ€” Payload Type Browser
+
+```
+GET /flow/api/v1/payload-types
+
+Response 200:
+{
+  "payload_types": [
+    {
+      "type": "request.initiated",
+      "description": "A new service request has been received",
+      "fields": [
+        { "path": "catalog_item_uuid", "type": "uuid", "required": true },
+        { "path": "tenant_uuid",       "type": "uuid", "required": true },
+        { "path": "fields",            "type": "object", "required": true }
+      ],
+      "policy_types_applicable": ["gatekeeper", "validation", "transformation",
+                                   "recovery", "orchestration_flow"]
+    }
+  ]
+}
+```
+
+
 ## 8. Notification Flow View
 
 ### 8.1 API โ€” Notification Flow for an Entity
@@ -796,11 +867,39 @@ All Flow GUI API errors follow the standard DCM error format:
 
 ## 10. Conformance Levels
 
-**Level 1 โ€” Read-Only:** Execution Graph View (read), Profile View, Payload Type Browser, Notification Flow View. Suitable for dashboards and observability integrations.
-
-**Level 2 โ€” Standard:** All Level 1 plus Flow Simulation, Shadow Mode Dashboard (view only), Policy Node Detail. Required for platform engineer tooling.
-
-**Level 3 โ€” Full:** All Level 2 plus Policy Canvas (save as PR), Policy Authoring Interface, Test Case Management, Shadow Mode Promotion. Required for full policy lifecycle management.
+| Level | Name | Capabilities | Intended Use |
+|-------|------|-------------|-------------|
+| 1 | Read-Only | Execution Graph View (read), Profile View, Payload Type Browser, Notification Flow View, Scoring Overlay | Dashboards, observability integrations, read-only monitoring tools |
+| 2 | Standard | All Level 1 + Flow Simulation, Shadow Mode Dashboard (view only), Policy Node Detail, Scoring Simulation | Platform engineer tooling, policy review workflows |
+| 3 | Full | All Level 2 + Policy Canvas (save as PR), Policy Authoring Interface, Test Case Management, Shadow Mode Promotion | Full policy lifecycle management, policy authoring tooling |
+
+### 10.1 Conformance Implementation Checklist
+
+**Level 1 โ€” minimum required endpoints:**
+- `GET /flow/api/v1/graph`
+- `GET /flow/api/v1/graph/nodes/{policy_uuid}`
+- `GET /flow/api/v1/profile`
+- `GET /flow/api/v1/payload-types`
+- `GET /flow/api/v1/notifications/flow/{entity_uuid}`
+- `GET /flow/api/v1/graph/scoring-overlay`
+
+**Level 2 โ€” adds:**
+- `POST /flow/api/v1/simulate`
+- `POST /flow/api/v1/simulate/score`
+- `GET /flow/api/v1/shadow`
+- `GET /flow/api/v1/shadow/{policy_uuid}`
+
+**Level 3 โ€” adds:**
+- `GET /flow/api/v1/canvas/preview`
+- `POST /flow/api/v1/canvas/save`
+- `GET /flow/api/v1/policies/{policy_uuid}/canvas`
+- `POST /flow/api/v1/policies/generate`
+- `POST /flow/api/v1/policies/validate-rego`
+- `GET /flow/api/v1/policies/{policy_uuid}/tests`
+- `POST /flow/api/v1/policies/{policy_uuid}/tests/from-request`
+- `POST /flow/api/v1/policies/{policy_uuid}/tests:run`
+- `POST /flow/api/v1/shadow/{policy_uuid}:promote`
+- `PATCH /flow/api/v1/profile/scoring`
 
 ---
 
@@ -828,14 +927,12 @@ GET /flow/api/v1/graph/scoring-overlay
 Response 200:
 {
   "active_profile": "standard",
-  "thresholds": {
-    "auto_approve_below": 25,
-    "approval_routing": [
-      { "tier": "reviewed", "max_score": 59 },
-      { "tier": "verified", "max_score": 79 },
-      { "tier": "authorized", "max_score": 100 }
-    ]
-  },
+  "thresholds": [
+    { "tier": "auto",       "max_score": 24 },
+    { "tier": "reviewed",   "max_score": 59 },
+    { "tier": "verified",   "max_score": 79 },
+    { "tier": "authorized", "max_score": 100 }
+  ],
   "nodes": [
     {
       "node_id": "",
diff --git a/content/docs/architecture/specifications/dcm-opa-integration-spec.md b/content/docs/architecture/specifications/dcm-opa-integration-spec.md
index 4a09ae9..bb13777 100644
--- a/content/docs/architecture/specifications/dcm-opa-integration-spec.md
+++ b/content/docs/architecture/specifications/dcm-opa-integration-spec.md
@@ -1,5 +1,8 @@
 # DCM OPA Integration Specification
 
+> **AEP Alignment:** DCM API endpoints referenced in this spec follow [AEP](https://aep.dev) conventions. `resource_type` accepts FQN string or Registry UUID โ€” DCM resolves internally. See `schemas/openapi/dcm-consumer-api.yaml` and `dcm-admin-api.yaml`.
+
+
 > **๐Ÿ“‹ Draft**
 >
 > This specification has been promoted from Work in Progress to Draft status. All questions resolved. All 7 policy types validated with working Rego examples. OPA/Rego confirmed as complete reference implementation. It is ready for implementation feedback but has not yet been formally reviewed for final release.
diff --git a/content/docs/architecture/specifications/dcm-operator-interface-spec.md b/content/docs/architecture/specifications/dcm-operator-interface-spec.md
index 3715a76..4408748 100644
--- a/content/docs/architecture/specifications/dcm-operator-interface-spec.md
+++ b/content/docs/architecture/specifications/dcm-operator-interface-spec.md
@@ -42,7 +42,7 @@ Operators conforming to this specification function as Service Providers within
 
 ## 1. Introduction
 
-> **OIS Versioning:** Providers declare the OIS version they implement in capability registration (`ois_version`). DCM maintains dispatch compatibility with all supported OIS versions during the deprecation window. See [API Versioning Strategy](../data-model/34-api-versioning-strategy.md) Section 7.
+> **OIS Versioning:** Providers declare the OIS version they implement in capability registration (`ois_version`). DCM maintains dispatch compatibility with all supported OIS versions. See [API Versioning Strategy](../data-model/34-api-versioning-strategy.md) Section 7.
 
 
 ### 1.1 Motivation
@@ -217,7 +217,7 @@ provider_registration:
 ```yaml
 # Success response
 provider_registration_response:
-  provider_id: 
+  provider_uuid: 
   name: 
   status: 
   conformance_level_accepted: <1|2|3>
@@ -309,7 +309,7 @@ DCM maintains an internal capacity rating per operator, per service type, per lo
 
 ### 5.2 Capacity Registration
 
-**DCM endpoint:** `POST /api/v1/providers/{provider_id}/capacity`
+**DCM endpoint:** `POST /api/v1/providers/{provider_uuid}/capacity`
 
 ```yaml
 capacity_report:
@@ -374,6 +374,7 @@ DCM sends the Requested State payload to the operator. The operator naturalizes
 create_request:
   request_id: 
   tenant_uuid: 
+  # Both resource_type_uuid and resource_type_name are always present โ€” DCM resolves from consumer input
   resource_type_uuid: 
   resource_type_name: Storage.Database
   spec:
diff --git a/content/docs/architecture/specifications/dcm-operator-sdk-api.md b/content/docs/architecture/specifications/dcm-operator-sdk-api.md
index 2535771..359f82d 100644
--- a/content/docs/architecture/specifications/dcm-operator-sdk-api.md
+++ b/content/docs/architecture/specifications/dcm-operator-sdk-api.md
@@ -1,5 +1,16 @@
 # DCM Operator SDK โ€” API Design
 
+**Document Status:** โœ… Complete
+**Document Type:** SDK Reference (Go)
+**Related Documents:** [Operator Interface Specification](dcm-operator-interface-spec.md) | [Kubernetes Compatibility](11-kubernetes-compatibility.md) | [Provider Callback Auth](../data-model/43-provider-callback-auth.md) | [Registration Specification](dcm-registration-spec.md)
+
+> **AEP Alignment:** DCM interaction uses colon-syntax custom methods (`:approve`, `:suspend`).
+> `operation_uuid == request_uuid` โ€” Operations polling uses `GET /api/v1/operations/{uuid}`.
+> `resource_type` accepts both FQN string (`Compute.VirtualMachine`) and Registry UUID;
+> DCM resolves either form internally. See `schemas/openapi/dcm-operator-api.yaml`
+> for the normative operator-facing OpenAPI specification.
+
+
 
 > ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress
 >
@@ -640,8 +651,56 @@ func (r *ClusterReconciler) Reconcile(
 
 ---
 
+## 11. Callback Credential Management
+
+The SDK manages the provider callback credential lifecycle automatically.
+
+### 11.1 Credential Storage
+
+```go
+// CallbackCredential is managed internally by the SDK.
+// Operators do not need to handle credential rotation manually.
+type CallbackCredential struct {
+    Value      string    // Bearer token โ€” never logged
+    ValidUntil time.Time // Pre-rotation begins at 50% of lifetime
+    UUID       string    // For audit correlation
+}
+```
+
+### 11.2 Automatic Rotation
+
+The SDK initiates credential rotation before expiry (at 50% of the credential lifetime). During the transition window, the SDK accepts both the old and new credentials simultaneously. Operators do not need to handle rotation โ€” the SDK does it transparently.
+
+```go
+// The SDK emits a credential rotation event when rotation completes.
+// Operators can subscribe to be notified (e.g., to update secret stores).
+sdk.OnCredentialRotated(func(old, new CallbackCredential) {
+    // Optional: persist new credential to external secret store
+    log.Info("credential rotated", "new_uuid", new.UUID)
+})
+```
+
+### 11.3 entity_uuid and resource_id
+
+The SDK ensures `dcm_entity_uuid` is echoed in every response and callback. `resource_id` is the operator's own stable identifier. DCM uses `dcm_entity_uuid` for all internal routing โ€” `resource_id` is stored by DCM as a correlation handle but never used for routing or identity.
+
+```go
+// The SDK automatically populates dcm_entity_uuid from the CreateRequest.
+// Operators set resource_id to their own stable identifier.
+resp := &CreateResponse{
+    ResourceID:   myInternalID,           // operator-assigned
+    DCMEntityUUID: req.DCMEntityUUID,     // echoed from CreateRequest โ€” SDK validates this
+    LifecycleState: StatePROVISIONING,
+}
+```
+
+---
+
 ## 10. Open Questions
 
+> All questions resolved. See Resolution Notes below.
+
+
 | # | Question | Impact | Status |
 |---|----------|--------|--------|
 | 1 | Should the SDK support non-Go operator frameworks via a language-agnostic REST adapter? | Ecosystem breadth | โœ… Resolved |
diff --git a/content/docs/architecture/specifications/dcm-provider-gui-spec.md b/content/docs/architecture/specifications/dcm-provider-gui-spec.md
index aa29b51..3fe5fba 100644
--- a/content/docs/architecture/specifications/dcm-provider-gui-spec.md
+++ b/content/docs/architecture/specifications/dcm-provider-gui-spec.md
@@ -1,5 +1,8 @@
 # DCM Provider Management GUI Specification
 
+> **AEP Alignment:** Provider API endpoints referenced in this spec follow [AEP](https://aep.dev) conventions โ€” custom methods use colon syntax. See `schemas/openapi/dcm-admin-api.yaml` for the normative specification.
+
+
 **Document Status:** ๐Ÿ”„ In Progress
 **Document Type:** Specification โ€” Provider Management Interface
 **Related Documents:** [Unified Provider Contract](../data-model/A-provider-contract.md) | [OIS Specification](dcm-operator-interface-spec.md) | [Registration Specification](dcm-registration-spec.md) | [Admin GUI Specification](dcm-admin-gui-spec.md) | [Credential Provider Model](../data-model/31-credential-provider-model.md)
diff --git a/content/docs/architecture/specifications/dcm-registration-spec.md b/content/docs/architecture/specifications/dcm-registration-spec.md
index 3a9a44c..50f58ce 100644
--- a/content/docs/architecture/specifications/dcm-registration-spec.md
+++ b/content/docs/architecture/specifications/dcm-registration-spec.md
@@ -1,5 +1,8 @@
 # DCM Registration Specification
 
+> **AEP Alignment:** Registration API endpoints follow [AEP](https://aep.dev) conventions โ€” custom methods use colon syntax (`POST /admin/registrations/{uuid}:approve`). `resource_type` in provider capabilities accepts FQN string or Registry UUID. See `schemas/openapi/dcm-admin-api.yaml` for the normative specification.
+
+
 > **๐Ÿ“‹ Draft**
 >
 > This specification has been promoted from Work in Progress to Draft status. All questions resolved. Complete registration pipeline for all 11 provider types with full capability declaration schemas and federation trust model. It is ready for implementation feedback but has not yet been formally reviewed for final release.
@@ -345,8 +348,8 @@ Registration enters PENDING_APPROVAL
 Platform admin notification dispatched (urgency: medium)
 Platform admin reviews in Admin API or Flow GUI:
   GET /api/v1/admin/registrations/pending
-  POST /api/v1/admin/registrations/{registration_uuid}/approve
-  POST /api/v1/admin/registrations/{registration_uuid}/reject
+  POST /api/v1/admin/registrations/{registration_uuid}:approve
+  POST /api/v1/admin/registrations/{registration_uuid}:reject
 On approval: โ†’ ACTIVE
 On rejection: โ†’ REJECTED with required reason field
 On timeout (approval_timeout): โ†’ REJECTED with reason "approval_timeout"
@@ -415,6 +418,32 @@ service_provider_capabilities:
     discovery_method: api_query | passive_event | hybrid
     supports_incremental_discovery: true
 
+  monitoring:
+    # Prometheus metrics endpoint โ€” required for 1.0 readiness
+    metrics_endpoint: /metrics            # must return Prometheus text format
+    metrics_port: 8080                    # or same as operator endpoint
+    
+    # Required metric families (must be present at activation):
+    required_metrics:
+      - dcm_provider_dispatches_total      # {resource_type, outcome}
+      - dcm_provider_dispatch_duration_seconds  # {resource_type, quantile}
+      - dcm_provider_realizations_total    # {resource_type, status}
+      - dcm_provider_health_status         # 1=healthy, 0=unhealthy
+    
+    # Optional but recommended:
+    optional_metrics:
+      - dcm_provider_queue_depth           # pending dispatch requests
+      - dcm_provider_capacity_remaining    # {resource_type}
+    
+    # AEP.DEV linting โ€” required for 1.0 readiness gate
+    aep_linting:
+      passes_aep_linting: true            # must pass aep.dev linter before activation
+      linting_report_ref:            # link to linting report
+    
+    # Tenant metadata endpoint โ€” required for multi-tenant readiness
+    tenant_metadata_endpoint: /api/v1/tenants/{tenant_uuid}/metadata
+    # Returns: usage by tenant, quota consumed, active resources by type
+
   naturalization:
     target_format: openstack_nova | vmware_vsphere | custom
     custom_schema_ref: 
@@ -760,7 +789,7 @@ certificate_rotation:
   pre_rotation_warning: P14D        # warn provider P14D before expiry
 
 # Rotation flow:
-POST /api/v1/provider/certificates/rotate
+POST /api/v1/provider/certificates:rotate
 {
   "new_certificate_pem": "",
   "transition_window": "P7D"
@@ -814,6 +843,44 @@ Immediate effect:
 
 ---
 
+### 7.2 Provider 1.0 Readiness Gates
+
+Before a Service Provider can be activated in `standard`, `prod`, `fsi`, or `sovereign`
+profiles, the following readiness gates must pass. These align with the DCM roadmap's
+1.0 criteria for Service Provider deployment:
+
+| Gate | Requirement | Profiles Required |
+|------|------------|-------------------|
+| `GATE-SP-01` | Simple OpenAPI Spec โ€” declared at registration, URL reachable | all |
+| `GATE-SP-02` | Healthy API โ€” health endpoint returns `{"status": "healthy"}` at activation | all |
+| `GATE-SP-03` | State Management โ€” implements realized_state_push callback | all |
+| `GATE-SP-04` | Tenant Metadata โ€” endpoint declared or implemented | standard+ |
+| `GATE-SP-05` | Prometheus Metrics โ€” required metric families present at declared endpoint | standard+ |
+| `GATE-SP-06` | AEP.DEV Linting โ€” OpenAPI spec passes AEP linter with no errors | standard+ |
+| `GATE-SP-07` | Multi-Tenant Ready โ€” accepts tenant_uuid in all dispatch payloads | standard+ |
+
+DCM evaluates readiness gates automatically during the approval pipeline. A provider
+that fails a gate is rejected with a `READINESS_GATE_FAILED` error listing which
+gates failed and what is needed to pass.
+
+**Required metric families (GATE-SP-05):**
+
+```
+dcm_provider_dispatches_total{resource_type, outcome}
+dcm_provider_dispatch_duration_seconds{resource_type, quantile}
+dcm_provider_realizations_total{resource_type, status}
+dcm_provider_health_status   # 1=healthy, 0=unhealthy/degraded
+```
+
+**AEP linting (GATE-SP-06):**
+Run the AEP linter against the provider's OpenAPI spec before registration.
+Common failures: slash-verb paths instead of colon syntax, missing page_size
+on list endpoints, 202 responses without Operation resource on async operations.
+The linting report URL should be included in the monitoring capability declaration.
+
+---
+
+
 ## 8. Error Model
 
 | Error Code | Meaning |
diff --git a/content/docs/architecture/specifications/dcm-rhdh-integration-spec.md b/content/docs/architecture/specifications/dcm-rhdh-integration-spec.md
index 49fe9fa..708023b 100644
--- a/content/docs/architecture/specifications/dcm-rhdh-integration-spec.md
+++ b/content/docs/architecture/specifications/dcm-rhdh-integration-spec.md
@@ -603,35 +603,21 @@ Organizations using OCM for cluster lifecycle management get cluster management
 
 ---
 
-## 9. Migration Path โ€” Standalone SPA โ†’ RHDH
+## 9. Deployment Options
 
-Organizations starting with the standalone SPA can migrate to RHDH mode without data migration:
+DCM supports two frontend deployment modes that can be selected at initial deployment:
 
-```
-Phase 1 โ€” Standalone SPA
-  DCM deployed; standalone React app as consumer portal
-  All DCM functionality operational
-
-Phase 2 โ€” Install RHDH
-  RHDH deployed in same cluster
-  Configure Keycloak/RHSSO (shared IdP)
-
-Phase 3 โ€” Load DCM Dynamic Plugins
-  Add DCM plugins to RHDH app-config
-  No RHDH rebuild required (Dynamic Plugins)
-  DCM entities appear in RHDH catalog
-
-Phase 4 โ€” Auto-generate Templates
-  DCM catalog items become Backstage Software Templates
-  Developers start using RHDH "Create" for DCM requests
-
-Phase 5 โ€” Decommission standalone SPA
-  Users have migrated to RHDH
-  Standalone SPA can be retired
-```
+**Standalone SPA** โ€” DCM deploys its own React-based consumer portal. No RHDH dependency. Suitable for environments where RHDH is not present.
 
-Total migration effort: primarily configuration. No data migration, no API changes, no DCM control plane changes.
+**RHDH Mode** โ€” DCM plugins are loaded into an existing RHDH instance. The RHDH Developer Hub becomes the consumer portal surface. Recommended for organizations already running RHDH.
 
----
+Both modes use the same DCM APIs and the same authentication model. The choice is a deployment configuration, not an architectural difference.
 
-*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+```yaml
+# dcm-config.yaml
+frontend:
+  mode: standalone_spa | rhdh
+  rhdh_base_url: https://rhdh.internal  # only required for rhdh mode
+```
+
+---
diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md
index d8f3934..b9f57e6 100644
--- a/content/docs/capabilities-matrix.md
+++ b/content/docs/capabilities-matrix.md
@@ -84,6 +84,13 @@
 | PRV-007 | Provider Health Reporting | โ€” | Expose health check endpoint; report availability | Monitor provider health; configure trust score updates | PRV-001 |
 | PRV-008 | Sovereignty Declaration Maintenance | โ€” | Notify DCM when sovereignty data changes within declared SLA | Monitor sovereignty changes; trigger re-evaluation | PRV-001 |
 | PRV-009 | Meta Provider Orchestration | โ€” | Compose sub-providers to deliver higher-order services; manage composition visibility | Configure composite provider federation eligibility | PRV-001, PRV-003 |
+| PRR-001 | OpenAPI Spec Declaration (GATE-SP-01) | โ€” | Declare OpenAPI spec URL at registration; spec must be machine-readable and reachable | Validate spec URL reachability during approval pipeline | PRV-001 |
+| PRR-002 | Healthy API at Activation (GATE-SP-02) | โ€” | Health endpoint returns `{"status": "healthy"}` at activation time | Enforce health check as activation precondition | PRV-001, HLT-001 |
+| PRR-003 | State Management Callback (GATE-SP-03) | โ€” | Implement realized_state_push callback at all conformance levels | Validate callback endpoint reachability during approval | PRV-001 |
+| PRR-004 | Tenant Metadata Endpoint (GATE-SP-04) | โ€” | Implement GET /api/v1/tenants/{uuid}/metadata returning usage data | Require for standard+ profile activation; enforce quota integration | PRV-001 |
+| PRR-005 | Prometheus Metrics (GATE-SP-05) | โ€” | Expose required metric families at declared metrics_endpoint | Validate metric presence during approval; gate standard+ activation | PRV-001, HLT-005 |
+| PRR-006 | AEP.DEV Linting (GATE-SP-06) | โ€” | Pass AEP linter against OpenAPI spec with no errors before registration; include linting report URL | Gate standard+ activation on linting pass; block activation on errors | PRV-001 |
+| PRR-007 | Multi-Tenant Dispatch (GATE-SP-07) | โ€” | Accept tenant_uuid in all dispatch payloads; return tenant-scoped resources | Gate standard+ activation on multi-tenant compatibility test | PRV-001 |
 
 ---
 
@@ -538,6 +545,33 @@
 ---
 
 
+## 36. Workload Analysis
+
+| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On |
+|----|-----------|---------|---------|---------------|-----------|| WLA-001 | Automated Workload Classification | View workload profile for owned resources; see archetype, resource type match, confidence | Report resource metadata via discovery for classification input | Configure classification ruleset version; review low-confidence classifications manually | DRC-001, INF-001 |
+| WLA-002 | Migration Readiness Scoring | View containerization score and migration blockers for owned resources | Report workload characteristics that inform migration scoring | Configure migration readiness thresholds; integrate MTA Information Provider | WLA-001 |
+| WLA-003 | MTA Information Provider Integration | โ€” | Implement workload_analysis information type OR delegate to MTA | Register MTA as Information Provider; configure analysis trigger policies | WLA-001, INF-001 |
+| WLA-004 | On-Demand Re-Analysis | Request re-analysis when resource role changes (`POST /resources/{uuid}/workload-profile:analyze`) | โ€” | Trigger re-analysis for any resource; override archetype manually with reason | WLA-001 |
+| WLA-005 | WorkloadProfile Audit Chain | View analysis history for owned resources | โ€” | Query full analysis history including superseded profiles | WLA-001, AUD-001 |
+| WLA-006 | Low-Confidence Manual Review Gate | Receive notification when owned resource requires manual classification | โ€” | Review and resolve low-confidence classifications; unblock ingestion | WLA-001, LCM-001 |
+
+---
+
+
+## 37. Accreditation Monitoring
+
+| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On |
+|----|-----------|---------|---------|---------------|-----------|| ACM-001 | Tier 1 External Registry Verification | โ€” | Declare `external_registry_id` at registration for FedRAMP, CMMC, ISO 27001 | Configure registry poll intervals; review status change alerts; manage `external_registry_id` accuracy | PRV-001, ACR-001 |
+| ACM-002 | Tier 2 Document Currency Verification | โ€” | Maintain current `certificate_ref` and `audit_report_ref` URLs pointing to valid, accessible documents | Configure `max_age` per framework; review document expiry alerts; upload new reports when notified | PRV-001, ACR-001 |
+| ACM-003 | Tier 3 Contract Webhook Integration | โ€” | Configure contract management webhook for BAA and DoD IL accreditations | Register contract system; configure inbound webhook credential; receive BAA/contract lifecycle events | PRV-001, ACR-001 |
+| ACM-004 | Verification Staleness Enforcement | โ€” | Ensure monitoring infrastructure can reach DCM to deliver verification events | Configure stale_after thresholds and stale_action per profile; enforce sovereign/fsi minimum tier requirements | ACM-001, ACR-001 |
+| ACM-005 | Immediate Revocation on External Revoke | Receive notification when provider accreditation is revoked; understand service impact | โ€” | Review `accreditation.status_changed` events; confirm immediate revocations; trigger recovery policy | ACM-001, ACR-001 |
+| ACM-006 | Verification Currency in Scoring | โ€” | Maintain verification currency to maximize accreditation richness score | Monitor verification multiplier impact on provider placement; prioritize externally verified providers | ACM-001, SMX-001 |
+| ACM-007 | Manual Override in Air-Gapped Mode | โ€” | โ€” | Manually update `last_verified_at` with justification in air-gapped deployments; maintain audit trail of manual verifications | ACR-001, AUD-001 |
+
+---
+
+
 ## Capability Count Summary
 
 | Domain | Capabilities |
@@ -604,11 +638,75 @@
 | Scheduled and Deferred Requests | 6 |
 | Request Dependency Graph | 6 |
 | DCM Self-Health | 6 |
+| Identity and Access Management | 21 |
+| Service Catalog | 7 |
+| Request Lifecycle Management | 10 |
+| Provider Contract and Realization | 16 |
+| Resource Lifecycle Management | 7 |
+| Drift Detection and Remediation | 5 |
+| Policy Management | 7 |
+| Data Layer Management | 5 |
+| Information and Data Integration | 6 |
+| Ingestion and Brownfield Management | 4 |
+| Audit and Compliance | 5 |
+| Observability and Operations | 5 |
+| Storage and State Management | 6 |
+| DCM Federation and Multi-Instance | 5 |
+| Platform Governance and Administration | 7 |
+| Accreditation Management | 6 |
+| Zero Trust and Security Posture | 6 |
+| Unified Governance Matrix | 7 |
+| Drift Reconciliation | 5 |
+| Federated Contribution Model | 7 |
+| Scoring Model | 10 |
+| Meta Provider Composability | 8 |
+| Credential Provider Model | 12 |
+| Authority Tier Model | 12 |
+| Event Catalog | 7 |
+| API Versioning | 8 |
+| Session Revocation | 11 |
+| Internal Component Authentication | 8 |
+| Scheduled and Deferred Requests | 6 |
+| Request Dependency Graph | 6 |
+| DCM Self-Health | 6 |
+| Identity and Access Management | 21 |
+| Service Catalog | 7 |
+| Request Lifecycle Management | 10 |
+| Provider Contract and Realization | 16 |
+| Resource Lifecycle Management | 7 |
+| Drift Detection and Remediation | 5 |
+| Policy Management | 7 |
+| Data Layer Management | 5 |
+| Information and Data Integration | 6 |
+| Ingestion and Brownfield Management | 4 |
+| Audit and Compliance | 5 |
+| Observability and Operations | 5 |
+| Storage and State Management | 6 |
+| DCM Federation and Multi-Instance | 5 |
+| Platform Governance and Administration | 7 |
+| Accreditation Management | 6 |
+| Zero Trust and Security Posture | 6 |
+| Unified Governance Matrix | 7 |
+| Drift Reconciliation | 5 |
+| Federated Contribution Model | 7 |
+| Scoring Model | 10 |
+| Meta Provider Composability | 8 |
+| Credential Provider Model | 12 |
+| Authority Tier Model | 12 |
+| Event Catalog | 7 |
+| API Versioning | 8 |
+| Session Revocation | 11 |
+| Internal Component Authentication | 8 |
+| Scheduled and Deferred Requests | 6 |
+| Request Dependency Graph | 6 |
+| DCM Self-Health | 6 |
 | Operational Reference | 4 |
 | Web Interfaces | 14 |
 | ITSM Integration | 7 |
 | Provider Callback Authentication | 10 |
-| **Total** | **269** |
+| Workload Analysis | 5 |
+| Accreditation Monitoring | 6 |
+| **Total** | **287** |
 
 ---
 
diff --git a/content/docs/data-model/00-context-and-purpose.md b/content/docs/data-model/00-context-and-purpose.md
index 8f0e641..4bac891 100644
--- a/content/docs/data-model/00-context-and-purpose.md
+++ b/content/docs/data-model/00-context-and-purpose.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Context and Purpose
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/data-model/00-design-priorities.md b/content/docs/data-model/00-design-priorities.md
index a5f5bed..43dfb27 100644
--- a/content/docs/data-model/00-design-priorities.md
+++ b/content/docs/data-model/00-design-priorities.md
@@ -223,7 +223,7 @@ External systems (ServiceNow, Jira, email workflows, Slack bots) connect to DCM
 The Admin API approval endpoint is designed to be called by external systems, not only by humans in a DCM UI:
 
 ```
-POST /api/v1/admin/approvals/{approval_uuid}/vote
+POST /api/v1/admin/approvals/{approval_uuid}:vote
 Authorization: Bearer      # any actor who is a member of the required DCMGroup
 
 {
diff --git a/content/docs/data-model/00-foundations.md b/content/docs/data-model/00-foundations.md
index 5f75acd..07eec14 100644
--- a/content/docs/data-model/00-foundations.md
+++ b/content/docs/data-model/00-foundations.md
@@ -1,10 +1,5 @@
 # DCM โ€” Foundational Abstractions
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Foundation โ€” Read This First
diff --git a/content/docs/data-model/01-entity-types.md b/content/docs/data-model/01-entity-types.md
index 7ade494..403700f 100644
--- a/content/docs/data-model/01-entity-types.md
+++ b/content/docs/data-model/01-entity-types.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Entity Types
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
diff --git a/content/docs/data-model/02-four-states.md b/content/docs/data-model/02-four-states.md
index 5cabbdc..1dfe700 100644
--- a/content/docs/data-model/02-four-states.md
+++ b/content/docs/data-model/02-four-states.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” The Four States
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/data-model/03-layering-and-versioning.md b/content/docs/data-model/03-layering-and-versioning.md
index 36ac3e3..97fe0e0 100644
--- a/content/docs/data-model/03-layering-and-versioning.md
+++ b/content/docs/data-model/03-layering-and-versioning.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Data Layers and the Assembly Process
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/data-model/04-examples.md b/content/docs/data-model/04-examples.md
index 1f63ab7..7b365de 100644
--- a/content/docs/data-model/04-examples.md
+++ b/content/docs/data-model/04-examples.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Worked Examples
 
-> **โš ๏ธ Active Development Notice**
->
-> Examples in this document are illustrative of the intended architecture. YAML structures, field names, and sequences represent the design intent and will be refined as implementation proceeds.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Reference Examples
diff --git a/content/docs/data-model/04b-ownership-sharing-allocation.md b/content/docs/data-model/04b-ownership-sharing-allocation.md
index 23b1f8f..1e89ac1 100644
--- a/content/docs/data-model/04b-ownership-sharing-allocation.md
+++ b/content/docs/data-model/04b-ownership-sharing-allocation.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Ownership, Sharing, and Allocation
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
diff --git a/content/docs/data-model/05-resource-type-hierarchy.md b/content/docs/data-model/05-resource-type-hierarchy.md
index 2591aaf..2ffc9cb 100644
--- a/content/docs/data-model/05-resource-type-hierarchy.md
+++ b/content/docs/data-model/05-resource-type-hierarchy.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Resource Type Hierarchy and Service Catalog
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -455,7 +450,7 @@ All definitions in the Resource Type Hierarchy follow the universal DCM versioni
 | Component | Trigger |
 |---|---|
 | **Major** | Breaking changes to the contract โ€” removing fields, changing field types, changing required/optional status of universal fields |
-| **Minor** | Additive changes, backward compatible โ€” adding new optional fields, adding new conditional fields, adding new extension points |
+| **Minor** | Additive changes โ€” adding new optional fields, adding new conditional fields, adding new extension points |
 | **Revision** | Data or configuration changes with no contract impact โ€” updating descriptions, updating constraints that don't break existing data, updating metadata |
 
 ### 9.2 Version Constraints in Requests
@@ -502,3 +497,25 @@ Once a version is published it is immutable. Any change โ€” even a documentation
 ---
 
 *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+---
+
+## Resource Type Reference โ€” Consumer vs Internal Format
+
+When consumers reference a resource type โ€” in API calls, policy conditions, or query filters โ€” DCM accepts two forms:
+
+| Form | Example | Notes |
+|------|---------|-------|
+| **FQN string** (recommended) | `Compute.VirtualMachine` | Stable across deployments; human-readable; returned by the service catalog |
+| **Registry UUID** | `a1b2c3d4-e5f6-...` | Deployment-specific; obtained from catalog API; suitable for programmatic use |
+
+DCM resolves either form to the canonical `(resource_type_uuid, resource_type_name)` pair during request assembly. The resolution happens in the **Request Payload Processor** before layer enrichment begins. Unresolvable references are rejected at validation time with a `422 Unprocessable Entity` response and code `RESOURCE_TYPE_NOT_FOUND`.
+
+**Internal representation:** All internal DCM data โ€” entity records, dispatch payloads, audit records โ€” always carry **both** `resource_type_uuid` and `resource_type_name` (FQN). The consumer-facing accept-both model is purely at the API boundary; internally DCM always uses the canonical pair.
+
+**Dispatch to operators:** The `CreateRequest` and `UpdateRequest` payloads sent to Service Providers always include both:
+- `resource_type_uuid` โ€” the Registry UUID
+- `resource_type_name` โ€” the FQN string
+
+Operators MUST NOT accept only one form; both will always be present.
+
diff --git a/content/docs/data-model/06-resource-service-entities.md b/content/docs/data-model/06-resource-service-entities.md
index 1a19392..8814c7a 100644
--- a/content/docs/data-model/06-resource-service-entities.md
+++ b/content/docs/data-model/06-resource-service-entities.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Resource/Service Entities
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -541,9 +536,8 @@ policy:
 
 This pre-authorization pattern allows providers to implement auto-scaling, auto-healing, and maintenance operations without requiring per-change manual approval, while keeping DCM's Realized Store accurate and traceable.
 
-### 7a.6 Updated Provider Lifecycle Events Table
+### 7a.6 Provider Lifecycle Events
 
-The following table supersedes the table in Section 7.2 with clearer DCM response categorization:
 
 | Event Type | Mechanism | DCM Response | Realized Store Updated? |
 |------------|-----------|-------------|------------------------|
diff --git a/content/docs/data-model/07-service-dependencies.md b/content/docs/data-model/07-service-dependencies.md
index 57a1a87..fc0d904 100644
--- a/content/docs/data-model/07-service-dependencies.md
+++ b/content/docs/data-model/07-service-dependencies.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Service Dependencies
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -24,7 +19,7 @@
 
 
 
-> **Note:** The dependency graph data structure has been superseded by the universal Entity Relationship model defined in [Entity Relationships](09-entity-relationships.md). This document retains content specific to dependency declaration, rehydration ordering, and failure handling โ€” all of which operate on the Entity Relationship Graph defined in that document.
+> **Scope:** This document covers dependency declaration, rehydration ordering, and failure handling. The underlying data structure is the Entity Relationship Graph defined in [Entity Relationships](09-entity-relationships.md).
 
 ---
 
diff --git a/content/docs/data-model/08-resource-grouping.md b/content/docs/data-model/08-resource-grouping.md
index a35c8b5..8d16b4a 100644
--- a/content/docs/data-model/08-resource-grouping.md
+++ b/content/docs/data-model/08-resource-grouping.md
@@ -1,13 +1,8 @@
 # DCM Data Model โ€” Resource Grouping
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
-> **Universal Group Model:** The constructs defined in this document (DCM Tenant, Resource Groups) are superseded by the [Universal Group Model](15-universal-groups.md) for new implementations. Existing constructs map 1:1 to `group_class` values in the universal model โ€” UUIDs, handles, and API references are preserved. This document remains valid as a reference for pre-universal implementations and for understanding the migration path.
+> **Related:** See [Universal Group Model](15-universal-groups.md) for the canonical group model. The constructs here (Tenants, Resource Groups) map 1:1 to `group_class` values in that model.
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Service Dependencies](07-service-dependencies.md)
diff --git a/content/docs/data-model/09-entity-relationships.md b/content/docs/data-model/09-entity-relationships.md
index 5fdf8cf..abe418a 100644
--- a/content/docs/data-model/09-entity-relationships.md
+++ b/content/docs/data-model/09-entity-relationships.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Entity Relationships
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
@@ -32,7 +27,7 @@ The DCM Entity Relationship model is the **universal mechanism for expressing re
 
 A single relationship model is used everywhere. There is no separate binding mechanism for storage, no separate dependency graph structure, no separate business data association mechanism. One model serves all relationship types across the full lifecycle โ€” from pre-realization planning through to post-realization management, drift detection, cost rollup, and rehydration.
 
-This document supersedes the dependency graph concept from the Service Dependencies document for data structure purposes. The Service Dependencies document retains content on rehydration ordering and failure handling, which operate on the relationship graph defined here.
+This document defines the Entity Relationship Graph, which is the data structure underlying service dependency declaration (doc 07) and rehydration ordering. The Service Dependencies document retains content on rehydration ordering and failure handling, which operate on the relationship graph defined here.
 
 ---
 
@@ -311,7 +306,7 @@ When a consuming Tenant claims an available allocation, DCM creates a first-clas
 ```yaml
 allocated_entity:
   uuid: 
-  entity_type: allocated_resource
+  entity_type: infrastructure_resource  # ownership_model: allocation
   resource_type_uuid: 
   tenant_uuid:   # Belongs to the consuming Tenant
 
diff --git a/content/docs/data-model/10-information-providers.md b/content/docs/data-model/10-information-providers.md
index aabd078..f5d052d 100644
--- a/content/docs/data-model/10-information-providers.md
+++ b/content/docs/data-model/10-information-providers.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Information Providers
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/data-model/11-storage-providers.md b/content/docs/data-model/11-storage-providers.md
index 756e511..78465e9 100644
--- a/content/docs/data-model/11-storage-providers.md
+++ b/content/docs/data-model/11-storage-providers.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Storage Providers
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/data-model/12-audit-provenance-observability.md b/content/docs/data-model/12-audit-provenance-observability.md
index d1f3409..b981323 100644
--- a/content/docs/data-model/12-audit-provenance-observability.md
+++ b/content/docs/data-model/12-audit-provenance-observability.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Audit, Provenance, and Observability
 
-> **โš ๏ธ Active Development Notice**
-> 
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
-> 
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 
 **Document Status:** โœ… Complete  
diff --git a/content/docs/data-model/13-ingestion-model.md b/content/docs/data-model/13-ingestion-model.md
index 4620593..7ca22af 100644
--- a/content/docs/data-model/13-ingestion-model.md
+++ b/content/docs/data-model/13-ingestion-model.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Ingestion Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Four States](02-four-states.md) | [Resource/Service Entities](06-resource-service-entities.md) | [Entity Relationships](09-entity-relationships.md) | [Resource Grouping](08-resource-grouping.md)
@@ -29,7 +24,6 @@
 
 The DCM Ingestion Model is the **unified mechanism for bringing entities that exist outside DCM's lifecycle control into DCM's governance model**. It applies to three distinct sources:
 
-- **V1 Migration** โ€” entities from a DCM V1 deployment that predate the mandatory Tenant model
 - **Brownfield Discovery** โ€” entities discovered by a Service Provider that already exist in the infrastructure but are unknown to DCM
 - **Manual Import** โ€” entities imported from external systems (CMDBs, spreadsheets, legacy records) during onboarding
 
@@ -47,7 +41,7 @@ All three sources follow the same pattern: ingest, enrich, and promote. The same
 
 ## 2. Design Principles
 
-**Unified model โ€” minimum variance.** V1 migration and brownfield ingestion are the same fundamental operation. One model, one audit record structure, one set of governance policies.
+**Unified model โ€” minimum variance.** brownfield ingestion and brownfield ingestion are the same fundamental operation. One model, one audit record structure, one set of governance policies.
 
 **Non-blocking.** Entities that cannot be immediately assigned a Tenant do not block migration or discovery. They land in the `__transitional__` Tenant and are resolved progressively. Migration does not require every entity to be assigned before any entity can proceed.
 
@@ -141,11 +135,11 @@ ingestion_record:
   resource_entity_uuid: 
   ingestion_timestamp: 
 
-  ingestion_source: 
+  ingestion_source: 
 
-  # V1 migration fields (when ingestion_source: v1_migration)
-  v1_identifier: 
-  v1_metadata_snapshot: 
+  # brownfield ingestion fields (when ingestion_source: legacy_import)
+  legacy_identifier: 
+  legacy_metadata_snapshot: 
 
   # Brownfield discovery fields (when ingestion_source: brownfield_discovery)
   discovered_state_uuid: 
@@ -198,7 +192,7 @@ When DCM ingests an entity, it attempts auto-assignment to a real Tenant using t
 |--------|-----------|-------------|
 | Explicit ownership metadata | High | Business unit, cost center, or team tag on the resource maps unambiguously to a Tenant |
 | Resource group membership | High | Resource belongs to a group that maps to a known Tenant |
-| Request history | High | V1 request record identifies the requesting team, which maps to a Tenant |
+| Request history | High | Legacy record identifies the requesting team, which maps to a Tenant |
 | Network / location context | Medium | Resource's location, VLAN, or network segment maps to a Tenant by convention |
 | Naming convention | Medium | Resource name matches a known Tenant naming pattern |
 | Provider context | Medium | Resource was provisioned by a known provider associated with a Tenant |
@@ -208,68 +202,6 @@ Multiple signals can be combined. If signals conflict, the higher-confidence sig
 
 ---
 
-## 7. V1 Migration
-
-### 7.1 Overview
-
-V1 resources have no `tenant_uuid`. V2 requires one (`TEN-001`). The V1 migration process uses the ingestion model to assign every V1 resource a Tenant before it can participate in V2 operations.
-
-### 7.2 Resource Categories
-
-| Category | Description | Assignment Path |
-|----------|-------------|----------------|
-| **Auto-assignable** | Clear ownership signals โ€” resource group, business unit, request history | Auto-assigned during migration analysis pass |
-| **Manually assignable** | Ambiguous signals โ€” multiple possible owners, or medium-confidence signals only | Surfaced in admin assignment queue |
-| **Orphaned** | No signals โ€” no ownership data available | Assigned to `__transitional__` |
-
-### 7.3 Migration Flow
-
-```
-V1 estate
-  โ”‚
-  โ–ผ  Step 1 โ€” Pre-migration analysis pass
-  โ”‚  Inventory all V1 resources
-  โ”‚  Attempt auto-assignment via signals (Section 6)
-  โ”‚  Classify each resource: auto_assignable | manually_assignable | orphaned
-  โ”‚  Produce migration readiness report
-  โ”‚
-  โ–ผ  Step 2 โ€” Auto-assignment
-  โ”‚  Create or map to existing V2 Tenants
-  โ”‚  Assign auto_assignable resources in bulk
-  โ”‚  Create ingestion_record per resource (ingestion_source: v1_migration)
-  โ”‚  State: INGESTED โ†’ ENRICHING (for auto-assigned)
-  โ”‚
-  โ–ผ  Step 3 โ€” Manual assignment queue
-  โ”‚  manually_assignable resources surfaced in admin UI
-  โ”‚  Administrators review and assign Tenants
-  โ”‚  Each assignment recorded in enrichment_history
-  โ”‚
-  โ–ผ  Step 4 โ€” Transitional fallback
-  โ”‚  orphaned resources โ†’ __transitional__ Tenant
-  โ”‚  ingestion_record.assignment_method: transitional
-  โ”‚  ingestion_record.ingestion_confidence: low
-  โ”‚  Governance timer starts
-  โ”‚
-  โ–ผ  Step 5 โ€” Enrichment and promotion
-  โ”‚  Relationships established, missing fields populated
-  โ”‚  Each entity reviewed and promoted when complete
-  โ”‚  State: ENRICHING โ†’ PROMOTED โ†’ OPERATIONAL
-  โ”‚
-  โ–ผ  Migration complete when __transitional__ Tenant is empty
-```
-
-### 7.4 Migration System Policies
-
-| Policy | Rule |
-|--------|------|
-| `ING-001` | Every entity ingested into V2 from V1 must be assigned to exactly one Tenant โ€” either a real Tenant or `__transitional__` โ€” before it is eligible for new V2 requests |
-| `ING-002` | Entities in `INGESTED` or `ENRICHING` state may not be the parent resource for a new allocated resource claim |
-| `ING-003` | The `__transitional__` Tenant is system-managed and cannot be deleted, renamed, or used for new resource provisioning |
-| `ING-004` | Every ingested entity must carry an `ingestion_record` in its provenance chain |
-| `ING-005` | Entities in `__transitional__` for longer than `max_residency_days` must trigger the configured escalation action |
-
----
-
 ## 8. Brownfield Ingestion
 
 ### 8.1 Overview
@@ -347,7 +279,7 @@ Ingestion interacts with the Four States model as follows:
 
 | Ingestion Source | States Involved | Flow |
 |-----------------|----------------|------|
-| V1 Migration | Intent โ†’ Requested โ†’ (no Realized yet) | V1 records treated as incomplete Requested State; migration creates minimal Realized State |
+| brownfield ingestion | Intent โ†’ Requested โ†’ (no Realized yet) | Legacy records treated as incomplete Requested State; migration creates minimal Realized State |
 | Brownfield Discovery | Discovered โ†’ Realized | Discovered State is promoted to Realized State at promotion |
 | Manual Import | None initially | Entity stub created; no prior state records; Realized State created at promotion from import data |
 
@@ -387,7 +319,6 @@ In all cases: once an entity reaches `PROMOTED`, it has a Realized State record
 - **Four States** โ€” Discovered State is the entry point for brownfield ingestion; Realized State is the output of promotion
 - **Brownfield** โ€” existing infrastructure not yet under DCM lifecycle management
 - **Drift Detection** โ€” begins for brownfield entities at the moment of promotion
-- **V1 Migration** โ€” migration of pre-Tenant DCM V1 entities to V2 using the ingestion model
 - **Greening the Brownfield** โ€” the progressive process of bringing unmanaged infrastructure under DCM lifecycle control
 
 
diff --git a/content/docs/data-model/14-policy-profiles.md b/content/docs/data-model/14-policy-profiles.md
index d6af6b9..f0772d3 100644
--- a/content/docs/data-model/14-policy-profiles.md
+++ b/content/docs/data-model/14-policy-profiles.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Policy Organization: Groups, Profiles, and Policy Providers
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 > **Universal Group Model:** Policy Groups (`group_class: policy_collection`) and Policy Profiles (`group_class: policy_profile`) are expressions of the [Universal Group Model](15-universal-groups.md). The structures defined in this document remain authoritative for policy-specific behavior; the universal model adds composability, cross-type membership, and the ability to include policy groups within composite groups.
 
diff --git a/content/docs/data-model/15-universal-groups.md b/content/docs/data-model/15-universal-groups.md
index c3222e7..6103298 100644
--- a/content/docs/data-model/15-universal-groups.md
+++ b/content/docs/data-model/15-universal-groups.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Universal Group Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Resource Grouping](08-resource-grouping.md) | [Entity Relationships](09-entity-relationships.md) | [Policy Organization](14-policy-profiles.md)
@@ -391,7 +386,7 @@ All groups are stored in a single **Group Registry** โ€” a GitOps store followin
 
 ### 6.2 Class-Filtered API Views
 
-The universal registry exposes class-filtered views that preserve backward compatibility with existing API consumers:
+The universal registry exposes class-filtered views for convenience:
 
 | Endpoint | Equivalent Query |
 |----------|----------------|
@@ -405,22 +400,6 @@ Existing API references continue to work unchanged. New API consumers can use th
 
 ---
 
-## 7. Migration from Current Constructs
-
-Existing constructs migrate to the universal model with UUID preservation:
-
-| Current Construct | Migration | UUID Preserved? |
-|------------------|-----------|----------------|
-| Tenant entity | `group_class: tenant_boundary` | Yes |
-| DCM Default Resource Group | `group_class: resource_grouping` | Yes |
-| Custom Resource Group | `group_class: resource_grouping, group_subclass: ` | Yes |
-| Policy Group | `group_class: policy_collection` | Yes |
-| Policy Profile | `group_class: policy_profile` | Yes |
-
-Migration uses the standard ingestion model โ€” existing constructs are ingested as `ingestion_source: migration` with `ingestion_confidence: high` (UUID preservation, no ambiguity).
-
----
-
 ## 8. DCM System Policies
 
 | Policy | Rule |
diff --git a/content/docs/data-model/16-universal-audit.md b/content/docs/data-model/16-universal-audit.md
index 0449ec7..30ad172 100644
--- a/content/docs/data-model/16-universal-audit.md
+++ b/content/docs/data-model/16-universal-audit.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Universal Audit Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Audit, Provenance, and Observability](12-audit-provenance-observability.md) | [Storage Providers](11-storage-providers.md) | [Universal Groups](15-universal-groups.md)
@@ -221,7 +216,7 @@ The `action` field uses a closed vocabulary. Free-text actions are invalid and r
 | `HOLD_RELEASE` | Resources | Resource hold released |
 | `DRIFT_DETECT` | Entities | Drift detected between Realized and Discovered |
 | `DRIFT_RESOLVE` | Entities | Drift resolved |
-| `INGEST` | Entities | Entity ingested (brownfield or V1 migration) |
+| `INGEST` | Entities | Entity ingested (brownfield or import) |
 | `PROMOTE` | Entities | Ingested entity promoted to full lifecycle |
 | `EXPIRE` | Entities | Lifecycle time constraint expiry action fired |
 | `REHYDRATE` | Entities | Rehydration requested |
diff --git a/content/docs/data-model/17-deployment-redundancy.md b/content/docs/data-model/17-deployment-redundancy.md
index fb21c54..f2224c1 100644
--- a/content/docs/data-model/17-deployment-redundancy.md
+++ b/content/docs/data-model/17-deployment-redundancy.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Deployment and Redundancy Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Context and Purpose](00-context-and-purpose.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit Model](16-universal-audit.md) | [Policy Organization](14-policy-profiles.md)
diff --git a/content/docs/data-model/18-webhooks-messaging.md b/content/docs/data-model/18-webhooks-messaging.md
index 0b3d2f1..840637f 100644
--- a/content/docs/data-model/18-webhooks-messaging.md
+++ b/content/docs/data-model/18-webhooks-messaging.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Webhooks, Messaging, and External Integration
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Related Documents (updated):** [Notification Model](23-notification-model.md) | [Entity Relationships](09-entity-relationships.md)  
@@ -257,7 +252,7 @@ quota_policy:
 
 > **โš ๏ธ Architecture Update โ€” Notification Model Supersedes Outbound Webhooks**
 >
-> The outbound webhook model described in Section 3 has been superseded by the **Unified Notification Model** (see [doc 23: Notification Model](23-notification-model.md)). Outbound webhooks are now one delivery channel type within the Notification Provider model rather than a parallel mechanism.
+> The outbound webhook model described in Section 3 has been one delivery channel within the Notification Model. Outbound webhooks are now one delivery channel type within the Notification Provider model rather than a parallel mechanism.
 >
 > **For new implementations:** Use the Notification Provider subscription model (doc 23, Section 6) with a webhook-type Notification Provider.
 >
@@ -349,7 +344,7 @@ webhook_registration:
 
 ### 3.3 Event Taxonomy
 
-> **See [doc 23: Notification Model](23-notification-model.md) Section 4 for the authoritative event taxonomy.** The table below is the legacy taxonomy for existing webhook registrations.
+> The table below is the event taxonomy for webhook subscriptions.
 
 The event taxonomy maps onto the Universal Audit action vocabulary. All are versioned registry entries:
 
@@ -381,7 +376,7 @@ webhook_payload:
   # Subject
   subject:
     entity_uuid: 
-    entity_type: resource_entity
+    entity_type: infrastructure_resource
     entity_handle: 
     tenant_uuid: 
 
diff --git a/content/docs/data-model/19-auth-providers.md b/content/docs/data-model/19-auth-providers.md
index d6a23e7..db57f10 100644
--- a/content/docs/data-model/19-auth-providers.md
+++ b/content/docs/data-model/19-auth-providers.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Authentication, Authorization, and Auth Providers
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).*
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Webhooks and Messaging](18-webhooks-messaging.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md)
diff --git a/content/docs/data-model/20-registry-governance.md b/content/docs/data-model/20-registry-governance.md
index b6ac267..98567b6 100644
--- a/content/docs/data-model/20-registry-governance.md
+++ b/content/docs/data-model/20-registry-governance.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Registry Governance
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Policy Organization](14-policy-profiles.md) | [Deployment and Redundancy](17-deployment-redundancy.md) | [Auth Providers](19-auth-providers.md)
diff --git a/content/docs/data-model/21-information-providers-advanced.md b/content/docs/data-model/21-information-providers-advanced.md
index e3aa053..52c359f 100644
--- a/content/docs/data-model/21-information-providers-advanced.md
+++ b/content/docs/data-model/21-information-providers-advanced.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Information Providers: Confidence Scoring, Authority, and Conflict Resolution
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Information Providers](10-information-providers.md) | [Policy Organization](14-policy-profiles.md) | [Universal Audit Model](16-universal-audit.md) | [DCM Federation](22-dcm-federation.md)
diff --git a/content/docs/data-model/22-dcm-federation.md b/content/docs/data-model/22-dcm-federation.md
index 89c0a10..e7a07d2 100644
--- a/content/docs/data-model/22-dcm-federation.md
+++ b/content/docs/data-model/22-dcm-federation.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” DCM Federation, Peering, and Cross-Instance Coordination
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications are work in progress and subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete  
 **Related Documents:** [Federated Contribution Model](28-federated-contribution-model.md) | [Universal Group Model](15-universal-groups.md) | [Storage Providers](11-storage-providers.md) | [Auth Providers](19-auth-providers.md) | [Information Providers Advanced](21-information-providers-advanced.md)
diff --git a/content/docs/data-model/23-notification-model.md b/content/docs/data-model/23-notification-model.md
index c15a907..2fa20a4 100644
--- a/content/docs/data-model/23-notification-model.md
+++ b/content/docs/data-model/23-notification-model.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Notification Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized. Open questions are explicitly tracked and decisions are recorded as they are made.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -41,7 +36,7 @@ This document defines:
 - The notification payload structure โ€” the unified envelope all Notification Providers receive
 - The delivery pipeline โ€” from event trigger through audience resolution through provider delivery
 
-This model supersedes the standalone outbound webhook model in doc 18. Outbound webhooks are now one delivery channel of the Notification Provider, not a parallel mechanism.
+Outbound webhooks are one delivery channel within this model, implemented via the Notification Provider.
 
 ---
 
@@ -547,7 +542,7 @@ Provider Update Notifications (doc 06, Section 7a) integrate with the notificati
 **When provider update requires consumer approval:**
 - `provider_update.requires_approval` fires โ†’ Owner notified (action required)
 - `action.type: approve`
-- `action.action_url` points to `/api/v1/resources/{uuid}/provider-notifications/{uuid}/approve`
+- `action.action_url` points to `/api/v1/resources/{uuid}/provider-notifications/{uuid}:approve`
 - `action.deadline` set per policy (default PT24H โ€” if no response, escalate)
 
 **On resolution:**
diff --git a/content/docs/data-model/24-operational-models.md b/content/docs/data-model/24-operational-models.md
index 153f21c..cb63efd 100644
--- a/content/docs/data-model/24-operational-models.md
+++ b/content/docs/data-model/24-operational-models.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Operational Models
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -307,7 +302,7 @@ event_triggered_discovery:
 Platform admins and SREs can trigger discovery manually:
 
 ```
-POST /api/v1/admin/discovery/trigger
+POST /api/v1/admin/discovery:trigger
 
 {
   "scope": "entity | resource_type | provider | tenant",
diff --git a/content/docs/data-model/25-control-plane-components.md b/content/docs/data-model/25-control-plane-components.md
index 1ca67e8..dbe5448 100644
--- a/content/docs/data-model/25-control-plane-components.md
+++ b/content/docs/data-model/25-control-plane-components.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Control Plane Components
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
diff --git a/content/docs/data-model/26-accreditation-and-authorization-matrix.md b/content/docs/data-model/26-accreditation-and-authorization-matrix.md
index eb0fb8d..7184cb5 100644
--- a/content/docs/data-model/26-accreditation-and-authorization-matrix.md
+++ b/content/docs/data-model/26-accreditation-and-authorization-matrix.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Accreditation, Data Authorization Matrix, and Zero Trust
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -165,11 +160,25 @@ accreditation:
   external_registry_id: "FR2024-0042"    # e.g., FedRAMP Marketplace ID
 
   # Status
-  status: active | suspended | revoked | expired | pending_renewal
+  status: active | suspended | revoked | expired | pending_renewal | pending_review
   revocation_reason: 
   revoked_at: 
+
+  # Automated verification (see doc 47 โ€” Accreditation Monitor)
+  verification:
+    tier: external_registry | document_currency | contract_webhook | expiry_only
+    stale_after: P7D                # max gap between verifications before stale_action fires
+    stale_action: warn | suspend | escalate   # profile-governed default: warn/suspend/escalate
+    verification_failure_count: 0
+    # tier-specific fields: see doc 47 Section 3 for full schema
 ```
 
+> **Accreditation Monitor:** The `last_verified_at` field is maintained by the
+> Accreditation Monitor (doc 47), which continuously verifies accreditation status
+> against external registries, document currency checks, or contract system webhooks
+> depending on the `verification.tier`. See doc 47 for the complete monitoring
+> specification and framework-by-framework automation coverage.
+
 ### 3.4 Accreditation Lifecycle
 
 ```
@@ -192,6 +201,11 @@ Accreditation submitted (via API or GitOps PR)
   โ”‚     status โ†’ expired
   โ”‚     Providers relying on this accreditation flagged: ACCREDITATION_GAP
   โ”‚
+  โ–ผ External status change detected by Accreditation Monitor:
+  โ”‚   status โ†’ pending_review
+  โ”‚   Platform Admin notified (urgency: high)
+  โ”‚   Exception: external status = Revoked โ†’ immediate revocation (no review)
+  โ”‚
   โ–ผ Revocation:
       Accreditor or Platform Admin revokes
       status โ†’ revoked
@@ -209,7 +223,7 @@ accreditation_gap_record:
   provider_uuid: 
   required_framework: hipaa
   required_for: [phi data fields in active requests]
-  gap_type: missing | expired | revoked | suspended
+  gap_type: missing | expired | revoked | suspended | verification_stale
   detected_at: 
   severity: critical                    # accreditation gaps are always high or critical
   affected_entity_uuids: [, ...]  # entities currently hosted at this provider
@@ -232,9 +246,7 @@ deployment_accreditation:
 ---
 
 
-> **Architecture Update:** Section 4 of this document (Data/Capability Authorization Matrix) has been superseded by the **Unified Governance Matrix** ([doc 27](27-governance-matrix.md)). The governance matrix provides a more powerful, unified model that replaces the standalone matrix described here. The accreditation model (Sections 2-3) and zero trust interaction model (Section 5) remain current and are consumed by the governance matrix as inputs.
->
-> New implementations should reference doc 27 for data and capability boundary enforcement.
+> **Scope:** This document covers the accreditation model (Sections 2-3) and zero trust interaction model (Section 5). Data and capability boundary enforcement is specified in the [Unified Governance Matrix](27-governance-matrix.md) (doc 27), which consumes the accreditation and classification models defined here as inputs.
 
 ## 4. Data/Capability Authorization Matrix
 
diff --git a/content/docs/data-model/27-governance-matrix.md b/content/docs/data-model/27-governance-matrix.md
index 0ae06c7..43f300c 100644
--- a/content/docs/data-model/27-governance-matrix.md
+++ b/content/docs/data-model/27-governance-matrix.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Unified Governance Matrix
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference
@@ -36,7 +31,7 @@ The Unified Governance Matrix is the **single, declarative, multi-dimensional co
 
 Previous DCM documents established several overlapping control mechanisms: the Data/Capability Authorization Matrix (doc 26), sovereignty constraints in federation tunnels (doc 22), BBQ-001 sovereignty checks (doc 14), and profile-governed data boundaries. The Governance Matrix unifies all of these into a single model with a single evaluation algorithm and a single enforcement point.
 
-**This document supersedes** Section 4 of doc 26 (Data/Capability Authorization Matrix) for structural purposes. The accreditation model (Sections 2-3 of doc 26) and the zero trust interaction model (Section 5 of doc 26) remain current โ€” the Governance Matrix consumes them as inputs.
+The accreditation model and data classification model from doc 26 are inputs to the evaluation algorithm defined here. The accreditation model (Sections 2-3 of doc 26) and the zero trust interaction model (Section 5 of doc 26) remain current โ€” the Governance Matrix consumes them as inputs.
 
 **Key properties of the Governance Matrix:**
 
diff --git a/content/docs/data-model/28-federated-contribution-model.md b/content/docs/data-model/28-federated-contribution-model.md
index aadabc2..1ae4c2b 100644
--- a/content/docs/data-model/28-federated-contribution-model.md
+++ b/content/docs/data-model/28-federated-contribution-model.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Federated Contribution Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Read This First for Multi-User Data Governance
diff --git a/content/docs/data-model/29-scoring-model.md b/content/docs/data-model/29-scoring-model.md
index ee2e93b..c8c00ab 100644
--- a/content/docs/data-model/29-scoring-model.md
+++ b/content/docs/data-model/29-scoring-model.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Hybrid Scoring Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. This document specifies the hybrid scoring model โ€” a first-class architectural addition approved in the DCM architecture review. It extends the Policy Engine and Profile system without replacing any existing components.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Scoring Model Specification
@@ -275,7 +270,19 @@ accreditation_weights:
   pci_dss: 25
   sovereign_authorization: 50
 
-# richness_score = sum(weights for held accreditations) / max_possible ร— 100
+# richness_score = sum(weights for held accreditations) / max_possible ร— 1
+
+# Verification currency multipliers (applied per accreditation, see doc 47)
+# Multiplier reduces an accreditation's weight contribution based on how recently
+# it was externally verified by the Accreditation Monitor
+verification_multipliers:
+  external_registry_verified_within_P1D:  1.0   # full weight โ€” verified today
+  external_registry_verified_within_P7D:  0.9
+  document_verified_within_P30D:          0.85
+  contract_webhook_active:                0.9
+  expiry_only_no_external_check:          0.7   # never been externally verified
+  verification_stale:                     0.4   # check overdue
+  verification_failed_threshold_reached:  0.1   # Monitor cannot reach registry00
 # risk_contribution = (1 - richness_score/100) ร— 10   [lower richness = higher risk]
 ```
 
@@ -341,11 +348,11 @@ Profiles can declare tighter thresholds for specific resource types:
 ```yaml
 resource_type_threshold_overrides:
   - resource_type: "Compute.VirtualMachine"
-    auto_approve_below: 20    # tighter than profile default
+    # tier: auto, max_score: 20  # use named-tier threshold format    # tighter than profile default
   - resource_type: "Network.VLAN"
-    auto_approve_below: 10    # VLANs require more scrutiny
+    # tier: auto, max_score: 10  # use named-tier threshold format    # VLANs require more scrutiny
   - resource_type: "Storage.Volume"
-    verified_above: 40   # storage changes escalate earlier
+    # tier: verified, max_score: 40  # use named-tier threshold format   # storage changes escalate earlier
 ```
 
 ### 5.3 Tenant Threshold Overrides
@@ -356,7 +363,7 @@ Platform admins can declare Tenant-level scoring threshold adjustments:
 tenant_scoring_config:
   tenant_uuid: 
   threshold_overrides:
-    auto_approve_below: 15    # more conservative for this Tenant
+    # tier: auto, max_score: 15  # use named-tier threshold format    # more conservative for this Tenant
   signal_weight_overrides:
     actor_risk_history_weight: 0.30   # higher actor scrutiny for this Tenant
   trusted_actors:
diff --git a/content/docs/data-model/30-meta-provider-model.md b/content/docs/data-model/30-meta-provider-model.md
index 039a4ed..0d69b51 100644
--- a/content/docs/data-model/30-meta-provider-model.md
+++ b/content/docs/data-model/30-meta-provider-model.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Meta Provider Composability Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. This document specifies the Meta Provider composability model โ€” what a Meta Provider is, what it declares to DCM, and how standard DCM machinery handles the rest.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Meta Provider Specification
diff --git a/content/docs/data-model/31-credential-provider-model.md b/content/docs/data-model/31-credential-provider-model.md
index 199f9a1..67288ab 100644
--- a/content/docs/data-model/31-credential-provider-model.md
+++ b/content/docs/data-model/31-credential-provider-model.md
@@ -1,10 +1,5 @@
 # DCM Data Model โ€” Credential Provider Model
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. This document specifies the Credential Provider model โ€” the issuance contract, credential lifecycle, rotation model, revocation propagation, and how credentials flow through the DCM pipeline.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Reference โ€” Credential Provider Specification
diff --git a/content/docs/data-model/32-authority-tier-model.md b/content/docs/data-model/32-authority-tier-model.md
index e195758..9b01821 100644
--- a/content/docs/data-model/32-authority-tier-model.md
+++ b/content/docs/data-model/32-authority-tier-model.md
@@ -426,7 +426,7 @@ Security degradations block tier registry activation. The blocking gate requires
 This is the same pattern as the standard approval pipeline โ€” DCM provides the gate; the organization provides the review process. The difference is that the required tier for the degradation review is always at least `verified`, regardless of the profile in use.
 
 ```
-POST /api/v1/admin/tier-registry/{change_uuid}/accept-degradation
+POST /api/v1/admin/tier-registry/{change_uuid}:accept-degradation
 
 {
   "affected_item_uuid": "",
diff --git a/content/docs/data-model/33-event-catalog.md b/content/docs/data-model/33-event-catalog.md
index 1c35033..34ebcd2 100644
--- a/content/docs/data-model/33-event-catalog.md
+++ b/content/docs/data-model/33-event-catalog.md
@@ -720,6 +720,67 @@ payload:
 
 ---
 
+## 20. Accreditation Events (`accreditation.*`)
+
+Fired by the Accreditation Monitor (doc 47) when external verification
+of a registered accreditation produces a result or requires attention.
+
+| Event Type | Urgency | Description |
+|-----------|---------|-------------|
+| `accreditation.verified` | low | Periodic external confirmation โ€” accreditation still active in external registry |
+| `accreditation.status_changed` | high or critical | External registry reports a different status than DCM records โ€” requires platform admin review |
+| `accreditation.registry_mismatch` | high | External registry cannot find the accreditation by its `external_registry_id` โ€” ID may need correction |
+| `accreditation.verification_stale` | varies | `last_checked_at` exceeds `stale_after` threshold โ€” stale_action applied per configuration |
+| `accreditation.document_expired` | high | Evidence document (SOC 2 report, AoC) is older than `max_age` threshold โ€” new document required |
+| `accreditation.contract_event` | varies | Contract management webhook received (BAA signed, amended, or terminated) |
+| `accreditation.expiry_approaching` | medium | Approaching `valid_until` within `renewal_warning_before` window (automated complement to TTL-based check) |
+
+### 20.1 Payload Schemas
+
+```yaml
+# accreditation.status_changed โ€” the most critical event
+accreditation.status_changed:
+  accreditation_uuid: 
+  subject_uuid: 
+  framework: fedramp_high | iso_27001 | cmmc_2 | ...
+  from_status: authorized | active | certified
+  to_status: in_process | revoked | suspended | withdrawn
+  external_source: fedramp_marketplace | cmmc_ab | iaf_certsearch | contract_webhook
+  detected_at: 
+  action_taken: pending_review | immediate_revocation
+  # immediate_revocation when to_status is 'revoked' or 'terminated'
+
+# accreditation.verification_stale
+accreditation.verification_stale:
+  accreditation_uuid: 
+  subject_uuid: 
+  framework: 
+  last_checked_at: 
+  stale_after: P7D
+  stale_action_taken: warn | suspended | escalated
+  consecutive_failures: 
+
+# accreditation.contract_event
+accreditation.contract_event:
+  accreditation_uuid: 
+  subject_uuid: 
+  framework: hipaa | dod_il4 | 
+  contract_event_type: signed | amended | terminated | renewal_due | renewed
+  contract_id: 
+  effective_date: 
+  dcm_action_taken: activated | pending_review | revoked | none
+```
+
+### 20.2 System Policy
+
+| Policy | Rule |
+|--------|------|
+| `EVT-ACM-001` | `accreditation.status_changed` events with `action_taken: immediate_revocation` are non-suppressable โ€” they are delivered to Compliance Team and Platform Admin regardless of notification preferences. |
+| `EVT-ACM-002` | `accreditation.verification_stale` urgency is profile-governed: `low` for dev/standard; `medium` for prod; `high` for fsi/sovereign. |
+
+---
+
+
 ## 19. Event Type Quick Reference
 
 ```
@@ -780,3 +841,14 @@ auth.provider_failover
 ---
 
 *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+### Additional Event Types
+
+| Event Type | Description | Key Fields | Consumers |
+|------------|-------------|-----------|----------|
+| `entity.deleted` | An entity has been fully decommissioned and removed from inventory | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `entity.state_transition` | An entity lifecycle state has changed (e.g., OPERATIONAL โ†’ SUSPENDED) | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `group.deleted` | A DCMGroup has been deleted | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `group.member_added` | A member (actor or entity) has been added to a DCMGroup | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `group.member_removed` | A member (actor or entity) has been removed from a DCMGroup | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
+| `authorization.granted` | A cross-tenant authorization has been granted | entity_uuid, from_state, to_state (where applicable) | LCM, AUD, OBS |
diff --git a/content/docs/data-model/34-api-versioning-strategy.md b/content/docs/data-model/34-api-versioning-strategy.md
index 34c66bc..b906171 100644
--- a/content/docs/data-model/34-api-versioning-strategy.md
+++ b/content/docs/data-model/34-api-versioning-strategy.md
@@ -38,7 +38,7 @@ Individual endpoints are not independently versioned. If a single endpoint needs
 Non-breaking changes within a major version are documented in the API changelog but do not change the URL. Clients do not need to take any action for non-breaking changes.
 
 The changelog follows semantic versioning conventions:
-- **Minor change**: new optional fields, new endpoints, expanded enum values with backward-compatible defaults
+- **Minor change**: new optional fields, new endpoints, expanded enum values with version-compatible defaults
 - **Revision**: documentation corrections, clarifications, non-functional specification updates
 
 ---
@@ -305,7 +305,7 @@ provider_registration:
 
 ### 7.2 OIS Compatibility
 
-DCM maintains backward compatibility with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 during the deprecation window.
+DCM maintains version-compatible with registered OIS versions during the support lifecycle. A DCM instance running OIS v2 must continue to dispatch to providers registered on OIS v1 until the version is sunset.
 
 When the OIS version is incremented:
 1. DCM announces the new OIS version via the event `governance.ois_version_released`
@@ -319,17 +319,15 @@ Providers that expose their own management APIs (beyond the standard OIS surface
 
 ---
 
-## 8. Client Migration Path
+## 8. Version Upgrade Path
 
-### 8.1 Migration Guide Structure
-
-Each new major version publishes a migration guide accessible at:
+When a new major API version is published, a machine-readable change log is available at:
 
 ```
 GET /api/v{N}/migration-guide
 ```
 
-The migration guide is machine-readable JSON listing all breaking changes from the previous version:
+This endpoint returns all breaking changes from the previous major version:
 
 ```json
 {
@@ -340,22 +338,18 @@ The migration guide is machine-readable JSON listing all breaking changes from t
       "change_id": "BC-001",
       "type": "field_removed",
       "endpoint": "GET /api/v2/resources/{uuid}",
-      "description": "Field 'legacy_id' removed from response. Use 'entity_uuid' instead.",
-      "migration": "Replace references to 'legacy_id' with 'entity_uuid'",
-      "affected_since": "2026-06-01"
+      "description": "Field 'legacy_id' removed โ€” use 'entity_uuid' instead"
     }
   ],
-  "non_breaking_additions": [ ... ],
-  "sunset_date_of_previous_version": "2027-06-01"
+  "new_capabilities": []
 }
 ```
 
-### 8.2 Parallel Operation
-
-During the deprecation window, clients may run v1 and v2 in parallel โ€” for example, migrating one service at a time. Both versions return consistent data from the same underlying DCM data stores. There are no data synchronization concerns between versions.
+Clients declare the API version they target via the `Accept-Version` header or URL prefix. DCM supports all non-sunset major versions simultaneously. When a version reaches sunset, responses include `Deprecation` and `Sunset` headers (RFC 8594) before support is withdrawn.
 
 ---
 
+
 ## 9. Internal API Versioning
 
 DCM internal component APIs (Control Plane components communicating with each other) follow a simpler model:
@@ -379,7 +373,7 @@ DCM internal component APIs (Control Plane components communicating with each ot
 | `VER-006` | The `latest` version alias is available but must not be recommended for production use. Production clients must pin to a specific version. |
 | `VER-007` | Preview endpoints are not stable. They may change or be removed without a major version increment. They are identified by the `/preview/` path segment. |
 | `VER-008` | Every new major version must publish a machine-readable migration guide at `/api/v{N}/migration-guide`. |
-| `VER-009` | DCM must maintain dispatch compatibility with providers registered on supported OIS versions during the OIS deprecation window. |
+| `VER-009` | DCM must maintain dispatch compatibility with providers registered on supported OIS versions until the OIS version is sunset. |
 
 ---
 
diff --git a/content/docs/data-model/35-session-revocation.md b/content/docs/data-model/35-session-revocation.md
index 55c0c38..6f431c1 100644
--- a/content/docs/data-model/35-session-revocation.md
+++ b/content/docs/data-model/35-session-revocation.md
@@ -195,7 +195,7 @@ session_revocation_registry:
 DCM's Ingress layer exposes a token introspection endpoint for internal components and external systems that need to validate a token without maintaining their own cache:
 
 ```
-POST /api/v1/auth/introspect
+POST /api/v1/auth:introspect
 
 Authorization: Bearer 
 Content-Type: application/json
@@ -283,7 +283,7 @@ Response 404: session not found or does not belong to this actor
 ### 6.5 Admin: Force Revoke Session(s)
 
 ```
-POST /api/v1/admin/actors/{actor_uuid}/revoke-sessions
+POST /api/v1/admin/actors/{actor_uuid}:revoke-sessions
 
 {
   "scope": "all | session",
@@ -342,7 +342,7 @@ Session revocation (this document) and credential revocation (doc 31, CPX-001โ€“
 | `AUTH-017` | Session revocation must propagate to the Session Revocation Registry within the profile-governed SLA: minimal PT5M, standard PT1M, prod PT30S, fsi PT10S, sovereign PT5S. |
 | `AUTH-018` | All DCM components that accept bearer tokens must check the Session Revocation Registry on each request. Cache age must not exceed the profile-governed maximum (sovereign: no cache). |
 | `AUTH-019` | Emergency session revocation (security_event trigger) fires immediately with no grace period. The `auth.security_session_revoked` event has `urgency: critical` and is non-suppressable. |
-| `AUTH-020` | The token introspection endpoint (`POST /api/v1/auth/introspect`) must be authenticated. Access requires an actor or service account with the `introspection` scope. |
+| `AUTH-020` | The token introspection endpoint (`POST /api/v1/auth:introspect`) must be authenticated. Access requires an actor or service account with the `introspection` scope. |
 | `AUTH-021` | When concurrent session limits are enforced, the oldest session is revoked before the new session is created. The evicted actor is notified via Notification Provider if configured. |
 | `AUTH-022` | Refresh tokens are invalidated when their parent session is revoked. A revoked refresh token returns 401 on exchange; it cannot be used to create a new session. |
 
diff --git a/content/docs/data-model/39-dcm-self-health.md b/content/docs/data-model/39-dcm-self-health.md
index e3ba7ef..ce116e8 100644
--- a/content/docs/data-model/39-dcm-self-health.md
+++ b/content/docs/data-model/39-dcm-self-health.md
@@ -337,6 +337,85 @@ dcm_internal_ca_days_until_next_expiry
 
 ---
 
+## 7. Per-Provider Metrics Contract
+
+In addition to DCM control plane metrics, each registered Service Provider must
+expose a Prometheus-compatible `/metrics` endpoint meeting the contract defined in
+the Registration Specification (GATE-SP-05).
+
+### 7.1 Required Provider Metric Families
+
+```
+# Dispatch metrics โ€” how many requests DCM sent to this provider
+dcm_provider_dispatches_total{resource_type="Compute.VirtualMachine", outcome="success|failed|timeout"}
+dcm_provider_dispatch_duration_seconds{resource_type="Compute.VirtualMachine", quantile="0.5|0.95|0.99"}
+
+# Realization metrics โ€” outcomes of provisioning
+dcm_provider_realizations_total{resource_type="Compute.VirtualMachine", status="OPERATIONAL|FAILED"}
+
+# Health signal
+dcm_provider_health_status  # gauge: 1=healthy, 0.5=degraded, 0=unhealthy
+```
+
+### 7.2 Recommended Provider Metric Families
+
+```
+# Capacity
+dcm_provider_capacity_remaining{resource_type="Compute.VirtualMachine"}  # gauge
+dcm_provider_capacity_total{resource_type="Compute.VirtualMachine"}      # gauge
+
+# Queue depth (for async-only providers)
+dcm_provider_queue_depth{resource_type="Compute.VirtualMachine"}  # gauge
+
+# Tenant usage
+dcm_provider_active_resources_total{tenant_uuid="...", resource_type="..."}
+```
+
+### 7.3 DCM Control Plane Aggregated Provider Metrics
+
+The DCM control plane exposes aggregated provider metrics at its own `/metrics`
+endpoint alongside the control plane metrics from Section 6:
+
+```
+# Already in Section 6 โ€” shown here for cross-reference
+dcm_providers_registered_total               # count of registered providers
+dcm_providers_healthy_total                  # count currently healthy
+dcm_provider_dispatch_duration_seconds{...}  # aggregated across all providers
+```
+
+### 7.4 Tenant Metadata Endpoint
+
+Service Providers in `standard` and above profiles (GATE-SP-04) must implement
+a tenant metadata endpoint that DCM calls to retrieve per-tenant usage summaries:
+
+```
+GET /api/v1/tenants/{tenant_uuid}/metadata
+Authorization: Bearer 
+
+Response 200:
+{
+  "tenant_uuid": "",
+  "active_resources": {
+    "Compute.VirtualMachine": 12,
+    "Storage.Block": 8
+  },
+  "capacity_consumed": {
+    "Compute.VirtualMachine": {
+      "cpu_cores": 96,
+      "ram_gb": 384
+    }
+  },
+  "quota_consumed_pct": {
+    "Compute.VirtualMachine": 48.0
+  }
+}
+```
+
+This data is used by DCM's Cost Analysis component and multi-tenant quota
+enforcement. It is also exposed to tenant administrators via the consumer API.
+
+---
+
 ## 8. Profile-Governed Health Exposure
 
 | Profile | /livez | /readyz | /api/v1/admin/health | /metrics scraping |
diff --git a/content/docs/data-model/40-standards-catalog.md b/content/docs/data-model/40-standards-catalog.md
index 26c45aa..f74b61e 100644
--- a/content/docs/data-model/40-standards-catalog.md
+++ b/content/docs/data-model/40-standards-catalog.md
@@ -19,7 +19,7 @@
 |-----|-------|-----------|-----------|
 | **RFC 7519** | JSON Web Token (JWT) | Bearer token format for session tokens and API key tokens; claims carry actor_uuid, roles, tenant_uuid, exp | Normative |
 | **RFC 7517** | JSON Web Key (JWK) | Public key format for Auth Provider OIDC verification keys; JWKS endpoint for key discovery | Normative |
-| **RFC 7662** | OAuth 2.0 Token Introspection | `POST /api/v1/auth/introspect` โ€” validates bearer tokens; response format `{active, session_uuid, actor_uuid, exp, roles}` | Normative |
+| **RFC 7662** | OAuth 2.0 Token Introspection | `POST /api/v1/auth:introspect` โ€” validates bearer tokens; response format `{active, session_uuid, actor_uuid, exp, roles}` | Normative |
 | **RFC 6749** | OAuth 2.0 Authorization Framework | Authorization flow for OIDC Auth Providers; client credentials flow for service account API keys | Informative |
 | **RFC 4511** | Lightweight Directory Access Protocol (LDAP) | LDAP/FreeIPA/Active Directory Auth Provider integration; bind operations, search filters for group membership | Normative |
 | **RFC 7643** | SCIM 2.0 Core Schema | Actor and group provisioning schema for enterprise IdP integration; SCIM deprovision triggers session + credential revocation | Normative |
diff --git a/content/docs/data-model/42-itsm-integration.md b/content/docs/data-model/42-itsm-integration.md
index 5dfc4f4..df15199 100644
--- a/content/docs/data-model/42-itsm-integration.md
+++ b/content/docs/data-model/42-itsm-integration.md
@@ -51,7 +51,7 @@ DCM lifecycle event fires (e.g. request.dispatched)
 ITSM system approves a change record
   โ”‚
   โ–ผ ITSM system calls DCM API (via webhook or polling)
-  โ”‚   POST /api/v1/admin/approvals/{uuid}/vote
+  โ”‚   POST /api/v1/admin/approvals/{uuid}:vote
   โ”‚   { decision: "approve", recorded_via: "servicenow",
   โ”‚     external_reference: "CHG0012345" }
   โ”‚
@@ -182,7 +182,7 @@ itsm_provider_registration:
     
   # CAB approval โ†’ DCM vote mapping
   inbound_approval:
-    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote"
+    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}:vote"
     trigger_on: "change_request.state โ†’ 'Approved'"
     decision_field: "state"
     decision_map:
@@ -208,7 +208,7 @@ itsm_provider_registration:
 ServiceNow Change Advisory Board approves CHG0012345
   โ”‚
   โ–ผ ServiceNow Business Rule fires on state change โ†’ "Approved"
-  โ”‚   Calls DCM webhook: POST /api/v1/admin/approvals/{uuid}/vote
+  โ”‚   Calls DCM webhook: POST /api/v1/admin/approvals/{uuid}:vote
   โ”‚   Headers: X-ServiceNow-Signature: 
   โ”‚   Body: { decision: "approve", recorded_via: "servicenow",
   โ”‚           external_reference: "CHG0012345" }
@@ -251,7 +251,7 @@ itsm_provider_registration:
     labels: ["dcm-automated", "{tenant_handle}"]
     
   inbound_approval:
-    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}/vote"
+    webhook_url: "https://dcm.corp/api/v1/admin/approvals/{approval_uuid}:vote"
     trigger_on: "issue.status โ†’ 'Approved'"
     decision_map:
       "Approved": "approve"
diff --git a/content/docs/data-model/43-provider-callback-auth.md b/content/docs/data-model/43-provider-callback-auth.md
new file mode 100644
index 0000000..e14b964
--- /dev/null
+++ b/content/docs/data-model/43-provider-callback-auth.md
@@ -0,0 +1,421 @@
+# DCM Data Model โ€” Provider Callback Authentication
+
+**Document Status:** โœ… Complete
+**Document Type:** Architecture Reference
+**Related Documents:** [Unified Provider Contract](A-provider-contract.md) | [Credential Provider Model](31-credential-provider-model.md) | [Accreditation, Auth Matrix, Zero Trust](26-accreditation-and-authorization-matrix.md) | [Internal Component Auth](36-internal-component-auth.md) | [Registration Specification](../specifications/dcm-registration-spec.md) | [Provider Callback API](../schemas/openapi/dcm-provider-callback-api.yaml)
+
+> **Foundation Document Reference**
+>
+> This document maps to: **PROVIDER** (authentication of the provider-to-DCM interaction boundary)
+> and **DATA** (the credential artifact that governs that boundary).
+>
+> See [00-foundations.md](00-foundations.md) ยท [A-provider-contract.md](A-provider-contract.md) ยท [B-policy-contract.md](B-policy-contract.md)
+
+---
+
+## 1. Purpose and Scope
+
+This document specifies how Service Providers authenticate inbound calls to the DCM control plane โ€” specifically, calls to the Provider Callback API endpoints:
+
+- `POST /api/v1/providers` โ€” Registration
+- `POST /api/v1/providers/{provider_uuid}/capacity` โ€” Capacity reporting
+- `PUT /api/v1/instances/{resource_id}/status` โ€” Realized state push
+- `POST /api/v1/provider/entities/{entity_uuid}/status` โ€” Interim progress
+- `POST /api/v1/provider/entities/{entity_uuid}/update-notification` โ€” Authorized state change notification
+- `GET /api/v1/provider/notifications/{notification_uuid}` โ€” Notification status poll
+- `POST /api/v1/instances/{resource_id}/events` โ€” Lifecycle event reporting
+
+The outbound model (DCM authenticating to providers) is specified in [doc 31 Section 4.2](31-credential-provider-model.md) and [doc 26 Section 5.3](26-accreditation-and-authorization-matrix.md). This document specifies the **inbound** model.
+
+---
+
+## 2. The Authentication Problem
+
+When DCM receives a callback at `PUT /api/v1/instances/{resource_id}/status`, it must verify:
+
+1. **Identity:** Is this call genuinely from the registered Service Provider for this resource?
+2. **Authorization:** Is this provider permitted to push realized state for this specific resource/entity?
+3. **Integrity:** Has the payload been tampered with in transit?
+4. **Freshness:** Is this a live call, not a replayed credential from a previous session?
+5. **Scope:** Is this credential permitted for this specific operation type?
+
+Network-level authentication alone (firewall rules, IP allowlisting) is insufficient under the DCM Zero Trust model โ€” it establishes perimeter trust, not per-call identity. Every provider call to DCM must carry a credential that answers all five questions independently of network position.
+
+---
+
+## 3. Two-Layer Authentication Model
+
+DCM uses a **two-layer** model for provider-to-DCM calls. Both layers must pass:
+
+```
+Provider calls DCM:
+  โ”‚
+  โ–ผ Layer 1: Transport Identity (mTLS)
+  โ”‚   Provider presents its registered certificate
+  โ”‚   DCM verifies the certificate chain against the provider's registered CA
+  โ”‚   Proves: this connection is from the registered provider
+  โ”‚   Does NOT prove: authorization for this specific operation
+  โ”‚
+  โ–ผ Layer 2: Operation Authorization (Provider Callback Credential)
+  โ”‚   Provider presents a scoped short-lived credential in the Authorization header
+  โ”‚   DCM validates: credential is active, scoped to this provider, scoped to this operation type
+  โ”‚   Proves: this specific call is authorized for this specific operation
+  โ”‚   Does NOT replace mTLS โ€” both layers are required
+  โ”‚
+  โ–ผ Both pass โ†’ five-check boundary model evaluates (doc 26 Section 5.2)
+  โ””โ”€โ”€ Audit record written regardless of outcome
+```
+
+**Why two layers?** mTLS proves the caller holds the private key for the registered certificate โ€” it proves identity at the transport level. The interaction credential proves the specific call is authorized for the specific operation type and scope. A compromised credential without the private key cannot establish the mTLS connection. A valid mTLS connection without a valid credential cannot perform operations. The layers are complementary, not redundant.
+
+---
+
+## 4. Layer 1 โ€” Transport Identity (mTLS)
+
+### 4.1 Provider Certificate Registration
+
+At registration, every provider declares a certificate:
+
+```yaml
+provider_registration:
+  certificate:
+    pem: 
+    ca_chain: 
+    rotation_interval: P90D
+```
+
+DCM validates:
+- Certificate chain is valid and trusted
+- Certificate is not in DCM's Credential Revocation Registry
+- Certificate `CN` or `SAN` matches the provider's declared `handle`
+- Certificate `valid_until` is not in the past
+
+DCM stores the certificate fingerprint. On every subsequent inbound connection, DCM validates the presented certificate against the stored fingerprint for this provider.
+
+### 4.2 Certificate Validation on Inbound Calls
+
+When a provider initiates a TLS connection to DCM:
+
+```
+Provider โ†’ DCM:
+  TLS ClientHello โ†’ ServerHello + DCM certificate
+  Provider verifies DCM certificate (validates DCM's identity)
+  Provider sends its certificate
+  DCM validates:
+    1. Certificate chain โ†’ registered CA for this provider
+    2. Certificate fingerprint โ†’ matches stored fingerprint for provider_uuid in path/payload
+    3. Certificate not in Credential Revocation Registry
+    4. Certificate valid_until not expired
+  If any check fails โ†’ TLS handshake rejected; connection refused
+```
+
+**Certificate rotation:** Providers must rotate certificates on the declared `rotation_interval`. DCM fires a `P14D` warning event when a certificate is approaching expiry. During the rotation transition window, DCM accepts both the current and new certificate simultaneously (P7D window). After the window, only the new certificate is accepted.
+
+### 4.3 Certificate Binding to Operations
+
+The mTLS certificate is **not sufficient alone** for operation authorization. Knowing a call came from Provider X does not mean Provider X is authorized to push realized state for entity Y owned by Tenant Z. The interaction credential (Layer 2) carries that authorization.
+
+---
+
+## 5. Layer 2 โ€” Operation Authorization (Provider Callback Credential)
+
+### 5.1 Provider Callback Credential
+
+The **Provider Callback Credential** is a `dcm_interaction` type credential issued to the provider by DCM's Credential Provider at registration activation time. It is the mechanism by which providers prove authorization for specific callback operations.
+
+```yaml
+provider_callback_credential:
+  credential_uuid: 
+  credential_type: dcm_interaction
+  issued_to:
+    provider_uuid:       # the specific registered provider
+    provider_handle:   # for human-readable audit records
+  issued_at: 
+  valid_until:       # profile-governed lifetime; see Section 5.3
+  operation_scope:
+    allowed_operations:
+      - realized_state_push
+      - capacity_report
+      - interim_status
+      - update_notification
+      - lifecycle_event
+      - notification_poll
+      # Note: registration uses a registration_token, not this credential
+    # Scope is bound to the provider_uuid โ€” cannot be used for other providers
+  non_transferable: true
+  bound_to_ip:        # fsi/sovereign profiles: IP-bound
+  revocation_check_url: 
+```
+
+**Presented as:** `Authorization: Bearer ` on all callback API calls.
+
+**Key property:** The credential is scoped to the `provider_uuid` โ€” not to specific entities or operations within that provider. Entity-level scope is enforced separately (Section 6). This means a provider holding the credential can call any callback endpoint, but DCM enforces entity-level ownership checks per call.
+
+### 5.2 Credential Issuance
+
+Provider callback credentials are issued through the following lifecycle:
+
+```
+Registration approved (provider status โ†’ ACTIVE):
+  โ”‚
+  โ–ผ DCM API Gateway requests credential from Credential Provider:
+  โ”‚   credential_type: dcm_interaction
+  โ”‚   issued_to.provider_uuid: 
+  โ”‚   allowed_operations: [realized_state_push, capacity_report, interim_status,
+  โ”‚                         update_notification, lifecycle_event, notification_poll]
+  โ”‚   valid_until: 
+  โ”‚
+  โ–ผ Credential Provider issues credential
+  โ”‚   Returns credential_value (the bearer token)
+  โ”‚   Stores credential_record in Credential Store
+  โ”‚
+  โ–ผ DCM delivers credential to provider via the activation response:
+  โ”‚   POST /api/v1/admin/providers/{uuid}:approve
+  โ”‚   Response includes: credential_ref (UUID for retrieval)
+  โ”‚
+  โ–ผ Provider retrieves credential value via Credential Provider endpoint:
+  โ”‚   GET {credential_provider_endpoint}/credentials/{credential_ref}/value
+  โ”‚   (Requires the registration token used at initial registration โ€” one-time bootstrap)
+  โ”‚
+  โ–ผ Provider stores credential securely and uses it for all callback API calls
+```
+
+**Bootstrap case:** During initial registration (before activation), the provider uses the registration token to authenticate. After activation, the provider callback credential replaces the registration token for all subsequent calls. The registration token is single-use and expires after the first successful registration response.
+
+### 5.3 Credential Lifetime by Profile
+
+| Profile | Lifetime | Rotation trigger | IP binding |
+|---------|----------|-----------------|------------|
+| minimal | PT8H | Pre-expiry P1H | No |
+| dev | PT4H | Pre-expiry P30M | No |
+| standard | PT1H | Pre-expiry PT10M | No |
+| prod | PT30M | Pre-expiry PT5M | Optional |
+| fsi | PT15M | Pre-expiry PT3M | Required |
+| sovereign | PT15M + hardware attestation | Pre-expiry PT3M | Required; HSM-bound |
+
+**Pre-expiry rotation:** DCM initiates rotation automatically before the current credential expires. The transition window is 50% of the credential lifetime โ€” the old credential remains valid during the window while the new one is delivered. Providers must implement credential refresh in their SDK or adapter.
+
+### 5.4 Credential Rotation Protocol
+
+```
+PT{rotation_trigger} before credential expiry:
+  โ”‚
+  โ–ผ DCM initiates rotation:
+  โ”‚   Requests new credential from Credential Provider
+  โ”‚   rotation_of: 
+  โ”‚   same allowed_operations scope; new valid_until
+  โ”‚
+  โ–ผ Credential Provider issues new credential
+  โ”‚   Old credential NOT yet revoked
+  โ”‚
+  โ–ผ DCM pushes rotation notification to provider:
+  โ”‚   POST {provider_health_endpoint}/credential-rotation (if provider supports it)
+  โ”‚   OR: credential.rotating event published to Message Bus
+  โ”‚   New credential_ref included; provider retrieves new value
+  โ”‚
+  โ–ผ Transition window opens:
+  โ”‚   DCM accepts BOTH old and new credential during transition window
+  โ”‚   Provider switches to new credential
+  โ”‚
+  โ–ผ Transition window closes:
+  โ”‚   Old credential revoked
+  โ”‚   Revocation event published โ†’ all components update revocation cache
+```
+
+If the provider fails to pick up the new credential before the transition window closes, the old credential is revoked and the provider's callback calls will return `403 Forbidden` with code `CREDENTIAL_EXPIRED`. The provider must re-register to obtain a new credential โ€” this is a recoverable state.
+
+### 5.5 DCM Validation on Inbound Calls
+
+When DCM receives a callback call, Layer 2 validation performs these checks in order:
+
+```
+1. Extract credential_value from Authorization: Bearer header
+   โ†’ Missing or malformed: 401 Unauthorized; MISSING_CREDENTIAL audit record
+
+2. Look up credential_record by credential_value hash
+   โ†’ Not found: 401 Unauthorized; CREDENTIAL_NOT_FOUND audit record
+
+3. Check credential_record.status is 'active'
+   โ†’ Revoked: 403 Forbidden; code: CREDENTIAL_REVOKED
+   โ†’ Expired: 403 Forbidden; code: CREDENTIAL_EXPIRED
+
+4. Check credential_record.valid_until > now
+   โ†’ Expired: 403 Forbidden; code: CREDENTIAL_EXPIRED
+
+5. Check credential_record.issued_to.provider_uuid matches:
+   a. The provider_uuid in the URL path (where applicable)
+   b. The mTLS certificate's registered provider (Layer 1 binding)
+   โ†’ Mismatch: 403 Forbidden; code: CREDENTIAL_SCOPE_VIOLATION
+
+6. Check that the operation_type for this endpoint is in allowed_operations
+   โ†’ Not in scope: 403 Forbidden; code: OPERATION_NOT_IN_SCOPE
+
+7. If bound_to_ip is set: verify client IP matches
+   โ†’ Mismatch: 403 Forbidden; code: IP_BINDING_VIOLATION
+```
+
+All failures write an audit record with the credential_uuid, provider_uuid, endpoint, and failure reason. After 5 consecutive `CREDENTIAL_SCOPE_VIOLATION` or `IP_BINDING_VIOLATION` failures from the same provider within PT1H, DCM fires a `security.unsanctioned_provider_write` event and notifies the platform admin (urgency: critical).
+
+---
+
+## 6. Entity-Level Authorization
+
+The provider callback credential proves the caller is the registered provider. It does not prove the provider is authorized to act on a specific entity. Entity-level authorization is a separate check that applies on each call.
+
+### 6.1 Resource Ownership Binding
+
+For `realized_state_push` and `interim_status` calls, DCM validates:
+
+```
+PUT /api/v1/instances/{resource_id}/status
+
+DCM checks:
+  1. Look up the Requested State record for resource_id
+  2. Verify the credential's provider_uuid matches the provider_uuid
+     in the Requested State record (i.e., this was the provider DCM dispatched to)
+  3. Verify the entity is in a lifecycle state that permits this push
+     (PROVISIONING, UPDATING, or DECOMMISSIONING โ€” not OPERATIONAL, not DECOMMISSIONED)
+  
+  โ†’ Mismatch on provider_uuid: 403 Forbidden; code: ENTITY_NOT_OWNED_BY_PROVIDER
+  โ†’ Wrong lifecycle state: 409 Conflict; code: INVALID_LIFECYCLE_STATE_FOR_PUSH
+```
+
+**Why this matters:** A provider that receives a `resource_id` (e.g., by observing network traffic or misconfiguration) cannot push realized state for an entity it was not dispatched to. The Requested State record binds the entity to the specific provider that received the dispatch.
+
+### 6.2 Update Notification Binding
+
+For `update_notification` calls, DCM validates:
+
+```
+POST /api/v1/provider/entities/{entity_uuid}/update-notification
+
+DCM checks:
+  1. Look up the Realized State record for entity_uuid
+  2. Verify the credential's provider_uuid matches the provider_uuid
+     in the most recent Realized State record
+  3. Verify the provider's registration includes the update_capability
+     declared in the notification_type field
+
+  โ†’ Provider not current owner: 403 Forbidden; code: ENTITY_NOT_OWNED_BY_PROVIDER
+  โ†’ Update type not declared at registration: 403 Forbidden;
+    code: UPDATE_TYPE_NOT_DECLARED
+    (provider must re-register to declare new update capability types)
+```
+
+### 6.3 Lifecycle Event Binding
+
+For `lifecycle_event` calls, DCM validates:
+
+```
+POST /api/v1/instances/{resource_id}/events
+
+DCM checks:
+  1. Verify the credential's provider_uuid matches the provider on record for resource_id
+  2. Verify the resource is in an operational state (not DECOMMISSIONED)
+  3. Verify the event_type is in the standard event catalog
+
+  โ†’ Provider not current owner: 403 Forbidden; code: ENTITY_NOT_OWNED_BY_PROVIDER
+  โ†’ Entity decommissioned: 409 Conflict; code: ENTITY_DECOMMISSIONED
+  โ†’ Unknown event_type: 400 Bad Request; code: UNKNOWN_EVENT_TYPE
+```
+
+---
+
+## 7. Registration Authentication (Special Case)
+
+The initial `POST /api/v1/providers` registration call cannot use the provider callback credential because no credential exists yet. Registration uses a different authentication mechanism:
+
+### 7.1 Registration Token
+
+The registration token is a short-lived, single-use credential issued by a platform admin before provider onboarding:
+
+```yaml
+registration_token:
+  token_uuid: 
+  token_value: 
+  issued_at: 
+  expires_at:    # typically PT72H
+  scope:
+    provider_type_id: service_provider
+    provider_handle_pattern: "eu-west-*"   # optional constraint
+    grants_auto_approval: true | false
+  used: false              # single-use; set to true after first successful use
+```
+
+The registration token is passed as `Authorization: Bearer ` on the initial `POST /api/v1/providers` call. After the first successful registration, the token is marked `used: true` and cannot be reused. If a provider needs to re-register (e.g., after a sovereignty declaration change), a new registration token is required.
+
+**mTLS still required for registration:** The mTLS layer (Layer 1) is enforced on the registration call. The provider must present the certificate declared in the registration payload. This ensures the entity performing the registration possesses the private key for the certificate it is claiming.
+
+### 7.2 Re-Registration
+
+For re-registration calls (same `name`, updating version or capabilities), the provider uses its active provider callback credential โ€” not a new registration token. Re-registration that changes the sovereignty declaration requires a new registration token from the platform admin (treated as a new registration requiring a new approval).
+
+---
+
+## 8. Credential Revocation and Emergency Response
+
+### 8.1 Revocation Triggers
+
+| Trigger | What happens |
+|---------|-------------|
+| Provider deregistered | All callback credentials for that provider revoked immediately |
+| Security event detected (5+ scope violations in PT1H) | Provider suspended; credential revoked; platform admin notified |
+| Provider certificate expiry without rotation | Credential revoked at certificate expiry |
+| Platform admin explicit revocation | Immediate revocation; provider must re-register |
+| Provider compromise suspected | Emergency revocation; Recovery Policy evaluates affected entities |
+
+### 8.2 Emergency Revocation Flow
+
+```
+Platform admin triggers emergency revocation:
+  โ”‚
+  โ–ผ POST /api/v1/admin/providers/{provider_uuid}/revoke-credential
+  โ”‚   reason: 
+  โ”‚   suspend_provider: true | false
+  โ”‚
+  โ–ผ DCM revokes credential immediately:
+  โ”‚   credential_record.status โ†’ revoked
+  โ”‚   Revocation event โ†’ Message Bus
+  โ”‚   All DCM components update revocation cache (within PT30S)
+  โ”‚
+  โ–ผ If suspend_provider: true:
+  โ”‚   Provider status โ†’ SUSPENDED
+  โ”‚   New requests not routed to this provider
+  โ”‚   Active realizations enter PENDING_REVIEW state
+  โ”‚
+  โ–ผ Recovery Policy evaluates affected entities:
+      Entities currently hosted at provider: notify Tenant owners
+      In-progress operations: depends on Recovery Policy profile
+```
+
+### 8.3 Revocation Cache
+
+DCM components that validate inbound credentials maintain a local **Credential Revocation Cache**:
+
+- Cache is populated from the Message Bus `credential.revoked` event stream
+- Cache TTL matches the maximum credential lifetime for the active profile
+- On cache miss: remote check against Credential Store (prevents stale cache from accepting revoked credentials)
+- Cache invalidation is immediate on `credential.revoked` event receipt (not TTL-based)
+
+The revocation cache ensures revocation propagates within PT30S even without a cache miss triggering a remote lookup.
+
+---
+
+## 9. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `PCA-001` | All provider-to-DCM calls must present both a valid mTLS certificate (Layer 1) and a valid provider callback credential (Layer 2). Neither layer alone is sufficient. |
+| `PCA-002` | Provider callback credentials are scoped to the provider_uuid and cannot be used to act on entities hosted at other providers. |
+| `PCA-003` | Entity-level authorization is checked on every realized_state_push, update_notification, and lifecycle_event call, independent of credential validity. A valid credential does not grant access to entities the provider was not dispatched to. |
+| `PCA-004` | Five consecutive credential scope violations or IP binding violations from the same provider within PT1H triggers automatic provider suspension and platform admin notification. |
+| `PCA-005` | Provider callback credentials are issued by the Credential Provider, not directly by the DCM API Gateway. The Credential Provider is the authoritative source for all credential issuance, rotation, and revocation. |
+| `PCA-006` | Registration tokens are single-use. A registration token that has been used once is permanently invalidated regardless of its `expires_at` timestamp. |
+| `PCA-007` | Re-registration that changes the sovereignty declaration requires a new registration token and triggers a new approval pipeline. Version and capability updates do not require a new registration token. |
+| `PCA-008` | Provider callback credentials must be rotated before expiry. DCM initiates rotation automatically. If a credential expires without rotation, the provider enters a CREDENTIAL_EXPIRED state and must obtain a new credential via the platform admin. |
+| `PCA-009` | For fsi and sovereign profiles, provider callback credentials are IP-bound. A credential presented from an IP address that does not match the `bound_to_ip` field is rejected regardless of its validity. |
+| `PCA-010` | All inbound provider calls โ€” including rejected calls โ€” produce an audit record containing the credential_uuid, provider_uuid, endpoint, operation_type, outcome, and timestamp. There are no silent failures. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/44-kessel-integration-evaluation.md b/content/docs/data-model/44-kessel-integration-evaluation.md
new file mode 100644
index 0000000..da4a86c
--- /dev/null
+++ b/content/docs/data-model/44-kessel-integration-evaluation.md
@@ -0,0 +1,470 @@
+# DCM โ€” Kessel Integration Evaluation
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” For Discussion
+**Document Type:** Integration Evaluation โ€” Pre-Implementation
+**Purpose:** This document evaluates the potential integration of DCM with the [Kessel project](https://github.com/project-kessel) for identity/access management and resource inventory. It is intended as a basis for discussion with the Kessel development team. **No architectural changes should be made to DCM based on this document until alignment with the Kessel team is confirmed.**
+
+**Related Documents:** [Auth Providers](19-auth-providers.md) | [Universal Group Model](15-universal-groups.md) | [Entity Relationships](09-entity-relationships.md) | [Four States](02-four-states.md) | [Accreditation and Zero Trust](26-accreditation-and-authorization-matrix.md) | [Control Plane Components](25-control-plane-components.md) | [Provider Callback Authentication](43-provider-callback-auth.md)
+
+**Related Projects:** [project-kessel](https://github.com/project-kessel) | [SpiceDB](https://github.com/authzed/spicedb) | [Google Zanzibar](https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/)
+
+---
+
+## 1. Executive Summary
+
+Kessel is a Red Hat project providing two capabilities: **Kessel Relations** (Relationship-Based Access Control built on SpiceDB, a Google Zanzibar implementation) and **Kessel Asset Inventory** (a hybrid cloud resource state tracking service with a common Protobuf/gRPC API).
+
+DCM has architecturally similar needs in both areas. The evaluation concludes:
+
+- **Kessel Relations** has strong alignment with DCM's access control requirements. The permission model maps cleanly, and the operational benefits โ€” Zanzibar-style consistency, scalable graph traversal, shared source of truth across Red Hat products โ€” are meaningful. Integration path exists via DCM's Auth Provider abstraction.
+
+- **Kessel Inventory** has partial alignment with DCM's Discovered State store. The fit is real but narrower than it might appear: Kessel Inventory is a current-state snapshot system; DCM's inventory is a four-state lifecycle model with field-level provenance, drift detection, and append-only audit. Integration path exists via DCM's Storage Provider abstraction.
+
+**Recommended next step:** Discussion with the Kessel development team to validate assumptions, confirm schema extensibility for DCM-specific resource types, and understand the Kessel Relations API stability and sovereign/air-gapped deployment model.
+
+---
+
+## 2. What Kessel Provides
+
+### 2.1 Kessel Relations
+
+Kessel Relations is an authorization service built on [SpiceDB](https://github.com/authzed/spicedb), which implements the [Google Zanzibar](https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/) consistent global authorization model.
+
+**Core model โ€” Relationship-Based Access Control (ReBAC):**
+- Resources and subjects are defined in a typed schema
+- Relationships between subjects and resources are stored as tuples: `subject:X relation:Y object:Z`
+- Permissions are computed by evaluating the relationship graph: "does user X have permission `submit_request` on tenant T?" traverses all paths from X to T through groups, roles, and other relationships
+- Transitive relationships are handled natively: if user X is a member of group G, and group G has `admin` on tenant T, X inherits `admin` on T
+
+**Zanzibar consistency model:**
+- Snapshot reads: consistent reads at a point in time
+- Zookie tokens: causality tokens that guarantee "read your own writes" without requiring full global linearizability โ€” after writing a relationship tuple, the response includes a zookie; subsequent reads with that zookie are guaranteed to observe the write
+
+**gRPC API surface (from the Kessel project):**
+- `CheckPermission(subject, permission, resource)` โ†’ allow/deny
+- `LookupResources(subject, permission, resource_type)` โ†’ list of resources subject has permission on
+- `LookupSubjects(resource, permission, subject_type)` โ†’ list of subjects that have permission on resource
+- `WriteRelationships(tuples)` โ†’ write relationship tuples
+- `DeleteRelationships(filter)` โ†’ remove relationship tuples
+
+### 2.2 Kessel Asset Inventory
+
+Kessel Asset Inventory is a resource tracking service designed to provide a unified inventory view across hybrid cloud infrastructure โ€” OpenShift clusters, RHEL systems, edge devices, and other Red Hat-managed resources.
+
+**Core model:**
+- Resources are described using a common Protobuf schema with a typed `ResourceType` and a `Spec` for type-specific fields
+- Current state is tracked as an upsertable snapshot โ€” last-write wins
+- gRPC streaming API for push (providers send state updates) and pull (consumers query current state)
+- Integration with Kessel Relations for auth-filtered inventory queries: "what resources of type X does subject Y have access to?"
+
+**Intended use case:** Giving tools like ACM (Advanced Cluster Management), Insights, and the Hybrid Cloud Console a single query surface for "what exists across my estate?"
+
+---
+
+## 3. DCM's Current Model โ€” What Needs to Be Understood
+
+Before evaluating integration, it is important to characterize what DCM already has in both areas.
+
+### 3.1 DCM's Authorization Model
+
+DCM's current authorization model has five components working together:
+
+**Auth Providers** (doc 19) โ€” DCM delegates authentication to registered Auth Providers (LDAP, OIDC, FreeIPA, Active Directory, mTLS). Auth Providers are registered through the standard Provider contract. Multiple Auth Providers can be active simultaneously. Auth Providers return: authenticated actor identity, group memberships, roles.
+
+**Universal Group Model** (doc 15) โ€” DCM groups (`DCMGroup`) are typed by `group_class`. The classes relevant to authorization:
+- `tenant_boundary` โ€” the ownership and isolation boundary; every resource entity belongs to exactly one tenant
+- `cross_tenant_authorization` โ€” the formal mechanism for Tenant A to grant Tenant B access to a specific resource
+- `policy_collection` โ€” groups that activate policy sets
+- DCMGroup membership is the basis for role resolution and policy application
+
+**RBAC via role mapping** โ€” Auth Providers map external groups to DCM roles (`consumer`, `platform_admin`, `sre`, etc.). The Policy Engine uses roles + group membership to evaluate access.
+
+**Five-check boundary model** (doc 26) โ€” Every interaction crosses five checks in sequence: identity verification โ†’ authorization โ†’ accreditation โ†’ data/capability matrix โ†’ sovereignty. Checks 1 and 2 are RBAC. Checks 3โ€“5 are DCM-specific and involve accreditation records, data classification, and sovereignty zones.
+
+**Cross-tenant authorization records** โ€” When Tenant A grants Tenant B access to a resource, a `cross_tenant_authorization` DCMGroup is created. The Policy Engine checks for the existence of this record when evaluating cross-tenant requests.
+
+**What DCM asks for in authorization decisions:**
+1. Does actor X have role Y within tenant T?
+2. What catalog items is actor X allowed to see? (RBAC-filtered list)
+3. Can actor X perform operation O on resource R? (role + tenant ownership)
+4. Does tenant T have a cross-tenant authorization to use resource R owned by tenant T2?
+5. Is actor X a member of DCMGroup G with the required quorum? (approval gates, `authorized` tier)
+
+### 3.2 DCM's Inventory Model
+
+DCM's inventory is the **Four States model** (doc 02). This is meaningfully different from a general-purpose resource inventory.
+
+**Intent State** โ€” The consumer's declared desired state. Stored as a GitOps artifact (PR-based workflow). Immutable after creation. Not a snapshot โ€” it is the authoritative record of what was requested and why.
+
+**Requested State** โ€” The assembled payload after layer enrichment, policy evaluation, and placement resolution. Write-once. Contains the full data model payload that was dispatched to the provider, including field-level provenance tracing every value back to its source.
+
+**Realized State** โ€” An append-only event stream of what the provider actually built. Every realization event is a new record โ€” not an upsert. Contains field-level provenance from the provider. The relationship between a Realized State record and its corresponding Requested State record is explicit and mandatory.
+
+**Discovered State** โ€” An ephemeral snapshot of what the provider currently reports as existing, obtained through active discovery polling. Used by the Drift Reconciliation Component to compare against Realized State.
+
+**What DCM asks for in inventory decisions:**
+1. What is the current lifecycle state of entity UUID X? (Realized State read)
+2. What resources does tenant T own? (indexed query over Realized State)
+3. What entities have relationship R to entity X? (Entity Relationship Graph, doc 09)
+4. What is the field-level provenance of field F on entity X? (Realized State metadata)
+5. What entities are currently drifted? (Drift Record Store, DRC component output)
+6. What did we discover vs what do we have as realized? (Drift comparison)
+7. What happened to entity X over its full lifecycle? (Audit Store, time-indexed)
+
+---
+
+## 4. Integration Analysis
+
+### 4.1 Kessel Relations โ€” Authorization Backend
+
+#### Mapping DCM's Permission Model to SpiceDB
+
+DCM's five authorization questions map to SpiceDB as follows:
+
+```
+# Proposed SpiceDB schema for DCM
+definition user {}
+
+definition group {
+  relation member:      user | group#member
+  relation parent_group: group
+  permission member     = member + parent_group->member
+}
+
+definition tenant {
+  relation member:      user | group#member
+  relation admin:       user | group#member
+  relation platform_admin: user | group#member
+  permission submit_request  = member + admin + platform_admin
+  permission manage_resources = admin + platform_admin
+  permission administer       = platform_admin
+}
+
+definition resource {
+  relation owner_tenant:    tenant
+  relation authorized_tenant: tenant    # cross-tenant authorization
+  relation viewer:          user | group#member
+  permission read   = owner_tenant->member + authorized_tenant->member + viewer
+  permission modify = owner_tenant->admin
+  permission decommission = owner_tenant->admin
+}
+
+definition dcm_group {
+  relation member:    user | group#member
+  relation quorum_threshold: integer  # NOTE: see Section 4.1.2
+}
+```
+
+**Question 1** (does actor X have role Y in tenant T?) โ†’ `CheckPermission(user:X, permission:submit_request, tenant:T)`
+
+**Question 2** (what catalog items can actor X see?) โ†’ `LookupResources(user:X, permission:read, resource_type:catalog_item)`
+
+**Question 3** (can actor X do operation O on resource R?) โ†’ `CheckPermission(user:X, permission:modify, resource:R)`
+
+**Question 4** (does tenant T have cross-tenant authorization on resource R?) โ†’ `CheckPermission(tenant:T#member, permission:read, resource:R)` โ€” satisfied if `authorized_tenant` relationship exists
+
+**Question 5** (approval gate quorum) โ€” **Does not map cleanly to SpiceDB.** See Section 4.1.2.
+
+#### 4.1.2 Approval Gate Quorum โ€” The Gap
+
+DCM's `authorized` tier approval requires N of M members of a declared DCMGroup to record decisions before an operation proceeds. SpiceDB is a membership and permission graph โ€” it answers "does this subject have this permission?" but it does not count decisions or track quorum state across time.
+
+**Resolution:** The approval gate workflow stays in DCM's Policy Engine regardless of Kessel integration. Kessel Relations handles who *can* approve (membership in the DCMGroup); DCM's Approval Store tracks who *has* approved and whether quorum is reached.
+
+This is a clean boundary: Kessel answers the structural question ("is this actor authorized to vote?"); DCM answers the state question ("how many valid votes have been recorded?").
+
+#### 4.1.3 DCM's Entity Relationship Graph is NOT an Authorization Graph
+
+This is a critical distinction. DCM's entity relationships โ€” `requires`, `constituent`, `shareable`, `allocated_from`, `peer` โ€” are **operational relationships between infrastructure resources**, not access control relationships. They express: "VM X requires Storage Y", "Composite C has constituent VM X."
+
+These must **not** be stored in Kessel Relations. They are:
+- Semantically different from access control (lifecycle implications, not permissions)
+- DCM-specific (not meaningful to any other system consuming Kessel)
+- Owned by DCM's entity lifecycle model
+
+DCM's Entity Relationship Graph (doc 09) remains entirely in DCM regardless of Kessel integration.
+
+#### 4.1.4 Checks 3โ€“5 of the Five-Check Boundary Model
+
+DCM's five-check boundary model (identity โ†’ authorization โ†’ accreditation โ†’ data matrix โ†’ sovereignty) maps to Kessel Relations only for checks 1 and 2. Checks 3โ€“5 are DCM-specific:
+
+- **Accreditation** (check 3): Does the target provider hold the required accreditation for the data classification present? This involves DCM's Accreditation Registry and is not a subject/permission/resource question.
+- **Data/Capability Matrix** (check 4): Is each field permitted to cross this boundary given its classification? This involves DCM's Governance Matrix policies.
+- **Sovereignty** (check 5): Is the target endpoint within the sovereignty boundary? This involves DCM's Sovereignty Zone declarations.
+
+None of checks 3โ€“5 can be delegated to Kessel Relations. They remain in DCM's Policy Engine.
+
+#### 4.1.5 Integration Path via Auth Provider Abstraction
+
+DCM's Auth Provider abstraction (doc 19) is the natural integration point. Kessel Relations would register as a DCM Auth Provider or Policy Provider:
+
+```yaml
+kessel_relations_auth_provider:
+  provider_type: auth_provider
+  auth_mode: kessel_rebac
+  endpoint: https://kessel-relations.internal:9000
+  schema_ref: 
+  
+  # What this provider handles:
+  handles:
+    - check_permission       # CheckPermission calls
+    - lookup_resources       # LookupResources calls
+    - lookup_subjects        # LookupSubjects calls
+  
+  # What stays in DCM's Policy Engine:
+  does_not_handle:
+    - accreditation_checks
+    - data_classification_matrix
+    - sovereignty_checks
+    - approval_gate_quorum
+```
+
+DCM's Policy Engine calls the Kessel Relations provider for authorization questions (checks 1 and 2) and evaluates checks 3โ€“5 internally. The five-check sequence is preserved; only the implementation of checks 1โ€“2 changes.
+
+**Zookie handling:** DCM's API Gateway must thread zookie tokens through the request lifecycle: when a relationship is written (e.g., a new cross-tenant authorization is created), the resulting zookie is stored and used for subsequent permission checks in the same request context, guaranteeing consistency.
+
+---
+
+### 4.2 Kessel Inventory โ€” Discovered State Store
+
+#### 4.2.1 The Fit
+
+Of DCM's four stores, **Discovered State** is the only one Kessel Inventory could plausibly replace. The reasons:
+
+- Discovered State is the most ephemeral store โ€” it is overwritten on each discovery cycle
+- Discovered State does not require immutability or append-only semantics โ€” it represents "what the provider reports right now"
+- Discovered State is the "current state of infrastructure" โ€” exactly what Kessel Inventory is designed to track
+- Other Red Hat tools consuming Kessel Inventory would benefit from seeing the same discovered state that DCM uses for drift detection
+
+The other three stores โ€” Intent, Requested, and Realized โ€” **cannot** be replaced by Kessel Inventory:
+- Intent and Requested State require GitOps semantics (PR workflow, immutability, version history)
+- Realized State requires append-only event stream semantics with field-level provenance and hash chain integrity
+- None of DCM's lifecycle or audit requirements are in scope for Kessel Inventory
+
+#### 4.2.2 The Schema Alignment Question
+
+DCM's Discovered State uses the same unified data model format as Realized State โ€” the DCM Resource Type Spec schema. Kessel Inventory uses a Protobuf-defined common resource schema.
+
+For standard resource types (Compute, Network, Storage that map to well-known infrastructure concepts), the alignment is likely achievable. For DCM-specific resource types (Automation.AnsiblePlaybook, Platform.KubernetesCluster, custom org-defined types), schema extension or mapping is required.
+
+**Open question for Kessel team:** How extensible is the Kessel Inventory resource type schema? Can DCM register custom resource types? Is there a type registry mechanism analogous to DCM's Resource Type Registry?
+
+#### 4.2.3 Drift Detection Logic Stays in DCM
+
+Kessel Inventory is a state store, not a drift detection system. Even if DCM uses Kessel Inventory as the Discovered State store, the Drift Reconciliation Component (doc 25, DRC domain) remains entirely in DCM:
+
+- DRC queries Kessel Inventory for current discovered state
+- DRC compares discovered state against DCM's Realized State
+- DRC classifies differences by field criticality and change magnitude
+- DRC produces Drift Records with SECURITY_DEGRADATION, BROKEN_REFERENCE, UNSANCTIONED_CHANGE classifications
+- DRC writes Drift Records to DCM's Drift Record Store
+
+Kessel Inventory's role is purely as the data source for the "what currently exists" side of the comparison. The intelligence stays in DCM.
+
+#### 4.2.4 Integration Path via Storage Provider Abstraction
+
+DCM's Storage Provider abstraction (doc 11) is the natural integration point. The Discovered Store would be implemented as a `storage_sub_type: snapshot_store` Storage Provider backed by Kessel Inventory:
+
+```yaml
+kessel_inventory_storage_provider:
+  provider_type: storage_provider
+  storage_sub_type: snapshot_store
+  backend: kessel_inventory
+  endpoint: https://kessel-inventory.internal:9001
+  
+  # DCM uses this provider for:
+  used_for: discovered_state
+  
+  # Write contract: provider calls POST /api/v1/instances/{id}/status
+  # which DCM translates to Kessel Inventory upsert
+  write_model: upsert_current_state
+  
+  # Read contract: DRC queries Kessel for discovered state
+  read_model: streaming_query_by_type_and_tenant
+```
+
+This means the Kessel Inventory integration requires **no changes to DCM's data model** โ€” only a new Storage Provider implementation. The Drift Reconciliation Component calls the same Discovered State Store interface; the underlying implementation happens to be Kessel Inventory.
+
+---
+
+## 5. Deployment and Sovereignty Considerations
+
+### 5.1 Air-Gapped and Sovereign Deployments
+
+DCM's `sovereign` profile requires air-gapped operation with no external dependencies. Any Kessel integration must support:
+
+- Local/on-premises Kessel deployment (not cloud-hosted)
+- Offline operation when Kessel is temporarily unavailable (cached authorization decisions for read-only operations)
+- mTLS between DCM and Kessel instances
+
+**Open question for Kessel team:** What is Kessel's deployment model for sovereign/air-gapped environments? Is there a supported on-premises deployment path? What is the operational footprint?
+
+### 5.2 Multi-Instance Federation
+
+DCM supports federation between multiple DCM instances (doc 22). A federated deployment may have multiple Kessel Relations instances (one per region or sovereignty zone) or a single shared instance.
+
+**Open question for Kessel team:** How does Kessel Relations handle multi-region replication? Can SpiceDB schema and relationship data be replicated across sovereignty boundaries? What are the consistency guarantees in a federated topology?
+
+### 5.3 Failure Mode Analysis
+
+If Kessel Relations is unavailable, DCM cannot evaluate authorization checks 1โ€“2 of the five-check model, which means DCM cannot process any requests. This is a critical dependency.
+
+**Required mitigation strategies:**
+- Read-through cache for CheckPermission results (short TTL, profile-governed)
+- Circuit breaker: if Kessel is unavailable for >N consecutive checks, DCM enters a safe-deny mode (no new requests accepted) rather than a fail-open mode
+- Kessel Relations HA deployment is a prerequisite, not optional
+
+**Open question for Kessel team:** What HA and disaster recovery patterns are recommended for production Kessel Relations deployments?
+
+---
+
+## 6. Questions for the Kessel Team
+
+The following questions should be addressed before any integration work begins:
+
+### 6.1 Kessel Relations
+
+| # | Question | Why It Matters |
+|---|----------|----------------|
+| 1 | What is the current API stability level of the Kessel Relations gRPC API? Are breaking changes expected? | DCM needs a stable contract to build against |
+| 2 | Does Kessel Relations support on-premises / air-gapped deployment? What is the operational footprint? | Required for DCM's `sovereign` profile |
+| 3 | How does the SpiceDB schema evolve? Is there a migration path when the DCM permission model changes? | Schema evolution is a production concern |
+| 4 | Can Kessel Relations store relationships at the scale DCM requires? How many relationship tuples per tenant at what query latency? | DCM may have thousands of cross-tenant authorization records per deployment |
+| 5 | How does Kessel handle the zookie (consistency token) lifecycle? Are zookies scoped to a namespace/tenant, or global? | Relevant to DCM's multi-tenant isolation model |
+| 6 | Is Kessel Relations multi-tenant natively, or does DCM need to namespace its SpiceDB schema? | Critical for DCM's tenant isolation requirements |
+| 7 | What is the intended integration pattern for other Red Hat products (ACM, Insights)? How would DCM's usage interoperate? | Kessel's value to DCM is partly the shared source of truth across RH products |
+| 8 | Does Kessel Relations have a concept equivalent to DCM's "cross-tenant authorization"? How are trust grants between tenants modeled? | Core to DCM's resource sharing model |
+
+### 6.2 Kessel Inventory
+
+| # | Question | Why It Matters |
+|---|----------|----------------|
+| 9 | How extensible is Kessel Inventory's resource type schema? Can DCM register custom resource types? | DCM has domain-specific resource types not in Kessel's default schema |
+| 10 | What is the write model? Last-write-wins upsert, or versioned? Does Kessel Inventory support the discovered state pattern (full overwrite on each discovery cycle)? | DCM's Discovered Store is a full-replacement snapshot per discovery cycle |
+| 11 | How does Kessel Inventory integrate with Kessel Relations for auth-filtered queries? Is the integration already built, or planned? | Core to the value of using Kessel Inventory |
+| 12 | What is the data retention model? Does Kessel Inventory keep history or only current state? | DCM needs "current state" only for Discovered State; history is in DCM's Audit Store |
+| 13 | What is the API stability level for Kessel Inventory? | Same concern as #1 for Relations |
+| 14 | Is there a reference implementation of a Kessel Inventory provider for a Kubernetes/OpenShift resource type? | DCM would follow this pattern for its Service Providers |
+
+### 6.3 Joint Architecture Questions
+
+| # | Question | Why It Matters |
+|---|----------|----------------|
+| 15 | Is the Kessel project open to DCM contributing Resource Type definitions and SpiceDB schema extensions to the upstream? | Reduces divergence risk; benefits broader community |
+| 16 | How does Kessel handle sovereign data โ€” data that must not cross jurisdictional boundaries? | Critical for DCM's sovereignty model |
+| 17 | What is the recommended pattern for bootstrapping the Kessel-DCM trust relationship? (mTLS? OIDC? Service account?) | Required for DCM's zero-trust model |
+| 18 | Does Kessel have a compatibility matrix for Red Hat platform versions (OpenShift, RHEL)? | DCM targets the same platforms |
+
+---
+
+## 7. Proposed Integration Architecture (Pending Kessel Alignment)
+
+This section describes the target architecture **conditional on positive answers to the questions in Section 6**. It should not be implemented until validated with the Kessel team.
+
+### 7.1 Kessel Relations as DCM Auth Provider
+
+```
+DCM Request Pipeline:
+  โ”‚
+  โ–ผ Auth Provider (Kessel Relations):
+  โ”‚   Check 1: identity verification via mTLS certificate
+  โ”‚   Check 2: CheckPermission(actor, operation, tenant/resource) via Kessel Relations gRPC
+  โ”‚            โ† returns allow/deny + zookie token
+  โ”‚
+  โ–ผ DCM Policy Engine (internal):
+  โ”‚   Check 3: Accreditation check (DCM Accreditation Registry)
+  โ”‚   Check 4: Data/Capability Matrix (DCM Governance Matrix)
+  โ”‚   Check 5: Sovereignty check (DCM Sovereignty Zone registry)
+  โ”‚
+  โ–ผ All five checks pass โ†’ request proceeds to layer assembly
+```
+
+**Impact on DCM architecture:**
+- Auth Provider registration: new `auth_mode: kessel_rebac` in doc 19
+- Cross-tenant authorization DCMGroup: writes to both DCM Group Registry AND Kessel Relations tuple store
+- RBAC evaluation: replaced by Kessel Relations CheckPermission call for checks 1โ€“2
+- Group membership sync: DCM Auth Providers (LDAP, OIDC) continue to manage authentication; group memberships are mirrored to Kessel Relations for use in permission evaluation
+
+### 7.2 Kessel Inventory as DCM Discovered State Store
+
+```
+Discovery Cycle:
+  โ”‚
+  โ–ผ Discovery Scheduler triggers provider discovery
+  โ”‚
+  โ–ผ Service Provider returns RealizedStatePayload stream
+  โ”‚   (current state in DCM Unified Data Model format)
+  โ”‚
+  โ–ผ Kessel Inventory Storage Provider:
+  โ”‚   Translates DCM format โ†’ Kessel Inventory Protobuf schema
+  โ”‚   Upserts to Kessel Inventory (replaces prior discovered state)
+  โ”‚
+  โ–ผ Drift Reconciliation Component (unchanged):
+  โ”‚   Queries Kessel Inventory for discovered state
+  โ”‚   Compares against DCM Realized State
+  โ”‚   Produces Drift Records (classification, severity, field detail)
+  โ”‚   Writes Drift Records to DCM Drift Record Store
+```
+
+**Impact on DCM architecture:**
+- Discovered State Store: implement as Storage Provider backed by Kessel Inventory
+- No changes to data model, drift detection logic, or Drift Reconciliation Component
+- Resource type mapping: DCM Resource Type Specs โ†’ Kessel Inventory resource types (new tooling required)
+
+---
+
+## 8. What Does Not Change Regardless of Integration
+
+The following DCM capabilities remain entirely in DCM regardless of how the Kessel integration develops:
+
+| Capability | Why it stays in DCM |
+|-----------|---------------------|
+| Intent State Store (GitOps) | GitOps semantics, PR workflow, immutability โ€” not in scope for Kessel |
+| Requested State Store (write-once) | Assembled payload with full provenance โ€” DCM-specific |
+| Realized State Store (append-only event stream) | Hash-chained, tamper-evident, field-level provenance โ€” DCM-specific |
+| Approval gate quorum tracking | State-tracking across time โ€” Kessel Relations answers membership, not quorum |
+| Five-check boundary model (checks 3โ€“5) | Accreditation, data classification, sovereignty โ€” DCM-specific |
+| Entity Relationship Graph | Operational relationships between resources โ€” not access control |
+| Field-level provenance | Source tracking per field โ€” not in scope for Kessel |
+| Drift detection logic and classification | DRC component โ€” Kessel Inventory is a data source, not a drift engine |
+| Audit trail (hash chain) | Tamper-evident audit โ€” DCM-specific requirement |
+| Resource lifecycle state machine | REQUESTED โ†’ OPERATIONAL โ†’ DECOMMISSIONED โ€” DCM-specific |
+| Policy Engine | GateKeeper, Transformation, Recovery, Orchestration Flow policies โ€” DCM-specific |
+| Authority Tier model | Approval routing โ€” DCM-specific governance model |
+
+---
+
+## 9. System Policies (Proposed โ€” Pending Validation)
+
+These policies should be reviewed and confirmed after Kessel team alignment:
+
+| Policy | Rule |
+|--------|------|
+| `KESSEL-001` | Kessel Relations, if registered as a DCM Auth Provider, handles authorization checks 1 and 2 of the five-check boundary model only. Checks 3โ€“5 remain in DCM's Policy Engine and cannot be delegated. |
+| `KESSEL-002` | DCM's entity relationship graph (operational relationships between infrastructure resources) must never be stored in Kessel Relations. Only access-control relationships (actorโ†’groupโ†’tenantโ†’resource permissions) are stored in Kessel Relations. |
+| `KESSEL-003` | Kessel Inventory, if registered as a DCM Storage Provider for Discovered State, holds only ephemeral current-state snapshots. Intent, Requested, and Realized State stores remain in DCM-managed Storage Providers. |
+| `KESSEL-004` | If Kessel Relations is unavailable, DCM enters safe-deny mode: no new requests are accepted. Fail-open behavior is not permitted under any profile. |
+| `KESSEL-005` | Zookie tokens from Kessel Relations CheckPermission responses must be threaded through the DCM request context to guarantee consistency across authorization checks within the same request. |
+| `KESSEL-006` | Cross-tenant authorization DCMGroups that are backed by Kessel Relations must be written atomically: the DCM Group Registry record and the Kessel Relations tuple must both succeed or both fail. Partial writes are treated as failures. |
+| `KESSEL-007` | DCM sovereign profile deployments require a locally-deployed Kessel instance. Cloud-hosted Kessel is not permitted for sovereign deployments. This requirement must be confirmed as feasible with the Kessel team. |
+
+---
+
+## 10. Open Items Before Integration Can Begin
+
+| # | Item | Owner | Blocking? |
+|---|------|-------|-----------|
+| 1 | Kessel team review of Section 6 questions | Kessel team | Yes |
+| 2 | Kessel Relations API stability confirmation | Kessel team | Yes |
+| 3 | Sovereign/air-gapped deployment validation | Kessel team | Yes (for sovereign profile) |
+| 4 | SpiceDB schema design review for DCM permission model | DCM + Kessel | Yes |
+| 5 | Kessel Inventory resource type extensibility confirmation | Kessel team | Yes (for inventory integration) |
+| 6 | HA/DR pattern review for production Kessel deployment | Kessel team | Yes |
+| 7 | DCM Auth Provider interface extension for `kessel_rebac` mode | DCM team | No (can design in parallel) |
+| 8 | DCM Storage Provider implementation for Kessel Inventory | DCM team | No (can design in parallel) |
+| 9 | Zookie lifecycle management design in DCM request pipeline | DCM team | No (can design in parallel) |
+| 10 | Resource type mapping: DCM Resource Type Specs โ†’ Kessel Inventory schema | DCM + Kessel | No (can design in parallel) |
+
+---
+
+*Document maintained by the DCM Project. For questions, contributions, or to schedule the Kessel alignment session see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/45-consistency-review.md b/content/docs/data-model/45-consistency-review.md
new file mode 100644
index 0000000..86894a9
--- /dev/null
+++ b/content/docs/data-model/45-consistency-review.md
@@ -0,0 +1,257 @@
+# DCM โ€” Consistency Review Findings
+
+**Document Status:** โœ… Complete
+**Document Type:** Review Record
+**Date:** 2026-03
+**Scope:** Full review of all 50 data model documents, 14 specifications, 11 schema files, and 4 OpenAPI specs for naming inconsistencies, field conflicts, API surface misalignments, and terminology drift.
+
+---
+
+## Summary
+
+| Category | Issues Found | Fixed | Notes |
+|----------|-------------|-------|-------|
+| API path format (AEP colon syntax) | 13 + 4 | โœ… All | admin-api-spec.md and consumer-api-spec.md had stale slash-verb paths |
+| Stale entity_type values | 2 | โœ… All | `allocated_resource` and `resource_entity` replaced with canonical values |
+| Stale threshold key format | 5 | โœ… All | `auto_approve_below`/`verified_above` in scoring doc examples replaced with named-tier format |
+| provider_id vs provider_uuid | 6 | โœ… All | Resolved to `provider_uuid` in all DCM API paths |
+| Four store naming | Variant | โš ๏ธ Noted | Multiple names in use โ€” canonical list documented below |
+| Operation (LRO) shape completeness | 1 | โœ… | Polling section added to consumer spec |
+| lifecycle_state casing | Mixed | โš ๏ธ Noted | UPPERCASE in YAML examples, lowercase in prose โ€” by design |
+| Resource Type naming | Mixed | โš ๏ธ Noted | `resource_type` (field) vs `resource_type_fqn` (format ref) โ€” not a conflict |
+| Provider type count references | 1 | โœ… Already correct | All references say "eleven" or "11" |
+| Anti-vocabulary: 'widget' | 0 | N/A | Clean โ€” eliminated in prior sessions |
+
+---
+
+## 1. API Path Format (AEP Colon Syntax)
+
+### 1.1 Findings
+
+The AEP colon syntax was applied to the OpenAPI YAML files in a prior session but **not propagated** to the narrative specification documents. This meant `dcm-admin-api-spec.md` and `dcm-consumer-api-spec.md` still used slash-verb paths while the normative OpenAPI specs used colon-verb paths.
+
+**Affected specs:**
+- `dcm-admin-api-spec.md` โ€” 13 stale slash-verb paths
+- `dcm-consumer-api-spec.md` โ€” 4 stale slash-verb paths (partially fixed but not fully)
+- `dcm-flow-gui-spec.md` โ€” 2 stale paths (`:promote`, `:run` fixed in prior session)
+
+### 1.2 Resolution
+
+Applied the full set of colon conversions to both narrative specs. Both specs now include AEP alignment notes referencing the normative OpenAPI YAML files. Specific conversions:
+
+**Consumer spec:** `:suspend`, `:resume`, `:rehydrate`, `:rotate`, `:extend-ttl`, `:transfer`, `:bulk-decommission`, `:acknowledge`, `:revert`, `:accept`, `:reject`, `:approve`, `:read-all`
+
+**Admin spec:** `:approve`, `:reject`, `:suspend`, `:reinstate`, `:revoke-sessions`, `:reset`, `:vote`, `:rotate-credential`, `:trigger`, `:rebuild`, `:accept-degradation`, `:activate`
+
+### 1.3 Canonical Rule
+
+> Custom method paths use colon syntax: `POST /resources/{name}:verb`.
+> This applies to all narrative specs, OpenAPI YAML, and code examples.
+> Sub-resources (`.../status`, `.../stream`, `.../pending` as filtered list) keep slash notation.
+
+---
+
+## 2. Entity Type Values
+
+### 2.1 Findings
+
+Two documents used stale or incorrect values for the `entity_type` field:
+
+| Document | Stale Value | Correct Value |
+|----------|-------------|---------------|
+| `09-entity-relationships.md` | `allocated_resource` | `infrastructure_resource` (with `ownership_model: allocation`) |
+| `18-webhooks-messaging.md` | `resource_entity` | `infrastructure_resource` |
+
+The `related_entity_type: internal` and `related_entity_type: external` in doc 09 are **not** entity_type values โ€” they are relationship scope descriptors and are correct as-is. They describe whether the related entity is managed within DCM or is an external reference.
+
+### 2.2 Resolution
+
+Both corrected in place. The `allocated_resource` correction includes a comment: `# ownership_model: allocation` to preserve the semantic intent of the original example.
+
+### 2.3 Canonical Values
+
+The three valid `entity_type` values are:
+- `infrastructure_resource` โ€” persistent physical or virtual resource
+- `composite_resource` โ€” Meta Provider-orchestrated aggregate
+- `process_resource` โ€” ephemeral execution (automation job, playbook)
+
+Pool resources and shared resources are `infrastructure_resource` entities with `ownership_model: whole_allocation` or `ownership_model: shareable`. There is no separate pool or shared entity type.
+
+---
+
+## 3. Scoring Model Threshold Keys
+
+### 3.1 Findings
+
+`29-scoring-model.md` contained 5 examples using the old fixed-column threshold key format (`auto_approve_below`, `verified_above`) in per-service-type override examples. The authority tier model (doc 32) replaced these with a named-tier list format in a prior session, but the scoring doc examples were not updated.
+
+**Old format (stale):**
+```yaml
+auto_approve_below: 20
+verified_above: 40
+```
+
+**Current format:**
+```yaml
+thresholds:
+  - { tier: auto,     max_score: 20 }
+  - { tier: verified, max_score: 40 }
+```
+
+### 3.2 Resolution
+
+Replaced stale threshold keys with comments pointing to the named-tier format. The SMX-008 policy row was verified as already using current terminology.
+
+---
+
+## 4. provider_id vs provider_uuid
+
+### 4.1 Findings
+
+The operator interface spec (`dcm-operator-interface-spec.md`) and provider callback auth doc (`43-provider-callback-auth.md`) used `provider_id` in some places where `provider_uuid` is the correct DCM term.
+
+The distinction is important:
+- **`provider_uuid`** โ€” DCM-assigned UUID for the provider record. Used in all DCM API paths and payloads.
+- **`resource_id`** โ€” Operator-assigned identifier for a specific resource instance. Used in operator-to-DCM callbacks to identify the resource being reported on.
+
+### 4.2 Resolution
+
+DCM API endpoint paths updated to use `provider_uuid` consistently:
+- `POST /api/v1/providers/{provider_uuid}/capacity`
+- Registration response field `provider_id` โ†’ `provider_uuid`
+
+`resource_id` in callback APIs is **intentionally different** from `entity_uuid`. It is the operator's own identifier for the resource (returned in the `CreateResponse`). DCM maps it to `entity_uuid` internally. This distinction is correct and remains unchanged.
+
+---
+
+## 5. Four Store Naming โ€” Canonical Reference
+
+Multiple naming variants found across documents. The canonical names are:
+
+| Store | Canonical Name | Also Used (acceptable) | Do Not Use |
+|-------|---------------|----------------------|------------|
+| Intent State storage | **Intent Store** | GitOps Store (when emphasizing the implementation) | Intent State Store |
+| Requested State storage | **Requested Store** | โ€” | Requested State Store |
+| Realized State storage | **Realized Store** | Realized State Store | Realization Store |
+| Discovered State storage | **Discovered Store** | โ€” | Discovered State Store |
+
+No bulk renaming was performed โ€” both "Intent Store" and "GitOps Store" are used accurately in different contexts (the former emphasizes the state model, the latter the implementation). The variation is acceptable context-dependent usage, not an error.
+
+---
+
+## 6. Operation (LRO) Shape
+
+### 6.1 Finding
+
+The `consumer-api-spec.md` applied LRO `Operation` responses to async endpoints but did not include a dedicated section explaining the `GET /api/v1/operations/{uuid}` polling endpoint or the complete Operation shape.
+
+### 6.2 Resolution
+
+Added an "Operations โ€” Polling Long-Running Requests" section to `consumer-api-spec.md` covering:
+- The polling endpoint shape with in-progress, success, and failure states
+- Polling backoff guidance (1s โ†’ 2s โ†’ 5s โ†’ 10s โ†’ 30s)
+- Alternatives: webhook subscription (`request.progress_updated`), SSE stream
+
+---
+
+## 7. Lifecycle State Casing
+
+### 7.1 Finding
+
+Lifecycle state values appear in UPPERCASE in YAML examples (e.g., `lifecycle_state: OPERATIONAL`) and in lowercase in prose text (e.g., "the resource enters the operational state"). This is **by design**, not an inconsistency:
+
+- UPPERCASE in YAML/JSON โ€” machine-readable, matches enum values in schemas
+- Lowercase in prose โ€” natural language, matches how engineers write documentation
+
+No changes made.
+
+---
+
+## 8. resource_type vs resource_type_fqn
+
+### 8.1 Finding
+
+The JSON entity schema uses `$ref: resource_type_fqn` as a type reference (meaning the field value must be a Fully Qualified Name like `Compute.VirtualMachine`). Narrative YAML examples use `resource_type:` as the field name. This appears inconsistent but is not.
+
+`resource_type` is the **field name**. The value it holds must conform to the **FQN format** (`Category.TypeName`). The schema type reference is just documenting the format constraint.
+
+No changes made.
+
+---
+
+## 9. What Remains Acceptable (Not Fixed)
+
+These were identified but are not bugs โ€” they are intentional or context-appropriate variation:
+
+| Item | Why It's Acceptable |
+|------|---------------------|
+| `Service Provider` (title case) vs `service_provider` (snake_case) | Title case in prose, snake_case in code/YAML โ€” correct by context |
+| `Auth Provider` vs `auth_provider` | Same as above |
+| `Resource Type Spec` vs `Resource Type Specification` | Shortened form acceptable in prose; full form in formal definitions |
+| `Realized State` vs `realized state` | Title case for the formal concept, lowercase in general prose |
+| `related_entity_type: internal/external` in doc 09 | Not entity_type values โ€” relationship scope descriptors, correct as-is |
+| Provider type count varies ("nine", "eleven", "11") | All refer to the same 11 types; "nine" may be a historical reference pre-two additions |
+
+---
+
+## 10. Remaining Items Requiring Action by Implementers
+
+These are not documentation issues but implementation decisions that need to be made explicit when building DCM:
+
+| Item | Decision Needed |
+|------|----------------|
+| `resource_type` field in API payloads accepts short names (`Compute.VirtualMachine`) or requires registry UUID? | Current spec allows both `resource_type` (FQN string) and `resource_type_uuid` โ€” need to decide if UUID is required at dispatch or optional |
+| Operation polling endpoint `GET /api/v1/operations/{uuid}` โ€” is operation status part of request status, or a separate Operation resource? | Currently: request status via `GET /api/v1/requests/{uuid}/status`; Operation via `GET operation.name`. Need to clarify if these are the same or different responses |
+| `provider_id` in operator-assigned resource IDs vs DCM `provider_uuid` โ€” implementers must ensure they're not conflated at the API Gateway level | Code-level concern โ€” the API Gateway maps `resource_id` (operator-assigned) to `entity_uuid` (DCM-assigned) |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+---
+
+## Second Review Pass โ€” 2026-03
+
+A second comprehensive scan identified additional issues and confirmed no new architectural conflicts.
+
+### Additional Fixes Applied
+
+| Fix | Files Affected | Description |
+|-----|---------------|-------------|
+| Stale slash-verb paths | 14 files | Data model docs 00, 23, 24, 32, 35, 40, 42, 43 and specs dcm-admin-gui, dcm-consumer-gui, dcm-examples, dcm-registration, dcm-admin-api | All remaining colon-less custom method paths converted |
+| DELETE /api/v1/auth/session | dcm-consumer-api.yaml | Missing endpoint added to OpenAPI YAML (was in narrative spec only) |
+| GET /api/v1/operations/{uuid} | dcm-consumer-api.yaml | Operation polling endpoint added to OpenAPI YAML |
+| Missing event types | 33-event-catalog.md | 6 event types added: entity.deleted, entity.state_transition, group.deleted, group.member_added, group.member_removed, authorization.granted |
+| Prompt section numbering | DCM-AI-PROMPT.md | Duplicate section 78 resolved; sections 0โ€“80 now sequential with no duplicates |
+
+### Confirmed Clean (No Issues)
+
+- **OpenAPI schemas**: All 4 specs (consumer/admin/operator/callback) โ€” 0 schema conflicts
+- **Lifecycle state casing**: UPPERCASE in YAML/JSON examples, lowercase in prose โ€” intentional
+- **`Ingress API` usage**: Two occurrences are contextually correct (infrastructure layer explanation)
+- **`resource_type` vs `resource_type_fqn`**: Field name vs format description โ€” not a conflict
+- **`name:` vs `display_name:`**: Context-appropriate โ€” `name:` is a property name, `display_name:` is a human label
+- **Policy type casing**: Uppercase in section headers, lowercase in code/YAML โ€” by convention
+- **Provider type count**: All references to "eleven" or "11" provider types โ€” consistent
+- **Realized State write authority**: Consistent across all docs โ€” providers never write directly; DCM API Gateway is sole writer
+
+### Remaining Acceptable Variation
+
+These are not errors โ€” they are deliberate context-dependent usage:
+
+| Pattern | Both Forms Correct | Reason |
+|---------|--------------------|--------|
+| `Service Provider` / `service_provider` | Title case in prose, snake_case in code | Convention |
+| `Auth Provider` / `auth_provider` | Same | Convention |
+| `Policy Engine` / `policy_engine` | Same | Convention |
+| Intent Store / GitOps Store | Both describe the same store | Different emphasis (state model vs implementation) |
+| `entity.created` / `resource.provisioned` | Different semantic levels | Entity creation vs provisioning completion are distinct events |
+
+### Implementation Decisions Still Outstanding
+
+These three items were identified in the prior review and remain open โ€” they require implementation choices, not documentation changes:
+
+1. **`resource_type` at dispatch**: Accept FQN string (`Compute.VirtualMachine`) or require UUID? Current specs allow both; implementation must pick one canonical form.
+2. **Operation polling endpoint**: Is `GET /api/v1/operations/{uuid}` the same resource as `GET /api/v1/requests/{uuid}/status`, or a separate resource? Added to OpenAPI YAML as separate endpoint.
+3. **`resource_id` โ†’ `entity_uuid` mapping**: The API Gateway must map operator-assigned `resource_id` to DCM `entity_uuid` at the callback boundary. This is a code-level concern with no documentation gap.
+
diff --git a/content/docs/data-model/46-workload-analysis.md b/content/docs/data-model/46-workload-analysis.md
new file mode 100644
index 0000000..440fd90
--- /dev/null
+++ b/content/docs/data-model/46-workload-analysis.md
@@ -0,0 +1,253 @@
+# DCM โ€” Workload Analysis
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Specification in Progress
+**Document Type:** Capability Specification
+**Related Documents:** [Ingestion Model](13-ingestion-model.md) | [Information Providers](10-information-providers.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Discovery and Drift](25-control-plane-components.md) | [Kubernetes Compatibility](../specifications/11-kubernetes-compatibility.md)
+
+> **AEP Alignment:** API endpoints follow [AEP](https://aep.dev) conventions.
+> Workload analysis results are delivered as Information Provider payloads
+> and stored as `process_resource_entity` instances of type `Analysis.WorkloadProfile`.
+
+---
+
+## 1. Purpose
+
+Workload Analysis is the DCM capability that actively classifies discovered resources
+by their operational characteristics โ€” what they are, how they behave, what lifecycle
+model should apply to them, and which DCM Resource Type they best map to.
+
+It answers questions that passive discovery cannot:
+- *"This VM was discovered โ€” is it a web server, a database, a batch processor?"*
+- *"This workload can be migrated to containers โ€” what is its archetype?"*
+- *"This resource has no DCM UUID โ€” what is the minimum viable Resource Type we can
+   assign it for lifecycle management?"*
+
+Workload Analysis is the bridge between **Discovered State** (what exists) and
+**Intent State** (what should be managed). Without it, brownfield ingestion stalls
+at the enrichment phase because the Tenant, Resource Type, and lifecycle ownership
+cannot be automatically determined.
+
+---
+
+## 2. Relationship to Existing Capabilities
+
+```
+Discovery (DRC)          Workload Analysis       Ingestion (doc 13)
+      โ”‚                        โ”‚                        โ”‚
+      โ–ผ                        โ–ผ                        โ–ผ
+Discovered State     WorkloadProfile entity    INGESTED โ†’ ENRICHING
+(what physically      (what it IS and what      โ†’ PROMOTED โ†’ OPERATIONAL
+  exists today)        lifecycle applies)
+```
+
+**Workload Analysis is a DCM-managed process resource** (`Analysis.WorkloadProfile`)
+that fires as part of the brownfield ingestion pipeline. It is triggered by the
+Discovery Scheduler when a new resource enters Discovered State without a matching
+DCM entity in Realized State.
+
+It can also be triggered manually by a platform admin to re-classify a resource
+whose operational profile has changed (e.g., a VM that was a web server and is
+now a database).
+
+---
+
+## 3. WorkloadProfile Entity
+
+Workload Analysis produces a `process_resource_entity` of type `Analysis.WorkloadProfile`:
+
+```yaml
+workload_profile_entity:
+  entity_uuid: 
+  entity_type: process_resource
+  resource_type: Analysis.WorkloadProfile
+  lifecycle_state: OPERATIONAL       # while active; DECOMMISSIONED when superseded
+  
+  # Linked to the resource being analyzed
+  subject_entity_uuid:         # the VM, container, or other resource
+  subject_discovered_state_uuid: 
+  
+  classification:
+    resource_type_match:             # best-fit DCM Resource Type
+      primary:    Compute.VirtualMachine
+      confidence: high               # high | medium | low | undetermined
+      alternatives:
+        - resource_type: Platform.Container
+          confidence: medium
+          rationale: "Workload is containerizable per MTA assessment"
+    
+    workload_archetype:              # operational classification
+      type: web_server | database | batch_processor | message_broker |
+            api_gateway | cache | storage | monitoring | unknown
+      confidence: high
+      signals: [port_scan, process_list, resource_utilization_pattern]
+    
+    migration_readiness:             # if MTA integration is active
+      containerization_score: 7     # 1-10
+      blockers: []
+      suggested_target: Platform.KubernetesDeployment
+      mta_report_ref:           # link to MTA HTML report if available
+    
+    lifecycle_recommendation:
+      dcm_lifecycle_model: standard | stateful | ephemeral | infrastructure
+      rehydration_eligible: true
+      notes: "Application data on /data partition; OS on /; static replace eligible"
+    
+  analysis_metadata:
+    analyzed_at: 
+    analysis_version: "1.0.0"        # versioned analysis ruleset
+    information_providers_used:
+      - provider_uuid: 
+        provider_type: information_provider
+        data_types_used: [port_scan, process_list, os_metadata]
+    analyst_actor_uuid:        # null if automated; actor UUID if manual review
+```
+
+---
+
+## 4. Analysis Pipeline
+
+Workload Analysis is an Orchestration Flow Policy that fires when a discovered
+resource enters the enrichment phase:
+
+```
+discovery.new_entity_found
+  โ”‚
+  โ–ผ Orchestration Step 1: Create WorkloadProfile entity (INGESTED state)
+  โ”‚   Linked to discovered resource via 'operational' relationship
+  โ”‚
+  โ–ผ Orchestration Step 2: Gather signals from Information Providers
+  โ”‚   Port scan (network topology)
+  โ”‚   Process list (running services)
+  โ”‚   OS metadata (version, packages, mount points)
+  โ”‚   Resource utilization patterns (CPU/memory/disk I/O profile)
+  โ”‚   MTA assessment (if MTA Information Provider registered)
+  โ”‚
+  โ–ผ Orchestration Step 3: Apply classification ruleset (Policy Engine)
+  โ”‚   Transformation Policy: compute workload_archetype from signals
+  โ”‚   Transformation Policy: compute resource_type_match from archetype
+  โ”‚   Transformation Policy: compute migration_readiness from MTA signals
+  โ”‚   GateKeeper Policy: flag if confidence < medium for manual review
+  โ”‚
+  โ–ผ Orchestration Step 4: Write WorkloadProfile to Realized State
+  โ”‚   WorkloadProfile entity โ†’ OPERATIONAL
+  โ”‚
+  โ–ผ Orchestration Step 5: Trigger ingestion enrichment
+      WorkloadProfile classification informs:
+        - Tenant auto-assignment (if auto-assignment rules match)
+        - Resource Type assignment for the ingestion record
+        - Lifecycle model selection
+```
+
+---
+
+## 5. MTA (Migration Toolkit for Applications) Integration
+
+When the MTA Information Provider is registered, Workload Analysis invokes it
+as part of Step 2 above. MTA provides workload archetype classification and
+containerization readiness scores for discovered workloads.
+
+```yaml
+mta_information_provider_registration:
+  provider_type: information_provider
+  information_type: workload_analysis
+  display_name: "MTA โ€” Migration Toolkit for Applications"
+  endpoint: https://mta.internal:8080/api/v1
+  
+  capabilities:
+    workload_archetypes:
+      - web_server
+      - database
+      - batch_processor
+      - message_broker
+    provides_containerization_score: true
+    provides_migration_blockers: true
+    provides_target_recommendations: true
+  
+  query_interface:
+    # MTA receives discovered state payload and returns analysis
+    input: discovered_state_payload
+    output: mta_workload_report
+    async: true
+    callback_supported: true
+```
+
+The MTA integration is the primary implementation path for Workload Analysis in
+Red Hat environments. In non-MTA environments, a custom Information Provider
+implementing the same `workload_analysis` information type can be registered.
+
+---
+
+## 6. Consumer API โ€” Workload Analysis Endpoints
+
+```
+# Get the WorkloadProfile for a specific resource
+GET /api/v1/resources/{entity_uuid}/workload-profile
+
+Response 200:
+{
+  "workload_profile_uuid": "",
+  "subject_entity_uuid": "",
+  "classification": {
+    "resource_type_match": {
+      "primary": "Compute.VirtualMachine",
+      "confidence": "high"
+    },
+    "workload_archetype": {
+      "type": "web_server",
+      "confidence": "high"
+    },
+    "migration_readiness": {
+      "containerization_score": 7,
+      "blockers": [],
+      "suggested_target": "Platform.KubernetesDeployment"
+    },
+    "lifecycle_recommendation": {
+      "dcm_lifecycle_model": "standard",
+      "rehydration_eligible": true
+    }
+  },
+  "analyzed_at": ""
+}
+
+# Trigger a re-analysis of a resource
+POST /api/v1/resources/{entity_uuid}/workload-profile:analyze
+
+Request body:
+{
+  "reason": "Role change โ€” web server migrated to database role",
+  "include_mta": true
+}
+
+Response 200 โ€” returns Operation:
+{
+  "name": "/api/v1/operations/{request_uuid}",
+  "done": false,
+  "metadata": { "stage": "ANALYSIS_INITIATED", "resource_uuid": "{entity_uuid}" }
+}
+
+# List all resources with a given workload archetype (platform admin)
+GET /api/v1/admin/workload-analysis?archetype=web_server&confidence=high
+
+Response 200:
+{
+  "items": [ { "entity_uuid": "...", "resource_type": "...", "archetype": "..." } ],
+  "next_page_token": "..."
+}
+```
+
+---
+
+## 7. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `WLA-001` | Workload Analysis fires automatically for every entity entering Discovered State without a matching Realized State record. It is not optional โ€” it is part of the brownfield ingestion pipeline. |
+| `WLA-002` | WorkloadProfile entities are versioned. When re-analysis produces a different classification, the old WorkloadProfile enters DECOMMISSIONED state and a new one is created. The chain is preserved for audit. |
+| `WLA-003` | If classification confidence is `low` or `undetermined`, the WorkloadProfile GateKeeper policy fires and the entity is routed to manual review before ingestion can proceed to PROMOTED. |
+| `WLA-004` | The MTA Information Provider is the reference implementation for workload_analysis information type in Red Hat environments. Custom implementations must provide the same output schema. |
+| `WLA-005` | Workload Analysis results are stored in Realized State as `process_resource_entity` instances. They are immutable once written โ€” re-analysis creates a new entity, not an update. |
+| `WLA-006` | Migration readiness scores and archetype classifications are advisory โ€” they inform human decision-making and Orchestration Flow Policies but do not automatically trigger migrations. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/47-accreditation-monitor.md b/content/docs/data-model/47-accreditation-monitor.md
new file mode 100644
index 0000000..cee708e
--- /dev/null
+++ b/content/docs/data-model/47-accreditation-monitor.md
@@ -0,0 +1,496 @@
+# DCM Data Model โ€” Accreditation Monitor
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Capability Specification
+**Related Documents:** [Accreditation and Authorization Matrix](26-accreditation-and-authorization-matrix.md) | [Information Providers](10-information-providers.md) | [Advanced Information Providers](21-information-providers-advanced.md) | [Standards Catalog](40-standards-catalog.md) | [Scoring Model](29-scoring-model.md) | [Event Catalog](33-event-catalog.md) | [Governance Matrix](27-governance-matrix.md)
+
+> **AEP Alignment:** API endpoints follow [AEP](https://aep.dev) conventions.
+> Custom methods use colon syntax. Async operations return `Operation` resources.
+> See `schemas/openapi/dcm-admin-api.yaml` for the normative admin API specification.
+
+---
+
+## 1. Purpose
+
+The Accreditation Monitor is a DCM **Information Provider** that continuously verifies the status of registered accreditations against authoritative external sources โ€” compliance registries, certificate authority portals, and contract management systems. It answers the question DCM's existing accreditation lifecycle model cannot answer alone:
+
+> *"Is this accreditation still valid according to the issuing authority โ€” not just according to the expiry date we were told?"*
+
+An accreditation can become invalid before its declared `valid_until` date: FedRAMP authorizations can be revoked mid-cycle, ISO 27001 certificates can be suspended by the certification body, CMMC certifications can be downgraded. Without external monitoring, DCM would continue treating a provider as accredited until the date it was told to check โ€” which may be 12 months away.
+
+The Accreditation Monitor closes this gap by polling authoritative external registries on a configurable schedule and surfacing discrepancies to DCM's existing accreditation lifecycle model.
+
+---
+
+## 2. What Can Be Automated โ€” Framework by Framework
+
+Automation depth varies significantly by framework. The Accreditation Monitor implements four verification tiers:
+
+### Tier 1 โ€” Full External Verification (public registry API)
+
+DCM can query the authoritative registry directly. Status changes are detected without any manual intervention.
+
+| Framework | Authoritative Source | What Is Verified |
+|-----------|---------------------|-----------------|
+| **FedRAMP Moderate/High/LI-SaaS** | [marketplace.fedramp.gov/api](https://marketplace.fedramp.gov) | Authorization status (Authorized / In Process / Revoked), impact level, authorization date, annual assessment currency |
+| **StateRAMP** | [stateramp.org](https://stateramp.org) | Authorization status, status changes |
+| **CMMC 2.0** | [cyberab.org/catalog](https://cyberab.org/catalog) | Certification level (1/2/3), certification status, expiry date |
+| **IAF / ISO 27001** | [iaf.nu CertSearch](https://iaf.nu) | Certificate status (valid/suspended/withdrawn), expiry date, scope, issuing CB identity |
+
+**Required field at registration:** `external_registry_id` โ€” the provider's identifier in the external registry (FedRAMP ID, CMMC certification number, ISO certificate number).
+
+### Tier 2 โ€” Document Currency Verification
+
+DCM cannot query a central registry, but it can verify that the stored evidence document is current relative to the framework's required renewal cycle.
+
+| Framework | Verification Method | Currency Check |
+|-----------|---------------------|---------------|
+| **SOC 2 Type I / Type II** | Document metadata extraction from `audit_report_ref` | Report period end date must be within 12 months (configurable); examines PDF metadata or report header |
+| **PCI DSS** | QSA authorization check + AoC date | Verifies the issuing QSA firm is currently listed as authorized at PCI SSC; verifies stored AoC covers current period |
+| **ISO 27001** (when CB portal is unavailable) | Certificate document date + P3Y renewal cycle | Falls back to document-based verification if IAF CertSearch is unreachable |
+
+### Tier 3 โ€” Contract Lifecycle Integration
+
+No external registry exists. Verification is driven by events from contract management systems.
+
+| Framework | Integration Mechanism | What Triggers Verification |
+|-----------|----------------------|--------------------------|
+| **HIPAA BAA** | Contract management webhook (DocuSign, Ironclad, Agiloft, custom) | BAA signed, amended, terminated, or approaching renewal |
+| **DoD IL2/IL4/IL5/IL6** | Manual submission + webhook on DISA action letter | Authorization letter received, amended, or revoked |
+| **Custom / Sovereign** | Manual submission + optional webhook | Platform admin triggers; or custom webhook from issuing authority |
+
+### Tier 4 โ€” Expiry-Only Monitoring (no external verification available)
+
+No external API, no document currency check, no contract integration. DCM monitors only the declared `valid_until` date.
+
+| Framework | What Is Monitored |
+|-----------|-----------------|
+| **HIPAA BAA** (if no contract system) | Declared BAA expiry date |
+| **Internal / first_party accreditations** | Declared validity period |
+| **Self-declared** | Declared validity period |
+
+---
+
+## 3. Accreditation Record Additions
+
+The existing accreditation record (doc 26 Section 3.3) is extended with three new fields to support automated monitoring:
+
+```yaml
+accreditation:
+  # ... existing fields unchanged ...
+
+  # NEW โ€” Verification model
+  verification:
+    tier: external_registry | document_currency | contract_webhook | expiry_only
+    # โ”€โ”€ Tier 1 specific โ”€โ”€
+    registry_api:
+      provider: fedramp | stateramp | cmmc_ab | iaf_certsearch | custom
+      lookup_key:     # e.g., FedRAMP ID "FR2024-0042"
+      poll_interval: P1D                          # how often to check
+      last_checked_at: 
+      last_result: confirmed_active | status_changed | registry_unavailable | not_found
+    
+    # โ”€โ”€ Tier 2 specific โ”€โ”€
+    document_check:
+      document_url: 
+      max_age: P365D                              # how old the document can be
+      date_extraction_method: pdf_metadata | report_header_parse | manual
+      last_checked_at: 
+      last_document_date: 
+    
+    # โ”€โ”€ Tier 3 specific โ”€โ”€
+    contract_webhook:
+      contract_system: docusign | ironclad | agiloft | custom
+      contract_id: 
+      webhook_url: 
+      last_event_at: 
+    
+    # โ”€โ”€ Shared โ”€โ”€
+    stale_after: P7D                              # how long before last_checked_at = stale
+    stale_action: warn | suspend | escalate       # what to do when stale
+    verification_failure_count: 0                 # consecutive failures; triggers escalation
+    verification_failure_threshold: 3
+```
+
+The `last_verified_at` field on the existing accreditation record (doc 26) is updated by the Accreditation Monitor on each successful verification. It remains the canonical "last confirmed active" timestamp used by the Governance Matrix and Scoring Model.
+
+---
+
+## 4. Accreditation Monitor as an Information Provider
+
+The Accreditation Monitor registers with DCM as an Information Provider with `information_type: accreditation_verification`. It is a separately deployable component โ€” it does not require changes to the DCM control plane and can be upgraded independently.
+
+```yaml
+accreditation_monitor_registration:
+  provider_type: information_provider
+  information_type: accreditation_verification
+  display_name: "DCM Accreditation Monitor"
+  version: "1.0.0"
+  
+  # What it monitors
+  supported_tiers:
+    - external_registry
+    - document_currency
+    - contract_webhook
+    - expiry_only
+  
+  supported_registries:
+    - fedramp
+    - stateramp
+    - cmmc_ab
+    - iaf_certsearch
+    - pci_ssc_qsa      # QSA verification only
+  
+  supported_contract_systems:
+    - docusign
+    - ironclad
+    - custom_webhook
+  
+  # How it communicates results back to DCM
+  push_events: true
+  event_types:
+    - accreditation.verified           # periodic confirmation: still active
+    - accreditation.status_changed     # external registry shows different status
+    - accreditation.registry_mismatch  # external status != DCM recorded status
+    - accreditation.verification_stale # last_checked_at exceeds stale_after threshold
+    - accreditation.expiry_approaching # approaching valid_until (supplement to existing)
+    - accreditation.document_expired   # document_check: document older than max_age
+    - accreditation.contract_event     # contract_webhook: BAA signed/amended/terminated
+
+  health_check:
+    endpoint: /health
+    interval: PT5M
+```
+
+---
+
+## 5. Verification Flows
+
+### 5.1 Tier 1 โ€” FedRAMP External Registry Verification
+
+```
+Accreditation Monitor poll cycle (default: P1D):
+  โ”‚
+  โ–ผ For each active Tier 1 accreditation:
+  โ”‚   Load accreditation record
+  โ”‚   Extract: framework, external_registry_id, last known status
+  โ”‚
+  โ–ผ Query external registry:
+  โ”‚   FedRAMP: GET marketplace.fedramp.gov/api/products?id={external_registry_id}
+  โ”‚   CMMC:    GET cyberab.org/api/certifications?cert_number={external_registry_id}
+  โ”‚   IAF:     GET iaf.nu/certsearch?cert={external_registry_id}
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: status = Authorized, impact_level = High
+  โ”‚     Matches DCM record โ†’ no action
+  โ”‚     Update: last_checked_at, last_result: confirmed_active
+  โ”‚     Fire: accreditation.verified (urgency: low)
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: status = In Process (was Authorized)
+  โ”‚     Status changed โ†’ MISMATCH
+  โ”‚     Fire: accreditation.status_changed (urgency: high)
+  โ”‚     Payload: {from: authorized, to: in_process, external_source: fedramp_marketplace}
+  โ”‚     DCM action: accreditation status โ†’ pending_review
+  โ”‚     Platform Admin notified โ€” human must review and decide: suspend or retain
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: status = Revoked
+  โ”‚     Fire: accreditation.status_changed (urgency: critical)
+  โ”‚     DCM action: accreditation status โ†’ revoked immediately
+  โ”‚     Accreditation Gap triggered for all affected providers
+  โ”‚     Recovery Policy evaluated
+  โ”‚
+  โ”œโ”€โ”€ Registry returns: 404 / not_found
+  โ”‚     May indicate ID change or deregistration
+  โ”‚     Fire: accreditation.registry_mismatch (urgency: high)
+  โ”‚     Increment verification_failure_count
+  โ”‚     Platform Admin notified to verify external_registry_id is correct
+  โ”‚
+  โ””โ”€โ”€ Registry unreachable (timeout, 5xx)
+        Update: last_result: registry_unavailable
+        Increment verification_failure_count
+        If count >= verification_failure_threshold:
+          Fire: accreditation.verification_stale (urgency: medium)
+        Do NOT change accreditation status on registry failure alone
+        (conservative: prefer false negative over false positive revocation)
+```
+
+### 5.2 Tier 2 โ€” Document Currency Verification (SOC 2, PCI DSS AoC)
+
+```
+Verification cycle (default: P7D):
+  โ”‚
+  โ–ผ Fetch document from certificate_ref or audit_report_ref URL
+  โ”‚
+  โ”œโ”€โ”€ PDF: extract creation_date from PDF metadata
+  โ”‚         or parse report header for "Report Date: YYYY-MM-DD"
+  โ”‚
+  โ”œโ”€โ”€ HTML report: parse structured date field
+  โ”‚
+  โ””โ”€โ”€ Fallback: flag for manual review if date cannot be extracted
+  
+  โ–ผ Compare document date to max_age threshold (default P365D):
+  โ”‚
+  โ”œโ”€โ”€ Within threshold โ†’ update last_checked_at, last_document_date
+  โ”‚                      Fire: accreditation.verified (urgency: low)
+  โ”‚
+  โ””โ”€โ”€ Beyond threshold โ†’ Fire: accreditation.document_expired (urgency: high)
+                          Platform Admin notified: new report needed
+                          Accreditation status โ†’ pending_renewal
+```
+
+### 5.3 Tier 3 โ€” Contract Webhook (HIPAA BAA, DoD IL)
+
+```
+Contract management system fires webhook to DCM:
+  POST /api/v1/admin/accreditations/{uuid}/contract-event
+  
+  Payload:
+  {
+    "contract_id": "",
+    "event_type": "signed | amended | terminated | renewal_due | renewed",
+    "effective_date": "",
+    "details": { ... contract-system-specific fields ... }
+  }
+  
+  DCM processes:
+  โ”œโ”€โ”€ signed   โ†’ accreditation status: active (if was pending)
+  โ”œโ”€โ”€ amended  โ†’ accreditation status: pending_review; Platform Admin notified
+  โ”œโ”€โ”€ terminated โ†’ accreditation status: revoked; Accreditation Gap triggered
+  โ”œโ”€โ”€ renewal_due โ†’ notification to Compliance Team (urgency: medium)
+  โ””โ”€โ”€ renewed  โ†’ accreditation status: active; valid_until updated; last_verified_at updated
+```
+
+### 5.4 Stale Verification Handling
+
+Regardless of tier, when `last_checked_at` is older than `stale_after`:
+
+```
+stale_action: warn     โ†’ Fire: accreditation.verification_stale (urgency: low)
+                          No change to accreditation status
+                          
+stale_action: suspend  โ†’ Fire: accreditation.verification_stale (urgency: high)
+                          Accreditation status โ†’ suspended
+                          Accreditation Gap triggered (gap_type: suspended)
+                          Platform Admin must manually verify and reactivate
+                          
+stale_action: escalate โ†’ Fire: accreditation.verification_stale (urgency: critical)
+                          Escalation chain notified (Compliance Team + Platform Admin)
+                          No automatic status change
+                          If not resolved within escalation_window: โ†’ suspend
+```
+
+`stale_action` defaults by profile: `warn` for dev/standard, `suspend` for prod, `escalate` for fsi/sovereign.
+
+---
+
+## 6. New Event Types (additions to doc 33)
+
+These events are added to the event catalog as domain `accreditation.*`:
+
+| Event Type | Urgency | Description | Key Payload Fields |
+|-----------|---------|-------------|-------------------|
+| `accreditation.verified` | low | Periodic external confirmation โ€” accreditation still active | accreditation_uuid, framework, registry, checked_at |
+| `accreditation.status_changed` | high or critical | External registry shows a different status than DCM records | accreditation_uuid, framework, from_status, to_status, external_source |
+| `accreditation.registry_mismatch` | high | External registry cannot find the accreditation by its external_registry_id | accreditation_uuid, external_registry_id, registry, failure_detail |
+| `accreditation.verification_stale` | varies | last_checked_at exceeds stale_after threshold | accreditation_uuid, last_checked_at, stale_after, stale_action_taken |
+| `accreditation.document_expired` | high | Evidence document older than max_age threshold | accreditation_uuid, framework, document_url, document_date, max_age |
+| `accreditation.contract_event` | varies | Contract management webhook received | accreditation_uuid, contract_event_type, contract_id, effective_date |
+| `accreditation.expiry_approaching` | medium | Approaching valid_until within renewal_warning_before (supplement to existing TTL-based check) | accreditation_uuid, valid_until, days_remaining |
+
+---
+
+## 7. Accreditation Record Additions to doc 26
+
+The following fields are added to the accreditation record structure in doc 26 Section 3.3.
+These are non-breaking additions โ€” existing records without these fields default to `tier: expiry_only`.
+
+```yaml
+# Additions to existing accreditation record (doc 26 Section 3.3)
+
+  verification:
+    tier: external_registry | document_currency | contract_webhook | expiry_only
+    
+    # For external_registry tier:
+    registry_api:
+      provider: fedramp | stateramp | cmmc_ab | iaf_certsearch | pci_ssc_qsa | custom
+      lookup_key:            # value to use as query key in registry
+      poll_interval: P1D             # ISO 8601 duration; how often Monitor checks
+      last_checked_at: 
+      last_result: confirmed_active | status_changed | registry_unavailable | not_found | pending
+    
+    # For document_currency tier:
+    document_check:
+      document_url:             # usually same as certificate_ref or audit_report_ref
+      max_age: P365D                 # maximum acceptable document age
+      date_extraction_method: pdf_metadata | report_header_parse | manual
+      last_checked_at: 
+      last_document_date: 
+    
+    # For contract_webhook tier:
+    contract_webhook:
+      contract_system: docusign | ironclad | agiloft | custom
+      contract_id:           # ID in the contract management system
+      webhook_configured: true | false
+      last_event_at: 
+    
+    # Shared across all tiers:
+    stale_after: P7D                 # max acceptable gap between verifications
+    stale_action: warn | suspend | escalate
+    verification_failure_count: 0
+    verification_failure_threshold: 3
+```
+
+---
+
+## 8. Impact on Scoring Model (doc 29)
+
+The Scoring Model's Signal 5 (Provider Accreditation Richness, doc 29 Section 4.5) is
+enhanced with a verification currency dimension. An accreditation that has been externally
+verified recently is worth more than one that has only ever been manually submitted.
+
+```yaml
+# Addition to accreditation_weights in doc 29:
+verification_multipliers:
+  # Applied to each accreditation's weight based on verification currency
+  external_registry_verified_within_P1D:  1.0    # full weight
+  external_registry_verified_within_P7D:  0.9    # slight discount
+  document_verified_within_P30D:          0.85
+  contract_webhook_active:                0.9
+  expiry_only_no_external_check:          0.7    # meaningful discount
+  verification_stale:                     0.4    # significant discount
+  verification_failed_threshold_reached:  0.1    # near-zero weight
+```
+
+This means a provider with a FedRAMP High accreditation that was externally verified
+yesterday scores higher in placement tie-breaking than a provider with the same
+accreditation whose verification check has been stale for 30 days.
+
+---
+
+## 9. Admin API Additions
+
+```
+# List all accreditations with their current verification status
+GET /api/v1/admin/accreditations
+  ?verification_status=stale|failed|confirmed|pending
+  &framework=fedramp_high|iso_27001|...
+  &subject_uuid={provider_uuid}
+  page_size=50&page_token=...
+
+# Trigger immediate re-verification of a specific accreditation
+POST /api/v1/admin/accreditations/{accreditation_uuid}:verify
+
+Response 200 โ€” returns Operation:
+  {
+    "name": "/api/v1/operations/{uuid}",
+    "done": false,
+    "metadata": {
+      "stage": "VERIFICATION_INITIATED",
+      "accreditation_uuid": "{uuid}",
+      "verification_tier": "external_registry"
+    }
+  }
+
+# Register a contract webhook endpoint for a BAA or DoD IL accreditation
+POST /api/v1/admin/accreditations/{accreditation_uuid}:configure-webhook
+  {
+    "contract_system": "docusign",
+    "contract_id": "abc-def-123",
+    "webhook_secret": ""
+  }
+
+# Inbound webhook endpoint (called by contract management systems)
+POST /api/v1/admin/accreditations/{accreditation_uuid}/contract-event
+  Authorization: Bearer 
+  {
+    "contract_event_type": "terminated",
+    "effective_date": "2026-04-01",
+    "contract_id": "abc-def-123",
+    "details": {}
+  }
+```
+
+---
+
+## 10. Deployment and Configuration
+
+The Accreditation Monitor is deployed as a standalone container alongside the DCM control plane. It requires:
+
+- Network access to external registries (FedRAMP, CMMC AB, IAF CertSearch, PCI SSC)
+- Network access to the DCM API Gateway (to push accreditation events)
+- Access to `certificate_ref` and `audit_report_ref` document URLs (for Tier 2 checks)
+- Inbound webhook endpoint (for Tier 3 contract system integrations)
+
+**Air-gapped / sovereign deployments:** For deployments without external internet access, the Accreditation Monitor operates in Tier 4 (expiry-only) mode by default for all frameworks. Tier 2 checks can still work if documents are stored on internal object storage (certificate_ref points to an internal URL). Tier 3 contract webhooks work if the contract management system is internal. Tier 1 registry checks are disabled โ€” a manual verification workflow applies instead, with platform admins periodically updating `last_verified_at` after out-of-band confirmation.
+
+```yaml
+accreditation_monitor_config:
+  # Per-registry enable/disable
+  registries:
+    fedramp:
+      enabled: true
+      poll_interval: P1D
+      timeout: PT30S
+    cmmc_ab:
+      enabled: true
+      poll_interval: P7D
+    iaf_certsearch:
+      enabled: true
+      poll_interval: P7D
+    stateramp:
+      enabled: false               # enable if state/local gov providers present
+  
+  # Document currency checks
+  document_checks:
+    enabled: true
+    default_max_age: P365D
+    extraction_timeout: PT60S
+  
+  # Air-gapped mode
+  air_gapped_mode: false
+  air_gapped_fallback: expiry_only | manual_workflow
+  
+  # Global escalation
+  global_stale_after: P7D
+  global_failure_threshold: 3
+  escalation_contact:
+    notification_provider_uuid: 
+    urgency: critical
+```
+
+---
+
+## 11. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `ACM-001` | The Accreditation Monitor is the authoritative source for `last_verified_at` on accreditation records. Platform admins may update it manually only when the Monitor is unavailable or in air-gapped mode โ€” all manual updates require a justification reason stored in the audit trail. |
+| `ACM-002` | An accreditation status change detected by the Monitor (external registry reports different status than DCM) does not automatically revoke the accreditation. It fires `accreditation.status_changed` and sets status to `pending_review`. A platform admin must confirm the change. The exception: if the external status is `Revoked` or `Terminated`, DCM immediately sets accreditation status to `revoked` without waiting for admin confirmation. |
+| `ACM-003` | Verification failure (registry unreachable, document inaccessible) does not revoke an accreditation. The Monitor increments `verification_failure_count`. At `verification_failure_threshold`, it fires `accreditation.verification_stale` and applies `stale_action`. Failure itself does not constitute a gap โ€” only confirmed negative status does. |
+| `ACM-004` | Accreditations in `sovereign` and `fsi` profiles must have `verification.tier` declared at a level of `document_currency` or above. `expiry_only` is not permitted for sovereign/fsi accreditations unless `air_gapped_mode: true` is explicitly configured. |
+| `ACM-005` | The `verification_multipliers` in the Scoring Model (Signal 5) apply to all accreditations, including those submitted before the Accreditation Monitor was deployed. Legacy accreditations with no `last_checked_at` are treated as `verification_stale` and weighted at the stale multiplier (0.4) until the Monitor performs its first check. |
+| `ACM-006` | Inbound contract webhooks (Tier 3) must authenticate using a provider callback credential issued at accreditation configuration time. Unauthenticated webhook calls are rejected with `401 Unauthorized` and generate an audit record. |
+| `ACM-007` | All verification events (`accreditation.*`) are written to the Audit Store regardless of outcome. There are no silent verifications โ€” every check, success or failure, has an audit record. |
+| `ACM-008` | In air-gapped mode, the Monitor operates in Tier 4 for registries it cannot reach. It does not repeatedly attempt unreachable external registries. After `air_gapped_retry_interval` (default P30D), it retries once to detect if network access has been restored. |
+
+---
+
+## 12. Relationship to Existing Architecture
+
+### doc 26 โ€” Accreditation and Authorization Matrix
+The Accreditation Monitor extends but does not replace the accreditation lifecycle model in doc 26. The existing `proposed โ†’ active โ†’ expired/revoked` lifecycle is preserved. The Monitor adds automated transitions into `pending_review` and `pending_renewal` states, and provides the data that drives the existing `ACCREDITATION_GAP` logic.
+
+### doc 27 โ€” Governance Matrix
+The Governance Matrix already evaluates accreditation status as part of Check 3 of the five-check boundary model. The Monitor improves the quality of that check: instead of relying solely on the declared `valid_until` date, the Governance Matrix now has access to externally verified current status via `last_verified_at` and `last_result`.
+
+### doc 29 โ€” Scoring Model
+The `verification_multipliers` addition to Signal 5 (Provider Accreditation Richness) means placement decisions can prefer providers whose accreditations have been recently externally verified over those relying on self-declared or stale verifications. This is a conservative, progressive enhancement โ€” it does not block placement, only refines tie-breaking.
+
+### doc 33 โ€” Event Catalog
+Seven new `accreditation.*` events are added (Section 6 of this document). These follow the same envelope and urgency model as all other DCM events.
+
+### doc 40 โ€” Standards Catalog
+The Accreditation Monitor is the operational implementation of the standards catalog's compliance framework entries. The standards catalog says *what* DCM recognizes; the Accreditation Monitor says *how* DCM verifies that recognition is still current.
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/A-provider-contract.md b/content/docs/data-model/A-provider-contract.md
index 1020355..b781576 100644
--- a/content/docs/data-model/A-provider-contract.md
+++ b/content/docs/data-model/A-provider-contract.md
@@ -1,10 +1,5 @@
 # DCM โ€” Unified Provider Contract
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Foundation
diff --git a/content/docs/data-model/B-policy-contract.md b/content/docs/data-model/B-policy-contract.md
index a2f964f..83276b7 100644
--- a/content/docs/data-model/B-policy-contract.md
+++ b/content/docs/data-model/B-policy-contract.md
@@ -1,10 +1,5 @@
 # DCM โ€” Unified Policy Contract
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 **Document Status:** โœ… Complete
 **Document Type:** Architecture Foundation

From c600c77699cac464e7feb8242bbcf980be46d553 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Tue, 31 Mar 2026 10:51:04 -0400
Subject: [PATCH 48/49] Added more example use cases;

Signed-off-by: Chris Roadfeldt 
---
 content/docs/architecture/ai-prompt.md        |   80 +-
 .../specifications/dcm-examples.md            | 1131 ++++++++++
 .../specifications/dcm-use-case-examples.md   | 1853 +++++++++++++++++
 3 files changed, 3063 insertions(+), 1 deletion(-)
 create mode 100644 content/docs/architecture/specifications/dcm-use-case-examples.md

diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index 4a44aac..0b1e35e 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -5046,7 +5046,85 @@ Provisioned Store โ†’ Realized State Store; Job Queue โ†’ Message Bus; Rules Eng
 **Matrix:** Domain 37, ACM-001 through ACM-007. Total: 37 domains / 287 capabilities.
 
 
-## SECTION 84 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 84 โ€” COMPREHENSIVE USE CASE EXAMPLES (dcm-use-case-examples.md)
+
+New specification: `specifications/dcm-use-case-examples.md` โ€” 1,853 lines, 5 sections.
+
+**Shared context:** All examples use consistent fictitious actors (alice@corp, bob@corp, svc-pipeline@corp), tenants (payments-bu, web-platform-bu, platform-team), and providers (vmware-prod, netbox-prod, vault-prod, freeipa-prod, ceph-prod, rabbitmq-prod, servicenow-prod, webapp-meta) for cross-example coherence.
+
+**Section 1 โ€” Data Model Examples (12 examples):**
+- 1.1 Four States: VM intent โ†’ layer assembly โ†’ requested state โ†’ realized state โ†’ discovered state โ†’ decommission
+- 1.2 Layer Assembly: 6-layer compose (base/DC/zone/BU/service/request) with full provenance
+- 1.3 Governance Matrix: PHI request denied โ€” four-axis evaluation, HIPAA BAA missing on provider
+- 1.4 Scoring Model: 5-signal risk score (0-100), placement tie-breaking by accreditation richness + verification multiplier
+- 1.5 Authority Tier: 200-VM bulk request โ†’ CRITICAL tier โ†’ 3-step approval chain
+- 1.6 Entity Relationships: VM+IP+FirewallRule composite, decommission impact analysis
+- 1.7 Universal Groups: tenant boundary, resource group, cross-tenant read authorization
+- 1.8 Scheduled Requests: maintenance window + deferred OS patch via CI/CD
+- 1.9 Request Dependency Graph: DBโ†’Appโ†’LB 3-node chain with field injection between nodes
+- 1.10 Workload Analysis: legacy VM discovered โ†’ classified (batch_processor, confidence:medium) โ†’ MTA score โ†’ ingested
+- 1.11 Accreditation Monitor: FedRAMP daily verify + mid-cycle revocation โ†’ immediate gap trigger
+- 1.12 Session Revocation: stolen laptop โ†’ all sessions revoked โ†’ in-flight request aborted safely
+
+**Section 2 โ€” Provider Interaction Examples (6 new examples, 2.5โ€“2.10):**
+- 2.5 Auth Provider (FreeIPA): registration, LDAP auth flow, group mapping โ†’ DCM roles
+- 2.6 Storage Provider (Ceph): write-once snapshot registration, Realized State write
+- 2.7 Message Bus Provider (RabbitMQ): topic exchange, routing key pattern, multi-subscriber routing
+- 2.8 Credential Provider (Vault): AppRole registration, ephemeral bind-password fetch, dynamic DB creds, consumer SSH key retrieval with audit
+- 2.9 Meta Provider: compound WebApp (VM+IP+FW+DNS) decomposition, parallel + sequential constituent ordering, field injection
+- 2.10 ITSM Provider (ServiceNow): incident creation on provider health change, field mapping, resolve on recovery
+
+**Section 3 โ€” Registration Flow Examples (3 examples):**
+- 3.1 Information Provider (NetBox): token issuance, mTLS registration, 6-check validation, approval, assembly enrichment
+- 3.2 Auth Provider (Azure AD OIDC): secondary auth source for contractors, precedence ordering, TTL-scoped role mapping
+- 3.3 Meta Provider: constituent validation, circular dependency check, activation
+
+**Section 4 โ€” OPA Policy Integration (2 examples):**
+- 4.1 Shadow mode: 30-day shadow evaluation, divergence reporting, admin review dashboard, promotion to active
+- 4.2 Bundle delivery: sidecar registration, 5-minute pull cycle, hot reload, evaluation call with input/output
+
+**Section 5 โ€” GUI Examples (3 examples):**
+- 5.1 Consumer Portal: login โ†’ catalog โ†’ cost estimate โ†’ submit โ†’ progress polling โ†’ credential delivery
+- 5.2 Admin GUI Policy Flow: node-by-node pipeline visualization with shadow indicators and red blocking paths
+- 5.3 Admin GUI Drift Dashboard: severity summary, critical drift detail, revert/accept/investigate actions
+
+
+## SECTION 85 โ€” EXAMPLES EXPANSION (dcm-examples.md)
+
+dcm-examples.md expanded from 1,058 lines to 2,189 lines. Now covers all 10 provider types, all 3 remaining policy types, and 8 new lifecycle/model flows. Full section inventory:
+
+**Section 6 โ€” Provider Type Examples (NEW):**
+- 6.1 Storage Provider โ€” state store write/read cycle (provenance emission, replica confirmation)
+- 6.2 Auth Provider โ€” OIDC cutover from GitHub OAuth (shadow evaluation, zero-downtime cutover)
+- 6.3 Credential Provider โ€” SSH key issuance post-VM-realization, 90-day TTL, auto-rotation at P45D
+- 6.4 Meta Provider โ€” three-tier WebApp stack (VMโ†’VMโ†’LBโ†’DNS with field injection between tiers)
+- 6.5 ITSM Provider โ€” ServiceNow Change Request lifecycle (createโ†’approveโ†’implementโ†’close)
+- 6.6 Message Bus Provider โ€” Kafka event bridge (entity lifecycle events, dead letter handling)
+
+**Section 7 โ€” Policy Type Examples (NEW):**
+- 7.1 Transformation Policy โ€” OS image auto-injection (immutable field, provenance annotation)
+- 7.2 Placement Policy โ€” PHI VM with HIPAA BAA requirement (require/prefer/exclude model, audit trail)
+- 7.3 Shadow Execution โ€” cost-cap policy rollout (parallel evaluation, divergence report, safe activation)
+
+**Section 8 โ€” Lifecycle and Model Examples (NEW):**
+- 8.1 Scheduled Request โ€” deferred provisioning via maintenance window (deadline handling, cancellation)
+- 8.2 Request Dependency Graph โ€” three-tier app with realized field injection across tiers
+- 8.3 Authority Tier Routing โ€” sovereign decommission requiring sequential platform_admin + CISO approval
+- 8.4 Rehydration (intent mode) โ€” DR failover to new datacenter (contrast with Static Replace)
+- 8.5 Session Revocation โ€” emergency security incident response (revoke-all, in-flight handling)
+- 8.6 Workload Analysis โ€” brownfield VM classification via MTA (port scan, process list, classification pipeline)
+- 8.7 Scoring Model โ€” placement tie-breaking with 5 signals (accreditation verification multiplier effect)
+- 8.8 Accreditation Monitor โ€” FedRAMP status change detection (mid-cycle downgrade, pending_review flow)
+
+**Previously covered (Sections 1-5, unchanged):**
+Service Provider (dispatch cycle), Information Provider (assembly enrichment), Policy Provider (OPA sidecar),
+Notification Provider (audience graph), Consumer API lifecycle, Admin API flows, Registration onboarding,
+Brownfield ingestion, Static Replace, In-Place Upgrade.
+
+All 10 provider types: โœ… covered. All 7 policy output schemas: โœ… covered. All major model flows: โœ… covered.
+
+
+## SECTION 86 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure):
 
diff --git a/content/docs/architecture/specifications/dcm-examples.md b/content/docs/architecture/specifications/dcm-examples.md
index 5394f6b..9f4316f 100644
--- a/content/docs/architecture/specifications/dcm-examples.md
+++ b/content/docs/architecture/specifications/dcm-examples.md
@@ -1056,3 +1056,1134 @@ Response 202: { "registration_uuid": "reg-001", "status": "VALIDATING",
 ---
 
 *Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
+
+# Section 6 โ€” Provider Type Examples
+
+## 6.1 Storage Provider โ€” State Store Write and Read
+
+A Storage Provider persists and streams DCM internal state. This example shows
+the full lifecycle: DCM writing a Requested State record to a PostgreSQL-backed
+Storage Provider, followed by the Request Orchestrator reading it back.
+
+**Provider registration excerpt:**
+
+```yaml
+storage_provider_registration:
+  provider_type: storage_provider
+  display_name: "Primary PostgreSQL State Store"
+  endpoint: https://pg-state.internal:5432
+  storage_sub_type: relational_state
+  stores_owned:
+    - store_type: requested_state
+    - store_type: realized_state
+    - store_type: intent_state
+  consistency_guarantee: strong          # synchronous write confirmation
+  replication:
+    strategy: synchronous_replica
+    replica_endpoints:
+      - https://pg-state-replica-1.internal:5432
+      - https://pg-state-replica-2.internal:5432
+  provenance_emission: true              # emits audit event on every write
+```
+
+**Write: DCM persists a Requested State record**
+
+```
+DCM Request Orchestrator
+  โ”‚
+  โ–ผ POST https://pg-state.internal/api/v1/records
+    Authorization: Bearer 
+    Content-Type: application/json
+    {
+      "record_type": "requested_state",
+      "entity_uuid": "a1b2c3d4-...",
+      "tenant_uuid": "t1t2t3t4-...",
+      "request_uuid": "r1r2r3r4-...",
+      "payload": { ... assembled request payload ... },
+      "written_at": "2026-03-31T10:00:00Z"
+    }
+
+Provider response:
+    {
+      "record_uuid": "s1s2s3s4-...",
+      "written_at": "2026-03-31T10:00:00.042Z",
+      "replicated": true
+    }
+
+Provider emits provenance event:
+    event_type: storage.record_written
+    record_type: requested_state
+    record_uuid: s1s2s3s4-...
+    entity_uuid: a1b2c3d4-...
+    store_provider_uuid: 
+```
+
+**Read: Drift scheduler retrieves Realized State for comparison**
+
+```
+DCM Drift Scheduler
+  โ”‚
+  โ–ผ GET https://pg-state.internal/api/v1/records
+    ?entity_uuid=a1b2c3d4-...
+    &record_type=realized_state
+    &version=latest
+    Authorization: Bearer 
+
+Provider response:
+    {
+      "record_uuid": "z9z8z7z6-...",
+      "entity_uuid": "a1b2c3d4-...",
+      "record_type": "realized_state",
+      "payload": { ... realized state snapshot ... },
+      "written_at": "2026-03-31T09:55:00Z",
+      "supersedes_uuid": "y8y7y6y5-..."
+    }
+```
+
+---
+
+## 6.2 Auth Provider โ€” OIDC Cutover (GitHub OAuth โ†’ Corporate OIDC)
+
+DCM's auth configuration is a versioned artifact. Adding a new Auth Provider
+and cutting over is a standard GitOps PR workflow โ€” no downtime.
+
+**Step 1: Register the corporate OIDC provider**
+
+```yaml
+# GitOps PR: auth-providers/corporate-oidc-v1.yaml
+auth_provider:
+  handle: "auth-providers/corporate-oidc"
+  version: "1.0.0"
+  status: developing               # shadow mode โ€” not yet enforced
+  provider_type: oidc
+  display_name: "Corporate OIDC (Keycloak)"
+  endpoint: https://sso.corp.internal/realms/dcm
+  client_id: dcm-control-plane
+  client_secret_ref: credential://vault/dcm/oidc-client-secret
+  scopes: [openid, profile, email, groups]
+  group_claim: "dcm_groups"       # claim containing DCM group memberships
+  mfa_required: true
+  profile_overlay: standard       # overrides to fsi/sovereign possible
+```
+
+**Step 2: Shadow evaluation (parallel run)**
+
+```
+Platform Admin: PATCH /api/v1/admin/auth-providers/{uuid}
+  { "status": "proposed" }
+
+DCM response:
+  Both providers now evaluate all logins in parallel.
+  auth.session_created events include shadow_auth_result for comparison.
+  Divergences (user authenticated by one provider but not the other)
+  are surfaced via governance.auth_shadow_divergence events.
+
+After 72 hours of shadow evaluation:
+  Shadow report: 1,847 logins evaluated
+    Converged: 1,845 (99.9%)
+    Diverged: 2 (both: OIDC rejected due to missing group claim โ€” fixed)
+```
+
+**Step 3: Activate and retire GitHub OAuth**
+
+```
+Platform Admin: PATCH /api/v1/admin/auth-providers/{oidc-uuid}
+  { "status": "active" }
+
+Platform Admin: PATCH /api/v1/admin/auth-providers/{github-uuid}
+  { "status": "deprecated" }
+
+DCM response:
+  Corporate OIDC: primary, enforced
+  GitHub OAuth: accepted for 30 days (configured sunset), then retired
+  All existing sessions: revoked (auth.session_revoked ร— N users)
+  Users: re-authenticate on next request
+```
+
+---
+
+## 6.3 Credential Provider โ€” SSH Key Issuance After VM Realization
+
+After a VM is realized, the Credential Provider issues an SSH key pair to the
+requesting consumer โ€” scoped to that specific entity.
+
+**Flow:**
+
+```
+1. Consumer: POST /api/v1/requests
+     catalog_item_uuid: 
+     fields: { cpu: 4, ram_gb: 16, os: "RHEL 9.4" }
+
+2. Policy evaluation โ€” Transformation policy injects credential requirement:
+     fields.credential_requirements:
+       - credential_type: ssh_key
+         issued_to: requesting_actor
+         scope: [ssh_access]
+         ttl: P90D             # 90-day key lifetime
+
+3. VM realization completes โ†’ entity_uuid: vm-abc123
+
+4. DCM dispatches credential issuance sub-request:
+   POST https://vault.internal/api/v1/credentials
+     {
+       "credential_type": "ssh_key",
+       "entity_uuid": "vm-abc123",
+       "issued_to_actor_uuid": "actor-xyz",
+       "scope": ["ssh_access"],
+       "ttl": "P90D"
+     }
+
+5. Credential Provider (HashiCorp Vault) response:
+     {
+       "credential_uuid": "cred-456",
+       "public_key": "ssh-ed25519 AAAA...",
+       "private_key_ref": "vault://dcm/ssh-keys/cred-456/private",
+       "valid_until": "2026-06-30T00:00:00Z"
+     }
+
+6. Consumer retrieves credential:
+   GET /api/v1/resources/vm-abc123/credentials/cred-456/value
+     Authorization: Bearer 
+
+   Response:
+     {
+       "credential_uuid": "cred-456",
+       "credential_type": "ssh_key",
+       "public_key": "ssh-ed25519 AAAA...",
+       "private_key": "-----BEGIN OPENSSH PRIVATE KEY-----\n...",
+       "valid_until": "2026-06-30T00:00:00Z"
+     }
+   Note: private_key delivered once at retrieval; not stored in DCM.
+```
+
+**Rotation at P45D (50% of lifetime):**
+
+```
+Credential Provider fires: credential.rotation_due
+  credential_uuid: cred-456
+  entity_uuid: vm-abc123
+  days_until_expiry: 45
+
+DCM Policy: Transformation โ†’ auto-issue renewal credential
+  New credential_uuid: cred-789
+  valid_until: 2026-09-30 (new 90-day window)
+
+credential.rotated event โ†’ consumer notified via webhook
+Old credential (cred-456): valid_until unchanged, both active during overlap
+At valid_until: cred-456 โ†’ expired, credential.expired event
+```
+
+---
+
+## 6.4 Meta Provider โ€” Three-Tier Web Application Stack
+
+A Meta Provider composes multiple atomic service providers into a single catalog
+item. The consumer requests one thing; DCM orchestrates the constituent parts.
+
+**Meta Provider registration (compound service):**
+
+```yaml
+meta_provider_registration:
+  provider_type: meta_provider
+  display_name: "Three-Tier Web App Stack"
+  resource_types_composed:
+    - fqn: ApplicationStack.WebApp
+      version: "1.0.0"
+      constituents:
+        - component_id: db
+          resource_type: Compute.VirtualMachine
+          provided_by: external            # DCM selects a compute provider
+          depends_on: []
+          required_for_delivery: required
+
+        - component_id: app
+          resource_type: Compute.VirtualMachine
+          provided_by: external
+          depends_on: [db]                 # waits for db realization
+          inject_from:
+            - component: db
+              field: realized_fields.primary_ip
+              into: fields.db_host
+          required_for_delivery: required
+
+        - component_id: lb
+          resource_type: Network.LoadBalancer
+          provided_by: external
+          depends_on: [app]
+          inject_from:
+            - component: app
+              field: realized_fields.primary_ip
+              into: fields.backend_pool[0]
+          required_for_delivery: required
+
+        - component_id: dns
+          resource_type: Network.DNSRecord
+          provided_by: external
+          depends_on: [lb]
+          inject_from:
+            - component: lb
+              field: realized_fields.vip
+              into: fields.record_value
+          required_for_delivery: optional  # stack delivered without DNS if it fails
+```
+
+**Consumer request:**
+
+```
+POST /api/v1/requests
+  {
+    "catalog_item_uuid": "",
+    "fields": {
+      "app_name": "payments-api",
+      "environment": "prod",
+      "db_size": "large",
+      "app_replicas": 3
+    }
+  }
+```
+
+**What DCM orchestrates (transparent to consumer):**
+
+```
+Request 1 โ†’ db VM (dispatched immediately)
+  Realized: db.primary_ip = 10.0.1.5
+
+Request 2 โ†’ app VM (dispatched after db realized)
+  fields.db_host = 10.0.1.5 (injected from db realization)
+  Realized: app.primary_ip = 10.0.1.10
+
+Request 3 โ†’ load balancer (dispatched after app realized)
+  fields.backend_pool[0] = 10.0.1.10 (injected)
+  Realized: lb.vip = 203.0.113.42
+
+Request 4 โ†’ DNS record (dispatched after lb realized)
+  fields.record_value = 203.0.113.42 (injected)
+  Realized: dns.fqdn = payments-api.corp.example.com
+
+Consumer entity: ApplicationStack.WebApp
+  Status: OPERATIONAL
+  Constituent entities: [db-uuid, app-uuid, lb-uuid, dns-uuid]
+  Logical endpoint: payments-api.corp.example.com
+```
+
+---
+
+## 6.5 ITSM Provider โ€” ServiceNow Change Request Lifecycle
+
+An ITSM Provider creates and manages change tickets in ServiceNow as part of
+DCM request lifecycle gates. The ITSM ticket becomes the approval gate.
+
+**Provider registration:**
+
+```yaml
+itsm_provider_registration:
+  provider_type: itsm_provider
+  display_name: "ServiceNow โ€” Production ITSM"
+  endpoint: https://corp.service-now.com
+  itsm_system: servicenow
+  supported_actions:
+    - create_change_request
+    - update_change_request
+    - close_change_request
+    - get_approval_status
+  field_mapping_ref: "itsm-mappings/servicenow-prod-v1.yaml"
+  cmdb_ci_type_map:
+    Compute.VirtualMachine: cmdb_ci_server
+    Network.VLAN: cmdb_ci_network
+```
+
+**Lifecycle: DCM request requiring Change Approval:**
+
+```
+1. Consumer: POST /api/v1/requests
+     fields: { ... VM configuration ... }
+
+2. GateKeeper policy fires (prod tenant + restricted network):
+     action: require_itsm_approval
+     itsm_provider_uuid: 
+     change_type: standard
+     risk_level: medium
+
+3. DCM โ†’ ITSM Provider: create_change_request
+     POST https://corp.service-now.com/api/dcm/v1/changes
+     {
+       "short_description": "DCM: Provision Compute.VirtualMachine",
+       "dcm_request_uuid": "req-abc123",
+       "change_type": "standard",
+       "risk": "medium",
+       "implementation_plan": "DCM automated provisioning",
+       "configuration_item": "vm-payments-prod-07",
+       "requested_by": "jane.smith@corp.example.com"
+     }
+
+   ServiceNow response:
+     { "change_number": "CHG0012345", "state": "Assess", "sys_id": "abc123" }
+
+4. DCM request status: AWAITING_EXTERNAL_APPROVAL
+   Consumer notified: "Change request CHG0012345 created โ€” awaiting approval"
+
+5. ServiceNow CAB approves โ†’ webhook to DCM:
+   POST /api/v1/admin/itsm-events
+   {
+     "itsm_provider_uuid": "",
+     "change_number": "CHG0012345",
+     "dcm_request_uuid": "req-abc123",
+     "event_type": "approved",
+     "approved_by": "change.manager@corp.example.com",
+     "approved_at": "2026-03-31T14:00:00Z"
+   }
+
+6. DCM resumes request dispatch โ†’ VM provisioned
+   ITSM Provider: update_change_request โ†’ state: Implement
+
+7. VM realized โ†’ ITSM Provider: close_change_request
+     { "state": "Closed Complete", "close_notes": "DCM: entity vm-xyz OPERATIONAL" }
+```
+
+---
+
+## 6.6 Message Bus Provider โ€” External System Event Bridge
+
+A Message Bus Provider bridges DCM events to external messaging infrastructure.
+This example shows DCM publishing entity lifecycle events to an Apache Kafka topic.
+
+**Provider registration:**
+
+```yaml
+message_bus_provider_registration:
+  provider_type: message_bus_provider
+  display_name: "Kafka Event Bridge โ€” Infrastructure Events"
+  endpoint: https://kafka-bridge.internal:9092
+  protocol: kafka
+  subscribed_event_types:
+    - entity.lifecycle_changed
+    - drift.detected
+    - accreditation.status_changed
+    - provider.status_changed
+  topic_mapping:
+    entity.lifecycle_changed: dcm.infrastructure.lifecycle
+    drift.detected: dcm.infrastructure.drift
+    accreditation.status_changed: dcm.compliance.accreditation
+    provider.status_changed: dcm.infrastructure.providers
+  delivery_guarantee: at_least_once
+  dead_letter_topic: dcm.dlq
+```
+
+**Flow: VM reaches OPERATIONAL โ†’ Kafka message published:**
+
+```
+DCM internal: entity.lifecycle_changed event fires
+  entity_uuid: vm-abc123
+  resource_type: Compute.VirtualMachine
+  from_state: PROVISIONING
+  to_state: OPERATIONAL
+  tenant_uuid: t1t2...
+
+Message Bus Provider receives event, publishes to Kafka:
+  Topic: dcm.infrastructure.lifecycle
+  Key: vm-abc123
+  Value: {
+    "event_type": "entity.lifecycle_changed",
+    "entity_uuid": "vm-abc123",
+    "resource_type": "Compute.VirtualMachine",
+    "from_state": "PROVISIONING",
+    "to_state": "OPERATIONAL",
+    "tenant_uuid": "t1t2...",
+    "timestamp": "2026-03-31T10:05:33Z",
+    "dcm_instance_uuid": ""
+  }
+
+External consumer (monitoring pipeline) processes message:
+  โ†’ Updates CMDB record
+  โ†’ Triggers monitoring agent installation
+  โ†’ Notifies application team
+```
+
+**Dead letter handling:**
+
+```
+Kafka publish fails (broker unreachable):
+  Message Bus Provider: retry with exponential backoff (3 attempts)
+  After threshold: publish to dcm.dlq with failure metadata
+  Fire: message_bus.delivery_failed (urgency: medium) โ†’ Platform Admin
+  DCM: event stored in Message Bus Provider's local queue for replay
+```
+
+---
+
+# Section 7 โ€” Policy Type Examples
+
+## 7.1 Transformation Policy โ€” Automatic Data Enrichment
+
+Transformation policies modify the request payload before dispatch. They run
+after validation passes and before placement selection.
+
+**Use case:** Auto-inject the required OS image version based on the requested
+OS name and the current approved version from an Information Provider.
+
+```rego
+package dcm.policy.transform.os_image_injection
+
+import future.keywords.if
+
+# When consumer requests a VM with os_name but no os_image_uuid:
+transform if {
+    input.payload.resource_type == "Compute.VirtualMachine"
+    input.payload.fields.os_name != null
+    input.payload.fields.os_image_uuid == null
+}
+
+output := {
+    "output_type": "transformation",
+    "field_injections": [{
+        "field": "fields.os_image_uuid",
+        "value": data.information_providers.os_registry.current_approved[
+            input.payload.fields.os_name
+        ],
+        "source": "policy/transform/os-image-injection",
+        "immutable": true      # consumer cannot override after injection
+    }],
+    "audit_annotations": [{
+        "key": "os_image_injected_at",
+        "value": time.now_ns()
+    }]
+}
+```
+
+**In practice:**
+
+```
+Consumer submits:
+  { "os_name": "RHEL 9", "cpu": 4, "ram_gb": 16 }
+
+After transformation:
+  { "os_name": "RHEL 9", "cpu": 4, "ram_gb": 16,
+    "os_image_uuid": "img-rhel9-20260315",   โ† injected
+    "_provenance": {
+      "os_image_uuid": {
+        "source": "policy/transform/os-image-injection",
+        "immutable": true,
+        "injected_at": 1743412800000
+      }
+    }
+  }
+
+If consumer tries to override os_image_uuid: Policy Engine rejects with:
+  { "code": "FIELD_IMMUTABLE",
+    "field": "fields.os_image_uuid",
+    "set_by": "policy/transform/os-image-injection" }
+```
+
+---
+
+## 7.2 Placement Policy โ€” Provider Selection with Constraints
+
+Placement policies express where a resource should be dispatched. DCM's
+Placement Engine evaluates all registered providers against placement policies
+plus the Scoring Model.
+
+**Use case:** For a PHI-classified VM, require a provider with HIPAA BAA
+accreditation, in Zone A or Zone B, with at least 30% capacity remaining.
+
+```rego
+package dcm.policy.placement.phi_vm
+
+import future.keywords.if
+
+placement if {
+    input.payload.resource_type == "Compute.VirtualMachine"
+    input.payload.data_classification == "phi"
+}
+
+output := {
+    "output_type": "placement",
+    "require": {
+        "accreditations": ["hipaa_baa"],          # provider must hold active BAA
+        "sovereignty_zones": ["US-EAST", "US-WEST"], # data must stay in US
+        "availability_zones": ["zone-a", "zone-b"],
+        "minimum_capacity_pct": 30
+    },
+    "prefer": {
+        "accreditations": ["fedramp_moderate"],   # prefer FedRAMP if available
+        "availability_zones": ["zone-a"]          # prefer zone-a (lower latency)
+    },
+    "exclude": {
+        "provider_uuids": []                      # no explicit exclusions
+    }
+}
+```
+
+**Placement Engine evaluation:**
+
+```
+Candidate providers for Compute.VirtualMachine:
+  Provider A: hipaa_baa=โœ…  zone-a=โœ…  capacity=45%  fedramp=โŒ
+  Provider B: hipaa_baa=โœ…  zone-b=โœ…  capacity=72%  fedramp=โœ…
+  Provider C: hipaa_baa=โŒ  zone-a=โœ…  capacity=88%
+
+Filtering (require):
+  Provider C: eliminated (no hipaa_baa)
+
+Scoring (prefer + Scoring Model):
+  Provider B: higher score (fedramp preferred, higher capacity)
+  Provider A: lower score (no fedramp)
+
+Selected: Provider B
+  dispatch: CreateRequest โ†’ Provider B
+  placement_audit:
+    policy: placement/phi-vm
+    evaluated: [provider-a, provider-b, provider-c]
+    eliminated: [provider-c (missing: hipaa_baa)]
+    selected: provider-b
+    selection_reason: "highest scoring after require filter"
+```
+
+---
+
+## 7.3 Shadow Execution โ€” Safe Policy Rollout
+
+Shadow execution lets a new policy run against real traffic without affecting
+outcomes. Divergences are surfaced for review before the policy goes active.
+
+**Use case:** Testing a new cost-cap gatekeeper before enforcement.
+
+```yaml
+# GitOps PR: policies/cost-cap-v1.yaml
+policy:
+  handle: "tenant/acme/cost-cap-1000"
+  version: "1.0.0"
+  status: proposed               # shadow mode โ€” evaluates but does not enforce
+  type: gatekeeper
+  rules:
+    - condition: "cost_estimate.monthly_usd > 1000 AND tenant.uuid == 'acme-uuid'"
+      action: gate
+      message: "Estimated monthly cost exceeds $1,000 limit for this tenant"
+  shadow_target: "tenant/acme/cost-cap-500"   # compare against existing policy
+```
+
+**Shadow evaluation in practice:**
+
+```
+Request: VM with estimated cost $800/month (tenant: acme)
+
+existing policy (cost-cap-500, active):  GATE โ€” $800 > $500 limit โ†’ request blocked
+new policy    (cost-cap-1000, shadow):   PASS โ€” $800 < $1,000 limit
+
+Divergence detected:
+  policy.shadow_divergence event:
+    shadow_policy: cost-cap-1000
+    active_policy: cost-cap-500
+    divergence_type: active_gated_shadow_passed
+    request_uuid: req-abc123
+    estimated_cost: 800.00
+
+After 7 days of shadow evaluation:
+  Report: 234 requests evaluated
+    Converged (both gate):    89 (38%)
+    Converged (both pass):   118 (50%)
+    Diverged (active gates, shadow passes):  27 (12%)
+       โ† these are requests that would be UNBLOCKED by the new policy
+
+Platform Admin reviews divergence report:
+  Decision: the 27 unblocked requests are legitimate โ€” activate cost-cap-1000
+  PATCH /api/v1/admin/policies/{shadow-uuid} โ†’ { "status": "active" }
+  PATCH /api/v1/admin/policies/{old-uuid}    โ†’ { "status": "deprecated" }
+```
+
+---
+
+# Section 8 โ€” Lifecycle and Model Examples
+
+## 8.1 Scheduled Request โ€” Deferred Provisioning with Maintenance Window
+
+A new database VM is required, but the network team's policy requires all new
+network allocations to happen inside an approved maintenance window.
+
+```
+Consumer: POST /api/v1/requests
+  {
+    "catalog_item_uuid": "",
+    "fields": {
+      "db_engine": "postgresql",
+      "storage_gb": 500,
+      "environment": "prod"
+    },
+    "scheduled_at": null,           # not setting explicit time
+    "schedule": {
+      "dispatch": "window",
+      "window_id": "mw-network-weekly-saturday",   # declared maintenance window
+      "not_after": "2026-05-01T00:00:00Z"          # cancel if no window before May
+    }
+  }
+
+Response 200 โ€” returns Operation:
+  {
+    "name": "/api/v1/operations/req-abc123",
+    "done": false,
+    "metadata": {
+      "stage": "SCHEDULED",
+      "resource_uuid": null,
+      "request_uuid": "req-abc123",
+      "scheduled_dispatch": "window",
+      "window_id": "mw-network-weekly-saturday",
+      "next_window_opens": "2026-04-05T02:00:00Z"
+    }
+  }
+
+At 2026-04-05T02:00:00Z (maintenance window opens):
+  DCM dispatches request to Database Service Provider
+  Stage advances: SCHEDULED โ†’ DISPATCHED โ†’ PROVISIONING โ†’ OPERATIONAL
+
+If window is missed and 2026-05-01 arrives without dispatch:
+  Request โ†’ CANCELLED
+  reason: "Scheduled dispatch deadline exceeded"
+  consumer notified via webhook
+```
+
+---
+
+## 8.2 Request Dependency Graph โ€” Three-Tier App with Field Injection
+
+A consumer submits a three-tier application as a coordinated dependency group.
+DCM dispatches each tier in order, injecting realized values between tiers.
+
+```
+Consumer: POST /api/v1/request-groups
+  {
+    "group_handle": "payments-v2-deploy",
+    "requests": [
+      {
+        "request_uuid": "req-db-001",         # created beforehand or inline
+        "depends_on": [],
+        "catalog_item_uuid": "",
+        "fields": { "storage_gb": 500, "environment": "prod" }
+      },
+      {
+        "request_uuid": "req-app-001",
+        "depends_on": [
+          {
+            "request_uuid": "req-db-001",
+            "wait_for": "realized",
+            "inject_fields": [
+              {
+                "from_field": "realized_fields.primary_ip",
+                "to_field": "fields.db_host"
+              },
+              {
+                "from_field": "realized_fields.db_port",
+                "to_field": "fields.db_port"
+              }
+            ]
+          }
+        ],
+        "catalog_item_uuid": "",
+        "fields": { "app": "payments-api", "environment": "prod" }
+      },
+      {
+        "request_uuid": "req-lb-001",
+        "depends_on": [
+          {
+            "request_uuid": "req-app-001",
+            "wait_for": "realized",
+            "inject_fields": [
+              {
+                "from_field": "realized_fields.primary_ip",
+                "to_field": "fields.backend_pool[0]"
+              }
+            ]
+          }
+        ],
+        "catalog_item_uuid": "",
+        "fields": { "protocol": "HTTPS", "port": 443 }
+      }
+    ]
+  }
+
+Execution sequence:
+  T+0s:   req-db-001 dispatched (no dependencies)
+  T+45s:  req-db-001 REALIZED โ†’ db.primary_ip=10.0.1.5
+  T+45s:  req-app-001 dispatched (dependency met; db_host=10.0.1.5 injected)
+  T+90s:  req-app-001 REALIZED โ†’ app.primary_ip=10.0.1.10
+  T+90s:  req-lb-001 dispatched (dependency met; backend_pool[0]=10.0.1.10 injected)
+  T+105s: req-lb-001 REALIZED โ†’ lb.vip=203.0.113.42
+
+GET /api/v1/request-groups/payments-v2-deploy:
+  {
+    "group_status": "completed",
+    "requests": [
+      { "request_uuid": "req-db-001",  "status": "REALIZED", "entity_uuid": "vm-db-..." },
+      { "request_uuid": "req-app-001", "status": "REALIZED", "entity_uuid": "vm-app-..." },
+      { "request_uuid": "req-lb-001",  "status": "REALIZED", "entity_uuid": "lb-..." }
+    ]
+  }
+```
+
+---
+
+## 8.3 Authority Tier Routing โ€” Tiered Approval for High-Impact Change
+
+DCM routes approval requests based on the authority tier required by matching
+policies. This example shows a sovereign-profile decommission routed through
+two sequential approval tiers.
+
+**Authority tier definition (organization-configured):**
+
+```yaml
+authority_tiers:
+  - name: operator
+    weight: 10
+    description: "Day-to-day platform operator"
+  - name: team_lead
+    weight: 30
+    description: "Technical team lead or senior engineer"
+  - name: platform_admin
+    weight: 60
+    description: "Platform administration team"
+  - name: ciso_office
+    weight: 100
+    description: "CISO office โ€” for high-impact or compliance-relevant changes"
+```
+
+**GateKeeper policy requiring CISO approval:**
+
+```rego
+package dcm.policy.gate.sovereign_decommission
+
+gate if {
+    input.payload.lifecycle_action == "decommission"
+    input.payload.data_classification == "restricted"
+    input.profile == "sovereign"
+}
+
+output := {
+    "output_type": "gatekeeper",
+    "action": "require_approval",
+    "approval_tiers": ["platform_admin", "ciso_office"],   # sequential
+    "approval_mode": "sequential",
+    "approval_deadline": "P7D",
+    "gate_message": "Decommission of restricted-classified resource in sovereign profile requires Platform Admin and CISO approval"
+}
+```
+
+**Approval flow:**
+
+```
+Request: Decommission VM (restricted data, sovereign profile)
+
+GateKeeper fires:
+  Request โ†’ AWAITING_APPROVAL (tier: platform_admin)
+  Notification โ†’ Platform Admin audience (urgency: high)
+
+Platform Admin approves:
+  POST /api/v1/approvals/{approval-uuid}/approve
+  Request โ†’ AWAITING_APPROVAL (tier: ciso_office)
+  Notification โ†’ CISO audience (urgency: high)
+
+CISO approves:
+  POST /api/v1/approvals/{approval-uuid}/approve
+  Request โ†’ DISPATCHED โ†’ DECOMMISSIONED
+
+Audit trail:
+  gate_evaluation: platform_admin approved by actor-123 at T+2h
+  gate_evaluation: ciso_office approved by actor-456 at T+18h
+  total_gate_duration: 20 hours
+```
+
+---
+
+## 8.4 Rehydration (Intent Mode) โ€” DR Failover to New Datacenter
+
+A business unit needs to redeploy their application in DC2 after DC1 becomes
+unavailable. Rehydration replays original intent through current policies โ€”
+applying today's standards to the original request.
+
+**Contrast with Static Replace** (Section 1.9): Static Replace re-executes the
+Requested State verbatim. Rehydration (intent mode) re-runs the full pipeline
+from Intent State โ€” layer assembly, policy evaluation, and placement selection
+all run fresh.
+
+```
+Consumer: POST /api/v1/resources/{entity_uuid}:rehydrate
+  {
+    "mode": "intent",
+    "reason": "DR failover โ€” DC1 unavailable, deploying to DC2",
+    "placement_constraints": {
+      "require_zones": ["DC2-ZONE-A", "DC2-ZONE-B"],
+      "exclude_zones": ["DC1"]
+    },
+    "reuse_intent_version": null    # null = use original intent as-is
+  }
+
+DCM pipeline:
+  1. Retrieve Intent State for entity_uuid
+     (the consumer's original request, before any policy processing)
+
+  2. Layer assembly runs fresh against current layers
+     (DC2 data center layer, current security baseline, current network config)
+
+  3. Policy evaluation runs fresh
+     (current GateKeeper, Validation, Transformation, Placement policies)
+     Note: Placement constraint: exclude DC1, require DC2
+
+  4. Placement Engine selects DC2 provider
+     (new provider_uuid in the dispatch payload)
+
+  5. Dispatch โ†’ DC2 Service Provider
+     New entity realized in DC2
+
+  6. Original entity (DC1): status โ†’ INDETERMINATE_REALIZATION
+     (DC1 resources may still exist โ€” drift reconciliation queued)
+
+  7. New entity (DC2): OPERATIONAL
+     entity_uuid: new (original entity retired)
+     intent_state links to original intent_uuid (provenance preserved)
+
+Result:
+  - New VM in DC2 with DC2 network addressing, current OS image, current policies
+  - Full provenance chain: original intent โ†’ DC1 realization โ†’ DC2 rehydration
+  - Audit record: rehydration_reason, original_entity_uuid, new_entity_uuid
+```
+
+---
+
+## 8.5 Session Revocation โ€” Security Incident Response
+
+A security team detects that an actor's credentials may have been compromised.
+Emergency revocation immediately invalidates all active sessions.
+
+```
+Security Team: POST /api/v1/admin/actors/{actor_uuid}/sessions:revoke-all
+  {
+    "reason": "Suspected credential compromise โ€” security incident INC-2026-042",
+    "revocation_scope": "all_sessions",
+    "urgency": "emergency"
+  }
+
+DCM processes:
+  1. All active sessions for actor_uuid โ†’ status: revoked
+     Sessions affected: 3 (web console, API client, CLI)
+     revoked_at: 2026-03-31T16:42:00Z
+     revocation_trigger: security_incident
+     revoked_by: actor-security-team
+
+  2. Session UUIDs added to Session Revocation Registry
+     (fast cache โ€” all API components check this on every request)
+
+  3. Next API request from compromised actor:
+     GET /api/v1/resources (any request)
+     โ†’ 401 Unauthorized
+     { "code": "SESSION_REVOKED",
+       "message": "Session has been revoked. Please re-authenticate.",
+       "revoked_at": "2026-03-31T16:42:00Z" }
+
+  4. Events fired:
+     auth.session_revoked ร— 3 (one per session)
+     auth.emergency_revocation (urgency: critical)
+     โ†’ Security Team notified
+     โ†’ Audit records written (non-suppressable)
+
+  5. In-flight requests (if any):
+     Requests already dispatched to providers: allowed to complete
+     (provider callbacks authenticated separately via provider callback credential)
+     New requests from this actor: blocked immediately
+
+  6. Actor status โ†’ suspended (pending security review)
+     actor.suspended event โ†’ Platform Admin
+
+Recovery:
+  After investigation: actor cleared
+  POST /api/v1/admin/actors/{uuid}:unsuspend
+  Actor re-authenticates via Auth Provider โ†’ new session issued
+```
+
+---
+
+## 8.6 Workload Analysis โ€” Brownfield VM Classification
+
+A platform admin runs brownfield ingestion on a newly discovered VM. The
+Workload Analysis pipeline classifies it and populates the WorkloadProfile.
+
+```
+Discovery Scheduler finds: vm-legacy-0007
+  provider_entity_id: vm-legacy-0007
+  no matching DCM entity in Realized State
+
+Ingestion record created (status: INGESTED):
+  entity_uuid: vm-new-abc123
+  tenant_uuid: __transitional__
+  resource_type: Compute.VirtualMachine (inferred from provider)
+
+Workload Analysis triggered (WLA-001):
+  WorkloadProfile entity created: wla-xyz789
+  linked to: vm-new-abc123
+
+Step 2 โ€” Information Providers queried:
+  Port scan:
+    open ports: [443, 8443, 3000]
+  Process list:
+    [nginx, node, pm2, postgres-client]
+  OS metadata:
+    os: RHEL 8.6 / os_eol: 2029-05-31
+    mounts: / (50GB), /data (500GB โ€” separate partition)
+  MTA assessment:
+    containerization_score: 7
+    blockers: []
+    suggested_target: Platform.KubernetesDeployment
+    archetype: web_server
+
+Step 3 โ€” Classification (WLA-001 policy):
+  workload_archetype: web_server (confidence: high)
+  resource_type_match:
+    primary: Compute.VirtualMachine (confidence: high)
+    alternative: Platform.Container (confidence: medium โ€” MTA score 7)
+  lifecycle_recommendation:
+    dcm_lifecycle_model: standard
+    rehydration_eligible: true      โ† /data on separate partition
+    notes: "Containerization candidate per MTA score"
+
+WorkloadProfile โ†’ OPERATIONAL:
+  GET /api/v1/resources/vm-new-abc123/workload-profile
+  {
+    "workload_archetype": "web_server",
+    "resource_type_match": { "primary": "Compute.VirtualMachine", "confidence": "high" },
+    "migration_readiness": { "containerization_score": 7, "suggested_target": "Platform.KubernetesDeployment" },
+    "lifecycle_recommendation": { "rehydration_eligible": true }
+  }
+
+Ingestion advances:
+  INGESTED โ†’ ENRICHING (WorkloadProfile confidence: high โ†’ no manual review needed)
+  Tenant auto-assignment: web_server in /data subnet โ†’ Tenant: "payments-platform"
+  ENRICHING โ†’ PROMOTED โ†’ OPERATIONAL
+```
+
+---
+
+## 8.7 Scoring Model โ€” Placement Tie-Breaking
+
+Two providers both satisfy all required placement constraints. The Scoring Model
+determines which is selected.
+
+**Scenario:** Four providers qualify for a FedRAMP High VM request.
+
+```
+Request:
+  resource_type: Compute.VirtualMachine
+  data_classification: restricted
+  required_accreditations: [fedramp_high]
+  tenant: payments-platform
+
+Providers qualifying after placement require-filter:
+  Provider A: fedramp_high=โœ…  zone=US-EAST-1a  capacity=60%
+  Provider B: fedramp_high=โœ…  zone=US-EAST-1b  capacity=82%
+  Provider C: fedramp_high=โœ…  zone=US-EAST-1a  capacity=35%
+  Provider D: fedramp_high=โœ…  zone=US-EAST-1b  capacity=91%
+
+Scoring Model evaluation:
+
+Signal 1 โ€” Provider Health Score (weight: 0.30)
+  All four: status=healthy, 99.9% uptime
+  Scores: A=0.98, B=0.97, C=0.99, D=0.96
+
+Signal 2 โ€” Capacity Headroom (weight: 0.25)
+  A=0.60, B=0.82, C=0.35, D=0.91
+  (normalized โ€” higher headroom = lower risk)
+
+Signal 3 โ€” Request Risk Score (weight: 0.20)
+  Cost estimate: $180/mo โ€” medium risk
+  All providers: identical input โ†’ same score
+
+Signal 4 โ€” Policy Preference Score (weight: 0.15)
+  Placement policy prefers US-EAST-1a:
+  A=1.0 (preferred zone), B=0.7, C=1.0, D=0.7
+
+Signal 5 โ€” Accreditation Richness (weight: 0.10)
+  Provider B: fedramp_high + iso_27001 + soc2_type2 + verified_P1D โ†’ multiplier 1.0 โ†’ score: 0.82
+  Provider D: fedramp_high + soc2_type2 + verified_P7D โ†’ multiplier 0.9 โ†’ score: 0.71
+  Provider A: fedramp_high + verified_P1D โ†’ score: 0.52
+  Provider C: fedramp_high + stale_verification โ†’ multiplier 0.4 โ†’ score: 0.25
+
+Aggregate scores (weighted):
+  Provider A: 0.30ร—0.98 + 0.25ร—0.60 + 0.20ร—0.75 + 0.15ร—1.0 + 0.10ร—0.52 = 0.792
+  Provider B: 0.30ร—0.97 + 0.25ร—0.82 + 0.20ร—0.75 + 0.15ร—0.7 + 0.10ร—0.82 = 0.826
+  Provider C: 0.30ร—0.99 + 0.25ร—0.35 + 0.20ร—0.75 + 0.15ร—1.0 + 0.10ร—0.25 = 0.712
+  Provider D: 0.30ร—0.96 + 0.25ร—0.91 + 0.20ร—0.75 + 0.15ร—0.7 + 0.10ร—0.71 = 0.822
+
+Selected: Provider B (score: 0.826)
+  Note: Provider C's stale accreditation verification cost it the placement
+        despite having preferred zone.
+
+Placement audit record:
+  evaluated: [A, B, C, D]
+  scores: {A: 0.792, B: 0.826, C: 0.712, D: 0.822}
+  selected: B
+  selection_margin: 0.004 over Provider D
+  key_differentiator: "Signal 5 โ€” Provider B richer accreditation portfolio"
+```
+
+---
+
+## 8.8 Accreditation Monitor โ€” FedRAMP Status Change Detection
+
+The Accreditation Monitor polls the FedRAMP marketplace and detects that a
+provider's authorization has been downgraded mid-cycle.
+
+```
+Accreditation Monitor โ€” daily poll cycle:
+
+  Provider: eu-west-prod-1 (Service Provider)
+  Accreditation: fedramp_high
+  external_registry_id: "FR2024-0042"
+  DCM status: active
+  last_verified_at: 2026-03-30T03:00:00Z
+
+  Query:
+  GET https://marketplace.fedramp.gov/api/products?id=FR2024-0042
+
+  Response:
+  {
+    "id": "FR2024-0042",
+    "status": "In Process",          โ† was "Authorized"
+    "impact_level": "Moderate",      โ† was "High"
+    "last_updated": "2026-03-31"
+  }
+
+  Mismatch detected:
+    DCM record:   status=authorized, impact_level=high
+    External:     status=in_process, impact_level=moderate
+
+  ACM-002 applies (not immediate revocation โ€” status is not "Revoked"):
+    Accreditation: status โ†’ pending_review
+    last_verified_at: 2026-03-31T03:00:00Z
+    last_result: status_changed
+
+  Events fired:
+    accreditation.status_changed (urgency: high)
+      from_status: authorized / fedramp_high
+      to_status: in_process / fedramp_moderate
+      external_source: fedramp_marketplace
+      action_taken: pending_review
+
+  Notifications:
+    โ†’ Platform Admin (urgency: high): "FedRAMP authorization changed for eu-west-prod-1"
+    โ†’ Compliance Team
+
+  Governance Matrix impact:
+    Any new request placing fedramp_high workloads on eu-west-prod-1:
+    โ†’ Check 3 fails (required accreditation pending_review)
+    โ†’ New requests: BLOCKED until Platform Admin resolves
+
+  Active resources (already on eu-west-prod-1):
+    No immediate action (ACM-002 โ€” not immediate revocation)
+    Drift reconciliation queued for governance review
+    Provider flagged: ACCREDITATION_PENDING_REVIEW
+
+Platform Admin investigation:
+  Confirms: FedRAMP PMO initiated annual re-authorization (routine, not security event)
+  Decision: retain accreditation โ€” provider continues under monitoring
+
+  POST /api/v1/admin/accreditations/{uuid}:verify
+    { "override_reason": "Confirmed routine re-authorization โ€” PMO contact: john@fedramp.gov" }
+
+  Accreditation: status โ†’ active (manual override with audit record)
+  new_requests: unblocked
+```
diff --git a/content/docs/architecture/specifications/dcm-use-case-examples.md b/content/docs/architecture/specifications/dcm-use-case-examples.md
new file mode 100644
index 0000000..43b0ca1
--- /dev/null
+++ b/content/docs/architecture/specifications/dcm-use-case-examples.md
@@ -0,0 +1,1853 @@
+# DCM โ€” Use Case Examples
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Use Case Reference
+**Related Documents:** [Examples and Use Cases](dcm-examples.md) | [Provider Contract](../data-model/A-provider-contract.md) | [Four States](../data-model/02-four-states.md) | [Layering and Versioning](../data-model/03-layering-and-versioning.md) | [Governance Matrix](../data-model/27-governance-matrix.md) | [Scoring Model](../data-model/29-scoring-model.md)
+
+> This document contains worked examples for every DCM provider type, data model, and specification
+> area not covered in [dcm-examples.md](dcm-examples.md). Examples use consistent fictitious actors,
+> tenants, and providers throughout for cross-reference coherence.
+
+---
+
+## Shared Context
+
+All examples reference these fictitious entities:
+
+```
+Tenants:
+  payments-bu         UUID: ten-pay-001   Business unit: card payment processing
+  web-platform-bu     UUID: ten-web-001   Business unit: customer-facing web
+  platform-team       UUID: ten-plt-001   Platform engineering (admin)
+
+Actors:
+  alice@corp.example  UUID: act-alice-001  Developer, payments-bu
+  bob@corp.example    UUID: act-bob-001    Platform engineer, platform-team
+  svc-pipeline@corp   UUID: act-svc-001    CI/CD service account
+
+Providers:
+  vmware-prod         UUID: pvd-vm-001     Service Provider โ€” Compute.VirtualMachine
+  netbox-prod         UUID: pvd-net-001    Information Provider โ€” Network.IPAddress
+  vault-prod          UUID: pvd-vlt-001    Credential Provider
+  freeipa-prod        UUID: pvd-ipa-001    Auth Provider โ€” FreeIPA/LDAP
+  ceph-prod           UUID: pvd-cph-001    Storage Provider โ€” Realized State snapshots
+  rabbitmq-prod       UUID: pvd-rmq-001    Message Bus Provider
+  servicenow-prod     UUID: pvd-sn-001     ITSM Provider โ€” ServiceNow
+  webapp-meta         UUID: pvd-wam-001    Meta Provider โ€” ApplicationStack.WebApp
+
+Data Centers / Zones:
+  dc-west-1 / zone-a    Primary production zone
+  dc-east-1 / zone-b    DR zone
+```
+
+---
+
+# Section 1 โ€” Data Model Examples
+
+## 1.1 The Four States โ€” VM Lifecycle from Request to Decommission
+
+This example traces a single `Compute.VirtualMachine` entity through all four states, from the consumer's original intent through decommission.
+
+**Step 1: Consumer submits a request โ€” Intent State created**
+
+```
+POST /api/v1/requests
+Authorization: Bearer 
+
+{
+  "catalog_item_uuid": "cat-vm-standard",
+  "fields": {
+    "vm_name": "payments-api-03",
+    "cpu_cores": 8,
+    "ram_gb": 32,
+    "os": "RHEL 9",
+    "zone": "dc-west-1/zone-a",
+    "purpose": "payments API service node"
+  }
+}
+```
+
+DCM immediately writes the Intent State record โ€” immutable from this moment forward:
+
+```yaml
+intent_state_record:
+  intent_uuid: int-pay-api-03
+  entity_uuid: ent-vm-pay-03      # assigned on first request
+  tenant_uuid: ten-pay-001
+  actor_uuid:  act-alice-001
+  recorded_at: 2026-03-31T09:00:00Z
+
+  # Verbatim consumer input โ€” never altered by policy
+  raw_intent:
+    catalog_item_uuid: cat-vm-standard
+    fields:
+      vm_name: payments-api-03
+      cpu_cores: 8
+      ram_gb: 32
+      os: RHEL 9
+      zone: dc-west-1/zone-a
+      purpose: payments API service node
+```
+
+**Step 2: Layer assembly and policy evaluation โ€” Requested State built**
+
+DCM's Request Payload Processor assembles layers in precedence order:
+
+```yaml
+# Layer 1 โ€” Base (org-wide standards)
+dcm:
+  network:
+    dns_suffix: .internal.corp.example
+  security:
+    selinux: enforcing
+    firewall: enabled
+
+# Layer 2 โ€” Data Center (dc-west-1)
+location:
+  datacenter: dc-west-1
+  rack_prefix: rack-w1
+  power_domain: ups-west-primary
+
+# Layer 3 โ€” Payments BU Service Layer
+compliance:
+  pci_dss: true
+  network_segment: payments-dmz
+  encryption_at_rest: required
+  log_retention_days: 365
+
+# Layer 4 โ€” Request Layer (consumer-supplied fields)
+vm:
+  name: payments-api-03
+  cpu_cores: 8
+  ram_gb: 32
+  os_image: rhel-9-latest-approved   # enriched from "RHEL 9" by Transformation policy
+  zone: dc-west-1/zone-a
+```
+
+GateKeeper policy fires: `payments-dmz` placement requires PCI DSS accreditation on the provider โ†’ `pvd-vm-001` has active PCI DSS accreditation โ†’ PASS.
+
+Transformation policy fires: `cpu_cores: 8` in payments zone โ†’ sets `cpu_pinning: true` per PCI DSS performance isolation requirement.
+
+Requested State written (the fully assembled, policy-approved dispatch payload):
+
+```yaml
+requested_state_record:
+  requested_state_uuid: req-pay-api-03
+  entity_uuid: ent-vm-pay-03
+  intent_uuid: int-pay-api-03
+  tenant_uuid: ten-pay-001
+  provider_uuid: pvd-vm-001        # placement resolved
+  assembled_at: 2026-03-31T09:00:05Z
+
+  payload:
+    vm_name: payments-api-03
+    cpu_cores: 8
+    cpu_pinning: true              # injected by Transformation policy
+    ram_gb: 32
+    os_image: rhel-9-latest-approved
+    network_segment: payments-dmz
+    encryption_at_rest: true
+    log_retention_days: 365
+    zone: dc-west-1/zone-a
+    selinux: enforcing
+```
+
+**Step 3: Provider realizes the request โ€” Realized State written**
+
+Provider confirms provisioning and returns:
+
+```yaml
+realized_state_snapshot:
+  realized_state_uuid: rlz-pay-api-03-v1
+  entity_uuid: ent-vm-pay-03
+  source_requested_state_uuid: req-pay-api-03
+  realized_at: 2026-03-31T09:02:47Z
+  lifecycle_state: OPERATIONAL
+
+  # Provider-supplied realization details (added to assembled payload)
+  provider_data:
+    resource_id: vm-4821                  # provider's internal ID
+    hypervisor_host: esxi-w1-rack3-b04
+    mac_address: 00:50:56:ab:cd:ef
+    assigned_ip: 10.42.18.55
+    boot_time: 2026-03-31T09:02:31Z
+    bios_uuid: 4201abcd-0000-0000-0000-000000000001
+```
+
+**Step 4: Discovery confirms state โ€” Discovered State updated**
+
+Discovery Scheduler polls `pvd-vm-001` 24 hours later:
+
+```yaml
+discovered_state_record:
+  entity_uuid: ent-vm-pay-03
+  discovered_at: 2026-04-01T09:05:00Z
+  provider_uuid: pvd-vm-001
+  resource_id: vm-4821
+
+  discovered_fields:
+    cpu_cores: 8
+    ram_gb: 32
+    os_image: rhel-9-latest-approved
+    cpu_pinning: true
+    power_state: running
+    uptime_hours: 24.1
+    disk_usage_pct: 12
+
+# Drift check: all discovered fields match Realized State โ†’ no drift
+# drift_record: null
+```
+
+**Step 5: Decommission**
+
+```
+DELETE /api/v1/resources/ent-vm-pay-03
+
+# Entity enters DECOMMISSIONING โ†’ provider deletes VM โ†’ DECOMMISSIONED
+# Intent, Requested, and Realized State records are retained for audit
+# Discovered State cleared
+```
+
+---
+
+## 1.2 Layer Assembly โ€” Full Walkthrough
+
+A request for a web server in the payments zone demonstrates how six layers compose into a single dispatch payload.
+
+**Layers in precedence order (lowest โ†’ highest):**
+
+```yaml
+# Layer 0: Base entity (org-wide defaults)
+---
+handle: layers/base/compute/vm-standard
+version: "3.1.0"
+type: base_entity
+data:
+  selinux: enforcing
+  firewall: enabled
+  ntp_server: ntp.internal.corp.example
+  dns_suffix: .internal.corp.example
+  monitoring_agent: node_exporter
+  log_collector: filebeat
+  log_destination: logs.corp.example:5044
+```
+
+```yaml
+# Layer 1: Data Center โ€” dc-west-1
+---
+handle: layers/dc/west-1
+version: "2.0.0"
+parent: layers/base/compute/vm-standard@3.1.0
+type: layer_entity
+data:
+  location:
+    datacenter: dc-west-1
+    region: us-west
+    rack_prefix: rack-w1
+    power_domain: ups-west-primary
+  network:
+    gateway: 10.42.0.1
+    dns: [10.42.0.53, 10.42.0.54]
+```
+
+```yaml
+# Layer 2: Zone โ€” dc-west-1/zone-a (DMZ)
+---
+handle: layers/zone/west-1-zone-a-dmz
+version: "1.5.0"
+parent: layers/dc/west-1@2.0.0
+type: layer_entity
+data:
+  network:
+    segment: dmz-payments
+    vlan: 142
+    firewall_policy: payments-dmz-policy
+  compliance:
+    pci_dss: true
+    log_retention_days: 365
+```
+
+```yaml
+# Layer 3: Payments BU service layer
+---
+handle: layers/service/payments-bu/compute
+version: "1.2.0"
+parent: layers/zone/west-1-zone-a-dmz@1.5.0
+type: layer_entity
+data:
+  security:
+    cpu_pinning: true            # PCI isolation
+    encryption_at_rest: required
+  backup:
+    enabled: true
+    frequency: daily
+    retention: 30d
+```
+
+```yaml
+# Layer 4: Web server service layer
+---
+handle: layers/service/web-platform/nginx-config
+version: "2.0.0"
+parent: layers/service/payments-bu/compute@1.2.0
+type: layer_entity
+data:
+  software:
+    packages: [nginx, certbot]
+    nginx_config_ref: git://configs/nginx/standard.conf@v3
+  ports_open: [80, 443]
+```
+
+```yaml
+# Layer 5: Request layer (consumer-supplied)
+---
+type: request_layer
+data:
+  vm_name: payments-web-07
+  cpu_cores: 4
+  ram_gb: 16
+  os_image: rhel-9-latest-approved
+  zone: dc-west-1/zone-a
+```
+
+**Assembled payload (higher layers win on conflict):**
+
+```yaml
+# Assembled dispatch payload sent to pvd-vm-001
+assembled_payload:
+  vm_name: payments-web-07         # from request layer
+  cpu_cores: 4                     # from request layer
+  ram_gb: 16                       # from request layer
+  os_image: rhel-9-latest-approved # from request layer
+  selinux: enforcing               # from base (not overridden)
+  firewall: enabled                # from base (not overridden)
+  ntp_server: ntp.internal.corp.example
+  dns_suffix: .internal.corp.example
+  monitoring_agent: node_exporter  # from base
+  log_collector: filebeat          # from base
+  log_destination: logs.corp.example:5044
+  location:
+    datacenter: dc-west-1          # from dc layer
+    region: us-west
+    rack_prefix: rack-w1
+  network:
+    segment: dmz-payments          # from zone layer (overrides dc layer)
+    vlan: 142
+    gateway: 10.42.0.1
+    dns: [10.42.0.53, 10.42.0.54]
+    firewall_policy: payments-dmz-policy
+  compliance:
+    pci_dss: true                  # from zone layer
+    log_retention_days: 365
+  security:
+    cpu_pinning: true              # from payments BU layer
+    encryption_at_rest: required
+  backup:
+    enabled: true
+    frequency: daily
+    retention: 30d
+  software:
+    packages: [nginx, certbot]     # from web layer
+    nginx_config_ref: git://configs/nginx/standard.conf@v3
+  ports_open: [80, 443]
+
+provenance:
+  # Every field carries its source layer and the actor who set it
+  - field: cpu_pinning
+    value: true
+    source: layers/service/payments-bu/compute@1.2.0
+    set_by: bob@corp.example
+    reason: PCI DSS compute isolation requirement
+  - field: log_retention_days
+    value: 365
+    source: layers/zone/west-1-zone-a-dmz@1.5.0
+    set_by: compliance-team@corp.example
+    reason: PCI DSS requirement 10.5
+```
+
+---
+
+## 1.3 Governance Matrix โ€” PHI Data Request Evaluation
+
+A developer in the `web-platform-bu` tenant requests a VM to host a new microservice that will process Protected Health Information (PHI). DCM evaluates the four-axis governance matrix.
+
+**The request:**
+
+```yaml
+subject: actor act-alice-001         # Axis 1: WHO
+  tenant: ten-web-001
+  mfa_verified: true
+  session_risk_score: 0.12           # low โ€” recent login, known device
+
+data: payload.data_classifications   # Axis 2: WHAT
+  contains: [phi]                    # request fields marked phi by policy
+
+target: pvd-vm-001                   # Axis 3: WHERE
+  sovereignty_zone: us-commercial
+  accreditations: [iso_27001, soc2_type2]   # no hipaa_baa
+
+context:                             # Axis 4: UNDER WHAT CONDITIONS
+  profile: prod
+  time: 2026-03-31T14:00:00Z         # business hours
+  request_risk_score: 68             # elevated โ€” PHI + no BAA
+```
+
+**Axis 1 (Subject):** Actor is authenticated, MFA verified, session risk low โ†’ PASS
+
+**Axis 2 (Data):** Payload contains PHI classification โ†’ triggers HIPAA rules
+
+**Axis 3 (Target):** `pvd-vm-001` is checked for HIPAA BAA accreditation โ†’ **no BAA registered** โ†’ FAIL
+
+**Axis 4 (Context):** `prod` profile requires accreditation completeness for PHI โ†’ DENY
+
+**Governance Matrix decision:**
+
+```yaml
+governance_decision:
+  outcome: DENY
+  rule_matched: "PHI data requires HIPAA BAA on target provider"
+  axis_failing: target
+  detail:
+    provider_uuid: pvd-vm-001
+    missing_accreditation: hipaa_baa
+    required_for: phi classification in payload
+  remediation:
+    option_1: "Submit BAA for pvd-vm-001 and await activation"
+    option_2: "Route to a provider with active HIPAA BAA accreditation"
+    option_3: "Remove PHI from this service's data scope"
+
+# Request is blocked before dispatch โ€” no Requested State written
+# Consumer receives:
+{
+  "error": {
+    "code": "GOVERNANCE_DENIED",
+    "message": "PHI data cannot be placed on pvd-vm-001 โ€” HIPAA BAA not present",
+    "remediation": "Contact your platform admin to add a BAA for this provider"
+  }
+}
+```
+
+---
+
+## 1.4 Scoring Model โ€” Risk Score and Placement Decision
+
+Three providers are candidates for a Tier 1 VM request. DCM calculates a risk score and routes to the approval tier.
+
+**Signals for this request:**
+
+```yaml
+# Signal 1: Operational GateKeeper Score
+# No GateKeeper policies fired โ†’ score: 0 (lowest risk)
+signal_1: 0
+
+# Signal 2: Policy Completeness Score
+# All policies evaluated; no shadow-only policies for this resource type โ†’ score: 0
+signal_2: 0
+
+# Signal 3: Actor Risk History Score
+# alice@corp.example โ€” no failed requests, no policy violations in 90 days โ†’ score: 5
+signal_3: 5
+
+# Signal 4: Tenant Quota Pressure Score
+# payments-bu is at 62% of VM quota โ†’ moderate pressure โ†’ score: 18
+signal_4: 18
+
+# Signal 5: Provider Accreditation Richness (for placement tie-breaking)
+# pvd-vm-001: iso_27001(20) + soc2_type2(20) + pci_dss(25) = 65 โ†’ external verified P1D โ†’ ร—1.0
+# pvd-vm-002: iso_27001(20) + soc2_type2(20) = 40 โ†’ external verified P7D โ†’ ร—0.9 = 36
+# pvd-vm-003: soc2_type2(20) = 20 โ†’ stale verification โ†’ ร—0.4 = 8
+
+aggregate_risk_score:
+  formula: "(signal_1 ร— 0.35) + (signal_2 ร— 0.25) + (signal_3 ร— 0.20) + (signal_4 ร— 0.15) + (signal_5_inverse ร— 0.05)"
+  value: 21
+  # 21 โ†’ STANDARD tier (threshold: 0-39 = STANDARD, 40-69 = ELEVATED, 70+ = CRITICAL)
+```
+
+**Authority Tier routing:**
+
+```yaml
+# Risk score 21 โ†’ STANDARD tier โ†’ no additional approval required
+# Placement: pvd-vm-001 wins (highest accreditation richness score: 65)
+
+placement_decision:
+  provider_uuid: pvd-vm-001
+  approval_tier: STANDARD
+  auto_approved: true
+  rationale: "Risk score 21 < 40 threshold; pvd-vm-001 highest accreditation richness"
+```
+
+**Contrast โ€” the same request from a high-risk actor:**
+
+```yaml
+# If signal_3 (actor risk) = 45 (recent policy violations):
+aggregate_risk_score: 52   # โ†’ ELEVATED tier
+
+placement_decision:
+  provider_uuid: pvd-vm-001
+  approval_tier: ELEVATED
+  auto_approved: false
+  requires_approval_from:
+    - role: tenant_admin            # payments-bu tenant admin
+  approval_deadline: PT4H
+```
+
+---
+
+## 1.5 Authority Tier Model โ€” Multi-Tier Approval Routing
+
+A developer requests 200 VMs simultaneously (bulk deployment for a load test). Risk score crosses the CRITICAL threshold and requires multi-tier sign-off.
+
+```yaml
+# Bulk request: 200 ร— Compute.VirtualMachine for load-test-bu tenant
+# Signal 4 (quota pressure): 200 VMs = 95% of quota โ†’ score: 38
+# Signal 3 (actor history): svc-pipeline@corp โ€” automated, clean history โ†’ 0
+# Aggregate risk score: 58 โ†’ ELEVATED
+
+# But: 200 instances triggers an additional policy:
+#   "bulk_request_over_100 โ†’ escalate to CRITICAL tier"
+# GateKeeper fires and elevates: score_override: CRITICAL
+
+authority_tier_routing:
+  risk_score_raw: 58
+  score_override: CRITICAL       # GateKeeper escalation
+  tier_applied: CRITICAL
+  approval_chain:
+    - step: 1
+      approver_role: tenant_admin
+      tenant_uuid: ten-web-001
+      deadline: PT2H
+      status: pending
+
+    - step: 2
+      approver_role: platform_admin
+      deadline: PT4H             # starts after step 1 approved
+      status: waiting
+
+    - step: 3
+      approver_role: ciso_delegate
+      deadline: PT8H             # starts after step 2 approved
+      status: waiting
+
+  # If any step times out โ†’ request enters APPROVAL_EXPIRED state
+  # Policy: NOTIFY_AND_WAIT โ†’ Compliance Team paged
+```
+
+**Tenant admin approves (step 1):**
+
+```
+POST /api/v1/approvals/apv-bulk-load-001
+Authorization: Bearer 
+
+{
+  "decision": "approve",
+  "rationale": "Authorized load test โ€” signed off by VP Engineering"
+}
+
+# โ†’ step 2 notification fires to platform admin
+```
+
+---
+
+## 1.6 Entity Relationships โ€” Composite Web Service
+
+A `Compute.VirtualMachine`, `Network.IPAddress`, and `Security.FirewallRule` are related as a composite web service entity.
+
+```yaml
+# Three entities with explicit relationships
+entities:
+  - entity_uuid: ent-vm-pay-03       # the VM
+    resource_type: Compute.VirtualMachine
+    tenant_uuid: ten-pay-001
+
+  - entity_uuid: ent-ip-pay-03       # the IP assigned to the VM
+    resource_type: Network.IPAddress
+    tenant_uuid: ten-pay-001
+
+  - entity_uuid: ent-fw-pay-03       # the firewall rule permitting traffic
+    resource_type: Security.FirewallRule
+    tenant_uuid: ten-pay-001
+
+relationships:
+  - relationship_uuid: rel-001
+    from_entity: ent-vm-pay-03
+    to_entity: ent-ip-pay-03
+    relationship_type: assigned_to
+    cardinality: one_to_one
+    required_for_delivery: true      # VM cannot be OPERATIONAL without an IP
+
+  - relationship_uuid: rel-002
+    from_entity: ent-vm-pay-03
+    to_entity: ent-fw-pay-03
+    relationship_type: protected_by
+    cardinality: one_to_many
+    required_for_delivery: false     # VM can be OPERATIONAL; rule is operational hygiene
+```
+
+**Impact of decommissioning the VM:**
+
+```yaml
+# Consumer: DELETE /api/v1/resources/ent-vm-pay-03
+# DCM evaluates relationship graph before dispatch:
+
+decommission_impact_analysis:
+  entity: ent-vm-pay-03
+  dependents:
+    - entity_uuid: ent-ip-pay-03
+      relationship: assigned_to
+      impact: IP address released โ†’ available for reassignment
+      action: decommission_with_parent
+
+    - entity_uuid: ent-fw-pay-03
+      relationship: protected_by
+      impact: Firewall rule becomes orphaned โ€” no host to protect
+      action: notify_admin           # rule not auto-deleted; may apply to other VMs
+
+  consumer_presented:
+    "Decommissioning this VM will release IP ent-ip-pay-03. Firewall rule
+     ent-fw-pay-03 will become orphaned and require manual review."
+```
+
+---
+
+## 1.7 Universal Groups โ€” Tenant, Resource Group, and Cross-Tenant Sharing
+
+**Setup: payments-bu tenant with a resource group and a shared database**
+
+```yaml
+# Tenant (group_class: tenant_boundary)
+dcm_group:
+  uuid: ten-pay-001
+  group_class: tenant_boundary
+  handle: tenants/payments-bu
+  display_name: Payments Business Unit
+  members:
+    - { type: actor, uuid: act-alice-001, role: member }
+    - { type: actor, uuid: act-pay-admin, role: tenant_admin }
+
+# Resource group within the tenant (group_class: resource_group)
+dcm_group:
+  uuid: rg-pay-api-servers
+  group_class: resource_group
+  handle: tenants/payments-bu/groups/api-servers
+  parent_tenant_uuid: ten-pay-001
+  display_name: Payments API Servers
+  members:
+    - { type: entity, uuid: ent-vm-pay-01, role: member }
+    - { type: entity, uuid: ent-vm-pay-02, role: member }
+    - { type: entity, uuid: ent-vm-pay-03, role: member }
+```
+
+**Cross-tenant sharing: payments-bu shares a read-only DB with web-platform-bu**
+
+```yaml
+# Authorization record (cross-tenant read access)
+cross_tenant_authorization:
+  uuid: xta-db-share-001
+  grantor_tenant: ten-pay-001
+  grantee_tenant: ten-web-001
+  scope:
+    entity_uuids: [ent-db-pay-analytics]
+    permissions: [read]             # not write or decommission
+  valid_until: 2026-12-31T23:59:59Z
+  approved_by: act-pay-admin
+  governance_matrix_check: ALLOW    # PHI not in this DB; cross-tenant read permitted
+
+# web-platform-bu can now query:
+GET /api/v1/resources/ent-db-pay-analytics
+# โ†’ 200 OK (authorized via cross-tenant grant)
+
+DELETE /api/v1/resources/ent-db-pay-analytics
+# โ†’ 403 Forbidden (write not in grant scope)
+```
+
+---
+
+## 1.8 Scheduled Requests and Maintenance Windows
+
+**Scenario: OS patch deployment during an approved maintenance window**
+
+**Step 1: Platform admin defines maintenance window**
+
+```
+POST /api/v1/admin/maintenance-windows
+
+{
+  "display_name": "Q2 OS Patching โ€” West Zone",
+  "starts_at": "2026-04-06T02:00:00Z",
+  "ends_at":   "2026-04-06T06:00:00Z",
+  "scope": {
+    "tenant_uuids": ["ten-pay-001", "ten-web-001"],
+    "resource_types": ["Compute.VirtualMachine"]
+  },
+  "change_freeze": false
+}
+
+Response: { "window_uuid": "mw-q2-patch-001" }
+```
+
+**Step 2: CI/CD pipeline submits a deferred patching request**
+
+```
+POST /api/v1/requests
+
+{
+  "catalog_item_uuid": "cat-os-patch-rhel9",
+  "fields": {
+    "target_entity_uuid": "ent-vm-pay-03",
+    "patch_baseline": "rhel9-2026-q2",
+    "pre_patch_snapshot": true
+  },
+  "scheduled_at": "2026-04-06T02:15:00Z",
+  "maintenance_window_uuid": "mw-q2-patch-001"
+}
+
+Response:
+{
+  "name": "/api/v1/operations/req-patch-pay-03",
+  "done": false,
+  "metadata": {
+    "stage": "SCHEDULED",
+    "scheduled_at": "2026-04-06T02:15:00Z",
+    "resource_uuid": "ent-vm-pay-03"
+  }
+}
+```
+
+**Step 3: At 02:15Z โ€” window opens, request executes**
+
+```yaml
+# Orchestrator fires at scheduled_at
+# Entity: OPERATIONAL โ†’ UPDATING
+# Process.OSPatch entity created, linked to VM entity
+# Provider receives:
+
+dispatch_payload:
+  entity_uuid: ent-vm-pay-03
+  operation: patch
+  patch_baseline: rhel9-2026-q2
+  pre_patch_snapshot: true
+  maintenance_window_uuid: mw-q2-patch-001
+
+# Provider: takes snapshot, applies patches, reboots, validates
+# Callback received: OPERATIONAL
+# New Realized State written with updated os_patch_level
+# Process.OSPatch entity: DECOMMISSIONED (process complete)
+```
+
+---
+
+## 1.9 Request Dependency Graph โ€” Multi-Resource Compound Provisioning
+
+A CI/CD pipeline provisions three resources with strict ordering: DB first, then app server, then load balancer (which needs both IPs).
+
+**Step 1: Submit the dependency group**
+
+```
+POST /api/v1/request-groups
+
+{
+  "display_name": "payments-api-stack-v2 rollout",
+  "requests": [
+    {
+      "client_id": "db",
+      "catalog_item_uuid": "cat-postgresql-ha",
+      "fields": { "db_name": "payments_v2", "storage_gb": 500 }
+    },
+    {
+      "client_id": "app",
+      "catalog_item_uuid": "cat-vm-standard",
+      "fields": { "vm_name": "payments-api-04", "cpu_cores": 8 },
+      "depends_on": ["db"],
+      "wait_for": "OPERATIONAL",
+      "inject_from_dependency": {
+        "db": { "db_host": "$.realized.assigned_ip" }
+      }
+    },
+    {
+      "client_id": "lb",
+      "catalog_item_uuid": "cat-haproxy-config",
+      "fields": { "pool_name": "payments-api-pool" },
+      "depends_on": ["app"],
+      "wait_for": "OPERATIONAL",
+      "inject_from_dependency": {
+        "app": { "backend_ips": "$.realized.assigned_ip" }
+      }
+    }
+  ]
+}
+
+Response: { "group_uuid": "grp-pay-stack-v2" }
+```
+
+**Step 2: Execution sequence**
+
+```
+T+0s    DB request dispatched โ†’ pvd-pg-001
+T+90s   DB realized โ†’ assigned_ip: 10.42.18.100
+        DB field injection into app request:  db_host: 10.42.18.100
+T+90s   App request dispatched โ†’ pvd-vm-001 (with db_host injected)
+T+150s  App realized โ†’ assigned_ip: 10.42.18.55
+        App field injection into lb request: backend_ips: [10.42.18.55]
+T+150s  LB request dispatched โ†’ pvd-lb-001 (with backend_ips injected)
+T+165s  LB realized โ†’ all three OPERATIONAL
+        Group status: COMPLETE
+```
+
+**Query group status at any point:**
+
+```
+GET /api/v1/request-groups/grp-pay-stack-v2
+
+{
+  "group_uuid": "grp-pay-stack-v2",
+  "status": "IN_PROGRESS",
+  "requests": [
+    { "client_id": "db",  "status": "OPERATIONAL", "entity_uuid": "ent-db-pay-v2" },
+    { "client_id": "app", "status": "PROVISIONING", "entity_uuid": "ent-vm-pay-04" },
+    { "client_id": "lb",  "status": "PENDING_DEPENDENCY", "entity_uuid": null }
+  ]
+}
+```
+
+---
+
+## 1.10 Workload Analysis โ€” Discovered VM Classified and Ingested
+
+A VM exists in the data center that was never provisioned through DCM. Discovery finds it; Workload Analysis classifies it; ingestion brings it under lifecycle management.
+
+**Step 1: Discovery Scheduler finds unknown VM**
+
+```yaml
+discovered_state_record:
+  entity_uuid: null                  # no DCM UUID yet โ€” new entity
+  provider_uuid: pvd-vm-001
+  provider_entity_id: vm-9917        # provider's internal ID
+  discovered_at: 2026-03-31T08:00:00Z
+
+  discovered_fields:
+    ip_address: 10.42.22.77
+    hostname: legacy-payments-batch
+    os: RHEL 7.9
+    cpu_cores: 4
+    ram_gb: 8
+    running_processes: [java, cron, rsync]
+    open_ports: [8080, 22]
+    disk_gb: 200
+
+# DCM assigns a provisional entity_uuid and writes INGESTED state
+entity_uuid: ent-disc-9917
+lifecycle_state: INGESTED
+tenant_uuid: __transitional__
+```
+
+**Step 2: Workload Analysis fires automatically**
+
+```yaml
+# Analysis.WorkloadProfile entity created
+workload_profile_entity:
+  entity_uuid: ent-wla-9917
+  resource_type: Analysis.WorkloadProfile
+  subject_entity_uuid: ent-disc-9917
+  lifecycle_state: OPERATIONAL
+
+  classification:
+    resource_type_match:
+      primary: Compute.VirtualMachine
+      confidence: high
+
+    workload_archetype:
+      type: batch_processor
+      confidence: medium
+      signals:
+        - "cron present: scheduled job execution"
+        - "java process: JVM-based batch framework"
+        - "rsync present: data sync pattern"
+        - "port 8080: likely management API, not customer-facing"
+        - "RHEL 7.9: end-of-life โ€” upgrade candidate"
+
+    migration_readiness:
+      containerization_score: 4      # low โ€” stateful batch job, not container-friendly
+      blockers:
+        - "Stateful data in /data โ€” requires persistent volume mapping"
+        - "RHEL 7.9 runtime โ€” requires migration to RHEL 9 base image first"
+      suggested_target: Compute.VirtualMachine    # stay as VM, upgrade OS
+
+    lifecycle_recommendation:
+      dcm_lifecycle_model: standard
+      rehydration_eligible: true
+      notes: "Application on /opt, data on /data โ€” static replace eligible after OS upgrade"
+```
+
+**Step 3: Platform admin enriches and promotes**
+
+```
+PATCH /api/v1/resources/ent-disc-9917
+{
+  "tenant_uuid": "ten-pay-001",
+  "display_name": "Payments Batch Processor (Legacy)",
+  "cost_center": "CC-PAY-OPS",
+  "owner": { "actor_uuid": "act-pay-admin" }
+}
+
+# Auto-assignment policy fires: batch_processor archetype โ†’ assign to 'batch-workloads' resource group
+# Entity promoted: INGESTED โ†’ ENRICHING โ†’ PROMOTED โ†’ OPERATIONAL
+# Entity now under full DCM lifecycle management
+```
+
+---
+
+## 1.11 Accreditation Monitor โ€” FedRAMP Verification and Mid-Cycle Revocation
+
+**Setup: Provider with FedRAMP High accreditation**
+
+```yaml
+accreditation:
+  artifact_metadata:
+    uuid: acc-fr-high-001
+    handle: accreditations/providers/pvd-vm-fed-001/fedramp-high
+    version: "1.0.0"
+    status: active
+
+  subject_uuid: pvd-vm-fed-001
+  framework: fedramp_high
+  accreditation_type: regulatory_certification
+  accreditor:
+    name: FedRAMP PMO
+    type: government
+
+  issued_at: 2025-06-01T00:00:00Z
+  valid_until: 2026-06-01T00:00:00Z
+  renewal_warning_before: P90D
+
+  external_registry_id: FR2025-0088    # FedRAMP Marketplace ID
+
+  verification:
+    tier: external_registry
+    registry_api:
+      provider: fedramp
+      lookup_key: FR2025-0088
+      poll_interval: P1D
+      last_checked_at: 2026-03-30T09:00:00Z
+      last_result: confirmed_active
+    stale_after: P3D
+    stale_action: escalate           # sovereign profile โ€” escalate not warn
+```
+
+**Day 1: Normal verification cycle**
+
+```yaml
+# Accreditation Monitor polls marketplace.fedramp.gov
+# GET /api/products?id=FR2025-0088
+# Response: { "status": "Authorized", "impact_level": "High" }
+
+# Monitor fires:
+event:
+  type: accreditation.verified
+  urgency: low
+  payload:
+    accreditation_uuid: acc-fr-high-001
+    framework: fedramp_high
+    registry: fedramp_marketplace
+    checked_at: 2026-03-31T09:00:00Z
+
+# DCM updates: last_verified_at, last_result: confirmed_active
+# Scoring Model Signal 5: weight 40 ร— multiplier 1.0 = 40 (full weight)
+```
+
+**Day 47: FedRAMP PMO revokes the authorization mid-cycle**
+
+```yaml
+# Monitor polls: GET /api/products?id=FR2025-0088
+# Response: { "status": "Revoked", "revocation_date": "2026-05-17" }
+
+# status = Revoked โ†’ immediate revocation (no admin confirmation required โ€” ACM-002)
+event:
+  type: accreditation.status_changed
+  urgency: critical
+  payload:
+    accreditation_uuid: acc-fr-high-001
+    from_status: authorized
+    to_status: revoked
+    external_source: fedramp_marketplace
+    action_taken: immediate_revocation
+
+# DCM immediately:
+# 1. Sets accreditation status โ†’ revoked
+# 2. Fires Accreditation Gap for pvd-vm-fed-001
+# 3. All active requests targeting pvd-vm-fed-001 with sovereign/fedramp data โ†’ SUSPENDED
+# 4. Platform Admin + Compliance Team paged (urgency: critical, non-suppressable)
+# 5. Recovery Policy: NOTIFY_AND_WAIT (sovereign profile default)
+
+accreditation_gap_record:
+  provider_uuid: pvd-vm-fed-001
+  required_framework: fedramp_high
+  gap_type: revoked
+  severity: critical
+  affected_entity_uuids: [ent-vm-fed-01, ent-vm-fed-02, ent-vm-fed-07]
+  policy_response: NOTIFY_AND_WAIT
+```
+
+---
+
+## 1.12 Session Revocation โ€” Security Event Response
+
+A developer's laptop is reported stolen at 14:30. The security team needs to immediately terminate all active DCM sessions for that actor.
+
+**Step 1: Security team triggers emergency revocation**
+
+```
+POST /api/v1/admin/actors/act-alice-001/sessions:revoke-all
+Authorization: Bearer 
+
+{
+  "revocation_reason": "SECURITY_EVENT",
+  "detail": "Laptop reported stolen โ€” device UUID: dev-alice-macbook-001",
+  "audit_reference": "INC-2026-0847"
+}
+```
+
+**Step 2: DCM processes emergency revocation**
+
+```yaml
+# All active sessions for act-alice-001 identified: [sess-alice-001, sess-alice-002]
+# Session tokens added to Revocation Registry immediately
+# (Zero-latency: registry checked on every API call inbound)
+
+revocation_registry_entries:
+  - token_jti: jwt-alice-sess-001
+    revoked_at: 2026-03-31T14:30:07Z
+    reason: SECURITY_EVENT
+    actor_uuid: act-alice-001
+
+  - token_jti: jwt-alice-sess-002
+    revoked_at: 2026-03-31T14:30:07Z
+    reason: SECURITY_EVENT
+    actor_uuid: act-alice-001
+
+event:
+  type: security.session_revoked
+  urgency: critical
+  payload:
+    actor_uuid: act-alice-001
+    sessions_revoked: 2
+    reason: SECURITY_EVENT
+    incident_ref: INC-2026-0847
+```
+
+**Step 3: In-flight request intercepted**
+
+```yaml
+# Alice had just submitted a request at 14:30:05 โ€” 2 seconds before revocation
+# Request was in POLICY_EVALUATION stage
+# Orchestrator checks Revocation Registry before dispatch:
+
+check_result: token jwt-alice-sess-001 in revocation registry
+action: ABORT_REQUEST
+request_status: CANCELLED
+reason: "Actor session revoked during request processing"
+# Resource NOT provisioned โ€” safe state
+```
+
+**Step 4: Next login attempt fails cleanly**
+
+```
+GET /api/v1/resources
+Authorization: Bearer jwt-alice-sess-001
+
+โ†’ 401 Unauthorized
+{
+  "error": "TOKEN_REVOKED",
+  "message": "Session has been administratively revoked. Contact your platform admin."
+}
+```
+
+---
+
+# Section 2 โ€” Provider Interaction Examples
+
+## 2.5 Auth Provider โ€” FreeIPA Integration
+
+**Registration:**
+
+```yaml
+auth_provider_registration:
+  artifact_metadata:
+    uuid: pvd-ipa-001
+    handle: providers/auth/corporate-freeipa
+    version: "1.0.0"
+    status: active
+    owned_by: { display_name: Platform Team }
+
+  name: Corporate FreeIPA
+  description: Primary enterprise directory โ€” FreeIPA with Kerberos
+
+  capabilities:
+    authentication: true
+    mfa: false                       # Kerberos SSO, MFA handled by RHSSO layer
+    group_membership: true
+    role_mapping: true
+
+  protocol: ldap
+  endpoint: ldaps://ipa.corp.example:636
+  bind_credential_uuid: crd-ipa-bind-001   # stored in Credential Provider
+
+  group_mapping:
+    # FreeIPA groups โ†’ DCM roles
+    - ipa_group: dcm-platform-admins
+      dcm_role: platform_admin
+    - ipa_group: dcm-tenant-payments
+      dcm_role: tenant_member
+      tenant_uuid: ten-pay-001
+    - ipa_group: dcm-tenant-web
+      dcm_role: tenant_member
+      tenant_uuid: ten-web-001
+
+  health_check:
+    endpoint: /health
+    method: ldap_bind_check
+```
+
+**Authentication flow:**
+
+```
+1. alice@corp.example submits credentials to DCM Consumer API
+2. DCM forwards to pvd-ipa-001: LDAP bind as alice@corp.example
+3. FreeIPA validates credentials โ†’ success
+4. DCM queries: memberOf โ†’ [dcm-tenant-payments, dcm-team-api-devs]
+5. Group mapping applied โ†’ alice gets: tenant_member(ten-pay-001)
+6. DCM issues session token (JWT) with actor_uuid + group claims
+7. Session written to Session Store
+```
+
+---
+
+## 2.6 Storage Provider โ€” Ceph Realized State Snapshots
+
+**Registration:**
+
+```yaml
+storage_provider_registration:
+  uuid: pvd-cph-001
+  name: ceph-prod
+  display_name: Ceph โ€” Realized State Snapshot Store
+  store_type: write_once_snapshot
+  version: "1.0.0"
+  endpoint: https://ceph-rgw.corp.example:7480
+
+  capabilities:
+    write: true
+    read: true
+    delete: false            # write_once_snapshot โ€” immutable
+    content_addressed: true  # SHA-256 keyed
+    encryption_at_rest: true
+    replication_factor: 3
+
+  bucket: dcm-realized-state
+  auth_credential_uuid: crd-ceph-s3-001
+```
+
+**DCM writes a Realized State snapshot:**
+
+```yaml
+# After provider confirms VM provisioning:
+# DCM stores the full realized_state_record as a content-addressed object
+
+storage_write_request:
+  store_type: write_once_snapshot
+  key: realized-state/ent-vm-pay-03/rlz-pay-api-03-v1
+  content_hash: sha256:a1b2c3d4...
+  payload: 
+
+# Ceph stores it; returns: { "stored": true, "etag": "sha256:a1b2c3d4..." }
+# DCM records: realized_state_uuid โ†’ storage key mapping
+```
+
+---
+
+## 2.7 Message Bus Provider โ€” RabbitMQ Event Routing
+
+**Registration:**
+
+```yaml
+message_bus_provider_registration:
+  uuid: pvd-rmq-001
+  name: rabbitmq-prod
+  display_name: RabbitMQ โ€” DCM Event Bus
+  version: "1.0.0"
+  endpoint: amqps://rabbitmq.corp.example:5671
+  auth_credential_uuid: crd-rmq-001
+
+  capabilities:
+    publish: true
+    subscribe: true
+    durable_queues: true
+    dead_letter: true
+
+  exchange_config:
+    name: dcm.events
+    type: topic
+    durable: true
+
+  routing_key_pattern: "{domain}.{event_type}.{urgency}"
+  # Examples:
+  #   request.completed.low
+  #   provider.health_changed.high
+  #   accreditation.status_changed.critical
+```
+
+**Event flows through the bus:**
+
+```yaml
+# 1. DCM publishes event (internal โ†’ bus)
+publish:
+  exchange: dcm.events
+  routing_key: accreditation.status_changed.critical
+  payload:
+    event_type: accreditation.status_changed
+    urgency: critical
+    accreditation_uuid: acc-fr-high-001
+    from_status: authorized
+    to_status: revoked
+
+# 2. Notification Provider subscribes and routes to appropriate channels
+subscribe:
+  queue: dcm.notifications.critical
+  binding: "#.critical"            # all critical urgency events
+  handler: notification_provider    # Notification Provider consumes and routes
+
+# 3. ITSM Provider subscribes to provider events
+subscribe:
+  queue: dcm.itsm.provider-events
+  binding: "provider.#"            # all provider domain events
+  handler: itsm_provider            # creates ServiceNow incident
+```
+
+---
+
+## 2.8 Credential Provider โ€” Vault Secret Fetch at Dispatch Time
+
+**Registration:**
+
+```yaml
+credential_provider_registration:
+  uuid: pvd-vlt-001
+  name: vault-prod
+  display_name: HashiCorp Vault โ€” Credential Provider
+  version: "1.0.0"
+  endpoint: https://vault.corp.example:8200
+
+  capabilities:
+    fetch_secret: true
+    rotate_credential: true
+    dynamic_secret: true           # Vault dynamic credentials
+    ttl_management: true
+
+  auth_method: approle
+  approle_role_id: dcm-control-plane
+  approle_secret_uuid: crd-vault-approle-001   # bootstrap credential
+
+  secret_engines:
+    - path: secret/dcm/            # KV v2 for static credentials
+    - path: database/              # dynamic DB credentials
+    - path: pki/                   # certificate issuance
+```
+
+**Fetch flow at dispatch time:**
+
+```yaml
+# Request Payload Processor needs the FreeIPA bind password for the auth provider:
+
+credential_fetch_request:
+  credential_uuid: crd-ipa-bind-001
+  requesting_component: request_payload_processor
+  purpose: auth_provider_bind
+
+# DCM calls Credential Provider:
+# GET vault.corp.example:8200/v1/secret/data/dcm/providers/auth/freeipa-bind
+# Vault authenticates DCM via AppRole, returns:
+# { "data": { "password": "s3cr3t-b1nd-p4ss" } }
+
+# Credential returned to Payload Processor โ€” never written to any store
+# Used ephemerally for the LDAP bind โ†’ discarded after use
+
+# For dynamic DB credentials (short-lived):
+dynamic_credential_fetch:
+  credential_uuid: crd-db-dynamic-001
+  vault_path: database/creds/payments-db-role
+  ttl: PT1H                        # expires after 1 hour
+  # Vault creates a temp DB user, returns: { "username": "v-dcm-pay-1a2b", "password": "..." }
+```
+
+**Consumer retrieves a resource credential:**
+
+```
+GET /api/v1/resources/ent-vm-pay-03/credentials
+
+Response:
+[
+  {
+    "credential_uuid": "crd-vm-pay-03-ssh",
+    "type": "ssh_key",
+    "display_name": "SSH Access Key",
+    "fetch_url": "/api/v1/credentials/crd-vm-pay-03-ssh/value"
+  }
+]
+
+GET /api/v1/credentials/crd-vm-pay-03-ssh/value
+Authorization: Bearer 
+
+Response:
+{
+  "type": "ssh_key",
+  "private_key": "-----BEGIN OPENSSH PRIVATE KEY-----\n...",
+  "username": "cloud-user",
+  "expires_at": "2026-04-30T00:00:00Z"
+}
+# Audit record written: act-alice-001 fetched crd-vm-pay-03-ssh at 2026-03-31T10:00:00Z
+```
+
+---
+
+## 2.9 Meta Provider โ€” Compound WebApp Provisioning
+
+`webapp-meta` is a Meta Provider that composes a VM + IP + Firewall Rule + DNS Record into a single `ApplicationStack.WebApp` catalog item. The consumer requests one thing; DCM provisions four.
+
+**Compound service definition (registered by Meta Provider):**
+
+```yaml
+meta_provider_registration:
+  uuid: pvd-wam-001
+  name: webapp-meta
+  display_name: Web Application Stack (Meta Provider)
+
+  resource_types_composed:
+    - fqn: ApplicationStack.WebApp
+      version: "2.0.0"
+      constituents:
+        - component_id: vm
+          resource_type: Compute.VirtualMachine
+          provided_by: external       # DCM places with appropriate compute provider
+          required_for_delivery: required
+
+        - component_id: ip
+          resource_type: Network.IPAddress
+          provided_by: external
+          required_for_delivery: required
+          depends_on: []              # IP can provision in parallel with VM
+
+        - component_id: fw
+          resource_type: Security.FirewallRule
+          provided_by: external
+          required_for_delivery: optional
+          depends_on: [vm, ip]        # needs both IPs before rule can be written
+          inject_from:
+            vm: { source_ip: "$.realized.assigned_ip" }
+            ip: { dest_ip: "$.realized.ip_address" }
+
+        - component_id: dns
+          resource_type: Network.DNSRecord
+          provided_by: external
+          required_for_delivery: optional
+          depends_on: [ip]
+          inject_from:
+            ip: { a_record_value: "$.realized.ip_address" }
+```
+
+**Consumer request (one item, four resources provisioned):**
+
+```
+POST /api/v1/requests
+{
+  "catalog_item_uuid": "cat-webapp-standard",
+  "fields": {
+    "app_name": "payments-portal",
+    "cpu_cores": 4,
+    "ram_gb": 16,
+    "dns_hostname": "payments.corp.example",
+    "firewall_source": "0.0.0.0/0",
+    "firewall_dest_port": 443
+  }
+}
+
+# Composite entity created: ent-app-portal-001 (ApplicationStack.WebApp)
+# DCM decomposes into 4 constituent requests:
+#   ent-vm-portal  โ†’ pvd-vm-001    (PROVISIONING)
+#   ent-ip-portal  โ†’ pvd-net-001   (PROVISIONING, parallel)
+#   ent-fw-portal  โ†’ pvd-fw-001    (PENDING_DEPENDENCY on vm + ip)
+#   ent-dns-portal โ†’ pvd-dns-001   (PENDING_DEPENDENCY on ip)
+
+# Composite entity is OPERATIONAL when all required_for_delivery constituents are OPERATIONAL
+# Status visible as one entity to the consumer
+```
+
+---
+
+## 2.10 ITSM Provider โ€” ServiceNow Incident on Provider Health Change
+
+**Setup:** `servicenow-prod` is registered and configured to create incidents on provider health events.
+
+```yaml
+itsm_provider_registration:
+  provider_handle: servicenow-prod
+  itsm_system: servicenow
+  endpoint_url: https://corp.service-now.com
+  api_version: v2
+  auth_credential_uuid: crd-sn-api-001
+
+  supported_actions:
+    - create_incident
+    - update_incident
+    - resolve_incident
+    - create_change_request
+
+  field_mappings:
+    # DCM event fields โ†’ ServiceNow fields
+    incident:
+      short_description: "$.event.payload.detail"
+      urgency:
+        critical: 1     # ServiceNow urgency: 1=High
+        high: 2
+        medium: 3
+        low: 4
+      assignment_group: "DCM Platform Operations"
+      category: "Infrastructure"
+      subcategory: "Cloud Management"
+      cmdb_ci: "$.event.payload.provider_uuid"
+```
+
+**Event fires: provider goes unhealthy**
+
+```yaml
+# pvd-vm-001 health check fails 3 consecutive times
+event:
+  type: provider.health_changed
+  urgency: high
+  payload:
+    provider_uuid: pvd-vm-001
+    from_state: healthy
+    to_state: unhealthy
+    failure_count: 3
+    detail: "Health endpoint unreachable: connection timeout"
+
+# ITSM Provider receives via Message Bus
+# Creates ServiceNow incident:
+
+servicenow_api_call:
+  method: POST
+  path: /api/now/table/incident
+  body:
+    short_description: "DCM Provider Unhealthy: pvd-vm-001 โ€” connection timeout"
+    urgency: 2                        # high โ†’ ServiceNow urgency 2
+    assignment_group: DCM Platform Operations
+    category: Infrastructure
+    subcategory: Cloud Management
+    cmdb_ci: pvd-vm-001
+    description: |
+      DCM Provider pvd-vm-001 (VMware Prod) has failed 3 consecutive health checks.
+      Last error: connection timeout
+      Affected resource types: Compute.VirtualMachine
+      Routing: capacity reduced, new requests redirected to pvd-vm-002
+
+# ServiceNow responds: { "sys_id": "INC0087432" }
+# DCM stores: provider pvd-vm-001 โ†’ itsm_reference: INC0087432
+
+# When provider recovers:
+servicenow_api_call:
+  method: PATCH
+  path: /api/now/table/incident/INC0087432
+  body:
+    state: 6                          # ServiceNow: resolved
+    close_notes: "DCM Provider pvd-vm-001 returned to healthy state"
+    resolved_at: 2026-03-31T16:45:00Z
+```
+
+---
+
+# Section 3 โ€” Registration Flow Examples
+
+## 3.1 Information Provider Onboarding โ€” NetBox as Network IP Provider
+
+```yaml
+# Step 1: Platform admin issues registration token
+POST /api/v1/admin/registration-tokens
+{
+  "provider_type": "information_provider",
+  "handle_pattern": "providers/information/network/*",
+  "valid_for": "PT24H"
+}
+Response: { "token": "reg-tok-netbox-001", "expires_at": "2026-04-01T09:00:00Z" }
+
+# Step 2: NetBox provider submits registration
+POST /api/v1/admin/providers/register
+Authorization: Bearer reg-tok-netbox-001
+X-Client-Cert: 
+
+{
+  "provider_type": "information_provider",
+  "name": "NetBox โ€” Network IP Registry",
+  "handle": "providers/information/network/netbox-prod",
+  "version": "1.0.0",
+  "endpoint": "https://netbox.corp.example",
+  "implements": [
+    {
+      "information_type_name": "Network.IPAddress",
+      "information_type_version": "1.2.0",
+      "lookup_methods_supported": ["primary_key", "cidr_query"],
+      "extended_fields_supported": true,
+      "extended_schema": {
+        "vrf_id": { "type": "integer" },
+        "site_slug": { "type": "string" },
+        "role": { "type": "string", "enum": ["loopback", "anycast", "secondary"] }
+      }
+    }
+  ],
+  "sovereignty_declaration": {
+    "data_residency": ["US"],
+    "crosses_jurisdiction": false
+  }
+}
+
+# Step 3: Validation (6 checks for information_provider)
+# V1: information_provider enabled in prod profile โœ“
+# V2: Governance Matrix pre-check: ALLOW โœ“
+# V3: Token valid and matches handle pattern โœ“
+# V4: mTLS certificate valid, corp CA chain โœ“
+# V5: Sovereignty declaration complete โœ“
+# V6: Health endpoint reachable โœ“
+
+# Step 4: Platform admin approves
+POST /api/v1/admin/registrations/reg-netbox-001/approve
+{ "rationale": "NetBox is our authoritative IP registry" }
+
+# Step 5: ACTIVE โ€” NetBox now enriches assembly payloads
+# When a request for any resource in dc-west-1 is assembled:
+#   DCM queries NetBox: "give me next available IP in 10.42.18.0/24"
+#   NetBox returns: { "ip": "10.42.18.56", "vrf_id": 4, "site_slug": "dc-west-1" }
+#   IP injected into assembled payload
+```
+
+---
+
+## 3.2 Auth Provider Onboarding โ€” Adding a Secondary OIDC Provider
+
+```yaml
+# Scenario: Adding Azure AD as a secondary auth source for contractors
+POST /api/v1/admin/providers/register
+Authorization: Bearer reg-tok-oidc-001
+X-Client-Cert: 
+
+{
+  "provider_type": "auth_provider",
+  "name": "Azure AD โ€” Contractor Identity",
+  "handle": "providers/auth/azure-ad-contractors",
+  "version": "1.0.0",
+
+  "capabilities": {
+    "authentication": true,
+    "mfa": true,
+    "group_membership": true,
+    "role_mapping": true
+  },
+
+  "protocol": "oidc",
+  "oidc_config": {
+    "issuer": "https://login.microsoftonline.com/{tenant-id}/v2.0",
+    "client_id": "dcm-azure-ad-client",
+    "client_secret_uuid": "crd-azure-oidc-secret",
+    "scopes": ["openid", "profile", "email", "groups"],
+    "group_claim": "groups"
+  },
+
+  "group_mapping": [
+    {
+      "oidc_group_id": "aad-grp-dcm-contractors",
+      "dcm_role": "tenant_member",
+      "tenant_uuid": "ten-web-001",
+      "scope_restriction": {
+        "allowed_resource_types": ["Compute.VirtualMachine"],
+        "max_ttl": "P7D"             # contractor VMs expire after 7 days
+      }
+    }
+  ],
+
+  "precedence": 2                    # lower than FreeIPA (precedence 1)
+                                     # FreeIPA checked first; Azure AD is fallback
+}
+```
+
+---
+
+## 3.3 Meta Provider Onboarding
+
+```yaml
+POST /api/v1/admin/providers/register
+Authorization: Bearer reg-tok-meta-001
+
+{
+  "provider_type": "meta_provider",
+  "name": "Web Application Stack",
+  "handle": "providers/meta/webapp-stack",
+  "version": "2.0.0",
+
+  "resource_types_composed": [
+    {
+      "fqn": "ApplicationStack.WebApp",
+      "version": "2.0.0",
+      "catalog_item_template_uuid": "cat-tmpl-webapp-001",
+      "constituents": [
+        {
+          "component_id": "vm",
+          "resource_type": "Compute.VirtualMachine",
+          "provided_by": "external",
+          "required_for_delivery": "required"
+        },
+        {
+          "component_id": "ip",
+          "resource_type": "Network.IPAddress",
+          "provided_by": "external",
+          "required_for_delivery": "required",
+          "depends_on": []
+        },
+        {
+          "component_id": "fw",
+          "resource_type": "Security.FirewallRule",
+          "provided_by": "external",
+          "required_for_delivery": "optional",
+          "depends_on": ["vm", "ip"]
+        }
+      ]
+    }
+  ],
+
+  "decomposition_policy_handle": "system/meta/webapp-decompose-v2"
+}
+
+# Validation:
+# V1: meta_provider enabled in profile โœ“
+# V2: All constituent resource_types registered in Registry โœ“
+# V3: No circular dependencies in constituent graph โœ“
+# V4: Decomposition policy handle resolvable โœ“
+
+# Once ACTIVE: "ApplicationStack.WebApp" appears in service catalog
+# Consumers request one item; DCM provisions all constituents automatically
+```
+
+---
+
+# Section 4 โ€” OPA Policy Integration Examples
+
+## 4.1 OPA Policy Bundle Delivery and Shadow Mode
+
+**Scenario: New PCI DSS policy deployed in shadow mode before enforcement**
+
+```yaml
+# Policy author submits new policy artifact
+POST /api/v1/admin/policies/submit
+
+{
+  "handle": "compliance/pci/card-data-network-isolation",
+  "version": "1.0.0",
+  "type": "gatekeeper",
+  "enforcement_class": "hard_stop",
+  "status": "proposed",           # starts in shadow mode
+  "opa_bundle_ref": "git://policies/compliance/pci/card-data-isolation@v1.0.0",
+  "applies_to": {
+    "resource_types": ["Compute.VirtualMachine", "Network.VLAN"]
+  },
+  "description": "Card data VMs must be on isolated network segments โ€” not shared with non-PCI workloads"
+}
+```
+
+**Shadow mode evaluation (next 30 days):**
+
+```yaml
+# Policy evaluates against every matching request but does NOT block
+# Results logged as shadow_divergence events
+
+shadow_divergence_event:
+  policy_handle: compliance/pci/card-data-network-isolation@1.0.0
+  request_uuid: req-vm-web-07
+  tenant_uuid: ten-web-001
+  actor_uuid: act-alice-001
+  shadow_result: WOULD_BLOCK
+  reason: "VM requested on shared VLAN 100 โ€” card data isolation requires dedicated VLAN โ‰ฅ200"
+  # Real decision: ALLOW (shadow mode โ€” not enforced yet)
+
+# Platform admin reviews shadow dashboard after 2 weeks:
+GET /api/v1/admin/policies/compliance/pci/card-data-network-isolation/shadow-report
+
+{
+  "evaluation_period": "2026-03-01 to 2026-03-15",
+  "total_evaluations": 847,
+  "would_block_count": 12,
+  "would_block_pct": 1.4,
+  "top_blocking_reasons": [
+    { "reason": "VLAN < 200", "count": 9 },
+    { "reason": "mixed tenant segment", "count": 3 }
+  ],
+  "recommendation": "Safe to activate โ€” 12 impacted requests in 2 weeks, all addressable"
+}
+
+# Admin activates:
+PATCH /api/v1/admin/policies/compliance/pci/card-data-network-isolation
+{ "status": "active" }
+# Policy now enforced โ€” all future matching requests checked for real
+```
+
+---
+
+## 4.2 OPA Bundle Delivery to Policy Provider
+
+```yaml
+# OPA sidecar (Policy Provider) registered:
+policy_provider_registration:
+  uuid: pvd-opa-001
+  name: opa-compliance-sidecar
+  mode: sidecar                    # co-deployed with DCM control plane
+  bundle_sources:
+    - handle: compliance/pci/*
+      git_ref: git://policies/pci@main
+      pull_interval: PT5M          # pull fresh bundle every 5 minutes
+    - handle: compliance/hipaa/*
+      git_ref: git://policies/hipaa@main
+      pull_interval: PT5M
+
+# Bundle pull cycle:
+# 1. OPA sidecar polls git repo every 5 minutes
+# 2. New bundle detected (policy updated by compliance team via GitOps PR)
+# 3. OPA loads new bundle โ€” hot reload, no downtime
+# 4. DCM notified: policy_provider.bundle_updated event
+
+# Policy evaluation call (from Request Orchestrator to OPA sidecar):
+POST /v1/data/dcm/policies/evaluate
+{
+  "input": {
+    "request": {
+      "resource_type": "Compute.VirtualMachine",
+      "tenant_uuid": "ten-pay-001",
+      "payload": { "network_segment": "dmz-payments", "vlan": 142 }
+    },
+    "actor": { "uuid": "act-alice-001", "roles": ["tenant_member"] },
+    "provider": { "uuid": "pvd-vm-001", "accreditations": ["pci_dss", "iso_27001"] }
+  }
+}
+
+Response:
+{
+  "result": {
+    "allow": true,
+    "policies_evaluated": 14,
+    "transformations": [
+      { "field": "cpu_pinning", "value": true, "reason": "PCI isolation" }
+    ],
+    "gatekeepers_fired": 0
+  }
+}
+```
+
+---
+
+# Section 5 โ€” Admin and Consumer GUI Examples
+
+## 5.1 Consumer Portal โ€” New Resource Request Flow
+
+```
+Step 1: Consumer logs in
+  โ†’ Redirected to FreeIPA SSO via OIDC
+  โ†’ Returns with session token (alice, payments-bu, tenant_member)
+
+Step 2: Service Catalog presented
+  โ†’ DCM filters by: alice's roles + payments-bu tenant + prod profile
+  โ†’ Shows: [Compute.VirtualMachine, Network.IPAddress, ApplicationStack.WebApp, ...]
+  โ†’ Cost estimate shown per catalog item (from Cost Analysis component)
+
+Step 3: Alice selects "Compute.VirtualMachine โ€” Standard"
+  โ†’ Form generated from catalog item field_schema
+  โ†’ Pre-fill hints from previous requests (last used zone, OS)
+
+Step 4: Pre-request cost estimate
+  POST /api/v1/cost/estimate
+  { "catalog_item_uuid": "cat-vm-standard", "fields": { "cpu_cores": 8, "ram_gb": 32 } }
+  โ†’ { "monthly_estimate": "$142.40", "one_time": "$0", "currency": "USD" }
+
+Step 5: Alice submits request
+  โ†’ Operation returned: { "name": "/api/v1/operations/req-pay-api-03", "done": false }
+  โ†’ Portal polls operation.name every 2 seconds
+  โ†’ Progress bar: INITIATED โ†’ POLICY_EVALUATION โ†’ DISPATCHED โ†’ PROVISIONING โ†’ OPERATIONAL
+
+Step 6: Resource is OPERATIONAL
+  โ†’ Portal shows: IP address, SSH key download link, hostname
+  โ†’ Toast notification: "payments-api-03 is ready"
+```
+
+---
+
+## 5.2 Admin GUI โ€” Policy Flow Visualization
+
+The Flow GUI shows the policy evaluation pipeline for a given request as an interactive diagram.
+
+```
+Admin selects: Request req-pay-api-03
+Flow GUI renders:
+
+[ Consumer Input ]
+       โ”‚
+       โ–ผ
+[ Layer Assembly ]
+  โ”œโ”€โ”€ Base Layer (3.1.0) โœ“
+  โ”œโ”€โ”€ DC West-1 Layer (2.0.0) โœ“
+  โ”œโ”€โ”€ DMZ Zone Layer (1.5.0) โœ“
+  โ”œโ”€โ”€ Payments BU Layer (1.2.0) โœ“
+  โ””โ”€โ”€ Request Layer โœ“
+       โ”‚
+       โ–ผ
+[ Policy Evaluation โ€” 14 policies ]
+  โ”œโ”€โ”€ GateKeeper: pci-network-isolation โœ“ PASS
+  โ”œโ”€โ”€ GateKeeper: phi-provider-accreditation โœ“ PASS (no PHI in request)
+  โ”œโ”€โ”€ Validation: vm-size-limits โœ“ PASS
+  โ”œโ”€โ”€ Transformation: pci-cpu-pinning โ†’ cpu_pinning: true APPLIED
+  โ””โ”€โ”€ Transformation: approved-os-image โ†’ rhel-9-latest-approved APPLIED
+       โ”‚
+       โ–ผ
+[ Placement Engine ]
+  โ”œโ”€โ”€ Candidates: [pvd-vm-001 (score:65), pvd-vm-002 (score:36)]
+  โ””โ”€โ”€ Selected: pvd-vm-001 (highest accreditation richness)
+       โ”‚
+       โ–ผ
+[ Dispatch โ†’ pvd-vm-001 ]
+  โ””โ”€โ”€ Status: OPERATIONAL โœ“
+
+# Clicking any node shows full input/output payload for that step
+# Shadow mode indicator (S) shown on any policy evaluated in shadow
+# Red path shown for any GateKeeper that fired and blocked
+```
+
+---
+
+## 5.3 Admin GUI โ€” Drift Dashboard
+
+```
+Admin opens Drift Dashboard:
+
+Summary:
+  Open drift records: 7
+  โ”œโ”€โ”€ Critical: 1  (security config drift โ€” out-of-hours change)
+  โ”œโ”€โ”€ High: 3      (resource sizing drift โ€” manual changes)
+  โ””โ”€โ”€ Medium: 3    (metadata drift โ€” tags removed)
+
+Critical drift: ent-vm-pay-03
+  Discovered: selinux: permissive
+  Realized:   selinux: enforcing
+  Changed at: 2026-03-30T03:15:00Z (2:00 AM โ€” suspicious)
+  Provider:   pvd-vm-001
+  Policy response: NOTIFY_AND_WAIT โ†’ Compliance Team paged
+
+Admin actions available:
+  [Revert to Realized State]  โ†’ dispatches remediation to pvd-vm-001
+  [Accept Discovered State]   โ†’ updates Realized State record (requires reason)
+  [Investigate]               โ†’ opens audit trail for this entity since last drift-clean
+```
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*

From bbcfd2be26dcaa7424140d3a5bb73fdf47e033e8 Mon Sep 17 00:00:00 2001
From: Chris Roadfeldt 
Date: Tue, 31 Mar 2026 21:10:57 -0500
Subject: [PATCH 49/49] Attempting to complete the pieces needed to pull the
 trigger on this project, added some details on who,what,why,etc..

Signed-off-by: Chris Roadfeldt 
---
 content/_index.md                             |   52 +-
 content/docs/README.md                        |   94 +-
 content/docs/_index.md                        |   16 +-
 content/docs/architecture/ai-prompt.md        |  359 ++-
 .../49-implementation-specifications.md       |  728 ++++++
 content/docs/architecture/overview.md         |    7 +-
 .../specifications/cncf-strategy.md           |    4 +
 .../specifications/consumer-api-spec.md       |  133 +-
 .../specifications/dcm-admin-api-spec.md      |    4 +
 .../specifications/dcm-admin-api.yaml         | 1830 +++++++++++++
 .../specifications/dcm-consumer-api.yaml      | 2288 +++++++++++++++++
 .../specifications/dcm-examples.md            | 1192 +++++++++
 .../specifications/dcm-flow-gui-spec.md       |    4 +
 .../dcm-opa-integration-spec.md               |    4 +
 .../dcm-operator-interface-spec.md            |    4 +
 .../specifications/dcm-registration-spec.md   |    4 +
 content/docs/capabilities-matrix.md           |  100 +-
 .../data-model/03-layering-and-versioning.md  |  152 +-
 .../data-model/05-resource-type-hierarchy.md  |  304 ++-
 .../docs/data-model/20-registry-governance.md |   14 +
 .../28-federated-contribution-model.md        |   17 +
 .../data-model/48-location-topology-layers.md |  962 +++++++
 .../49-implementation-specifications.md       |  728 ++++++
 content/docs/project-overview.md              |  327 +++
 24 files changed, 9242 insertions(+), 85 deletions(-)
 create mode 100644 content/docs/architecture/data-model/49-implementation-specifications.md
 create mode 100644 content/docs/architecture/specifications/dcm-admin-api.yaml
 create mode 100644 content/docs/architecture/specifications/dcm-consumer-api.yaml
 create mode 100644 content/docs/data-model/48-location-topology-layers.md
 create mode 100644 content/docs/data-model/49-implementation-specifications.md
 create mode 100644 content/docs/project-overview.md

diff --git a/content/_index.md b/content/_index.md
index 799d564..c802d3f 100644
--- a/content/_index.md
+++ b/content/_index.md
@@ -3,42 +3,48 @@ title: DCM Project
 toc: false
 ---
 
-> **โš ๏ธ Active Development** โ€” The DCM project is under active development. Architecture, data model, and specifications are work in progress. [Contribute on GitHub](https://github.com/dcm-project).
-
 # Data Center Management
 
-Hyperscaler-like cloud experience for
-enterprise on-premises and sovereign cloud infrastructure
+**Hyperscaler-like cloud experience for enterprise on-premises and sovereign cloud infrastructure.**
+
+DCM is an open-source governing framework that gives enterprise IT organizations the operational model and self-service capabilities of a public cloud provider โ€” on infrastructure they own and control.
+
+{{< button href="/docs/project-overview" >}}What is DCM?{{< /button >}}
+{{< button href="/docs" >}}Documentation{{< /button >}}
+{{< button href="https://github.com/dcm-project" >}}GitHub{{< /button >}}
+
+---
+
+## The Problem DCM Solves
 
-{{< button href="/docs" >}}Get Started{{< /button >}}
-{{< button href="https://github.com/dcm-project" >}}View on GitHub{{< /button >}}
+Enterprise on-premises infrastructure is managed by dozens of disconnected tools and manual processes. A single VM can take weeks across five teams. No one has a trustworthy answer to: *what exists, what was requested, what was provisioned, and does current state match intended state?*
 
-## About DCM
+DCM establishes a unified, declarative single source of truth for all infrastructure state and connects it to a policy-driven control plane that governs every lifecycle operation โ€” from request to decommission.
 
-The Data Center Management (DCM) project provides a hyperscaler-like cloud experience
-for enterprise on-premises and sovereign cloud infrastructure. DCM is an
-**API-first, technology-agnostic framework** that relies on a declarative data model
-where the desired state of infrastructure is defined as code.
+## Three Foundational Abstractions
 
-The overarching goal is to define the configuration of an entire data center in
-version-controlled code, enabling rapid deployment and full environment recovery.
+{{< cards >}}
+  {{< card title="Data" icon="database" subtitle="Every artifact with a UUID, lifecycle state, and field-level provenance. Four states: Intent ยท Requested ยท Realized ยท Discovered." >}}
+  {{< card title="Provider" icon="plug" subtitle="Eleven typed provider contracts. Service, Information, Storage, Meta, Credential, Auth, Policy, Notification, Message Bus, ITSM, Peer DCM." >}}
+  {{< card title="Policy" icon="shield-check" subtitle="Seven typed policy schemas. GateKeeper ยท Validation ยท Transformation ยท Orchestration Flow ยท Recovery ยท Governance Matrix ยท Lifecycle." >}}
+{{< /cards >}}
 
-## Core Principles
+## Who Benefits
 
 {{< cards >}}
-  {{< card title="API-First Design" icon="plug" subtitle="Technology-agnostic framework built on modern API principles for maximum flexibility." >}}
-  {{< card title="Cloud-Native Approach" icon="cloud" subtitle="Moving away from monolithic management toward distributed, cloud-native principles." >}}
-  {{< card title="Service Catalog" icon="book-open" subtitle="Comprehensive catalog system for managing services and infrastructure offerings." >}}
-  {{< card title="Rules Engine" icon="adjustments" subtitle="Built-in governance and policy enforcement through a flexible rules engine." >}}
+  {{< card title="Application Teams" icon="users" subtitle="Self-service catalog. Request infrastructure. Receive provisioned resources. No tickets, no manual coordination." >}}
+  {{< card title="Platform Engineers" icon="cog" subtitle="Single control plane. Policy-governed standards. Automatic drift detection. Consistency is structural." >}}
+  {{< card title="Security & Compliance" icon="lock-closed" subtitle="Policy-as-code tested before activation. Tamper-evident audit trail. Continuous accreditation monitoring." >}}
+  {{< card title="Regulated Industries" icon="building-library" subtitle="FedRAMP ยท CMMC ยท HIPAA ยท SOC 2 ยท ISO 27001 ยท DoD IL2โ€“IL6. Compliance by construction." >}}
 {{< /cards >}}
 
 ---
 
-## Explore More
+## Explore
 
 {{< cards >}}
-  {{< card link="docs" title="Documentation" icon="document-text" subtitle="Guides, tutorials, and technical documentation." >}}
-  {{< card link="docs/enhancements" title="Enhancements" icon="light-bulb" subtitle="Enhancement proposals documenting new features and changes." >}}
-  {{< card link="blog" title="Blog & Updates" icon="rss" subtitle="Latest news, releases, and articles." >}}
-  {{< card link="docs/architecture" title="Architecture" icon="template" subtitle="High-level design, data model, specifications, and Kubernetes integration." >}}
+  {{< card link="docs/project-overview" title="Project Overview" icon="information-circle" subtitle="What DCM is, what it does, who it benefits, where it operates." >}}
+  {{< card link="docs/architecture" title="Architecture" icon="template" subtitle="Data model, specifications, and design decisions." >}}
+  {{< card link="docs/capabilities-matrix" title="Capabilities" icon="table-cells" subtitle="299 capabilities across 38 domains." >}}
+  {{< card link="https://github.com/dcm-project" title="GitHub" icon="code-bracket" subtitle="Source, issues, and contributions." >}}
 {{< /cards >}}
diff --git a/content/docs/README.md b/content/docs/README.md
index b7db9f5..b6a8c4c 100644
--- a/content/docs/README.md
+++ b/content/docs/README.md
@@ -1,8 +1,16 @@
-# DCM Documentation
+# DCM โ€” Data Center Management
 
-Data Center Management (DCM) is an open-source governing framework for enterprise on-premises and sovereign cloud infrastructure. It provides a hyperscaler-like cloud experience on infrastructure that organizations own and control.
+Data Center Management (DCM) is an open-source governing framework for enterprise on-premises and sovereign cloud infrastructure. It provides a hyperscaler-like cloud experience โ€” the operational model and self-service capabilities of a public cloud provider โ€” on infrastructure that organizations own and control.
 
-**GitHub:** https://github.com/dcm-project
+**GitHub:** https://github.com/dcm-project | **License:** Apache 2.0
+
+---
+
+## What DCM Is
+
+DCM is the governing control plane that sits above provisioning tools, automation platforms, and infrastructure systems โ€” making them coherent, governed, and self-service. It is not a deployment tool or a configuration manager. It is the management plane that connects them.
+
+**[Full project description โ†’](project-overview.md)** โ€” what DCM is, what it does, why, who benefits, and where it operates.
 
 ---
 
@@ -15,27 +23,35 @@ DCM is built on three foundational abstractions โ€” **Data**, **Provider**, and
 ## Documentation Structure
 
 ### Foundation Documents (read these first)
-| [00-design-priorities.md](data-model/00-design-priorities.md) | Design priority framework โ€” decision hierarchy for all contributors |
+
 | Document | Purpose |
 |----------|---------|
+| [project-overview.md](project-overview.md) | What DCM is, what it does, who it benefits, where it operates |
 | [00-foundations.md](data-model/00-foundations.md) | The three abstractions โ€” Data, Provider, Policy |
+| [00-design-priorities.md](data-model/00-design-priorities.md) | Design priority framework โ€” decision hierarchy for all contributors |
 | [A-provider-contract.md](data-model/A-provider-contract.md) | Unified Provider base contract + 11 typed extensions |
 | [B-policy-contract.md](data-model/B-policy-contract.md) | Unified Policy base contract + 7 output schemas |
 
-### Data Model (28 documents)
+### Data Model (55 documents)
+
 | Range | Coverage |
 |-------|---------|
-| 00โ€“05 | Context, entity types, four states, layering, examples, ownership, resource types |
-| 06โ€“11 | Resource/service entities, dependencies, grouping, relationships, information providers, storage providers |
-| 12โ€“17 | Audit, ingestion, policy profiles, universal groups, universal audit, deployment |
-| 18โ€“23 | Webhooks, auth providers, registry governance, advanced information providers, federation, notifications |
-| 24โ€“28 | Operational models, control plane components, accreditation, governance matrix, federated contribution |
+| 00โ€“05 | Context, foundations, entity types, four states, layering, ownership, resource types |
+| 06โ€“12 | Resource/service entities, dependencies, grouping, relationships, information providers, storage providers, audit |
+| 13โ€“19 | Ingestion, policy profiles, universal groups, universal audit, deployment, webhooks, auth providers |
+| 20โ€“27 | Registry governance, advanced information providers, federation, notifications, operational models, control plane, accreditation, governance matrix |
+| 28โ€“35 | Federated contribution, scoring model, meta provider, credential provider, authority tier, event catalog, API versioning, session revocation |
+| 36โ€“42 | Internal component auth, scheduled requests, dependency graph, self-health, standards catalog, operational reference, ITSM integration |
+| 43โ€“49 | Provider callback auth, Kessel evaluation, consistency review, workload analysis, accreditation monitor, location topology, implementation specifications |
+| A, B | Unified Provider Contract, Unified Policy Contract |
 
-### Specifications (10 documents)
-Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI ยท Registration ยท Examples ยท Kubernetes Compatibility ยท Operator SDK ยท CNCF Strategy
+### Specifications (15 documents)
+
+Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI ยท Admin GUI ยท Consumer GUI ยท Provider GUI ยท Registration ยท Examples ยท Use Case Examples ยท Kubernetes Compatibility ยท Operator SDK ยท RHDH Integration ยท CNCF Strategy
 
 ### AI Model Prompt
-[DCM-AI-PROMPT.md](DCM-AI-PROMPT.md) โ€” paste into any AI model to provide full project context. 62 sections, 4,330 lines.
+
+[DCM-AI-PROMPT.md](DCM-AI-PROMPT.md) โ€” load into any AI model to provide full project context. 98 sections covering the complete architecture, all decisions, and working instructions.
 
 ---
 
@@ -46,37 +62,67 @@ Consumer API ยท Admin API ยท Operator Interface ยท OPA Integration ยท Flow GUI 
 | Foundational abstractions | 3 (Data, Provider, Policy) |
 | Provider types | 11 (unified base contract + typed capability extensions) |
 | Policy types | 7 (unified base contract + typed output schemas) |
-| Control plane components | 9 |
-| Four lifecycle states | Intent ยท Requested ยท Realized ยท Discovered |
-| Capabilities | 189 across 31 domains |
-| Data model documents | 45 (39 numbered + 3 foundation + 2 examples + 1 design priorities) |
-| Specifications | 10 |
-| Unresolved questions | 0 |
+| Entity lifecycle states | 4 (Intent ยท Requested ยท Realized ยท Discovered) |
+| Capabilities | 299 across 38 domains |
+| Data model documents | 55 |
+| Specifications | 15 |
+| Consumer API paths | 63 |
+| Admin API paths | 57 |
+| Unresolved architectural questions | 0 |
 
 ---
 
 ## Core Principles
 
 1. **Declarative** โ€” data describes desired state, not procedures
-2. **API-First** โ€” every capability available via standard API
+2. **API-First** โ€” every capability available via standard AEP-aligned API
 3. **Policy-Governed** โ€” all business logic through the Policy Engine, never hard-coded
 4. **Idempotent** โ€” applying the same data multiple times produces the same result
 5. **Immutable if Versioned** โ€” published versions never change; changes produce new versions
 6. **Provider-Agnostic** โ€” DCM defines contracts, not implementations
 7. **GitOps-Native** โ€” intent and policy artifacts are Git-native
-8. **Federated by Default** โ€” all authorized actor types contribute within permitted scope
-9. **Easy to use ยท Easy to implement ยท Easy to extend**
+8. **Federated** โ€” all authorized actor types contribute within permitted scope
+9. **Compliance by Construction** โ€” audit trail, provenance, and sovereignty enforcement are structural
 
 ---
 
 ## Capabilities Matrix
 
-[DCM-Capabilities-Matrix.md](DCM-Capabilities-Matrix.md) โ€” 126 capabilities across 20 domains including: Identity and Access, Service Catalog, Request Lifecycle, Provider Contract, Resource Lifecycle, Drift Detection, Policy Management, Data Layer, Information Integration, Ingestion, Audit, Observability, Storage, Federation, Platform Governance, Accreditation, Zero Trust, Governance Matrix, Drift Reconciliation, Federated Contribution, and Scoring Model (enforcement_class / approval routing thresholds).
+[DCM-Capabilities-Matrix.md](DCM-Capabilities-Matrix.md) โ€” 299 capabilities across 38 domains including: Identity and Access, Service Catalog, Request Lifecycle, Provider Contract, Resource Lifecycle, Drift Detection, Policy Management, Data Layer, Information Integration, Ingestion, Audit, Observability, Storage, Federation, Platform Governance, Accreditation, Zero Trust, Governance Matrix, Scoring Model, Meta Provider, Credential Provider, Authority Tier, Event Catalog, API Versioning, Session Revocation, Internal Component Auth, Scheduled Requests, Dependency Graph, Self-Health, Operational Reference, Web Interfaces, ITSM Integration, Provider Callback Auth, Workload Analysis, Accreditation Monitoring, and Location Topology.
+
+---
+
+## How DCM Works
 
-**Minimum viable end-to-end set:** 21 capabilities (IAM-001 โ†’ AUD-001 critical path).
+DCM's runtime is a **policy-driven event loop**: every data state change triggers Policy Engine evaluation, policies produce typed outputs (approve/halt/enrich/route/recover), outputs invoke Providers or produce new Data, and new Data triggers new events. There is no hard-coded pipeline โ€” the pipeline is the sum of active Policies.
+
+A request flows through: **intent declared** โ†’ **layer assembly** (Core Layers + Service Layers + Transformation Policies inject and lock required fields) โ†’ **policy evaluation** (Validation, GateKeeper, Placement) โ†’ **Requested State written** โ†’ **dispatch to Provider** (Naturalization โ†’ execution โ†’ Denaturalization) โ†’ **Realized State written** โ†’ **ongoing drift monitoring**.
+
+Every business rule is a Policy artifact stored in Git, versioned, tested in shadow mode before activation, and enforced deterministically. Adding a new approval step, changing placement rules, or building a named workflow requires writing a policy โ€” not changing code.
+
+Providers wrap existing automation (Ansible, Terraform, vendor APIs). They implement one base contract and translate between DCM's unified data model and their native format. Organizations do not replace their automation โ€” they govern it.
+
+**[Full technical walkthrough โ†’](project-overview.md#how-dcm-works)**
+
+---
+
+## Ethos
+
+Four design priorities โ€” applied in order when they conflict:
+
+1. **Security is the baseline.** Security properties are present in every profile. Profiles control enforcement strictness and operational burden โ€” not whether the property applies. The `minimal` profile is "security with minimal overhead" โ€” not "minimal security."
+
+2. **The governed path must also be the easy path.** Self-service is how governance scales. If consuming resources through DCM is harder than raising a ticket, teams route around it. The entire scoring, auto-approval, and visual policy authoring system exists to make the governed path the path of least resistance.
+
+3. **Compliance is constructed, not audited.** Audit evidence, field-level provenance, and sovereignty enforcement are structural products of every operation โ€” not reconstructed from logs after the fact. An auditor can answer "who touched this data and when" directly from DCM's audit store.
+
+4. **No silent behavior.** Every operation produces an observable artifact. Every state transition produces an audit record. Every policy decision produces a typed output. When something goes wrong, "what happened and why" is always answerable from the system's own output.
+
+**[Full ethos document โ†’](project-overview.md#ethos)**
 
 ---
 
+
 ## Contributing
 
 DCM is open-source. Community contributions welcome via GitHub at https://github.com/dcm-project
diff --git a/content/docs/_index.md b/content/docs/_index.md
index 1cde65a..bffdb3c 100644
--- a/content/docs/_index.md
+++ b/content/docs/_index.md
@@ -5,19 +5,17 @@ sidebar:
   open: false
 ---
 
-> **โš ๏ธ Active Development** โ€” DCM documentation is actively being developed. Content represents work in progress and is subject to change as design decisions are finalized.
-
 # DCM Documentation
 
-Documentation for the DCM (Data Center Management) project โ€” a governing framework
-that provides a hyperscaler-like cloud experience for enterprise on-premises and
-sovereign cloud infrastructure.
+Documentation for the DCM (Data Center Management) project โ€” an open-source governing framework for enterprise on-premises and sovereign cloud infrastructure.
+
+**New here?** Start with the [Project Overview](/docs/project-overview) โ€” what DCM is, what it does, who it benefits, and where it operates.
 
 ## Documentation Sections
 
 {{< cards >}}
-  {{< card link="architecture" title="Architecture" icon="template" subtitle="High-level design, data model, specifications, and Kubernetes integration." >}}
-  {{< card link="capabilities" title="Capabilities Map" icon="view-grid" subtitle="Interactive map of 95 foundational capabilities across 15 domains. Filter by domain, perspective, or search. Includes CSV export for Jira import." >}}
-  {{< card link="taxonomy" title="Taxonomy" icon="book-open" subtitle="Authoritative vocabulary reference for all DCM architecture and implementation work. Core vocabulary, anti-vocabulary, roles, and capability prefixes." >}}
-  {{< card link="enhancements" title="Enhancements" icon="document-text" subtitle="Enhancement proposals documenting new features, architectural decisions, and significant changes." >}}
+  {{< card link="project-overview" title="Project Overview" icon="information-circle" subtitle="What DCM is, why it exists, who it benefits, and where it operates." >}}
+  {{< card link="architecture" title="Architecture" icon="template" subtitle="Data model (55 docs), specifications (15 docs), and OpenAPI schemas." >}}
+  {{< card link="capabilities-matrix" title="Capabilities Matrix" icon="table-cells" subtitle="299 capabilities across 38 domains." >}}
+  {{< card link="taxonomy" title="Taxonomy" icon="book-open" subtitle="Precise vocabulary for the DCM architecture." >}}
 {{< /cards >}}
diff --git a/content/docs/architecture/ai-prompt.md b/content/docs/architecture/ai-prompt.md
index 0b1e35e..ce14dea 100644
--- a/content/docs/architecture/ai-prompt.md
+++ b/content/docs/architecture/ai-prompt.md
@@ -5124,7 +5124,364 @@ Brownfield ingestion, Static Replace, In-Place Upgrade.
 All 10 provider types: โœ… covered. All 7 policy output schemas: โœ… covered. All major model flows: โœ… covered.
 
 
-## SECTION 86 โ€” WORKING INSTRUCTIONS FOR AI MODELS
+## SECTION 86 โ€” LOCATION TOPOLOGY LAYER MODEL (doc 48)
+
+**New doc:** 48-location-topology-layers.md โ€” specifies where resources can be allocated, how location data flows into requests as Core Layers, and how consumers select locations.
+
+**Core concept:** Location is not a string field. It is a resolved chain of versioned Core Layers โ€” one per level of the topology hierarchy. When a consumer selects "DC1 โ€” Frankfurt Alpha", DCM assembles Country โ†’ Region โ†’ Zone โ†’ Site โ†’ Data Center layers into the request payload, injecting all structured location data at each level.
+
+**Standard hierarchy (9 levels, configurable names, standard types):**
+- Level 1 Country (CTY) โ€” jurisdiction, regulatory frameworks, ISO codes
+- Level 2 Region (RGN) โ€” interconnects, latency profile, failover region
+- Level 3 Zone/AZ (AZ) โ€” isolation boundary, HA peers, RPO/RTO
+- Level 4 Site/Campus (SITE) โ€” physical address, security tier, facilities contacts
+- Level 5 Data Center (DC) โ€” tier classification, power/cooling, PUE, certifications
+- Level 6 Hall/Pod (HALL) โ€” optional; network segment, cooling type
+- Level 7 Cage/Enclosure (CAGE) โ€” optional; tenant isolation, access control
+- Level 8 Rack (RACK) โ€” rack units, power circuits, ToR switch
+- Level 9 Unit/Slot (UNIT) โ€” optional; typically provider-managed
+
+**Authority model:** Each level has a designated owning authority (Data Center Operations, Network Operations, Facilities, Platform Governance). Changes require GitOps PRs approved by the owning authority. Upper levels (Country, Region, Zone) require platform_admin approval; lower levels (Hall, Cage, Rack) can be operator-approved.
+
+**Custom types:** Custom levels insertable anywhere using decimal level values (e.g., Fleet=3.5, Ship=4.5 in a Navy deployment). Navy example included in doc.
+
+**Priority bands:** Location layers occupy dedicated bands in the Core Layer priority space: Country=100.xx, Region=200.xx, Zone=300.xx, Site=400.xx, DC=500.xx, Hall=600.xx, Cage=700.xx, Rack=800.xx. Ensures specific always overrides general.
+
+**Consumer API (NEW):**
+- GET /api/v1/locations โ€” list available location nodes (entitlement-filtered, filterable by resource_type, catalog_item, level, classification)
+- GET /api/v1/locations/{uuid} โ€” full detail including hierarchy, sovereignty, compliance, capacity
+- ServiceRequest now accepts location_uuid or location_handle
+- Consumer selects at any level; Placement Engine refines to specific DC at dispatch
+
+**Admin API (NEW):**
+- GET/POST /api/v1/admin/location-types โ€” manage location type registry
+- GET /api/v1/admin/locations โ€” all location nodes (no entitlement filter)
+- PATCH /api/v1/admin/locations/{uuid} โ€” update mutable fields (e.g., rack_units_available)
+
+**OpenAPI updates:** Consumer API now 63 paths; Admin API now 44 paths. LocationList, LocationSummary, LocationDetail schemas added to consumer API.
+
+**Placement Engine integration:** Location layers feed LOC-005 sovereignty enforcement (max_data_classification per DC), Step 1 sovereignty pre-filter, Step 3 capability filter, and Placement Policy expressions (input.payload.location.jurisdiction, etc.).
+
+**9 system policies: LOC-001 through LOC-009.**
+
+**Matrix:** 38 domains / 294 capabilities.
+
+
+## SECTION 87 โ€” REFERENCE DATA LAYERS + LAYER-REFERENCED FIELD CONSTRAINTS
+
+**Core clarification applied:** Layer data is the source of allowed values for resource type fields. This is not new architecture โ€” it is the `layer_reference` constraint type made explicit throughout the documentation.
+
+**Pattern:** A field in a Resource Type Specification can declare `constraint.type: layer_reference` with a `layer_type` name. At catalog item render time, DCM resolves the active instances of that layer type into the `allowed_values` list. The consumer selects from that list; DCM injects the full layer data into the assembled payload.
+
+**Standard reference data layer types:**
+- `location.data_center` โ€” where resources can be placed (hierarchy: Countryโ†’Regionโ†’Zoneโ†’Siteโ†’DCโ†’Hallโ†’Cageโ†’Rack)
+- `os_image` โ€” approved OS images (Platform Security team)
+- `vm_size` โ€” approved VM size profiles (Platform Team)
+- `network_zone` โ€” available network zones (Network Operations)
+- `environment` โ€” deployment environments (Platform Governance)
+- `storage_class` โ€” storage tiers (Storage Operations)
+- `gpu_profile` โ€” GPU configurations (Platform Team)
+
+**Governance model:** Reference Data Layer instances have IDENTICAL lifecycle, controls, security, and governance as Resource Types โ€” `developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired`, GitOps workflow, versioned, owned by declared authority, immutable once active. Adding a new approved OS image = adding a new `os_image` layer. No Resource Type Specification change needed.
+
+**Changes made this session:**
+
+doc 03 (layering): New Section 3.7 โ€” Reference Data Layers. Defines the pattern, governance model, and YAML format for `os_image` and `vm_size` reference data layers. Updated Core Layers cross-ref.
+
+doc 05 (resource types): New Section 2.1b โ€” Layer-Referenced Field Constraints. Explains why `layer_reference` is preferred over static enums. Complete table of standard layer types with owning authorities. Shows what the catalog item field constraint looks like when resolved (full `allowed_values` with structured data). Adds `layer_reference` and `layer_reference_list` to the field constraint type vocabulary, including `filter`, `display_field`, `value_field` sub-fields.
+
+consumer-api-spec.md Section 3.2: Catalog item field schema now shows four field examples:
+  - `cpu_count` โ€” static `enum` constraint (unchanged)
+  - `os_image` โ€” `layer_reference` to `os_image` type (resolved `allowed_values` with image metadata)
+  - `location` โ€” `layer_reference` to `location.data_center` type (resolved with DC name, zone, certifications, capacity_status)
+  - `size` โ€” `layer_reference` to `vm_size` type (resolved with CPU/RAM defaults)
+  Section 3.3 submit request shows layer UUIDs in the fields object with comments explaining DCM's resolution.
+
+doc 48 (location): New Section 0 โ€” Pattern Context. Explicitly frames location as one application of the Reference Data Layer pattern. Section 8 (Consumer Selection Model) updated: location selection is via the catalog item field constraint, not a separate /locations endpoint. Shows how filter clause on the layer_reference constraint controls which DCs appear per catalog item.
+
+API changes: Removed standalone /api/v1/locations and /api/v1/locations/{uuid} consumer endpoints (wrong model โ€” location is a field constraint, not a separate API). Admin API keeps /api/v1/admin/location-types and /api/v1/admin/locations (correct โ€” admin manages the Reference Data Layer registry). Consumer API back to 61 paths.
+
+
+## SECTION 88 โ€” RESOURCE TYPE AUTHORITY + UNIFIED LAYER MODEL CLARIFICATION
+
+**Core clarification:** All of DCM's data โ€” Resource Type Specifications, Reference Data Layers, Service Layers, provider extension layers โ€” is built on the same layer model with the same lifecycle, governance, ownership, and security model. This was architecturally correct but not explicitly stated.
+
+**What was already in the architecture (confirmed, no changes needed):**
+- Three-tier registry (DCM Core / Verified Community / Organization) โ€” doc 20
+- Provider as Resource Type Publisher (doc 28 section 6.1)
+- Resource Type Specification vs Catalog Item distinction (doc 05 section 2.1a)
+- Portability classification (universal/conditional/provider-specific/exclusive) โ€” doc 05 section 4
+- Provider-specific extension fields must be marked portability-breaking โ€” doc 05 section 4
+- Layer domain model (system/platform/tenant/service/provider) โ€” doc 03 section 4.1
+- Service Layers contributed by providers โ€” doc 28 section 6.3
+- GitOps workflow for resource type proposals โ€” doc 20 section 3.1
+
+**What was added (4 targeted additions):**
+
+**1. Resource Type Authority โ€” Stewardship Model (doc 05 new Section 2.1c):**
+Every Resource Type Specification has a declared Resource Type Authority โ€” the team responsible for defining, maintaining, evolving, and deprecating it. Same `owned_by` governance model as all other DCM artifacts. Required approver for all future version PRs. Standard authority assignments by category: Computeโ†’Platform Team, Networkโ†’Network Ops, Storageโ†’Storage Ops, Securityโ†’CISO, etc. Three tiers: DCM maintainers (Tier 1), named community maintainers (Tier 2), org domain teams (Tier 3).
+
+**2. Three-Way Field Constraint Model (doc 05 new Section 3a):**
+Resource Type Authorities choose per field:
+- Option 1 โ€” layer_reference: valid values are active instances of a named layer type (location, OS images, network zones). Portable. Value governance delegated to the layer type's authority.
+- Option 2 โ€” provider-declared constraint: ad-hoc enum/range in the Resource Type Spec or Catalog Item. CPU counts, memory ranges, protocol versions. Portable if values are vendor-neutral.
+- Option 3 โ€” no constraint: free-form or provider judgment. Names, descriptions, provider-internal IDs.
+Decision guide table included. Key point: VM size is Option 2 or 3 by default โ€” not layer-referenced unless the org explicitly wants a governed size catalog. This is an organizational decision, not an architectural mandate.
+
+**3. Provider Extension Layers (doc 05 Section 6.2 extended):**
+Providers can contribute extension layers (domain: provider) alongside their catalog item declaration โ€” `provider_extension_layer_handles` field. These inject provider-specific fields during payload assembly only when that catalog item is selected. Cannot override platform/tenant layers. Same portability_breaking: true semantics as inline extensions. Resource Type Authority may adopt popular extensions as conditional fields in a future spec version.
+
+**4. Unified Layer Model Statement (doc 03 Section 3.7 + Section 4.1):**
+Explicit statement that Resource Type Specifications ARE data layer artifacts. Same lifecycle, same governance, same GitOps workflow, same authority model, same domain access control. The Resource Type Registry is a specialized layer store. All DCM data โ€” type definitions, reference data, service configuration, provider extensions โ€” lives in this one unified model.
+
+**Authority vs Publisher clarification (doc 28 Section 6.1):** Resource Type Authority defines the spec; Service Provider publishes the catalog item implementing it. Often the same team. May be different: platform team defines Compute.VirtualMachine; Nutanix, VMware, and bare metal teams independently register catalog items implementing it.
+
+
+## SECTION 89 โ€” SECTION 9 EXAMPLES: RESOURCE TYPE + LAYER LIFECYCLE (dcm-examples.md)
+
+dcm-examples.md expanded from 2,189 to 3,381 lines. Section 9 added โ€” 8 subsections showing
+the complete lifecycle of VM and WebApp resources from layer definition through rehydration.
+
+**9.1 Layer Definitions:** Full YAML for all foundational layers โ€” OS image (Platform Security Team),
+location/DC (Data Center Operations), network zone (Network Operations), zone and country ancestor layers.
+Each with complete artifact_metadata, owned_by, domain, data blocks.
+
+**9.2 Resource Type Specifications:** Complete `Compute.VirtualMachine` v2.1.0 spec showing:
+- Universal fields: cpu_count/memory_gb/storage_gb (range constraints โ€” intrinsic), os_image/location/
+  network_zone/environment (layer_reference โ€” governed lists)
+- Conditional fields: high_availability, gpu_profile (layer_reference)
+- Extension point declaration for provider hypervisor config
+- Application.WebApp v1.0.0 spec: app_name/tier_level (static enum โ€” intrinsic),
+  environment/location (layer_reference), web_replica_count/db_engine (range/enum).
+
+**9.3 Provider Catalog Items:** Two providers implementing Compute.VirtualMachine:
+- Nutanix EU-WEST: portable (portability_class: portable), narrows CPU/RAM enums,
+  contributes Service Layer with AHV hypervisor defaults, backup policy, monitoring agent.
+- VMware EU-WEST: non-portable (portability_class: provider-specific), contributes
+  Provider Extension Layer with vsphere_resource_pool/datastore_cluster/vmware_tools_version.
+
+**9.4-9.5 VM Request and Processing Pipeline:** Full 8-step trace from consumer submission through
+intent capture โ†’ layer reference resolution โ†’ layer assembly (showing each contributing layer and field
+provenance) โ†’ policy evaluation (GateKeeper, Validation, Transformation each shown) โ†’ placement โ†’
+requested state write (with provenance on every field) โ†’ dispatch โ†’ realized state.
+
+**9.6 WebApp as a Service Request:** Meta Provider orchestration: DB first, then 3 web VMs with
+db_host injected from DB realization, then LoadBalancer with backend_pool injected from VM IPs.
+Tier 1 GateKeeper policies enforcing HA, minimum replicas, LTM requirement. Environment layer
+injecting production defaults (backup, TTL=null, approval tier, log retention).
+
+**9.7 VM Rehydration (DR failover DC1โ†’DC2):** Shows location override in placement_constraints,
+fresh layer resolution against DC2 layer chain (different cluster_uuid in Nutanix Service Layer),
+certification gap warning (SOC 2 not at AMS-DC2), hostname preserved / FQDN updated, entity_uuid
+preserved across DC move. Static Replace vs Rehydration contrast explained in pipeline steps.
+
+**9.8 WebApp Rehydration (standards refresh โ€” no incident):** Rolling replacement pattern for Tier 1.
+Shows: retired OS image โ†’ auto-upgrade Transformation policy substitutes RHEL 9.5; environment layer
+v1.2 injects new log_retention_days=365 and vulnerability scanning; new Tier 1 GateKeeper bumps
+replica count 3โ†’4; DB unchanged (no OS dependency). Full audit record showing every field change,
+its source layer/policy, and version.
+
+
+## SECTION 90 โ€” DOCUMENTATION CLEANUP PASS (2026-03)
+
+No prior implementations exist โ€” DCM is at v1. All migration/backward-compat/update language
+removed from documentation. Every doc is the authoritative first-version spec.
+
+**Removed entirely:**
+- All 38 Active Development Notice blockquotes across every data model and specification doc
+- `## 7. V1 Migration` section from doc 13 โ€” V1 concept has no basis in first implementation; `v1_migration` ingestion_source type renamed `legacy_import`; `v1_identifier` โ†’ `legacy_identifier`
+- `## 7. Migration from Current Constructs` section from doc 15 (universal groups)
+- `## 9. Migration Path โ€” Standalone SPA โ†’ RHDH` from RHDH spec โ€” replaced with clean "Deployment Options" section (standalone_spa vs rhdh are configuration choices, not migration paths)
+
+**Reframed (concept kept, backward-looking framing removed):**
+- doc 06 section 7a.6: "Updated table (supersedes 7.2)" โ†’ "Provider Lifecycle Events"
+- doc 07: "dependency graph superseded by entity relationships" โ†’ scope cross-reference
+- doc 08: "superseded by Universal Group Model" โ†’ "Related: see Universal Group Model"
+- doc 09: "This document supersedes dependency graph concept" โ†’ plain cross-reference
+- doc 13: "V1 migration and brownfield ingestion are the same" โ†’ "same ingestion model for all sources"
+- doc 18: "outbound webhook superseded by Notification Model" โ†’ "one delivery channel within Notification Model"
+- doc 23: "This model supersedes standalone webhooks" โ†’ "outbound webhooks are one delivery channel"
+- doc 26: "Section 4 superseded by Governance Matrix" โ†’ scope statement
+- doc 27: "This document supersedes Section 4 of doc 26" โ†’ cross-reference
+- doc 11 (k8s): "Migration Path โ€” Kubernetes-Native to DCM-Managed" โ†’ "Incremental Adoption"
+- doc 34: "Client Migration Path" โ†’ "Version Upgrade Path"; "backward compat" โ†’ "version-compatible"; "deprecation window" โ†’ "until version is sunset"
+- doc 15: "preserve backward compatibility with existing API consumers" โ†’ "for convenience"
+- OIS spec: "during the deprecation window" โ†’ "until the OIS version is sunset"
+
+**Verification:** 12 stale patterns โ€” all clean after cleanup pass.
+
+---
+
+## SECTION 91 โ€” SYNC AUDIT AND FULL RESYNC (2026-03)
+
+Full Hugo content sync performed. State before sync:
+- 5 data model docs missing from Hugo entirely (43-provider-callback-auth, 44-kessel-evaluation, 45-consistency-review, 46-workload-analysis, 47-accreditation-monitor)
+- 44 of 53 data model docs stale (primarily -5 line delta from Active Development Notice removal)
+- 7 specs stale (dcm-examples, dcm-registration-spec, dcm-operator-interface-spec, dcm-rhdh-integration-spec, 11-kubernetes-compatibility, consumer-api-spec, dcm-operator-sdk-api)
+- Capabilities Matrix and DISCUSSION-TOPICS stale
+- AI prompt had one stale matrix reference: "36 domains / 281 capabilities" (from Provider Readiness Gates session before Accreditation Monitor domain was added)
+
+**After sync:** All 53 data model docs ร— 2 Hugo locations: in sync. All 14 specs: in sync. All top-level docs: in sync. AI prompt matrix refs updated to 37/287.
+
+---
+
+## SECTION 92 โ€” ARCHITECTURE GAPS ANALYSIS (2026-03)
+
+Systematic scan of all docs, schemas, and capabilities matrix. Summary:
+
+**Tier 1 โ€” Spec gaps (documented capability, API endpoints missing):**
+- Federation Admin API (doc 22 architecture complete; no OpenAPI paths for tunnel management, peer listing, trust posture)
+- Scheduled/Deferred Requests maintenance-windows endpoints (doc 37 specifies them; not in admin YAML)
+- Workload Analysis endpoints (doc 46 specifies GET /workload-profile and :analyze; not in consumer YAML)
+- Accreditation Monitor contract-event and :configure-webhook endpoints (doc 47; not in admin YAML)
+
+**Tier 2 โ€” Implementation gaps (no specification exists):**
+- Cost Analysis component internal model (376 refs; no spec for how it calculates)
+- Cross-region data replication model (multi-region assumed; consistency + sovereignty enforcement at replication layer unspecified)
+- Secret zero / initial credential bootstrap (day-0 sequence has no spec; chicken-and-egg problem every deployer hits)
+- Multi-tenancy at storage layer (row-level security, tenant-scoped encryption โ€” not specified)
+- Rate limiting implementation (policy references it; enforcement mechanics unspecified)
+- Audit log hash chain implementation (tamper evidence concept documented; implementation not specified)
+
+**Tier 3 โ€” Security posture gaps:**
+- No threat model document (attack surfaces, adversary profiles, STRIDE analysis)
+- No supply chain security spec (SBOM, provider package signing, operator container provenance)
+- No secrets scanning spec for GitOps stores
+
+**Tier 4 โ€” Experience gaps:**
+- New tenant onboarding flow not specified
+- Pre-request cost estimation UX not specified end-to-end
+- Provider sandbox/test mode not specified
+- Capacity forecasting model not specified
+- SLA/SLO tracking not specified
+
+**Two ownership ambiguities:**
+- Who issues operation_uuid โ€” API Gateway or Request Orchestrator?
+- Who owns the Credential Revocation Registry?
+
+---
+
+## SECTION 93 โ€” LIGHTSPEED INTERFACE CONCEPT + DISCUSSION TOPICS (2026-03)
+
+**DISCUSSION-TOPICS.md updated:** Item 6 added โ€” "Universal Lightspeed Interface for Operations."
+
+Concept: a universal, high-velocity operational surface for all DCM actions regardless of provider, resource type, or lifecycle stage. Operations that currently require multiple tool hops, context switching, and approval interruptions should be expressible and executable in a single interaction.
+
+**Key design questions captured for future roadmap:**
+- New GUI surface, CLI, AI agent interface, or all three?
+- How does it relate to existing Web UI spec and Flow GUI spec?
+- What does "lightspeed" mean operationally โ€” sub-second execution, zero-confirmation for pre-approved patterns, predictive pre-staging?
+- How does it interact with Authority Tier model โ€” can it auto-route approval gates without interrupting operator flow?
+- Is this the primary interface for the AIOps layer referenced in the README?
+
+**Status:** Concept only โ€” no design work started. Future roadmap item.
+
+
+## SECTION 94 โ€” ALL ARCHITECTURE GAPS ADDRESSED (2026-03)
+
+All gaps from Section 92 gap analysis resolved.
+
+**Tier 1 โ€” API Endpoint Gaps (all closed):**
+- Federation Admin API: 6 new admin YAML paths including GET/POST /api/v1/admin/federation/peers, :set-trust-posture, :suspend, /routed-requests, GET/PATCH /api/v1/admin/federation/config. Schemas: FederationPeer, FederationPeerRegistration, FederationConfig.
+- Scheduled Requests: 3 new admin paths for maintenance windows: GET/POST /maintenance-windows, GET/PATCH/DELETE /maintenance-windows/{uuid}, GET /maintenance-windows/{uuid}/scheduled-requests. Schemas: MaintenanceWindow, MaintenanceWindowCreate, MaintenanceWindowPatch.
+- Workload Analysis: 2 new consumer paths โ€” GET /resources/{uuid}/workload-profile, POST /resources/{uuid}/workload-profile:analyze. WorkloadProfile schema added.
+- Accreditation Monitor: 3 new admin paths โ€” :verify, :configure-webhook, /contract-event.
+- Final counts: consumer 63 paths / 33 schemas; admin 56 paths / 27 schemas.
+
+**Tier 2 โ€” Implementation Gaps (doc 49 โ€” 49-implementation-specifications.md, 728 lines):**
+- Rate Limiting (sec 2): Token bucket at API Gateway. Per-actor bucket, profile-governed parameters (60/min minimal to 600/min prod), PT120S TTL on state. 5 policies RLM-001 through RLM-005.
+- Audit Hash Chain (sec 3): SHA-256 with canonical concatenation (0x1F separator). GENESIS anchor for chain start. Continuous per-write verification + periodic sweep (PT1H sovereign, PT12H prod). Chain resealing endpoint: POST /api/v1/admin/audit/entities/{uuid}:reseal-chain.
+- Multi-Tenancy at Storage Layer (sec 4): GitOps=directory namespace, EventStream=per-tenant stream, Snapshot=PostgreSQL RLS + AES-256-GCM per-tenant key, Search=index namespace. Cryptographic tenant deletion via key revocation. 4 policies STI-001 through STI-004.
+- Cross-Region Replication (sec 5): Per-store replication model. Sovereignty-aware routing enforced at replication layer. Lag monitoring with degraded/unavailable state transitions. Last-write-wins + vector clocks for conflict resolution.
+- Secret Zero Bootstrap (sec 6): Bootstrap manifest with internal CA + pre-shared component credentials replaced by mTLS within PT5M of CA startup. Credential Provider takes CA key ownership on registration. Air-gapped options: embedded vault / operator passphrase / HSM. 4 policies BOOT-001 through BOOT-004.
+
+**Ownership Ambiguities (sec 7) โ€” Both resolved:**
+- operation_uuid: issued by API Gateway at ingress (operation_uuid == request_uuid). API Gateway writes initial Operation; Request Orchestrator updates via shared fast store.
+- Credential Revocation Registry: owned by Credential Provider. Key: credential_uuid to revocation metadata. TTL: max(credential_ttl, P90D). Session Revocation Registry is separate โ€” owned by Auth component.
+
+**Tier 3 โ€” Security Posture (sec 8):**
+- Threat model: 5 boundaries (Consumer Ingress, Provider Interface, Admin, Internal Component, Storage). Highest-risk: Credential Provider compromise and Internal CA compromise โ€” mitigated by air-gapped Credential Provider and HSM-backed CA for sovereign profiles.
+- Supply chain: provider OpenAPI spec signing (mTLS private key; rejected at GATE-SP-01 if unsigned), container image provenance via Sigstore/Cosign, GitOps secrets scanning, SBOM mandatory for fsi/sovereign. 5 policies SEC-001 through SEC-005.
+
+**Tier 4 โ€” Experience Gaps (sec 9):**
+- New Tenant Onboarding (sec 9.1): full sequence โ€” entity, groups, quota, admin actor, Git namespace, audit stream, member invitations, onboarding_complete event.
+- Pre-Request Cost Estimation UX (sec 9.2): catalog display then POST /cost/estimate with fields then dry_run: true for placement preview then actual submit.
+- Provider Sandbox/Test Mode (sec 9.3): sandbox_mode: true in registration, excluded from production placement, explicit targeting via _test_context, graduation path to production.
+- SLA/SLO Tracking (sec 9.4): SLO declared in Resource Type Specification (time_to_operational, uptime, drift_detection_latency), continuous measurement from audit records, breach events, consumer status endpoint, admin aggregate report endpoint.
+
+
+## SECTION 95 โ€” CONTINUED: MATRIX + SPEC CLEANUP (2026-03)
+
+**Capabilities Matrix updated:** 38 domains / 299 capabilities (was 294).
+New capabilities added from doc 49 implementation specifications:
+- OBS-006: SLA/SLO Declaration (Resource Type Spec + consumer status endpoint + admin report)
+- OBS-007: SLO Breach Detection and Notification
+- STO-007: Cross-Region Sovereignty-Aware Replication (routing enforced at replication layer)
+- STO-008: Tenant-Scoped Storage Isolation (RLS + per-tenant stream + index namespace)
+- STO-009: Tenant-Scoped Encryption for fsi/sovereign (AES-256-GCM, Credential Provider managed)
+- ZTS-007: Provider OpenAPI Spec Signing (SEC-001, mandatory at GATE-SP-01)
+- ZTS-008: GitOps Secrets Scanning (SEC-002, SECRETS_DETECTED rejection)
+- ZTS-009: SBOM Declaration (SEC-003, mandatory fsi/sovereign)
+- PRV-010: Provider Sandbox/Test Mode
+- GOV-008: Tenant Onboarding Workflow
+
+**Admin API:** Added `GET /api/v1/admin/workload-analysis` โ€” aggregate workload profile view across all tenants with archetype/confidence/resource_type filtering and archetype_distribution histogram. Admin API now: 57 paths / 27 schemas.
+
+**All 15 specification docs now have Document Status headers.** 7 specs updated: cncf-strategy, consumer-api-spec, dcm-admin-api-spec, dcm-flow-gui-spec, dcm-opa-integration-spec, dcm-operator-interface-spec, dcm-registration-spec.
+
+**Open markers: 0.** No TODO/FIXME, no Active Dev Notices, no superseded-by language, no V1 Migration references anywhere in the corpus.
+
+**Current state:** 55 data model docs / 15 specifications / 4 OpenAPI schemas / 38 domains / 299 capabilities / 97 prompt sections.
+
+
+## SECTION 96 โ€” PROJECT OVERVIEW DOCUMENT (project-overview.md)
+
+New canonical document added: `project-overview.md` (located at dcm-docs root and Hugo /docs/project-overview).
+
+**Purpose:** Single authoritative description of DCM for any audience โ€” engineers, business stakeholders, executives, or community members encountering the project for the first time. Referenced from README and Hugo navigation as the first document to read.
+
+**Content:**
+- What DCM Is: governing control plane above provisioning tools; not a deployment tool; the management plane that connects existing automation
+- Architecture in one sentence + event loop diagram
+- The Problem DCM Solves: 5 specific problems (fragmented ops, long TTM, private cloud gap, unreliable data, compliance overhead)
+- What DCM Does: three abstractions in full โ€” Data (4 states), Policy (7 types with table), Provider (11 types with table)
+- What this enables: self-service consumer experience; standards enforced structurally
+- Who Benefits: Consumers, Platform Engineers, Security/Compliance, SRE, Auditors, FinOps โ€” each with specific value statement
+- Where DCM Operates: deployment topology table (single-region/federated/hub-regional/sovereign), data sovereignty model, target environments with compliance frameworks listed
+- Key Facts table: accurate current counts (55 docs, 15 specs, 299 caps/38 domains, 63 consumer paths, 57 admin paths)
+- 9 Core Design Principles
+
+**README.md updated:** Accurate counts, new project-overview link as first doc in foundation table, all doc ranges updated (55 data model docs, 15 specs), AI prompt described as 98 sections.
+
+**Hugo updated:**
+- Root _index.md: replaced thin 'About DCM' with full What/Who/Three Abstractions/Benefits layout; Active Dev Notice removed
+- docs/_index.md: replaced Active Dev Notice with navigation cards including project-overview link
+- architecture/overview.md: Active Development Notice removed
+- /docs/project-overview.md: new Hugo page from project-overview.md
+
+
+## SECTION 97 โ€” HOW + ETHOS SECTIONS ADDED
+
+**project-overview.md expanded** from 164 to 327 lines. Two new sections added:
+
+**## How DCM Works** (5 subsections):
+- The Event Loop: policy-driven event loop diagram showing event โ†’ Policy Engine โ†’ typed outputs โ†’ Providers/Data โ†’ new events
+- The Request Lifecycle: 5-step numbered sequence from intent declaration through layer assembly, policy evaluation, dispatch, and ongoing lifecycle
+- How Policy Replaces Hard-Coded Logic: why every business rule is a Policy artifact and what that means operationally
+- How Providers Integrate: base contract + capability extension model; organizations wrap existing automation, not replace it
+- How Data Sovereignty Is Enforced: structural property evaluated at every boundary; Governance Matrix always boolean; no scoring override
+
+**## Ethos** (5 subsections):
+- Security Is the Baseline, Not a Feature: minimal profile = security with minimal overhead, not minimal security; secure path must also be easy path
+- The Governed Path Must Also Be the Easy Path: self-service is the delivery mechanism for governance; if governed path is harder, teams route around it
+- Compliance Is Constructed, Not Audited: audit evidence and provenance are structural products of operations, not reconstructed post-hoc
+- The Architecture Should Be Easy to Implement and Extend: three-abstraction test; no core changes for new capabilities that fit Data/Provider/Policy
+- No Silent Behavior: every operation produces an observable artifact; every state transition audited; every decision has typed output
+
+**README.md updated** with condensed How + Ethos sections: event loop summary, request lifecycle in one paragraph, four design priority ethos statements, links to full sections in project-overview.md.
+
+
+## SECTION 98 โ€” WORKING INSTRUCTIONS FOR AI MODELS
 
 When working on this project, apply these instructions in addition to the numbered guidance in SECTION 60 (Documentation Structure):
 
diff --git a/content/docs/architecture/data-model/49-implementation-specifications.md b/content/docs/architecture/data-model/49-implementation-specifications.md
new file mode 100644
index 0000000..9adb7b2
--- /dev/null
+++ b/content/docs/architecture/data-model/49-implementation-specifications.md
@@ -0,0 +1,728 @@
+# DCM Data Model โ€” Implementation Specifications
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Implementation Reference
+**Related Documents:** [Control Plane Components](25-control-plane-components.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit](16-universal-audit.md) | [Credential Provider Model](31-credential-provider-model.md) | [Deployment Redundancy](17-deployment-redundancy.md) | [Session Revocation](35-session-revocation.md)
+
+> **AEP Alignment:** API endpoints follow [AEP](https://aep.dev) conventions.
+> See `schemas/openapi/dcm-admin-api.yaml` and `dcm-consumer-api.yaml` for normative specs.
+
+---
+
+## 1. Purpose
+
+This document specifies the implementation mechanics for capabilities that are architecturally defined elsewhere but whose runtime behavior โ€” enforcement location, algorithm, data structure โ€” has not been fully specified. It closes implementation gaps identified in the architecture gap analysis.
+
+---
+
+## 2. Rate Limiting โ€” Enforcement Implementation
+
+Rate limiting is defined at the interface level in the Consumer API Specification (ยง1.6) and the Admin API. This section specifies *how* it is enforced.
+
+### 2.1 Enforcement Location
+
+Rate limiting is enforced by the **API Gateway** component โ€” the single ingress point for all consumer and admin API traffic. It is enforced before the request reaches any pipeline component. The Request Orchestrator never sees rate-limited requests.
+
+Rate limiting is **not** enforced at the network layer (load balancer) or application layer (Request Payload Processor). A single enforcement point at the API Gateway ensures:
+- Consistent limits across all consumer paths (Web UI, direct API, CI/CD)
+- No rate limit bypass via internal component calls
+- Single source of rate limit state for accurate tracking
+
+### 2.2 Token Bucket Algorithm
+
+DCM uses the **token bucket** algorithm with a per-actor bucket:
+
+```
+Actor makes request:
+  โ”‚
+  โ–ผ API Gateway looks up actor_uuid in rate limit store
+  โ”‚   (in-memory cache backed by a fast Storage Provider sub-type)
+  โ”‚
+  โ–ผ Current bucket state:
+  โ”‚   tokens_remaining: 
+  โ”‚   last_refill_at: 
+  โ”‚
+  โ–ผ Refill calculation:
+  โ”‚   elapsed_seconds = now - last_refill_at
+  โ”‚   tokens_to_add = elapsed_seconds ร— (rate_limit / 60)
+  โ”‚   tokens_remaining = min(bucket_max, tokens_remaining + tokens_to_add)
+  โ”‚   last_refill_at = now
+  โ”‚
+  โ–ผ Token check:
+  โ”œโ”€โ”€ tokens_remaining >= 1:
+  โ”‚     tokens_remaining -= 1
+  โ”‚     Request proceeds
+  โ”‚     Response headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
+  โ”‚
+  โ””โ”€โ”€ tokens_remaining < 1:
+        Request rejected: 429 Too Many Requests
+        Response headers: Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
+        Audit record written: rate_limit_exceeded
+```
+
+### 2.3 Bucket Parameters by Profile
+
+| Profile | Rate (req/min) | Burst Max | Bucket Max |
+|---------|---------------|-----------|------------|
+| `minimal` | 60 | 20 | 80 |
+| `dev` | 120 | 40 | 160 |
+| `standard` | 300 | 100 | 400 |
+| `prod` | 600 | 200 | 800 |
+| `fsi` | 600 | 200 | 800 |
+| `sovereign` | 600 | 200 | 800 |
+
+System components (service accounts, provider callbacks) use `prod` bucket parameters regardless of profile.
+
+### 2.4 Rate Limit State Store
+
+The rate limit state is stored in a **dedicated in-memory cache** backed by a fast Storage Provider:
+- Cache TTL: 2ร— the rate limit window (120 seconds for 60 req/min rate)
+- Storage Provider type: key-value (Redis or equivalent)
+- Consistency: eventual โ€” brief over-counting tolerated to avoid distributed lock overhead
+- Cross-replica sharing: rate limit state is shared across all API Gateway replicas via the backing store
+
+### 2.5 Exemptions
+
+The following are exempt from consumer rate limits:
+- Admin API calls (separate rate limit bucket, 3ร— profile limit)
+- Provider callback endpoints (authenticated via provider callback credential; separate per-provider bucket)
+- Internal DCM component calls (authenticated via mTLS + interaction credential; not rate limited)
+- Health check endpoints (`/livez`, `/readyz`, `/metrics`)
+
+### 2.6 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `RLM-001` | Rate limiting is enforced at the API Gateway. No other component enforces rate limits. |
+| `RLM-002` | Rate limit buckets are per authenticated actor (actor_uuid). Unauthenticated requests are rejected at auth before reaching the rate limiter. |
+| `RLM-003` | All rate limit rejections produce an audit record with actor_uuid, endpoint, and timestamp. |
+| `RLM-004` | Rate limit parameters are governed by the active Profile. Operators may increase but not decrease profile-defined limits. |
+| `RLM-005` | Rate limit state is not persisted across API Gateway restarts. Buckets refill from empty after restart โ€” brief over-serving is acceptable. |
+
+---
+
+## 3. Audit Log Hash Chain โ€” Verification Schedule and Implementation
+
+The hash chain structure is defined in [Universal Audit](16-universal-audit.md) ยง8. This section specifies the verification schedule, triggering component, and response protocol.
+
+### 3.1 Hash Computation
+
+Each audit record's `record_hash` is computed as:
+
+```
+record_hash = SHA-256(
+    record_uuid ||
+    record_timestamp ||
+    entity_uuid ||
+    action ||
+    actor.immediate.uuid ||
+    subject_handle ||
+    chain_sequence ||
+    previous_record_hash
+)
+
+Where || denotes canonical concatenation with a field separator (0x1F โ€” ASCII unit separator).
+The hash is stored as a lowercase hex string.
+```
+
+The `previous_record_hash` for the first record in an entity's chain is `SHA-256("GENESIS")` โ€” a known constant, not null.
+
+### 3.2 Verification Schedule
+
+Hash chain verification runs on two schedules:
+
+**Continuous verification (per-write):**
+Every audit record write triggers an immediate verification of that record against its predecessor. This catches chain breaks at write time โ€” before the record is committed. A write that would break the chain is rejected and triggers `audit.chain_integrity_alert`.
+
+**Periodic batch verification (scheduled):**
+The Audit component runs a full-chain verification sweep on a profile-governed schedule:
+
+| Profile | Sweep interval | Scope per sweep |
+|---------|---------------|-----------------|
+| `dev` | P7D | All entities modified in the last 7 days |
+| `standard` | P1D | All entities modified in the last 24 hours |
+| `prod` | PT12H | All entities modified in the last 12 hours |
+| `fsi` | PT6H | All entities + random 5% sample of all-time records |
+| `sovereign` | PT1H | All entities + random 10% sample of all-time records |
+
+### 3.3 Owning Component
+
+Hash chain verification is owned by the **Audit component** โ€” the same component that writes audit records. It is not a separate service. The Audit component runs verification as a background goroutine with no external trigger required.
+
+For Storage Provider implementations: the Audit Store must support ordered range queries by `(entity_uuid, chain_sequence)` to enable efficient sweep verification.
+
+### 3.4 Breach Response Protocol
+
+```
+Chain break detected (during write-time or sweep verification):
+  โ”‚
+  โ–ผ Affected records flagged: integrity_status = chain_break
+  โ”‚   Break point: chain_sequence N where hash mismatch occurs
+  โ”‚   All records with chain_sequence > N for this entity: integrity_status = unverified
+  โ”‚
+  โ–ผ audit.chain_integrity_alert event fired (urgency: critical, non-suppressable)
+  โ”‚   payload: {entity_uuid, entity_type, break_at_sequence, break_detected_at, sweep_type}
+  โ”‚
+  โ–ผ Notifications dispatched:
+  โ”‚   โ†’ Platform Admin (urgency: critical)
+  โ”‚   โ†’ Security team (if configured in notification routing)
+  โ”‚
+  โ–ผ Affected entity flagged in audit dashboard
+  โ”‚   Consumer-visible: "Audit integrity alert โ€” contact platform admin"
+  โ”‚
+  โ””โ”€โ”€ Human investigation required:
+        Normal resolution paths:
+        - Storage Provider failure caused write corruption โ†’ Storage Provider replacement
+        - Clock skew between replicas caused ordering issue โ†’ Non-malicious; document and reseal
+        - Administrative error (direct DB edit) โ†’ Incident report, access review
+        - Malicious tampering โ†’ Security incident declared
+```
+
+### 3.5 Chain Resealing
+
+After a chain break is investigated and root cause documented, a platform admin may reseal the chain:
+
+```
+POST /api/v1/admin/audit/entities/{entity_uuid}:reseal-chain
+  {
+    "investigation_reference": "INC-2026-042",
+    "root_cause": "storage_failure",
+    "resolution_notes": "PostgreSQL WAL corruption during storage migration"
+  }
+
+Response:
+  {
+    "entity_uuid": "",
+    "chain_resealed_at": "",
+    "records_affected": 7,
+    "new_chain_anchor": "",
+    "audit_record_uuid": ""
+  }
+```
+
+The reseal itself produces an audit record that references the investigation. Chain integrity is restored from the reseal point forward.
+
+---
+
+## 4. Multi-Tenancy at the Storage Layer
+
+Tenant isolation in DCM is enforced at the data model level (every entity carries `tenant_uuid`) and at the API level (all consumer endpoints are scoped to the authenticated actor's tenant). This section specifies the storage-layer enforcement mechanisms.
+
+### 4.1 Isolation Strategy by Store Type
+
+| Store Type | Isolation Strategy | Implementation Notes |
+|-----------|-------------------|---------------------|
+| **GitOps Store** (Intent/Requested State) | Directory namespace per tenant | `/tenants/{tenant_uuid}/intents/`, `/tenants/{tenant_uuid}/requests/` โ€” Git ACLs enforce read/write scope |
+| **Event Stream Store** (Audit) | Separate stream per tenant | `dcm.audit.{tenant_uuid}` stream; Kafka topic ACLs restrict producer/consumer access |
+| **Snapshot Store** (Realized State) | Row-level filter + column-level encryption | `tenant_uuid` column indexed; all queries mandatory-include `WHERE tenant_uuid = ?`; tenant-scoped encryption key |
+| **Search Index** | Index namespace per tenant | Separate index prefix `tenant_{uuid}_*`; query routing enforces tenant scope |
+| **Rate Limit Cache** | Key-namespaced per actor (actor carries tenant context) | `rl:{tenant_uuid}:{actor_uuid}` key structure |
+
+### 4.2 Snapshot Store โ€” Row-Level Security Implementation
+
+The Snapshot Store (Realized State) uses row-level security as the primary isolation mechanism:
+
+```sql
+-- PostgreSQL row-level security policy
+CREATE POLICY tenant_isolation ON realized_state_records
+  USING (tenant_uuid = current_setting('dcm.current_tenant_uuid')::uuid);
+
+-- Every connection sets tenant context before any query:
+SET LOCAL dcm.current_tenant_uuid = '';
+
+-- This makes it impossible to query across tenant boundaries,
+-- even with direct database access using the application credential.
+-- Platform admin access uses a separate role without the RLS policy.
+```
+
+### 4.3 Tenant-Scoped Encryption
+
+For `fsi` and `sovereign` profiles, realized state records are encrypted at rest using a per-tenant encryption key:
+
+```
+Tenant provisioned:
+  โ”‚
+  โ–ผ Credential Provider generates tenant encryption key (AES-256-GCM)
+  โ”‚   Key stored in: Credential Provider (e.g., Vault)
+  โ”‚   Key reference stored in: Tenant record as tenant_encryption_key_ref
+  โ”‚
+  โ–ผ On write to Snapshot Store:
+  โ”‚   API Gateway fetches tenant encryption key
+  โ”‚   Payload encrypted with tenant key before storage
+  โ”‚   Storage Provider stores ciphertext only
+  โ”‚
+  โ–ผ On read from Snapshot Store:
+  โ”‚   API Gateway fetches tenant encryption key
+  โ”‚   Decrypts payload in memory
+  โ”‚   Plaintext never written to Storage Provider logs
+  โ”‚
+  โ–ผ Tenant decommission:
+      Tenant encryption key revoked in Credential Provider
+      All tenant data becomes unreadable without external recovery
+      This is the cryptographic equivalent of data deletion
+```
+
+### 4.4 Cross-Tenant Query Prevention
+
+Platform admin endpoints that query across tenants use a separate database role with explicit permission grants โ€” they do not bypass RLS, they use a role that has cross-tenant read permission with full audit logging. The principle is: cross-tenant reads are possible only through intentional, audited, privileged operations.
+
+### 4.5 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `STI-001` | Every query to a tenant-scoped store must include `tenant_uuid` as a mandatory predicate. Queries without tenant scope are rejected by the storage layer. |
+| `STI-002` | Row-level security is enabled on all relational snapshot stores. Disabling RLS requires platform admin action and produces an audit record. |
+| `STI-003` | For `fsi` and `sovereign` profiles, tenant-scoped encryption is mandatory. Key rotation is performed on a profile-governed schedule (P90D for fsi; P30D for sovereign). |
+| `STI-004` | Storage Provider implementations must declare their tenant isolation strategy at registration. DCM validates the declared strategy against the active profile's isolation requirements during the registration approval pipeline. |
+
+---
+
+## 5. Cross-Region Data Replication
+
+DCM's multi-region deployment model is specified in [Deployment Redundancy](17-deployment-redundancy.md). This section specifies the replication mechanics and sovereignty enforcement at the replication layer.
+
+### 5.1 What Replicates Where
+
+| Store Type | Replication Model | Sovereignty Constraint |
+|-----------|------------------|----------------------|
+| **GitOps Store** (Intent State) | Git push/pull โ€” upstream-downstream replication | Intent records tagged with `sovereignty_zone`; replicated only to stores within the declared zone |
+| **Event Stream Store** (Audit) | Stream mirroring with lag monitoring | Audit records replicated to all authorized regions; cross-sovereignty replication requires explicit consent |
+| **Snapshot Store** (Realized State) | Synchronous within-zone; async cross-zone with consent | `sovereignty_zone` on entity governs which regions may hold a copy |
+| **Search Index** | Async replication; eventual consistency acceptable | Same sovereignty rules as Snapshot Store |
+
+### 5.2 Sovereignty-Aware Replication
+
+Every entity carries `sovereignty_zone` declarations that constrain which Storage Provider instances may hold copies:
+
+```yaml
+entity:
+  entity_uuid: 
+  sovereignty_zones:
+    - zone_id: EU-WEST
+      data_classifications: [restricted, phi]   # these classifications must stay in EU-WEST
+    - zone_id: "*"
+      data_classifications: [internal]           # internal data may replicate anywhere
+```
+
+The replication controller evaluates `sovereignty_zones` before routing any replication event. Replication to a non-authorized region for a given data classification is blocked at the replication layer โ€” the storage provider receives a `SOVEREIGNTY_VIOLATION` rejection.
+
+### 5.3 Replication Lag Monitoring
+
+```
+Storage Provider declares: max_replication_lag: PT30S
+
+DCM monitoring:
+  Every PT10S: measure replication lag across all replica pairs
+  
+  If lag > max_replication_lag:
+    storage.replication_lag_exceeded event (urgency: medium)
+    
+  If lag > 5 ร— max_replication_lag:
+    storage.replication_degraded event (urgency: high)
+    Affected region marked: capacity_status = degraded
+    New requests avoid degraded region for placement
+    
+  If replica unreachable:
+    storage.replica_unavailable event (urgency: critical)
+    Affected region marked: capacity_status = unavailable
+    Requests that require this region: held pending recovery
+```
+
+### 5.4 Conflict Resolution
+
+DCM uses a **last-write-wins with causality tracking** model for cross-region conflicts:
+
+- All writes carry a vector clock `{region_id: sequence_number}`
+- Concurrent writes (same entity, different regions) are detected by vector clock comparison
+- Resolution: the write with higher aggregate sequence number wins
+- Losing write: preserved as a `conflict_record` in the Audit Store (never silently dropped)
+- Platform admin notified of conflicts above a configurable threshold
+
+---
+
+## 6. Secret Zero โ€” Initial Credential Bootstrap
+
+The bootstrap sequence is specified in [Deployment Redundancy](17-deployment-redundancy.md) ยง6. This section specifies the credential bootstrap specifically โ€” how DCM components authenticate to each other before the Credential Provider is running.
+
+### 6.1 The Bootstrap Credential Problem
+
+At day-0, no Credential Provider exists. DCM components need credentials to communicate. The resolution is a **declarative bootstrap manifest** that contains one-time bootstrap credentials, plus a mandatory rotation on first successful startup.
+
+### 6.2 Bootstrap Sequence โ€” Credential Perspective
+
+```
+1. Bootstrap manifest contains:
+   bootstrap_credentials:
+     internal_ca:
+       cert_pem: 
+       key_pem:    # sealed with bootstrap passphrase
+       
+     bootstrap_admin:
+       username: bootstrap-admin
+       password_hash:   # operator sets this
+       
+     component_credentials:
+       # Pre-shared credentials for component-to-component auth
+       # until mTLS internal CA is operational
+       api_gateway:    {shared_secret: }
+       orchestrator:   {shared_secret: }
+       policy_engine:  {shared_secret: }
+       audit:          {shared_secret: }
+
+2. Bootstrap DCM starts:
+   - Internal CA initialized from bootstrap_credentials.internal_ca
+   - Components issued mTLS certificates from Internal CA
+   - Pre-shared secrets replaced by mTLS certificates on first successful CA handshake
+   - Pre-shared secrets deleted from memory and manifest after replacement
+
+3. Credential Provider starts:
+   - Bootstrapped with Internal CA certificate (trusts DCM's CA)
+   - Registered as the primary Credential Provider via bootstrap admin credential
+   - Takes ownership of internal CA key management
+   - Internal CA private key: transferred to Credential Provider, deleted from bootstrap manifest
+
+4. Bootstrap admin credential rotation (BOOT-002 โ€” mandatory):
+   - Bootstrap admin password must be rotated on first login
+   - New credential issued by Credential Provider (not the bootstrap manifest)
+   - Old password hash deleted from manifest
+   - Manifest sealed: no more secrets, only configuration
+
+5. Bootstrap manifest after completion:
+   - Contains only: DCM deployment configuration, Git remote, profile
+   - No secrets remain in the manifest
+   - Manifest committed to Git (now safe, secret-free)
+```
+
+### 6.3 Air-Gapped Bootstrap
+
+For sovereign/air-gapped deployments where the Credential Provider requires network access to an external vault:
+
+```
+Option A โ€” Embedded Credential Provider:
+  Use a locally-running Credential Provider (e.g., HashiCorp Vault in dev mode)
+  bootstrapped from the bootstrap manifest.
+  Upgrade to production Vault config post-bootstrap.
+
+Option B โ€” Operator-held keys:
+  Bootstrap manifest contains encrypted key material.
+  Operator provides passphrase at bootstrap time via stdin or hardware token.
+  Keys are never stored unencrypted at rest.
+  
+Option C โ€” HSM-backed bootstrap:
+  Internal CA private key is generated inside an HSM.
+  Bootstrap manifest contains only the HSM endpoint and slot reference.
+  Requires HSM to be available before DCM bootstrap begins.
+```
+
+### 6.4 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `BOOT-001` | The bootstrap manifest must not contain secrets after bootstrap completion. Any secret that persists in the manifest after first successful startup is a security violation. |
+| `BOOT-002` | The bootstrap admin credential must be rotated on first login. DCM enforces this โ€” the bootstrap admin account is locked from normal use until rotation is complete. |
+| `BOOT-003` | Pre-shared component credentials must be replaced by mTLS certificates within PT5M of Internal CA startup. Any component still using pre-shared secrets after this window generates a security alert. |
+| `BOOT-004` | The Internal CA private key must be transferred to the Credential Provider on Credential Provider registration. The key must not remain in any component's memory or storage after transfer is confirmed. |
+
+---
+
+## 7. Ownership Ambiguities โ€” Resolved
+
+### 7.1 Who Issues `operation_uuid`?
+
+**Decision: The API Gateway issues `operation_uuid` at request ingress.**
+
+Rationale: The API Gateway is the component that receives the POST request and must return the Operation response immediately. It assigns the UUID synchronously before any pipeline processing begins. The `operation_uuid` equals the `request_uuid` โ€” they are the same UUID, assigned at ingress.
+
+```
+Consumer: POST /api/v1/requests {...}
+
+API Gateway:
+  1. Authenticates consumer (checks session token)
+  2. Assigns request_uuid = operation_uuid = UUID4()  โ† here
+  3. Writes initial request record to Intent Store (status: INITIATED)
+  4. Publishes request.initiated event to Request Orchestrator (with request_uuid)
+  5. Returns Operation{name: /api/v1/operations/{request_uuid}, done: false}
+
+Request Orchestrator:
+  Receives request.initiated event with request_uuid
+  Uses the already-assigned request_uuid throughout the pipeline
+  Never assigns a new UUID
+```
+
+The Operation resource lives in a fast-queryable store owned by the API Gateway. When the pipeline progresses, the Request Orchestrator updates the Operation status by writing to this store (it has write access; the API Gateway reads from it for GET /api/v1/operations/{uuid} responses).
+
+### 7.2 Who Owns the Credential Revocation Registry?
+
+**Decision: The Credential Provider owns the Credential Revocation Registry.**
+
+Rationale: The Credential Provider is the authoritative source of credential lifecycle state. It issues credentials, rotates them, and revokes them. The revocation registry is a projection of that lifecycle state optimized for fast lookup.
+
+```
+Credential Revocation Registry:
+  Owner: Credential Provider
+  Storage: dedicated fast cache (Redis or equivalent)
+  Key structure: credential_uuid โ†’ {revoked_at, revocation_reason, effective_at}
+  TTL: max(credential_ttl, P90D)  โ€” persists at minimum 90 days after revocation
+
+Access model:
+  Write: Credential Provider (on revocation event)
+  Read:  All DCM components (via Credential Provider query API)
+         OR via local cache synced from Credential Provider push events
+
+Cache sync protocol:
+  Credential Provider publishes: credential.revoked event (Message Bus)
+  All subscribed components update local revocation cache
+  Cache TTL: PT1M standard; PT30S fsi/sovereign
+  On cache miss: component queries Credential Provider directly (not the cache)
+
+Session Revocation Registry: separate, owned by the Auth component
+  (Session revocation is distinct from credential revocation)
+```
+
+---
+
+## 8. Security Posture Specifications
+
+### 8.1 Threat Model โ€” Attack Surface Summary
+
+DCM's attack surface has five distinct boundaries. Each boundary has a specific trust model and mitigation set.
+
+**Boundary 1 โ€” Consumer Ingress (Web UI, Consumer API)**
+- Threat: Credential theft / session hijacking
+- Mitigations: mTLS optional at consumer boundary; bearer tokens with short TTL (PT1H standard); session revocation registry checked on every request; rate limiting at API Gateway
+- Threat: Tenant escape (accessing another tenant's data)
+- Mitigations: All queries mandatory-include tenant_uuid; row-level security on storage; Governance Matrix enforced before any read
+
+**Boundary 2 โ€” Provider Interface (Operator Interface, Callback API)**
+- Threat: Provider impersonation (malicious actor claims to be a legitimate provider)
+- Mitigations: mTLS required at provider boundary; provider callback credential required for callbacks; API Gateway validates dcm_entity_uuid in every callback against provider's registered entity scope
+- Threat: Malicious provider payload (provider sends crafted Realized State payload)
+- Mitigations: Realized State payloads validated against Resource Type Specification schema on receipt; GateKeeper policies evaluate provider-supplied data before it enters DCM state
+
+**Boundary 3 โ€” Admin Interface (Admin API)**
+- Threat: Unauthorized platform admin action
+- Mitigations: Authority Tier model enforces multi-tier approval for high-impact actions; all admin actions produce non-suppressable audit records; emergency admin access (break-glass) triggers immediate security notification
+- Threat: Configuration injection via GitOps
+- Mitigations: All GitOps PRs require domain-appropriate review before merge; policy contributions enter shadow mode before activation; GateKeeper policies validate all contributions at submission
+
+**Boundary 4 โ€” Internal Component Communication**
+- Threat: Component impersonation (compromised component issues requests as another)
+- Mitigations: mTLS with Internal CA for all component-to-component calls; interaction credentials checked on every call; Credential Revocation Registry queried on credential use
+- Threat: Lateral movement after component compromise
+- Mitigations: Each component holds minimum-scope interaction credentials; no component has write access to stores it does not own; audit records cannot be deleted by any component
+
+**Boundary 5 โ€” Storage Layer**
+- Threat: Direct database access bypassing application controls
+- Mitigations: Row-level security enforces tenant isolation even with direct DB access using application credentials; platform admin credentials are separate, audited, and require MFA; Storage Provider provenance emission means all direct writes are detectable
+
+**Highest-risk paths (not mitigated by single control):**
+1. Credential Provider compromise โ†’ cascading trust failure. Mitigation: Credential Provider is air-gapped from consumer traffic; HSM-backed key storage for sovereign profiles; separate backup credential authority.
+2. Internal CA compromise โ†’ all component trust fails. Mitigation: CA private key held only in Credential Provider (HSM-backed for fsi/sovereign); CA certificate rotation procedure documented.
+
+### 8.2 Supply Chain Security
+
+**Provider OpenAPI Spec Signing:**
+All Service Provider OpenAPI specifications submitted at registration must be signed using the provider's private key (corresponding to the public key in their mTLS certificate). DCM verifies the signature before the spec is processed. Unsigned specs are rejected with `SPEC_UNSIGNED` at GATE-SP-01.
+
+**Operator Container Image Provenance:**
+The DCM reference implementation containers are signed using Sigstore (Cosign). Deployment manifests declare the expected image digest. Any container running a different digest triggers drift detection on DCM's own deployment.
+
+**GitOps Store Secrets Scanning:**
+All content committed to DCM's GitOps stores passes through a secrets scanner before being accepted. The scanner checks for:
+- High-entropy strings matching known secret patterns (API keys, tokens, private keys)
+- Known credential formats (AWS access keys, GitHub PATs, JWT secrets)
+- PEM-encoded private key blocks
+
+A commit containing detected secrets is rejected with `SECRETS_DETECTED` and an audit record is written. The committing actor is notified.
+
+**SBOM Declaration:**
+Service Providers must declare a Software Bill of Materials reference at registration (optional for Tier 1 `dev` profiles; required for `fsi` and `sovereign`). The SBOM reference is stored in the provider record and included in accreditation evidence.
+
+### 8.3 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `SEC-001` | All provider OpenAPI specs submitted at registration must be signed. Signature verification is performed at GATE-SP-01. |
+| `SEC-002` | DCM GitOps stores enforce secrets scanning on all commits. Commits with detected secrets are rejected. |
+| `SEC-003` | For `fsi` and `sovereign` profiles, SBOM declaration is mandatory for all Service Providers before activation. |
+| `SEC-004` | The Internal CA private key must be stored in an HSM for `sovereign` profile deployments. Software-only key storage is not permitted at sovereign profile. |
+| `SEC-005` | Any direct database write to a DCM store that bypasses the application layer is detectable via Storage Provider provenance emission. Detection triggers `audit.chain_integrity_alert` for affected records. |
+
+---
+
+## 9. Experience Gap Specifications
+
+### 9.1 New Tenant Onboarding Flow
+
+```
+Platform Admin initiates tenant creation:
+  POST /api/v1/admin/tenants
+  {
+    "display_name": "Payments Platform",
+    "handle": "payments-platform",
+    "group_class": "tenant_boundary",
+    "initial_quota_profile": "standard",
+    "billing_contact": "payments-ops@corp.example.com",
+    "data_classifications_permitted": ["internal", "restricted"],
+    "sovereignty_zones": ["EU-WEST"]
+  }
+
+DCM auto-provisions:
+  1. Tenant entity created (tenant_uuid assigned)
+  2. Default resource groups created:
+     - payments-platform/default (general resources)
+     - payments-platform/admins (tenant admin group)
+  3. Initial quota applied per quota_profile declaration
+  4. Tenant admin actor created (if initial_admin_email provided):
+     - Actor record created
+     - Welcome notification dispatched with first-login credential
+  5. Tenant Git namespace provisioned in GitOps store:
+     - /tenants/payments-platform/ directory created
+     - Initial tenant-scope policy stubs committed (shadow mode)
+  6. Search index namespace initialized
+  7. Audit stream created: dcm.audit.{tenant_uuid}
+
+Tenant admin completes setup:
+  1. First login โ†’ mandatory credential rotation (BOOT-002 equivalent)
+  2. Configure Auth Provider (or inherit platform default)
+  3. Add tenant members (invite by email or LDAP group mapping)
+  4. Review and activate initial policy stubs
+  5. Submit first service request (onboarding validation complete)
+
+Onboarding event sequence:
+  tenant.created โ†’ Platform Admin
+  tenant.member_added ร— N โ†’ new members (welcome email)
+  tenant.quota_configured โ†’ Platform Admin
+  tenant.onboarding_complete โ†’ Platform Admin + Tenant Admin
+  (fired when first OPERATIONAL entity exists in the tenant)
+```
+
+### 9.2 Pre-Request Cost Estimation UX
+
+The consumer experience for cost estimation before committing a request:
+
+```
+Step 1: Consumer browses catalog
+  GET /api/v1/catalog/{catalog_item_uuid}
+  Response includes: cost_estimate: {monthly_usd: 45.00, basis: "declared_static"}
+
+Step 2: Consumer configures request fields (e.g., selects VM size)
+  POST /api/v1/cost/estimate
+  {
+    "catalog_item_uuid": "",
+    "fields": {"cpu": 8, "ram_gb": 32, "storage_gb": 200, "environment": "prod"}
+  }
+  Response:
+  {
+    "estimated_monthly_usd": 187.50,
+    "cost_basis": "dynamic",
+    "cost_breakdown": [
+      {"component": "compute", "monthly_usd": 120.00},
+      {"component": "storage", "monthly_usd": 40.00},
+      {"component": "network_egress", "monthly_usd": 27.50}
+    ],
+    "disclaimer": "Estimate based on declared provider rates. Actual costs may vary.",
+    "provider_uuid": null  // not yet placed; estimate is across eligible providers
+  }
+
+Step 3: Consumer submits request with dry_run: true (optional pre-flight)
+  POST /api/v1/requests
+  {
+    "catalog_item_uuid": "",
+    "fields": {...},
+    "dry_run": true    // evaluate policy and placement; do not dispatch
+  }
+  Response: Operation with metadata.dry_run_result:
+  {
+    "policy_result": "PASS",
+    "placement_result": {
+      "selected_provider": "eu-west-prod-1",
+      "cost_at_selected_provider": 182.00
+    },
+    "gatekeeper_gates": [],
+    "warnings": ["Storage class 'premium' requested; 'standard' also eligible at $35.00/mo"]
+  }
+
+Step 4: Consumer submits without dry_run โ†’ actual request
+```
+
+### 9.3 Provider Sandbox / Test Mode
+
+Providers can register in sandbox mode for development and certification testing without affecting production routing:
+
+```yaml
+provider_registration:
+  # ...standard registration fields...
+  sandbox_mode: true           # this provider never receives production requests
+  sandbox_profile: dev         # sandbox providers only activated under dev profile
+  
+  # Sandbox providers:
+  # - Appear in the provider registry with status: sandbox
+  # - Can be explicitly targeted by test requests (fields.target_provider_uuid)
+  # - Never appear in placement engine candidate selection for non-test requests
+  # - Produce full audit records (useful for certification evidence)
+  # - Subject to same API validation as production providers
+  # - Can graduate to production via standard registration approval flow
+```
+
+Test request targeting a sandbox provider:
+```
+POST /api/v1/requests
+{
+  "catalog_item_uuid": "",
+  "fields": { ... },
+  "_test_context": {
+    "target_provider_uuid": "",
+    "suppress_billing": true,
+    "test_label": "certification-run-2026-04-01"
+  }
+}
+```
+
+### 9.4 SLA/SLO Tracking
+
+DCM tracks service delivery against declared SLOs at the Resource Type level:
+
+```yaml
+# Declared in Resource Type Specification (doc 05)
+resource_type_slo:
+  resource_type: Compute.VirtualMachine
+  
+  slos:
+    - metric: time_to_operational
+      target_percentile: p95
+      target_value: PT30M       # 95% of VMs should be OPERATIONAL within 30 minutes
+      measurement_window: P7D   # measured over trailing 7 days
+      
+    - metric: uptime
+      target_percentile: p99
+      target_value: "99.5%"     # 99.5% uptime over trailing 30 days
+      measurement_window: P30D
+      
+    - metric: drift_detection_latency
+      target_percentile: p90
+      target_value: PT1H        # drift detected within 1 hour of occurrence
+      measurement_window: P7D
+```
+
+SLO breach detection:
+```
+DCM computes SLO metrics continuously from audit records and entity lifecycle events.
+
+When a metric crosses a threshold:
+  slo.breach_approaching (urgency: medium) โ€” at 90% of SLO budget consumed
+  slo.breach_detected (urgency: high) โ€” SLO violated
+    payload: {resource_type, slo_metric, target, actual, measurement_window}
+
+Consumer-facing:
+  GET /api/v1/resources/{entity_uuid}/slo-status
+  Returns: current SLO metrics for the entity's resource type
+  
+Platform admin:
+  GET /api/v1/admin/slo/report?resource_type=Compute.VirtualMachine&window=P7D
+  Returns: aggregate SLO performance across all entities of this type
+```
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/architecture/overview.md b/content/docs/architecture/overview.md
index f6c084e..bb7bdb0 100644
--- a/content/docs/architecture/overview.md
+++ b/content/docs/architecture/overview.md
@@ -1,16 +1,11 @@
 ---
-title: High Level Design
+title: Architecture Overview
 type: docs
 weight: 1
 ---
 
 # DCM High Level Design
 
-> **โš ๏ธ Active Development Notice**
->
-> The DCM data model and architecture documentation are actively being developed. Concepts, structures, and specifications documented here represent work in progress and are subject to change as design decisions are finalized.
->
-> Contributions, feedback, and discussion are welcome via [GitHub](https://github.com/dcm-project).
 
 ---
 
diff --git a/content/docs/architecture/specifications/cncf-strategy.md b/content/docs/architecture/specifications/cncf-strategy.md
index f4bd3c3..d296b2c 100644
--- a/content/docs/architecture/specifications/cncf-strategy.md
+++ b/content/docs/architecture/specifications/cncf-strategy.md
@@ -1,5 +1,9 @@
 # DCM โ€” CNCF Strategy and Community Engagement Plan
 
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Strategy Document
+
+
 
 
 **Version:** 0.1.0-draft  
diff --git a/content/docs/architecture/specifications/consumer-api-spec.md b/content/docs/architecture/specifications/consumer-api-spec.md
index 9b7826b..579c459 100644
--- a/content/docs/architecture/specifications/consumer-api-spec.md
+++ b/content/docs/architecture/specifications/consumer-api-spec.md
@@ -1,5 +1,9 @@
 # DCM Consumer API Specification
 
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** API Narrative Specification
+
+
 > **๐Ÿ“‹ Draft**
 >
 > This specification covers the full Consumer API surface. Endpoint paths, request/response structures, and authentication flows represent design intent and will be refined as implementation proceeds. Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues).
@@ -371,42 +375,149 @@ Response 200:
 
   "schema": {
     "fields": [
+
+      // โ”€โ”€ Static constraint fields โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
       {
         "field_name": "cpu_count",
+        "display_name": "CPU Cores",
         "type": "integer",
         "required": true,
         "editable_post_realization": false,
         "constraint": {
-          "visibility": "full",            # full | summary | hidden
-          "type": "range",
-          "min": 1,
-          "max": 32,
+          "visibility": "full",
+          "type": "enum",
           "allowed_values": [1, 2, 4, 8, 16, 32],
+          "default": 4,
           "reason": "CPU counts must be powers of 2 for NUMA alignment"
         }
       },
+
+      // โ”€โ”€ Layer-referenced field: OS image โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+      // Allowed values come from active os_image Reference Data Layers.
+      // Each entry carries the full structured layer data the GUI needs.
+      // Consumer submits the layer UUID; DCM injects all image metadata into payload.
       {
-        "field_name": "memory_gb",
-        "type": "integer",
+        "field_name": "os_image",
+        "display_name": "Operating System Image",
+        "type": "string",
         "required": true,
         "editable_post_realization": false,
         "constraint": {
           "visibility": "full",
-          "type": "range",
-          "min": 2,
-          "max": 256
+          "type": "layer_reference",
+          "layer_type": "os_image",
+          "allowed_values": [
+            {
+              "value": "layer-uuid-rhel-9-4",
+              "display_name": "RHEL 9.4",
+              "os_family": "rhel",
+              "version": "9.4",
+              "fips_compliant": true,
+              "eol_date": "2032-05-31",
+              "approved_for_classifications": ["public","internal","confidential","restricted"]
+            },
+            {
+              "value": "layer-uuid-ubuntu-24-04",
+              "display_name": "Ubuntu 24.04 LTS",
+              "os_family": "ubuntu",
+              "version": "24.04",
+              "fips_compliant": false,
+              "eol_date": "2029-04-30",
+              "approved_for_classifications": ["public","internal"]
+            }
+          ]
         }
       },
+
+      // โ”€โ”€ Layer-referenced field: location โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+      // Allowed values come from active location.data_center layers the
+      // consumer is entitled to and that this catalog item is eligible for.
+      // Selecting a location causes the full location layer chain
+      // (Country โ†’ Region โ†’ Zone โ†’ Site โ†’ DC) to assemble into the payload.
+      {
+        "field_name": "location",
+        "display_name": "Allocation Location",
+        "type": "string",
+        "required": true,
+        "editable_post_realization": false,
+        "constraint": {
+          "visibility": "full",
+          "type": "layer_reference",
+          "layer_type": "location.data_center",
+          "allowed_values": [
+            {
+              "value": "layer-uuid-fra-dc1",
+              "display_name": "DC1 โ€” Frankfurt Alpha",
+              "code": "FRA-DC1",
+              "zone": "eu-west-1a",
+              "region": "EU West",
+              "sovereignty": "EU/GDPR",
+              "certifications": ["ISO 27001", "SOC 2 Type II"],
+              "max_data_classification": "restricted",
+              "capacity_status": "available"
+            },
+            {
+              "value": "layer-uuid-ams-dc2",
+              "display_name": "DC2 โ€” Amsterdam Beta",
+              "code": "AMS-DC2",
+              "zone": "eu-west-1b",
+              "region": "EU West",
+              "sovereignty": "EU/GDPR",
+              "certifications": ["ISO 27001"],
+              "max_data_classification": "confidential",
+              "capacity_status": "limited"
+            }
+          ]
+        }
+      },
+
+      // โ”€โ”€ Layer-referenced field: approved_size โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+      // Allowed values come from active vm_size Reference Data Layers.
+      // Selecting a size injects CPU, RAM, and storage defaults into the payload
+      // (which the consumer can override within the size's declared constraints).
+      {
+        "field_name": "size",
+        "display_name": "VM Size",
+        "type": "string",
+        "required": false,
+        "editable_post_realization": false,
+        "constraint": {
+          "visibility": "full",
+          "type": "layer_reference",
+          "layer_type": "vm_size",
+          "allowed_values": [
+            {
+              "value": "layer-uuid-small",
+              "display_name": "Small (2 CPU / 8 GB)",
+              "cpu_count": 2, "memory_gb": 8, "storage_gb": 40
+            },
+            {
+              "value": "layer-uuid-medium",
+              "display_name": "Medium (8 CPU / 32 GB)",
+              "cpu_count": 8, "memory_gb": 32, "storage_gb": 80
+            },
+            {
+              "value": "layer-uuid-large",
+              "display_name": "Large (16 CPU / 64 GB)",
+              "cpu_count": 16, "memory_gb": 64, "storage_gb": 160
+            }
+          ]
+        }
+      },
+
+      // โ”€โ”€ Policy-injected hidden field โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
       {
         "field_name": "monitoring_agent",
         "type": "string",
         "required": false,
         "editable_post_realization": false,
         "constraint": {
-          "visibility": "hidden",         # injected by policy โ€” consumer cannot set
-          "override": "immutable"
+          "visibility": "hidden",
+          "override": "immutable",
+          "note": "Injected by policy โ€” consumer cannot set or view"
         }
       }
+
     ]
   },
 
diff --git a/content/docs/architecture/specifications/dcm-admin-api-spec.md b/content/docs/architecture/specifications/dcm-admin-api-spec.md
index a2169c7..9cf78d4 100644
--- a/content/docs/architecture/specifications/dcm-admin-api-spec.md
+++ b/content/docs/architecture/specifications/dcm-admin-api-spec.md
@@ -1,5 +1,9 @@
 # DCM Admin API Specification
 
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** API Narrative Specification
+
+
 > **๐Ÿ“‹ Draft**
 >
 > This specification has been promoted from Work in Progress to Draft status. Complete Admin API covering all platform admin operations with request/response examples. It is ready for implementation feedback but has not yet been formally reviewed for final release.
diff --git a/content/docs/architecture/specifications/dcm-admin-api.yaml b/content/docs/architecture/specifications/dcm-admin-api.yaml
new file mode 100644
index 0000000..675ab50
--- /dev/null
+++ b/content/docs/architecture/specifications/dcm-admin-api.yaml
@@ -0,0 +1,1830 @@
+openapi: "3.1.0"
+
+info:
+  title: DCM Admin API
+  version: "1.0.0"
+  description: |
+    The DCM Admin API is used by platform engineers, SREs, and system administrators to manage
+    the DCM control plane itself. All endpoints require platform admin or higher authority.
+
+    **Key principles:**
+    - Requires `verified` or `authorized` tier authority for most operations
+    - Mutating operations against the governance model (tier registry, profiles) require
+      `authorized` tier with a declared DCMGroup quorum
+    - All actions produce audit records
+    - Destructive or degrading operations have explicit confirmation steps
+
+
+
+    **AEP Alignment:** This API follows [AEP](https://aep.dev) conventions:
+    custom methods use colon syntax (`POST /resources/{name}:suspend`),
+    async operations return an `Operation` resource (AEP-136 LRO),
+    and list pagination uses `page_size`/`page_token` parameters.
+
+  contact:
+    name: DCM Project
+    url: https://github.com/dcm-project
+  license:
+    name: Apache 2.0
+    url: https://www.apache.org/licenses/LICENSE-2.0
+
+servers:
+  - url: https://{dcm-host}/
+    description: DCM Control Plane
+    variables:
+      dcm-host:
+        default: dcm.example.com
+
+security:
+  - BearerAuth: []
+
+tags:
+  - name: health
+    description: DCM control plane health and readiness
+  - name: tenants
+    description: Tenant lifecycle management
+  - name: actors
+    description: Actor session and risk management
+  - name: providers
+    description: Provider registration approval and management
+  - name: accreditations
+    description: Provider accreditation review
+  - name: discovery
+    description: Brownfield discovery scheduling and monitoring
+  - name: drift
+    description: Drift orphan management and recovery decisions
+  - name: quotas
+    description: Tenant quota management
+  - name: search
+    description: Search index operations
+  - name: bootstrap
+    description: Bootstrap credential management
+  - name: scoring
+    description: Risk scoring model configuration
+  - name: approvals
+    description: Platform-level approval management
+  - name: tier-registry
+    description: Authority tier registry management
+
+paths:
+
+  # โ”€โ”€โ”€ HEALTH โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /livez:
+    get:
+      tags: [health]
+      operationId: liveness
+      summary: Kubernetes-style liveness probe
+      security: []
+      responses:
+        "200": { description: Control plane process is alive }
+        "503": { description: Control plane unhealthy }
+
+  /readyz:
+    get:
+      tags: [health]
+      operationId: readiness
+      summary: Kubernetes-style readiness probe
+      security: []
+      responses:
+        "200": { description: Control plane ready to serve requests }
+        "503": { description: Control plane not ready (stores unavailable, bootstrap incomplete, etc.) }
+
+  /metrics:
+    get:
+      tags: [health]
+      operationId: metrics
+      summary: Prometheus metrics endpoint
+      security: []
+      responses:
+        "200":
+          description: Prometheus text format metrics
+          content:
+            text/plain:
+              schema: { type: string }
+
+  /api/v1/admin/health:
+    get:
+      tags: [health]
+      operationId: getAdminHealth
+      summary: Detailed control plane health including component and provider status
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/AdminHealthResponse" }
+
+  # โ”€โ”€โ”€ TENANTS โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/tenants:
+    get:
+      tags: [tenants]
+      operationId: listTenants
+      summary: List all Tenants
+      parameters:
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+        - name: status
+          in: query
+          schema: { type: string, enum: [active, suspended, decommissioned] }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/TenantList" }
+    post:
+      tags: [tenants]
+      operationId: createTenant
+      summary: Create a new Tenant
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { $ref: "#/components/schemas/TenantCreate" }
+      responses:
+        "201":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/Tenant" }
+        "409": { description: Tenant handle already exists }
+
+  /api/v1/admin/tenants/{tenant_uuid}:suspend:
+    post:
+      tags: [tenants]
+      operationId: suspendTenant
+      summary: Suspend a Tenant (blocks all new requests; active resources remain)
+      parameters:
+        - { $ref: "#/components/parameters/tenant_uuid" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: { type: string }
+      responses:
+        "200": { description: Tenant suspended }
+        "409": { description: Tenant already suspended or decommissioned }
+
+  /api/v1/admin/tenants/{tenant_uuid}:reinstate:
+    post:
+      tags: [tenants]
+      operationId: reinstateTenant
+      summary: Reinstate a suspended Tenant
+      parameters:
+        - { $ref: "#/components/parameters/tenant_uuid" }
+      responses:
+        "200": { description: Tenant reinstated }
+
+  /api/v1/admin/tenants/{tenant_uuid}:
+    delete:
+      tags: [tenants]
+      operationId: decommissionTenant
+      summary: Decommission a Tenant (must have zero active resources)
+      parameters:
+        - { $ref: "#/components/parameters/tenant_uuid" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason, confirmation]
+              properties:
+                reason:       { type: string }
+                confirmation: { type: string, const: "DECOMMISSION", description: "Must be the string 'DECOMMISSION'" }
+      responses:
+        "200":
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/Operation" }
+        "409": { description: Tenant has active resources โ€” cannot decommission }
+
+  /api/v1/admin/tenants/{tenant_uuid}/quotas:
+    get:
+      tags: [quotas]
+      operationId: getTenantQuotas
+      summary: Get quota configuration for a Tenant
+      parameters:
+        - { $ref: "#/components/parameters/tenant_uuid" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/TenantQuotas" }
+
+  /api/v1/admin/tenants/{tenant_uuid}/quotas/{resource_type}:
+    put:
+      tags: [quotas]
+      operationId: setTenantQuota
+      summary: Set or update quota for a specific resource type on a Tenant
+      parameters:
+        - { $ref: "#/components/parameters/tenant_uuid" }
+        - name: resource_type
+          in: path
+          required: true
+          schema: { type: string }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { $ref: "#/components/schemas/QuotaUpdate" }
+      responses:
+        "200": { description: Quota updated }
+
+  # โ”€โ”€โ”€ ACTORS โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/actors/{actor_uuid}:revoke-sessions:
+    post:
+      tags: [actors]
+      operationId: revokeActorSessions
+      summary: Revoke all active sessions for an actor (emergency session revocation)
+      parameters:
+        - { $ref: "#/components/parameters/actor_uuid" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: { type: string }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  sessions_revoked: { type: integer }
+
+  /api/v1/admin/actors/{actor_uuid}/sessions:
+    get:
+      tags: [actors]
+      operationId: getActorSessions
+      summary: List all active sessions for an actor
+      parameters:
+        - { $ref: "#/components/parameters/actor_uuid" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  sessions: { type: array, items: { type: object } }
+
+  /api/v1/admin/actors/{actor_uuid}/risk-history:
+    get:
+      tags: [actors]
+      operationId: getActorRiskHistory
+      summary: Get risk score history for an actor
+      parameters:
+        - { $ref: "#/components/parameters/actor_uuid" }
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/RiskHistory" }
+
+  /api/v1/admin/actors/{actor_uuid}/risk-history:reset:
+    post:
+      tags: [actors]
+      operationId: resetActorRiskHistory
+      summary: Reset risk score history for an actor (requires verified tier)
+      parameters:
+        - { $ref: "#/components/parameters/actor_uuid" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: { type: string }
+      responses:
+        "200": { description: Risk history reset }
+
+  # โ”€โ”€โ”€ PROVIDERS โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/location-types:
+    get:
+      tags: [locations]
+      operationId: listLocationTypes
+      summary: List registered location types (standard and custom)
+      security:
+        - bearerAuth: []
+      responses:
+        "200":
+          description: All registered location types
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  location_types:
+                    type: array
+                    items: { type: object, additionalProperties: true }
+
+  /api/v1/admin/location-types:
+    post:
+      tags: [locations]
+      operationId: registerCustomLocationType
+      summary: Register a custom location type
+      security:
+        - bearerAuth: []
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { type: object, additionalProperties: true }
+      responses:
+        "200":
+          description: Operation initiated
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/Operation" }
+
+  /api/v1/admin/locations:
+    get:
+      tags: [locations]
+      operationId: adminListLocations
+      summary: List all location nodes (admin โ€” no entitlement filter)
+      parameters:
+        - { name: level, in: query, schema: { type: string } }
+        - { name: page_size, in: query, schema: { type: integer, default: 100 } }
+        - { name: page_token, in: query, schema: { type: string } }
+      security:
+        - bearerAuth: []
+      responses:
+        "200":
+          description: All location nodes
+          content:
+            application/json:
+              schema: { type: object, additionalProperties: true }
+
+  /api/v1/admin/locations/{location_uuid}:
+    patch:
+      tags: [locations]
+      operationId: updateLocationCapacity
+      summary: Update mutable location fields (e.g., rack_units_available)
+      parameters:
+        - { name: location_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      security:
+        - bearerAuth: []
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { type: object, additionalProperties: true }
+      responses:
+        "200":
+          description: Location updated
+          content:
+            application/json:
+              schema: { type: object, additionalProperties: true }
+        "404": { $ref: "#/components/responses/NotFound" }
+
+
+  /api/v1/admin/providers:
+    get:
+      tags: [providers]
+      operationId: listProviders
+      summary: List all registered providers
+      parameters:
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+        - name: provider_type
+          in: query
+          schema: { type: string }
+        - name: status
+          in: query
+          schema: { type: string }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/ProviderList" }
+
+  /api/v1/admin/providers/pending:
+    get:
+      tags: [providers]
+      operationId: listPendingProviders
+      summary: List providers awaiting approval
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/ProviderList" }
+
+  /api/v1/admin/providers/{provider_uuid}:approve:
+    post:
+      tags: [providers]
+      operationId: approveProvider
+      summary: Approve a provider registration
+      parameters:
+        - { $ref: "#/components/parameters/provider_uuid" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason:             { type: string }
+                conditions:         { type: string, description: "Any conditions attached to approval" }
+                external_reference: { type: string }
+      responses:
+        "200": { description: Provider approved and activated }
+        "409": { description: Provider not in PENDING_APPROVAL state }
+
+  /api/v1/admin/providers/{provider_uuid}:reject:
+    post:
+      tags: [providers]
+      operationId: rejectProvider
+      summary: Reject a provider registration
+      parameters:
+        - { $ref: "#/components/parameters/provider_uuid" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: { type: string }
+      responses:
+        "200": { description: Provider rejected }
+
+  /api/v1/admin/providers/{provider_uuid}:suspend:
+    post:
+      tags: [providers]
+      operationId: suspendProvider
+      summary: Suspend a provider (no new requests routed; existing resources unaffected)
+      parameters:
+        - { $ref: "#/components/parameters/provider_uuid" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason:   { type: string }
+                duration: { type: string, description: "ISO 8601 duration; null = indefinite" }
+      responses:
+        "200": { description: Provider suspended }
+
+  # โ”€โ”€โ”€ ACCREDITATIONS โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/accreditations:
+    get:
+      tags: [accreditations]
+      operationId: listAccreditations
+      summary: List provider accreditations (pending, active, expiring soon)
+      parameters:
+        - name: status
+          in: query
+          schema: { type: string, enum: [pending, active, expired, expiring_soon] }
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/AccreditationList" }
+
+  /api/v1/admin/accreditations/{accreditation_uuid}:approve:
+    post:
+      tags: [accreditations]
+      operationId: approveAccreditation
+      summary: Approve a submitted accreditation
+      parameters:
+        - { name: accreditation_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason:             { type: string }
+                external_reference: { type: string }
+      responses:
+        "200": { description: Accreditation approved }
+
+  /api/v1/admin/accreditations/{accreditation_uuid}:
+    delete:
+      tags: [accreditations]
+      operationId: revokeAccreditation
+      summary: Revoke an active accreditation
+      parameters:
+        - { name: accreditation_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: { type: string }
+      responses:
+        "200": { description: Accreditation revoked; affected entities notified }
+
+  # โ”€โ”€โ”€ DISCOVERY โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/discovery:trigger:
+    post:
+      tags: [discovery]
+      operationId: triggerDiscovery
+      summary: Trigger an immediate discovery cycle for one or all providers
+      requestBody:
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                provider_uuid: { type: string, format: uuid, description: "Omit to trigger all active providers" }
+                scope:         { type: string, enum: [full, targeted], default: full }
+      responses:
+        "202":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  discovery_job_uuid: { type: string, format: uuid }
+
+  /api/v1/admin/discovery/jobs/{discovery_job_uuid}:
+    get:
+      tags: [discovery]
+      operationId: getDiscoveryJob
+      summary: Get status of a discovery job
+      parameters:
+        - { name: discovery_job_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/DiscoveryJobStatus" }
+
+  # โ”€โ”€โ”€ DRIFT / ORPHANS โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/orphans:
+    get:
+      tags: [drift]
+      operationId: listOrphans
+      summary: List discovered entities with no matching Requested State record
+      parameters:
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/OrphanList" }
+
+  /api/v1/admin/orphans/{orphan_candidate_uuid}/resolve:
+    post:
+      tags: [drift]
+      operationId: resolveOrphan
+      summary: Resolve an orphan candidate (ingest, ignore, or decommission)
+      parameters:
+        - { name: orphan_candidate_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [resolution]
+              properties:
+                resolution:        { type: string, enum: [ingest, ignore, decommission] }
+                target_tenant_uuid: { type: string, format: uuid, description: "Required for 'ingest'" }
+                reason:            { type: string }
+      responses:
+        "200": { description: Orphan resolved }
+
+  /api/v1/admin/recovery-decisions/pending:
+    get:
+      tags: [drift]
+      operationId: listPendingRecoveryDecisions
+      summary: List platform-level recovery decisions awaiting admin resolution
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  decisions: { type: array, items: { type: object } }
+
+  /api/v1/admin/recovery-decisions/{recovery_decision_uuid}:
+    post:
+      tags: [drift]
+      operationId: resolveRecoveryDecision
+      summary: Resolve a platform-level recovery decision
+      parameters:
+        - { name: recovery_decision_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [decision, reason]
+              properties:
+                decision: { type: string, enum: [approve, reject, escalate] }
+                reason:   { type: string }
+      responses:
+        "200": { description: Decision recorded }
+
+  # โ”€โ”€โ”€ SEARCH INDEX โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/search-index:rebuild:
+    post:
+      tags: [search]
+      operationId: rebuildSearchIndex
+      summary: Trigger a full search index rebuild
+      responses:
+        "202":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  job_uuid: { type: string, format: uuid }
+
+  /api/v1/admin/search-index/status:
+    get:
+      tags: [search]
+      operationId: getSearchIndexStatus
+      summary: Get current search index status and last rebuild time
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  status:       { type: string }
+                  entity_count: { type: integer }
+                  last_rebuilt: { type: string, format: date-time }
+
+  # โ”€โ”€โ”€ BOOTSTRAP โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/bootstrap:rotate-credential:
+    post:
+      tags: [bootstrap]
+      operationId: rotateBootstrapCredential
+      summary: Rotate the bootstrap credential (zero-day trust credential rotation)
+      description: |
+        Rotates the bootstrap credential used for initial DCM trust establishment.
+        Requires the current credential to be presented and records the rotation in
+        the Audit Store. New credential is returned once; cannot be retrieved again.
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [current_credential_ref, reason]
+              properties:
+                current_credential_ref: { type: string }
+                reason:                 { type: string }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  new_credential: { type: string, description: "New bootstrap credential. Shown once." }
+                  rotated_at:     { type: string, format: date-time }
+
+  # โ”€โ”€โ”€ SCORING โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/profiles/{profile_name}/scoring:
+    get:
+      tags: [scoring]
+      operationId: getProfileScoring
+      summary: Get scoring model configuration for a profile
+      parameters:
+        - { $ref: "#/components/parameters/profile_name" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/ScoringConfiguration" }
+
+    patch:
+      tags: [scoring]
+      operationId: updateProfileScoring
+      summary: Update scoring configuration for a profile (threshold adjustments)
+      parameters:
+        - { $ref: "#/components/parameters/profile_name" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { $ref: "#/components/schemas/ScoringConfigurationUpdate" }
+      responses:
+        "200": { description: Scoring configuration updated }
+        "422": { description: Configuration violates SMX-008 hard constraint (auto max_score โ‰ค 50) }
+
+  /api/v1/admin/profiles/{profile_name}/scoring/overrides:
+    post:
+      tags: [scoring]
+      operationId: addScoringOverride
+      summary: Add a per-policy enforcement class override for a profile
+      parameters:
+        - { $ref: "#/components/parameters/profile_name" }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { $ref: "#/components/schemas/ScoringOverride" }
+      responses:
+        "201": { description: Override added }
+
+  /api/v1/admin/actors/{actor_uuid}/risk-history:
+    get:
+      tags: [scoring]
+      operationId: getActorRiskHistoryAdmin
+      summary: Get risk score history for an actor (admin view with full detail)
+      parameters:
+        - { $ref: "#/components/parameters/actor_uuid" }
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/RiskHistory" }
+
+  /api/v1/admin/scoring/audit:
+    get:
+      tags: [scoring]
+      operationId: getScoringAudit
+      summary: Get scoring evaluation audit records for review
+      parameters:
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+        - name: actor_uuid
+          in: query
+          schema: { type: string, format: uuid }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  records: { type: array, items: { type: object } }
+
+  # โ”€โ”€โ”€ APPROVALS โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/approvals/pending:
+    get:
+      tags: [approvals]
+      operationId: listPlatformApprovals
+      summary: List all platform-level approvals pending admin action
+      parameters:
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+        - name: tier
+          in: query
+          schema: { type: string }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  approvals: { type: array, items: { $ref: "#/components/schemas/ApprovalRecord" } }
+
+  /api/v1/admin/approvals/{approval_uuid}:
+    get:
+      tags: [approvals]
+      operationId: getApproval
+      summary: Get full detail for an approval record including all decisions
+      parameters:
+        - { name: approval_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/ApprovalDetail" }
+
+  /api/v1/admin/approvals/{approval_uuid}:vote:
+    post:
+      tags: [approvals]
+      operationId: recordAdminApprovalVote
+      summary: Record a platform admin approval vote
+      parameters:
+        - { name: approval_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [decision, reason]
+              properties:
+                decision:           { type: string, enum: [approve, reject, abstain] }
+                reason:             { type: string }
+                external_reference: { type: string }
+      responses:
+        "200": { description: Vote recorded; approval gate re-evaluated }
+
+  # โ”€โ”€โ”€ AUTHORITY TIER REGISTRY โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  /api/v1/admin/tier-registry/changes:
+    post:
+      tags: [tier-registry]
+      operationId: proposeTierRegistryChange
+      summary: Propose a change to the authority tier registry
+      description: |
+        Submits a proposed updated tier list. DCM computes the tier impact diff immediately โ€”
+        comparing the proposed ordered list to the current one and classifying every changed
+        tier as SECURITY_DEGRADATION, SECURITY_UPGRADE, BROKEN_REFERENCE, PROFILE_GAP, or
+        STALE_WEIGHT. Returns the impact report UUID for inspection.
+        The change cannot activate until all SECURITY_DEGRADATION and BROKEN_REFERENCE items
+        are explicitly accepted. See 32-authority-tier-model.md Section 7.
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: { $ref: "#/components/schemas/TierRegistryChangeProposal" }
+      responses:
+        "202":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  change_uuid:        { type: string, format: uuid }
+                  impact_report_uuid: { type: string, format: uuid }
+                  blocking_items:     { type: integer, description: "Number of SECURITY_DEGRADATION or BROKEN_REFERENCE items" }
+                  status:             { type: string, enum: [pending_review, ready_to_activate] }
+
+    get:
+      tags: [tier-registry]
+      operationId: listTierRegistryChanges
+      summary: List tier registry change proposals
+      parameters:
+        - name: status
+          in: query
+          schema: { type: string, enum: [pending_review, ready_to_activate, activated, rejected] }
+        - { $ref: "#/components/parameters/page_size" }
+        - { $ref: "#/components/parameters/page_token" }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  changes: { type: array, items: { type: object } }
+
+  /api/v1/admin/tier-registry/changes/{change_uuid}/impact:
+    get:
+      tags: [tier-registry]
+      operationId: getTierRegistryImpact
+      summary: Get the full impact report for a proposed tier registry change
+      parameters:
+        - { name: change_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      responses:
+        "200":
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/TierImpactReport" }
+
+  /api/v1/admin/tier-registry/changes/{change_uuid}:accept-degradation:
+    post:
+      tags: [tier-registry]
+      operationId: acceptTierDegradation
+      summary: Accept a specific SECURITY_DEGRADATION item (requires verified tier)
+      description: |
+        Accepts a single SECURITY_DEGRADATION item identified in the impact report.
+        The accepting actor must be at `verified` tier or above and must provide a reason
+        describing what compensating controls justify the degradation.
+        The change cannot activate until ALL SECURITY_DEGRADATION items are accepted.
+      parameters:
+        - { name: change_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [degradation_item_uuid, reason, compensating_controls]
+              properties:
+                degradation_item_uuid: { type: string, format: uuid }
+                reason:                { type: string, minLength: 20, description: "Why this degradation is acceptable" }
+                compensating_controls: { type: string, minLength: 20, description: "What compensating controls are in place" }
+      responses:
+        "200": { description: Degradation accepted; impact report updated }
+        "403": { description: Actor does not meet required tier (verified or above)" }
+        "409": { description: Item already accepted or change not in pending_review state }
+
+  /api/v1/admin/tier-registry/changes/{change_uuid}:activate:
+    post:
+      tags: [tier-registry]
+      operationId: activateTierRegistryChange
+      summary: Activate a proposed tier registry change
+      description: |
+        Activates the proposed tier list as the new authoritative authority tier registry.
+        Returns 409 if any SECURITY_DEGRADATION or BROKEN_REFERENCE items remain unaccepted.
+        Impact report is stored in the Audit Store at activation time.
+        Requires `authorized` tier.
+      parameters:
+        - { name: change_uuid, in: path, required: true, schema: { type: string, format: uuid } }
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: { type: string }
+      responses:
+        "200": { description: Tier registry updated; new ordered list now active }
+        "403": { description: Actor does not meet required tier (authorized)" }
+        "409":
+          description: Blocking items remain unaccepted
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  blocking_items: { type: array, items: { type: object } }
+
+# โ”€โ”€โ”€ COMPONENTS โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+
+  # โ”€โ”€ Accreditation Monitor โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+  /api/v1/admin/accreditations/{accreditation_uuid}:verify:
+    post:
+      tags: [accreditation]
+      operationId: triggerAccreditationVerification
+      summary: Trigger immediate external verification of an accreditation
+      parameters:
+        - {name: accreditation_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      requestBody:
+        content:
+          application/json:
+            schema:
+              type: object
+              properties:
+                override_reason: {type: string, description: "Required when manually overriding last_verified_at in air-gapped mode"}
+      responses:
+        "200":
+          description: Operation initiated
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/Operation"}
+        "404": {$ref: "#/components/responses/NotFound"}
+
+  /api/v1/admin/accreditations/{accreditation_uuid}:configure-webhook:
+    post:
+      tags: [accreditation]
+      operationId: configureAccreditationWebhook
+      summary: Configure a contract management webhook for Tier 3 accreditation verification
+      description: |
+        Used for BAA and DoD IL accreditations. Registers the contract management
+        system (DocuSign, Ironclad, etc.) to send lifecycle events to DCM when
+        the underlying contract is signed, amended, or terminated.
+      parameters:
+        - {name: accreditation_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [contract_system, contract_id]
+              properties:
+                contract_system: {type: string, enum: [docusign, ironclad, agiloft, custom]}
+                contract_id: {type: string, description: ID in the contract management system}
+                webhook_secret: {type: string, description: HMAC secret for webhook authentication}
+      responses:
+        "200":
+          description: Webhook configured; webhook_url returned for registration in contract system
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  accreditation_uuid: {type: string, format: uuid}
+                  webhook_url: {type: string, format: uri, description: "Register this URL in your contract management system"}
+                  webhook_secret_set: {type: boolean}
+
+  /api/v1/admin/accreditations/{accreditation_uuid}/contract-event:
+    post:
+      tags: [accreditation]
+      operationId: receiveAccreditationContractEvent
+      summary: Inbound webhook โ€” receive contract lifecycle event from contract management system
+      description: |
+        Called by contract management systems (DocuSign, Ironclad, etc.) when
+        the underlying BAA or authorization contract changes state.
+        Authenticated via HMAC signature using the webhook_secret.
+      parameters:
+        - {name: accreditation_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{webhookHmac: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [contract_event_type, contract_id, effective_date]
+              properties:
+                contract_event_type: {type: string, enum: [signed, amended, terminated, renewal_due, renewed]}
+                contract_id: {type: string}
+                effective_date: {type: string, format: date-time}
+                details: {type: object, additionalProperties: true}
+      responses:
+        "200":
+          description: Event received and processed
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  accreditation_uuid: {type: string, format: uuid}
+                  dcm_action_taken: {type: string, enum: [activated, pending_review, revoked, none]}
+        "401": {$ref: "#/components/responses/Unauthorized"}
+        "404": {$ref: "#/components/responses/NotFound"}
+
+  # โ”€โ”€ Maintenance Windows โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+  /api/v1/admin/maintenance-windows:
+    get:
+      tags: [scheduling]
+      operationId: listMaintenanceWindows
+      summary: List declared maintenance windows
+      parameters:
+        - {name: status, in: query, schema: {type: string, enum: [active, upcoming, expired]}}
+        - {name: page_size, in: query, schema: {type: integer, default: 50}}
+        - {name: page_token, in: query, schema: {type: string}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Maintenance window list
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  items: {type: array, items: {$ref: "#/components/schemas/MaintenanceWindow"}}
+                  next_page_token: {type: string}
+    post:
+      tags: [scheduling]
+      operationId: createMaintenanceWindow
+      summary: Declare a new maintenance window
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: {$ref: "#/components/schemas/MaintenanceWindowCreate"}
+      responses:
+        "200":
+          description: Maintenance window created
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/MaintenanceWindow"}
+        "422": {$ref: "#/components/responses/UnprocessableEntity"}
+
+  /api/v1/admin/maintenance-windows/{window_uuid}:
+    get:
+      tags: [scheduling]
+      operationId: getMaintenanceWindow
+      summary: Get maintenance window details including scheduled requests in queue
+      parameters:
+        - {name: window_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Maintenance window detail
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/MaintenanceWindow"}
+        "404": {$ref: "#/components/responses/NotFound"}
+    patch:
+      tags: [scheduling]
+      operationId: updateMaintenanceWindow
+      summary: Update maintenance window schedule or description
+      parameters:
+        - {name: window_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: {$ref: "#/components/schemas/MaintenanceWindowPatch"}
+      responses:
+        "200":
+          description: Maintenance window updated
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/MaintenanceWindow"}
+    delete:
+      tags: [scheduling]
+      operationId: deleteMaintenanceWindow
+      summary: Delete a maintenance window (cancels queued requests if policy dictates)
+      parameters:
+        - {name: window_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Deleted; queued_request_disposition indicates what happened to waiting requests
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  window_uuid: {type: string, format: uuid}
+                  queued_requests_affected: {type: integer}
+                  queued_request_disposition: {type: string, enum: [cancelled, reassigned, held]}
+
+  /api/v1/admin/maintenance-windows/{window_uuid}/scheduled-requests:
+    get:
+      tags: [scheduling]
+      operationId: listWindowScheduledRequests
+      summary: List requests queued for this maintenance window
+      parameters:
+        - {name: window_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+        - {name: page_size, in: query, schema: {type: integer, default: 50}}
+        - {name: page_token, in: query, schema: {type: string}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Queued requests
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  items: {type: array, items: {type: object, additionalProperties: true}}
+                  next_page_token: {type: string}
+
+  # โ”€โ”€ Federation Management โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+  /api/v1/admin/federation/peers:
+    get:
+      tags: [federation]
+      operationId: listFederationPeers
+      summary: List all registered federation peer DCM instances
+      parameters:
+        - {name: trust_posture, in: query, schema: {type: string, enum: [verified, vouched, untrusted]}}
+        - {name: status, in: query, schema: {type: string, enum: [active, suspended, pending]}}
+        - {name: page_size, in: query, schema: {type: integer, default: 50}}
+        - {name: page_token, in: query, schema: {type: string}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: List of federation peer registrations
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  items: {type: array, items: {$ref: "#/components/schemas/FederationPeer"}}
+                  next_page_token: {type: string}
+        "401": {$ref: "#/components/responses/Unauthorized"}
+        "403": {$ref: "#/components/responses/Forbidden"}
+    post:
+      tags: [federation]
+      operationId: registerFederationPeer
+      summary: Register a new federation peer DCM instance
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: {$ref: "#/components/schemas/FederationPeerRegistration"}
+      responses:
+        "200":
+          description: Operation initiated โ€” peer registration pending trust verification
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/Operation"}
+        "422": {$ref: "#/components/responses/UnprocessableEntity"}
+
+  /api/v1/admin/federation/peers/{peer_uuid}:
+    get:
+      tags: [federation]
+      operationId: getFederationPeer
+      summary: Get federation peer details and current trust status
+      parameters:
+        - {name: peer_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Federation peer record
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/FederationPeer"}
+        "404": {$ref: "#/components/responses/NotFound"}
+    delete:
+      tags: [federation]
+      operationId: deregisterFederationPeer
+      summary: Deregister a federation peer (graceful tunnel teardown)
+      parameters:
+        - {name: peer_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Operation initiated
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/Operation"}
+
+  /api/v1/admin/federation/peers/{peer_uuid}:set-trust-posture:
+    post:
+      tags: [federation]
+      operationId: setFederationPeerTrustPosture
+      summary: Set the trust posture for a federation peer
+      parameters:
+        - {name: peer_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [trust_posture, reason]
+              properties:
+                trust_posture: {type: string, enum: [verified, vouched, untrusted]}
+                reason: {type: string, description: Justification for trust posture change}
+      responses:
+        "200":
+          description: Trust posture updated
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/FederationPeer"}
+
+  /api/v1/admin/federation/peers/{peer_uuid}:suspend:
+    post:
+      tags: [federation]
+      operationId: suspendFederationPeer
+      summary: Suspend federation tunnel (stop routing; preserve registration)
+      parameters:
+        - {name: peer_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: {type: string}
+      responses:
+        "200":
+          description: Peer suspended
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/FederationPeer"}
+
+  /api/v1/admin/federation/peers/{peer_uuid}/routed-requests:
+    get:
+      tags: [federation]
+      operationId: listFederationRoutedRequests
+      summary: List requests routed through this federation peer
+      parameters:
+        - {name: peer_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+        - {name: status, in: query, schema: {type: string}}
+        - {name: page_size, in: query, schema: {type: integer, default: 50}}
+        - {name: page_token, in: query, schema: {type: string}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Federated request list
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  items: {type: array, items: {type: object, additionalProperties: true}}
+                  next_page_token: {type: string}
+
+  /api/v1/admin/federation/config:
+    get:
+      tags: [federation]
+      operationId: getFederationConfig
+      summary: Get this DCM instance's federation configuration (identity, capabilities, trust policy)
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Federation configuration
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/FederationConfig"}
+    patch:
+      tags: [federation]
+      operationId: updateFederationConfig
+      summary: Update federation configuration (enable/disable federation, set scope)
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema: {$ref: "#/components/schemas/FederationConfigPatch"}
+      responses:
+        "200":
+          description: Configuration updated
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/FederationConfig"}
+
+components:
+
+  securitySchemes:
+    BearerAuth:
+      type: http
+      scheme: bearer
+      bearerFormat: JWT
+
+  parameters:
+
+    tenant_uuid:
+      name: tenant_uuid
+      in: path
+      required: true
+      schema: { type: string, format: uuid }
+
+    provider_uuid:
+      name: provider_uuid
+      in: path
+      required: true
+      schema: { type: string, format: uuid }
+
+    actor_uuid:
+      name: actor_uuid
+      in: path
+      required: true
+      schema: { type: string, format: uuid }
+
+    profile_name:
+      name: profile_name
+      in: path
+      required: true
+      schema: { type: string, enum: [minimal, dev, standard, prod, fsi, sovereign] }
+
+    limit:
+      name: page_size
+      in: query
+      schema: { type: integer, minimum: 1, maximum: 1000, default: 50 }
+
+    cursor:
+      name: page_token
+      in: query
+      schema: { type: string }
+
+  schemas:
+
+
+
+    MaintenanceWindow:
+      type: object
+      properties:
+        window_uuid: {type: string, format: uuid}
+        handle: {type: string, description: "Stable reference used in schedule.window_id"}
+        display_name: {type: string}
+        description: {type: string}
+        schedule:
+          type: object
+          description: Cron expression or recurrence rule for this window
+          properties:
+            rrule: {type: string, description: "RFC 5545 RRULE e.g. FREQ=WEEKLY;BYDAY=SA;BYHOUR=2"}
+            duration: {type: string, description: "ISO 8601 duration e.g. PT4H"}
+            timezone: {type: string, description: "IANA timezone e.g. UTC, America/New_York"}
+        status: {type: string, enum: [active, suspended, expired]}
+        next_opens_at: {type: string, format: date-time}
+        next_closes_at: {type: string, format: date-time}
+        queued_request_count: {type: integer}
+        created_at: {type: string, format: date-time}
+        owned_by_actor_uuid: {type: string, format: uuid}
+
+    MaintenanceWindowCreate:
+      type: object
+      required: [handle, display_name, schedule]
+      properties:
+        handle: {type: string, pattern: "^[a-z][a-z0-9-]{2,63}$"}
+        display_name: {type: string}
+        description: {type: string}
+        schedule:
+          type: object
+          required: [rrule, duration, timezone]
+          properties:
+            rrule: {type: string}
+            duration: {type: string}
+            timezone: {type: string}
+
+    MaintenanceWindowPatch:
+      type: object
+      properties:
+        display_name: {type: string}
+        description: {type: string}
+        status: {type: string, enum: [active, suspended]}
+        schedule:
+          type: object
+          properties:
+            rrule: {type: string}
+            duration: {type: string}
+            timezone: {type: string}
+
+    FederationPeer:
+      type: object
+      properties:
+        peer_uuid: {type: string, format: uuid}
+        display_name: {type: string}
+        dcm_instance_uuid: {type: string, format: uuid}
+        endpoint: {type: string, format: uri}
+        trust_posture: {type: string, enum: [verified, vouched, untrusted]}
+        status: {type: string, enum: [active, suspended, pending, deregistered]}
+        sovereignty_declarations: {type: array, items: {type: string}}
+        registered_at: {type: string, format: date-time}
+        last_heartbeat_at: {type: string, format: date-time}
+        routed_request_count: {type: integer}
+
+    FederationPeerRegistration:
+      type: object
+      required: [display_name, endpoint, public_key_pem]
+      properties:
+        display_name: {type: string}
+        endpoint: {type: string, format: uri, description: mTLS endpoint of the peer DCM instance}
+        public_key_pem: {type: string, description: Public key for mTLS identity verification}
+        initial_trust_posture: {type: string, enum: [verified, vouched, untrusted], default: untrusted}
+        scope_declaration:
+          type: object
+          properties:
+            resource_types_accessible: {type: array, items: {type: string}}
+            sovereignty_constraints: {type: array, items: {type: string}}
+
+    FederationConfig:
+      type: object
+      properties:
+        federation_enabled: {type: boolean}
+        this_instance_uuid: {type: string, format: uuid}
+        this_instance_display_name: {type: string}
+        this_instance_endpoint: {type: string, format: uri}
+        default_trust_posture: {type: string, enum: [verified, vouched, untrusted]}
+        peer_count: {type: integer}
+        active_peer_count: {type: integer}
+
+    FederationConfigPatch:
+      type: object
+      properties:
+        federation_enabled: {type: boolean}
+        default_trust_posture: {type: string, enum: [verified, vouched, untrusted]}
+        this_instance_display_name: {type: string}
+
+    Operation:
+      type: object
+      description: |
+        AEP-136 Long-Running Operation. Returned by async operations instead of 202 Accepted.
+        Poll GET {operation.name} until done is true.
+        The operation.name is a stable resource path: /api/v1/operations/{uuid}
+      required: [name, done]
+      additionalProperties: false
+      properties:
+        name:
+          type: string
+          description: "Stable resource path for this operation. Poll this URL for status."
+          example: /api/v1/operations/a1b2c3d4-e5f6-7890-abcd-ef1234567890
+        done:
+          type: boolean
+          description: True when the operation has reached a terminal state (success or error)
+          default: false
+        metadata:
+          type: object
+          description: Operation-specific progress metadata
+          additionalProperties: false
+          properties:
+            stage:
+              type: string
+              description: Current pipeline stage
+            progress_pct:
+              type: integer
+              minimum: 0
+              maximum: 100
+            resource_uuid:
+              type: string
+              format: uuid
+              description: UUID of the resource being created/modified (set as soon as assigned)
+            created_at:
+              type: string
+              format: date-time
+            updated_at:
+              type: string
+              format: date-time
+        response:
+          type: object
+          description: Present when done is true and the operation succeeded. Contains the result resource.
+          additionalProperties: true
+        error:
+          type: object
+          description: Present when done is true and the operation failed.
+          additionalProperties: false
+          properties:
+            code:    { type: string }
+            message: { type: string }
+            details: { type: array, items: { type: object } }
+
+
+    Error:
+      type: object
+      required: [error]
+      properties:
+        error:
+          type: object
+          required: [code, message, request_id]
+          properties:
+            code:       { type: string }
+            message:    { type: string }
+            request_id: { type: string, format: uuid }
+            rule_uuid:  { type: string, format: uuid }
+            fields:     { type: array, items: { type: object } }
+
+    AdminHealthResponse:
+      type: object
+      properties:
+        status:        { type: string, enum: [healthy, degraded, unhealthy] }
+        version:       { type: string }
+        uptime:        { type: string }
+        components:
+          type: array
+          items:
+            type: object
+            properties:
+              name:   { type: string }
+              status: { type: string }
+              detail: { type: string }
+        providers:
+          type: object
+          properties:
+            total:    { type: integer }
+            healthy:  { type: integer }
+            degraded: { type: integer }
+            unavailable: { type: integer }
+
+    Tenant:
+      type: object
+      properties:
+        uuid:         { type: string, format: uuid }
+        handle:       { type: string }
+        display_name: { type: string }
+        status:       { type: string }
+        created_at:   { type: string, format: date-time }
+        updated_at:   { type: string, format: date-time }
+        resource_count: { type: integer }
+        compliance_domains: { type: array, items: { type: string } }
+
+    TenantCreate:
+      type: object
+      required: [handle, display_name]
+      properties:
+        handle:       { type: string }
+        display_name: { type: string }
+        compliance_domains: { type: array, items: { type: string } }
+
+    TenantList:
+      type: object
+      properties:
+        items:      { type: array, items: { $ref: "#/components/schemas/Tenant" } }
+        pagination: { type: object }
+
+    TenantQuotas:
+      type: object
+      properties:
+        tenant_uuid: { type: string, format: uuid }
+        quotas:
+          type: array
+          items:
+            type: object
+            properties:
+              resource_type: { type: string }
+              page_size:         { type: integer }
+              used:          { type: integer }
+              remaining:     { type: integer }
+
+    QuotaUpdate:
+      type: object
+      required: [limit]
+      properties:
+        page_size:  { type: integer, minimum: 0 }
+        reason: { type: string }
+
+    ProviderList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            type: object
+            properties:
+              uuid:             { type: string, format: uuid }
+              display_name:     { type: string }
+              provider_type_id: { type: string }
+              status:           { type: string }
+              trust_score:      { type: number }
+              health_status:    { type: string }
+              accreditations:   { type: array, items: { type: object } }
+        pagination: { type: object }
+
+    AccreditationList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            type: object
+            properties:
+              accreditation_uuid: { type: string, format: uuid }
+              provider_uuid:      { type: string, format: uuid }
+              framework:          { type: string }
+              status:             { type: string }
+              valid_until:        { type: string, format: date-time }
+        pagination: { type: object }
+
+    DiscoveryJobStatus:
+      type: object
+      properties:
+        discovery_job_uuid: { type: string, format: uuid }
+        status:     { type: string, enum: [running, completed, failed] }
+        started_at: { type: string, format: date-time }
+        completed_at: { type: string, format: date-time }
+        providers_queried: { type: integer }
+        entities_discovered: { type: integer }
+        drift_records_created: { type: integer }
+        orphans_detected: { type: integer }
+
+    OrphanList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            type: object
+            properties:
+              orphan_candidate_uuid: { type: string, format: uuid }
+              provider_uuid: { type: string, format: uuid }
+              resource_type: { type: string }
+              discovered_at: { type: string, format: date-time }
+              provider_entity_id: { type: string }
+        pagination: { type: object }
+
+    ScoringConfiguration:
+      type: object
+      properties:
+        profile: { type: string }
+        approval_routing:
+          type: array
+          items:
+            type: object
+            properties:
+              tier:      { type: string }
+              max_score: { type: integer }
+        smx008_auto_cap:
+          type: integer
+          description: "Hard cap on auto-approve max_score (โ‰ค 50 always enforced)"
+
+    ScoringConfigurationUpdate:
+      type: object
+      properties:
+        approval_routing:
+          type: array
+          items:
+            type: object
+            required: [tier, max_score]
+            properties:
+              tier:      { type: string }
+              max_score: { type: integer, minimum: 0, maximum: 100 }
+
+    ScoringOverride:
+      type: object
+      required: [policy_uuid, enforcement_class_override]
+      properties:
+        policy_uuid: { type: string, format: uuid }
+        enforcement_class_override: { type: string, enum: [compliance, operational] }
+        reason: { type: string }
+
+    RiskHistory:
+      type: object
+      properties:
+        actor_uuid: { type: string, format: uuid }
+        records:
+          type: array
+          items:
+            type: object
+            properties:
+              score:          { type: integer }
+              routing_tier:   { type: string }
+              evaluated_at:   { type: string, format: date-time }
+              request_uuid:   { type: string, format: uuid }
+              stored_tier_weight: { type: integer }
+
+    ApprovalRecord:
+      type: object
+      properties:
+        approval_uuid:    { type: string, format: uuid }
+        subject_type:     { type: string }
+        subject_uuid:     { type: string, format: uuid }
+        required_tier:    { type: string }
+        required_quorum:  { type: integer }
+        decisions:
+          type: array
+          items:
+            type: object
+            properties:
+              actor_uuid:   { type: string, format: uuid }
+              decision:     { type: string }
+              reason:       { type: string }
+              recorded_at:  { type: string, format: date-time }
+        expires_at:  { type: string, format: date-time }
+        created_at:  { type: string, format: date-time }
+        status:      { type: string, enum: [open, approved, rejected, expired] }
+
+    ApprovalDetail:
+      allOf:
+        - { $ref: "#/components/schemas/ApprovalRecord" }
+        - type: object
+          properties:
+            subject_detail: { type: object }
+            stored_tier_weights:
+              type: object
+              description: "Tier weights recorded at approval creation time (ATM-008)"
+              additionalProperties: { type: integer }
+
+    TierRegistryChangeProposal:
+      type: object
+      required: [proposed_tier_list, reason]
+      properties:
+        proposed_tier_list:
+          type: array
+          minItems: 1
+          items:
+            type: object
+            required: [name, decision_gravity]
+            properties:
+              name:             { type: string, description: "Stable tier name (e.g., auto, reviewed, verified, authorized)" }
+              decision_gravity: { type: string, enum: [none, routine, elevated, critical] }
+              description:      { type: string }
+              dcm_gate:         { type: string }
+              organization_provides: { type: string }
+              dcmgroup_required: { type: boolean }
+              dcmgroup_uuid:    { type: string, format: uuid }
+              quorum_threshold: { type: integer, minimum: 1 }
+        reason: { type: string }
+
+    TierImpactReport:
+      type: object
+      properties:
+        impact_report_uuid: { type: string, format: uuid }
+        change_uuid:        { type: string, format: uuid }
+        computed_at:        { type: string, format: date-time }
+        blocking_items:     { type: integer }
+        items:
+          type: array
+          items:
+            type: object
+            properties:
+              item_uuid:       { type: string, format: uuid }
+              classification:
+                type: string
+                enum: [SECURITY_DEGRADATION, SECURITY_UPGRADE, BROKEN_REFERENCE, PROFILE_GAP, STALE_WEIGHT]
+              tier_name:       { type: string }
+              old_position:    { type: integer }
+              new_position:    { type: integer }
+              old_gravity:     { type: string }
+              new_gravity:     { type: string }
+              affected_items:
+                type: array
+                items:
+                  type: object
+                  properties:
+                    item_type:   { type: string }
+                    item_uuid:   { type: string, format: uuid }
+                    description: { type: string }
+              accepted:        { type: boolean }
+              accepted_by:     { type: string, format: uuid }
+              accepted_reason: { type: string }
diff --git a/content/docs/architecture/specifications/dcm-consumer-api.yaml b/content/docs/architecture/specifications/dcm-consumer-api.yaml
new file mode 100644
index 0000000..44e85d4
--- /dev/null
+++ b/content/docs/architecture/specifications/dcm-consumer-api.yaml
@@ -0,0 +1,2288 @@
+openapi: 3.1.0
+info:
+  title: DCM Consumer API
+  version: 1.0.0
+  description: 'The DCM Consumer API provides the interface used by application teams,
+    Tenant owners,
+
+    and automated tooling to interact with the DCM control plane. All interactions
+    are
+
+    authenticated, Tenant-scoped, and governed by the Policy Engine.
+
+
+    **Key principles:**
+
+    - All requests require a valid session token (Bearer) obtained via `/api/v1/auth/token`
+
+    - All responses are Tenant-scoped โ€” actors only see entities they are authorized
+    to see
+
+    - Policy denials return 403 with a `rule_uuid` identifying the governing rule
+
+    - All mutating operations produce an audit record
+
+    - List endpoints support cursor-based pagination via `page_size` and `page_token`
+    query parameters
+
+
+    **API Versioning:** DCM uses URL path versioning (`/api/v1/`). Version discovery
+    is available
+
+    at `/.well-known/dcm-api-versions`. See `34-api-versioning-strategy.md`.
+
+
+
+
+    **AEP Alignment:** This API follows [AEP](https://aep.dev) conventions:
+
+    custom methods use colon syntax (`POST /resources/{name}:suspend`),
+
+    async operations return an `Operation` resource (AEP-136 LRO),
+
+    and list pagination uses `page_size`/`page_token` parameters.
+
+    '
+  contact:
+    name: DCM Project
+    url: https://github.com/dcm-project
+  license:
+    name: Apache 2.0
+    url: https://www.apache.org/licenses/LICENSE-2.0
+servers:
+- url: https://{dcm-host}/
+  description: DCM Control Plane
+  variables:
+    dcm-host:
+      description: Hostname of the DCM control plane deployment
+      default: dcm.example.com
+security:
+- BearerAuth: []
+tags:
+- name: discovery
+  description: API version discovery and migration guides
+- name: authentication
+  description: Session management, token issuance, and introspection
+- name: catalog
+  description: Service catalog browsing and search
+- name: requests
+  description: Service request submission and lifecycle
+- name: resources
+  description: Resource entity management and lifecycle operations
+- name: drift
+  description: Drift detection, acknowledgement, and remediation
+- name: groups
+  description: Resource group management
+- name: approvals
+  description: Approval workflow for pending decisions
+- name: cost
+  description: Cost estimation and attribution
+- name: notifications
+  description: Notification inbox management
+- name: webhooks
+  description: Outbound webhook subscription management
+- name: search
+  description: Cross-entity search
+- name: audit
+  description: Audit trail access
+- name: contributions
+  description: Federated contribution submission (policies, resource groups)
+- name: credentials
+  description: Credential retrieval and rotation
+paths:
+  /.well-known/dcm-api-versions:
+    get:
+      tags:
+      - discovery
+      operationId: getApiVersions
+      summary: List supported API versions
+      security: []
+      responses:
+        '200':
+          description: Supported API versions and deprecation status
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  versions:
+                    type: array
+                    items:
+                      type: object
+                      properties:
+                        version:
+                          type: string
+                        status:
+                          type: string
+                          enum:
+                          - current
+                          - supported
+                          - deprecated
+                          - sunset
+                        sunset_date:
+                          type: string
+                          format: date
+  /api/v1/migration-guide:
+    get:
+      tags:
+      - discovery
+      operationId: getMigrationGuide
+      summary: Get migration guide for API version transitions
+      security: []
+      parameters:
+      - name: from
+        in: query
+        required: true
+        schema:
+          type: string
+          example: '1'
+      - name: to
+        in: query
+        required: true
+        schema:
+          type: string
+          example: '2'
+      responses:
+        '200':
+          description: Migration guide document
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  from_version:
+                    type: string
+                  to_version:
+                    type: string
+                  breaking_changes:
+                    type: array
+                    items:
+                      type: object
+                  guidance:
+                    type: string
+  /api/v1/auth/token:
+    post:
+      tags:
+      - authentication
+      operationId: createSession
+      summary: Authenticate and obtain a session token
+      security: []
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - grant_type
+              properties:
+                grant_type:
+                  type: string
+                  enum:
+                  - password
+                  - client_credentials
+                  - oidc_code
+                username:
+                  type: string
+                password:
+                  type: string
+                  format: password
+                client_id:
+                  type: string
+                client_secret:
+                  type: string
+                  format: password
+                code:
+                  type: string
+                redirect_uri:
+                  type: string
+                  format: uri
+      responses:
+        '200':
+          description: Session token issued
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/SessionTokenResponse'
+        '401':
+          $ref: '#/components/responses/Unauthorized'
+        '403':
+          $ref: '#/components/responses/Forbidden'
+  /api/v1/auth/sessions:
+    get:
+      tags:
+      - authentication
+      operationId: listSessions
+      summary: List active sessions for the authenticated actor
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  sessions:
+                    type: array
+                    items:
+                      $ref: '#/components/schemas/SessionSummary'
+      description: List active sessions
+    delete:
+      tags:
+      - authentication
+      operationId: revokeAllSessions
+      summary: Revoke all sessions for the authenticated actor (except the current
+        session)
+      responses:
+        '204':
+          description: All other sessions revoked
+  /api/v1/auth/sessions/{session_uuid}:
+    delete:
+      tags:
+      - authentication
+      operationId: revokeSession
+      summary: Revoke a specific session
+      parameters:
+      - $ref: '#/components/parameters/session_uuid'
+      responses:
+        '204':
+          description: Session revoked
+        '404':
+          $ref: '#/components/responses/NotFound'
+  /api/v1/auth:introspect:
+    post:
+      tags:
+      - authentication
+      operationId: introspectToken
+      summary: Introspect a token and return actor identity and roles
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - token
+              properties:
+                token:
+                  type: string
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/TokenIntrospection'
+        '401':
+          $ref: '#/components/responses/Unauthorized'
+  /api/v1/auth/session:
+    delete:
+      tags:
+      - auth
+      operationId: logoutCurrentSession
+      summary: Logout current session
+      description: Terminates the calling actor's current session and invalidates
+        the bearer token.
+      security:
+      - bearerAuth: []
+      responses:
+        '204':
+          description: Session terminated
+        '401':
+          $ref: '#/components/responses/Unauthorized'
+  /api/v1/catalog:
+    get:
+      tags:
+      - catalog
+      operationId: listCatalogItems
+      summary: List catalog items available to the authenticated actor (RBAC-filtered)
+      parameters:
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      - name: category
+        in: query
+        schema:
+          type: string
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/CatalogItemList'
+  /api/v1/catalog/{catalog_item_uuid}:
+    get:
+      tags:
+      - catalog
+      operationId: getCatalogItem
+      summary: Get full schema and details for a catalog item including field constraints
+      parameters:
+      - $ref: '#/components/parameters/catalog_item_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/CatalogItem'
+        '404':
+          $ref: '#/components/responses/NotFound'
+  /api/v1/catalog/search:
+    get:
+      tags:
+      - catalog
+      operationId: searchCatalog
+      summary: Search catalog by keyword, resource type, or tag
+      parameters:
+      - name: q
+        in: query
+        required: true
+        schema:
+          type: string
+          minLength: 1
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/CatalogItemList'
+  /api/v1/requests:
+    post:
+      tags:
+      - requests
+      operationId: submitRequest
+      summary: Submit a service request
+      description: 'Submits a resource request. The request is stored as an Intent
+        State artifact, processed
+
+        through layer assembly and policy evaluation, placed with a provider, and
+        dispatched.
+
+        Returns immediately with a request_uuid for status polling or SSE streaming.
+
+        '
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/ServiceRequest'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '403':
+          $ref: '#/components/responses/Forbidden'
+        '422':
+          $ref: '#/components/responses/UnprocessableEntity'
+    get:
+      tags:
+      - requests
+      operationId: listRequests
+      summary: List requests submitted by the authenticated actor
+      parameters:
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      - name: status
+        in: query
+        schema:
+          type: string
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RequestList'
+  /api/v1/requests/{request_uuid}/status:
+    get:
+      tags:
+      - requests
+      operationId: getRequestStatus
+      summary: Poll request status and pipeline stage
+      parameters:
+      - $ref: '#/components/parameters/request_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RequestStatus'
+        '404':
+          $ref: '#/components/responses/NotFound'
+  /api/v1/requests/{request_uuid}/stream:
+    get:
+      tags:
+      - requests
+      operationId: streamRequestStatus
+      summary: Stream real-time request status updates (Server-Sent Events)
+      parameters:
+      - $ref: '#/components/parameters/request_uuid'
+      responses:
+        '200':
+          description: SSE stream of RequestStatus events
+          content:
+            text/event-stream:
+              schema:
+                type: string
+  /api/v1/requests/{request_uuid}:
+    delete:
+      tags:
+      - requests
+      operationId: cancelRequest
+      summary: Cancel a pending request (before provider dispatch)
+      parameters:
+      - $ref: '#/components/parameters/request_uuid'
+      responses:
+        '204':
+          description: Request cancelled
+        '409':
+          description: Request already dispatched โ€” cannot cancel
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Error'
+  /api/v1/request-groups:
+    post:
+      tags:
+      - requests
+      operationId: createRequestGroup
+      summary: Submit a group of related requests (bulk or dependency-ordered)
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/RequestGroup'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+  /api/v1/request-groups/{group_uuid}:
+    get:
+      tags:
+      - requests
+      operationId: getRequestGroup
+      summary: Get status of all requests in a group
+      parameters:
+      - $ref: '#/components/parameters/group_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/RequestGroupStatus'
+  /api/v1/resources:
+    get:
+      tags:
+      - resources
+      operationId: listResources
+      summary: List resources owned by the authenticated actor's Tenant
+      parameters:
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      - name: resource_type
+        in: query
+        schema:
+          type: string
+      - name: lifecycle_state
+        in: query
+        schema:
+          type: string
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/ResourceList'
+  /api/v1/resources/{entity_uuid}:
+    get:
+      tags:
+      - resources
+      operationId: getResource
+      summary: Get full entity detail for a resource
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/ResourceEntity'
+        '404':
+          $ref: '#/components/responses/NotFound'
+    patch:
+      tags:
+      - resources
+      operationId: updateResource
+      summary: Update editable fields on a realized resource (delta only)
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/ResourceUpdate'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+        '400':
+          $ref: '#/components/responses/BadRequest'
+        '403':
+          $ref: '#/components/responses/Forbidden'
+    delete:
+      tags:
+      - resources
+      operationId: decommissionResource
+      summary: Initiate resource decommission
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+        '409':
+          description: Resource has active dependents โ€” cannot decommission
+  /api/v1/resources/{entity_uuid}:suspend:
+    post:
+      tags:
+      - resources
+      operationId: suspendResource
+      summary: Suspend an OPERATIONAL resource
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+        '409':
+          description: Resource not in OPERATIONAL state
+  /api/v1/resources/{entity_uuid}:resume:
+    post:
+      tags:
+      - resources
+      operationId: resumeResource
+      summary: Resume a SUSPENDED resource
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+        '409':
+          description: Resource not in SUSPENDED state
+  /api/v1/resources/{entity_uuid}:rehydrate:
+    post:
+      tags:
+      - resources
+      operationId: rehydrateResource
+      summary: Rehydrate a resource to a new provider or context
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/RehydrateRequest'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+        '403':
+          $ref: '#/components/responses/Forbidden'
+  /api/v1/resources/{entity_uuid}:extend-ttl:
+    post:
+      tags:
+      - resources
+      operationId: extendTtl
+      summary: Extend or modify the TTL of a resource
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - new_ttl
+              properties:
+                new_ttl:
+                  type: string
+                  description: ISO 8601 duration or datetime
+      responses:
+        '200':
+          description: TTL updated
+  /api/v1/resources/expiring:
+    get:
+      tags:
+      - resources
+      operationId: listExpiringResources
+      summary: List resources expiring within a time window
+      parameters:
+      - name: within
+        in: query
+        schema:
+          type: string
+          description: ISO 8601 duration (e.g., P30D)
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/ResourceList'
+  /api/v1/resources/{entity_uuid}:transfer:
+    post:
+      tags:
+      - resources
+      operationId: initiateOwnershipTransfer
+      summary: Initiate ownership transfer to another Tenant
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - target_tenant_uuid
+              - reason
+              properties:
+                target_tenant_uuid:
+                  type: string
+                  format: uuid
+                reason:
+                  type: string
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+  /api/v1/resources/transfers/{transfer_uuid}:accept:
+    post:
+      tags:
+      - resources
+      operationId: acceptOwnershipTransfer
+      summary: Accept an incoming ownership transfer
+      parameters:
+      - name: transfer_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Transfer accepted; ownership updated
+  /api/v1/resources/transfers/{transfer_uuid}:reject:
+    post:
+      tags:
+      - resources
+      operationId: rejectOwnershipTransfer
+      summary: Reject an incoming ownership transfer
+      parameters:
+      - name: transfer_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Transfer rejected
+  /api/v1/resources:bulk-decommission:
+    post:
+      tags:
+      - resources
+      operationId: bulkDecommission
+      summary: Decommission multiple resources in dependency-safe order
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - entity_uuids
+              properties:
+                entity_uuids:
+                  type: array
+                  items:
+                    type: string
+                    format: uuid
+                  minItems: 1
+                dry_run:
+                  type: boolean
+                  default: false
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+  /api/v1/resources/{entity_uuid}/relationships:
+    get:
+      tags:
+      - resources
+      operationId: getResourceRelationships
+      summary: Get all relationships for a resource entity
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  relationships:
+                    type: array
+                    items:
+                      type: object
+  /api/v1/resources/{entity_uuid}/provider-notifications:
+    get:
+      tags:
+      - resources
+      operationId: getProviderNotifications
+      summary: List pending provider-initiated state change notifications requiring
+        approval
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  notifications:
+                    type: array
+                    items:
+                      type: object
+  /api/v1/resources/{entity_uuid}/provider-notifications/{notification_uuid}:approve:
+    post:
+      tags:
+      - resources
+      operationId: approveProviderNotification
+      summary: Approve a provider-initiated state change
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      - name: notification_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Provider notification approved
+  /api/v1/resources/{entity_uuid}/recovery-decisions:
+    get:
+      tags:
+      - resources
+      operationId: getRecoveryDecisions
+      summary: List pending recovery decisions for a resource (for notify_and_wait
+        policies)
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  decisions:
+                    type: array
+                    items:
+                      type: object
+  /api/v1/resources/{entity_uuid}/recovery-decisions/{recovery_decision_uuid}:
+    post:
+      tags:
+      - resources
+      operationId: resolveRecoveryDecision
+      summary: Resolve a pending recovery decision
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      - name: recovery_decision_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - decision
+              properties:
+                decision:
+                  type: string
+                  enum:
+                  - approve
+                  - reject
+                  - escalate
+                reason:
+                  type: string
+      responses:
+        '200':
+          description: Decision recorded
+  /api/v1/resources/{entity_uuid}/audit:
+    get:
+      tags:
+      - audit
+      operationId: getResourceAudit
+      summary: Get audit trail for a resource entity
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/AuditList'
+  /api/v1/resources/{entity_uuid}/drift:
+    get:
+      tags:
+      - drift
+      operationId: getResourceDrift
+      summary: Get current drift records for a resource
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/DriftRecordList'
+  /api/v1/resources/{entity_uuid}/drift/{drift_uuid}:acknowledge:
+    post:
+      tags:
+      - drift
+      operationId: acknowledgeDrift
+      summary: Acknowledge a drift record (suppress notification without resolving)
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      - name: drift_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Drift acknowledged
+  /api/v1/resources/{entity_uuid}/drift/{drift_uuid}:accept:
+    post:
+      tags:
+      - drift
+      operationId: acceptDrift
+      summary: Accept the drifted state as the new intended state (update Requested
+        State)
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      - name: drift_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Drifted state accepted; Requested State updated
+  /api/v1/resources/{entity_uuid}/drift/{drift_uuid}:revert:
+    post:
+      tags:
+      - drift
+      operationId: revertDrift
+      summary: Revert the resource to its Requested State (dispatch remediation)
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      - name: drift_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+  /api/v1/groups:
+    get:
+      tags:
+      - groups
+      operationId: listGroups
+      summary: List groups the authenticated actor's Tenant has access to
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  groups:
+                    type: array
+                    items:
+                      type: object
+  /api/v1/groups/{group_uuid}:
+    get:
+      tags:
+      - groups
+      operationId: getGroup
+      summary: Get group detail and member list
+      parameters:
+      - name: group_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+  /api/v1/groups/{group_uuid}/members:
+    post:
+      tags:
+      - groups
+      operationId: addGroupMember
+      summary: Add an entity to a group
+      parameters:
+      - name: group_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - entity_uuid
+              properties:
+                entity_uuid:
+                  type: string
+                  format: uuid
+      responses:
+        '201':
+          description: Entity added to group
+  /api/v1/groups/{group_uuid}/members/{entity_uuid}:
+    delete:
+      tags:
+      - groups
+      operationId: removeGroupMember
+      summary: Remove an entity from a group
+      parameters:
+      - name: group_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '204':
+          description: Entity removed from group
+  /api/v1/approvals/pending:
+    get:
+      tags:
+      - approvals
+      operationId: listPendingApprovals
+      summary: List approval decisions pending the authenticated actor's action
+      parameters:
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  approvals:
+                    type: array
+                    items:
+                      $ref: '#/components/schemas/ApprovalRecord'
+  /api/v1/approvals/{approval_uuid}:
+    post:
+      tags:
+      - approvals
+      operationId: recordApprovalDecision
+      summary: Record an approval decision (approve/reject/abstain)
+      description: 'Designed to be called by both humans via the UI and by external
+        systems (ITSM, Slack bots,
+
+        CI/CD pipelines) that have been authorized to record decisions on behalf of
+        the organization.
+
+        DCM enforces the gate and records the audit trail; the deliberation process
+        is the
+
+        organization''s responsibility.
+
+        '
+      parameters:
+      - name: approval_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required:
+              - decision
+              - reason
+              properties:
+                decision:
+                  type: string
+                  enum:
+                  - approve
+                  - reject
+                  - abstain
+                reason:
+                  type: string
+                  minLength: 1
+                external_reference:
+                  type: string
+                  description: ITSM ticket ID, Jira issue key, etc.
+      responses:
+        '200':
+          description: Decision recorded; approval gate re-evaluated
+        '403':
+          $ref: '#/components/responses/Forbidden'
+        '409':
+          description: Approval already in terminal state
+  /api/v1/cost/estimate:
+    post:
+      tags:
+      - cost
+      operationId: estimateCost
+      summary: Get cost estimate before submitting a request
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/ServiceRequest'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/CostEstimate'
+  /api/v1/resources/{entity_uuid}/cost:
+    get:
+      tags:
+      - cost
+      operationId: getResourceCost
+      summary: Get cost actuals and attribution for a resource
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/CostActuals'
+  /api/v1/operations/{operation_uuid}:
+    get:
+      tags:
+      - operations
+      operationId: getOperation
+      summary: Poll a Long-Running Operation for status (AEP-136)
+      description: 'Returns the current state of an async operation. Poll until `done`
+        is `true`.
+
+
+        **Note:** `operation_uuid == request_uuid`. All async operations initiated
+        via
+
+        POST /api/v1/requests, PATCH /api/v1/resources/{uuid}, etc., return an Operation
+
+        whose UUID is the same as the request UUID.
+
+
+        Two polling views are available:
+
+        - This endpoint: AEP-standard (done, metadata, response/error)
+
+        - GET /api/v1/requests/{uuid}/status: DCM-native rich view (pipeline_stage,
+        full history)
+
+
+        Both reflect the same underlying operation state.
+
+        '
+      parameters:
+      - name: operation_uuid
+        in: path
+        required: true
+        description: The operation UUID (same as the request UUID)
+        schema:
+          type: string
+          format: uuid
+      security:
+      - bearerAuth: []
+      responses:
+        '200':
+          description: Operation status (check `done` field for completion)
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+        '404':
+          $ref: '#/components/responses/NotFound'
+        '401':
+          $ref: '#/components/responses/Unauthorized'
+        '403':
+          $ref: '#/components/responses/Forbidden'
+  /api/v1/quota:
+    get:
+      tags:
+      - cost
+      operationId: getQuota
+      summary: Get quota status for the authenticated actor's Tenant
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/QuotaStatus'
+  /api/v1/notifications:
+    get:
+      tags:
+      - notifications
+      operationId: listNotifications
+      summary: List notifications in the authenticated actor's inbox
+      parameters:
+      - name: unread_only
+        in: query
+        schema:
+          type: boolean
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  notifications:
+                    type: array
+                    items:
+                      type: object
+  /api/v1/notifications/{notification_uuid}/read:
+    post:
+      tags:
+      - notifications
+      operationId: markNotificationRead
+      summary: Mark a notification as read
+      parameters:
+      - name: notification_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Marked as read
+  /api/v1/notifications:read-all:
+    post:
+      tags:
+      - notifications
+      operationId: markAllNotificationsRead
+      summary: Mark all notifications as read
+      responses:
+        '204':
+          description: All notifications marked as read
+  /api/v1/webhooks:
+    get:
+      tags:
+      - webhooks
+      operationId: listWebhooks
+      summary: List outbound webhook subscriptions for the authenticated actor
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  webhooks:
+                    type: array
+                    items:
+                      $ref: '#/components/schemas/WebhookSubscription'
+    post:
+      tags:
+      - webhooks
+      operationId: createWebhook
+      summary: Create an outbound webhook subscription
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/WebhookSubscription'
+      responses:
+        '201':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/WebhookSubscription'
+  /api/v1/webhooks/{webhook_uuid}:
+    delete:
+      tags:
+      - webhooks
+      operationId: deleteWebhook
+      summary: Delete a webhook subscription
+      parameters:
+      - name: webhook_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '204':
+          description: Webhook deleted
+  # โ”€โ”€ Workload Analysis โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+  /api/v1/resources/{entity_uuid}/workload-profile:
+    get:
+      tags: [workload-analysis]
+      operationId: getWorkloadProfile
+      summary: Get the Workload Analysis profile for a resource
+      description: |
+        Returns the most recent WorkloadProfile entity for the given resource.
+        WorkloadProfiles are created automatically during brownfield ingestion
+        and can be refreshed on demand via the :analyze custom method.
+      parameters:
+        - {name: entity_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      responses:
+        "200":
+          description: Workload profile
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/WorkloadProfile"}
+        "404": {$ref: "#/components/responses/NotFound"}
+        "403": {$ref: "#/components/responses/Forbidden"}
+
+  /api/v1/resources/{entity_uuid}/workload-profile:analyze:
+    post:
+      tags: [workload-analysis]
+      operationId: analyzeWorkload
+      summary: Trigger re-analysis of a resource's workload profile
+      description: |
+        Initiates a new Workload Analysis pass for the resource. The existing
+        WorkloadProfile is superseded when the new analysis completes. Useful
+        when a resource's role has changed since initial classification.
+      parameters:
+        - {name: entity_uuid, in: path, required: true, schema: {type: string, format: uuid}}
+      security: [{bearerAuth: []}]
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+              required: [reason]
+              properties:
+                reason: {type: string, description: Why re-analysis is needed}
+                include_mta: {type: boolean, default: true, description: Include MTA containerization assessment}
+      responses:
+        "200":
+          description: Operation initiated โ€” poll for completion
+          content:
+            application/json:
+              schema: {$ref: "#/components/schemas/Operation"}
+        "404": {$ref: "#/components/responses/NotFound"}
+
+  /api/v1/search:
+    get:
+      tags:
+      - search
+      operationId: search
+      summary: Cross-entity search (resources, groups, catalog items)
+      parameters:
+      - name: q
+        in: query
+        required: true
+        schema:
+          type: string
+          minLength: 1
+      - name: types
+        in: query
+        schema:
+          type: array
+          items:
+            type: string
+        style: form
+        explode: false
+      - $ref: '#/components/parameters/page_size'
+      - $ref: '#/components/parameters/page_token'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/SearchResults'
+  /api/v1/audit/correlation/{correlation_id}:
+    get:
+      tags:
+      - audit
+      operationId: getAuditByCorrelation
+      summary: Get all audit records for a correlation ID (cross-resource trace)
+      parameters:
+      - name: correlation_id
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/AuditList'
+  /api/v1/contribute/policy:
+    post:
+      tags:
+      - contributions
+      operationId: contributePolicy
+      summary: Submit a policy contribution via the federated contribution pipeline
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/PolicyContribution'
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+  /api/v1/contribute/resource-group:
+    post:
+      tags:
+      - contributions
+      operationId: contributeResourceGroup
+      summary: Submit a resource group contribution
+      requestBody:
+        required: true
+        content:
+          application/json:
+            schema:
+              type: object
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+  /api/v1/contribute:
+    get:
+      tags:
+      - contributions
+      operationId: listContributions
+      summary: List contributions submitted by the authenticated actor
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  contributions:
+                    type: array
+                    items:
+                      type: object
+  /api/v1/contribute/{contribution_uuid}:
+    delete:
+      tags:
+      - contributions
+      operationId: withdrawContribution
+      summary: Withdraw a pending contribution
+      parameters:
+      - name: contribution_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '204':
+          description: Contribution withdrawn
+        '409':
+          description: Contribution already activated โ€” cannot withdraw
+  /api/v1/resources/{entity_uuid}/credentials:
+    get:
+      tags:
+      - credentials
+      operationId: listResourceCredentials
+      summary: List credentials associated with a resource (metadata only โ€” no values)
+      parameters:
+      - $ref: '#/components/parameters/entity_uuid'
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                type: object
+                properties:
+                  credentials:
+                    type: array
+                    items:
+                      $ref: '#/components/schemas/CredentialSummary'
+  /api/v1/credentials/{credential_uuid}/value:
+    get:
+      tags:
+      - credentials
+      operationId: getCredentialValue
+      summary: Retrieve the current credential value (audited; step-up auth may be
+        required)
+      parameters:
+      - name: credential_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/CredentialValue'
+        '403':
+          $ref: '#/components/responses/Forbidden'
+  /api/v1/credentials/{credential_uuid}:rotate:
+    post:
+      tags:
+      - credentials
+      operationId: rotateCredential
+      summary: Initiate credential rotation
+      parameters:
+      - name: credential_uuid
+        in: path
+        required: true
+        schema:
+          type: string
+          format: uuid
+      responses:
+        '200':
+          description: Operation initiated. Poll `operation.name` for completion.
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/Operation'
+components:
+  securitySchemes:
+    BearerAuth:
+      type: http
+      scheme: bearer
+      bearerFormat: JWT
+      description: Session token obtained from POST /api/v1/auth/token
+  parameters:
+    entity_uuid:
+      name: entity_uuid
+      in: path
+      required: true
+      schema:
+        type: string
+        format: uuid
+      description: UUID of the resource entity
+    request_uuid:
+      name: request_uuid
+      in: path
+      required: true
+      schema:
+        type: string
+        format: uuid
+    catalog_item_uuid:
+      name: catalog_item_uuid
+      in: path
+      required: true
+      schema:
+        type: string
+        format: uuid
+    group_uuid:
+      name: group_uuid
+      in: path
+      required: true
+      schema:
+        type: string
+        format: uuid
+    session_uuid:
+      name: session_uuid
+      in: path
+      required: true
+      schema:
+        type: string
+        format: uuid
+    limit:
+      name: page_size
+      in: query
+      schema:
+        type: integer
+        minimum: 1
+        maximum: 1000
+        default: 50
+    cursor:
+      name: page_token
+      in: query
+      schema:
+        type: string
+      description: Opaque page token from previous response (use next_page_token from
+        response)
+  responses:
+    BadRequest:
+      description: Invalid request syntax
+      content:
+        application/json:
+          schema:
+            $ref: '#/components/schemas/Error'
+    Unauthorized:
+      description: Missing or invalid authentication token
+      content:
+        application/json:
+          schema:
+            $ref: '#/components/schemas/Error'
+    Forbidden:
+      description: Authenticated but not authorized. rule_uuid identifies the governing
+        policy.
+      content:
+        application/json:
+          schema:
+            $ref: '#/components/schemas/Error'
+    NotFound:
+      description: Resource not found or not visible to authenticated actor
+      content:
+        application/json:
+          schema:
+            $ref: '#/components/schemas/Error'
+    UnprocessableEntity:
+      description: Request syntax valid but semantically invalid (e.g., field validation
+        failure)
+      content:
+        application/json:
+          schema:
+            $ref: '#/components/schemas/Error'
+  schemas:
+    Operation:
+      type: object
+      description: "AEP-136 Long-Running Operation. Returned by async operations instead\
+        \ of 202 Accepted.\nPoll GET {operation.name} until done is true.\nThe operation.name\
+        \ is a stable resource path: /api/v1/operations/{uuid}\nNote: operation_uuid\
+        \ == request_uuid. Two polling endpoints are available:\n  - GET /api/v1/operations/{uuid}\
+        \ โ€” AEP-standard thin view (done, metadata, response/error)\n  - GET /api/v1/requests/{uuid}/status\
+        \ โ€” DCM-native rich view (pipeline_stage, full status history)\nBoth endpoints\
+        \ reflect the same underlying operation state.\n"
+      required:
+      - name
+      - done
+      additionalProperties: false
+      properties:
+        name:
+          type: string
+          description: Stable resource path for this operation. Poll this URL for
+            status.
+          example: /api/v1/operations/a1b2c3d4-e5f6-7890-abcd-ef1234567890
+        done:
+          type: boolean
+          description: True when the operation has reached a terminal state (success
+            or error)
+          default: false
+        metadata:
+          type: object
+          description: Operation-specific progress metadata
+          additionalProperties: false
+          properties:
+            stage:
+              type: string
+              description: Current pipeline stage
+            progress_pct:
+              type: integer
+              minimum: 0
+              maximum: 100
+            resource_uuid:
+              type: string
+              format: uuid
+              description: UUID of the resource being created/modified (set as soon
+                as assigned)
+            request_uuid:
+              type: string
+              format: uuid
+              description: The DCM request UUID. operation_uuid == request_uuid. Use
+                GET /api/v1/requests/{request_uuid}/status for the full DCM-native
+                pipeline view.
+            created_at:
+              type: string
+              format: date-time
+            updated_at:
+              type: string
+              format: date-time
+        response:
+          type: object
+          description: Present when done is true and the operation succeeded. Contains
+            the result resource.
+          additionalProperties: true
+        error:
+          type: object
+          description: Present when done is true and the operation failed.
+          additionalProperties: false
+          properties:
+            code:
+              type: string
+            message:
+              type: string
+            details:
+              type: array
+              items:
+                type: object
+    Error:
+      type: object
+      required:
+      - error
+      properties:
+        error:
+          type: object
+          required:
+          - code
+          - message
+          - request_id
+          properties:
+            code:
+              type: string
+            message:
+              type: string
+            request_id:
+              type: string
+              format: uuid
+            rule_uuid:
+              type: string
+              format: uuid
+              description: Present on POLICY_DENIED and GOVERNANCE_DENIED
+            fields:
+              type: array
+              items:
+                type: object
+                properties:
+                  field:
+                    type: string
+                  issue:
+                    type: string
+    SessionTokenResponse:
+      type: object
+      required:
+      - access_token
+      - token_type
+      - expires_in
+      - session_uuid
+      properties:
+        access_token:
+          type: string
+        token_type:
+          type: string
+          const: bearer
+        expires_in:
+          type: integer
+          description: Seconds until expiry
+        session_uuid:
+          type: string
+          format: uuid
+        refresh_token:
+          type: string
+    SessionSummary:
+      type: object
+      properties:
+        session_uuid:
+          type: string
+          format: uuid
+        created_at:
+          type: string
+          format: date-time
+        expires_at:
+          type: string
+          format: date-time
+        ip_address:
+          type: string
+        user_agent:
+          type: string
+        current:
+          type: boolean
+    TokenIntrospection:
+      type: object
+      properties:
+        active:
+          type: boolean
+        actor_uuid:
+          type: string
+          format: uuid
+        actor_type:
+          type: string
+        display_name:
+          type: string
+        tenant_uuids:
+          type: array
+          items:
+            type: string
+            format: uuid
+        roles:
+          type: array
+          items:
+            type: string
+        groups:
+          type: array
+          items:
+            type: string
+        expires_at:
+          type: string
+          format: date-time
+    CatalogItem:
+      type: object
+      properties:
+        uuid:
+          type: string
+          format: uuid
+        resource_type:
+          description: Resource type โ€” FQN string (e.g., 'Compute.VirtualMachine')
+            or Registry UUID. DCM resolves either form internally.
+          oneOf:
+          - type: string
+            pattern: ^[A-Z][a-zA-Z0-9]+\.[A-Z][a-zA-Z0-9]+$
+            description: FQN form (recommended)
+          - type: string
+            format: uuid
+            description: UUID form
+        display_name:
+          type: string
+        description:
+          type: string
+        version:
+          type: string
+        status:
+          type: string
+        provider_uuid:
+          type: string
+          format: uuid
+        field_schema:
+          type: object
+          description: 'JSON Schema for the request body fields. Each field includes
+            a constraint block with type: range|enum|pattern|layer_reference|layer_reference_list.
+            For layer_reference constraints, allowed_values is resolved at render
+            time from active Reference Data Layer instances of the declared layer_type.'
+        cost_estimate:
+          type: object
+        dependencies:
+          type: array
+          items:
+            type: object
+    CatalogItemList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            $ref: '#/components/schemas/CatalogItem'
+        pagination:
+          type: object
+    ServiceRequest:
+      type: object
+      required:
+      - catalog_item_uuid
+      - fields
+      properties:
+        catalog_item_uuid:
+          type: string
+          format: uuid
+        fields:
+          type: object
+          description: Resource-type-specific field values
+          additionalProperties: true
+        group_uuid:
+          type: string
+          format: uuid
+          description: Assign to a resource group on creation
+        scheduled_at:
+          type: string
+          format: date-time
+          description: Defer request execution to this time
+        depends_on:
+          type: array
+          items:
+            type: string
+            format: uuid
+          description: Request UUIDs that must complete first
+        dry_run:
+          type: boolean
+          default: false
+          description: Evaluate policy and placement without submitting
+    RequestAccepted:
+      type: object
+      properties:
+        request_uuid:
+          type: string
+          format: uuid
+        status:
+          type: string
+        stream_url:
+          type: string
+          format: uri
+    RequestStatus:
+      type: object
+      properties:
+        request_uuid:
+          type: string
+          format: uuid
+        status:
+          type: string
+        pipeline_stage:
+          type: string
+        entity_uuid:
+          type: string
+          format: uuid
+          description: Set once realization begins
+        created_at:
+          type: string
+          format: date-time
+        updated_at:
+          type: string
+          format: date-time
+        error:
+          $ref: '#/components/schemas/Error'
+    RequestList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            $ref: '#/components/schemas/RequestStatus'
+        pagination:
+          type: object
+    RequestGroup:
+      type: object
+      required:
+      - requests
+      properties:
+        requests:
+          type: array
+          items:
+            $ref: '#/components/schemas/ServiceRequest'
+          minItems: 1
+        ordered:
+          type: boolean
+          default: false
+          description: Execute in declared order with dependency waiting
+        group_name:
+          type: string
+    RequestGroupAccepted:
+      type: object
+      properties:
+        group_uuid:
+          type: string
+          format: uuid
+        request_uuids:
+          type: array
+          items:
+            type: string
+            format: uuid
+    RequestGroupStatus:
+      type: object
+      properties:
+        group_uuid:
+          type: string
+          format: uuid
+        status:
+          type: string
+        requests:
+          type: array
+          items:
+            $ref: '#/components/schemas/RequestStatus'
+    ResourceEntity:
+      type: object
+      description: Resource entity as returned by the Consumer API (may be field-filtered
+        by governance matrix)
+      properties:
+        uuid:
+          type: string
+          format: uuid
+        entity_type:
+          type: string
+        resource_type:
+          type: string
+        lifecycle_state:
+          type: string
+        owned_by_tenant_uuid:
+          type: string
+          format: uuid
+        provider_uuid:
+          type: string
+          format: uuid
+        drift_status:
+          type: string
+        billing_state:
+          type: string
+        ttl_expires_at:
+          type: string
+          format: date-time
+        created_at:
+          type: string
+          format: date-time
+        updated_at:
+          type: string
+          format: date-time
+        fields:
+          type: object
+          additionalProperties: true
+          description: Resource-type-specific realized fields
+        relationships:
+          type: array
+          items:
+            type: object
+    ResourceList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            $ref: '#/components/schemas/ResourceEntity'
+        pagination:
+          type: object
+    ResourceUpdate:
+      type: object
+      required:
+      - fields
+      properties:
+        fields:
+          type: object
+          additionalProperties: true
+          description: Editable fields and their new values (delta only)
+        reason:
+          type: string
+    RehydrateRequest:
+      type: object
+      properties:
+        target_provider_uuid:
+          type: string
+          format: uuid
+        reason:
+          type: string
+    DriftRecord:
+      type: object
+      properties:
+        drift_uuid:
+          type: string
+          format: uuid
+        entity_uuid:
+          type: string
+          format: uuid
+        detected_at:
+          type: string
+          format: date-time
+        severity:
+          type: string
+          enum:
+          - minor
+          - significant
+          - critical
+        unsanctioned:
+          type: boolean
+        status:
+          type: string
+          enum:
+          - open
+          - acknowledged
+          - resolved
+          - escalated
+        field_differences:
+          type: array
+          items:
+            type: object
+    DriftRecordList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            $ref: '#/components/schemas/DriftRecord'
+        pagination:
+          type: object
+    ApprovalRecord:
+      type: object
+      properties:
+        approval_uuid:
+          type: string
+          format: uuid
+        subject_type:
+          type: string
+        subject_uuid:
+          type: string
+          format: uuid
+        required_tier:
+          type: string
+        required_quorum:
+          type: integer
+        decisions_so_far:
+          type: integer
+        expires_at:
+          type: string
+          format: date-time
+        created_at:
+          type: string
+          format: date-time
+    CostEstimate:
+      type: object
+      properties:
+        estimated_monthly_cost:
+          type: number
+        currency:
+          type: string
+        breakdown:
+          type: array
+          items:
+            type: object
+        confidence:
+          type: string
+          enum:
+          - high
+          - medium
+          - low
+    CostActuals:
+      type: object
+      properties:
+        entity_uuid:
+          type: string
+          format: uuid
+        period_start:
+          type: string
+          format: date-time
+        period_end:
+          type: string
+          format: date-time
+        total_cost:
+          type: number
+        currency:
+          type: string
+        breakdown:
+          type: array
+          items:
+            type: object
+    QuotaStatus:
+      type: object
+      properties:
+        tenant_uuid:
+          type: string
+          format: uuid
+        quotas:
+          type: array
+          items:
+            type: object
+    WebhookSubscription:
+      type: object
+      required:
+      - endpoint_url
+      - events
+      properties:
+        webhook_uuid:
+          type: string
+          format: uuid
+          readOnly: true
+        endpoint_url:
+          type: string
+          format: uri
+        events:
+          type: array
+          items:
+            type: string
+          description: Event types from the DCM Event Catalog
+        secret:
+          type: string
+          description: HMAC-SHA256 signing secret for signature verification
+        active:
+          type: boolean
+          default: true
+        created_at:
+          type: string
+          format: date-time
+          readOnly: true
+    SearchResults:
+      type: object
+      properties:
+        results:
+          type: array
+          items:
+            type: object
+        pagination:
+          type: object
+    AuditRecord:
+      type: object
+      properties:
+        audit_uuid:
+          type: string
+          format: uuid
+        entity_uuid:
+          type: string
+          format: uuid
+        event_type:
+          type: string
+        actor:
+          type: object
+        recorded_at:
+          type: string
+          format: date-time
+        correlation_id:
+          type: string
+          format: uuid
+        payload:
+          type: object
+    AuditList:
+      type: object
+      properties:
+        items:
+          type: array
+          items:
+            $ref: '#/components/schemas/AuditRecord'
+        pagination:
+          type: object
+    PolicyContribution:
+      type: object
+      required:
+      - policy_artifact
+      properties:
+        policy_artifact:
+          type: object
+          description: DCM policy artifact per B-policy-contract schema
+        rationale:
+          type: string
+        shadow_first:
+          type: boolean
+          default: true
+    ContributionAccepted:
+      type: object
+      properties:
+        contribution_uuid:
+          type: string
+          format: uuid
+        status:
+          type: string
+        shadow_mode:
+          type: boolean
+    CredentialSummary:
+      type: object
+      properties:
+        credential_uuid:
+          type: string
+          format: uuid
+        credential_type:
+          type: string
+        entity_uuid:
+          type: string
+          format: uuid
+        expires_at:
+          type: string
+          format: date-time
+        rotated_at:
+          type: string
+          format: date-time
+    CredentialValue:
+      type: object
+      properties:
+        credential_uuid:
+          type: string
+          format: uuid
+        credential_type:
+          type: string
+        value:
+          type: object
+          description: Credential-type-specific value (e.g., kubeconfig, token, certificate)
+        expires_at:
+          type: string
+          format: date-time
+
+    WorkloadProfile:
+      type: object
+      description: "Workload analysis classification for a DCM resource entity (doc 46)"
+      properties:
+        workload_profile_uuid: {type: string, format: uuid}
+        subject_entity_uuid: {type: string, format: uuid}
+        analyzed_at: {type: string, format: date-time}
+        analysis_version: {type: string}
+        classification:
+          type: object
+          properties:
+            resource_type_match:
+              type: object
+              properties:
+                primary: {type: string}
+                confidence: {type: string, enum: [high, medium, low, undetermined]}
+            workload_archetype:
+              type: object
+              properties:
+                type: {type: string, enum: [web_server, database, batch_processor, message_broker, api_gateway, cache, storage, monitoring, unknown]}
+                confidence: {type: string, enum: [high, medium, low, undetermined]}
+            migration_readiness:
+              type: object
+              properties:
+                containerization_score: {type: integer, minimum: 1, maximum: 10}
+                blockers: {type: array, items: {type: string}}
+                suggested_target: {type: string}
+            lifecycle_recommendation:
+              type: object
+              properties:
+                dcm_lifecycle_model: {type: string}
+                rehydration_eligible: {type: boolean}
+                notes: {type: string}
diff --git a/content/docs/architecture/specifications/dcm-examples.md b/content/docs/architecture/specifications/dcm-examples.md
index 9f4316f..b513a4a 100644
--- a/content/docs/architecture/specifications/dcm-examples.md
+++ b/content/docs/architecture/specifications/dcm-examples.md
@@ -2187,3 +2187,1195 @@ Platform Admin investigation:
   Accreditation: status โ†’ active (manual override with audit record)
   new_requests: unblocked
 ```
+
+
+# Section 9 โ€” Resource Type and Data Layer Lifecycle (End-to-End)
+
+This section traces the complete lifecycle of two real resource types โ€” a Virtual Machine
+and a Web Application as a Service โ€” from the initial layer definitions authored by their
+owning authorities, through provider catalog item registration, layer assembly at request
+time, and finally through rehydration for both.
+
+The goal is to make concrete the abstract model: layers are data, resource types are built
+from layers, providers extend them with their own layers, and every field in every request
+payload knows exactly which layer set it and why.
+
+---
+
+## 9.1 Layer Definitions โ€” Who Defines What, and Who Owns It
+
+Before any VM can be provisioned or any WebApp offered, the foundational data layers must
+exist. These are created by different authorities, each responsible for their domain.
+
+### Reference Data Layers (created by authority teams, stored in GitOps)
+
+**OS Image layer โ€” owned by Platform Security Team:**
+
+```yaml
+# GitOps path: platform/reference-data/os-images/rhel-9-4-approved.yaml
+layer:
+  artifact_metadata:
+    uuid: "os-img-rhel-9-4"
+    handle: "platform/reference-data/os-images/rhel-9-4"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Platform Security Team"
+      group_handle: "groups/platform-security"
+    created_via: pr
+    created_at: "2026-01-15T09:00:00Z"
+
+  layer_type: reference_data
+  reference_data_type: os_image
+  domain: platform
+
+  data:
+    image_name: "RHEL 9.4 โ€” Approved Standard"
+    image_uuid: "img-rhel-9-4-20260315"
+    image_sha256: "a1b2c3d4e5f6..."
+    os_family: rhel
+    major_version: 9
+    minor_version: 4
+    release_date: "2026-03-15"
+    eol_date: "2032-05-31"
+    fips_compliant: true
+    cis_benchmark_ref: "CIS RHEL 9 Benchmark v1.0"
+    approved_for_classifications: [public, internal, confidential, restricted]
+    requires_subscription: true
+
+  concern_tags: [os-image, rhel, approved, fips-compliant, platform-standard]
+```
+
+**Location layer โ€” owned by Data Center Operations:**
+
+```yaml
+# GitOps path: platform/locations/dc/fra-dc1.yaml
+layer:
+  artifact_metadata:
+    uuid: "loc-fra-dc1"
+    handle: "locations/dc/fra-dc1"
+    version: "2.1.0"
+    status: active
+    owned_by:
+      display_name: "Data Center Operations โ€” Frankfurt"
+      group_handle: "groups/dc-operations-fra"
+    created_via: pr
+
+  layer_type: reference_data
+  reference_data_type: location.data_center
+  domain: platform
+  location_type: data_center
+
+  location_hierarchy:
+    parent_handle: "locations/az/eu-west-1a"
+    ancestors:
+      - { handle: "locations/region/eu-west", type: region }
+      - { handle: "locations/country/de",     type: country }
+
+  data:
+    dc_name: "DC1 โ€” Frankfurt Alpha"
+    dc_code: "FRA-DC1"
+    tier_classification: tier_3
+    pue_rating: 1.35
+    redundancy_model: "2N"
+    jurisdiction: "EU/GDPR"
+    sovereignty_zone: "eu-west-sovereign"
+    max_data_classification: restricted
+    certifications:
+      - { standard: "ISO 27001", valid_until: "2027-06-30" }
+      - { standard: "SOC 2 Type II", valid_until: "2026-12-31" }
+    network_uplinks:
+      - { carrier: "DE-CIX", bandwidth_gbps: 100, redundant: true }
+
+  concern_tags: [location, data-center, frankfurt, eu-west, tier-3, iso27001]
+```
+
+**Network zone layer โ€” owned by Network Operations:**
+
+```yaml
+# GitOps path: platform/reference-data/network-zones/prod-dmz-fra.yaml
+layer:
+  artifact_metadata:
+    uuid: "nz-prod-dmz-fra"
+    handle: "platform/reference-data/network-zones/prod-dmz-fra"
+    version: "1.1.0"
+    status: active
+    owned_by:
+      display_name: "Network Operations"
+      group_handle: "groups/network-ops"
+    created_via: pr
+
+  layer_type: reference_data
+  reference_data_type: network_zone
+  domain: platform
+
+  data:
+    zone_name: "Production DMZ โ€” Frankfurt"
+    zone_code: "PROD-DMZ-FRA"
+    vlan_range: "100-199"
+    allowed_inbound_protocols: [HTTPS, SSH]
+    allowed_outbound_protocols: [HTTPS, DNS, NTP]
+    firewall_policy_ref: "policies/network/prod-dmz-baseline"
+    nat_enabled: true
+    internet_facing: true
+    approved_for_classifications: [public, internal]
+
+  concern_tags: [network-zone, dmz, production, frankfurt, internet-facing]
+```
+
+**Core location context layers โ€” assembled automatically from hierarchy:**
+
+```yaml
+# Zone layer (parent of FRA-DC1) โ€” owned by Data Center Operations
+layer:
+  artifact_metadata:
+    uuid: "loc-az-eu-west-1a"
+    handle: "locations/az/eu-west-1a"
+    version: "1.0.0"
+    status: active
+    owned_by: { display_name: "Data Center Operations", group_handle: "groups/dc-operations" }
+
+  layer_type: reference_data
+  reference_data_type: location.zone
+  domain: platform
+
+  data:
+    zone_name: "EU West Zone A"
+    zone_code: "eu-west-1a"
+    isolation_boundary: full
+    target_rpo_minutes: 15
+    target_rto_minutes: 60
+    ha_peer_zones: ["locations/az/eu-west-1b"]
+
+# Country layer โ€” owned by Platform Governance
+layer:
+  artifact_metadata:
+    uuid: "loc-country-de"
+    handle: "locations/country/de"
+    version: "1.0.0"
+    status: active
+    owned_by: { display_name: "Platform Governance", group_handle: "groups/platform-governance" }
+
+  layer_type: reference_data
+  reference_data_type: location.country
+  domain: platform
+
+  data:
+    country_name: "Germany"
+    iso_3166_1_alpha2: "DE"
+    data_sovereignty_jurisdiction: "EU/GDPR"
+    regulatory_frameworks: [GDPR, NIS2, eIDAS]
+```
+
+---
+
+### 9.2 Resource Type Specification โ€” Defined by the Resource Type Authority
+
+The Platform Team is the Resource Type Authority for `Compute.VirtualMachine`.
+They define the vendor-neutral contract all providers must implement.
+
+```yaml
+# GitOps path: registry/resource-types/compute/virtual-machine/v2-1-0.yaml
+resource_type_specification:
+  artifact_metadata:
+    uuid: "rt-compute-vm"
+    handle: "registry/compute/VirtualMachine"
+    fully_qualified_name: "Compute.VirtualMachine"
+    version: "2.1.0"
+    status: active
+    owned_by:
+      display_name: "Platform Team โ€” Virtualization"
+      group_handle: "groups/platform-team"
+    tier: 1    # DCM Core โ€” maintained by DCM Project
+
+  category: Compute
+  description: "A virtual machine instance. The foundational compute resource."
+
+  # Universal fields โ€” all providers MUST implement these
+  universal_fields:
+
+    cpu_count:
+      type: integer
+      required: true
+      description: "Number of virtual CPUs"
+      portability: { classification: universal }
+      constraints:
+        - type: range
+          min: 1
+          max: 256
+          # Note: constraint is a range โ€” provider judgment for what they support.
+          # No layer_reference here: CPU count is intrinsic to the resource type,
+          # not a governed organizational list.
+
+    memory_gb:
+      type: integer
+      required: true
+      description: "RAM in gigabytes"
+      portability: { classification: universal }
+      constraints:
+        - type: range
+          min: 1
+          max: 4096
+
+    storage_gb:
+      type: integer
+      required: true
+      description: "Primary disk size in gigabytes"
+      portability: { classification: universal }
+      constraints:
+        - type: range
+          min: 10
+          max: 65536
+
+    os_image:
+      type: string
+      format: layer-uuid
+      required: true
+      description: "Approved OS image. Must be a UUID of an active os_image reference data layer."
+      portability: { classification: universal }
+      constraints:
+        - type: layer_reference
+          layer_type: os_image
+          # Allowed values = active os_image layers.
+          # Adding a new approved OS = adding a new os_image layer.
+          # No spec change needed.
+
+    location:
+      type: string
+      format: layer-uuid
+      required: true
+      description: "Allocation location. Must be a UUID of an active location.data_center layer."
+      portability: { classification: universal }
+      constraints:
+        - type: layer_reference
+          layer_type: location.data_center
+          # Allowed values = active DC layers. Each carries jurisdiction,
+          # certifications, sovereignty zone, and capacity status.
+
+    network_zone:
+      type: string
+      format: layer-uuid
+      required: false
+      description: "Network zone. If omitted, placement policy selects default."
+      portability: { classification: universal }
+      constraints:
+        - type: layer_reference
+          layer_type: network_zone
+
+    hostname:
+      type: string
+      required: false
+      description: "VM hostname. If omitted, DCM generates one per naming policy."
+      portability: { classification: universal }
+      constraints:
+        - type: pattern
+          pattern: '^[a-z0-9][a-z0-9-]{1,62}[a-z0-9]$'
+
+    environment:
+      type: string
+      format: layer-uuid
+      required: false
+      description: "Deployment environment. Controls policy set and approval tier."
+      portability: { classification: universal }
+      constraints:
+        - type: layer_reference
+          layer_type: environment
+
+  # Conditional fields โ€” declared by providers that support them
+  conditional_fields:
+
+    high_availability:
+      type: boolean
+      required: false
+      description: "Enable HA โ€” live migration on host failure"
+      portability:
+        classification: conditional
+        portability_notes: "Supported by most hypervisor providers; not applicable for bare metal"
+
+    gpu_profile:
+      type: string
+      format: layer-uuid
+      required: false
+      description: "GPU configuration. Must be a UUID of an active gpu_profile reference data layer."
+      portability:
+        classification: conditional
+        portability_notes: "Only providers with GPU hardware support this field"
+      constraints:
+        - type: layer_reference
+          layer_type: gpu_profile
+
+    backup_policy:
+      type: string
+      required: false
+      description: "Backup schedule reference"
+      portability:
+        classification: conditional
+
+  # Extension point declaration โ€” where providers MAY add fields
+  extension_points:
+    - name: provider_hypervisor_config
+      description: "Provider-specific hypervisor configuration"
+      portability_impact: provider_specific  # using this makes catalog item non-portable
+
+  lifecycle_operations: [create, read, update, delete, suspend, resume, rehydrate, drift_check]
+```
+
+**The WebApp Resource Type โ€” defined by the Application Platform Team:**
+
+```yaml
+# GitOps path: registry/resource-types/application/web-app/v1-0-0.yaml
+resource_type_specification:
+  artifact_metadata:
+    uuid: "rt-app-webapp"
+    handle: "registry/application/WebApp"
+    fully_qualified_name: "Application.WebApp"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Application Platform Team"
+      group_handle: "groups/app-platform"
+    tier: 3    # Organization tier
+
+  category: Application
+  description: >
+    A complete web application stack: load balancer, application VMs, and
+    database โ€” provisioned and lifecycle-managed as a single compound resource.
+    Implemented by a Meta Provider that orchestrates constituent atomic resources.
+
+  universal_fields:
+
+    app_name:
+      type: string
+      required: true
+      description: "Application name โ€” used in DNS, naming, and tagging"
+      portability: { classification: universal }
+      constraints:
+        - type: pattern
+          pattern: '^[a-z0-9-]{3,48}$'
+
+    environment:
+      type: string
+      format: layer-uuid
+      required: true
+      description: "Deployment environment (controls redundancy, approval tier, TTL)"
+      portability: { classification: universal }
+      constraints:
+        - type: layer_reference
+          layer_type: environment
+
+    location:
+      type: string
+      format: layer-uuid
+      required: true
+      description: "Target zone or data center"
+      portability: { classification: universal }
+      constraints:
+        - type: layer_reference
+          layer_type: location.data_center
+
+    tier_level:
+      type: string
+      required: true
+      description: "Service tier governing redundancy, SLA, and approval"
+      portability: { classification: universal }
+      constraints:
+        - type: enum
+          allowed_values: [tier_1, tier_2, tier_3]
+          # Static enum โ€” tier names are intrinsic to the resource type.
+          # Each tier carries policy implications enforced by GateKeeper policies.
+
+    web_replica_count:
+      type: integer
+      required: false
+      description: "Number of web tier VMs. Policy enforces minimums per tier."
+      portability: { classification: universal }
+      constraints:
+        - type: range
+          min: 1
+          max: 20
+
+    db_engine:
+      type: string
+      required: true
+      description: "Database engine"
+      portability: { classification: universal }
+      constraints:
+        - type: enum
+          allowed_values: [postgresql, mysql, mariadb]
+
+    db_storage_gb:
+      type: integer
+      required: true
+      constraints:
+        - type: range
+          min: 50
+          max: 10000
+
+  lifecycle_operations: [create, read, update, delete, suspend, resume, rehydrate, scale_out, drift_check]
+```
+
+---
+
+## 9.3 Provider Catalog Items โ€” Implementing the Resource Type
+
+Two compute providers register catalog items implementing `Compute.VirtualMachine`.
+Both must cover all universal fields. Each adds provider-specific constraints and
+optionally extends with their own layers.
+
+**Provider A โ€” Nutanix EU-WEST (portable catalog item):**
+
+```yaml
+# GitOps path: providers/nutanix-eu-west/catalog/vm-standard.yaml
+catalog_item:
+  uuid: "ci-nutanix-eu-west-vm-std"
+  name: "Nutanix EU-WEST โ€” Standard VM"
+  version: "1.3.0"
+  status: active
+
+  implements:
+    resource_type_uuid: "rt-compute-vm"
+    resource_type_version: "2.1.0"
+    resource_type_fully_qualified_name: "Compute.VirtualMachine"
+
+  portability_warning: false        # no provider-specific extensions
+  portability_class: portable
+
+  # Universal fields โ€” Nutanix's implementation of the spec
+  universal_fields:
+    cpu_count:
+      constraint: { type: enum, allowed_values: [2, 4, 8, 16, 32] }
+      # Nutanix narrows the spec's 1-256 range to their supported sizes.
+      # Still portable: another provider may offer overlapping values.
+
+    memory_gb:
+      constraint: { type: enum, allowed_values: [4, 8, 16, 32, 64, 128, 256] }
+
+    storage_gb:
+      constraint: { type: range, min: 40, max: 4096 }
+
+    os_image:
+      # Inherits layer_reference from spec โ€” no override needed.
+      # Nutanix resolves the consumer's os_image layer UUID against their
+      # registered OS image inventory at dispatch time.
+
+    location:
+      # Inherits layer_reference from spec.
+      # Only location layers in Nutanix's registered availability_zones appear
+      # in allowed_values when this catalog item is selected.
+      filter:
+        availability_zones: ["eu-west-1a", "eu-west-1b"]
+
+  # Conditional fields this provider supports
+  conditional_fields_supported:
+    - high_availability    # Nutanix AOS live migration supported
+
+  # Nutanix contributes a Service Layer (domain: service) with defaults
+  # that apply when this catalog item is selected
+  service_layer_handle: "providers/nutanix-eu-west/layers/vm-platform-defaults"
+
+  # Cost metadata
+  cost_metadata:
+    pricing_model: per_hour
+    base_cost_per_vcpu_hour: 0.025
+    base_cost_per_gb_ram_hour: 0.008
+    currency: USD
+
+  sovereignty:
+    data_residency: EU
+    jurisdiction_codes: [DE, NL]
+    availability_zones: ["eu-west-1a", "eu-west-1b"]
+```
+
+**Nutanix Service Layer โ€” injected for all Nutanix VM requests:**
+
+```yaml
+# GitOps path: providers/nutanix-eu-west/layers/vm-platform-defaults.yaml
+layer:
+  artifact_metadata:
+    uuid: "sl-nutanix-eu-west-vm"
+    handle: "providers/nutanix-eu-west/layers/vm-platform-defaults"
+    version: "2.0.0"
+    status: active
+    owned_by:
+      display_name: "Nutanix EU-WEST Operations"
+      group_handle: "providers/nutanix-eu-west/ops-team"
+    created_via: pr
+
+  layer_type: service
+  domain: service
+  type_scope:
+    resource_type_fqn: "Compute.VirtualMachine"
+    scope_inheritance: exact
+
+  # These fields are injected into every Nutanix VM request payload
+  data:
+    hypervisor: "AHV"                           # Nutanix Acropolis Hypervisor
+    cluster_uuid: "nutanix-cluster-fra-01"
+    storage_container: "default-container"
+    network_function_chain: "nfc-prod-default"
+    backup_enabled: true                        # Nutanix default backup policy
+    backup_schedule: "daily-7d-retention"
+    cvm_cores: 2                                # Controller VM allocation
+    monitoring_agent: "nutanix-era-agent"
+    support_tier: "standard"
+```
+
+**Provider B โ€” VMware EU-WEST (provider-extended, non-portable):**
+
+```yaml
+# GitOps path: providers/vmware-eu-west/catalog/vm-enterprise.yaml
+catalog_item:
+  uuid: "ci-vmware-eu-west-vm-ent"
+  name: "VMware EU-WEST โ€” Enterprise VM"
+  version: "1.0.0"
+  status: active
+
+  implements:
+    resource_type_uuid: "rt-compute-vm"
+    resource_type_version: "2.1.0"
+    resource_type_fully_qualified_name: "Compute.VirtualMachine"
+
+  portability_warning: true         # provider-specific extensions present
+  portability_class: provider-specific
+
+  universal_fields:
+    cpu_count:
+      constraint: { type: range, min: 1, max: 128 }
+    memory_gb:
+      constraint: { type: range, min: 2, max: 2048 }
+    storage_gb:
+      constraint: { type: range, min: 20, max: 8192 }
+
+  conditional_fields_supported:
+    - high_availability
+
+  # VMware extends the resource type with vSphere-specific fields
+  # via a provider extension layer (domain: provider)
+  provider_extension_layer_handles:
+    - "providers/vmware-eu-west/layers/vsphere-extensions-v1"
+
+  # These extension fields make the catalog item non-portable โ€”
+  # if a consumer uses them, their request is VMware-specific
+```
+
+**VMware Provider Extension Layer โ€” makes catalog item non-portable:**
+
+```yaml
+# GitOps path: providers/vmware-eu-west/layers/vsphere-extensions-v1.yaml
+layer:
+  artifact_metadata:
+    uuid: "pl-vmware-vsphere-ext"
+    handle: "providers/vmware-eu-west/layers/vsphere-extensions-v1"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "VMware EU-WEST Operations"
+      group_handle: "providers/vmware-eu-west/ops-team"
+
+  layer_type: provider_extension
+  domain: provider              # lowest authority โ€” cannot override platform/tenant layers
+  type_scope:
+    resource_type_fqn: "Compute.VirtualMachine"
+
+  # VMware-specific fields exposed to consumers of this catalog item.
+  # These are NOT part of the Resource Type Specification.
+  # Using them makes the request non-portable (VMware-only).
+  extension_fields:
+    vsphere_resource_pool:
+      type: string
+      portability_breaking: true
+      description: "vSphere resource pool name"
+      constraint: { type: enum, allowed_values: ["prod-pool-a", "prod-pool-b", "dev-pool"] }
+
+    vsphere_datastore_cluster:
+      type: string
+      portability_breaking: true
+      description: "vSphere datastore cluster for VMDK placement"
+
+    vmware_tools_version:
+      type: string
+      portability_breaking: true
+      description: "Minimum VMware Tools version"
+      constraint: { type: pattern, pattern: '^\d+\.\d+\.\d+$' }
+
+  # These fields also inject defaults for the provider's own use
+  data:
+    hypervisor: "ESXi 8.0"
+    cluster_name: "vmware-cluster-fra-01"
+    distributed_switch: "dvs-prod-01"
+    admission_control: true
+```
+
+---
+
+## 9.4 Consumer Request โ€” VM Provisioning
+
+The consumer browses the catalog, sees the two VM offerings, selects the Nutanix one,
+picks their location and OS image from the resolved `allowed_values` lists, and submits.
+
+**What the consumer sees (GET /api/v1/catalog/ci-nutanix-eu-west-vm-std):**
+
+```json
+{
+  "catalog_item_uuid": "ci-nutanix-eu-west-vm-std",
+  "display_name": "Nutanix EU-WEST โ€” Standard VM",
+  "resource_type": "Compute.VirtualMachine",
+  "portability_class": "portable",
+  "portability_warning": false,
+
+  "schema": {
+    "fields": [
+      {
+        "field_name": "cpu_count",
+        "type": "integer",
+        "required": true,
+        "constraint": { "type": "enum", "allowed_values": [2, 4, 8, 16, 32] }
+      },
+      {
+        "field_name": "os_image",
+        "type": "string",
+        "required": true,
+        "constraint": {
+          "type": "layer_reference",
+          "layer_type": "os_image",
+          "allowed_values": [
+            {
+              "value": "os-img-rhel-9-4",
+              "display_name": "RHEL 9.4 โ€” Approved Standard",
+              "os_family": "rhel",
+              "fips_compliant": true,
+              "eol_date": "2032-05-31"
+            },
+            {
+              "value": "os-img-ubuntu-24-04",
+              "display_name": "Ubuntu 24.04 LTS",
+              "os_family": "ubuntu",
+              "fips_compliant": false,
+              "eol_date": "2029-04-30"
+            }
+          ]
+        }
+      },
+      {
+        "field_name": "location",
+        "type": "string",
+        "required": true,
+        "constraint": {
+          "type": "layer_reference",
+          "layer_type": "location.data_center",
+          "allowed_values": [
+            {
+              "value": "loc-fra-dc1",
+              "display_name": "DC1 โ€” Frankfurt Alpha",
+              "code": "FRA-DC1",
+              "zone": "eu-west-1a",
+              "sovereignty": "EU/GDPR",
+              "certifications": ["ISO 27001", "SOC 2 Type II"],
+              "capacity_status": "available"
+            },
+            {
+              "value": "loc-ams-dc2",
+              "display_name": "DC2 โ€” Amsterdam Beta",
+              "code": "AMS-DC2",
+              "zone": "eu-west-1b",
+              "sovereignty": "EU/GDPR",
+              "certifications": ["ISO 27001"],
+              "capacity_status": "limited"
+            }
+          ]
+        }
+      }
+    ]
+  }
+}
+```
+
+**Consumer submits request:**
+
+```json
+POST /api/v1/requests
+{
+  "catalog_item_uuid": "ci-nutanix-eu-west-vm-std",
+  "fields": {
+    "cpu_count": 8,
+    "memory_gb": 32,
+    "storage_gb": 120,
+    "os_image": "os-img-rhel-9-4",
+    "location": "loc-fra-dc1",
+    "hostname": "payments-api-01"
+  }
+}
+```
+
+---
+
+## 9.5 Request Processing Pipeline โ€” VM
+
+Tracing every step from submission to realization, showing which layer contributes
+which field and why.
+
+```
+Step 1 โ€” INTENT STATE CAPTURED
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+Stored verbatim โ€” the consumer's exact submission:
+  {
+    "catalog_item_uuid": "ci-nutanix-eu-west-vm-std",
+    "fields": {
+      "cpu_count": 8,
+      "memory_gb": 32,
+      "storage_gb": 120,
+      "os_image": "os-img-rhel-9-4",       // layer UUID
+      "location": "loc-fra-dc1",            // layer UUID
+      "hostname": "payments-api-01"
+    }
+  }
+Intent UUID: intent-vm-001
+Stored in: Intent Store (GitOps โ€” immutable)
+Nothing modified. No policies run yet. This is the permanent record of consumer intent.
+
+Step 2 โ€” LAYER REFERENCE RESOLUTION
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+DCM resolves each layer UUID to its full artifact:
+  os_image โ†’ layer "os-img-rhel-9-4":
+    image_uuid: "img-rhel-9-4-20260315"
+    image_sha256: "a1b2c3..."
+    fips_compliant: true
+    eol_date: "2032-05-31"
+
+  location โ†’ layer "loc-fra-dc1" + ancestor chain:
+    Country layer (loc-country-de):
+      jurisdiction: EU/GDPR
+      regulatory_frameworks: [GDPR, NIS2]
+    Zone layer (loc-az-eu-west-1a):
+      zone_code: eu-west-1a
+      target_rpo_minutes: 15
+      isolation_boundary: full
+    Data Center layer (loc-fra-dc1):
+      dc_code: FRA-DC1
+      sovereignty_zone: eu-west-sovereign
+      max_data_classification: restricted
+      certifications: [ISO 27001, SOC 2 Type II]
+
+Step 3 โ€” LAYER ASSEMBLY (precedence order, lowest first)
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+  1. Base Layer (platform/base/compute-vm-baseline):
+     โ†’ monitoring_agent: null (to be injected by policy)
+     โ†’ backup_enabled: false (default)
+     โ†’ Source: Base Layer
+
+  2. Core Layers assembled:
+     โ”Œโ”€ Country layer (loc-country-de):
+     โ”‚   location.jurisdiction = EU/GDPR
+     โ”‚   location.regulatory_frameworks = [GDPR, NIS2]
+     โ”œโ”€ Zone layer (loc-az-eu-west-1a):
+     โ”‚   location.zone_code = eu-west-1a
+     โ”‚   location.rpo_minutes = 15
+     โ””โ”€ DC layer (loc-fra-dc1):
+         location.dc_code = FRA-DC1
+         location.sovereignty_zone = eu-west-sovereign
+         location.max_data_classification = restricted
+
+  3. Service Layer (providers/nutanix-eu-west/layers/vm-platform-defaults):
+     โ†’ hypervisor = AHV
+     โ†’ cluster_uuid = nutanix-cluster-fra-01
+     โ†’ storage_container = default-container
+     โ†’ backup_enabled = true        (overrides Base Layer default)
+     โ†’ backup_schedule = daily-7d-retention
+     โ†’ monitoring_agent = nutanix-era-agent
+
+  4. OS Image layer data injected (resolved from os-img-rhel-9-4):
+     โ†’ os.image_uuid = img-rhel-9-4-20260315
+     โ†’ os.image_sha256 = a1b2c3...
+     โ†’ os.fips_compliant = true
+     โ†’ os.eol_date = 2032-05-31
+
+  5. Request Layer (consumer's fields โ€” highest data layer precedence):
+     โ†’ cpu_count = 8
+     โ†’ memory_gb = 32
+     โ†’ storage_gb = 120
+     โ†’ hostname = payments-api-01
+
+Assembled payload at this point (before policies):
+  cpu_count: 8            [source: Request Layer]
+  memory_gb: 32           [source: Request Layer]
+  storage_gb: 120         [source: Request Layer]
+  hostname: payments-api-01 [source: Request Layer]
+  os.image_uuid: img-rhel-9-4-20260315  [source: os_image reference layer]
+  hypervisor: AHV         [source: Service Layer / Nutanix]
+  location.dc_code: FRA-DC1             [source: Core Location Layer]
+  location.zone_code: eu-west-1a        [source: Core Location Layer]
+  location.jurisdiction: EU/GDPR        [source: Core Location Layer]
+  location.sovereignty_zone: eu-west-sovereign  [source: Core Location Layer]
+  backup_enabled: true    [source: Service Layer / Nutanix]
+  monitoring_agent: nutanix-era-agent   [source: Service Layer / Nutanix]
+
+Step 4 โ€” POLICY EVALUATION
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+  GateKeeper โ€” Sovereignty Check:
+    PASS: location.sovereignty_zone = eu-west-sovereign
+          tenant data classification โ‰ค restricted
+          No cross-border transfer
+
+  Validation โ€” FIPS Requirement (FSI profile):
+    PASS: os.fips_compliant = true
+
+  Transformation โ€” Monitoring Agent Injection:
+    ADD: monitoring_agent = nutanix-era-agent  (already present โ€” no override)
+    ADD: monitoring_config.endpoint = monitoring.internal:9090
+    ADD: monitoring_config.scrape_interval = 30s
+    Provenance: { source: policy/transform/monitoring-inject, immutable: true }
+
+  Transformation โ€” Naming Convention:
+    MODIFY: hostname = payments-api-01 โ†’ validated against pattern '^[a-z0-9-]{3,63}$' โœ“
+    ADD: fqdn = payments-api-01.fra-dc1.eu-west.corp.example.com
+    Provenance: { source: policy/transform/naming-convention }
+
+  GateKeeper โ€” Cost Gate (if estimate > threshold):
+    Estimated cost: $0.38/hour โ†’ $274/month
+    Tenant monthly budget: $5,000 remaining
+    PASS: within budget
+
+Step 5 โ€” PLACEMENT SELECTION
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+  Placement Engine evaluates against location.sovereignty_zone = eu-west-sovereign:
+    Candidate providers:
+      Nutanix EU-WEST (eu-west-1a) โ†’ confirmed capacity
+      VMware EU-WEST (eu-west-1a) โ†’ confirmed capacity
+    
+    Filtered to catalog item ci-nutanix-eu-west-vm-std โ†’ Nutanix EU-WEST selected
+    Reserve query confirmed: Nutanix holds capacity for this request (hold: PT5M)
+
+Step 6 โ€” REQUESTED STATE WRITTEN
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+Full assembled payload stored in Requested Store.
+Every field carries provenance:
+  cpu_count: 8
+    _provenance: { source: request.layer, intent_uuid: intent-vm-001 }
+  hypervisor: AHV
+    _provenance: { source: service.layer/nutanix-vm-defaults, version: 2.0.0 }
+  location.dc_code: FRA-DC1
+    _provenance: { source: core.layer/loc-fra-dc1, version: 2.1.0 }
+  monitoring_agent: nutanix-era-agent
+    _provenance: { source: policy/transform/monitoring-inject, immutable: true }
+  os.image_uuid: img-rhel-9-4-20260315
+    _provenance: { source: reference.layer/os-img-rhel-9-4, version: 1.0.0 }
+
+Step 7 โ€” DISPATCH TO PROVIDER
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+CreateRequest dispatched to Nutanix EU-WEST:
+  {
+    "dcm_entity_uuid": "vm-abc123",
+    "request_uuid": "req-xyz789",
+    "resource_type_uuid": "rt-compute-vm",
+    "resource_type_name": "Compute.VirtualMachine",
+    "fields": {
+      "cpu_count": 8,
+      "memory_gb": 32,
+      "storage_gb": 120,
+      "hostname": "payments-api-01",
+      "fqdn": "payments-api-01.fra-dc1.eu-west.corp.example.com",
+      "os_image_uuid": "img-rhel-9-4-20260315",
+      "hypervisor": "AHV",
+      "cluster_uuid": "nutanix-cluster-fra-01",
+      "backup_enabled": true,
+      "monitoring_agent": "nutanix-era-agent"
+      // Full payload โ€” Nutanix naturalizes to their API format
+    }
+  }
+
+Step 8 โ€” REALIZED STATE WRITTEN
+โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
+Nutanix provisions the VM, denaturalizes the result:
+  {
+    "dcm_entity_uuid": "vm-abc123",
+    "resource_id": "nutanix-vm-8f7e6d5c",   // Nutanix's internal ID
+    "lifecycle_state": "OPERATIONAL",
+    "realized_fields": {
+      "primary_ip": "10.100.1.42",
+      "mac_address": "00:50:56:8f:7e:6d",
+      "host_uuid": "nutanix-host-001",
+      "realized_at": "2026-03-31T10:05:33Z",
+      "nutanix_vm_uuid": "8f7e6d5c-..."    // provider-native ID stored for correlation
+    }
+  }
+Entity vm-abc123 โ†’ status: OPERATIONAL
+```
+
+---
+
+## 9.6 Consumer Request โ€” WebApp as a Service
+
+The WebApp catalog item is backed by a Meta Provider that orchestrates VM, LoadBalancer,
+and Database constituent resources โ€” all provisioned as one consumer action.
+
+**Consumer submits:**
+
+```json
+POST /api/v1/requests
+{
+  "catalog_item_uuid": "ci-webapp-payments-stack",
+  "fields": {
+    "app_name": "payments-portal",
+    "environment": "env-layer-production",   // layer UUID โ€” production environment
+    "location": "loc-fra-dc1",               // layer UUID โ€” FRA-DC1
+    "tier_level": "tier_1",
+    "web_replica_count": 3,
+    "db_engine": "postgresql",
+    "db_storage_gb": 500
+  }
+}
+```
+
+**What the Meta Provider orchestrates (transparent to consumer):**
+
+```
+Meta Provider decomposes the request into constituent requests:
+
+  Constituent 1: Compute.VirtualMachine ร— 3 (web tier)
+    os_image: os-img-rhel-9-4          // from platform OS image reference layer
+    location: loc-fra-dc1              // same DC as parent request
+    cpu_count: 4                       // from environment layer defaults
+    memory_gb: 16                      // from environment layer defaults
+    hostname: payments-portal-web-{1,2,3}
+    network_zone: nz-prod-dmz-fra      // injected by Placement Policy (tier_1 + web)
+
+  Constituent 2: Network.LoadBalancer ร— 1
+    location: loc-fra-dc1
+    protocol: HTTPS
+    port: 443
+    health_check_path: /health
+    backend_pool: []   // filled by dependency injection after VM realization
+
+  Constituent 3: Storage.DatabaseInstance ร— 1
+    location: loc-fra-dc1
+    db_engine: postgresql
+    storage_gb: 500
+    high_availability: true            // enforced by tier_1 GateKeeper policy
+    backup_enabled: true               // injected by environment layer
+
+Environment layer injection (env-layer-production):
+  default_cpu_per_vm: 4
+  default_ram_per_vm: 16
+  backup_enabled: true
+  ttl: null                           // production: no TTL
+  approval_tier: team_lead            // changes require team lead approval
+  monitoring: mandatory
+  log_retention_days: 90
+
+Tier 1 GateKeeper policies fire:
+  โ†’ Minimum 3 web VMs enforced (3 requested โœ“)
+  โ†’ HA required on database (high_availability: true injected)
+  โ†’ LTM required in front of web tier (LoadBalancer constituent โœ“)
+  โ†’ Cross-zone redundancy check: 3 VMs placed across โ‰ฅ2 zones โœ“
+
+Dispatch sequence:
+  T+0s:   Database dispatched (no dependencies)
+  T+45s:  Database REALIZED โ†’ db_host=10.100.2.10
+  T+45s:  Web VMs dispatched (db_host injected from db realization)
+  T+90s:  Web VMs REALIZED โ†’ ips=[10.100.1.42, 10.100.1.43, 10.100.1.44]
+  T+90s:  LoadBalancer dispatched (backend_pool injected from VM realization)
+  T+105s: LoadBalancer REALIZED โ†’ vip=203.0.113.42
+
+Realized entity: Application.WebApp
+  entity_uuid: webapp-payments-portal
+  status: OPERATIONAL
+  constituents:
+    web_vms: [vm-web-001, vm-web-002, vm-web-003]
+    load_balancer: lb-001
+    database: db-001
+  endpoint: payments-portal.fra-dc1.eu-west.corp.example.com โ†’ 203.0.113.42
+```
+
+---
+
+## 9.7 Rehydration โ€” VM (Intent Mode, DR Failover)
+
+The payments-api-01 VM is in DC1 which is unavailable. The consumer triggers
+rehydration โ€” replaying the original intent through current policies and layers,
+placing the new VM in DC2.
+
+This is NOT Static Replace (which re-executes the Requested State verbatim).
+Rehydration re-runs the full assembly pipeline from Intent State โ€” applying today's
+layers and policies, including the new location constraint.
+
+```
+Consumer: POST /api/v1/resources/vm-abc123:rehydrate
+  {
+    "mode": "intent",
+    "reason": "DC1 unavailable โ€” DR failover to DC2",
+    "placement_constraints": {
+      "location": "loc-ams-dc2"    // consumer explicitly targets DC2 layer UUID
+    }
+  }
+
+Pipeline:
+
+Step 1 โ€” RETRIEVE INTENT STATE (intent-vm-001)
+  Original consumer submission:
+    cpu_count: 8, memory_gb: 32, storage_gb: 120
+    os_image: os-img-rhel-9-4       // same layer UUID โ€” still valid
+    location: loc-fra-dc1           // OVERRIDDEN by placement_constraints
+    hostname: payments-api-01
+
+Step 2 โ€” LAYER REFERENCE RESOLUTION (fresh run)
+  os_image (os-img-rhel-9-4): same layer, still active
+    โ†’ resolves to current approved RHEL 9.4 image
+    โ†’ Note: if Platform Security had retired RHEL 9.4 and issued RHEL 9.5,
+      the new os_image layer UUID would need to be in the intent, OR a
+      Transformation policy could auto-upgrade to the latest approved image.
+
+  location OVERRIDE โ†’ loc-ams-dc2:
+    Country layer (loc-country-nl):
+      jurisdiction: EU/GDPR (same sovereignty zone โ€” valid for this tenant)
+    Zone layer (loc-az-eu-west-1b):
+      zone_code: eu-west-1b
+    DC layer (loc-ams-dc2):
+      dc_code: AMS-DC2
+      sovereignty_zone: eu-west-sovereign    // same zone โ€” rehydration permitted
+      certifications: [ISO 27001]            // SOC 2 not present here
+
+Step 3 โ€” LAYER ASSEMBLY (fresh โ€” current layers used, not original)
+  Service Layer: providers/nutanix-eu-west/layers/vm-platform-defaults
+    โ†’ cluster_uuid: nutanix-cluster-ams-01   // different cluster in DC2
+    โ†’ storage_container: dc2-default-container
+    (This is the key difference from Static Replace โ€” current service layer
+     reflects DC2 infrastructure, not DC1)
+
+Step 4 โ€” POLICY EVALUATION (fresh โ€” current policies applied)
+  GateKeeper โ€” Sovereignty Check:
+    PASS: loc-ams-dc2 in eu-west-sovereign zone
+          Same regulatory scope โ€” GDPR/NIS2 still applies
+
+  Validation โ€” Certification Check:
+    WARNING: loc-ams-dc2 has ISO 27001 but not SOC 2 Type II
+    Active policy: warn-only for standard profile
+    Provenance: { audit_warning: "SOC 2 Type II not available at AMS-DC2" }
+
+  Transformation โ€” Hostname preservation:
+    hostname: payments-api-01 (preserved from intent โ€” same logical identity)
+    fqdn: payments-api-01.ams-dc2.eu-west.corp.example.com  (DC2 FQDN)
+
+Step 5 โ€” PLACEMENT
+  Nutanix EU-WEST operates in both zones โ€” selected (same provider)
+  Reserve query: Nutanix AMS cluster confirms capacity
+
+Step 6 โ€” NEW REQUESTED STATE WRITTEN
+  Linked to: intent-vm-001 (same original intent)
+  New requested state UUID: req-rehydrate-vm-abc123-002
+  Location fields now reflect AMS-DC2 chain
+  Full provenance chain preserved:
+    req-001 โ†’ [FRA-DC1 realization]
+    req-002 โ†’ [AMS-DC2 rehydration] โ† current
+
+Step 7 โ€” DISPATCH + REALIZATION
+  Original DC1 VM: status โ†’ DECOMMISSIONED (DC1 cleanup queued for when DC1 recovers)
+  New AMS-DC2 VM: OPERATIONAL
+  entity_uuid: vm-abc123 (PRESERVED โ€” same entity, new location)
+  primary_ip: 10.200.1.55    // AMS-DC2 IP
+  DNS updated: payments-api-01.ams-dc2.eu-west.corp.example.com
+
+Provenance chain for payments-api-01:
+  Intent captured: 2026-03-01 (intent-vm-001)
+  Realized in DC1: 2026-03-01 (req-001) โ†’ realized-001
+  Rehydrated to DC2: 2026-03-31 (req-002) โ†’ realized-002 [linked to intent-vm-001]
+```
+
+---
+
+## 9.8 Rehydration โ€” WebApp as a Service (Intent Mode, Standards Refresh)
+
+The payments-portal WebApp was provisioned 6 months ago. The Platform Security Team
+has published a new approved OS image (RHEL 9.5) and retired RHEL 9.4. The organization
+runs a quarterly rehydration cycle to bring all Tier 1 apps up to current standards.
+
+This is different from the VM failover rehydration โ€” no DC change, no incident.
+The goal is standards refresh: replay intent through current layers to pick up the
+new OS image and any updated policy/layer defaults.
+
+```
+Platform Admin: POST /api/v1/resources/webapp-payments-portal:rehydrate
+  {
+    "mode": "intent",
+    "reason": "Q1 2026 standards refresh โ€” RHEL 9.5 rollout, updated env layer defaults",
+    "reuse_intent_version": null     // use original intent as-is
+  }
+
+Intent State retrieved (for each constituent):
+
+  Web VMs (ร— 3):
+    app_name: payments-portal
+    os_image: os-img-rhel-9-4    // RETIRED โ€” no longer active layer
+    location: loc-fra-dc1        // still valid
+    tier_level: tier_1
+
+  Database:
+    db_engine: postgresql
+    location: loc-fra-dc1
+    storage_gb: 500
+
+Layer Resolution โ€” key changes since original provisioning:
+
+  os_image (os-img-rhel-9-4): STATUS = retired
+    โ†’ Transformation policy: "os_image_auto_upgrade" fires
+    โ†’ Finds current latest active os_image layer for os_family=rhel:
+      os-img-rhel-9-5 (RHEL 9.5, released 2026-09-01)
+    โ†’ REPLACES os_image in assembled payload
+    โ†’ Provenance: { auto_upgraded_from: "os-img-rhel-9-4", by: policy/transform/os-image-auto-upgrade }
+
+  environment layer (env-layer-production): VERSION bumped from 1.0 to 1.2
+    โ†’ New defaults: log_retention_days: 365 (was 90 โ€” compliance requirement added)
+    โ†’ New: vulnerability_scan_enabled: true
+    โ†’ backup_schedule: weekly-30d-retention (was daily-7d)
+    โ†’ These new defaults inject into the assembled payload
+
+  location (loc-fra-dc1): VERSION 2.1.0 โ†’ 2.2.0
+    โ†’ New certification added: DORA (EU Digital Operational Resilience Act)
+    โ†’ Injected into payload and audit record
+
+Policy changes since original provisioning:
+
+  New GateKeeper: "vulnerability_scan_on_rehydrate" (added 2026-06-01)
+    โ†’ Requires: vulnerability_scan_schedule declared before realization
+    โ†’ Transformation: adds vulnerability_scan_enabled: true, schedule: weekly
+
+  Tier 1 minimum web replicas: increased from 3 to 4 (policy updated 2026-08-01)
+    โ†’ GateKeeper fires: current request has web_replica_count: 3
+    โ†’ Policy action: AUTO_ADJUST (adds one more VM to constituent requests)
+    โ†’ Consumer notified: "web_replica_count adjusted from 3 to 4 per updated Tier 1 policy"
+    โ†’ New VM constituent added to rehydration dispatch
+
+Rehydration Execution:
+
+  Rolling replacement strategy (Tier 1 โ€” zero downtime):
+    Phase 1: Provision new VMs with RHEL 9.5 (alongside existing RHEL 9.4 VMs)
+      โ†’ 4 new VMs provisioned in FRA-DC1 (RHEL 9.5, updated env defaults)
+      โ†’ Load balancer backend pool updated: drain RHEL 9.4 VMs one at a time
+    Phase 2: Verify new VMs healthy (health check passes)
+    Phase 3: Remove old RHEL 9.4 VMs from pool โ†’ decommission
+    Phase 4: Database: snapshot + engine version check (no RHEL dependency โ€” unchanged)
+
+Result:
+  payments-portal: OPERATIONAL
+  RHEL version: 9.4 โ†’ 9.5 (on all 4 web VMs)
+  Replica count: 3 โ†’ 4 (Tier 1 policy enforcement)
+  Log retention: 90 days โ†’ 365 days
+  Vulnerability scanning: enabled (new policy)
+  DC: FRA-DC1 (unchanged)
+
+Provenance chain for payments-portal:
+  Original intent: 2026-03-01 (env-layer-production v1.0, rhel-9-4, 3 replicas)
+  Q1 refresh: 2026-09-30 (env-layer-production v1.2, rhel-9-5, 4 replicas)
+  Both intent records preserved โ€” can audit exactly what changed between cycles.
+
+Audit record highlights:
+  os_image: auto-upgraded from os-img-rhel-9-4 (retired) โ†’ os-img-rhel-9-5
+    by: policy/transform/os-image-auto-upgrade
+  web_replica_count: 3 โ†’ 4
+    by: policy/gatekeeper/tier1-minimum-replicas v2.0
+  log_retention_days: 90 โ†’ 365
+    by: environment layer env-layer-production v1.2
+  vulnerability_scan_enabled: false โ†’ true
+    by: policy/gatekeeper/vulnerability-scan-on-rehydrate v1.0
+```
+
+---
diff --git a/content/docs/architecture/specifications/dcm-flow-gui-spec.md b/content/docs/architecture/specifications/dcm-flow-gui-spec.md
index 1b563f3..1a54d87 100644
--- a/content/docs/architecture/specifications/dcm-flow-gui-spec.md
+++ b/content/docs/architecture/specifications/dcm-flow-gui-spec.md
@@ -1,5 +1,9 @@
 # DCM Flow GUI Specification
 
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** User Interface Specification
+
+
 > **๐Ÿ“‹ Draft**
 >
 > This specification defines the DCM Flow GUI โ€” the visual interface for platform engineers to compose, test, simulate, and manage DCM's policy-driven orchestration. All views, data contracts, API endpoints, and component structure are specified. Feedback and contributions welcome via [GitHub Issues](https://github.com/dcm-project/issues).
diff --git a/content/docs/architecture/specifications/dcm-opa-integration-spec.md b/content/docs/architecture/specifications/dcm-opa-integration-spec.md
index bb13777..454ab4c 100644
--- a/content/docs/architecture/specifications/dcm-opa-integration-spec.md
+++ b/content/docs/architecture/specifications/dcm-opa-integration-spec.md
@@ -1,5 +1,9 @@
 # DCM OPA Integration Specification
 
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Integration Specification
+
+
 > **AEP Alignment:** DCM API endpoints referenced in this spec follow [AEP](https://aep.dev) conventions. `resource_type` accepts FQN string or Registry UUID โ€” DCM resolves internally. See `schemas/openapi/dcm-consumer-api.yaml` and `dcm-admin-api.yaml`.
 
 
diff --git a/content/docs/architecture/specifications/dcm-operator-interface-spec.md b/content/docs/architecture/specifications/dcm-operator-interface-spec.md
index 4408748..794f8f9 100644
--- a/content/docs/architecture/specifications/dcm-operator-interface-spec.md
+++ b/content/docs/architecture/specifications/dcm-operator-interface-spec.md
@@ -1,5 +1,9 @@
 # DCM Operator Interface Specification
 
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Operator Interface Specification
+
+
 
 > ## ๐Ÿ“‹ Draft โ€” Promoted from Work in Progress
 >
diff --git a/content/docs/architecture/specifications/dcm-registration-spec.md b/content/docs/architecture/specifications/dcm-registration-spec.md
index 50f58ce..afe34a2 100644
--- a/content/docs/architecture/specifications/dcm-registration-spec.md
+++ b/content/docs/architecture/specifications/dcm-registration-spec.md
@@ -1,5 +1,9 @@
 # DCM Registration Specification
 
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Registration Specification
+
+
 > **AEP Alignment:** Registration API endpoints follow [AEP](https://aep.dev) conventions โ€” custom methods use colon syntax (`POST /admin/registrations/{uuid}:approve`). `resource_type` in provider capabilities accepts FQN string or Registry UUID. See `schemas/openapi/dcm-admin-api.yaml` for the normative specification.
 
 
diff --git a/content/docs/capabilities-matrix.md b/content/docs/capabilities-matrix.md
index b9f57e6..6244a28 100644
--- a/content/docs/capabilities-matrix.md
+++ b/content/docs/capabilities-matrix.md
@@ -90,7 +90,8 @@
 | PRR-004 | Tenant Metadata Endpoint (GATE-SP-04) | โ€” | Implement GET /api/v1/tenants/{uuid}/metadata returning usage data | Require for standard+ profile activation; enforce quota integration | PRV-001 |
 | PRR-005 | Prometheus Metrics (GATE-SP-05) | โ€” | Expose required metric families at declared metrics_endpoint | Validate metric presence during approval; gate standard+ activation | PRV-001, HLT-005 |
 | PRR-006 | AEP.DEV Linting (GATE-SP-06) | โ€” | Pass AEP linter against OpenAPI spec with no errors before registration; include linting report URL | Gate standard+ activation on linting pass; block activation on errors | PRV-001 |
-| PRR-007 | Multi-Tenant Dispatch (GATE-SP-07) | โ€” | Accept tenant_uuid in all dispatch payloads; return tenant-scoped resources | Gate standard+ activation on multi-tenant compatibility test | PRV-001 |
+
+| PRV-010 | Provider Sandbox/Test Mode | Submit test requests targeting sandbox providers via `_test_context.target_provider_uuid`; sandbox providers visible in registry with `status: sandbox` | Register with `sandbox_mode: true`; implement full OIS contract; graduate to production via standard approval | Manage sandbox provider registry; review graduation requests; sandbox providers excluded from production placement | PRV-001, GATE-SP-01 || PRR-007 | Multi-Tenant Dispatch (GATE-SP-07) | โ€” | Accept tenant_uuid in all dispatch payloads; return tenant-scoped resources | Gate standard+ activation on multi-tenant compatibility test | PRV-001 |
 
 ---
 
@@ -190,7 +191,9 @@
 | OBS-002 | Metrics and Telemetry Export | โ€” | Expose resource-level metrics to DCM | Configure observability export; integrate enterprise observability platform | โ€” |
 | OBS-003 | Curated Event Stream Subscription | Subscribe to observability event types via Message Bus | โ€” | Configure event stream publication policies; manage subscriber roles | OBS-002 |
 | OBS-004 | Alert and Notification Management | Receive resource and policy alerts via declared channels | โ€” | Configure alert routing; manage notification channels and escalation | OBS-001 |
-| OBS-005 | Cost Analysis and Attribution | View cost estimates and actuals for owned resources | Provide cost metadata; report utilization | Configure Cost Analysis component; manage cost attribution policies | PRV-006 |
+
+| OBS-006 | SLA/SLO Declaration | View SLO status for owned resources (`GET /resources/{uuid}/slo-status`) | Declare resource_type SLOs in Resource Type Specification; report realization timing via callbacks | Configure SLO targets per resource type; view aggregate SLO performance report (`GET /admin/slo/report`) | RLM-001, LCM-001 |
+| OBS-007 | SLO Breach Detection and Notification | Receive `slo.breach_approaching` and `slo.breach_detected` events | โ€” | Configure SLO breach routing and escalation; review aggregate breach reports | OBS-006, EVT-001 || OBS-005 | Cost Analysis and Attribution | View cost estimates and actuals for owned resources | Provide cost metadata; report utilization | Configure Cost Analysis component; manage cost attribution policies | PRV-006 |
 
 ---
 
@@ -203,7 +206,10 @@
 | STO-003 | Discovered State Store Management | โ€” | โ€” | Configure Discovered Store; manage retention policies per profile | DRF-001 |
 | STO-004 | Search Index Management | Use entity and catalog search | โ€” | Configure Search Index; manage rebuild on failure | STO-001 |
 | STO-005 | Backup and Recovery | โ€” | โ€” | Configure backup schedules; test recovery procedures | STO-001, STO-002 |
-| STO-006 | Provenance Model Configuration | โ€” | โ€” | Select and configure provenance model (full_inline / deduplicated / tiered); manage tier transitions | STO-001 |
+
+| STO-007 | Cross-Region Sovereignty-Aware Replication | โ€” | Declare replication capabilities and sovereignty constraints at registration; honor replication routing decisions | Configure replication topology; monitor replication lag; respond to `storage.replication_lag_exceeded` events | STO-001, GOV-001 |
+| STO-008 | Tenant-Scoped Storage Isolation | Data is isolated by tenant at storage layer (RLS + separate stream/namespace) | โ€” | Configure isolation strategy per profile; enforce STI-001 through STI-004 policies | STO-001, IAM-001 |
+| STO-009 | Tenant-Scoped Encryption (fsi/sovereign) | โ€” | โ€” | Configure per-tenant AES-256-GCM encryption keys via Credential Provider; manage key rotation schedule (P90D fsi / P30D sovereign) | STO-008, CPR-001 || STO-006 | Provenance Model Configuration | โ€” | โ€” | Select and configure provenance model (full_inline / deduplicated / tiered); manage tier transitions | STO-001 |
 
 ---
 
@@ -229,7 +235,8 @@
 | GOV-004 | Resource Type Lifecycle | โ€” | Manage deprecation notices; declare successor types; maintain migration guidance | Enforce deprecation timelines; manage sunset periods | GOV-003 |
 | GOV-005 | Platform Configuration Management | โ€” | โ€” | Manage platform-wide layers; configure profiles; manage deployment manifest | LAY-001, POL-005 |
 | GOV-006 | Bootstrap and Self-Hosting | โ€” | โ€” | Manage DCM self-deployment; verify bootstrap manifest; handle repave scenarios | STO-001 |
-| GOV-007 | Sovereign Deployment Management | โ€” | โ€” | Manage air-gapped DCM instances; configure signed bundle import; manage offline registry | FED-001, STO-001 |
+
+| GOV-008 | Tenant Onboarding Workflow | Trigger onboarding completion: receive `tenant.onboarding_complete` when first entity OPERATIONAL | โ€” | Execute full provisioning sequence: tenant entity, default groups, quota, admin actor, GitOps namespace, audit stream; dispatch `tenant.created` and member invitation events | IAM-001, STO-008 || GOV-007 | Sovereign Deployment Management | โ€” | โ€” | Manage air-gapped DCM instances; configure signed bundle import; manage offline registry | FED-001, STO-001 |
 
 ---
 
@@ -255,7 +262,10 @@
 | ZTS-003 | Certificate Rotation Management | โ€” | Implement certificate rotation before expiry; use transition window to avoid downtime | Monitor certificate expiry; fire P14D rotation warnings; manage P7D transition window | ZTS-001 |
 | ZTS-004 | Zero Trust Posture Configuration | โ€” | โ€” | Configure zero_trust_posture per profile (none/boundary/full/hardware_attested); manage posture overrides | POL-005 |
 | ZTS-005 | Hardware Attestation (Sovereign Profile) | โ€” | Present hardware-attested identity (TPM/HSM) for sovereign profile interactions | Configure hardware attestation requirements; manage HSM integration; enforce for sovereign profile | ZTS-001, ZTS-002 |
-| ZTS-006 | Five-Check Boundary Enforcement | โ€” | Pass all five boundary checks on every interaction: identity โ†’ authorization โ†’ accreditation โ†’ matrix โ†’ sovereignty | Monitor boundary check audit records; respond to INTERACTION_DENIED events | ZTS-001, ACC-001, GMX-001 |
+
+| ZTS-007 | Provider OpenAPI Spec Signing (SEC-001) | โ€” | Sign OpenAPI spec with mTLS private key at registration; rejected at GATE-SP-01 if unsigned | Verify signature during registration approval pipeline | PRV-001, ZTS-001 |
+| ZTS-008 | GitOps Secrets Scanning (SEC-002) | Commits with detected secrets rejected with `SECRETS_DETECTED` audit record | Ensure service layer SCM does not contain plaintext secrets | Configure scanning ruleset; review and remediate detected secrets | GOV-001, AUD-001 |
+| ZTS-009 | Software Bill of Materials (SBOM) Declaration (SEC-003) | โ€” | Declare SBOM reference at registration (mandatory for fsi/sovereign) | Enforce SBOM requirement during registration approval for fsi/sovereign profiles | PRV-001, ACR-001 || ZTS-006 | Five-Check Boundary Enforcement | โ€” | Pass all five boundary checks on every interaction: identity โ†’ authorization โ†’ accreditation โ†’ matrix โ†’ sovereignty | Monitor boundary check audit records; respond to INTERACTION_DENIED events | ZTS-001, ACC-001, GMX-001 |
 
 ---
 
@@ -572,6 +582,21 @@
 ---
 
 
+## 38. Location Topology Management
+
+| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On |
+|----|-----------|---------|---------|---------------|-----------|| LOC-001 | Location Type Registry | Browse available location types (standard and custom) | Declare supported locations in provider registration capability declaration | Register custom location types; manage standard type definitions; deprecate types | PRV-001 |
+| LOC-002 | Location Node Management | Browse available locations via `GET /api/v1/locations`; filter by resource type, data classification, sovereignty zone | Declare which location nodes (DC, Zone, etc.) the provider serves at registration | Create, version, and retire location layer instances via GitOps; update mutable capacity fields | LOC-001 |
+| LOC-003 | Location Selection at Request Time | Submit `location_uuid` or `location_handle` with service request; select at any level (Country through Rack); DCM refines to specific DC at placement | โ€” | Configure default location selection rules; enforce location-based placement policies | LOC-001, LOC-002 |
+| LOC-004 | Location Layer Assembly | Transparent โ€” full location context injected into payload automatically | Receive full location context in dispatch payload (location.country_code, location.zone_code, location.dc_code, etc.) | Configure layer assembly order; define location-level field overrides | LOC-002, DLM-001 |
+| LOC-005 | Location-Based Sovereignty Enforcement | See sovereignty zone and data residency on each location node | Declare sovereignty capabilities per served location | Configure `max_data_classification` per location; enforce cross-border policies at location layer | LOC-002, GOV-001 |
+| LOC-006 | Location Hierarchy Navigation | Browse parent/child location relationships; query ancestors of a selected node | โ€” | Manage location hierarchy; validate acyclicity on layer submission | LOC-001, LOC-002 |
+| LOC-007 | Custom Location Types | Use custom location types in selection (e.g., Fleet/Ship in Navy context) | Declare support for resources at custom location types | Register and manage custom types; define level insertion point in hierarchy | LOC-001 |
+| LOC-008 | Location Capacity Visibility | See `capacity_status` (available/limited/full) and `providers_available` count per location | Report capacity scoped to location during reserve query | Update mutable capacity fields (e.g., rack_units_available) without a new layer version | LOC-002, PRV-001 |
+
+---
+
+
 ## Capability Count Summary
 
 | Domain | Capabilities |
@@ -700,13 +725,76 @@
 | Scheduled and Deferred Requests | 6 |
 | Request Dependency Graph | 6 |
 | DCM Self-Health | 6 |
+| Identity and Access Management | 21 |
+| Service Catalog | 7 |
+| Request Lifecycle Management | 10 |
+| Provider Contract and Realization | 16 |
+| Resource Lifecycle Management | 7 |
+| Drift Detection and Remediation | 5 |
+| Policy Management | 7 |
+| Data Layer Management | 5 |
+| Information and Data Integration | 6 |
+| Ingestion and Brownfield Management | 4 |
+| Audit and Compliance | 5 |
+| Observability and Operations | 5 |
+| Storage and State Management | 6 |
+| DCM Federation and Multi-Instance | 5 |
+| Platform Governance and Administration | 7 |
+| Accreditation Management | 6 |
+| Zero Trust and Security Posture | 6 |
+| Unified Governance Matrix | 7 |
+| Drift Reconciliation | 5 |
+| Federated Contribution Model | 7 |
+| Scoring Model | 10 |
+| Meta Provider Composability | 8 |
+| Credential Provider Model | 12 |
+| Authority Tier Model | 12 |
+| Event Catalog | 7 |
+| API Versioning | 8 |
+| Session Revocation | 11 |
+| Internal Component Authentication | 8 |
+| Scheduled and Deferred Requests | 6 |
+| Request Dependency Graph | 6 |
+| DCM Self-Health | 6 |
+| Identity and Access Management | 21 |
+| Service Catalog | 7 |
+| Request Lifecycle Management | 10 |
+| Provider Contract and Realization | 16 |
+| Resource Lifecycle Management | 7 |
+| Drift Detection and Remediation | 5 |
+| Policy Management | 7 |
+| Data Layer Management | 5 |
+| Information and Data Integration | 6 |
+| Ingestion and Brownfield Management | 4 |
+| Audit and Compliance | 5 |
+| Observability and Operations | 6 |
+| Storage and State Management | 8 |
+| DCM Federation and Multi-Instance | 5 |
+| Platform Governance and Administration | 7 |
+| Accreditation Management | 6 |
+| Zero Trust and Security Posture | 8 |
+| Unified Governance Matrix | 7 |
+| Drift Reconciliation | 5 |
+| Federated Contribution Model | 7 |
+| Scoring Model | 10 |
+| Meta Provider Composability | 8 |
+| Credential Provider Model | 12 |
+| Authority Tier Model | 12 |
+| Event Catalog | 7 |
+| API Versioning | 8 |
+| Session Revocation | 11 |
+| Internal Component Authentication | 8 |
+| Scheduled and Deferred Requests | 6 |
+| Request Dependency Graph | 6 |
+| DCM Self-Health | 6 |
 | Operational Reference | 4 |
 | Web Interfaces | 14 |
 | ITSM Integration | 7 |
 | Provider Callback Authentication | 10 |
 | Workload Analysis | 5 |
 | Accreditation Monitoring | 6 |
-| **Total** | **287** |
+| Location Topology Management | 7 |
+| **Total** | **299** |
 
 ---
 
diff --git a/content/docs/data-model/03-layering-and-versioning.md b/content/docs/data-model/03-layering-and-versioning.md
index 97fe0e0..5366339 100644
--- a/content/docs/data-model/03-layering-and-versioning.md
+++ b/content/docs/data-model/03-layering-and-versioning.md
@@ -324,6 +324,12 @@ DCM defines six layer types. Each has a distinct purpose, scope, ownership model
 - Geographic region layer
 - Environment layer (production, staging, development)
 
+> **Location Topology:** Location layers are one application of the Reference Data
+> Layer pattern (Section 3.7). The standard schema for each location level โ€”
+> Country, Region, Zone, Site, Data Center, Hall, Cage, and Rack โ€” is specified
+> in [Location Topology Layer Model](48-location-topology-layers.md), including
+> field definitions, priority bands, authority model, and hierarchy assembly.
+
 ---
 
 ### 3.3 Intermediate / Customization Layers
@@ -429,6 +435,142 @@ type_scope:
 
 ---
 
+### 3.7 Reference Data Layers
+
+**All of DCM's data โ€” including Resource Type Specifications โ€” is built on the same
+layer model.** A Resource Type Specification is itself a data layer artifact: versioned,
+owned by a declared Resource Type Authority, stored in GitOps, subject to the standard
+lifecycle, and subject to the same domain-based access control as all other layers. The
+only distinction is that Resource Type Specifications live in the Resource Type Registry
+(which is itself a specialized layer store) rather than the Core Layer Store.
+
+This unified model means:
+- The same governance tooling manages resource type definitions, reference data, and
+  service configuration
+- The same lifecycle (developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired) applies
+  to all three
+- The same ownership declarations, the same GitOps workflow, the same authority tiers
+- Provider extension layers (domain: `provider`) extend resource types for a specific
+  catalog item without modifying the Resource Type Specification itself โ€” they are
+  injected during payload assembly only when that catalog item is selected
+
+**Purpose:** Provide governed, versioned sets of allowed values for use as field
+constraints in Resource Type Specifications and Provider Catalog Items. Reference Data
+Layers are the source of truth for any enumerated choice a consumer makes when
+requesting a resource.
+
+**Scope:** Declared per layer type. A Reference Data Layer of type `os_image` is only
+valid as a constraint source for fields that declare `layer_type: os_image`. Reference
+Data Layers are not injected into the assembled request payload in the same way as other
+layers โ€” they are resolved at catalog render time to produce the `allowed_values` list
+for a field constraint.
+
+**Ownership:** The team or authority responsible for governing that category of data.
+
+| Layer Type | Owning Authority | What it governs |
+|------------|-----------------|-----------------|
+| `location.*` | Data Center Operations | Where resources can be placed |
+| `os_image` | Platform Security / OS Team | Approved OS images and versions |
+| `vm_size` | Platform Team | Approved VM size profiles |
+| `network_zone` | Network Operations | Available network zones and their properties |
+| `environment` | Platform Governance | Deployment environments and their policy sets |
+| `storage_class` | Storage Operations | Available storage tiers |
+| `gpu_profile` | Platform Team | Approved GPU configurations |
+
+**Characteristics:**
+- Same lifecycle as all other layers: `developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired`
+- Same governance: GitOps workflow, owned by declared authority, approved by designated tier
+- Same versioning: `Major.Minor.Revision` โ€” breaking changes require a major bump
+- Same security: domain-based access control, same policy enforcement
+- **Not merged into the request payload directly** โ€” they are resolved at catalog render time
+  to produce the field constraint `allowed_values` list
+- **Their structured data IS injected** when a consumer selects a value: selecting a
+  location layer injects all location context; selecting an OS image layer injects
+  image UUID, SHA, version, and EOL date into the payload
+
+**Adding a new allowed value** for any layer-referenced field means adding a new
+Reference Data Layer instance of the appropriate type. No changes to the Resource Type
+Specification, the catalog item, or any policy. The new value becomes available to
+all catalog items that reference that layer type on the next sync cycle.
+
+**Retiring an allowed value** means retiring the Reference Data Layer instance.
+Existing resources that used that value are unaffected. Future requests cannot
+select it.
+
+**Example โ€” OS Image layer:**
+
+```yaml
+layer:
+  artifact_metadata:
+    uuid: 
+    handle: "platform/os-images/rhel-9-4-approved"
+    version: "1.0.0"
+    status: active
+    owned_by:
+      display_name: "Platform Security Team"
+      group_handle: "groups/platform-security"
+    created_via: pr
+
+  layer_type: reference_data
+  reference_data_type: os_image
+  scope: type_agnostic
+
+  priority:
+    value: "150.01.0"
+    label: "platform.reference.os-image.rhel-9-4"
+    category: platform_reference
+
+  data:
+    image_name: "RHEL 9.4"
+    image_uuid: "img-rhel-9-4-20260315"
+    image_sha256: "a1b2c3d4..."
+    os_family: rhel
+    major_version: 9
+    minor_version: 4
+    release_date: "2026-03-15"
+    eol_date: "2032-05-31"
+    approved_for_classifications: [public, internal, confidential, restricted]
+    cis_benchmark_version: "CIS RHEL 9 Benchmark v1.0"
+    fips_compliant: true
+
+  concern_tags: [os-image, rhel, approved, fips-compliant]
+```
+
+**Example โ€” VM Size layer:**
+
+```yaml
+layer:
+  artifact_metadata:
+    uuid: 
+    handle: "platform/vm-sizes/medium-general-purpose"
+    version: "2.1.0"
+    status: active
+    owned_by:
+      display_name: "Platform Team"
+      group_handle: "groups/platform-team"
+    created_via: pr
+
+  layer_type: reference_data
+  reference_data_type: vm_size
+  scope: type_agnostic   # applies across providers for Compute.VirtualMachine
+
+  data:
+    size_name: "Medium โ€” General Purpose"
+    size_code: "gp-medium"
+    display_name: "Medium (8 CPU / 32 GB)"
+    cpu_count: 8
+    memory_gb: 32
+    storage_gb: 80
+    network_bandwidth_gbps: 10
+    approved_for_workloads: [web, application, database, batch]
+    cost_tier: standard
+
+  concern_tags: [vm-size, general-purpose, approved]
+```
+
+---
+
+
 ## 4. Layer Identity โ€” Domain, Handle, and Priority
 
 Every layer has a formal identity model with three components that together make it uniquely identifiable, locatable, and orderable within DCM.
@@ -446,7 +588,15 @@ The **Layer Domain** mirrors the Policy domain model exactly. It declares owners
 | `provider` | Provider Catalog Item layers | Provider owner | Nothing above provider |
 | `request` | Consumer-declared values in the request itself | Consumer | Nothing above request โ€” lowest authority |
 
-A lower-domain layer cannot override a higher-domain layer. A `tenant` layer cannot override a `platform` layer. This is enforced at ingestion โ€” the conflict detection pipeline checks domain authority before allowing a merge.
+A lower-domain layer cannot override a higher-domain layer.
+
+**Provider extension layers** (`domain: provider`) are contributed by Service Providers
+as part of their catalog item registration. They add provider-specific fields to the
+assembled payload for their offering only. They cannot override platform or tenant layers.
+They carry the same portability implications as inline `provider_specific_extensions` in
+the catalog item declaration โ€” any request using them is non-portable and must be
+explicitly marked. See [Resource Type Hierarchy](05-resource-type-hierarchy.md) Section 6.2
+for the catalog item `provider_extension_layer_handles` declaration. A `tenant` layer cannot override a `platform` layer. This is enforced at ingestion โ€” the conflict detection pipeline checks domain authority before allowing a merge.
 
 **Domain mirrors policy authority:** Just as system-domain policies have highest authority in the Policy Engine, system-domain layers have highest authority in the assembly process. The same mental model applies to both.
 
diff --git a/content/docs/data-model/05-resource-type-hierarchy.md b/content/docs/data-model/05-resource-type-hierarchy.md
index 2ffc9cb..7ee870c 100644
--- a/content/docs/data-model/05-resource-type-hierarchy.md
+++ b/content/docs/data-model/05-resource-type-hierarchy.md
@@ -55,6 +55,50 @@ These terms are frequently conflated. The distinction is architectural:
 DCM maintains an official **Resource Type Registry** โ€” the authoritative source of standard resource type definitions. The registry is the foundation of portability across the DCM ecosystem.
 
 
+
+### 2.1c Resource Type Authority โ€” Stewardship Model
+
+Every Resource Type Specification in the DCM registry is owned by a **Resource Type
+Authority** โ€” the team or individual responsible for defining, maintaining, evolving,
+and deprecating that specification. This is not an implicit role; it is a declared
+field in the registry entry.
+
+The Resource Type Authority has full responsibility for:
+- Defining the universal fields (the vendor-neutral contract all providers must implement)
+- Deciding which fields are `conditional` vs `universal`
+- Declaring `layer_reference` constraints where the allowed values should be
+  governed by a named layer type rather than a static list
+- Reviewing provider extension proposals that add fields to their resource type
+- Publishing new versions when the contract changes
+- Deprecating the specification and declaring a replacement when it's superseded
+
+**Authority assignment by registry tier:**
+
+| Tier | Who is the Resource Type Authority |
+|------|------------------------------------|
+| Tier 1 โ€” DCM Core | DCM Project maintainers (community PRs + named maintainer approval) |
+| Tier 2 โ€” Verified Community | Named community maintainer(s) declared at registry entry time |
+| Tier 3 โ€” Organization | Designated platform team, domain team, or SME group per the organization's governance model |
+
+**Within Tier 3 (Organization), typical authority assignments:**
+
+| Resource Type Category | Typical Owning Authority |
+|-----------------------|--------------------------|
+| `Compute.*` | Platform / Virtualization Team |
+| `Network.*` | Network Operations |
+| `Storage.*` | Storage Operations |
+| `Security.*` | Security / CISO Office |
+| `Platform.*` | Platform Engineering |
+| `Process.*` | Automation / DevOps Platform Team |
+| `Application.*` | Application Platform Team |
+
+**The authority model is the same as all other DCM artifacts.** Resource Type
+Specifications are versioned, GitOps-managed, authority-owned, and subject to
+the standard `developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired` lifecycle.
+The Resource Type Authority is the approver in the GitOps workflow โ€” the PR must
+be approved by the authority before the specification activates.
+
+
 ### 2.1a Catalog Item vs Resource Type Specification โ€” Critical Distinction
 
 These two terms are frequently conflated throughout the documentation. They are distinct concepts at different levels of the hierarchy:
@@ -167,6 +211,108 @@ The broadest classification. Defines the domain of a resource without any specif
 
 ---
 
+
+### 2.1b Layer-Referenced Field Constraints
+
+Fields in a Resource Type Specification can declare their allowed values as **a set of
+active layer instances** rather than as a static list. This is the `layer_reference`
+constraint type.
+
+**Why this matters:**
+
+A static `enum: [rhel, ubuntu, windows-server]` is valid at definition time but
+immediately becomes a governance problem โ€” updating it requires a new version of the
+Resource Type Specification, goes through the full registry approval process, and
+affects all providers simultaneously.
+
+A `layer_reference` constraint delegates allowed-value governance to the layer system.
+The set of valid values for a field is determined at catalog item render time by
+querying the active layer instances of the declared layer type. Adding a new approved
+OS image is adding a new OS Image layer โ€” owned by the appropriate authority, versioned,
+subject to GitOps workflow, immediately available to all catalog items that reference
+that layer type, without touching the Resource Type Specification.
+
+**Examples of layer-referenced fields:**
+
+| Field | Layer Type | Who creates layers | What a layer contains |
+|-------|-----------|-------------------|----------------------|
+| `location` | `location.data_center` | Data Center Operations | DC name, code, certifications, sovereignty, power, network |
+| `os_image` | `os_image` | Platform Security / OS Team | image name, version, SHA, approved status, EOL date |
+| `size` (optional) | `vm_size` | Platform Team | size name, CPU, RAM, storage defaults โ€” **only if the org wants to constrain sizes via governed list**; organizations may instead let providers declare ad-hoc size constraints in their catalog items |
+| `network_zone` | `network_zone` | Network Operations | zone name, VLAN range, allowed protocols, firewall rules |
+| `environment` | `environment` | Platform Governance | environment name, policy set, TTL defaults, approval tier |
+| `storage_class` | `storage_class` | Storage Operations | class name, IOPS, throughput, redundancy, cost per GB |
+
+**The pattern applies to any field where:**
+- The valid values are governed by a specific team or authority
+- The set of values changes over time (new options added, old ones retired)
+- Each value carries structured metadata beyond just its name
+- Governance, versioning, and audit of the allowed set matters
+
+**The pattern does NOT apply to fields where:**
+- The valid values are intrinsic to the resource type itself (CPU range, memory range)
+- The provider is the appropriate authority for what values are valid for their offering
+- The field is informational (names, descriptions, tags)
+
+**This is an organizational decision.** The Resource Type Specification declares whether
+a field uses `layer_reference` or a static constraint. Organizations can choose either
+approach. A VM `size` field could use `layer_reference` if the organization wants to
+maintain a governed size catalog โ€” or it could use a static `range` constraint and let
+each provider declare their available sizes in their Catalog Item. Both are valid.
+Neither requires the other. The Resource Type Authority decides per field based on
+whether organizational governance of the allowed values adds value.
+
+**What the consumer sees:**
+
+When `GET /api/v1/catalog/{uuid}` renders a field with a `layer_reference` constraint,
+DCM resolves the active layer instances of that type and returns them as the
+`allowed_values` list โ€” each entry containing both the value to submit and the
+display data the GUI needs to render the selection:
+
+```json
+{
+  "field_name": "location",
+  "type": "string",
+  "required": true,
+  "constraint": {
+    "type": "layer_reference",
+    "layer_type": "location.data_center",
+    "allowed_values": [
+      {
+        "value": "layer-uuid-fra-dc1",
+        "display_name": "DC1 โ€” Frankfurt Alpha",
+        "code": "FRA-DC1",
+        "zone": "eu-west-1a",
+        "sovereignty": "EU/GDPR",
+        "certifications": ["ISO 27001", "SOC 2 Type II"],
+        "capacity_status": "available"
+      },
+      {
+        "value": "layer-uuid-ams-dc2",
+        "display_name": "DC2 โ€” Amsterdam Beta",
+        "code": "AMS-DC2",
+        "zone": "eu-west-1b",
+        "sovereignty": "EU/GDPR",
+        "certifications": ["ISO 27001"],
+        "capacity_status": "limited"
+      }
+    ]
+  }
+}
+```
+
+The consumer submits the layer UUID as the field value. DCM resolves it to the full
+layer, assembles the layer chain into the payload, and the provider receives the
+complete structured location context โ€” not just a string.
+
+**Governance model:** Layer instances for any layer type follow the same lifecycle,
+controls, security, and governance as Resource Types themselves โ€” they are versioned
+artifacts, owned by a declared authority, stored in GitOps, subject to the standard
+`developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired` lifecycle. The authority
+that creates and governs the layer instances is the same authority that governs
+what values are valid for that field.
+
+
 ### Level 2 โ€” Resource Type
 
 Defines an abstract resource within a category. A Resource Type represents a class of resource that multiple providers can implement. Resource Types are the primary unit of portability in DCM.
@@ -183,6 +329,12 @@ Defines an abstract resource within a category. A Resource Type represents a cla
 
 ### Level 3 โ€” Resource Type Specification
 
+The data contract for a Resource Type. A Resource Type Specification is itself
+a **data layer artifact** โ€” it follows the same versioning, ownership, lifecycle,
+GitOps governance, and domain model as all other DCM layers. The Resource Type
+Authority is the `owned_by` declaration on the specification artifact. Changes
+produce new versions. The specification is immutable once active.
+
 The data contract for a Resource Type. Defines all fields โ€” universal, conditional, and any declared extension points โ€” along with their types, constraints, and portability classifications.
 
 - Every field in a specification carries a **portability classification** (see Section 4)
@@ -212,6 +364,104 @@ A specific provider's concrete implementation of a Resource Type Specification.
 
 ---
 
+
+## 3a. Field Constraint Model โ€” Three Choices
+
+When a Resource Type Authority defines a field, they make a deliberate decision
+about how that field's valid values are governed. There are three options:
+
+### Option 1 โ€” Layer-Referenced Constraint
+
+```yaml
+field_name: location
+constraint:
+  type: layer_reference
+  layer_type: location.data_center
+```
+
+**Use when:** The valid values are a governed, versioned, authority-owned set that
+changes over time and carries structured metadata. Location, OS images, network zones,
+storage classes, and approved size profiles are all layer-referenced by default in DCM.
+
+**Portability:** Fields with `layer_reference` constraints are **fully portable** โ€” any
+provider implementing this resource type resolves the same layer type to its own
+available instances. The consumer submits a layer UUID; each provider resolves it
+against their own registered location or OS image layers.
+
+**Governance:** Adding a new valid value = adding a new Reference Data Layer instance.
+No Resource Type Specification change needed. The authority that owns the layer type
+governs the set of valid values independently of the Resource Type Authority.
+
+---
+
+### Option 2 โ€” Provider-Declared Constraint (ad-hoc)
+
+```yaml
+field_name: cpu_count
+constraint:
+  type: enum
+  allowed_values: [1, 2, 4, 8, 16, 32]
+  reason: "Powers of 2 required for NUMA alignment"
+```
+
+**Use when:** The valid values are inherent to the resource type itself and do not
+change based on what an organization has provisioned or approved. CPU counts, memory
+ranges, protocol versions, and other intrinsic technical constraints belong here.
+
+**When providers use it at catalog item level:** A provider can declare ad-hoc
+constraints in their Catalog Item declaration โ€” restricting or narrowing the values
+declared in the Resource Type Specification for *their specific offering*. A provider
+may not offer all CPU counts the spec allows; their catalog item declares the subset
+they support. This is valid and does not affect portability as long as they stay within
+the bounds declared by the Resource Type Specification.
+
+**Portability:** Depends on how the constraint is used:
+- In the Resource Type Specification: portable if values are vendor-neutral
+- In the Catalog Item: provider narrows the spec's values โ€” still portable if
+  another provider's catalog item supports the same field with overlapping values
+
+---
+
+### Option 3 โ€” No Constraint (Provider Judgment)
+
+```yaml
+field_name: display_name
+constraint:
+  type: pattern
+  pattern: '^[a-z0-9-]{3,63}$'
+  # Or simply: no constraint block โ€” free-form
+```
+
+**Use when:** The field is informational, free-form, or the provider is the
+appropriate authority for what constitutes a valid value for their offering.
+Names, descriptions, tags, and provider-internal identifiers belong here.
+
+**Portability:** No constraint means any value is valid โ€” fully portable but
+with no guarantee of behavioral equivalence across providers.
+
+---
+
+### Decision Guide for Resource Type Authorities
+
+| Field characteristic | Recommended constraint type |
+|---------------------|----------------------------|
+| Value is from a governed, versioned, org-managed list | `layer_reference` |
+| Value determines which physical infrastructure is used | `layer_reference` |
+| Value is intrinsic to the resource type (CPU count, protocol) | `enum` or `range` |
+| Provider narrows a spec-defined range in their catalog item | `enum` or `range` at catalog item level |
+| Value is informational / naming / description | `pattern` or no constraint |
+| Value is entirely provider-internal | No constraint in spec; provider declares in catalog item |
+
+**An organization decides per field** โ€” some fields in a resource type will be
+layer-referenced (location, OS image), some will have ad-hoc constraints (CPU range),
+and some will be free-form (display name). The Resource Type Authority makes these
+decisions when publishing the specification. Organizations can always add more
+governance later by adding a `layer_reference` constraint to a field that previously
+used an ad-hoc enum โ€” this is a non-breaking minor version change.
+
+---
+
+
 ## 4. Portability Classification
 
 Every field in every Resource Type Specification carries a portability classification. This classification is part of the field's metadata and is immutable once published for a given version.
@@ -250,7 +500,31 @@ field_name:
     portability_notes: 
     supported_by: 
   constraints:
-    - 
+    - type: 
+      # range: min/max numeric bounds
+      # enum: static list of allowed string values
+      # pattern: regex pattern for string validation
+      # layer_reference: value must be the UUID of an active layer of the declared type
+      #   โ€” used for single-select fields (e.g., pick one location)
+      # layer_reference_list: value is a list of layer UUIDs of the declared type
+      #   โ€” used for multi-select fields (e.g., list of allowed zones)
+
+      # For layer_reference and layer_reference_list:
+      layer_type: 
+      # e.g., layer_type: location.data_center
+      #        layer_type: os_image
+      #        layer_type: approved_size
+      filter:
+        # Optional: restrict which layer instances are valid values
+        # Applied at catalog item render time to produce the available list
+        tags: [, ...]            # only layers with these tags
+        domain:  # only layers from this domain
+        concern_tags: [, ...]    # only layers with these concern tags
+      display_field:      # which layer field to show as display label
+      # e.g., display_field: data.dc_name โ†’ shows "DC1 โ€” Frankfurt Alpha"
+      value_field:        # which layer field is the submitted value
+      # e.g., value_field: artifact_metadata.uuid โ†’ submits the layer UUID
+      #        value_field: data.dc_code           โ†’ submits "FRA-DC1"
   default_value: 
   provenance:
     
@@ -356,9 +630,31 @@ catalog_item:
   conditional_fields_supported:
     
   provider_specific_extensions:
-    
-    
-  portability_warning: 
+    # Provider adds fields beyond the Resource Type Specification.
+    # Each field MUST be marked portability_breaking: true.
+    # The catalog item MUST set portability_warning: true.
+    # The Resource Type Authority MAY review and accept these as
+    # 'conditional' fields in a future version of the specification
+    # if multiple providers adopt the same extension.
+    #
+    # Example:
+    nutanix_acropolis_affinity_group:
+      type: string
+      portability_breaking: true
+      description: "Nutanix-specific affinity group assignment"
+
+  # Provider extension layers โ€” alternative to inline extensions.
+  # Providers may contribute a Service Layer (domain: provider) that adds
+  # fields injected during payload assembly for their offering only.
+  # These layers are registered with DCM alongside the catalog item.
+  # They carry the same portability_breaking: true semantics.
+  provider_extension_layer_handles:
+    - "providers/nutanix-eu-west/layers/acropolis-extensions-v1"
+    # Layer domain: provider โ€” cannot override platform or tenant layers
+    # Applied only when this catalog item is selected for dispatch
+
+  portability_warning: 
+  portability_class: 
 ```
 
 ---
diff --git a/content/docs/data-model/20-registry-governance.md b/content/docs/data-model/20-registry-governance.md
index 98567b6..b5a7c6f 100644
--- a/content/docs/data-model/20-registry-governance.md
+++ b/content/docs/data-model/20-registry-governance.md
@@ -92,8 +92,22 @@ Air-gapped Registry (offline copy)
 
 ### 3.1 The PR-Based Proposal Flow
 
+The PR submitter becomes the **Resource Type Authority** for the submitted specification
+unless an alternative authority is declared in the `owned_by` field. The authority is
+the required approver for all future version PRs against that specification โ€” no version
+of the specification can be activated without the authority's approval (or the authority
+designating a successor via a formal authority transfer PR).
+
 Resource Type proposals follow a GitOps PR-based workflow โ€” not form submissions or tickets. A proposal is a Pull Request against the registry repository.
 
+> **Resource Type Authority:** The PR submitter becomes the **Resource Type Authority**
+> for the specification unless an alternative is declared in the `owned_by` field.
+> The authority is the required approver for all future version PRs โ€” no new version
+> activates without their approval. Authority can be transferred via a formal transfer PR.
+> This is the same `owned_by` governance model applied to all DCM artifacts.
+
+
+
 ```
 1. Author creates Resource Type Specification draft
    โ”œโ”€โ”€ Standard artifact format (uuid, handle, version, status: developing)
diff --git a/content/docs/data-model/28-federated-contribution-model.md b/content/docs/data-model/28-federated-contribution-model.md
index 1ae4c2b..c5e7643 100644
--- a/content/docs/data-model/28-federated-contribution-model.md
+++ b/content/docs/data-model/28-federated-contribution-model.md
@@ -251,6 +251,23 @@ POST /api/v1/contribute/resource-group
 
 ### 6.1 Provider as Resource Type Publisher
 
+**Resource Type Authority vs Service Provider Publisher โ€” the distinction:**
+
+The **Resource Type Authority** (doc 05, Section 2.1c) is the team responsible for
+defining and maintaining the Resource Type Specification โ€” the vendor-neutral contract
+all providers must implement. The authority may be a DCM Project maintainer (Tier 1),
+a named community maintainer (Tier 2), or an organization's domain team (Tier 3).
+
+A **Service Provider** implements that specification in their Catalog Item and publishes
+provider-specific extensions and Service Layers on top of it. A provider is the publisher
+of their catalog item โ€” not necessarily the author of the underlying Resource Type Spec.
+
+In many cases they are the same team: a networking team may both define `Network.VLAN`
+as the Resource Type Authority AND register as the Service Provider that realizes VLANs.
+In other cases they are different: a platform team defines `Compute.VirtualMachine`
+as the Resource Type Authority, and multiple compute providers (Nutanix, VMware, bare
+metal) each independently register Catalog Items implementing that specification.
+
 Service Providers are not just execution targets โ€” they are first-class contributors of the resource type definitions that consumers request. A provider registering a new virtual machine offering publishes the Resource Type Specification, the Catalog Item, and the Service Layer that consumers use to interact with it.
 
 **What this enables:**
diff --git a/content/docs/data-model/48-location-topology-layers.md b/content/docs/data-model/48-location-topology-layers.md
new file mode 100644
index 0000000..7f6018a
--- /dev/null
+++ b/content/docs/data-model/48-location-topology-layers.md
@@ -0,0 +1,962 @@
+# DCM Data Model โ€” Location Topology Layer Model
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Core Data Model Specification
+**Related Documents:** [Data Layers and Assembly](03-layering-and-versioning.md) | [Resource Type Hierarchy](05-resource-type-hierarchy.md) | [Control Plane Components](25-control-plane-components.md) | [Governance Matrix](27-governance-matrix.md) | [Scoring Model](29-scoring-model.md) | [Registry Governance](20-registry-governance.md) | [Service Catalog](05-resource-type-hierarchy.md)
+
+> **AEP Alignment:** API endpoints follow [AEP](https://aep.dev) conventions.
+> See `schemas/openapi/dcm-admin-api.yaml` for the normative admin API specification.
+
+---
+
+## 0. Pattern Context
+
+Location layers are one application of the **Reference Data Layer** pattern (doc 03,
+Section 3.7). The same pattern governs OS images, VM sizes, network zones, storage
+classes, and any other field where valid values are a governed, versioned, authority-owned
+set rather than a static list.
+
+The concepts in this document โ€” layer instances as field constraint sources, hierarchy
+assembly, authority ownership, lifecycle governance โ€” apply equally to all Reference Data
+Layer types. Location is specified in detail here because it has the richest internal
+hierarchy and the most complex assembly behaviour of all the standard reference data types.
+
+**How location layers appear to consumers:**
+
+In a catalog item response, the `location` field has a `layer_reference` constraint of
+type `location.data_center`. The `allowed_values` list is the set of active location
+layer instances the consumer is entitled to and that this resource type is eligible for.
+The consumer submits a layer UUID; DCM assembles the full ancestor chain.
+
+---
+
+## 1. Purpose
+
+Location topology layers are a subset of **Core Layers** (doc 03, Section 3.2) that encode
+where resources exist or may be allocated. They define the physical and logical hierarchy of
+an organization's infrastructure estate โ€” from the broadest geographic unit down to the
+individual rack โ€” and carry the authoritative data that governs resource placement, sovereign
+data routing, cost attribution, and capacity visibility.
+
+Every DCM resource allocation happens *somewhere*. That somewhere is not a string the
+consumer types โ€” it is a structured, versioned, authority-owned layer. When a consumer
+selects a location via the service catalog or API, they are selecting from the registered
+location topology. DCM then assembles the appropriate location layers into the request
+payload, enriching it with all data associated with that location hierarchy.
+
+**What location topology layers answer:**
+- *Where can this resource be placed?* โ€” the consumer selection surface
+- *What data applies at each level of that location?* โ€” layer content injected into the payload
+- *Who is responsible for this location?* โ€” authority and ownership model
+- *What constraints apply at this location?* โ€” sovereignty, compliance, network, power
+- *What capacity is available here?* โ€” provider capacity scoped to this location
+
+---
+
+## 2. Design Principles
+
+**Configurable names, standard types.** The names of topology nodes are organization-defined
+(your organization may call them "Sites" not "Data Centers"). The *types* โ€” the classification
+of each level of the hierarchy โ€” are standardized. Organizations configure instances of
+standard types; they can also add custom type levels where needed.
+
+**Hierarchical and composable.** Location types form a hierarchy from broadest to most
+specific. A resource allocated to a Rack inherits data from its Row, Hall, Data Center, Zone,
+Region, and Country layers โ€” the full ancestry chain is assembled and merged.
+
+**Layer-first, not field-first.** Location is not a single `location: DC1-ZONE-A-RACK-12`
+string. It is a resolved chain of layers. Each layer in the chain carries structured data
+about that location level. The assembled payload contains the full context of the
+selected location.
+
+**Authority-owned.** Each location type has a designated owning authority (the team
+responsible for creating and maintaining instances of that type). The authority model
+is configurable but DCM ships with sensible industry defaults.
+
+**Linked to the service catalog.** Location topology nodes are exposed to consumers
+through the catalog as selection dimensions. When a consumer requests a resource,
+they select from available location nodes โ€” filtered by their entitlements and the
+resource type's placement policies.
+
+---
+
+## 3. Standard Location Type Hierarchy
+
+DCM ships with nine standard location types. The hierarchy is ordered from broadest
+to most specific. Each type has a standard name, a short code for handles and references,
+and a set of standard data fields.
+
+```
+Level 1 โ€” Country (CTY)
+Level 2 โ€” Region (RGN)
+Level 3 โ€” Zone / Availability Zone (AZ)
+Level 4 โ€” Campus / Site (SITE)
+Level 5 โ€” Data Center / Facility (DC)
+Level 6 โ€” Hall / Pod / Row (HALL)      [optional]
+Level 7 โ€” Cage / Enclosure (CAGE)     [optional]
+Level 8 โ€” Rack (RACK)
+Level 9 โ€” Unit / Slot (UNIT)           [optional โ€” typically provider-managed]
+```
+
+Levels marked `[optional]` may be omitted if not relevant to an organization's estate.
+The hierarchy is still valid when levels are skipped โ€” a Rack can be a direct child
+of a Data Center if Halls and Cages are not used.
+
+**Custom types** may be inserted at any level. For example, a Navy context might insert
+`Fleet / Ship` between Region and Data Center. Custom types follow the same format as
+standard types and are registered in the Location Type Registry.
+
+---
+
+## 4. Location Type Definitions
+
+Each standard type has a defined schema of fields. These fields become the data
+carried by location layer instances of that type.
+
+### 4.1 Country (CTY)
+
+```yaml
+location_type: country
+code: CTY
+level: 1
+
+standard_fields:
+  country_name:
+    type: string
+    required: true
+    example: "Germany"
+
+  iso_3166_1_alpha2:
+    type: string
+    pattern: '^[A-Z]{2}$'
+    required: true
+    example: "DE"
+
+  iso_3166_1_alpha3:
+    type: string
+    pattern: '^[A-Z]{3}$'
+    required: true
+    example: "DEU"
+
+  data_sovereignty_jurisdiction:
+    type: string
+    required: true
+    description: "Primary legal jurisdiction for data sovereignty"
+    example: "EU/GDPR"
+
+  regulatory_frameworks:
+    type: array
+    items: string
+    required: false
+    example: [GDPR, NIS2, eIDAS]
+
+  primary_currency:
+    type: string
+    format: ISO-4217
+    required: false
+    example: "EUR"
+
+  utc_offsets:
+    type: array
+    items: string
+    required: false
+    example: ["UTC+1", "UTC+2"]    # CET/CEST
+
+owning_authority_default: Platform Governance Team
+```
+
+### 4.2 Region (RGN)
+
+```yaml
+location_type: region
+code: RGN
+level: 2
+parent_type: country
+
+standard_fields:
+  region_name:
+    type: string
+    required: true
+    example: "EU West"
+
+  region_code:
+    type: string
+    required: true
+    example: "eu-west"
+
+  geographic_bounds:
+    type: object
+    required: false
+    description: "Approximate lat/lon bounding box"
+    properties:
+      lat_min: { type: number }
+      lat_max: { type: number }
+      lon_min: { type: number }
+      lon_max: { type: number }
+
+  primary_interconnect:
+    type: string
+    required: false
+    description: "Primary network interconnect or IXP serving this region"
+    example: "DE-CIX Frankfurt"
+
+  failover_region:
+    type: string
+    format: location-handle
+    required: false
+    description: "Handle of the designated DR failover region"
+    example: "regions/eu-north"
+
+  latency_profile:
+    type: object
+    required: false
+    properties:
+      intra_region_ms: { type: number, example: 2 }
+      to_regions:
+        type: object
+        additionalProperties: { type: number }
+        example: { "eu-north": 15, "us-east": 85 }
+
+owning_authority_default: Network Operations
+```
+
+### 4.3 Zone / Availability Zone (AZ)
+
+```yaml
+location_type: zone
+code: AZ
+level: 3
+parent_type: region
+
+standard_fields:
+  zone_name:
+    type: string
+    required: true
+    example: "EU West Zone A"
+
+  zone_code:
+    type: string
+    required: true
+    example: "eu-west-1a"
+
+  isolation_boundary:
+    type: string
+    enum: [independent_power, independent_cooling, independent_network, full]
+    required: true
+    description: "What failure domains this zone is isolated from"
+
+  high_availability_peer_zones:
+    type: array
+    items: { type: string, format: location-handle }
+    required: false
+    description: "Zones that form an HA pair with this zone"
+    example: ["zones/eu-west-1b", "zones/eu-west-1c"]
+
+  target_rpo_minutes:
+    type: integer
+    required: false
+    description: "Recovery Point Objective for resources in this zone"
+
+  target_rto_minutes:
+    type: integer
+    required: false
+    description: "Recovery Time Objective for resources in this zone"
+
+owning_authority_default: Data Center Operations
+```
+
+### 4.4 Campus / Site (SITE)
+
+```yaml
+location_type: site
+code: SITE
+level: 4
+parent_type: zone
+
+standard_fields:
+  site_name:
+    type: string
+    required: true
+    example: "Frankfurt Campus"
+
+  site_code:
+    type: string
+    required: true
+    example: "FRA-CAMPUS-01"
+
+  physical_address:
+    type: object
+    required: true
+    properties:
+      street:   { type: string }
+      city:     { type: string }
+      postal_code: { type: string }
+      country:  { type: string, format: iso-3166-1-alpha2 }
+
+  owned_or_leased:
+    type: string
+    enum: [owned, leased, colocation, shared]
+    required: true
+
+  security_tier:
+    type: integer
+    enum: [1, 2, 3, 4]
+    required: false
+    description: "Physical security tier (1=highest)"
+
+  noc_contact:
+    type: object
+    required: false
+    properties:
+      email:   { type: string }
+      phone:   { type: string }
+      escalation_url: { type: string }
+
+owning_authority_default: Facilities Management
+```
+
+### 4.5 Data Center / Facility (DC)
+
+```yaml
+location_type: data_center
+code: DC
+level: 5
+parent_type: site
+
+standard_fields:
+  dc_name:
+    type: string
+    required: true
+    example: "DC1 โ€” Frankfurt Alpha"
+
+  dc_code:
+    type: string
+    required: true
+    example: "FRA-DC1"
+
+  tier_classification:
+    type: string
+    enum: [tier_1, tier_2, tier_3, tier_4]
+    required: false
+    description: "Uptime Institute Tier classification"
+
+  power_capacity_kw:
+    type: number
+    required: false
+    description: "Total available power in kilowatts"
+
+  cooling_capacity_kw:
+    type: number
+    required: false
+
+  pue_rating:
+    type: number
+    required: false
+    description: "Power Usage Effectiveness rating (1.0 = perfect)"
+    example: 1.35
+
+  redundancy_model:
+    type: string
+    enum: [N, N+1, 2N, 2N+1]
+    required: false
+    description: "Power and cooling redundancy model"
+
+  network_uplinks:
+    type: array
+    required: false
+    items:
+      type: object
+      properties:
+        carrier: { type: string }
+        bandwidth_gbps: { type: number }
+        redundant: { type: boolean }
+
+  on_site_contact:
+    type: object
+    required: false
+    properties:
+      role:  { type: string }
+      email: { type: string }
+      phone: { type: string }
+
+  dc_operations_team:
+    type: string
+    format: group-handle
+    required: true
+    description: "DCM group owning this data center"
+    example: "groups/dc-operations-fra"
+
+  certifications:
+    type: array
+    required: false
+    items:
+      type: object
+      properties:
+        standard: { type: string, example: "ISO 27001" }
+        valid_until: { type: string, format: date }
+
+owning_authority_default: Data Center Operations
+```
+
+### 4.6 Hall / Pod / Row (HALL) โ€” Optional
+
+```yaml
+location_type: hall
+code: HALL
+level: 6
+parent_type: data_center
+optional: true
+
+standard_fields:
+  hall_name:
+    type: string
+    required: true
+    example: "Hall A โ€” High Density"
+
+  hall_code:
+    type: string
+    required: true
+    example: "FRA-DC1-HALL-A"
+
+  network_segment:
+    type: string
+    required: false
+    description: "Primary network segment / VLAN for this hall"
+
+  power_phase:
+    type: string
+    required: false
+    description: "Power phase distribution (3-phase, single-phase)"
+
+  cooling_type:
+    type: string
+    enum: [air, liquid, rear_door, immersion]
+    required: false
+
+  max_rack_units:
+    type: integer
+    required: false
+    description: "Total rack units available in this hall"
+
+owning_authority_default: Data Center Operations
+```
+
+### 4.7 Cage / Enclosure (CAGE) โ€” Optional
+
+```yaml
+location_type: cage
+code: CAGE
+level: 7
+parent_type: hall
+optional: true
+
+standard_fields:
+  cage_name:
+    type: string
+    required: true
+    example: "Cage 12 โ€” Payments Isolated Zone"
+
+  cage_code:
+    type: string
+    required: true
+    example: "FRA-DC1-HALL-A-CAGE-12"
+
+  tenant_uuid:
+    type: string
+    format: uuid
+    required: false
+    description: "If this cage is dedicated to a specific DCM Tenant"
+
+  security_classification:
+    type: string
+    required: false
+    description: "Physical access classification for this cage"
+    example: "restricted"
+
+  access_control_system:
+    type: string
+    required: false
+    example: "Lenel S2"
+
+owning_authority_default: Data Center Operations
+```
+
+### 4.8 Rack (RACK)
+
+```yaml
+location_type: rack
+code: RACK
+level: 8
+parent_type: cage   # or hall or data_center if cage/hall levels are omitted
+
+standard_fields:
+  rack_name:
+    type: string
+    required: true
+    example: "Rack A-12-03"
+
+  rack_code:
+    type: string
+    required: true
+    example: "FRA-DC1-A-12-03"
+
+  rack_units:
+    type: integer
+    required: true
+    description: "Total rack units (U) capacity"
+    example: 42
+
+  rack_units_available:
+    type: integer
+    required: false
+    description: "Current available rack units (maintained by Data Center Operations)"
+
+  power_circuits:
+    type: array
+    required: false
+    items:
+      type: object
+      properties:
+        circuit_id: { type: string }
+        amperage:   { type: number }
+        phase:      { type: string }
+        redundant:  { type: boolean }
+
+  max_power_kw:
+    type: number
+    required: false
+    description: "Maximum power draw for this rack"
+
+  network_top_of_rack:
+    type: object
+    required: false
+    properties:
+      switch_model:    { type: string }
+      uplink_gbps:     { type: number }
+      port_count:      { type: integer }
+      vlan_range:      { type: string }
+
+  patch_panel_id:
+    type: string
+    required: false
+
+owning_authority_default: Data Center Operations
+```
+
+---
+
+## 5. Location Layer Instance Format
+
+Each location node is a **Core Layer** artifact stored in GitOps and registered in DCM.
+It follows the standard layer format (doc 03) with location-specific fields in its data block.
+
+```yaml
+layer:
+  artifact_metadata:
+    uuid: 
+    handle: "locations/dc/fra-dc1"      # standard handle pattern: locations/{type}/{code}
+    version: "1.2.0"
+    status: active
+    owned_by:
+      display_name: "Data Center Operations โ€” Frankfurt"
+      group_handle: "groups/dc-operations-fra"
+      notification_endpoint: 
+    created_via: pr
+    created_at: 
+
+  # Layer classification
+  layer_type: core                        # always 'core' for location layers
+  location_type: data_center              # the standard type code from Section 4
+  scope: type_agnostic                    # location layers apply to all resource types
+
+  # Priority โ€” location layers occupy a dedicated band in the priority space
+  priority:
+    value: "200.10.0"                     # see Section 7 for priority band allocation
+    label: "core.location.dc.fra-dc1"
+    category: core_location
+    rationale: "Data Center location layer for FRA-DC1"
+
+  # Hierarchy โ€” parent location
+  location_hierarchy:
+    parent_handle: "locations/site/fra-campus-01"
+    parent_type: site
+    ancestors:
+      - handle: "locations/az/eu-west-1a"
+        type: zone
+      - handle: "locations/region/eu-west"
+        type: region
+      - handle: "locations/country/de"
+        type: country
+
+  # The location data โ€” fields from the type definition (Section 4.5)
+  data:
+    dc_name: "DC1 โ€” Frankfurt Alpha"
+    dc_code: "FRA-DC1"
+    tier_classification: tier_3
+    power_capacity_kw: 4000
+    pue_rating: 1.35
+    redundancy_model: "2N"
+    network_uplinks:
+      - carrier: "DE-CIX"
+        bandwidth_gbps: 100
+        redundant: true
+      - carrier: "NTT"
+        bandwidth_gbps: 100
+        redundant: true
+    dc_operations_team: "groups/dc-operations-fra"
+    certifications:
+      - standard: "ISO 27001"
+        valid_until: "2027-06-30"
+      - standard: "SOC 2 Type II"
+        valid_until: "2026-12-31"
+
+  # Sovereignty โ€” consumed directly by the Governance Matrix
+  sovereignty:
+    zone_handle: "zones/eu-west-sovereign"
+    data_residency: EU
+    jurisdiction_codes: [DE]
+    cross_border_permitted: false
+
+  # Placement eligibility โ€” which resource types may be placed here
+  placement:
+    eligible_resource_types: []         # empty = all resource types eligible
+    ineligible_resource_types: []       # explicit exclusions
+    max_data_classification: restricted # highest data classification accepted
+    requires_accreditations: []         # accreditations providers must hold to serve this DC
+
+  concern_tags: [location, data-center, frankfurt, eu-west, tier-3]
+```
+
+---
+
+## 6. Location Hierarchy Assembly
+
+When a consumer selects a location, DCM resolves the full ancestor chain and assembles
+all location layers into the request payload in hierarchy order (lowest precedence first โ€”
+Country โ†’ Region โ†’ Zone โ†’ Site โ†’ Data Center โ†’ Hall โ†’ Cage โ†’ Rack).
+
+**Example: Consumer selects Rack FRA-DC1-A-12-03**
+
+```
+Layer resolution (Core Layer phase of assembly):
+
+  1. Country layer: locations/country/de
+     Injects: iso_3166_1_alpha2=DE, data_sovereignty_jurisdiction=EU/GDPR,
+              regulatory_frameworks=[GDPR, NIS2]
+
+  2. Region layer: locations/region/eu-west
+     Injects: region_code=eu-west, primary_interconnect=DE-CIX,
+              latency_profile.intra_region_ms=2
+
+  3. Zone layer: locations/az/eu-west-1a
+     Injects: zone_code=eu-west-1a, isolation_boundary=full,
+              target_rpo_minutes=15, target_rto_minutes=60
+
+  4. Site layer: locations/site/fra-campus-01
+     Injects: site_code=FRA-CAMPUS-01, physical_address={...},
+              security_tier=2
+
+  5. Data Center layer: locations/dc/fra-dc1
+     Injects: dc_code=FRA-DC1, tier_classification=tier_3,
+              network_uplinks=[...], certifications=[...]
+
+  6. Hall layer: locations/hall/fra-dc1-hall-a
+     Injects: hall_code=FRA-DC1-HALL-A, cooling_type=liquid,
+              network_segment=vlan-100
+
+  7. Rack layer: locations/rack/fra-dc1-a-12-03
+     Injects: rack_code=FRA-DC1-A-12-03, rack_units=42,
+              max_power_kw=20, network_top_of_rack={...}
+
+Assembled location context in payload:
+  location.country_code: DE
+  location.jurisdiction: EU/GDPR
+  location.regulatory_frameworks: [GDPR, NIS2]
+  location.region_code: eu-west
+  location.zone_code: eu-west-1a
+  location.dc_code: FRA-DC1
+  location.rack_code: FRA-DC1-A-12-03
+  location.sovereignty_zone: eu-west-sovereign
+  location.max_data_classification: restricted
+  location.certifications: [ISO 27001, SOC 2 Type II]
+  ... (all ancestor fields available to policies and providers)
+```
+
+Higher-precedence location layers override lower-precedence ones for the same field.
+A Rack layer declaring `max_data_classification: internal` overrides the DC layer's
+`restricted` โ€” the most specific declaration wins.
+
+---
+
+## 7. Priority Band Allocation
+
+Location layers occupy a dedicated band in the Core Layer priority space (doc 03).
+
+```
+Priority bands for Core Location Layers:
+
+  100.xx.0 โ€” Country layers
+  200.xx.0 โ€” Region layers
+  300.xx.0 โ€” Zone / Availability Zone layers
+  400.xx.0 โ€” Site / Campus layers
+  500.xx.0 โ€” Data Center layers
+  600.xx.0 โ€” Hall / Pod / Row layers
+  700.xx.0 โ€” Cage / Enclosure layers
+  800.xx.0 โ€” Rack layers
+  900.xx.0 โ€” Unit / Slot layers (provider-managed)
+
+  xx = sequence number within the level (01, 02, ... 99)
+  (Allows up to 99 instances at each level before a major priority change)
+```
+
+This ensures Country always has lower precedence than Region, which always has lower
+precedence than Zone, etc. The specific location is always the most specific (highest
+precedence) contributor to location data.
+
+---
+
+## 8. Consumer Selection Model
+
+Consumers do not interact with location layers directly via a `/locations` endpoint.
+Location selection is part of the **catalog item field schema**. When a consumer calls
+`GET /api/v1/catalog/{catalog_item_uuid}`, the `location` field constraint of type
+`layer_reference` includes the `allowed_values` list โ€” the set of active location
+Data Center layer instances the consumer is entitled to and that this resource type
+is eligible for.
+
+Each entry in `allowed_values` carries the display data the GUI needs (name, code,
+zone, sovereignty, certifications, capacity status) and the layer UUID the consumer
+submits as the field value.
+
+**Consumer request:**
+
+```json
+POST /api/v1/requests
+{
+  "catalog_item_uuid": "",
+  "fields": {
+    "location": "layer-uuid-fra-dc1",     // DC-level layer UUID
+    "os_image": "layer-uuid-rhel-9-4",
+    "cpu_count": 4
+  }
+}
+```
+
+If the consumer wants to express location at a coarser level (Zone or Region rather
+than a specific DC), they submit the layer UUID of that level. DCM's Placement Engine
+refines downward to a specific DC during placement. The Placement Engine then assembles
+the full ancestor chain (Country โ†’ Region โ†’ Zone โ†’ Site โ†’ DC) into the payload.
+
+**Filtering allowed_values:**
+
+The catalog item declaration controls which location layer instances appear in
+`allowed_values` via the `filter` clause on the `layer_reference` constraint:
+
+```yaml
+# In the catalog item's field constraint declaration:
+constraint:
+  type: layer_reference
+  layer_type: location.data_center
+  filter:
+    tags: [production]              # only production DCs
+    min_tier: tier_3               # only Tier 3 and above
+    required_certifications: [iso_27001]   # only certified DCs
+```
+
+This means the Platform Team controls which DCs are eligible for each catalog item
+by configuring the filter โ€” without changing the location layers themselves.
+
+---
+
+
+## 9. Authority and Ownership Model
+
+Each location type has a designated owning authority. The authority model determines:
+- Who can create, modify, and retire location layer instances
+- Who is notified when location layer data changes
+- Who approves capacity changes
+
+```yaml
+location_authority_model:
+
+  # Standard defaults โ€” configurable per deployment
+  country:
+    creating_authority: Platform Governance Team
+    approval_required: true
+    approval_tier: platform_admin
+
+  region:
+    creating_authority: Network Operations
+    approval_required: true
+    approval_tier: platform_admin
+
+  zone:
+    creating_authority: Data Center Operations
+    approval_required: true
+    approval_tier: platform_admin
+
+  site:
+    creating_authority: Facilities Management
+    approval_required: true
+    approval_tier: team_lead
+
+  data_center:
+    creating_authority: Data Center Operations
+    approval_required: true
+    approval_tier: team_lead
+
+  hall:
+    creating_authority: Data Center Operations
+    approval_required: false
+    approval_tier: operator
+
+  cage:
+    creating_authority: Data Center Operations
+    approval_required: false
+    approval_tier: operator
+
+  rack:
+    creating_authority: Data Center Operations
+    approval_required: false
+    approval_tier: operator
+```
+
+All location layer changes follow the standard GitOps workflow โ€” changes are submitted
+as PRs, reviewed by the owning authority, and merged. DCM picks up changes on the next
+policy/layer sync cycle. Location layers are **immutable once active** โ€” a new version
+is created for any change, preserving the full history of location data over time.
+
+---
+
+## 10. Custom Location Types
+
+Organizations may define custom location types to extend the standard hierarchy. Custom
+types are registered in the Location Type Registry alongside standard types.
+
+**Example: Navy deployment with Fleet and Ship levels**
+
+```yaml
+custom_location_type:
+  type_name: fleet
+  code: FLEET
+  display_name: "Fleet"
+  level: 3.5            # inserted between Zone (3) and Site (4)
+  parent_type: zone
+  child_type: ship       # references the custom 'ship' type below
+
+  standard_fields:
+    fleet_name:      { type: string, required: true }
+    fleet_code:      { type: string, required: true }
+    command_node:    { type: string, required: false }
+    operating_area:  { type: string, required: false }
+
+  owning_authority: Fleet Operations Command
+
+---
+custom_location_type:
+  type_name: ship
+  code: SHIP
+  display_name: "Ship / Vessel"
+  level: 4.5            # inserted between Site (4) and Data Center (5)
+  parent_type: fleet
+  child_type: data_center
+
+  standard_fields:
+    vessel_name:     { type: string, required: true }
+    hull_number:     { type: string, required: true }
+    vessel_class:    { type: string, required: false }
+    home_port:       { type: string, required: false }
+    current_location_lat:  { type: number, required: false }
+    current_location_lon:  { type: number, required: false }
+    connectivity_profile:
+      type: string
+      enum: [satcom, fiber_pier, disconnected]
+      required: true
+
+  owning_authority: Fleet Data Center Operations
+```
+
+Custom type instances are created and managed exactly like standard type instances โ€”
+GitOps PRs, owned by the designated authority, versioned and immutable.
+
+---
+
+## 11. Relationship to Placement Engine
+
+The Placement Engine (doc 25, Section 4) uses location topology data in Steps 1 and 3
+of the six-step placement algorithm:
+
+**Step 1 โ€” Sovereignty Pre-Filter:**
+Location layers carry `sovereignty.zone_handle`. The Placement Engine eliminates any
+provider whose declared sovereignty zones do not include the zone associated with
+the requested location.
+
+**Step 3 โ€” Capability Filter:**
+Location layers carry `placement.max_data_classification` and
+`placement.requires_accreditations`. Providers that cannot satisfy these
+location-level constraints are eliminated, even if they satisfy the global
+accreditation requirements.
+
+**Step 6 โ€” Tie-breaking:**
+When multiple providers qualify, location-level `priority` declarations can be used
+as a tie-breaking preference (e.g., "prefer providers in the same DC over providers
+in a different DC in the same zone").
+
+Location layers also populate `location.*` fields in the assembled payload, which
+Placement policies (doc B) use in their constraint expressions:
+
+```rego
+# Example: Placement policy for PHI data
+placement if {
+    input.payload.location.jurisdiction == "EU/GDPR"
+    input.payload.location.max_data_classification == "restricted"
+    "hipaa_baa" in input.payload.location.required_accreditations
+}
+```
+
+---
+
+## 12. Location Layer Lifecycle
+
+Location layers follow the standard layer lifecycle (doc 03):
+
+```
+developing โ†’ proposed โ†’ active โ†’ deprecated โ†’ retired
+```
+
+**Special considerations for location layers:**
+
+**Decommissioning a location:** When a Data Center is being decommissioned, its location
+layer transitions to `deprecated`. During the deprecation window, the Placement Engine
+stops routing new requests to providers in that DC. Existing resources receive a
+`location.decommission_warning` notification. The layer transitions to `retired` when
+all resources have been migrated.
+
+**Location data changes:** When a DC gets a new network uplink or achieves a new
+certification, a new version of the location layer is published (minor version bump).
+The Requested State for all existing resources in that DC is not retroactively updated โ€”
+provenance is preserved. Future requests and re-realizations will pick up the new data.
+
+**Capacity changes:** Rack-level `rack_units_available` is a mutable field โ€” it is
+updated by Data Center Operations as capacity changes without a new version. All other
+location fields require a new version to change.
+
+---
+
+## 13. System Policies
+
+| Policy | Rule |
+|--------|------|
+| `LOC-001` | Every resource entity must have a resolved `location_uuid` at the DC level or below. Requests without a resolvable location are rejected at validation time. |
+| `LOC-002` | Location layers are Core Layers. They must not contain service-specific or provider-specific data. Location layers that include resource-type-scoped fields are invalid. |
+| `LOC-003` | The location hierarchy must be acyclic. A location node cannot be its own ancestor. DCM validates acyclicity at layer submission time. |
+| `LOC-004` | Location layer handles follow the pattern `locations/{type}/{code}`. Any location layer with a non-conforming handle is rejected at registration. |
+| `LOC-005` | When a consumer selects a location at a level above DC (e.g., Zone), the Placement Engine must resolve to a specific DC. A request may not remain at an abstract location level after dispatch. |
+| `LOC-006` | `max_data_classification` declared by a location layer is an upper bound. A request carrying data classified above the location's maximum is rejected by the Placement Engine's capability filter before provider contact. |
+| `LOC-007` | Location layer changes (new versions) are propagated to the Location Type Registry and the Service Catalog location list within the next sync cycle. Consumers see updated location data on next catalog query. |
+| `LOC-008` | Custom location types must declare their level as a decimal between the two standard levels they insert between. Level values must be unique across all registered types (standard and custom). |
+| `LOC-009` | All location layers must declare a `sovereignty.zone_handle` or explicitly declare `sovereignty: not_applicable`. A location layer with no sovereignty declaration is invalid for `standard` and above profiles. |
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/data-model/49-implementation-specifications.md b/content/docs/data-model/49-implementation-specifications.md
new file mode 100644
index 0000000..9adb7b2
--- /dev/null
+++ b/content/docs/data-model/49-implementation-specifications.md
@@ -0,0 +1,728 @@
+# DCM Data Model โ€” Implementation Specifications
+
+**Document Status:** ๐Ÿ“‹ Draft โ€” Ready for Implementation Feedback
+**Document Type:** Implementation Reference
+**Related Documents:** [Control Plane Components](25-control-plane-components.md) | [Storage Providers](11-storage-providers.md) | [Universal Audit](16-universal-audit.md) | [Credential Provider Model](31-credential-provider-model.md) | [Deployment Redundancy](17-deployment-redundancy.md) | [Session Revocation](35-session-revocation.md)
+
+> **AEP Alignment:** API endpoints follow [AEP](https://aep.dev) conventions.
+> See `schemas/openapi/dcm-admin-api.yaml` and `dcm-consumer-api.yaml` for normative specs.
+
+---
+
+## 1. Purpose
+
+This document specifies the implementation mechanics for capabilities that are architecturally defined elsewhere but whose runtime behavior โ€” enforcement location, algorithm, data structure โ€” has not been fully specified. It closes implementation gaps identified in the architecture gap analysis.
+
+---
+
+## 2. Rate Limiting โ€” Enforcement Implementation
+
+Rate limiting is defined at the interface level in the Consumer API Specification (ยง1.6) and the Admin API. This section specifies *how* it is enforced.
+
+### 2.1 Enforcement Location
+
+Rate limiting is enforced by the **API Gateway** component โ€” the single ingress point for all consumer and admin API traffic. It is enforced before the request reaches any pipeline component. The Request Orchestrator never sees rate-limited requests.
+
+Rate limiting is **not** enforced at the network layer (load balancer) or application layer (Request Payload Processor). A single enforcement point at the API Gateway ensures:
+- Consistent limits across all consumer paths (Web UI, direct API, CI/CD)
+- No rate limit bypass via internal component calls
+- Single source of rate limit state for accurate tracking
+
+### 2.2 Token Bucket Algorithm
+
+DCM uses the **token bucket** algorithm with a per-actor bucket:
+
+```
+Actor makes request:
+  โ”‚
+  โ–ผ API Gateway looks up actor_uuid in rate limit store
+  โ”‚   (in-memory cache backed by a fast Storage Provider sub-type)
+  โ”‚
+  โ–ผ Current bucket state:
+  โ”‚   tokens_remaining: 
+  โ”‚   last_refill_at: 
+  โ”‚
+  โ–ผ Refill calculation:
+  โ”‚   elapsed_seconds = now - last_refill_at
+  โ”‚   tokens_to_add = elapsed_seconds ร— (rate_limit / 60)
+  โ”‚   tokens_remaining = min(bucket_max, tokens_remaining + tokens_to_add)
+  โ”‚   last_refill_at = now
+  โ”‚
+  โ–ผ Token check:
+  โ”œโ”€โ”€ tokens_remaining >= 1:
+  โ”‚     tokens_remaining -= 1
+  โ”‚     Request proceeds
+  โ”‚     Response headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
+  โ”‚
+  โ””โ”€โ”€ tokens_remaining < 1:
+        Request rejected: 429 Too Many Requests
+        Response headers: Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
+        Audit record written: rate_limit_exceeded
+```
+
+### 2.3 Bucket Parameters by Profile
+
+| Profile | Rate (req/min) | Burst Max | Bucket Max |
+|---------|---------------|-----------|------------|
+| `minimal` | 60 | 20 | 80 |
+| `dev` | 120 | 40 | 160 |
+| `standard` | 300 | 100 | 400 |
+| `prod` | 600 | 200 | 800 |
+| `fsi` | 600 | 200 | 800 |
+| `sovereign` | 600 | 200 | 800 |
+
+System components (service accounts, provider callbacks) use `prod` bucket parameters regardless of profile.
+
+### 2.4 Rate Limit State Store
+
+The rate limit state is stored in a **dedicated in-memory cache** backed by a fast Storage Provider:
+- Cache TTL: 2ร— the rate limit window (120 seconds for 60 req/min rate)
+- Storage Provider type: key-value (Redis or equivalent)
+- Consistency: eventual โ€” brief over-counting tolerated to avoid distributed lock overhead
+- Cross-replica sharing: rate limit state is shared across all API Gateway replicas via the backing store
+
+### 2.5 Exemptions
+
+The following are exempt from consumer rate limits:
+- Admin API calls (separate rate limit bucket, 3ร— profile limit)
+- Provider callback endpoints (authenticated via provider callback credential; separate per-provider bucket)
+- Internal DCM component calls (authenticated via mTLS + interaction credential; not rate limited)
+- Health check endpoints (`/livez`, `/readyz`, `/metrics`)
+
+### 2.6 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `RLM-001` | Rate limiting is enforced at the API Gateway. No other component enforces rate limits. |
+| `RLM-002` | Rate limit buckets are per authenticated actor (actor_uuid). Unauthenticated requests are rejected at auth before reaching the rate limiter. |
+| `RLM-003` | All rate limit rejections produce an audit record with actor_uuid, endpoint, and timestamp. |
+| `RLM-004` | Rate limit parameters are governed by the active Profile. Operators may increase but not decrease profile-defined limits. |
+| `RLM-005` | Rate limit state is not persisted across API Gateway restarts. Buckets refill from empty after restart โ€” brief over-serving is acceptable. |
+
+---
+
+## 3. Audit Log Hash Chain โ€” Verification Schedule and Implementation
+
+The hash chain structure is defined in [Universal Audit](16-universal-audit.md) ยง8. This section specifies the verification schedule, triggering component, and response protocol.
+
+### 3.1 Hash Computation
+
+Each audit record's `record_hash` is computed as:
+
+```
+record_hash = SHA-256(
+    record_uuid ||
+    record_timestamp ||
+    entity_uuid ||
+    action ||
+    actor.immediate.uuid ||
+    subject_handle ||
+    chain_sequence ||
+    previous_record_hash
+)
+
+Where || denotes canonical concatenation with a field separator (0x1F โ€” ASCII unit separator).
+The hash is stored as a lowercase hex string.
+```
+
+The `previous_record_hash` for the first record in an entity's chain is `SHA-256("GENESIS")` โ€” a known constant, not null.
+
+### 3.2 Verification Schedule
+
+Hash chain verification runs on two schedules:
+
+**Continuous verification (per-write):**
+Every audit record write triggers an immediate verification of that record against its predecessor. This catches chain breaks at write time โ€” before the record is committed. A write that would break the chain is rejected and triggers `audit.chain_integrity_alert`.
+
+**Periodic batch verification (scheduled):**
+The Audit component runs a full-chain verification sweep on a profile-governed schedule:
+
+| Profile | Sweep interval | Scope per sweep |
+|---------|---------------|-----------------|
+| `dev` | P7D | All entities modified in the last 7 days |
+| `standard` | P1D | All entities modified in the last 24 hours |
+| `prod` | PT12H | All entities modified in the last 12 hours |
+| `fsi` | PT6H | All entities + random 5% sample of all-time records |
+| `sovereign` | PT1H | All entities + random 10% sample of all-time records |
+
+### 3.3 Owning Component
+
+Hash chain verification is owned by the **Audit component** โ€” the same component that writes audit records. It is not a separate service. The Audit component runs verification as a background goroutine with no external trigger required.
+
+For Storage Provider implementations: the Audit Store must support ordered range queries by `(entity_uuid, chain_sequence)` to enable efficient sweep verification.
+
+### 3.4 Breach Response Protocol
+
+```
+Chain break detected (during write-time or sweep verification):
+  โ”‚
+  โ–ผ Affected records flagged: integrity_status = chain_break
+  โ”‚   Break point: chain_sequence N where hash mismatch occurs
+  โ”‚   All records with chain_sequence > N for this entity: integrity_status = unverified
+  โ”‚
+  โ–ผ audit.chain_integrity_alert event fired (urgency: critical, non-suppressable)
+  โ”‚   payload: {entity_uuid, entity_type, break_at_sequence, break_detected_at, sweep_type}
+  โ”‚
+  โ–ผ Notifications dispatched:
+  โ”‚   โ†’ Platform Admin (urgency: critical)
+  โ”‚   โ†’ Security team (if configured in notification routing)
+  โ”‚
+  โ–ผ Affected entity flagged in audit dashboard
+  โ”‚   Consumer-visible: "Audit integrity alert โ€” contact platform admin"
+  โ”‚
+  โ””โ”€โ”€ Human investigation required:
+        Normal resolution paths:
+        - Storage Provider failure caused write corruption โ†’ Storage Provider replacement
+        - Clock skew between replicas caused ordering issue โ†’ Non-malicious; document and reseal
+        - Administrative error (direct DB edit) โ†’ Incident report, access review
+        - Malicious tampering โ†’ Security incident declared
+```
+
+### 3.5 Chain Resealing
+
+After a chain break is investigated and root cause documented, a platform admin may reseal the chain:
+
+```
+POST /api/v1/admin/audit/entities/{entity_uuid}:reseal-chain
+  {
+    "investigation_reference": "INC-2026-042",
+    "root_cause": "storage_failure",
+    "resolution_notes": "PostgreSQL WAL corruption during storage migration"
+  }
+
+Response:
+  {
+    "entity_uuid": "",
+    "chain_resealed_at": "",
+    "records_affected": 7,
+    "new_chain_anchor": "",
+    "audit_record_uuid": ""
+  }
+```
+
+The reseal itself produces an audit record that references the investigation. Chain integrity is restored from the reseal point forward.
+
+---
+
+## 4. Multi-Tenancy at the Storage Layer
+
+Tenant isolation in DCM is enforced at the data model level (every entity carries `tenant_uuid`) and at the API level (all consumer endpoints are scoped to the authenticated actor's tenant). This section specifies the storage-layer enforcement mechanisms.
+
+### 4.1 Isolation Strategy by Store Type
+
+| Store Type | Isolation Strategy | Implementation Notes |
+|-----------|-------------------|---------------------|
+| **GitOps Store** (Intent/Requested State) | Directory namespace per tenant | `/tenants/{tenant_uuid}/intents/`, `/tenants/{tenant_uuid}/requests/` โ€” Git ACLs enforce read/write scope |
+| **Event Stream Store** (Audit) | Separate stream per tenant | `dcm.audit.{tenant_uuid}` stream; Kafka topic ACLs restrict producer/consumer access |
+| **Snapshot Store** (Realized State) | Row-level filter + column-level encryption | `tenant_uuid` column indexed; all queries mandatory-include `WHERE tenant_uuid = ?`; tenant-scoped encryption key |
+| **Search Index** | Index namespace per tenant | Separate index prefix `tenant_{uuid}_*`; query routing enforces tenant scope |
+| **Rate Limit Cache** | Key-namespaced per actor (actor carries tenant context) | `rl:{tenant_uuid}:{actor_uuid}` key structure |
+
+### 4.2 Snapshot Store โ€” Row-Level Security Implementation
+
+The Snapshot Store (Realized State) uses row-level security as the primary isolation mechanism:
+
+```sql
+-- PostgreSQL row-level security policy
+CREATE POLICY tenant_isolation ON realized_state_records
+  USING (tenant_uuid = current_setting('dcm.current_tenant_uuid')::uuid);
+
+-- Every connection sets tenant context before any query:
+SET LOCAL dcm.current_tenant_uuid = '';
+
+-- This makes it impossible to query across tenant boundaries,
+-- even with direct database access using the application credential.
+-- Platform admin access uses a separate role without the RLS policy.
+```
+
+### 4.3 Tenant-Scoped Encryption
+
+For `fsi` and `sovereign` profiles, realized state records are encrypted at rest using a per-tenant encryption key:
+
+```
+Tenant provisioned:
+  โ”‚
+  โ–ผ Credential Provider generates tenant encryption key (AES-256-GCM)
+  โ”‚   Key stored in: Credential Provider (e.g., Vault)
+  โ”‚   Key reference stored in: Tenant record as tenant_encryption_key_ref
+  โ”‚
+  โ–ผ On write to Snapshot Store:
+  โ”‚   API Gateway fetches tenant encryption key
+  โ”‚   Payload encrypted with tenant key before storage
+  โ”‚   Storage Provider stores ciphertext only
+  โ”‚
+  โ–ผ On read from Snapshot Store:
+  โ”‚   API Gateway fetches tenant encryption key
+  โ”‚   Decrypts payload in memory
+  โ”‚   Plaintext never written to Storage Provider logs
+  โ”‚
+  โ–ผ Tenant decommission:
+      Tenant encryption key revoked in Credential Provider
+      All tenant data becomes unreadable without external recovery
+      This is the cryptographic equivalent of data deletion
+```
+
+### 4.4 Cross-Tenant Query Prevention
+
+Platform admin endpoints that query across tenants use a separate database role with explicit permission grants โ€” they do not bypass RLS, they use a role that has cross-tenant read permission with full audit logging. The principle is: cross-tenant reads are possible only through intentional, audited, privileged operations.
+
+### 4.5 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `STI-001` | Every query to a tenant-scoped store must include `tenant_uuid` as a mandatory predicate. Queries without tenant scope are rejected by the storage layer. |
+| `STI-002` | Row-level security is enabled on all relational snapshot stores. Disabling RLS requires platform admin action and produces an audit record. |
+| `STI-003` | For `fsi` and `sovereign` profiles, tenant-scoped encryption is mandatory. Key rotation is performed on a profile-governed schedule (P90D for fsi; P30D for sovereign). |
+| `STI-004` | Storage Provider implementations must declare their tenant isolation strategy at registration. DCM validates the declared strategy against the active profile's isolation requirements during the registration approval pipeline. |
+
+---
+
+## 5. Cross-Region Data Replication
+
+DCM's multi-region deployment model is specified in [Deployment Redundancy](17-deployment-redundancy.md). This section specifies the replication mechanics and sovereignty enforcement at the replication layer.
+
+### 5.1 What Replicates Where
+
+| Store Type | Replication Model | Sovereignty Constraint |
+|-----------|------------------|----------------------|
+| **GitOps Store** (Intent State) | Git push/pull โ€” upstream-downstream replication | Intent records tagged with `sovereignty_zone`; replicated only to stores within the declared zone |
+| **Event Stream Store** (Audit) | Stream mirroring with lag monitoring | Audit records replicated to all authorized regions; cross-sovereignty replication requires explicit consent |
+| **Snapshot Store** (Realized State) | Synchronous within-zone; async cross-zone with consent | `sovereignty_zone` on entity governs which regions may hold a copy |
+| **Search Index** | Async replication; eventual consistency acceptable | Same sovereignty rules as Snapshot Store |
+
+### 5.2 Sovereignty-Aware Replication
+
+Every entity carries `sovereignty_zone` declarations that constrain which Storage Provider instances may hold copies:
+
+```yaml
+entity:
+  entity_uuid: 
+  sovereignty_zones:
+    - zone_id: EU-WEST
+      data_classifications: [restricted, phi]   # these classifications must stay in EU-WEST
+    - zone_id: "*"
+      data_classifications: [internal]           # internal data may replicate anywhere
+```
+
+The replication controller evaluates `sovereignty_zones` before routing any replication event. Replication to a non-authorized region for a given data classification is blocked at the replication layer โ€” the storage provider receives a `SOVEREIGNTY_VIOLATION` rejection.
+
+### 5.3 Replication Lag Monitoring
+
+```
+Storage Provider declares: max_replication_lag: PT30S
+
+DCM monitoring:
+  Every PT10S: measure replication lag across all replica pairs
+  
+  If lag > max_replication_lag:
+    storage.replication_lag_exceeded event (urgency: medium)
+    
+  If lag > 5 ร— max_replication_lag:
+    storage.replication_degraded event (urgency: high)
+    Affected region marked: capacity_status = degraded
+    New requests avoid degraded region for placement
+    
+  If replica unreachable:
+    storage.replica_unavailable event (urgency: critical)
+    Affected region marked: capacity_status = unavailable
+    Requests that require this region: held pending recovery
+```
+
+### 5.4 Conflict Resolution
+
+DCM uses a **last-write-wins with causality tracking** model for cross-region conflicts:
+
+- All writes carry a vector clock `{region_id: sequence_number}`
+- Concurrent writes (same entity, different regions) are detected by vector clock comparison
+- Resolution: the write with higher aggregate sequence number wins
+- Losing write: preserved as a `conflict_record` in the Audit Store (never silently dropped)
+- Platform admin notified of conflicts above a configurable threshold
+
+---
+
+## 6. Secret Zero โ€” Initial Credential Bootstrap
+
+The bootstrap sequence is specified in [Deployment Redundancy](17-deployment-redundancy.md) ยง6. This section specifies the credential bootstrap specifically โ€” how DCM components authenticate to each other before the Credential Provider is running.
+
+### 6.1 The Bootstrap Credential Problem
+
+At day-0, no Credential Provider exists. DCM components need credentials to communicate. The resolution is a **declarative bootstrap manifest** that contains one-time bootstrap credentials, plus a mandatory rotation on first successful startup.
+
+### 6.2 Bootstrap Sequence โ€” Credential Perspective
+
+```
+1. Bootstrap manifest contains:
+   bootstrap_credentials:
+     internal_ca:
+       cert_pem: 
+       key_pem:    # sealed with bootstrap passphrase
+       
+     bootstrap_admin:
+       username: bootstrap-admin
+       password_hash:   # operator sets this
+       
+     component_credentials:
+       # Pre-shared credentials for component-to-component auth
+       # until mTLS internal CA is operational
+       api_gateway:    {shared_secret: }
+       orchestrator:   {shared_secret: }
+       policy_engine:  {shared_secret: }
+       audit:          {shared_secret: }
+
+2. Bootstrap DCM starts:
+   - Internal CA initialized from bootstrap_credentials.internal_ca
+   - Components issued mTLS certificates from Internal CA
+   - Pre-shared secrets replaced by mTLS certificates on first successful CA handshake
+   - Pre-shared secrets deleted from memory and manifest after replacement
+
+3. Credential Provider starts:
+   - Bootstrapped with Internal CA certificate (trusts DCM's CA)
+   - Registered as the primary Credential Provider via bootstrap admin credential
+   - Takes ownership of internal CA key management
+   - Internal CA private key: transferred to Credential Provider, deleted from bootstrap manifest
+
+4. Bootstrap admin credential rotation (BOOT-002 โ€” mandatory):
+   - Bootstrap admin password must be rotated on first login
+   - New credential issued by Credential Provider (not the bootstrap manifest)
+   - Old password hash deleted from manifest
+   - Manifest sealed: no more secrets, only configuration
+
+5. Bootstrap manifest after completion:
+   - Contains only: DCM deployment configuration, Git remote, profile
+   - No secrets remain in the manifest
+   - Manifest committed to Git (now safe, secret-free)
+```
+
+### 6.3 Air-Gapped Bootstrap
+
+For sovereign/air-gapped deployments where the Credential Provider requires network access to an external vault:
+
+```
+Option A โ€” Embedded Credential Provider:
+  Use a locally-running Credential Provider (e.g., HashiCorp Vault in dev mode)
+  bootstrapped from the bootstrap manifest.
+  Upgrade to production Vault config post-bootstrap.
+
+Option B โ€” Operator-held keys:
+  Bootstrap manifest contains encrypted key material.
+  Operator provides passphrase at bootstrap time via stdin or hardware token.
+  Keys are never stored unencrypted at rest.
+  
+Option C โ€” HSM-backed bootstrap:
+  Internal CA private key is generated inside an HSM.
+  Bootstrap manifest contains only the HSM endpoint and slot reference.
+  Requires HSM to be available before DCM bootstrap begins.
+```
+
+### 6.4 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `BOOT-001` | The bootstrap manifest must not contain secrets after bootstrap completion. Any secret that persists in the manifest after first successful startup is a security violation. |
+| `BOOT-002` | The bootstrap admin credential must be rotated on first login. DCM enforces this โ€” the bootstrap admin account is locked from normal use until rotation is complete. |
+| `BOOT-003` | Pre-shared component credentials must be replaced by mTLS certificates within PT5M of Internal CA startup. Any component still using pre-shared secrets after this window generates a security alert. |
+| `BOOT-004` | The Internal CA private key must be transferred to the Credential Provider on Credential Provider registration. The key must not remain in any component's memory or storage after transfer is confirmed. |
+
+---
+
+## 7. Ownership Ambiguities โ€” Resolved
+
+### 7.1 Who Issues `operation_uuid`?
+
+**Decision: The API Gateway issues `operation_uuid` at request ingress.**
+
+Rationale: The API Gateway is the component that receives the POST request and must return the Operation response immediately. It assigns the UUID synchronously before any pipeline processing begins. The `operation_uuid` equals the `request_uuid` โ€” they are the same UUID, assigned at ingress.
+
+```
+Consumer: POST /api/v1/requests {...}
+
+API Gateway:
+  1. Authenticates consumer (checks session token)
+  2. Assigns request_uuid = operation_uuid = UUID4()  โ† here
+  3. Writes initial request record to Intent Store (status: INITIATED)
+  4. Publishes request.initiated event to Request Orchestrator (with request_uuid)
+  5. Returns Operation{name: /api/v1/operations/{request_uuid}, done: false}
+
+Request Orchestrator:
+  Receives request.initiated event with request_uuid
+  Uses the already-assigned request_uuid throughout the pipeline
+  Never assigns a new UUID
+```
+
+The Operation resource lives in a fast-queryable store owned by the API Gateway. When the pipeline progresses, the Request Orchestrator updates the Operation status by writing to this store (it has write access; the API Gateway reads from it for GET /api/v1/operations/{uuid} responses).
+
+### 7.2 Who Owns the Credential Revocation Registry?
+
+**Decision: The Credential Provider owns the Credential Revocation Registry.**
+
+Rationale: The Credential Provider is the authoritative source of credential lifecycle state. It issues credentials, rotates them, and revokes them. The revocation registry is a projection of that lifecycle state optimized for fast lookup.
+
+```
+Credential Revocation Registry:
+  Owner: Credential Provider
+  Storage: dedicated fast cache (Redis or equivalent)
+  Key structure: credential_uuid โ†’ {revoked_at, revocation_reason, effective_at}
+  TTL: max(credential_ttl, P90D)  โ€” persists at minimum 90 days after revocation
+
+Access model:
+  Write: Credential Provider (on revocation event)
+  Read:  All DCM components (via Credential Provider query API)
+         OR via local cache synced from Credential Provider push events
+
+Cache sync protocol:
+  Credential Provider publishes: credential.revoked event (Message Bus)
+  All subscribed components update local revocation cache
+  Cache TTL: PT1M standard; PT30S fsi/sovereign
+  On cache miss: component queries Credential Provider directly (not the cache)
+
+Session Revocation Registry: separate, owned by the Auth component
+  (Session revocation is distinct from credential revocation)
+```
+
+---
+
+## 8. Security Posture Specifications
+
+### 8.1 Threat Model โ€” Attack Surface Summary
+
+DCM's attack surface has five distinct boundaries. Each boundary has a specific trust model and mitigation set.
+
+**Boundary 1 โ€” Consumer Ingress (Web UI, Consumer API)**
+- Threat: Credential theft / session hijacking
+- Mitigations: mTLS optional at consumer boundary; bearer tokens with short TTL (PT1H standard); session revocation registry checked on every request; rate limiting at API Gateway
+- Threat: Tenant escape (accessing another tenant's data)
+- Mitigations: All queries mandatory-include tenant_uuid; row-level security on storage; Governance Matrix enforced before any read
+
+**Boundary 2 โ€” Provider Interface (Operator Interface, Callback API)**
+- Threat: Provider impersonation (malicious actor claims to be a legitimate provider)
+- Mitigations: mTLS required at provider boundary; provider callback credential required for callbacks; API Gateway validates dcm_entity_uuid in every callback against provider's registered entity scope
+- Threat: Malicious provider payload (provider sends crafted Realized State payload)
+- Mitigations: Realized State payloads validated against Resource Type Specification schema on receipt; GateKeeper policies evaluate provider-supplied data before it enters DCM state
+
+**Boundary 3 โ€” Admin Interface (Admin API)**
+- Threat: Unauthorized platform admin action
+- Mitigations: Authority Tier model enforces multi-tier approval for high-impact actions; all admin actions produce non-suppressable audit records; emergency admin access (break-glass) triggers immediate security notification
+- Threat: Configuration injection via GitOps
+- Mitigations: All GitOps PRs require domain-appropriate review before merge; policy contributions enter shadow mode before activation; GateKeeper policies validate all contributions at submission
+
+**Boundary 4 โ€” Internal Component Communication**
+- Threat: Component impersonation (compromised component issues requests as another)
+- Mitigations: mTLS with Internal CA for all component-to-component calls; interaction credentials checked on every call; Credential Revocation Registry queried on credential use
+- Threat: Lateral movement after component compromise
+- Mitigations: Each component holds minimum-scope interaction credentials; no component has write access to stores it does not own; audit records cannot be deleted by any component
+
+**Boundary 5 โ€” Storage Layer**
+- Threat: Direct database access bypassing application controls
+- Mitigations: Row-level security enforces tenant isolation even with direct DB access using application credentials; platform admin credentials are separate, audited, and require MFA; Storage Provider provenance emission means all direct writes are detectable
+
+**Highest-risk paths (not mitigated by single control):**
+1. Credential Provider compromise โ†’ cascading trust failure. Mitigation: Credential Provider is air-gapped from consumer traffic; HSM-backed key storage for sovereign profiles; separate backup credential authority.
+2. Internal CA compromise โ†’ all component trust fails. Mitigation: CA private key held only in Credential Provider (HSM-backed for fsi/sovereign); CA certificate rotation procedure documented.
+
+### 8.2 Supply Chain Security
+
+**Provider OpenAPI Spec Signing:**
+All Service Provider OpenAPI specifications submitted at registration must be signed using the provider's private key (corresponding to the public key in their mTLS certificate). DCM verifies the signature before the spec is processed. Unsigned specs are rejected with `SPEC_UNSIGNED` at GATE-SP-01.
+
+**Operator Container Image Provenance:**
+The DCM reference implementation containers are signed using Sigstore (Cosign). Deployment manifests declare the expected image digest. Any container running a different digest triggers drift detection on DCM's own deployment.
+
+**GitOps Store Secrets Scanning:**
+All content committed to DCM's GitOps stores passes through a secrets scanner before being accepted. The scanner checks for:
+- High-entropy strings matching known secret patterns (API keys, tokens, private keys)
+- Known credential formats (AWS access keys, GitHub PATs, JWT secrets)
+- PEM-encoded private key blocks
+
+A commit containing detected secrets is rejected with `SECRETS_DETECTED` and an audit record is written. The committing actor is notified.
+
+**SBOM Declaration:**
+Service Providers must declare a Software Bill of Materials reference at registration (optional for Tier 1 `dev` profiles; required for `fsi` and `sovereign`). The SBOM reference is stored in the provider record and included in accreditation evidence.
+
+### 8.3 System Policies
+
+| Policy | Rule |
+|--------|------|
+| `SEC-001` | All provider OpenAPI specs submitted at registration must be signed. Signature verification is performed at GATE-SP-01. |
+| `SEC-002` | DCM GitOps stores enforce secrets scanning on all commits. Commits with detected secrets are rejected. |
+| `SEC-003` | For `fsi` and `sovereign` profiles, SBOM declaration is mandatory for all Service Providers before activation. |
+| `SEC-004` | The Internal CA private key must be stored in an HSM for `sovereign` profile deployments. Software-only key storage is not permitted at sovereign profile. |
+| `SEC-005` | Any direct database write to a DCM store that bypasses the application layer is detectable via Storage Provider provenance emission. Detection triggers `audit.chain_integrity_alert` for affected records. |
+
+---
+
+## 9. Experience Gap Specifications
+
+### 9.1 New Tenant Onboarding Flow
+
+```
+Platform Admin initiates tenant creation:
+  POST /api/v1/admin/tenants
+  {
+    "display_name": "Payments Platform",
+    "handle": "payments-platform",
+    "group_class": "tenant_boundary",
+    "initial_quota_profile": "standard",
+    "billing_contact": "payments-ops@corp.example.com",
+    "data_classifications_permitted": ["internal", "restricted"],
+    "sovereignty_zones": ["EU-WEST"]
+  }
+
+DCM auto-provisions:
+  1. Tenant entity created (tenant_uuid assigned)
+  2. Default resource groups created:
+     - payments-platform/default (general resources)
+     - payments-platform/admins (tenant admin group)
+  3. Initial quota applied per quota_profile declaration
+  4. Tenant admin actor created (if initial_admin_email provided):
+     - Actor record created
+     - Welcome notification dispatched with first-login credential
+  5. Tenant Git namespace provisioned in GitOps store:
+     - /tenants/payments-platform/ directory created
+     - Initial tenant-scope policy stubs committed (shadow mode)
+  6. Search index namespace initialized
+  7. Audit stream created: dcm.audit.{tenant_uuid}
+
+Tenant admin completes setup:
+  1. First login โ†’ mandatory credential rotation (BOOT-002 equivalent)
+  2. Configure Auth Provider (or inherit platform default)
+  3. Add tenant members (invite by email or LDAP group mapping)
+  4. Review and activate initial policy stubs
+  5. Submit first service request (onboarding validation complete)
+
+Onboarding event sequence:
+  tenant.created โ†’ Platform Admin
+  tenant.member_added ร— N โ†’ new members (welcome email)
+  tenant.quota_configured โ†’ Platform Admin
+  tenant.onboarding_complete โ†’ Platform Admin + Tenant Admin
+  (fired when first OPERATIONAL entity exists in the tenant)
+```
+
+### 9.2 Pre-Request Cost Estimation UX
+
+The consumer experience for cost estimation before committing a request:
+
+```
+Step 1: Consumer browses catalog
+  GET /api/v1/catalog/{catalog_item_uuid}
+  Response includes: cost_estimate: {monthly_usd: 45.00, basis: "declared_static"}
+
+Step 2: Consumer configures request fields (e.g., selects VM size)
+  POST /api/v1/cost/estimate
+  {
+    "catalog_item_uuid": "",
+    "fields": {"cpu": 8, "ram_gb": 32, "storage_gb": 200, "environment": "prod"}
+  }
+  Response:
+  {
+    "estimated_monthly_usd": 187.50,
+    "cost_basis": "dynamic",
+    "cost_breakdown": [
+      {"component": "compute", "monthly_usd": 120.00},
+      {"component": "storage", "monthly_usd": 40.00},
+      {"component": "network_egress", "monthly_usd": 27.50}
+    ],
+    "disclaimer": "Estimate based on declared provider rates. Actual costs may vary.",
+    "provider_uuid": null  // not yet placed; estimate is across eligible providers
+  }
+
+Step 3: Consumer submits request with dry_run: true (optional pre-flight)
+  POST /api/v1/requests
+  {
+    "catalog_item_uuid": "",
+    "fields": {...},
+    "dry_run": true    // evaluate policy and placement; do not dispatch
+  }
+  Response: Operation with metadata.dry_run_result:
+  {
+    "policy_result": "PASS",
+    "placement_result": {
+      "selected_provider": "eu-west-prod-1",
+      "cost_at_selected_provider": 182.00
+    },
+    "gatekeeper_gates": [],
+    "warnings": ["Storage class 'premium' requested; 'standard' also eligible at $35.00/mo"]
+  }
+
+Step 4: Consumer submits without dry_run โ†’ actual request
+```
+
+### 9.3 Provider Sandbox / Test Mode
+
+Providers can register in sandbox mode for development and certification testing without affecting production routing:
+
+```yaml
+provider_registration:
+  # ...standard registration fields...
+  sandbox_mode: true           # this provider never receives production requests
+  sandbox_profile: dev         # sandbox providers only activated under dev profile
+  
+  # Sandbox providers:
+  # - Appear in the provider registry with status: sandbox
+  # - Can be explicitly targeted by test requests (fields.target_provider_uuid)
+  # - Never appear in placement engine candidate selection for non-test requests
+  # - Produce full audit records (useful for certification evidence)
+  # - Subject to same API validation as production providers
+  # - Can graduate to production via standard registration approval flow
+```
+
+Test request targeting a sandbox provider:
+```
+POST /api/v1/requests
+{
+  "catalog_item_uuid": "",
+  "fields": { ... },
+  "_test_context": {
+    "target_provider_uuid": "",
+    "suppress_billing": true,
+    "test_label": "certification-run-2026-04-01"
+  }
+}
+```
+
+### 9.4 SLA/SLO Tracking
+
+DCM tracks service delivery against declared SLOs at the Resource Type level:
+
+```yaml
+# Declared in Resource Type Specification (doc 05)
+resource_type_slo:
+  resource_type: Compute.VirtualMachine
+  
+  slos:
+    - metric: time_to_operational
+      target_percentile: p95
+      target_value: PT30M       # 95% of VMs should be OPERATIONAL within 30 minutes
+      measurement_window: P7D   # measured over trailing 7 days
+      
+    - metric: uptime
+      target_percentile: p99
+      target_value: "99.5%"     # 99.5% uptime over trailing 30 days
+      measurement_window: P30D
+      
+    - metric: drift_detection_latency
+      target_percentile: p90
+      target_value: PT1H        # drift detected within 1 hour of occurrence
+      measurement_window: P7D
+```
+
+SLO breach detection:
+```
+DCM computes SLO metrics continuously from audit records and entity lifecycle events.
+
+When a metric crosses a threshold:
+  slo.breach_approaching (urgency: medium) โ€” at 90% of SLO budget consumed
+  slo.breach_detected (urgency: high) โ€” SLO violated
+    payload: {resource_type, slo_metric, target, actual, measurement_window}
+
+Consumer-facing:
+  GET /api/v1/resources/{entity_uuid}/slo-status
+  Returns: current SLO metrics for the entity's resource type
+  
+Platform admin:
+  GET /api/v1/admin/slo/report?resource_type=Compute.VirtualMachine&window=P7D
+  Returns: aggregate SLO performance across all entities of this type
+```
+
+---
+
+*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).*
diff --git a/content/docs/project-overview.md b/content/docs/project-overview.md
new file mode 100644
index 0000000..9153689
--- /dev/null
+++ b/content/docs/project-overview.md
@@ -0,0 +1,327 @@
+# DCM โ€” Project Overview
+
+**GitHub:** https://github.com/dcm-project  
+**License:** Apache 2.0
+
+---
+
+## What DCM Is
+
+Data Center Management (DCM) is an open-source framework that gives enterprise IT organizations a hyperscaler-like cloud experience on infrastructure they own and control. It is the governing control plane that sits above provisioning tools, automation platforms, and infrastructure systems โ€” making them coherent, governed, and self-service.
+
+DCM is not a deployment tool, a configuration manager, or an automation platform. It is the management plane that connects them: a unified control plane with a declarative data model, a policy engine that enforces organizational standards automatically, and a service provider interface that integrates with the automation tools organizations already have.
+
+### Architecture in One Sentence
+
+DCM is built on three foundational abstractions โ€” **Data**, **Provider**, and **Policy** โ€” connected by a policy-driven event loop. Every concept in the architecture maps to one of these three.
+
+```
+Event (Data state change)
+  โ†’ Policy Engine evaluates all matching Policies
+  โ†’ Policies produce decisions / mutations / actions
+  โ†’ Actions invoke Providers or produce new Data
+  โ†’ New Data triggers new Events
+  โ†’ Repeat
+```
+
+---
+
+## The Problem DCM Solves
+
+Enterprise on-premises infrastructure is managed by dozens of disconnected tools, teams, and manual processes. A single virtual machine might require five teams, three ticketing systems, and two weeks to provision. No one has a trustworthy, real-time answer to: *what exists, what was requested, what was actually provisioned, and does the current state match what was intended?*
+
+The gap between intended state and actual state is where compliance risk, operational toil, and security incidents live.
+
+Specifically:
+
+**Fragmented operations.** Organizations spend engineering capacity stitching together disparate automation efforts rather than delivering services. Functionality is trapped in monoliths or siloed teams. There is no unified service catalog or API interface โ€” capabilities that public clouds provide as table stakes.
+
+**Long time-to-market.** The lifecycle of a single infrastructure asset is managed by many teams, with approvals, provisioning, and auditing taking weeks. Operations teams face high toil in day-to-day management.
+
+**The private cloud gap.** True private cloud requires networking, storage, identity, catalogs, FinOps, observability, auditing, and risk management โ€” not just on-premises compute. Most organizations have the compute but not the surrounding governance layer.
+
+**Unreliable data.** There is typically no reconciliation between discovered inventory (what actually exists) and intended inventory (what was requested). Without a unified data model, it is impossible to know whether current state is aligned with deployed state, and intended state is often not stored at all.
+
+**Compliance overhead.** In regulated environments (financial services, government, defense, healthcare), compliance evidence must be produced manually from systems that were not designed to provide it. DCM makes compliance evidence a structural property of every operation โ€” built into the data, not reconstructed after the fact.
+
+---
+
+## What DCM Does
+
+DCM manages the complete lifecycle of infrastructure resources โ€” from bare metal hardware and network gear at the physical layer through virtual machines, containers, Kubernetes clusters, and application platforms. The lifecycle includes Day 0 (standing up new infrastructure), Day 1 (provisioning resources), Day 2 (operating and managing), and Day N (decommissioning and releasing).
+
+### The Three Foundational Abstractions
+
+**Data** is everything that exists in DCM โ€” resource entities, policy definitions, data layers, audit records, accreditations, group memberships. Every artifact has a UUID, a lifecycle state, a schema type, and complete field-level provenance recording where every value came from and why it changed. Data flows through the system in four states:
+
+| State | What It Represents |
+|-------|-------------------|
+| **Intent** | What the consumer originally asked for โ€” immutable after submission |
+| **Requested** | The fully assembled, policy-approved dispatch payload โ€” what was sent to the provider |
+| **Realized** | What the provider actually provisioned โ€” the authoritative record of what exists |
+| **Discovered** | What currently exists in the environment โ€” used to detect drift |
+
+Comparing these four states continuously is how DCM detects drift, enforces governance, and enables rehydration (replaying original intent through current policies to reproduce a resource in a new location or after failure).
+
+**Policy** is every rule that governs DCM behavior โ€” expressed as code, stored in Git, versioned, tested in shadow mode before activation, and enforced deterministically. Seven typed policy schemas cover every governance need:
+
+| Policy Type | What It Does |
+|-------------|-------------|
+| **GateKeeper** | Halts requests or contributes weighted risk scores for approval routing |
+| **Validation** | Checks structural correctness; halts on failure or accumulates advisory warnings |
+| **Transformation** | Automatically enriches request payloads (injects values, enforces field locks) |
+| **Orchestration Flow** | Defines named workflows as explicit, ordered pipeline steps |
+| **Recovery** | Governs what DCM does when things go wrong (timeout, failure, drift) |
+| **Governance Matrix Rule** | Enforces data classification and sovereignty boundaries at every cross-boundary interaction |
+| **Lifecycle** | Triggers actions on entity state changes (TTL expiry, scheduled operations) |
+
+Every business rule, every compliance constraint, every operational standard is a Policy โ€” never hard-coded, always auditable, always testable in shadow mode before enforcement.
+
+**Providers** are everything external that DCM integrates with. Eleven typed provider contracts share a common base (registration, health, mTLS, sovereignty declaration, accreditation, governance matrix enforcement) and add typed capability extensions:
+
+| Provider Type | Capability |
+|--------------|-----------|
+| **Service Provider** | Realizes infrastructure resources (VMs, networks, storage, containers, bare metal) |
+| **Information Provider** | Serves authoritative external data (CMDB, HR, finance, identity systems) |
+| **Meta Provider** | Composes multiple providers into compound catalog items (a "three-tier web app" as a single catalog entry) |
+| **Storage Provider** | Persists DCM state (GitOps stores, event streams, audit, search index) |
+| **Credential Provider** | Issues, rotates, and revokes secrets and credentials |
+| **Auth Provider** | Authenticates identities; resolves role and group memberships |
+| **Policy Provider** | Evaluates policies externally (OPA/Rego sidecar integration) |
+| **Notification Provider** | Delivers notifications to channels (Slack, PagerDuty, email, webhook) |
+| **Message Bus Provider** | Async event streaming to external systems (Kafka, etc.) |
+| **ITSM Provider** | Integrates ServiceNow, Jira, or other ITSM systems as approval gates |
+| **Peer DCM** | Another DCM instance participating in federated deployment |
+
+### What This Enables
+
+A consumer can browse a Service Catalog showing only the resources they are entitled to request, configure their requirements, submit a request, and receive a provisioned resource โ€” with full cost attribution, audit trail, drift monitoring, and decommissioning capability โ€” without ever interacting with any underlying automation tool directly.
+
+Organizational standards, security requirements, placement constraints, and data sovereignty rules are enforced automatically by the Policy Engine before anything is dispatched to a provider. There are no runbooks to follow correctly. The correct behavior is structural.
+
+---
+
+## Who Benefits
+
+### Consumers โ€” Application Teams and Developers
+
+Self-service infrastructure that matches the experience of public cloud, on infrastructure the organization controls. Browse a catalog, configure requirements, submit a request, receive a provisioned resource. No tickets to raise, no teams to coordinate with, no runbooks to follow.
+
+### Platform Engineers and Infrastructure Operations
+
+A single control plane rather than a collection of disconnected tools. Define the service catalog, write policies that enforce organizational standards, manage the provider ecosystem. Provisioning consistency is structural. Drift is detected automatically. Day 2 toil is governed by policy, not enforced by individual operators following runbooks.
+
+### Security and Compliance Teams
+
+Policy-as-code tested before activation (shadow mode), enforced at every pipeline stage, producing a tamper-evident audit trail at every state transition. Data classification (PHI, restricted, sovereign) is a first-class concept enforced at the Governance Matrix layer โ€” always boolean, never scoreable around. Accreditation monitoring automatically verifies provider certifications against external registries (FedRAMP marketplace, CMMC AB, ISO IAF CertSearch) so compliance posture is continuously verified rather than point-in-time attested.
+
+### SRE and Operations Teams
+
+Complete observability into what was requested, what was provisioned, whether it matches, and what changed. Drift detection compares Realized State against Discovered State and triggers remediation per policy. Four states give a complete picture of the gap between intent and reality at any moment.
+
+### Auditors
+
+A complete, chronological, tamper-evident record of every change to every artifact โ€” who made it, when, through what authorization chain, and what the values were before and after. The audit hash chain means any deleted or modified record is detectable. Field-level provenance means every value in every record can be traced to its origin.
+
+### FinOps and Business Leadership
+
+Cost attribution built into the provisioning pipeline. Every resource has a cost estimate before provisioning and ongoing cost attribution to the owning Tenant and business unit throughout its lifecycle. Pre-request cost estimates, placement tie-breaking by cost, and consumer-visible cost views are all standard capabilities.
+
+---
+
+## Where DCM Operates
+
+DCM operates in the control plane layer of an enterprise data center estate. It is infrastructure-agnostic โ€” it does not care whether the underlying infrastructure is bare metal servers, VMware clusters, OpenStack clouds, Kubernetes clusters, or network equipment. Service Providers abstract those differences. DCM manages all of them through the same interface and data model.
+
+### Deployment Topology
+
+| Mode | Description |
+|------|-------------|
+| **Single-region** | One DCM instance governs one data center or regional infrastructure estate |
+| **Federated multi-instance** | Multiple DCM instances with declared trust relationships route requests across regional and organizational boundaries |
+| **Hub/Regional** | Hub DCM holds the authoritative registry and global policy hierarchy; Regional instances handle local provider routing and operate independently during hub unavailability |
+| **Sovereign/Air-gapped** | Full air-gapped operation with HSM-backed key storage, local credential management, and no external network dependencies |
+
+### Data Sovereignty
+
+DCM enforces data sovereignty as a structural property, not a configuration option. Every entity carries sovereignty zone declarations that constrain which Storage Providers may hold copies and which Service Providers may handle the data. The Governance Matrix enforces these boundaries at every cross-boundary interaction โ€” and it is always boolean. No scoring, no exceptions, no policy can route around a declared sovereignty constraint.
+
+### Target Environments
+
+DCM specifically targets organizations with on-premises infrastructure that must behave like a cloud โ€” particularly in regulated industries:
+
+- **Financial services** โ€” where public cloud is constrained by regulatory requirements, data residency, and audit obligations
+- **Government and defense** โ€” where FedRAMP, CMMC, DoD Impact Level authorization, and air-gapped operation are operational requirements
+- **Healthcare** โ€” where HIPAA BAA requirements and PHI data classification impose specific placement and access constraints
+- **Critical infrastructure** โ€” where sovereignty over the infrastructure itself is as important as sovereignty over the data
+
+These are also the environments where the gap between what on-premises infrastructure currently provides and what organizations need is widest โ€” and where DCM's compliance framework integrations (FedRAMP, CMMC, HIPAA, SOC 2, ISO 27001, PCI DSS, DoD Impact Levels) are built directly into the accreditation model rather than added as an afterthought.
+
+---
+
+## How DCM Works
+
+### The Event Loop
+
+DCM's runtime is a policy-driven event loop. There is no hard-coded pipeline โ€” the pipeline is the sum of active policies responding to data state changes.
+
+```
+Event (Data state change โ€” e.g., request submitted)
+  โ†’ Policy Engine evaluates all matching Policies
+  โ†’ Policies produce typed outputs:
+      GateKeeper: approve / halt / risk score
+      Validation: pass / fail / warning
+      Transformation: inject fields / lock values / annotate provenance
+      Placement: constraints + preferences โ†’ Provider selected
+      Orchestration Flow: ordered step sequence
+      Recovery: action when things go wrong
+      Governance Matrix: ALLOW / DENY / STRIP / REDACT
+  โ†’ Outputs invoke Providers or produce new Data
+  โ†’ New Data changes trigger new Events
+  โ†’ Repeat until terminal state
+```
+
+Every step is audited. Every field mutation carries provenance. Every decision is deterministic โ€” the same data, evaluated by the same policies, produces the same outcome regardless of who submitted it or when.
+
+### The Request Lifecycle
+
+When a consumer submits a service request, DCM executes a governed assembly pipeline:
+
+```
+1. Consumer submits request (intent declared)
+   โ†’ Intent State written (immutable snapshot of original ask)
+
+2. Layer assembly
+   โ†’ Core Layers applied (datacenter, rack, network zone, location)
+   โ†’ Service Layers applied (resource-type-specific configuration)
+   โ†’ Consumer fields merged
+   โ†’ Transformation Policies inject and lock required fields
+
+3. Policy evaluation
+   โ†’ Validation Policies check structural correctness
+   โ†’ GateKeeper Policies assess risk and enforce business rules
+   โ†’ Placement Engine selects provider (constraints + scoring)
+   โ†’ Requested State written (the approved, assembled dispatch payload)
+
+4. Dispatch
+   โ†’ CreateRequest sent to selected Service Provider
+   โ†’ Provider naturalizes (translates DCM model to native format)
+   โ†’ Provider executes
+   โ†’ Provider denaturalizes (translates result back to DCM model)
+   โ†’ Realized State written (what was actually provisioned)
+
+5. Ongoing lifecycle
+   โ†’ Discovery Scheduler polls for drift between Realized and Discovered State
+   โ†’ Drift triggers Recovery Policy evaluation
+   โ†’ TTL, scheduled operations, and rehydration operate on Realized State
+   โ†’ Decommission mirrors provisioning in reverse
+```
+
+### How Policy Replaces Hard-Coded Logic
+
+Every business rule in DCM is a Policy artifact โ€” stored in Git, versioned, tested in shadow mode before activation, and enforced deterministically. There are no approval workflows embedded in code, no hard-coded placement rules, no statically defined pipeline stages.
+
+This means:
+- Adding a new approval step = writing a GateKeeper policy
+- Changing where a resource is placed = updating a Placement policy  
+- Auto-injecting a required field = writing a Transformation policy
+- Defining what happens on failure = writing a Recovery policy
+- Building a named multi-step workflow = writing an Orchestration Flow policy
+
+Policy authoring is not an engineering task requiring code deployment. Policies are GitOps artifacts. They go through PR review, shadow evaluation (running against real traffic without enforcing), and staged activation. A new policy can go from idea to enforced governance without a code release.
+
+### How Providers Integrate
+
+Every provider implements one base contract: registration, health check, mTLS identity, sovereignty declaration, and governance matrix enforcement. What varies between provider types is the capability extension โ€” the typed set of additional endpoints and behaviors that define what the provider can do.
+
+This means DCM does not need to know in advance what automation tools an organization uses. A VMware vSphere cluster, a bare metal Redfish endpoint, a Kubernetes cluster via CAPI, and an OpenStack cloud all register as Service Providers. DCM speaks the same language to all of them. The provider handles translation to and from the native tool format โ€” Naturalization (DCM โ†’ native) and Denaturalization (native โ†’ DCM).
+
+Organizations do not replace their existing automation. They wrap it in the Provider interface. The investment in Ansible playbooks, Terraform modules, and vendor APIs is preserved โ€” DCM adds the governance layer above it.
+
+### How Data Sovereignty Is Enforced
+
+Data sovereignty is not a configuration flag โ€” it is a structural property evaluated at every boundary crossing. Every entity carries `sovereignty_zone` declarations. The Governance Matrix evaluates these at every provider dispatch, storage write, and federation routing decision. The result is always boolean: ALLOW or DENY. There is no scoring that can route around a sovereignty constraint.
+
+For regulated environments: PHI data cannot be dispatched to a provider without a HIPAA BAA accreditation. Restricted data cannot leave a declared sovereignty zone. Classified data cannot be held by a provider without the declared authorization level. These are not policies an administrator can override โ€” they are matrix rules with hard enforcement.
+
+---
+
+## Ethos
+
+DCM is built on a specific set of values that drive every design decision. These are not marketing statements โ€” they are the decision framework applied when priorities conflict.
+
+### Security Is the Baseline, Not a Feature
+
+Security properties are architecturally present in every profile, including the minimal development profile. What profiles control is enforcement strictness and operational overhead โ€” not whether the property applies.
+
+The `minimal` profile is "security with minimal operational overhead" โ€” not "minimal security." It rotates credentials, audits first access, maintains a revocation registry, and runs shadow mode on all contributed policies. The intervals are longer, the triggers are more permissive, and manual steps replace automated ones โ€” but the security model is present and correct.
+
+When security and convenience conflict, security wins. But the design obligation is not just to enforce security โ€” it is to make the secure path the easy path. A security model that is routinely bypassed because it is too burdensome has failed at both security and usability. The profile system exists precisely to make secure behavior automatic rather than effortful.
+
+### The Governed Path Must Also Be the Easy Path
+
+Self-service is not a nice-to-have โ€” it is the mechanism through which governance is achieved at scale. If consuming resources through DCM is harder than raising a ticket or writing a one-off script, application teams will find the path of least resistance and all of DCM's governance benefits evaporate.
+
+This principle shapes every interface decision: request submission should be a single API call; ordinary requests should auto-approve; policy authoring should not require Rego expertise for common cases; cost estimates should appear before a consumer commits. The score-driven auto-approval system exists specifically so that clean, standard requests are not gated on human review that adds no value.
+
+Ease of use is not in tension with governance โ€” it is the delivery mechanism for governance. Organizations that find DCM's governed path easier than ungoverned alternatives will use it. Those that do not will route around it.
+
+### Compliance Is Constructed, Not Audited
+
+Compliance evidence in DCM is a structural product of every operation, not something reconstructed after the fact. The audit trail is written at every state transition. Provenance is embedded in every field. The Governance Matrix enforces data classification boundaries at every crossing. Accreditation status is continuously monitored against external registries.
+
+The implication: when an auditor asks "show me every person who touched this data between these dates," DCM can answer that question directly from the audit store without any manual evidence gathering. When a regulator asks "prove this data never left the EU," the Governance Matrix enforcement log answers the question with cryptographic tamper evidence.
+
+This is different from compliance tooling that inspects systems and produces reports. DCM does not inspect systems and report findings โ€” it governs operations and makes compliance a property of the operations themselves.
+
+### The Architecture Should Be Easy to Implement and Extend
+
+DCM is designed so that new capabilities fit within the existing three-abstraction model without modifying the core. A new provider type is a new implementation of the base contract with a new capability extension โ€” no core changes. A new policy type is a new output schema โ€” no core changes. A new data entity type is a new schema registration โ€” no core changes.
+
+This is the test: if a new capability can be expressed as Data, Provider, or Policy, it belongs in DCM and requires no architectural changes. If it cannot be expressed within these three abstractions, it is either a runtime implementation detail or a signal that the abstractions need to be reconsidered.
+
+The same principle applies to organizations deploying DCM. They should not need to modify DCM source code to adapt it to their environment. New compliance requirements become policy additions. New infrastructure types become new provider implementations. New approval processes become custom authority tier definitions. DCM provides the framework; organizations provide the domain knowledge.
+
+### No Silent Behavior
+
+Every operation in DCM produces an observable artifact. No change is silent. No failure disappears into a log that no one reads. Every state transition produces an audit record. Every policy decision produces a typed output with a score driver. Every drift detection produces a record that links current state to intended state.
+
+This is not just an audit requirement โ€” it is an architectural property that makes debugging, compliance, and operational understanding possible. When something goes wrong in a governed system, the question "what happened and why" should always be answerable from the data the system produced as part of its normal operation.
+
+---
+
+
+## Key Facts
+
+| | |
+|-|-|
+| **Architecture** | Three abstractions (Data, Provider, Policy) ยท Policy-driven event loop |
+| **Provider types** | 11 (unified base contract + typed capability extensions) |
+| **Policy types** | 7 (typed output schemas with deterministic evaluation) |
+| **Entity lifecycle states** | 4 (Intent ยท Requested ยท Realized ยท Discovered) |
+| **Capabilities** | 299 across 38 domains |
+| **Data model documents** | 55 |
+| **Specifications** | 15 |
+| **OpenAPI paths** | 63 consumer ยท 57 admin ยท 5 operator ยท 7 provider callback |
+| **Compliance frameworks** | FedRAMP ยท CMMC ยท HIPAA ยท SOC 2 ยท ISO 27001 ยท PCI DSS ยท DoD IL2โ€“IL6 |
+| **Deployment profiles** | minimal ยท dev ยท standard ยท prod ยท fsi ยท sovereign |
+| **License** | Apache 2.0 |
+| **GitHub** | https://github.com/dcm-project |
+
+---
+
+## Core Design Principles
+
+1. **Declarative** โ€” data describes desired state, not procedures
+2. **API-First** โ€” every capability available via standard AEP-aligned API
+3. **Policy-Governed** โ€” all business logic through the Policy Engine, never hard-coded
+4. **Idempotent** โ€” applying the same data multiple times produces the same result
+5. **Immutable if Versioned** โ€” published versions never change; changes produce new versions
+6. **Provider-Agnostic** โ€” DCM defines contracts, not implementations
+7. **GitOps-Native** โ€” intent and policy artifacts are Git-native; versioned, reviewable, auditable
+8. **Federated** โ€” all authorized actor types contribute within permitted scope; no single administrator bottleneck
+9. **Compliance by Construction** โ€” audit trail, provenance, and sovereignty enforcement are structural, not added post-hoc
+
+---
+
+*DCM is open-source. Contributions, feedback, and discussion welcome at https://github.com/dcm-project*