From a616a4926b094f2df4412fb33323e3966193e9b3 Mon Sep 17 00:00:00 2001 From: Chris Roadfeldt Date: Tue, 16 Jun 2026 16:43:20 -0500 Subject: [PATCH 1/2] architecture: capabilities, requirements and catalog Capabilities matrix, platform requirements, pattern-catalog overlay, and discussion topics. Signed-off-by: Chris Roadfeldt --- architecture/DCM-Capabilities-Matrix.md | 717 ++++++++++++++++++++ architecture/DISCUSSION-TOPICS.md | 644 ++++++++++++++++++ architecture/dcm-pattern-catalog-overlay.md | 408 +++++++++++ architecture/dcm-platform-requirements.md | 701 +++++++++++++++++++ 4 files changed, 2470 insertions(+) create mode 100644 architecture/DCM-Capabilities-Matrix.md create mode 100644 architecture/DISCUSSION-TOPICS.md create mode 100644 architecture/dcm-pattern-catalog-overlay.md create mode 100644 architecture/dcm-platform-requirements.md diff --git a/architecture/DCM-Capabilities-Matrix.md b/architecture/DCM-Capabilities-Matrix.md new file mode 100644 index 0000000..ff3547e --- /dev/null +++ b/architecture/DCM-Capabilities-Matrix.md @@ -0,0 +1,717 @@ +# DCM — Foundational Capabilities Matrix + +> **Purpose:** This document defines the core operational capabilities required for DCM to perform lifecycle management as defined by the data model. Each capability maps to a consumer/service provider perspective and will be used to drive implementation work in Jira. +> +> **How to read this document:** +> - **Capability Domain** — the architectural area the capability belongs to +> - **Capability** — a discrete operational function; the smallest unit of independently implementable behavior +> - **Consumer perspective** — what the end user / application team experiences +> - **Service Provider perspective** — what the Service Provider or platform component must implement +> - **Platform/Admin perspective** — what the platform engineer or SRE must configure or operate +> - **Depends on** — other capabilities that must exist first + +--- + +## 1. Identity and Access Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| IAM-001 | Actor Authentication | Authenticate to DCM via configured IdP | — | Register and configure Auth Providers; manage local user store | — | +| IAM-002 | Session Token Management | Receive and use session tokens; token refresh | — | Configure session TTL, failover chain | IAM-001 | +| IAM-003 | Role-Based Access Control | Receive role-appropriate service catalog and API responses | — | Declare role mappings; assign roles to actors | IAM-001 | +| IAM-004 | Group Membership Resolution | Group memberships automatically applied from IdP | — | Map IdP groups to DCM groups; declare group-role relationships | IAM-001, IAM-003 | +| IAM-005 | Multi-Factor Authentication | Satisfy per-session and step-up MFA challenges | — | Configure MFA methods; declare step-up operations | IAM-001 | +| IAM-006 | SCIM Automated Provisioning | Actor created/updated/deprovisioned from IdP automatically | — | Configure SCIM endpoint and attribute mappings | IAM-001 | +| IAM-007 | Tenant Scope Enforcement | Access restricted to authorized Tenants | — | Declare Tenant membership; configure cross-tenant policies | IAM-003, IAM-004 | +| AUTH-002 | Multi-Auth-Provider Routing | — | — | Register multiple Auth Providers simultaneously; ingress routes by authentication signal | IAM-001 | +| AUTH-003 | Auth Provider Trust Level Enforcement | Requests evaluated per provider trust level (authoritative / verified / advisory) | — | Configure trust level per registered Auth Provider | IAM-001 | +| AUTH-004 | Auth Provider Artifact Versioning | — | — | Manage role/tenant mapping versioning through standard DCM artifact lifecycle; activate/deprecate mappings | IAM-003 | +| AUTH-005 | Auth Provider Failover | Existing sessions remain valid on provider failure; new auth routes to failover chain | — | Configure failover chain; monitor provider health; manage session cache TTL | IAM-001 | +| AUTH-006 | Auth Context in Audit Trail | — | — | Auth Provider identity and ingress context automatically recorded in all audit records | IAM-001 | +| AUTH-007 | Auth Provider Credential Security | — | — | Enforce Auth Provider config credentials reference secrets management; no plaintext credentials | IAM-001 | +| AUTH-008 | No Anonymous Access | — | — | Enforce authenticated access at all ingress surfaces across all profiles | IAM-001 | +| AUTH-009 | Webhook and Message Bus Authentication | Authenticate webhook registrations | — | Enforce authentication on all inbound surfaces regardless of profile | IAM-001 | +| AUTH-010 | Per-Actor Rate Limiting | Receive 429 responses when rate limit exceeded | — | Configure rate limits per actor; manage burst allowances | IAM-001 | +| AUTH-011 | Git PR Identity Resolution | Git PR submissions resolve to same actor identity as API/UI login | — | Configure Auth Provider to trust Git server's identity assertion | IAM-001 | +| AUTH-012 | SCIM Automated Provisioning | Actor created/updated/deprovisioned from IdP automatically via SCIM | — | Configure SCIM 2.0 endpoint; manage suspension-on-deprovision policy | IAM-001 | +| AUTH-013 | In-Flight Request Continuity on Auth Failure | In-flight requests before auth failure are not interrupted | — | Configure session cache TTL; manage graceful degradation | IAM-001 | +| AUTH-014 | Two-Tier MFA Enforcement | Satisfy per-session MFA at login and step-up MFA for high-risk operations | — | Configure per-session and step-up MFA; declare step-up trigger operations | IAM-005 | +| AUTH-015 | Built-In Auth Provider Storage Backend | — | — | Configure built-in Auth Provider storage backend (SQLite for minimal/dev; PostgreSQL/MySQL for standard+) | IAM-001 | + +--- + +## 2. Service Catalog + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| CAT-001 | Service Catalog Presentation | Browse available services filtered by RBAC | Declare catalog items for offered resource types | Activate catalog items; configure catalog visibility policies | IAM-003, IAM-007 | +| CAT-002 | Service Schema Discovery | View field schemas, constraints, and edit constraints for a catalog item | Declare field schemas in Resource Type Spec | Configure constraint visibility level per profile | CAT-001 | +| CAT-003 | Catalog Item Search and Filter | Search catalog by keyword, resource type, tag | — | Configure Search Index for catalog | CAT-001 | +| CAT-004 | Catalog Item Versioning | Request a specific version of a catalog item | Publish new catalog item versions following semver | Manage version lifecycle; enforce deprecation timelines | CAT-001 | +| CAT-005 | Cost Estimation | Receive estimated cost before submitting a request | Declare cost metadata on provider registration | Configure Cost Analysis component | CAT-001 | +| CAT-006 | Dependency Visualization | See required dependencies for a catalog item before requesting | Declare dependency graph in Resource Type Spec | — | CAT-001 | +| CAT-007 | Catalog Item Deprecation | Receive deprecation warnings on deprecated catalog items | Declare successor types in deprecation notice | Manage deprecation lifecycle; notify consumers | CAT-004 | + +--- + +## 3. Request Lifecycle Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| REQ-001 | Submit Service Request | Submit a resource request via UI, API, or Git PR | — | Configure request ingress surfaces | IAM-007, CAT-001 | +| REQ-002 | Intent State Capture | Request stored as versioned GitOps artifact before processing | — | Configure Intent Store; manage Git repository structure | REQ-001 | +| REQ-003 | Layer Assembly | Request enriched with organizational defaults and context layers | Contribute Service Layers for resource types | Manage Core Layers; configure Layer Cache | REQ-002 | +| REQ-004 | Policy Evaluation | Request validated, transformed, and gated by applicable policies | Contribute provider-specific policies | Manage Policy Engine; configure Policy Groups and Profiles | REQ-003 | +| REQ-005 | Placement Engine Execution | Resource placed with the best available provider instance | Implement capacity reserve_query response | Configure placement constraints; manage provider priorities | REQ-004 | +| REQ-006 | Requested State Persistence | Assembled payload stored as authoritative GitOps record | — | Configure Requested Store; manage storage redundancy | REQ-005 | +| REQ-007 | Provider Dispatch | Request payload delivered to selected provider | Implement Services API to receive DCM payloads | Configure API Gateway and egress | REQ-006 | +| REQ-008 | Request Status Tracking | Monitor request status from submitted through realized | Report realization status back to DCM | Configure observability for request tracking | REQ-007 | +| REQ-009 | Request Cancellation | Cancel a pending request before realization | Handle cancellation payloads | Configure cancellation policies | REQ-002 | +| REQ-010 | Git PR Request Ingress | Submit requests via Git Pull Request with policy dry-run feedback | — | Configure Git Request Watcher; manage repository structure | REQ-001, IAM-001 | + +--- + +## 4. Provider Contract and Realization + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| PRV-001 | Provider Registration | — | Register provider with DCM: declare type, capabilities, sovereignty, cost metadata | Configure Provider Registry; validate sovereignty declarations | IAM-001 | +| PRV-002 | Naturalization | — | Convert DCM unified payload to provider-native format | — | PRV-001, REQ-007 | +| PRV-003 | Realization | — | Execute required actions to provision/configure/change resource | — | PRV-002 | +| PRV-004 | Denaturalization | — | Convert provider-native result back to DCM unified format | — | PRV-003 | +| PRV-005 | Realized State Reporting | — | Report realized payload and status to DCM API Gateway | Configure Realized State Store; manage Event Stream | PRV-004 | +| PRV-006 | Capacity Reporting | — | Respond to reserve_query with current capacity and availability | Configure placement engine; manage capacity confidence | PRV-001 | +| PRV-007 | Provider Health Reporting | — | Expose health check endpoint; report availability | Monitor provider health; configure trust score updates | PRV-001 | +| PRV-008 | Sovereignty Declaration Maintenance | — | Notify DCM when sovereignty data changes within declared SLA | Monitor sovereignty changes; trigger re-evaluation | PRV-001 | +| PRV-009 | Composite Service Composition | — | Register Composite Service catalog items declaring constituent resource types, dependencies, and delivery requirements | Configure Composite Service registration eligibility | PRV-001, PRV-003 | +| PRR-001 | OpenAPI Spec Declaration (GATE-SP-01) | — | Declare OpenAPI spec URL at registration; spec must be machine-readable and reachable | Validate spec URL reachability during approval pipeline | PRV-001 | +| PRR-002 | Healthy API at Activation (GATE-SP-02) | — | Health endpoint returns `{"status": "healthy"}` at activation time | Enforce health check as activation precondition | PRV-001, HLT-001 | +| PRR-003 | State Management Callback (GATE-SP-03) | — | Implement realized_state_push callback at all conformance levels | Validate callback endpoint reachability during approval | PRV-001 | +| PRR-004 | Tenant Metadata Endpoint (GATE-SP-04) | — | Implement GET /api/v1/tenants/{uuid}/metadata returning usage data | Require for standard+ profile activation; enforce quota integration | PRV-001 | +| PRR-005 | Prometheus Metrics (GATE-SP-05) | — | Expose required metric families at declared metrics_endpoint | Validate metric presence during approval; gate standard+ activation | PRV-001, HLT-005 | +| PRR-006 | AEP.DEV Linting (GATE-SP-06) | — | Pass AEP linter against OpenAPI spec with no errors before registration; include linting report URL | Gate standard+ activation on linting pass; block activation on errors | PRV-001 | + +| PRV-010 | Provider Sandbox/Test Mode | Submit test requests targeting sandbox providers via `_test_context.target_provider_uuid`; sandbox providers visible in registry with `status: sandbox` | Register with `sandbox_mode: true`; implement full OIS contract; graduate to production via standard approval | Manage sandbox provider registry; review graduation requests; sandbox providers excluded from production placement | PRV-001, GATE-SP-01 || PRR-007 | Multi-Tenant Dispatch (GATE-SP-07) | — | Accept tenant_uuid in all dispatch payloads; return tenant-scoped resources | Gate standard+ activation on multi-tenant compatibility test | PRV-001 | + +--- + +## 5. Resource Lifecycle Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| LCM-001 | Resource State Transitions | Trigger lifecycle actions: suspend, resume, decommission | Handle state transition payloads | Configure lifecycle policies; manage state machine | REQ-008 | +| LCM-002 | Post-Realization Field Updates | Update editable fields on realized resources (targeted delta) | Handle delta update payloads; apply partial changes | Configure editable field declarations; manage edit policies | PRV-005 | +| LCM-003 | Resource TTL Management | Declare and extend resource TTLs; receive expiry notifications | Handle TTL-triggered decommission payloads | Configure Lifecycle Constraint Enforcer; manage expiry policies | LCM-001 | +| LCM-004 | Ownership Transfer | Transfer resource ownership to a different Tenant | — | Authorize and execute ownership transfers; record transfer history | IAM-007, LCM-001 | +| LCM-005 | Rehydration | Replay a resource's intent state to a new provider or context | Receive and execute rehydration payloads | Manage rehydration leases; configure auth level requirements | REQ-002, PRV-003 | +| LCM-006 | Billing State Management | — | — | Configure billing state policies; integrate with Cost Analysis | LCM-001 | +| LCM-007 | Resource Decommission | Decommission resources individually or as part of group decommission | Handle decommission payloads; release resources | Manage decommission workflows; coordinate dependency teardown | LCM-001 | + +--- + +## 6. Drift Detection and Remediation + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| DRF-001 | Active Discovery | — | Expose discovery endpoint; respond to interrogation queries | Configure discovery schedules; manage Discovered Store | PRV-005 | +| DRF-002 | Drift Comparison | Receive drift notifications for owned resources | — | Configure drift detection policies; manage comparison logic | DRF-001, PRV-005 | +| DRF-003 | Drift Notification | Receive actionable drift alerts with field-level detail | — | Configure drift notification channels and escalation policies | DRF-002 | +| DRF-004 | Drift Remediation | Approve or reject automatic drift remediation | Execute remediation payloads | Configure remediation policies (revert/update/alert/escalate) | DRF-002, LCM-002 | +| DRF-005 | Unsanctioned Change Detection | Receive alerts on unauthorized resource modifications | Report all external state changes to DCM | Configure unsanctioned change policies | DRF-001 | + +--- + +## 7. Policy Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| POL-001 | Policy Authoring | — | Contribute provider-specific policy rules | Author and manage policies via API or GitOps ingress | IAM-003 | +| POL-002 | Policy Validation and Shadow Mode | View shadow evaluation results on own requests | — | Configure shadow mode; review shadow results | POL-001 | +| POL-003 | Policy Activation and Review | — | — | Manage policy review periods; authorize policy activation | POL-001, POL-002 | +| POL-004 | Policy Group Management | — | — | Compose Policy Groups; manage profile assignments | POL-003 | +| POL-005 | Profile Management | — | — | Configure deployment profiles; manage compliance domain groups | POL-004 | +| POL-006 | External Policy Evaluation | — | Register as external evaluation endpoint; implement BBQ-001–009 governance | Configure external evaluation trust levels; manage trust elevation | PRV-001, POL-001 | +| POL-007 | Policy Override and Constraint Visibility | View constraint details for service catalog fields | Declare constraint schemas on Resource Type Specs | Configure constraint visibility levels per profile | CAT-002, POL-003 | +| POL-008 | Constraint Type Registry | — | — | Register constraint types with OpenAPI v3 schemas; configure emittable_by/consumable_by; manage core and organization tiers | POL-001 | +| POL-009 | Evaluation Context and Multi-Pass Convergence | — | — | Configure max evaluation passes; monitor convergence; manage escalation for unresolvable conflicts | POL-001, POL-008 | +| POL-010 | Policy Templates | — | Contribute policy templates with parameterized Rego | Register templates (Gatekeeper ConstraintTemplate pattern); validate parameter schemas and constraint type references | POL-001, POL-008 | +| POL-011 | DCM Constraint Types Library | — | — | Manage auto-generated Rego library (data.dcm.constraint_types); sync with Constraint Type Registry | POL-008, POL-010 | +| POL-012 | Data-Driven Policy Matching | — | — | Configure match sources (request payload, operation context, evaluation context, entity metadata); validate match fields at activation | POL-001 | +| POL-013 | Lifecycle-Scoped Policy Evaluation | — | — | Configure lifecycle_scope per policy (which operation types trigger it); enforce profile minimums (fsi/sovereign require sovereignty policies on all operations); configure changed_field_filter for update/scale operations | POL-001, POL-012 | +| POL-014 | Override Policies | — | — | Author override policies targeting specific policies for defined scopes; enforce expiry and review dates; cannot target hard enforcement policies | POL-001, POL-003 | +| POL-015 | Exception Grants | — | — | Create time-bounded, scope-limited waivers with compensating controls; track usage count; enforce dual-approval for hard policies; manage renewal limits | POL-001, AUD-008 | +| POL-016 | Manual Override | Submit override request for blocked request with justification | — | Grant single-request overrides; enforce dual-approval for hard policies; manage override authority roles | POL-001, IAM-001 | +| POL-017 | Dual-Approval Escalation | — | — | Configure role separation requirements; manage approval workflow for hard policy overrides; enforce fsi/sovereign dual-approval on all overrides | POL-001 | +| POL-018 | Compensating Control Substitution | — | — | Define substitute control sets that satisfy policy intent through different mechanisms; require compliance officer validation | POL-001 | +| POL-019 | Override Approval Flow | Submit override request for blocked request; view pending overrides awaiting approval | — | Configure override notification routing (internal, webhook); configure timeout per profile; configure escalation rules; manage override authority roles | POL-016, IAM-001 | +| POL-020 | Override Notification Routing | — | — | Configure per-profile notification channels (LISTEN/NOTIFY, webhook to ServiceNow/Jira/Slack); configure routing by policy domain and enforcement level; configure escalation timeouts | POL-019 | +| POL-021 | Policy Block Resolution | View blocking details, resolution guidance with compliant values; choose resolution action (modify, request override, cancel, escalate) | — | Configure block timeout per profile; configure resolution guidance generation; manage escalation routing | POL-001 | + +--- + +## 8. Data Layer Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| LAY-001 | Core Layer Authoring | — | — | Author and manage Core and Organizational Layers in GitOps | IAM-003 | +| LAY-002 | Service Layer Contribution | — | Contribute Service Layers for offered resource types | Manage layer compatibility declarations | PRV-001, LAY-001 | +| LAY-003 | Layer Cache Management | — | — | Manage Layer Cache synchronization; handle cache invalidation | LAY-001, LAY-002 | +| LAY-004 | Layer Exclusion | Declare layer exclusions on specific requests | — | Configure which layers may be excluded; manage non-excludable declarations | REQ-003 | +| LAY-005 | Layer Versioning and Lifecycle | — | — | Manage layer versions; handle deprecation; enforce immutability | LAY-001 | + +--- + +## 9. Information and Data Integration + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| INF-001 | Information Provider Registration | — | Register Information Provider; declare authority scope and schema | Configure Information Provider Registry; manage authority layers | IAM-001 | +| INF-002 | Information Provider Push | — | Push field value updates to DCM; respond to conflict notifications | Configure ingestion pipeline; manage conflict resolution policies | INF-001 | +| INF-003 | Information Provider Pull / Discovery | — | Expose data query endpoint for DCM pull operations | Configure pull schedules; manage cache TTLs | INF-001 | +| INF-004 | Write-Back | — | Implement write-back endpoint to receive DCM-initiated updates | Configure write-back triggers via policy | INF-001, INF-002 | +| INF-005 | Confidence Score Visibility | View confidence bands on entity field values; query confidence aggregation API | — | Configure confidence scoring formula; manage trust score thresholds | INF-001 | +| INF-006 | Conflict Resolution Management | — | — | Review and resolve contested field values; manage conflict escalation | INF-002 | + +--- + +## 10. Ingestion and Brownfield Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ING-001 | Resource Discovery and Ingestion | — | Expose discovery endpoints for brownfield resources | Configure ingestion pipeline; manage __transitional__ Tenant | DRF-001 | +| ING-002 | Ingested Entity Review | — | — | Review ingested entities; resolve conflicts; promote to active Tenants | ING-001 | +| ING-003 | Bulk Promotion | — | — | Execute bulk entity promotions with preview and rollback | ING-002 | +| ING-004 | Catalog Item Association | — | — | Associate ingested entities with Resource Type Specs; create catalog items | ING-002, CAT-001 | + +--- + +## 11. Audit and Compliance + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| AUD-001 | Audit Trail Access | Query audit records for own resources | — | Configure Audit Store; manage retention policies | IAM-003 | +| AUD-002 | Compliance Reporting | — | — | Generate compliance reports; manage report schedules | AUD-001 | +| AUD-003 | Merkle Tree Verification | — | — | Run inclusion proofs, consistency proofs, request chain verification; manage integrity incidents | AUD-001 | +| AUD-004 | Cross-DCM Audit Correlation | — | — | Correlate audit records across DCM instances via correlation_id; authorize cross-DCM pulls | AUD-001, DCM-001 | +| AUD-005 | Audit Record Retention Management | — | — | Configure reference-based retention; manage post-lifecycle retention | AUD-001 | +| AUD-006 | Audit Granularity Configuration | — | — | Configure granularity level per profile (stage, mutation, field); enforce minimum for fsi/sovereign | AUD-001 | +| AUD-007 | Signed Tree Heads | — | — | Configure STH interval; manage audit signing keys; publish STH for external verification | AUD-001 | +| AUD-008 | Payload Chain of Custody | View chain-of-custody proof for own requests | Verify dispatched payload matches DCM's signed output | Verify full pipeline integrity via request chain verification API | AUD-003 | +| AUD-009 | Inter-Stage Verification | — | — | Configure verification mode per profile (synchronous, asynchronous, disabled) | AUD-001 | + +--- + +## 12. Observability and Operations + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| OBS-001 | Operational Dashboard | View health and status of own resources | — | Configure and manage observability dashboard | — | +| OBS-002 | Metrics and Telemetry Export | — | Expose resource-level metrics to DCM | Configure observability export; integrate enterprise observability platform, or deploy the packaged dcm-observability stack as the authoritative platform | — | +| OBS-003 | Curated Event Stream Subscription | Subscribe to observability event types via Message Bus | — | Configure event stream publication policies; manage subscriber roles | OBS-002 | +| OBS-004 | Alert and Notification Management | Receive resource and policy alerts via declared channels | — | Configure alert routing; manage notification channels and escalation | OBS-001 | + +| OBS-006 | SLA/SLO Declaration | View SLO status for owned resources (`GET /resources/{uuid}/slo-status`) | Declare resource_type SLOs in Resource Type Specification; report realization timing via callbacks | Configure SLO targets per resource type; view aggregate SLO performance report (`GET /admin/slo/report`) | RLM-001, LCM-001 | +| OBS-007 | SLO Breach Detection and Notification | Receive `slo.breach_approaching` and `slo.breach_detected` events | — | Configure SLO breach routing and escalation; review aggregate breach reports | OBS-006, EVT-001 | +| OBS-005 | Cost Analysis and Attribution | View cost estimates and actuals for owned resources | Provide cost metadata; report utilization | Configure Cost Analysis component; manage cost attribution policies | PRV-006 | +| OBS-008 | Group-Scoped Observability | View dashboards, reports, and alerts scoped to the business/operational groups (DCMGroup) own resources belong to | Attribute telemetry to entity UUIDs so group scoping resolves from resource definitions | Scope dashboards, reporting, alerting, and their management to DCMGroups; scoping derives from data in the resource definitions themselves (group membership, ownership), never side-channel configuration | OBS-001, OBS-002, OBS-004 | + +--- + +## 13. Storage and State Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| STO-001 | Data Store Management | — | — | Configure PostgreSQL data domains (intent, requested, realized, discovered); manage schema, RLS, and tenant isolation | — | +| STO-002 | Realized State Management | — | — | Configure realized_entities table; manage version retention and is_current flag | PRV-005 | +| STO-003 | Discovered State Management | — | — | Configure discovered_records table; manage retention policies per profile | DRF-001 | +| STO-004 | Search and Query | Use entity and catalog search | — | Configure materialized views and indexes; manage cache refresh | STO-001 | +| STO-005 | Backup and Recovery | — | — | Configure PostgreSQL backup (PITR); test recovery; verify audit hash chain integrity | STO-001 | +| STO-006 | Provenance Model Configuration | — | — | Select and configure provenance model (full_inline / deduplicated / tiered); manage tier transitions | STO-001 | +| STO-007 | Sovereignty Partitioning | — | Declare sovereignty constraints at registration | Configure separate PostgreSQL instances per sovereignty zone; manage cross-zone prohibition | STO-001, GOV-001 | +| STO-008 | Tenant-Scoped Storage Isolation | Data is isolated by tenant via RLS | — | Configure RLS policies per table; enforce STI-001 through STI-004 | STO-001, IAM-001 | +| STO-009 | Tenant-Scoped Encryption (fsi/sovereign) | — | — | Configure per-tenant AES-256-GCM encryption via secrets management; manage key rotation | STO-008, CPX-001 | +| STO-010 | Internal Secrets Management | — | — | Configure envelope encryption (KEK source: env var, K8s secret, or HSM); manage secrets table; optional Vault external backend | STO-001 | +| STO-011 | Pipeline Event Routing | — | — | Configure LISTEN/NOTIFY for pipeline events; optional Kafka for high-throughput; manage consumption tracking | STO-001 | +| STO-012 | Internal Authentication | — | — | Configure local actor accounts (argon2id hashes, DCM-issued JWT); optional OIDC/SAML external auth_provider | STO-001, IAM-001 | + +--- + +## 14. DCM Federation and Multi-Instance + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| FED-001 | DCM Provider Registration | Submit requests that are routed to peer DCMs | Register as DCM Provider in peer instances | Configure DCM Provider registrations; manage federation trust | PRV-001, IAM-001 | +| FED-002 | Federation Routing | Requests automatically routed to appropriate Regional/Sovereign DCM | Respond to reserve queries from Hub DCM | Configure federation placement policies; manage sovereignty pre-filters | FED-001, REQ-005 | +| FED-003 | Federation Trust Management | — | — | Manage mTLS certificates; monitor federation trust scores; handle cert rotation | FED-001 | +| FED-004 | Cross-DCM Drift Detection | Receive drift alerts for federated resources | Publish Discovered State events to federation Message Bus | Configure federated drift detection; manage alert-and-hold policies | FED-001, DRF-002 | +| FED-005 | DCM Export and Import | — | — | Export and import DCM state packages; verify import trust scores | STO-001, STO-002 | + +--- + +## 15. Platform Governance and Administration + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| GOV-001 | Tenant Management | — | — | Create, configure, and decommission Tenants; manage compliance overlays | IAM-007 | +| GOV-002 | Group Management | — | — | Create and manage DCM Groups; configure sovereignty rules; manage time-bounded memberships | IAM-003 | +| GOV-003 | Registry Management | — | Register and maintain Resource Type Specifications in organization registry | Manage registry sync; configure registry policies; manage Tier 3 types | PRV-001 | +| GOV-004 | Resource Type Lifecycle | — | Manage deprecation notices; declare successor types; maintain migration guidance | Enforce deprecation timelines; manage sunset periods | GOV-003 | +| GOV-005 | Platform Configuration Management | — | — | Manage platform-wide layers; configure profiles; manage deployment manifest | LAY-001, POL-005 | +| GOV-006 | Bootstrap and Self-Hosting | — | — | Manage DCM self-deployment; verify bootstrap manifest; handle repave scenarios | STO-001 | + +| GOV-008 | Tenant Onboarding Workflow | Trigger onboarding completion: receive `tenant.onboarding_complete` when first entity OPERATIONAL | — | Execute full provisioning sequence: tenant entity, default groups, quota, admin actor, GitOps namespace, audit stream; dispatch `tenant.created` and member invitation events | IAM-001, STO-008 || GOV-007 | Sovereign Deployment Management | — | — | Manage air-gapped DCM instances; configure signed bundle import; manage offline registry | FED-001, STO-001 | + +--- + +## 16. Accreditation Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ACC-001 | Accreditation Submission | — | Submit accreditation records (BAA, ISO 27001, FedRAMP, etc.) referencing external certificate evidence | Register accrediting bodies; configure minimum accreditation types per profile | PRV-001 | +| ACC-002 | Accreditation Review and Approval | — | Receive approval/rejection notification | Review submitted accreditations; verify certificate references; approve or reject via Admin API | ACC-001 | +| ACC-003 | Accreditation Lifecycle Monitoring | Receive notification when a provider's accreditation is nearing expiry or revoked | Renew accreditations before expiry; submit renewal documentation | Monitor expiry timelines; fire P90D renewal warnings; handle accreditation gaps | ACC-001 | +| ACC-004 | Accreditation Gap Response | Receive notification when a provider enters accreditation gap affecting owned resources | — | Configure Recovery Policy for accreditation gap events; manage affected entity remediation | ACC-003, POL-005 | +| ACC-005 | Data Classification Enforcement | Receive enforcement feedback when request payload contains data the selected provider cannot handle | Declare max_data_classification_accepted in capability registration | Configure classification immutability rules; manage phi/sovereign classification locks | ACC-001, PRV-001 | +| ACC-006 | DCM Deployment Accreditation | — | — | Register DCM deployment-level accreditations; expose to federation peers for trust verification | PRV-001, FED-001 | + +--- + +## 17. Zero Trust and Security Posture + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ZTS-001 | Mutual TLS Enforcement | All interactions authenticated via mTLS at the client side | Present valid mTLS certificate on every interaction; rotate certificates on declared schedule | Configure trust anchors; manage CA chain; enforce mTLS at all interaction boundaries | IAM-001 | +| ZTS-002 | Scoped Interaction Credentials | Receive scoped short-lived credentials for authorized operations | Validate credential scope before executing operations; reject out-of-scope credentials | Configure credential lifetime per profile; manage credential issuance via secrets management | IAM-001, PRV-001 | +| ZTS-003 | Certificate Rotation Management | — | Implement certificate rotation before expiry; use transition window to avoid downtime | Monitor certificate expiry; fire P14D rotation warnings; manage P7D transition window | ZTS-001 | +| ZTS-004 | Zero Trust Posture Configuration | — | — | Configure zero_trust_posture per profile (none/boundary/full/hardware_attested); manage posture overrides | POL-005 | +| ZTS-005 | Hardware Attestation (Sovereign Profile) | — | Present hardware-attested identity (TPM/HSM) for sovereign profile interactions | Configure hardware attestation requirements; manage HSM integration; enforce for sovereign profile | ZTS-001, ZTS-002 | + +| ZTS-007 | Provider OpenAPI Spec Signing (SEC-001) | — | Sign OpenAPI spec with mTLS private key at registration; rejected at GATE-SP-01 if unsigned | Verify signature during registration approval pipeline | PRV-001, ZTS-001 | +| ZTS-008 | GitOps Secrets Scanning (SEC-002) | Commits with detected secrets rejected with `SECRETS_DETECTED` audit record | Ensure service layer SCM does not contain plaintext secrets | Configure scanning ruleset; review and remediate detected secrets | GOV-001, AUD-001 | +| ZTS-009 | Software Bill of Materials (SBOM) Declaration (SEC-003) | — | Declare SBOM reference at registration (mandatory for fsi/sovereign) | Enforce SBOM requirement during registration approval for fsi/sovereign profiles | PRV-001, ACR-001 || ZTS-006 | Five-Check Boundary Enforcement | — | Pass all five boundary checks on every interaction: identity → authorization → accreditation → matrix → sovereignty | Monitor boundary check audit records; respond to INTERACTION_DENIED events | ZTS-001, ACC-001, GMX-001 | + +--- + +## 18. Unified Governance Matrix + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| GMX-001 | Governance Matrix Rule Authoring | — | — | Author governance matrix rules in GitOps; declare match conditions across four axes (subject/data/target/context); declare field permissions | POL-001 | +| GMX-002 | Boundary Enforcement Evaluation | Receive DENY response with governing rule_uuid and human-readable reason when a request crosses a prohibited boundary | Receive field-stripped or redacted payloads when STRIP_FIELD/REDACT decisions apply | Monitor GMX evaluation audit records; respond to DENY events | GMX-001, ZTS-006 | +| GMX-003 | Field-Level Data Control | Receive request feedback when specific payload fields are stripped or redacted by active matrix rules | Receive filtered payloads; handle missing optional fields gracefully | Configure allowlist/blocklist field permissions per rule; manage STRIP_FIELD vs REDACT vs DENY_REQUEST escalation | GMX-001 | +| GMX-004 | Sovereignty Zone Management | — | Declare operating sovereignty zones in provider registration | Register sovereignty zones; declare jurisdictions, regulatory frameworks, inter-zone agreements | PRV-001, GMX-001 | +| GMX-005 | Compliance Domain Matrix Activation | — | — | Activate compliance domain matrix rules (HIPAA, GDPR, etc.) by enabling compliance domain in profile; rules apply automatically | POL-005, GMX-001 | +| GMX-006 | Tenant and Resource-Type Matrix Overrides | — | — | Declare Tenant-level and resource-type-level matrix rules that tighten (never relax) platform defaults | GMX-001, GOV-001 | +| GMX-007 | Matrix Rule Lifecycle Management | — | — | Manage governance matrix rule lifecycle (developing → proposed → active); use shadow mode for safe validation before activation | GMX-001, POL-002 | + +--- + +## 19. Drift Reconciliation + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| DRC-001 | Drift Record Production | Receive drift records with field-level detail: realized value, discovered value, field criticality, severity, unsanctioned flag | — | Configure Drift Reconciliation Component; manage comparison algorithm and severity thresholds | DRF-001, PRV-005 | +| DRC-002 | Unsanctioned Change Classification | Receive elevated-severity alert when change has no corresponding Requested State record | — | Configure unsanctioned change detection; manage severity escalation rules | DRC-001 | +| DRC-003 | Drift Severity Classification | Receive severity-classified drift records (minor/significant/critical) based on field criticality × change magnitude | — | Declare field criticality in Resource Type Specifications; configure magnitude thresholds per profile | DRC-001, GOV-003 | +| DRC-004 | Drift Resolution Tracking | View drift record status (open/acknowledged/resolved/escalated); receive resolved notification when next discovery confirms clean state | — | Monitor drift resolution rates; configure escalation policies for aged-open drift records | DRC-001, DRF-004 | +| DRC-005 | Governance Matrix Drift Integration | — | — | Configure governance matrix check in drift comparison pipeline: expected provider changes are not flagged as drift | DRC-001, GMX-001 | + + +## 20. Federated Contribution Model + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| FCM-001 | Consumer Policy Authoring | Author and submit tenant-domain policies (GateKeeper, Transformation, Recovery, Lifecycle, Orchestration Flow, Governance Matrix rules) via API or Flow GUI; receive PR URL and shadow mode results | — | Configure consumer policy authoring permissions (policy_author role); manage review requirements per profile | POL-001, IAM-003, IAM-007 | +| FCM-002 | Provider Resource Type Publication | — | Publish Resource Type Specifications and Catalog Items for offered resource types via provider contribution API; receive registry PR for platform admin review | Manage provider contribution registry; configure review requirements for provider specs; manage Organization-tier registry | PRV-001, GOV-003 | +| FCM-003 | Provider Service Layer Contribution | — | Contribute Service Layers for offered resource types; layers applied during request assembly for all consumers requesting that resource type | Review and activate provider-contributed layers; manage layer compatibility | PRV-001, LAY-002 | +| FCM-004 | Consumer Resource Group and Definition Contribution | Author and manage resource groups, notification subscriptions, webhook registrations, and cross-tenant authorization records within own Tenant | — | Configure contribution permissions per role; manage Tenant-scoped artifact lifecycle | IAM-007, GOV-002 | +| FCM-005 | Federation Contribution (Peer DCM) | — | Peer DCM contributes registry entries, policy templates, and service layers via federation channels | Manage federation contribution trust posture (verified/vouched/provisional); configure review requirements per trust posture; manage cross-DCM artifact lifecycle | FED-001, GOV-003, POL-003 | +| FCM-006 | Contribution Review and Lifecycle | View contribution status (proposed, pending_review, active, withdrawn); withdraw a pending contribution; receive notification when contribution is approved or rejected | Receive notification when provider contributions are reviewed | Review and approve/reject contributions via Admin API; manage shadow review periods; assign new owners to orphaned artifacts | POL-002, POL-003 | +| FCM-007 | Contributor Scope Enforcement | Receive clear DENY response when attempting to contribute outside permitted domain scope | Receive DENY when contributing specs for resource types not offered | Monitor Governance Matrix enforcement at contribution time; configure scope violation audit and notification | GMX-001, GMX-002 | + +--- + + +## 21. Scoring Model + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| SMX-001 | Operational GateKeeper Scoring | Receive risk score and score_drivers with request acknowledgment; understand why score is at its level | Declare `enforcement_class: operational` and `scoring_weight` on contributed GateKeeper policies | Configure operational GateKeeper policies with appropriate weights; manage per-policy enforcement class | POL-001, REQ-004 | +| SMX-002 | Advisory Validation and Completeness Score | Receive advisory_warnings list with request acknowledgment; understand what optional improvements exist | Declare `output_class: advisory` on advisory Validation policies | Configure advisory Validation policies; manage completeness score thresholds | POL-001, REQ-004 | +| SMX-003 | Actor Risk History Tracking | View own risk history score and contributing events via Consumer API | — | Monitor actor risk history; reset scores for trusted automation accounts; configure decay parameters | AUD-001, IAM-001 | +| SMX-004 | Quota Pressure Scoring | Receive quota_pressure as a score driver when approaching Tenant quota limits | — | Configure per-resource-type quota limits; manage free_threshold parameter | IAM-007, REQ-004 | +| SMX-005 | Provider Accreditation Richness Scoring | — | Benefit from lower risk contribution by maintaining rich accreditation portfolio | Configure accreditation richness weights; manage portfolio scoring | ACC-001, PRV-001 | +| SMX-006 | Profile Scoring Threshold Management | — | — | Configure approval routing thresholds per profile (auto/reviewed/verified/authorized + custom tiers via named-tier list); manage signal weights; enforce SMX-008 (max auto_approve_below: 50) | POL-005, REQ-004 | +| SMX-007 | Policy Enforcement Class Override | — | Contribute policies with declared enforcement_class; receive notification when profile overrides enforcement class | Declare per-profile enforcement class overrides; manage regulatory_mandate flag to protect compliance-class policies from demotion | POL-004, POL-005 | +| SMX-008 | Score Audit Trail | Query risk score and routing decision for own requests; view score_drivers and advisory_warnings | — | Query full Score Record detail including signal breakdown and actor risk history; manage score audit retention | AUD-001, REQ-004 | +| SMX-009 | Scoring Weight Range Enforcement | — | Declare operational GateKeeper scoring_weight between 1 and 100 | Enforce weight range at policy activation; reject out-of-range weights | SMX-001 | +| SMX-010 | Score Breakdown Audit Inclusion | View score breakdown in request audit record | — | Configure score breakdown storage in Audit Store for all scored requests | SMX-001, AUD-001 | + +--- + + +## 22. Composite Service Composition + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| CMP-001 | Composite Service Request | Request a Composite Service as a single catalog item; receive composite entity UUID; track composite execution status via standard request status endpoint | Register Composite Service with constituent specification; implement standard Services API for `self` constituents | Configure Composite Service registration; manage composite catalog items | CAT-001, REQ-007, PRV-001 | +| CMP-002 | Dependency-Ordered Constituent Execution | — | Implement standard Services API for `self` constituents (DCM derives ordering from depends_on declarations) | Configure composition model; monitor execution round progress via status events | CMP-001, PRV-003 | +| CMP-003 | Partial Delivery and DEGRADED State | Receive DEGRADED composite entity when partial delivery is accepted; choose to accept or reject degraded state | Declare partial_delivery_supported and required_for_delivery per constituent; return standard realized state per constituent | Configure accept_degraded_delivery per profile; manage degraded notification urgency | CMP-001, PRV-005 | +| CMP-004 | Composite Compensation | Receive notification and recovery decision when a Composite Service fails; approve or reject compensation | Implement standard decommission for `self` constituents (DCM dispatches in dependency-reverse order); guarantee idempotent decommission calls | Configure compensation timeout; manage PARTIALLY_COMPENSATED orphan detection | CMP-001, LCM-007, DRC-001 | +| CMP-005 | Transparent Constituent Visibility | Query and manage DCM-visible constituent entities independently (when transparency mode); receive constituent-level drift alerts | Declare composition_visibility mode; register transparent constituents with deterministic UUIDs | Configure visibility mode per composite resource type; manage constituent entity lifecycle policies | CMP-001, DRF-001 | +| CMP-006 | Composite Execution Status Tracking | Monitor composite execution round progress via request status; see component-level status during long-running compositions | Send intermediate status events to DCM during execution; declare status_reporting.interval | Monitor composite execution health; configure execution timeout alerts | CMP-001, REQ-008 | +| CMP-007 | Nested Composite Service Composition | Request high-order Composite Services composed of other Composite Services (max depth 3) | Register a Composite Service whose constituents include other Composite Services; declare max_nesting_depth | Configure nesting depth limits; manage nested compensation chains | CMP-001, PRV-009 | +| CMP-008 | Composite Service Nesting Depth Enforcement | — | Declare nesting depth in Composite Service registration | Enforce maximum nesting depth of 3 at placement time; reject deeper compositions | CMP-001 | + +--- + + +## 23. Credential Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| CPX-001 | Resource Credential Issuance | Receive credential metadata and retrieval URL with realized resource; retrieve credential value via authenticated endpoint | Declare credential requirements in Resource Type Spec; receive credential issuance confirmation | Register service_provider with Credential.* resource types; configure credential types and lifetimes per resource type; manage issuance policies | PRV-005, ZTS-002 | +| CPX-002 | DCM Interaction Credential Issuance | — | Validate scoped interaction credential on every DCM dispatch; reject interactions without valid scoped credential (CPX-002) | Configure interaction credential lifetime per profile; manage secrets table for DCM-internal credentials | ZTS-002, PRV-001 | +| CPX-003 | Credential Rotation | Receive rotation notification before old credential expires; retrieve new credential during transition window | Implement rotate endpoint; honor transition window; notify DCM when rotation is complete | Configure rotation schedules and transition windows per credential type; manage pre-expiry rotation warnings | CPX-001, IAM-001 | +| CPX-004 | Emergency Rotation and Security Event Response | Receive immediate notification on emergency rotation; retrieve new credential via fastest channel | Implement immediate revocation with no transition window on security_event trigger | Configure security event triggers; manage emergency rotation audit trail; notify platform admin | CPX-003, OBS-004 | +| CPX-005 | Credential Revocation | Receive revocation notification when credentials are revoked (actor deprovisioned, entity decommissioned); confirm transition to new credential | Implement revoke endpoint with declared SLA; invalidate value immediately on emergency revocation | Manage Credential Revocation Registry; configure revocation cache TTL per profile (PT1M standard, PT30S fsi/sovereign); enforce CPX-007 (decommission blocks on credential revocation) | CPX-001, LCM-007 | +| CPX-006 | Revocation Propagation | — | Refresh revocation cache within profile-governed TTL; validate credential UUID against cache at use time (not only at receipt) | Configure revocation cache TTL; monitor revocation propagation latency; alert on SLA violations | CPX-005, IAM-001 | +| CPX-007 | Audit Trail for Credential Lifecycle | View own credential record history (issue, rotate, revoke events); every value retrieval audited with retrieval_uuid | — | Query full credential audit trail including retrieval count; manage credential audit retention | CPX-001, AUD-001 | +| CPX-008 | IP-Bound Credentials for fsi/sovereign | — | — | Enforce IP binding (bound_to_ip) on all credentials issued for fsi and sovereign profiles | CPX-001, ZTS-002 | +| CPX-009 | Algorithm and Key Usage Declaration | — | Declare algorithm and key_usage on credential records at issuance | Enforce declaration at issuance; reject credentials without declared algorithm | CPX-001 | +| CPX-010 | Idle Credential Detection | Receive notification when credential reaches idle threshold | — | Configure idle_detection_threshold per profile; enforce alert-only action | CPX-001 | +| CPX-011 | Compliance Domain Additive Credential Requirements | — | — | Enforce additive credential requirements when compliance domains are active on a profile | CPX-001, POL-005 | +| CPX-012 | Credential Value Store Isolation (All Profiles) | — | — | Enforce credential values never stored in DCM stores in ALL profiles including minimal | CPX-001 | + +--- + + +## 24. Authority Tier Model + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ATM-001 | Authority Tier Registry | Reference approval decisions by tier name; tier weight resolved dynamically from ordered list | Implement approval workflows that reference tier names (not hardcoded weights) | Manage the ordered authority tier list; control tier positions and gravity values | POL-005 | +| ATM-002 | Custom Tier Definition | — | — | Contribute custom tiers between existing tiers; declare decision_gravity and dcm_gate semantics; requires verified-tier approval | ATM-001, FCM-001 | +| ATM-003 | Dynamic Threshold Configuration | View which tier an action routes to | — | Configure profile approval_routing as named-tier threshold list; adjust score ranges when new tiers inserted | ATM-001, SMX-001 | +| ATM-004 | Tier Registry Change Impact Detection | Receive notification when tier changes affect owned resources or pending approvals | — | Propose tier registry changes; receive tier impact diff report; review SECURITY_DEGRADATION and BROKEN_REFERENCE items; accept degradations via Admin API | ATM-001, AUD-001 | +| ATM-005 | Degradation Review Gate | — | — | Review and explicitly accept each SECURITY_DEGRADATION item before a registry change activates; provide compensating control rationale; must hold verified or authorized tier reviewer role | ATM-004, IAM-001 | +| ATM-006 | Profile Gap Detection | — | — | Receive PROFILE_GAP warnings when tier registry changes leave profile threshold lists incomplete; update threshold lists or acknowledge gap within approval window | ATM-003, ATM-004 | +| ATM-007 | Tier Registry Audit Trail | Query historical tier registry versions and impact reports | — | Access full audit trail of all tier registry changes: proposal, impact assessment, degradation acceptances, activation | ATM-004, AUD-001 | +| ATM-008 | Approval Record Tier Weight Snapshot | — | — | Store tier name and resolved weight at approval record creation; historical records retain weight for audit comparison across regime changes | ATM-001 | +| ATM-009 | Tier Registry Degradation Gate | Receive notification when tier change produces degradation affecting owned resources | — | Block tier registry activation on SECURITY_DEGRADATION items; require verified-tier acceptance per item | ATM-001 | +| ATM-010 | Broken Reference Gate | — | — | Block tier registry activation when BROKEN_REFERENCE items exist (removed tier still referenced in active config) | ATM-001 | +| ATM-011 | Tier Change Impact Report | — | — | Generate and store tier impact report in Audit Store at proposal and activation time | ATM-001, AUD-001 | +| ATM-012 | Profile Gap Warning on Tier Insertion | — | — | Detect PROFILE_GAP when new tier inserted but profile threshold list not updated; emit non-blocking warning | ATM-001, POL-005 | + +--- + + +## 25. Event Catalog + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| EVT-001 | Event Subscription | Subscribe to DCM events via notification service_provider or Message Bus; filter by event type, entity type, urgency; idempotency via event_uuid | Publish standard events when provider actions occur; use reverse-DNS prefix for non-standard events | Configure notification service_provider channels and audience routing | IAM-001, OBS-001 | +| EVT-002 | Request Pipeline Events | Receive real-time status of own requests (submitted → intent_captured → policies_evaluated → requires_approval → approved → dispatched → realized/failed) | — | Configure request event delivery per profile; manage urgency routing | REQ-001 | +| EVT-003 | Entity Lifecycle Events | Receive entity lifecycle events (realized, state_changed, ttl_warning, decommissioning, etc.) for owned entities and entities with stakes | — | Configure entity event delivery; manage stakeholder audience routing | LCM-001 | +| EVT-004 | Security and Critical Events | Receive critical security events (audit chain alerts, sovereignty violations, unsanctioned provider writes) regardless of subscription preferences | — | Configure non-suppressable event delivery; manage security team routing | AUD-001, ZTS-001 | +| EVT-005 | Approval Pipeline Events | Receive approval events (requires_approval, decision_recorded, quorum_reached, window_expiring, expired) for own requests and approvals | — | Configure reviewer notification routing; manage approval window alerts | ATM-001, IAM-001 | +| EVT-006 | Provider and Infrastructure Events | — | Publish provider health events (registered, healthy, unhealthy, degraded); publish provider_update events on entity changes | Monitor provider health events; configure provider degradation alerts | PRV-001 | +| EVT-007 | Tier Registry and Governance Events | — | — | Receive tier_registry events (proposed, impact_assessed, degradation_detected, activated); configure governance event routing | ATM-004 | + +--- + + +## 26. API Versioning + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| VER-001 | Version Discovery | Discover available API versions and their status via `GET /.well-known/dcm-api-versions`; learn current, supported, and deprecated versions; get changelog and migration guide URLs | Declare supported OIS version in capability registration | Monitor version adoption; manage sunset schedules | — | +| VER-002 | Breaking Change Governance | Receive at least the profile-governed deprecation notice period before a breaking change takes effect; continue using deprecated versions until sunset date | Receive OIS version deprecation notice; migrate to new OIS version before sunset | Declare new major versions; configure deprecation timeline per profile; ensure VER-002 (breaking change definition) is applied | VER-001 | +| VER-003 | Deprecation Headers | Receive `Deprecation`, `Sunset`, and `Link` headers on all responses from deprecated API versions (RFC 8594/RFC 9745); use these to drive migration priority | — | Configure header injection for deprecated versions; ensure headers are accurate | VER-002 | +| VER-004 | Migration Guide | Access machine-readable migration guide at `GET /api/v{N}/migration-guide`; understand all breaking changes from previous version with migration instructions | Access OIS migration guide at `GET /provider/api/v{N}/migration-guide` | Maintain migration guides for all new major versions (required by VER-008) | VER-002 | +| VER-005 | Preview Endpoints | Access preview endpoints at `/api/v{N}/preview/`; understand stability commitment is none; provide feedback before graduation | — | Mark endpoints as preview; graduate to stable in new major version | VER-001 | +| VER-006 | Latest Alias Production Warning | — | — | Support `latest` version alias; emit response header discouraging production use | VER-001 | +| VER-007 | Preview Endpoint Instability Declaration | — | — | Mark preview endpoints explicitly; may change or be removed without major version bump | VER-001 | +| VER-009 | Provider Dispatch Compatibility | — | Maintain backward compatibility with DCM dispatch payloads from supported prior versions | Manage provider dispatch versioning; maintain supported version matrix | VER-001 | + +--- + + +## 27. Session Revocation + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| SES-001 | Session Lifecycle Management | View own active sessions; logout single session (DELETE /api/v1/auth/session); logout all sessions; revoke specific session by UUID | — | Force revoke sessions for any actor; view session store health | IAM-001, AUTH-001 | +| SES-002 | Actor Deprovisioning Session Revocation | — | — | Parallel session + credential revocation on actor deprovisioning; deprovisioning not acknowledged until both complete (AUTH-016) | SES-001, CPX-005 | +| SES-003 | Emergency Session Revocation | Receive critical notification on security-event session revocation | — | Trigger emergency revocation (security_event); revocation propagates within profile SLA (PT5S sovereign to PT30S standard) | SES-001, EVT-001 | +| SES-004 | Token Introspection | — | Call POST /api/v1/auth/introspect to validate bearer tokens without maintaining own revocation cache | Configure introspection endpoint access; manage introspection scope grants | SES-001, IAM-001 | +| SES-005 | Concurrent Session Enforcement | Oldest session auto-revoked when new session exceeds concurrent limit; receive notification via notification service_provider | — | Configure concurrent_sessions limit per profile; monitor session counts | SES-001 | +| AUTH-017 | Session Revocation Propagation SLA | — | — | Propagate revocation to Session Revocation Registry within profile SLA (PT5S sovereign → PT30S standard) | SES-001 | +| AUTH-018 | Per-Request Revocation Registry Check | — | Check Session Revocation Registry on each request bearing a bearer token | Configure revocation registry query path; manage registry availability | SES-001 | +| AUTH-019 | Emergency Revocation No-Grace Period | Receive critical notification on emergency revocation | — | Emergency session revocation fires immediately with no grace period | SES-003 | +| AUTH-020 | Introspection Endpoint Authentication | — | Authenticate introspection calls using provider interaction credential | Configure introspection scope grants; manage endpoint access | SES-004 | +| AUTH-021 | Oldest Session Revocation on Limit | Receive notification when oldest session is auto-revoked | — | Configure concurrent_sessions limit; enforce oldest-first revocation order | SES-005 | +| AUTH-022 | Refresh Token Invalidation on Session Revoke | — | — | Invalidate refresh token when parent session is revoked; return REVOKED_SESSION error on use | SES-001 | + +--- + +## 28. Internal Component Authentication + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ICOM-001 | Component Identity and mTLS | — | — | Manage Internal CA; issue and rotate component certificates; all inter-component calls use mTLS (ICOM-001) | ZTS-001, CPX-001 | +| ICOM-002 | Component Bootstrap | — | — | Generate one-time bootstrap tokens (PT1H max lifetime); components acquire first certificate via bootstrap token; token invalidated after single use (ICOM-007) | ICOM-001 | +| ICOM-003 | Internal Call Authorization | — | — | Declare allowed_sources per internal endpoint; declare allowed_targets per component; unauthorized source calls rejected with ICOM_UNAUTHORIZED_SOURCE audit record (urgency: high) | ICOM-001, AUD-001 | +| ICOM-004 | Internal Interaction Credentials | — | — | Every internal call presents a scoped ZTS-002 interaction credential in addition to mTLS; credential scoped to specific operation and target component | ICOM-001, CPX-002, ZTS-002 | +| ICOM-005 | Component Certificate Revocation | — | — | Compromised component certificates added to Internal CA CRL immediately; CRL cache refresh within profile SLA (PT15S sovereign to PT1M standard); ICOM_CERT_COMPROMISED audit record (urgency: critical) | ICOM-001, AUD-001 | +| ICOM-006 | Component Certificate Maximum Validity | — | — | Issue internal component certificates with maximum validity P90D; enforce expiry and rotation | ICOM-001 | +| ICOM-008 | Compromised Certificate Immediate CRL | — | — | Add compromised internal component certificates to Internal CA CRL immediately; propagate within PT60S | ICOM-001 | +| ICOM-009 | Trust Anchor Registration | — | — | Register root or intermediate CA as trust anchor for internal mTLS; reject certificates not chaining to registered trust anchor | ICOM-001 | + +--- + + +## 29. Scheduled and Deferred Requests + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| SCH-001 | Request Scheduling | Submit requests with schedule.dispatch: at/window/recurring; SCHEDULED requests visible in GET /api/v1/requests; cancellable before dispatch; receive request.scheduled event | — | Manage Maintenance Windows; configure Request Scheduler; monitor scheduled queue depth | REQ-001 | +| SCH-002 | Maintenance Windows | Reference maintenance windows in scheduled requests; view available windows at GET /api/v1/maintenance-windows | — | Create/manage/suspend maintenance windows; approve window schedules; configure platform-wide windows | SCH-001, GOV-001 | +| SCH-003 | Dual Policy Evaluation | — | — | Understand that scheduled requests run GateKeeper at declaration AND at dispatch; dispatch-time failure → FAILED with schedule_policy_rejection (SCH-003) | SCH-001, POL-001 | +| SCH-004 | Deadline Enforcement | Set not_after on scheduled requests; receive request.failed(schedule_deadline_missed) if deadline passes without dispatch | — | Monitor deadline miss rates; configure alerting on deadline misses | SCH-001, EVT-001 | +| SCH-005 | Not-After Expiry Failure | Receive FAILED status when scheduled request expires before dispatch | — | Configure not_after enforcement; manage SCHEDULE_EXPIRED recovery policy | SCH-001 | +| SCH-006 | Maintenance Window Platform Authorization | — | — | Require platform_admin or tenant_admin authority to create/modify Maintenance Windows | SCH-004 | + +--- + +## 30. Request Dependency Graph + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| RDG-001 | Dependency Group Submission | Submit POST /api/v1/request-groups with requests and depends_on declarations; local refs within submission; receive group_uuid and per-request entity_uuids | — | Monitor group queue depth; configure max group size | REQ-001 | +| RDG-002 | Field Injection | Declare inject_fields to pass realized output fields (e.g. IP address) from dependency into dependent request fields automatically at dispatch time | — | — | RDG-001, REQ-001 | +| RDG-003 | PENDING_DEPENDENCY Status | Track dependent requests in PENDING_DEPENDENCY status; cancel pending requests individually or cancel whole group; receive request.pending_dependency and request.dependency_met events | — | Monitor PENDING_DEPENDENCY queue depth; detect stalled groups | RDG-001, EVT-001 | +| RDG-004 | Group Failure Handling | Configure on_failure: cancel_remaining or continue; group-level timeout; group status via GET /api/v1/request-groups/{uuid} | — | Monitor group failure rates | RDG-001 | +| RDG-005 | Group-Level Timeout Enforcement | Receive TIMEOUT failure when group-level timeout elapses | — | Configure group_timeout independent of individual request timeouts | RDG-001 | +| RDG-006 | Single Group Membership Enforcement | — | — | Reject attempts to add a request to more than one dependency group | RDG-001 | + +--- + +## 31. DCM Self-Health + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| HLT-001 | Liveness Probe | — | — | GET /livez: fast liveness check (PT5S max, no external calls); Kubernetes restarts pod on failure; unauthenticated | — | +| HLT-002 | Readiness Probe | — | — | GET /readyz: checks Session Store, Audit Store, Policy Engine, Message Bus, Auth Provider connectivity; Kubernetes removes from LB on failure; startup sequence observable via readyz | — | +| HLT-003 | Component Health Detail | — | — | GET /api/v1/admin/health: per-component status (pass/warn/fail), metrics, queue depths, provider/auth summary; admin auth required | IAM-001 | +| HLT-004 | Prometheus Metrics | — | — | GET /metrics: Prometheus scrape endpoint; request pipeline, policy, session, drift, provider, internal CA metrics | OBS-001 | +| HLT-005 | Prometheus Metrics Endpoint | — | — | Expose Prometheus-compatible metrics at GET /metrics including request throughput, store latency, policy eval time, provider health counters | HLT-001 | +| HLT-006 | Startup Readiness via /readyz | — | — | Report PASSING on /readyz only after all required stores available and bootstrap complete | HLT-001 | + +--- + + +## 32. Operational Reference + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| OPS-001 | Data Store Partitioning | — | — | Declare partitioning strategy in deployment manifest; execute tenant-shard, per-tenant, or time-based archiving migration; configure shard routing and mirror lag monitoring | STO-001 | +| OPS-002 | Store Migration | — | — | Execute dual-write migration between store implementations; maintain audit chain continuity across cutover; enforce profile-governed burn-in before source decommission | STO-001, AUD-001 | +| OPS-003 | Disaster Recovery | — | — | Execute scenario-specific recovery procedures (component/store/full-CP/repave); meet profile-governed RTOs (PT1M–PT15M component, PT5M–PT2H store, PT5M–PT30M full-CP); complete post-recovery validation checklist | HLT-001, AUD-001 | +| OPS-004 | Backup Management | — | — | Configure PostgreSQL backup schedules (PITR for all data domains); enforce P365D minimum audit retention | STO-001 | + +--- + + +## 33. Web Interfaces + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| GUI-001 | Consumer Portal — Catalog, Requests, Resources | Browse catalog; live cost estimate; submit requests with scheduling and dependency groups; live SSE status with constituent tracking; cross-resource drift report; consumer-scoped audit trail with correlation ID trace | — | — | CAT-001, REQ-001, SCH-001, RDG-001, EVT-002 | +| GUI-002 | Consumer Portal — Live Request Status | Real-time status via SSE stream (status_change, progress_updated, approval events, heartbeat); constituent status for composite requests; approval flow inline; fallback to polling | — | — | REQ-001, EVT-002 | +| GUI-003 | Consumer Portal — Resource Management | View/filter owned resources by state and type; lifecycle state badges; drift indicator; resource detail with Overview/Drift/Audit/Cost/Credentials/Relationships/Groups tabs; state-sensitive action buttons; bulk operations | — | — | LCM-001, DRF-001, AUD-001 | +| GUI-004 | Consumer Portal — Session and Security | View active sessions; revoke individual or all other sessions; step-up MFA prompt for gated operations; tenant context selector; role-gated navigation (hide not disable) | — | — | SES-001, IAM-001 | +| GUI-005 | Admin Panel — Platform Dashboard | Control plane component health grid; provider health summary; pending approvals count; open drift records by severity; request throughput; all driven by GET /api/v1/admin/health | — | Platform Admins, SREs configure dashboard widgets; role-gated sections | HLT-003 | +| GUI-006 | Admin Panel — Governance and Approvals | Approval queue (all tenants); approval detail with risk score breakdown; authority tier registry editor (drag-and-drop reordering, impact report visualization, degradation acceptance flow); scoring threshold editor (auto_approve_below ≤ 50 hard-stop) | — | Policy Owners and Platform Admins | ATM-004, SMX-001 | +| GUI-007 | Admin Panel — Audit and Compliance | Platform-wide cross-tenant audit trail; pre-built compliance reports (SOC 2, FedRAMP, HIPAA); audit chain integrity status; correlation ID trace; session and security event feed | — | Auditors, Security team, Platform Admins | AUD-001, SES-003 | +| GUI-008 | Provider Management — Common Shell | Overview, configuration, health history, audit trail, and notification tabs for all 11 provider types; provider owner role gates access; Platform Admins see all providers | — | Provider owners manage own providers; Platform Admins manage all | PRV-001, IAM-001 | +| GUI-009 | Provider Management — Type Extensions | Service Provider: capacity, managed entities, naturalization mapping, realization history; secrets management: inventory, rotation, revocation, external CA config, algorithm compliance; Auth Provider: session stats, SCIM sync, connection status; external policy evaluation: trust level, contribution pipeline | — | Provider owners access type-specific tabs for their provider type | GUI-008, PRV-001 | +| GUI-011 | RHDH Plugin Suite | Use DCM capabilities within Red Hat Developer Hub (RHDH) or Backstage via Dynamic Plugins (@dcm/backstage-plugin-*); no RHDH rebuild required for updates | — | Configure RHDH app-config.yaml with DCM connection; configure Dynamic Plugin loading | +| GUI-012 | Scaffolder Template Auto-Generation | DCM catalog items automatically generate Backstage Software Templates; new resource types appear as templates without UI code; field schema → JSON Schema → Scaffolder form | — | Configure @dcm/backstage-plugin-catalog-backend; template generation is automatic | +| GUI-013 | DCM Entity Provider | DCMService (catalog items) and DCMResource (realized entities) appear in RHDH Software Catalog; entities sync every PT5M; search-indexed; tenancy enforced via namespace | — | Service account credential configuration; sync interval configuration | +| GUI-014 | ITSM Integration Bridge | View ITSM references (ServiceNow, Jira) on resource entity Overview tab; link change records to DCM requests; see ITSM-sourced approval votes in request status; CMDB reference on entity pages | ITSM systems receive DCM lifecycle events via notification service_provider and call Admin API to record approval votes; CMDB sync via webhook subscription | Configure ITSM notification service_provider; map DCM event → ITSM action; configure CMDB field mapping | EVT-001, GUI-002 | +| GUI-010 | Unified Shell | Single DCM web application with role-gated surfaces: Consumer Portal (all actors), Admin Panel (platform roles), Provider Management (provider_owner role), Flow GUI link (policy_owner/sre); one login, one session; no separate applications | — | Platform Admins configure which surfaces are available | IAM-001, SES-001 | + +--- + + +## 34. ITSM Integration + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| ITSM-001 | ITSM integration Registration | — | Register as ITSM integration with declared capabilities (supported_actions, itsm_system, field_mapping_ref, cmdb_ci_type_map); implement standard OIS health check | Register ITSM integrations; review and approve ITSM integration registrations; configure inbound webhook authentication | PRV-001, CPX-001 | +| ITSM-002 | Outbound ITSM Record Creation | View ITSM references on resource entities (change request, incident, CMDB CI links with deep links to ITSM system) | Receive action requests from DCM; create/update records in ITSM system; return record ID for storage on entity | Configure ITSM Policies (create_change_request, create_incident, update_cmdb_ci); configure block_until_created for compliance gates | ITSM-001, POL-001 | +| ITSM-003 | Inbound ITSM Approval Routing | — | Verify HMAC signature on inbound webhook; forward ITSM approval decisions to DCM Admin API approval vote endpoint | Configure inbound webhook secret (secrets management); monitor approval routing from ITSM systems (ServiceNow CAB, Jira workflow) | ITSM-001, CPX-001, ATM-001 | +| ITSM-004 | ITSM Policy Authoring | — | — | Author ITSM Action policies with template expressions; configure shadow validation; configure on_failure behavior; use block_until_created for pipeline gates (with mandatory timeout per ITSM-005) | ITSM-001, POL-001 | +| ITSM-005 | CMDB Synchronization | View CMDB CI reference on resource entities; CI auto-created on realization, auto-retired on decommission | Receive create_cmdb_ci and retire_cmdb_ci actions; maintain dcm_entity_uuid correlation on CMDB CI records | Configure CMDB CI type mapping per resource type; monitor CMDB sync failures | ITSM-001, ITSM-002 | +| ITSM-006 | ITSM Field Mapping Declaration | — | — | Declare field mappings between DCM entity fields and ITSM CI types in ITSM integration config; validate against Resource Type Specs | ITSM-001 | +| ITSM-007 | ITSM Policy Template Expression Validation | — | — | Validate template expressions in ITSM Policy action_payload at policy activation; reject unresolvable expressions | ITSM-001, POL-003 | + +--- + + +## 35. Provider Callback Authentication + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------| +| PCA-001 | Two-Layer Provider Callback Authentication | — | Present valid mTLS certificate (Layer 1) and provider callback credential (Layer 2) on every call to DCM callback endpoints | Configure DCM CA trust anchor; issue provider callback credentials at activation; enforce both layers | PRV-001, ZTS-001 | +| PCA-002 | Provider Callback Credential Scope Enforcement | — | Use callback credential scoped to own provider_uuid only; cannot act on other providers | Enforce credential scope at validation; reject cross-provider credential use | PRV-001 | +| PCA-003 | Entity-Level Callback Authorization | — | Receive 403 ENTITY_NOT_OWNED_BY_PROVIDER when pushing state for entities not dispatched to this provider | Enforce per-call entity ownership check independent of credential validity | PRV-001, REQ-007 | +| PCA-004 | Scope Violation Auto-Suspension | Receive critical notification when owned provider is suspended due to scope violations | — | Auto-suspend provider and notify platform admin after 5 consecutive scope violations within PT1H | PRV-001, ZTS-001 | +| PCA-005 | Callback Credential Issued by secrets management | — | Retrieve callback credential via secrets management at activation; not directly from API Gateway | Issue callback credentials exclusively through secrets management; reject direct credential issuance requests | PRV-001, CPX-001 | +| PCA-006 | Registration Token Single-Use Enforcement | — | Use registration token for initial registration only; obtain callback credential after activation | Invalidate registration token after first successful use regardless of expiry timestamp | PRV-001 | +| PCA-007 | Sovereignty Change Re-Registration | — | Submit new registration with new registration token when sovereignty declaration changes | Require new registration token and approval pipeline for sovereignty declaration changes | PRV-001, GMX-004 | +| PCA-008 | Callback Credential Pre-Expiry Rotation | — | Implement credential refresh; receive new credential before old expires during transition window | Initiate rotation before expiry; maintain transition window (50% of credential lifetime) | PRV-001, CPX-001 | +| PCA-009 | IP-Bound Callback Credentials for fsi/sovereign | — | Present callback calls from declared bound_to_ip address for fsi/sovereign profiles | Enforce IP binding on callback credentials for fsi and sovereign profiles | PCA-001, CPX-008 | +| PCA-010 | All Inbound Provider Calls Produce Audit Records | — | — | Write audit record for every inbound provider call including rejected calls; no silent failures | PCA-001, AUD-001 | + +--- + + +## 36. Workload Analysis + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------|| WLA-001 | Automated Workload Classification | View workload profile for owned resources; see archetype, resource type match, confidence | Report resource metadata via discovery for classification input | Configure classification ruleset version; review low-confidence classifications manually | DRC-001, INF-001 | +| WLA-002 | Migration Readiness Scoring | View containerization score and migration blockers for owned resources | Report workload characteristics that inform migration scoring | Configure migration readiness thresholds; integrate MTA Information Provider | WLA-001 | +| WLA-003 | MTA Information Provider Integration | — | Implement workload_analysis information type OR delegate to MTA | Register MTA as Information Provider; configure analysis trigger policies | WLA-001, INF-001 | +| WLA-004 | On-Demand Re-Analysis | Request re-analysis when resource role changes (`POST /resources/{uuid}/workload-profile:analyze`) | — | Trigger re-analysis for any resource; override archetype manually with reason | WLA-001 | +| WLA-005 | WorkloadProfile Audit Chain | View analysis history for owned resources | — | Query full analysis history including superseded profiles | WLA-001, AUD-001 | +| WLA-006 | Low-Confidence Manual Review Gate | Receive notification when owned resource requires manual classification | — | Review and resolve low-confidence classifications; unblock ingestion | WLA-001, LCM-001 | + +--- + + +## 37. Accreditation Monitoring + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------|| ACM-001 | Tier 1 External Registry Verification | — | Declare `external_registry_id` at registration for FedRAMP, CMMC, ISO 27001 | Configure registry poll intervals; review status change alerts; manage `external_registry_id` accuracy | PRV-001, ACR-001 | +| ACM-002 | Tier 2 Document Currency Verification | — | Maintain current `certificate_ref` and `audit_report_ref` URLs pointing to valid, accessible documents | Configure `max_age` per framework; review document expiry alerts; upload new reports when notified | PRV-001, ACR-001 | +| ACM-003 | Tier 3 Contract Webhook Integration | — | Configure contract management webhook for BAA and DoD IL accreditations | Register contract system; configure inbound webhook credential; receive BAA/contract lifecycle events | PRV-001, ACR-001 | +| ACM-004 | Verification Staleness Enforcement | — | Ensure monitoring infrastructure can reach DCM to deliver verification events | Configure stale_after thresholds and stale_action per profile; enforce sovereign/fsi minimum tier requirements | ACM-001, ACR-001 | +| ACM-005 | Immediate Revocation on External Revoke | Receive notification when provider accreditation is revoked; understand service impact | — | Review `accreditation.status_changed` events; confirm immediate revocations; trigger recovery policy | ACM-001, ACR-001 | +| ACM-006 | Verification Currency in Scoring | — | Maintain verification currency to maximize accreditation richness score | Monitor verification multiplier impact on provider placement; prioritize externally verified providers | ACM-001, SMX-001 | +| ACM-007 | Manual Override in Air-Gapped Mode | — | — | Manually update `last_verified_at` with justification in air-gapped deployments; maintain audit trail of manual verifications | ACR-001, AUD-001 | + +--- + + +## 38. Location Topology Management + +| ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | +|----|-----------|---------|---------|---------------|-----------|| LOC-001 | Location Type Registry | Browse available location types (standard and custom) | Declare supported locations in provider registration capability declaration | Register custom location types; manage standard type definitions; deprecate types | PRV-001 | +| LOC-002 | Location Node Management | Browse available locations via `GET /api/v1/locations`; filter by resource type, data classification, sovereignty zone | Declare which location nodes (DC, Zone, etc.) the provider serves at registration | Create, version, and retire location layer instances via GitOps; update mutable capacity fields | LOC-001 | +| LOC-003 | Location Selection at Request Time | Submit `location_uuid` or `location_handle` with service request; select at any level (Country through Rack); DCM refines to specific DC at placement | — | Configure default location selection rules; enforce location-based placement policies | LOC-001, LOC-002 | +| LOC-004 | Location Layer Assembly | Transparent — full location context injected into payload automatically | Receive full location context in dispatch payload (location.country_code, location.zone_code, location.dc_code, etc.) | Configure layer assembly order; define location-level field overrides | LOC-002, DLM-001 | +| LOC-005 | Location-Based Sovereignty Enforcement | See sovereignty zone and data residency on each location node | Declare sovereignty capabilities per served location | Configure `max_data_classification` per location; enforce cross-border policies at location layer | LOC-002, GOV-001 | +| LOC-006 | Location Hierarchy Navigation | Browse parent/child location relationships; query ancestors of a selected node | — | Manage location hierarchy; validate acyclicity on layer submission | LOC-001, LOC-002 | +| LOC-007 | Custom Location Types | Use custom location types in selection (e.g., Fleet/Ship in Navy context) | Declare support for resources at custom location types | Register and manage custom types; define level insertion point in hierarchy | LOC-001 | +| LOC-008 | Location Capacity Visibility | See `capacity_status` (available/limited/full) and `providers_available` count per location | Report capacity scoped to location during reserve query | Update mutable capacity fields (e.g., rack_units_available) without a new layer version | LOC-002, PRV-001 | + +--- + + + + +## 39. Subscription Management (SUB) + +| ID | Capability | Consumer | Provider | Platform/Admin | Dependencies | +|----|-----------|----------|----------|----------------|-------------| +| SUB-001 | Create subscription | Submit subscription request with tier and terms | Acknowledge subscription, provision initial entities | Approve subscription creation | CAT-001, REQ-001, POL-001 | +| SUB-002 | Manage subscription lifecycle | Suspend, resume, cancel own subscriptions | Report managed entity status | Force-cancel, view all subscriptions | LCM-001, AUD-001 | +| SUB-003 | Change subscription tier | Request tier upgrade/downgrade | Adjust capacity for managed entities | Approve tier changes requiring admin | POL-001, POL-003 | +| SUB-004 | Subscription renewal | Approve/decline renewal; view renewal status | Honor renewed terms | Configure renewal policies | SCH-001, POL-001 | +| SUB-005 | Provider-originated updates | Approve/reject non-auto updates | Submit updates via standard callback | View update audit trail | PRV-003, AUD-001 | +| SUB-006 | Update channel management | Configure auto_apply per channel | Declare available update channels | Set organization-wide channel policies | POL-001, POL-003 | +| SUB-007 | Subscription cost attribution | View subscription cost | Declare cost model per tier | Configure cost allocation rules | — | +| SUB-008 | Entitlement enforcement | View remaining capacity | Stay within entitlement bounds | Override entitlements in exceptional cases | POL-002 | +| SUB-009 | Grace period management | View grace period; request extension | Continue health reporting during grace | Configure grace period defaults | LCM-001 | +| SUB-010 | Subscription audit trail | View own subscription audit history | — | View all subscription audit records | AUD-001, AUD-002 | + +## Capability Count Summary + +| Domain | Capabilities | +|--------|-------------| +| Identity and Access Management | 21 | +| Service Catalog | 7 | +| Request Lifecycle Management | 10 | +| Provider Contract and Realization | 16 | +| Resource Lifecycle Management | 7 | +| Drift Detection and Remediation | 5 | +| Policy Management | 7 | +| Data Layer Management | 5 | +| Information and Data Integration | 6 | +| Ingestion and Brownfield Management | 4 | +| Audit and Compliance | 5 | +| Observability and Operations | 8 | +| Storage and State Management | 8 | +| DCM Federation and Multi-Instance | 5 | +| Platform Governance and Administration | 7 | +| Accreditation Management | 6 | +| Zero Trust and Security Posture | 8 | +| Unified Governance Matrix | 7 | +| Drift Reconciliation | 5 | +| Federated Contribution Model | 7 | +| Scoring Model | 10 | +| Composite Service Composition | 8 | +| secrets management Model | 12 | +| Authority Tier Model | 12 | +| Event Catalog | 7 | +| API Versioning | 8 | +| Session Revocation | 11 | +| Internal Component Authentication | 8 | +| Scheduled and Deferred Requests | 6 | +| Request Dependency Graph | 6 | +| DCM Self-Health | 6 | +| Operational Reference | 4 | +| Web Interfaces | 14 | +| ITSM Integration | 7 | +| Provider Callback Authentication | 10 | +| Workload Analysis | 5 | +| Accreditation Monitoring | 6 | +| Location Topology Management | 7 | +| Subscription Management | 10 | +| **Total** | **311** | +--- + +## Dependency Map — Critical Path Capabilities + +These capabilities block the most downstream work and should be implemented first: + +``` +IAM-001 (Actor Authentication) + └── IAM-002 (Session Tokens) → IAM-003 (RBAC) → IAM-007 (Tenant Scope) + └── CAT-001 (Service Catalog) + └── REQ-001 (Submit Request) + └── REQ-002 (Intent State) → REQ-003 (Layer Assembly) + └── REQ-004 (Policy Evaluation) → REQ-005 (Placement) + └── REQ-007 (Provider Dispatch) + └── PRV-003 (Realization) → PRV-005 (Realized State) + └── LCM-001 (State Transitions) + └── DRF-001 (Discovery) → DRF-002 (Drift) + +PRV-001 (Provider Registration) — parallel critical path + └── PRV-002 (Naturalization) → PRV-003 (Realization) + └── PRV-006 (Capacity Reporting) → REQ-005 (Placement) +``` + +**Minimum viable DCM capability set (to demonstrate end-to-end lifecycle):** + +IAM-001 → IAM-002 → IAM-003 → IAM-007 → CAT-001 → REQ-001 → REQ-002 → REQ-003 → REQ-004 → REQ-005 → REQ-006 → REQ-007 → PRV-001 → PRV-002 → PRV-003 → PRV-004 → PRV-005 → LCM-001 → DRF-001 → DRF-002 → AUD-001 + +**21 capabilities for a functional end-to-end demonstration.** + +**Note:** FCM-001 through FCM-007 (Federated Contribution Model) are not on the critical path — they extend DCM's multi-user capabilities but are not required for the initial end-to-end lifecycle demonstration. + +--- + +*Document maintained by the DCM Project. For questions or contributions see [GitHub](https://github.com/dcm-project).* diff --git a/architecture/DISCUSSION-TOPICS.md b/architecture/DISCUSSION-TOPICS.md new file mode 100644 index 0000000..844e6f8 --- /dev/null +++ b/architecture/DISCUSSION-TOPICS.md @@ -0,0 +1,644 @@ +# DCM — Discussion Topics + +> **Status Update (2026-03):** The DCM architecture has been substantially completed. All original 19 architecture review items, 10 session-added items, and 21 community questions have been resolved. The items below represent topics for ongoing community discussion and future evolution. +> +> **Current state:** 0 unresolved architectural questions · 331 capabilities across 39 domains · 58 data model documents · 12 specifications · 4 OpenAPI schemas · 6 JSON schemas · unified provider model (5 capability types) · 2 policy evaluation modes · 9 control plane services +> +> **Foundation:** Three abstractions (Data · Provider · Policy) · Unified Governance Matrix · Federated Contribution Model · Full OPA/Rego validation complete + +## Open Community Discussion Topics + +### 1. Kubernetes / CNCF Strategy (community decisions) +- CNCF submission scope: Operator Interface Spec as specification project first; DCM project after Level 2 reference implementation +- SIG App Delivery and SIG Cluster Lifecycle engagement before Sandbox submission +- Named adopters and TOC sponsor: project team action items +- Level 2 conformance scope now formally defined + +### 2. Implementation Decisions (engineering decisions) +- KubeVirt reference implementation timeline → team estimates against defined Level 2 scope +- SDK language support beyond Go → community SDKs encouraged; Go SDK is reference implementation +- Non-Kubernetes container runtime support → implementation detail + +### 3. Future Evolution Topics +- Normative data specifications (JSON Schema / OpenAPI) — the code-generation layer +- AI/ML Provider type — as DCM becomes AI-ready, a dedicated ML workload provider type +- Billing Provider type — deeper integration with enterprise billing and showback systems +- CMDB Provider type — dedicated contract for CMDB integration +- Multi-cloud federation model — extending DCM federation to public cloud providers +- GitOps PR UX improvements — better tooling for policy review workflow +- **UDLM as the universal observability export** *(added 2026-06-07)* — make + UDLM-modeled data THE export surface for monitoring, alerting, logging, and + audit consumption: telemetry entities + the event catalog exposed with + discoverable schemas so *any* tool (metrics TSDB, log aggregator, SIEM, + alerting pipeline) subscribes through one uniform, policy-scoped, + retention-governed interface — no per-tool adapters. This is the UDLM goal + ("one universal way to export data, consumable by any tool") applied to the + observability domain; pairs OBS-002/OBS-003 with the UDLM event catalog and + schema-sharing contracts. Validation use case: + `dav/use-cases/observability/udlm-universal-telemetry-export.yaml`. + DCM does not have to be the arbiter of the data — but it MAY be: where no + platform exists or a canned solution is desired, a packaged + **dcm-observability** component serves as the authoritative + telemetry/monitoring platform (provider-contract §7 / PRV-007). + First consumer and reference-implementation test bed: the roadfeldt + homelab observability stack (`roadfeldt-observability`). +- **Brownfield inventory ingestion — adopt, don't recreate** *(added + 2026-06-07)* — migrating existing configuration-management estates (Ansible + inventory) into DCM. Patterns established from live homelab migrations: + (1) inventory groups → DCMGroups, host vars → entity attributes, with + brownfield provenance; (2) **adopt-in-place is a hard requirement** — a + Quay bucket migration to claim-based provisioning forced a data copy + because the provisioner could not adopt the existing bucket; at estate + scale that is prohibitive, so DCM ingestion must adopt resources where + they stand; (3) embedded plaintext credentials (real finding: inventory + carried jump-host/Pi/PiKVM passwords in cleartext) convert to vault-backed + credential resources and get flagged for rotation; (4) coexistence — + legacy-tool changes surface as drift, and ingestion stays reversible until + cutover. Validation use case: + `dav/use-cases/cross-domain/ansible-inventory-brownfield-ingestion.yaml`. + + +### 5. Kessel Integration (pre-implementation evaluation) +- Evaluation document written: `44-kessel-integration-evaluation.md` +- **Action required:** Discussion with Kessel development team to validate assumptions before any implementation work begins +- Key questions: API stability, sovereign/air-gapped deployment model, SpiceDB schema extensibility, resource type registry extensibility, HA/DR patterns +- **Do not implement until alignment confirmed** — document is for discussion only +- Two integration paths evaluated: Kessel Relations as Auth Provider (checks 1-2 of five-check model), Kessel Inventory as Discovered State data store +- 10 blocking items identified in doc 44 Section 10 + +### 4. Governance Questions +- Community governance model — how will the DCM project make decisions once public? +- Certified Profile Program — self-certified vs project-reviewed for compliance profiles +- Registry tier promotion — Tier 3 (Organization) to Tier 2 (Verified Community) pathway + +--- + +*See [00-foundations.md](https://github.com/croadfeldt/udlm/blob/main/foundations/foundations.md) for the three-abstraction model that resolved the major architectural questions.* + + +--- + +**Original discussion topics (archived):** + +# DCM — Topics for Discussion and Modification + +**Document Status:** 🔄 Active +**Purpose:** A living document capturing topics that require further discussion, design decisions that need revisiting, and new capabilities to be incorporated into the DCM architecture and data model. +**Process:** When a topic is resolved, move it to the appropriate architecture or data model document and mark it resolved here with a reference to where it was documented. + +--- + +## How to Use This Document + +- **Add** any topic that surfaces during design, review, or implementation that needs a decision or deeper discussion +- **Tag** each item with its area, priority, and status +- **Resolve** items by documenting the decision in the appropriate document and updating the status here +- **Never delete** resolved items — keep the full history for audit and traceability + +--- + +## Status Key + +| Status | Meaning | +|--------|---------| +| 🔴 Blocking | Must be resolved before dependent work can proceed | +| 🟡 Active | Under active discussion | +| 🟢 Resolved | Decision made — documented in referenced document | +| ⚪ Parked | Acknowledged but deferred — revisit later | + +--- + +## Priority Key + +| Priority | Meaning | +|----------|---------| +| P1 | Critical — affects foundational architecture | +| P2 | High — affects multiple components or documents | +| P3 | Medium — affects a specific component or document | +| P4 | Low — enhancement or refinement | + +--- + +## Open Topics + +--- + +### TOPIC-001 — Webhook Integration + +**Area:** Control Plane, Provider Contract, Egress +**Priority:** P2 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Notification Model (doc 23) supersedes standalone webhooks. Webhooks are one delivery channel within the notification model. Full event taxonomy in doc 33 (Event Catalog). See AI Prompt Sections 14, 25, 48. +**Raised:** 2026-03 + +#### Description + +DCM needs a webhook integration model that allows external systems to be notified of DCM events and state changes in real time. Webhooks are a standard integration pattern that complements the existing API-first model and Message Bus — they enable a push-based notification model for consumers, providers, and external systems that cannot or do not poll DCM. + +#### Use Cases + +**Consumer Notifications** +- Notify a consumer's CI/CD pipeline when a resource request transitions to REALIZED state +- Notify an application team when their Resource/Service Entity enters DEGRADED state +- Notify a Tenant owner when an ownership transfer is initiated or completed +- Notify consumers when a dependency graph node fails during realization + +**Provider Notifications** +- Notify a provider when a new request payload is dispatched to them +- Notify a provider when DCM initiates a discovery request +- Notify a provider when a decommission is requested for one of their entities + +**External System Integration** +- Notify an external ITSM system (ServiceNow, Jira) when a request is created, updated, or completed +- Notify a monitoring system when Entity lifecycle state changes +- Notify a FinOps platform when new Resource/Service Entities are realized or decommissioned +- Notify a compliance system when GateKeeper policies fire or sovereignty constraints are applied + +**Operational Notifications** +- Notify SRE teams when provider capacity falls below threshold +- Notify security teams when unsanctioned changes are detected +- Notify auditors when specific policy types are triggered + +#### Design Questions to Resolve + +1. **Webhook registration model** — how do consumers, providers, and external systems register webhooks with DCM? Is registration via the Consumer API, Provider Registration, or a dedicated Webhook API? + +2. **Event taxonomy** — what is the full list of events DCM can emit via webhook? Should this be an extensible registry similar to the Resource Type Registry? + +3. **Payload format** — should webhook payloads use the DCM unified data model format, or a simplified event notification format? Should the full state payload be included or just a reference + event type? + +4. **Authentication and security** — how does DCM authenticate outbound webhook calls? Options include: HMAC signatures, OAuth tokens, mTLS, API keys. How does the receiving system verify the webhook is genuinely from DCM? + +5. **Retry and reliability** — what is DCM's obligation if a webhook delivery fails? Should DCM retry? How many times? With what backoff strategy? What happens if a webhook endpoint is consistently unavailable? + +6. **Ordering guarantees** — are webhook events delivered in order? What happens if events arrive out of order at the receiving end? + +7. **Filtering** — can webhook registrations declare filters — only receive events of specific types, for specific Resource Types, for specific Tenants, or for specific Resource Groups? + +8. **Policy Engine integration** — should GateKeeper and other policy types be able to trigger webhook notifications as a policy action? This would make webhooks a first-class policy response alongside ALERT, REVERT, etc. + +9. **Provider webhook obligations** — should providers be required to support webhook endpoints as part of their Provider Contract? Or is webhook support optional for providers? + +10. **Tenant scoping** — should webhook registrations be scoped to a Tenant, meaning a webhook can only receive events for resources owned by the registering Tenant? Or should there be platform-level webhooks that span Tenants (for SRE/Audit personas)? + +11. **Webhook versioning** — as DCM evolves, webhook payload schemas will change. How are webhook payload versions managed? Should webhook registrations declare which payload schema version they expect? + +12. **Relationship to Message Bus** — DCM already has a Message Bus component. What is the distinction between webhook integration and Message Bus integration? Are webhooks the outbound consumer-facing layer on top of the Message Bus? + +#### Initial Design Thoughts + +Webhooks fit naturally as an **Egress capability** — they are outbound notifications from DCM to external systems, which is consistent with the existing Egress zone in the architecture (Messaging Protocol, Interoperability API). + +The webhook registration model should likely be part of the **Consumer API** for consumer-facing webhooks and part of the **Provider Registration** for provider-facing webhooks. + +Webhook events should be **typed and versioned** — consistent with DCM's universal versioning model. An event type like `entity.state.changed` should have a version, and webhook registrations should declare which version they support. + +Webhook payloads should carry **provenance information** — the event payload should include enough context to trace back to the originating request, entity, and policy that caused the event. This is consistent with DCM's auditability requirements. + +**Policy Engine integration** is particularly interesting — if the Policy Engine can fire webhooks as a response action, it enables real-time governance notifications without requiring consumers to poll DCM. This aligns with the DCM goal of getting actionable information to the right people as fast as possible. + +#### References +- [Resource/Service Entities](https://github.com/croadfeldt/udlm/blob/main/entities/resource-service-entities.md) — provider lifecycle events +- [Service Dependencies](https://github.com/croadfeldt/udlm/blob/main/entities/service-dependencies.md) — dependency failure notifications +- Architecture: Egress zone, Message Bus, API Gateway + +--- + +### TOPIC-002 — Intent Store and Intent Payload Structure + +**Area:** Data Model +**Priority:** P1 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Intent State fully specified in doc 02 (Four States), doc 04 (Examples), consumer-api-spec. Resource type referenced by FQN or UUID (both accepted). See AI Prompt Sections 4, 5, 80. +**Raised:** 2026-03 + +#### Description + +The Intent State is captured when a consumer submits a request but the exact structure of the Intent payload has not been formally specified. The Intent payload is the consumer's raw declared desire — what they asked for before any processing, enrichment, or policy application. It needs a formal definition that is consistent with the four-state model and the layering model. + +#### Questions to Resolve + +1. What fields are required in an Intent payload vs optional? +2. How does the Intent payload reference a Resource Type — by UUID, by fully qualified name, or both? +3. How does the Intent payload declare its Tenant membership? +4. How does the Intent payload declare dependency requirements at the intent level? +5. How does the Intent payload declare group memberships? +6. How is the Intent payload versioned — does it carry a version or is it always a snapshot? + +#### References +- [Context and Purpose](https://github.com/croadfeldt/udlm/blob/main/foundations/context-and-purpose.md) — four states +- [Layering and Versioning](https://github.com/croadfeldt/udlm/blob/main/foundations/layering-and-versioning.md) — Request Layer +- [Resource Grouping](https://github.com/croadfeldt/udlm/blob/main/entities/resource-grouping.md) — Tenant and group membership + +--- + +### TOPIC-003 — GateKeeper vs Validation Policy Distinction + +**Area:** Policy Engine +**Priority:** P2 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** GateKeeper has enforcement_class (compliance=boolean deny, operational=risk score). Validation has output_class (structural=boolean fail, advisory=completeness score). GateKeeper can both block AND modify. See doc 29 (Scoring Model), AI Prompt Sections 17, 61. +**Raised:** 2026-03 + +#### Description + +The distinction between GateKeeper and Validation policy categories needs better examples and a clearer formal definition. Both involve checking data against rules, but GateKeeper has override authority while Validation is pass/fail only. The boundary between them needs to be unambiguous. + +#### Questions to Resolve + +1. What is the precise trigger condition that makes a policy a GateKeeper vs a Validation policy? +2. Can a GateKeeper policy both block AND modify in the same execution? +3. Are there cases where Validation and GateKeeper would produce different outcomes for the same rule? +4. Should GateKeeper policies require explicit authorization (e.g., only CISO-owned policies can be GateKeeper)? + +#### References +- [Layering and Versioning](https://github.com/croadfeldt/udlm/blob/main/foundations/layering-and-versioning.md) — Policy Layer section + +--- + +### TOPIC-004 — Audit vs Observability Component Separation + +**Area:** Control Plane +**Priority:** P3 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Definitively separate (AUD-013). Opposite trade-offs: audit is 100% accuracy, append-only, 7+ year retention; observability is statistical sampling, downsampled, operational window. Cannot be combined. See doc 16, AI Prompt Section 37.3. +**Raised:** 2026-03 + +#### Description + +The original architecture documents noted that Audit and Observability were separated for a reason that was later forgotten. This needs to be formally resolved — are they truly separate components with distinct responsibilities, or should they be merged? + +#### Initial Thinking + +**Audit** — focused on compliance evidence and transaction traceability. Reads provenance data intrinsic to data objects. Produces compliance reports, audit trails, and interrogation capability for Auditors, Security teams, and SRE personas. Historical record oriented. + +**Observability** — focused on operational visibility — metrics, health, performance, real-time monitoring. Consumes provider lifecycle events, Entity state changes, and system health data. Operational present-state oriented. + +These are likely genuinely separate concerns. Audit is about what happened and why. Observability is about what is happening now. + +#### Questions to Resolve + +1. Are Audit and Observability separate Atomic Components with separate APIs? +2. Do they share a data store or maintain separate stores? +3. How do provider lifecycle events flow to both components? + +--- + +### TOPIC-005 — Message Bus Consumer Ingress Question + +**Area:** Control Plane, Consumer Ingress +**Priority:** P3 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Message Bus supports bidirectional integration. Inbound messages processed as authenticated API calls via registered webhook actor identity. Same Policy Engine evaluation as any API call. See doc 18, AI Prompt Section 25. +**Raised:** 2026-03 + +#### Description + +The original architecture documents raised an unresolved question: should the Message Bus be offered as consumer ingress (inbound) in addition to egress (outbound)? The API-first principle suggests it should be egress only, but there are valid integration scenarios where external systems need to push data into DCM asynchronously. + +#### Questions to Resolve + +1. Should consumers be able to submit requests via the Message Bus, or only via the Consumer API? +2. If Message Bus ingress is supported, how are requests authenticated and authorized? +3. How does Message Bus ingress interact with the Intent State capture — is the message treated as an Intent payload? + +--- + +### TOPIC-006 — Cache Architecture + +**Area:** Data Model, Control Plane +**Priority:** P2 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Hybrid push-pull model. Caches placed closest to consumption, subject to sovereignty. GitOps and Event Streams always authoritative; caches are derived projections. Sovereign DCM uses signed bundles only. See CACHE-001 through CACHE-004, AI Prompt Section 40.2-40.4. +**Raised:** 2026-03 + +#### Description + +Several unresolved questions exist about where data caches live, how they are synchronized, and which cache is authoritative when caches diverge. This has implications for distributed DCM deployments and sovereignty scenarios. + +#### Questions to Resolve + +1. Where should data caches live? Hub DCM? Regional DCM? Sovereign DCM? All locations? +2. Should cache synchronization be push, pull, or both? +3. Which cache is authoritative when caches diverge? +4. What mechanism maintains consistency across distributed caches? +5. How do cache architecture decisions interact with sovereignty constraints — can cached data cross sovereignty boundaries? + +--- + +### TOPIC-007 — Cross-Tenant Dependencies + +**Area:** Data Model, Multi-Tenancy +**Priority:** P2 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Relationship nature governs cross-tenant: constituent=never, operational=dual authorization, informational=unless deny_all. explicit_only is default stance. cross_tenant_authorization DCMGroup required. See doc 09, REL-010/011/012, XTA-001-005, AI Prompt Sections 11.13-11.14, 21.6. +**Raised:** 2026-03 + +#### Description + +The dependency model currently assumes dependencies are resolved within a single Tenant. However, real-world deployments will have cross-tenant dependencies — a Payments Tenant application that depends on a shared DNS service owned by a Platform Tenant. The data model needs to formally address how cross-tenant dependencies are declared, resolved, and governed. + +#### Questions to Resolve + +1. How is a cross-tenant dependency declared — does it reference the Tenant UUID of the dependency owner? +2. What authorization is required for a cross-tenant dependency? Does the owning Tenant need to approve? +3. How does cost attribution work for cross-tenant service consumption? +4. How does drift detection work when a dependency is in another Tenant? +5. Can a GateKeeper policy block cross-tenant dependencies? + +--- + +### TOPIC-008 — Provider Trust Validation Mechanism + +**Area:** Service Providers, Provider Contract +**Priority:** P1 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Registration spec defines full approval pipeline (auto/reviewed/verified/authorized). Provider Callback Auth (doc 43, PCA-001-010) specifies two-layer mTLS + credential model. Trust levels: trusted/verified/untrusted. Accreditation model (doc 26) governs ongoing trust. See AI Prompt Sections 53.2, 76. +**Raised:** 2026-03 + +#### Description + +The Provider Contract includes a Trust Contract — providers must be validated and certified to participate in the DCM ecosystem. The mechanism for establishing, maintaining, and revoking trust has not been designed. + +#### Questions to Resolve + +1. What is the certification process for a new provider? +2. What technical mechanism validates trust at request time? +3. How is trust revoked if a provider violates their contract? +4. Should trust be per-provider or per-catalog-item? +5. How does the trust chain interact with sovereignty requirements? + +--- + +### TOPIC-009 — Physical Shared Infrastructure Tenancy + +**Area:** Data Model, Tenancy +**Priority:** P2 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** Three ownership patterns: Whole Allocation (indivisible, provider retains), Allocation (pool yields sub-resources), Shareable (one resource, multiple stakeholders). Physical infrastructure owned by __platform__ Tenant. See doc 04b, OWN-001 through OWN-008, AI Prompt Sections 8.3, 46.2. +**Raised:** 2026-03 + +#### Description + +The current model assigns every Resource/Service Entity to exactly one DCM Tenant. This works cleanly for most cases but edge cases exist for truly shared physical infrastructure — a rack, a network switch, a power circuit — where the ownership model may be genuinely joint or ambiguous. + +#### Questions to Resolve + +1. Is jointly-owned physical infrastructure a valid DCM use case, or is it always owned by a Platform/Infrastructure Tenant? +2. If joint ownership is valid, how is it modeled without breaking the single-Tenant rule? +3. Does the Whole Allocation model cover all physical infrastructure scenarios? + +--- + +### TOPIC-010 — Embedded Technology-Specific Data Bundles + +**Area:** Data Model +**Priority:** P3 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** native_passthrough field sanctioned for genuinely untranslatable provider-specific data. Always audit-logged (content if transparent, hash if opaque). Opaque passthrough blocked in fsi/sovereign by default. See DATA-001, AI Prompt Section 40.5. +**Raised:** 2026-03 + +#### Description + +The original data model discussion raised the question of whether the data model should allow embedded target-technology-specific data bundles — for example, a Terraform HCL block or an Ansible vars file embedded directly in an entity definition. The authors noted "not convinced this is something we want." + +#### Questions to Resolve + +1. Should technology-specific data bundles be allowed in entity definitions? +2. If allowed, must they be in clear text and appropriate for Git storage? +3. How would embedded bundles interact with the portability model — they would clearly be portability-breaking? +4. Is there a better mechanism — such as a provider-specific extension field with an appropriate portability classification? + +--- + +## Resolved Topics + +--- + +### TOPIC-R001 — Field Override Control Mechanism + +**Area:** Data Model, Policy Engine, Layering +**Priority:** P1 +**Status:** 🟢 Resolved +**Raised:** 2026-03 +**Resolved:** 2026-03 + +#### Decision + +Field override control is implemented as a **standard Policy Engine mechanism** using a graduated three-level model: Level 1 (no declaration — fully overridable), Level 2 (simple `override: allow|constrained|immutable`), Level 3 (full actor matrix with per-actor permissions, trusted grants, and expansion rules). The Policy Engine is the sole authority for setting override control. The Request Payload Processor enforces structural layer rules only. + +#### Documented In +- [Data Layers and Assembly Process](https://github.com/croadfeldt/udlm/blob/main/foundations/layering-and-versioning.md) — Section 5a +- [Context and Purpose](https://github.com/croadfeldt/udlm/blob/main/foundations/context-and-purpose.md) — Section 4.4 + +--- + +### TOPIC-R002 — Storage/Networking Bundling vs Dependency Model (Q53) + +**Area:** Data Model, Entity Relationships, Service Dependencies +**Priority:** P1 +**Status:** 🟢 Resolved +**Raised:** 2026-03 +**Resolved:** 2026-03 + +#### Decision + +The conflict between the enhancement documents (storage bundled in VM schema) and the data model (separate first-class entities) is resolved through a **universal Entity Relationship model**: + +- Bundled consumer declarations are expanded by the Request Payload Processor into first-class Resource/Service Entities with their own UUIDs +- The relationship between parent and child entities is expressed using the universal relationship model — bidirectional, UUID-keyed, with lifecycle policies +- The same relationship model is used for ALL entity relationships — compute to storage, application to web server, resource to business unit — minimizing variance +- Lifecycle policies (destroy|retain|detach|notify on parent destroy/suspend/modify) replace the ephemeral/persistent classification +- Expansion rules live in the Resource Type Specification — portable and declarative +- The dependency graph concept is unified into the Entity Relationship Graph + +#### Documented In +- [Entity Relationships](https://github.com/croadfeldt/udlm/blob/main/entities/entity-relationships.md) — complete relationship model +- [Information Providers](https://github.com/croadfeldt/udlm/blob/main/contracts/information-providers.md) — external data relationships +- [Service Dependencies](https://github.com/croadfeldt/udlm/blob/main/entities/service-dependencies.md) — updated to reference entity relationships + +--- + +### TOPIC-R003 — Information Provider Model + +**Area:** Data Model, Provider Contract +**Priority:** P1 +**Status:** 🟢 Resolved +**Raised:** 2026-03 +**Resolved:** 2026-03 + +#### Decision + +Information Providers are a first-class provider type in DCM. They follow the same registration, health check, trust, and contract model as Service Providers where applicable. Key decisions: + +- Information types live in the same DCM registry as Resource Types — distinguished by category prefix (Business.*, Identity.*, Compliance.*, Operations.*) +- Standard vs extended data — DCM only relies on standard fields for operational decisions; organizations can extend with domain-specific fields +- Stable external key model — DCM UUID wraps external UUID; if external system changes its UUID, only the reference record changes +- Three-mode verification — scheduled (Mode 1), provider push (Mode 2, contractual obligation), on-demand (Mode 3, fallback) +- Internal business data follows the standard resource entity model when DCM owns it; external references use the Information Provider model + +#### Documented In +- [Information Providers](https://github.com/croadfeldt/udlm/blob/main/contracts/information-providers.md) +- [Entity Relationships](https://github.com/croadfeldt/udlm/blob/main/entities/entity-relationships.md) — external relationship structure +- [Resource Type Hierarchy](https://github.com/croadfeldt/udlm/blob/main/entities/resource-type-hierarchy.md) — information type categories added + +--- + +--- + +### TOPIC-011 — Enhancement Document Compatibility Findings + +**Area:** Data Model, Provider Contract, Policy Engine, Catalog +**Priority:** P1 +**Status:** 🟢 Resolved +**Resolved:** 2026-03 +**Resolution:** All cross-cutting gaps resolved: field-level provenance (doc 00, Section 4.3), UUIDs (universal identity requirement), portability classification (doc 05), sovereignty declarations (doc A provider contract), tenancy (doc 15 universal groups), versioning (semver throughout), deprecation (five-status lifecycle). Enhancement docs superseded by current architecture. +**Raised:** 2026-03 + +#### Description + +A compatibility review of six DCM enhancement documents against the data model identified areas of strong alignment and cross-cutting gaps that need resolution before the enhancements can be considered fully aligned with the data model. + +#### Cross-Cutting Gaps (affect all or most enhancements) + +1. **Field-Level Provenance** — absent in all enhancement documents. The Policy Engine spec mutates request payloads without recording provenance. The KubeVirt SP returns minimal status rather than full DCM-format realized payloads. + +2. **UUIDs as Primary Identifiers** — most specs use `name` as the natural key rather than UUID as the primary key. The data model requires UUIDs as primary identifiers on all entities. + +3. **Portability Classification** — absent from all provider and catalog specs. Every field must declare `universal|conditional|provider-specific|exclusive` classification. + +4. **Sovereignty Declarations** — entirely absent from SP Registration and KubeVirt SP. Provider registration must include sovereignty capability declarations as a contractual obligation. + +5. **Tenant Support** — explicitly deferred in V1 in the Policy Engine spec. The data model treats Tenant as mandatory and non-overridable. A clear migration path from V1 to Tenant support is needed. + +6. **Universal Versioning** — catalog items use `apiVersion: v1alpha1` rather than Major.Minor.Revision. All definitions require universal versioning. + +7. **Deprecation Model** — absent from all enhancement documents. All definitions require the `active → deprecated → retired` lifecycle. + +#### Document-Specific Gaps + +**SP Registration Flow:** +- Registration payload missing: sovereignty declarations, ownership model declaration, dependency declarations, capacity model mode declaration +- Uses `name` as natural key rather than UUID as primary key +- Capacity data (totalCpu, totalMemory) treated as static rather than dynamic — needs reconciliation with three-mode capacity model + +**Service Type Definitions:** +- `providerHints` not formally marked as portability-breaking — silently bypasses portability enforcement +- Storage and networking bundled with compute — conflicts with the service dependency model which treats these as separate dependent services with their own lifecycle +- No UUIDs on field definitions, no provenance metadata, no versioning beyond v1alpha1 + +**Service Provider Health Check:** +- Covers liveness only — insufficient for full Provider Lifecycle Events contract +- Does not address DEGRADATION, MAINTENANCE, UNSANCTIONED_CHANGE, CAPACITY_CHANGE events +- Binary 200/non-200 model needs to coexist with structured event payload model + +**Policy Engine:** +- `selected_provider` as a direct policy output conflicts with the data model's specificity narrowing model for provider selection +- No policy versioning or GitOps integration — data model requires policies maintained via GitOps +- No provenance recording on field mutations — data model requires provenance on every modification +- Constraint immutability model is compatible and can be mapped to `override_preference` field metadata + +**Catalog Item Schema:** +- No UUIDs, no Tenant scoping, no portability classification, no deprecation model, no provenance +- `editable` field concept is valuable and not explicitly covered in data model — worth incorporating into the Resource Type Specification field definition +- `dependsOn` conditional field pattern maps to `conditional` portability classification but needs formal alignment + +**KubeVirt Service Provider:** +- Realized payload returns minimal status rather than complete DCM-format payload — Denaturalization requirement not met +- No unsanctioned change detection — only VMI phase changes reported +- Registration gaps same as SP Registration Flow spec +- `namespace` in response payload is provider-native concept with no DCM equivalent + +#### Questions to Resolve + +1. Should the enhancement documents be updated to align with the data model, or should the data model be adjusted where the enhancements reveal practical implementation constraints? +2. For the storage/networking bundling vs. dependency model tension — is this a V1 simplification that gets resolved in V2, or does the data model need a "monolithic service" concept? +3. For `selected_provider` in the Policy Engine — should provider selection remain a policy output, or should it be moved to a dedicated placement component that consumes narrowed field sets from the Policy Engine? +4. What is the migration path from V1 (no Tenant support) to Tenant-mandatory? +5. Should the `editable` field concept from the Catalog Item Schema be formally incorporated into the Resource Type Specification? + +#### References +- SP Registration: https://github.com/dcm-project/enhancements/blob/main/enhancements/sp-registration-flow/sp-registration-flow.md +- Service Types: https://github.com/dcm-project/enhancements/blob/main/enhancements/service-type-definitions/service-type-definitions.md +- Health Check: https://github.com/dcm-project/enhancements/blob/main/enhancements/service-provider-health-check/service-provider-health-check.md +- Policy Engine: https://github.com/dcm-project/enhancements/blob/main/enhancements/policy-engine/policy-engine.md +- Catalog Item: https://github.com/dcm-project/enhancements/blob/main/enhancements/catalog-item-schema/catalog-item-schema.md +- KubeVirt SP: https://github.com/dcm-project/enhancements/blob/main/enhancements/kubevirt-sp/kubevirt-sp.md + +--- + +Copy the following template and fill in the fields: + +```markdown +### TOPIC-NNN — Title + +**Area:** +**Priority:** +**Status:** 🟡 Active +**Raised:** + +#### Description + + + +#### Questions to Resolve + +1. + +#### References +- +``` + +--- + +*This document is maintained by the DCM Project team. Add topics freely — no topic is too small if it needs a decision.* + +--- + +### 6. Universal Lightspeed Interface for Operations (future concept) + +**Note captured for future exploration.** + +A universal operations interface concept — working title "Lightspeed" — for DCM. Intent is a unified, high-velocity operational surface for all DCM actions regardless of the underlying provider, resource type, or lifecycle stage. Think of it as the operational equivalent of what the unified data model is to data: a single consistent interaction model for humans and automation alike across the full DCM estate. + +**Initial thoughts to explore:** +- Single interface for any operation on any resource managed by DCM — no provider-specific tooling, no context switching +- "Lightspeed" implies minimal friction: operations that currently take multiple steps, approvals, and tool handoffs should be expressible and executable in a single interaction +- Applicable to both human operators (Web UI / CLI) and automation (agentic workflows, AIOps) +- Likely surfaces DCM's existing policy engine, lifecycle model, and unified data model as the execution layer — the interface is the innovation, not new backend capability +- Could be the primary surface for DCM's "AI Ready" design principle (README): agentic workflows operating over DCM's control plane via a natural-language-capable interface that still enforces all policy, sovereignty, and accreditation constraints + +**Questions to answer when this gets scoped:** +- Is this a new GUI surface, a CLI, an AI agent interface, or all three? +- How does it relate to the existing Web UI spec and Flow GUI spec? +- What does "lightspeed" mean operationally — sub-second execution, zero-confirmation for pre-approved patterns, predictive pre-staging? +- How does it interact with the Authority Tier model — can it auto-route approval gates without interrupting the operator's flow? +- Is this the primary interface for the AIOps layer referenced in the README? + +**Status:** Concept note — no design work started. Capture for roadmap planning. + +### 7. Spectral AEP Linter + CI Workflow Adoption (PR #18 review) + +**Source:** [PR #18](https://github.com/dcm-project/dcm/pull/18) by Fale — first draft of Interoperability API and Service Spec design guidelines. + +**PR #18 disposition:** The interoperability API and object design RFC are superseded by the current architecture (4 canonical OpenAPI specs, 58 data model docs). Several conventions in PR #18 (camelCase fields, PascalCase enums, YAML content type, K8s apiVersion/kind/spec/status structure) contradict the team's established patterns (snake_case fields, UPPER_SNAKE_CASE enums, JSON, flat REST entities). PR should be closed with acknowledgment and pointer to current architecture. + +**What to adopt:** The `.spectral.yaml` AEP linter config and the GitHub Actions CI workflow that validates OpenAPI specs against AEP rules. This enforces PRR-006 (AEP.DEV Linting) in CI. + +**Blockers before adoption:** +- The team's greenfield specs use OpenAPI 3.1.0; the AEP Spectral ruleset was built for 3.0.x. Must test for false positives on 3.1 constructs before enabling CI enforcement. +- Determine scope: lint only greenfield specs in `src/api/v1alpha1/`, or also the canonical specs in `dcm/schemas/openapi/`? +- Decide if this goes in dcm-examples only, or also in the dcm architecture repo for the canonical specs. + +**Status:** Future — pending OpenAPI 3.1 compatibility verification. + diff --git a/architecture/dcm-pattern-catalog-overlay.md b/architecture/dcm-pattern-catalog-overlay.md new file mode 100644 index 0000000..9ade886 --- /dev/null +++ b/architecture/dcm-pattern-catalog-overlay.md @@ -0,0 +1,408 @@ +# Deployment Pattern Catalog — How It Overlays on DCM + +**Date:** April 2026 +**Status:** Architecture Overlay — For Team Discussion + +--- + +## What a Deployment Pattern Is + +A deployment pattern is a **reusable, provider-agnostic blueprint** that defines a collection of resources, their dependencies, their runtime wiring, and their operational policies — together delivering a service that no single provider offers. + +Examples: + +| Pattern | Constituents | What it delivers | +|---------|-------------|-----------------| +| **Standard Web Application** | 2 app server VMs, 1 PostgreSQL DB, 1 load balancer, 1 network segment, 3 DNS records | A production web app with HA, monitoring, and DNS | +| **Secure Data Pipeline** | 1 Kafka cluster, 2 worker VMs, 1 S3-compatible store, 1 network policy, 1 encryption key | An encrypted ingest pipeline with data residency controls | +| **Developer Sandbox** | 1 VM, 1 namespace, 1 ephemeral DB, 1 port forward | A disposable dev environment with TTL auto-cleanup | +| **Regulated Database Service** | 1 PostgreSQL cluster (HA), 1 backup schedule, 1 encryption key, 1 audit log sink, 2 DNS records | A database meeting FSI data handling requirements | +| **Edge Compute Node** | 1 bare metal host, 1 MicroShift cluster, 1 VPN tunnel, 1 monitoring agent, 1 cert | A self-contained edge node with central management | + +The key property: **no single provider owns the pattern.** The PostgreSQL DB might come from one provider, the VMs from another, the load balancer from a third, and the DNS records from a fourth. The pattern defines *what* is needed and *how the pieces connect* — DCM figures out *who* provides each piece. + +--- + +## The Layering Model + +``` +┌─────────────────────────────────────────────────────────┐ +│ PATTERN CATALOG │ +│ Curated library of reusable deployment blueprints │ +│ "Standard Web App" · "Data Pipeline" · "Dev Sandbox" │ +│ │ +│ Authored by: Platform Engineers │ +│ Consumed by: Consumer Developers │ +│ Stored in: Resource Type Registry (composite types) │ +├─────────────────────────────────────────────────────────┤ +│ SERVICE CATALOG │ +│ Provider-specific offerings of patterns + atomic items │ +│ "EU-WEST Web App — Standard" · "APAC VM — Large" │ +│ │ +│ Populated by: Providers (atomic) + Patterns (composite) │ +│ Consumed by: Consumer Developers │ +├─────────────────────────────────────────────────────────┤ +│ DCM CONTROL PLANE │ +│ Decompose → Policy → Placement → Dispatch → Audit │ +│ │ +│ Processes: Both atomic requests and pattern requests │ +│ Each constituent → full pipeline independently │ +├─────────────────────────────────────────────────────────┤ +│ SERVICE PROVIDERS │ +│ VM · Network · Database · DNS · Storage · Container │ +│ │ +│ Fulfill: Individual constituents of a pattern │ +│ Report back: Realized state per constituent │ +└─────────────────────────────────────────────────────────┘ +``` + +**The Pattern Catalog is not a new architectural component.** It is a curated view of the Resource Type Registry filtered to composite resource types. DCM already has all the machinery to execute patterns — the composite service model, dependency graphs, binding fields, and constituent dispatch. What the Pattern Catalog adds is the **curation and consumer experience layer** on top of that machinery. + +--- + +## How a Pattern Maps to DCM Constructs + +A single deployment pattern maps to these existing DCM concepts: + +| Pattern concept | DCM construct | Where it lives | +|----------------|--------------|----------------| +| The pattern itself | Composite Resource Type Specification | Resource Type Registry | +| The constituents | Resource Type references with dependency declarations | `constituents[]` in the composite spec | +| How pieces connect | Binding fields — runtime values from one constituent injected into another | `binding_fields[]` on dependent constituents | +| What the consumer fills in | Parameterized fields exposed at the pattern level | `fields_from_parent[]` mapping pattern params → constituent fields | +| Who provides each piece | `provided_by: external` (DCM places) or `provided_by: self` (composite service handles) | Per-constituent declaration | +| What happens on failure | Lifecycle policy on the composite spec | `on_constituent_failure: rollback_all | continue_degraded | notify` | +| Operational policies | Standard DCM policies scoped to the pattern's resource type | Policy match on `resource_type = ApplicationStack.WebApp` | + +--- + +## Concrete Example: "Standard Web Application" Pattern + +### Pattern Definition (authored by Platform Engineer) + +```yaml +# Registered in Resource Type Registry as a composite resource type +resource_type: ApplicationStack.WebApp +version: "1.0.0" +entity_type: composite_resource +description: "Production web application with database, app servers, load balancer, and DNS" + +# What the consumer fills in when requesting this pattern +parameters: + app_name: + type: string + required: true + description: "Application name — used for hostnames, DNS, and resource tagging" + environment: + type: string + required: true + constraint: { layer_reference: "environment" } # values governed by environment layers + db_engine: + type: string + required: true + default: postgresql + constraint: { enum: [postgresql, mysql, mariadb] } + db_storage_gb: + type: integer + required: true + default: 50 + constraint: { min: 10, max: 1000 } + app_replicas: + type: integer + required: true + default: 2 + constraint: { min: 1, max: 10 } + expose_public: + type: boolean + default: false + description: "Whether to create a public DNS record and public LB listener" + +# The constituents and how they connect +constituents: + - name: network_segment + resource_type: Network.Segment + provided_by: external + depends_on: [] + required_for_delivery: required + fields_from_parent: + - source: "environment" + target: "environment" + - source: "app_name" + target: "segment_name_prefix" + + - name: database + resource_type: Database.Managed + provided_by: external + depends_on: [network_segment] + required_for_delivery: required + binding_fields: + - source: "network_segment.subnet_cidr" + target: "database.network_cidr" + - source: "network_segment.security_group_id" + target: "database.security_group_id" + fields_from_parent: + - source: "db_engine" + target: "engine" + - source: "db_storage_gb" + target: "storage_gb" + - source: "environment" + target: "environment" + + - name: app_server + resource_type: Compute.VirtualMachine + provided_by: external + depends_on: [database, network_segment] + required_for_delivery: required + binding_fields: + - source: "database.ip_address" + target: "app_server.config.db_host" + - source: "database.port" + target: "app_server.config.db_port" + - source: "database.credentials_ref" + target: "app_server.config.db_credentials_ref" + - source: "network_segment.subnet_cidr" + target: "app_server.network_cidr" + fields_from_parent: + - source: "app_name" + target: "hostname_prefix" + - source: "app_replicas" + target: "replicas" + - source: "environment" + target: "environment" + + - name: load_balancer + resource_type: Network.LoadBalancer + provided_by: external + depends_on: [app_server] + required_for_delivery: required + binding_fields: + - source: "app_server.ip_addresses" + target: "load_balancer.backend_pool" + - source: "app_server.port" + target: "load_balancer.backend_port" + fields_from_parent: + - source: "app_name" + target: "lb_name" + - source: "expose_public" + target: "public_listener" + + - name: dns_internal + resource_type: DNS.Record + provided_by: external + depends_on: [load_balancer] + required_for_delivery: partial + binding_fields: + - source: "load_balancer.vip_address" + target: "dns_internal.target_address" + fields_from_parent: + - source: "app_name" + target: "hostname" + + - name: dns_public + resource_type: DNS.Record + provided_by: external + depends_on: [load_balancer] + required_for_delivery: optional + condition: "parent.expose_public == true" # only created if consumer wants public access + binding_fields: + - source: "load_balancer.public_vip_address" + target: "dns_public.target_address" + fields_from_parent: + - source: "app_name" + target: "hostname" + +lifecycle_policy: + on_constituent_failure: rollback_all + decommission_order: reverse_dependency # DNS first, then LB, then app, then DB, then network +``` + +### What the Consumer Sees + +The consumer browses the service catalog, finds "Standard Web Application," and submits: + +```json +POST /api/v1/requests +{ + "catalog_item_uuid": "webapp-standard-uuid", + "fields": { + "app_name": "pet-clinic", + "environment": "production", + "db_engine": "postgresql", + "db_storage_gb": 100, + "app_replicas": 3, + "expose_public": true + } +} +``` + +Six fields. The consumer has no idea that this will produce 6 resources across potentially 4 different providers. + +### What DCM Does + +``` +1. Intent captured — consumer's 6 fields stored + +2. Pattern decomposed — DCM reads the composite resource type spec + → 6 constituents identified + → Dependency graph resolved: + Network Segment (no deps) + ├── Database (needs network) + │ └── App Server (needs DB + network) + │ └── Load Balancer (needs app server) + │ ├── DNS Internal (needs LB) + │ └── DNS Public (needs LB, conditional on expose_public=true) + +3. Each constituent gets its own full pipeline: + → Layer assembly (datacenter, environment, tenant, compliance layers merge in) + → Policy evaluation (sovereignty, sizing, naming, monitoring — per constituent) + → Placement (each constituent placed independently, all honoring sovereignty) + +4. Execution in dependency order: + + Round 1: Network Segment + → Placed with EU-WEST network provider + → Realized: subnet_cidr=10.5.0.0/24, security_group_id=sg-abc123 + + Round 2: Database (binding fields inject network values) + → config.network_cidr = 10.5.0.0/24 (from network_segment) + → config.security_group_id = sg-abc123 (from network_segment) + → Placed with EU-WEST database provider + → Realized: ip_address=10.5.0.50, port=5432, credentials_ref=vault:secret/pet-clinic-db + + Round 3: App Server (binding fields inject DB + network values) + → config.db_host = 10.5.0.50 (from database) + → config.db_port = 5432 (from database) + → config.db_credentials_ref = vault:secret/pet-clinic-db (from database) + → config.network_cidr = 10.5.0.0/24 (from network_segment) + → Placed with EU-WEST compute provider, 3 replicas + → Realized: ip_addresses=[10.5.0.10, 10.5.0.11, 10.5.0.12], port=8080 + + Round 4: Load Balancer (binding fields inject app server IPs) + → config.backend_pool = [10.5.0.10, 10.5.0.11, 10.5.0.12] (from app_server) + → config.backend_port = 8080 (from app_server) + → Placed with EU-WEST network provider + → Realized: vip_address=10.5.0.100, public_vip_address=203.0.113.50 + + Round 5: DNS records (binding fields inject LB addresses) + → DNS Internal: pet-clinic.internal → 10.5.0.100 + → DNS Public: pet-clinic.example.com → 203.0.113.50 (created because expose_public=true) + +5. All constituents realized → composite entity status: OPERATIONAL + → Consumer receives: entity_uuid, status, and connection details + → 6 entities, each with independent audit trail, drift detection, lifecycle +``` + +### What Policies See + +Policies don't need special awareness of patterns. Each constituent is a standard DCM request with a standard resource type. Existing policies apply naturally: + +| Policy | Fires on which constituent | What it does | +|--------|---------------------------|-------------| +| Sovereignty GateKeeper | All 6 | Ensures all constituents land in EU-WEST | +| VM sizing limits | App Server only | Validates replica count and VM size within tenant tier | +| DB storage limits | Database only | Validates db_storage_gb within allowed range | +| Network naming | Network Segment, DNS | Enforces naming conventions | +| Monitoring injection | App Server, Database | Injects monitoring agent config | +| Backup policy | Database | Injects backup schedule based on environment | + +**No new policy types are needed.** The pattern decomposes into standard resource types, and standard policies match on those types. + +--- + +## How the Pattern Catalog Surfaces in DCM + +### In the Resource Type Registry + +Patterns are compound Resource Type Specifications with `entity_type: composite_resource`. They live alongside atomic resource types in the same registry: + +``` +Resource Type Registry +├── Compute.VirtualMachine (atomic) +├── Network.Segment (atomic) +├── Database.Managed (atomic) +├── DNS.Record (atomic) +├── Network.LoadBalancer (atomic) +├── ApplicationStack.WebApp (composite ← this is a pattern) +├── ApplicationStack.DataPipeline (composite ← this is a pattern) +├── Environment.DevSandbox (composite ← this is a pattern) +└── Platform.EdgeNode (composite ← this is a pattern) +``` + +### In the Service Catalog + +Provider catalog items can reference either atomic or composite resource types. For patterns, the catalog item represents the pattern itself — the consumer requests the pattern, not the individual constituents: + +``` +Service Catalog +├── "EU-WEST VM — Standard" → Compute.VirtualMachine (atomic, provider-specific) +├── "EU-WEST VM — Large" → Compute.VirtualMachine (atomic, provider-specific) +├── "Standard Web Application" → ApplicationStack.WebApp (composite, multi-provider) +├── "Secure Data Pipeline" → ApplicationStack.DataPipeline (composite, multi-provider) +└── "Developer Sandbox" → Environment.DevSandbox (composite, multi-provider) +``` + +### In the Consumer API + +No API changes. The consumer requests a catalog item. Whether it's atomic or compound is transparent — the same `POST /api/v1/requests` endpoint handles both. The response includes constituent status for composite requests. + +### In RHDH + +The RHDH catalog page shows patterns alongside atomic offerings. Patterns have a "Components" view showing the constituent resources, their dependency graph, and (after realization) the binding field values. Platform engineers use RHDH scaffolding templates to create new patterns. + +--- + +## Who Authors Patterns vs Who Consumes Them + +| Role | What they do with patterns | +|------|--------------------------| +| **Platform Engineer** | Authors pattern definitions (composite resource type specs). Defines constituents, dependencies, binding fields, parameters, lifecycle policies. Registers patterns in the Resource Type Registry. Creates service catalog items for patterns. | +| **Policy/Compliance Owner** | Writes policies that apply to pattern constituents. Does not need to know about patterns specifically — policies match on resource types, which patterns decompose into. May also write pattern-level policies (e.g., "all ApplicationStack.* types require monitoring on every constituent"). | +| **Consumer Developer** | Browses the catalog, selects a pattern, fills in parameters, submits. Sees aggregate status. Can drill into constituent detail. Does not need to understand the decomposition. | +| **Infrastructure Operator** | Provides the atomic services that patterns compose. Registers providers for Compute, Network, Database, DNS — not for the pattern itself. | + +--- + +## Pattern Lifecycle + +Patterns follow the standard DCM artifact lifecycle: + +``` +developing → proposed → active → deprecated → retired +``` + +**Versioning:** Patterns are versioned (`ApplicationStack.WebApp v1.0.0`). A new version can add optional constituents, change defaults, or add new binding fields without breaking existing deployments. Removing a required constituent is a major version bump. + +**Deprecation:** When a pattern version is deprecated, existing realized instances continue operating. New requests are redirected to the successor version. Consumers are notified of the deprecation timeline. + +**Pattern evolution:** Adding an optional constituent (e.g., adding a cache layer to the web app pattern) is a minor version bump. Existing deployments don't gain the new constituent automatically — but new requests do. Consumers with existing deployments can opt in via an update request. + +--- + +## Interaction with Other DCM Features + +| Feature | How it interacts with patterns | +|---------|-------------------------------| +| **Drift detection** | Each constituent is independently discoverable. Drift on any constituent is detected and attributed to that constituent — not to the pattern as a whole. | +| **Decommission** | Pattern decommission triggers reverse-dependency-order teardown of all constituents. Consumer can also decommission individual constituents (e.g., remove the public DNS record) without tearing down the pattern. | +| **Rehydration** | Pattern rehydration rebuilds all constituents in dependency order with current policies. Binding fields resolve against newly realized values. | +| **Sovereignty** | Every constituent is independently sovereignty-checked. A pattern cannot span sovereignty zones unless every constituent passes its own sovereignty policy. | +| **Cost estimation** | Pattern cost is the sum of constituent costs. Each constituent's cost comes from its provider catalog item. | +| **Audit** | Each constituent has its own Merkle audit trail. The pattern entity has a composite audit record linking all constituent entity_uuids. | +| **Override** | A policy block on any constituent blocks the entire pattern. The consumer resolves the block for that specific constituent — modify, override, cancel, or escalate to the responsible policy domain owner. | +| **Federation** | Pattern constituents can be placed across DCM instances. The network segment might be local while the database is federated to a remote DCM with a specialized database provider. | + +--- + +## New Use Case for Requirements Document + +### UC-100: Deploy a Resource Pattern from the Pattern Catalog + +A consumer browses the Pattern Catalog section of the service catalog and selects "Standard Web Application." The catalog shows the pattern's components (network, database, app servers, load balancer, DNS), the parameters the consumer needs to provide, the dependency graph, and estimated cost. + +The consumer fills in 6 parameters (app_name, environment, db_engine, db_storage_gb, app_replicas, expose_public) and submits. DCM decomposes the pattern into 6 constituent resources, resolves the dependency graph, and processes each constituent through the full pipeline — layer assembly, policy evaluation, placement, dispatch — independently. Binding fields inject runtime values (IP addresses, connection strings, credentials references) from realized constituents into dependent ones. + +The consumer monitors aggregate progress ("3 of 6 constituents realized") and can drill into individual constituent status. On completion, the consumer has a fully wired application environment: database with data, app servers connected to the database, load balancer distributing traffic, and DNS records resolving. + +If any required constituent fails (e.g., the database provider reports an error), the pattern's lifecycle policy determines the response — rollback all realized constituents, continue in degraded mode, or notify and hold for manual intervention. + +**Success criteria:** Single request produces a complete, wired application environment. Runtime values flow correctly between constituents via binding fields. Each constituent is independently managed (own entity_uuid, audit trail, drift detection, lifecycle). Decommission reverses dependency order. Sovereignty enforced per-constituent. No new control plane services, policy types, or API endpoints required — patterns use existing DCM machinery. diff --git a/architecture/dcm-platform-requirements.md b/architecture/dcm-platform-requirements.md new file mode 100644 index 0000000..76a398d --- /dev/null +++ b/architecture/dcm-platform-requirements.md @@ -0,0 +1,701 @@ +# DCM PLATFORM REQUIREMENTS DOCUMENT + +**Data Center Management — Sovereign Cloud Framework** + +*Requirements for the DCM control plane, data model, and provider ecosystem enabling enterprise organizations to manage infrastructure lifecycle with policy-as-code governance, tamper-evident audit, and multi-provider abstraction.* + +**Date:** April 2026 +**Author:** Chris Roadfeldt, Principal Architect +**Project:** Red Hat FlightPath — github.com/dcm-project + +--- + +# Engagement Context + +DCM addresses a fundamental gap in enterprise infrastructure management: on-premises data centers lack the unified control plane that public clouds provide as table stakes. Organizations operating large-scale, multi-platform infrastructure spend disproportionate engineering effort stitching together disparate automation tools, enforcing governance manually, and reconciling inventory that diverges silently between intended and actual state. + +**Target Organizations** + +- Enterprise data center operators running heterogeneous infrastructure (VM, container, network, storage) across multiple platforms +- Organizations with regulatory requirements (financial services, government, healthcare) requiring provable audit trails and data residency enforcement +- Platform engineering teams seeking to offer self-service infrastructure to development teams with built-in governance + +**DCM's Value Proposition** + +DCM is the management plane that sits above provisioning tools (Ansible, Terraform, Kubernetes operators) and governs what gets requested, approved, built, owned, and decommissioned. It provides: a unified data model and API across all infrastructure platforms; policy-as-code enforcement on every request before provisioning; full lifecycle management from request through decommission with tamper-evident audit; and a provider abstraction with capability discovery that makes any infrastructure platform consumable through the same interface. + +**Delivery Context** + +DCM is an open-source project under the Red Hat FlightPath initiative, licensed under Apache 2.0. The engineering team spans Red Hat and community contributors. The initial milestone is a summit demonstration showing end-to-end provisioning with policy enforcement and audit verification. + +--- + +# Executive Summary + +DCM requires a control plane that enables: declarative lifecycle management of infrastructure resources across heterogeneous providers; policy-as-code governance enforced on every request with configurable override mechanisms; a provider abstraction that makes any infrastructure platform — VM, container, network, storage — consumable through the same interface and data model; tamper-evident audit with mathematical provability (Merkle tree) at configurable granularity; sovereignty and data residency enforcement as first-class concepts, not afterthoughts; and a consumer experience that abstracts infrastructure complexity behind a service catalog. + +The platform is designed as a management plane — it orchestrates lifecycle, enforces governance, and maintains state. It does not provision infrastructure directly. Provisioning is delegated to service providers (OpenStack, KubeVirt, ACM, Ansible, Terraform) that implement DCM's provider contract. + +--- + +# Personas + +## Consumer Developer + +A developer or application owner who requests and manages infrastructure resources through the service catalog. Interacts with DCM via API, Web UI (RHDH), or Git PR. Does not need to understand which provider, datacenter, or network configuration fulfills their request — DCM handles that. + +- **Key activities:** Browse service catalog, submit resource requests, track request status, handle policy blocks, manage running resources, view cost estimates, trigger decommission + +## Platform Engineer + +Defines the organizational infrastructure standards that DCM enforces. Authors resource type specifications, data layers (datacenter configs, environment defaults, tenant overrides), policies, and composite resource type specifications (three-tier applications, data pipelines). + +- **Key activities:** Define resource types and catalog items, author data layers, write and test policies (shadow mode), create composite service definitions, manage the resource type registry + +## Infrastructure Operator + +Manages the physical and virtual infrastructure that DCM's providers abstract. Implements and operates service providers, manages provider health and capacity, handles accreditation and sovereignty declarations, and responds to drift remediation alerts. + +- **Key activities:** Implement and register service providers, manage provider health and capacity reporting, handle naturalization/denaturalization, respond to discovery and drift events + +## Policy and Compliance Owner + +Defines and manages the governance rules that DCM enforces. Authors GateKeeper policies (allow/deny), validation policies, sovereignty constraints, and override approval rules. Reviews audit trails and compliance reports. + +- **Key activities:** Author and activate policies, configure policy profiles (minimal through sovereign), review override requests (dual-approval), verify audit integrity, manage compliance rescans + +## Platform Administrator + +Deploys and operates the DCM control plane itself. Manages the 9 control plane services, database, tenant configuration, auth provider setup, and platform health monitoring. + +- **Key activities:** Deploy and upgrade DCM, onboard tenants, configure auth providers, manage profiles and deployment configuration, monitor platform health, rotate credentials + +--- + +# Use Cases + +The following use cases capture DCM's core requirements, organized by lifecycle phase and prioritized for the summit demonstration and subsequent delivery phases. + +| **ID** | **Use Case** | **Priority** | **Personas** | +|--------|-------------|-------------|-------------| +| **UC-001** | Deploy DCM Control Plane | **P0** | Administrator | +| **UC-002** | Register Service Providers | **P0** | Operator, Administrator | +| **UC-003** | Populate Service Catalog | **P0** | Platform Engineer | +| **UC-004** | Configure Organizational Policies | **P0** | Policy Owner, Platform Engineer | +| **UC-005** | Configure Deployment Profiles | **P1** | Administrator | +| **UC-010** | Provision a Virtual Machine | **P0** | Consumer | +| **UC-011** | Provision a Three-Tier Application | **P0** | Consumer | +| **UC-012** | Handle a Policy-Blocked Request | **P0** | Consumer | +| **UC-013** | View Cost Before Committing | **P1** | Consumer | +| **UC-014** | Track Request Progress | **P0** | Consumer | +| **UC-020** | Update a Running Resource | **P0** | Consumer | +| **UC-021** | Scale Resource Capacity | **P1** | Consumer | +| **UC-022** | Detect and Remediate Drift | **P0** | Operator, Consumer | +| **UC-023** | Transfer Resource Ownership | **P2** | Consumer, Administrator | +| **UC-024** | Manage Subscription Lifecycle | **P2** | Consumer, Administrator | +| **UC-030** | Rebuild from Stored State (DR) | **P1** | Operator, Administrator | +| **UC-031** | Migrate Resources to New Provider | **P2** | Operator, Administrator | +| **UC-040** | Decommission a Resource | **P0** | Consumer | +| **UC-050** | Enforce Sovereignty and Data Residency | **P0** | Policy Owner | +| **UC-051** | Rescan Existing Resources Against New Policy | **P1** | Policy Owner | +| **UC-052** | Prove Pipeline Integrity to Auditor | **P0** | Policy Owner | +| **UC-053** | Prove Override Authorization | **P1** | Policy Owner | +| **UC-060** | Cross-Instance Placement (Federation) | **P2** | Administrator | +| **UC-070** | Monitor DCM Platform Health | **P1** | Administrator | +| **UC-071** | Onboard a New Tenant | **P0** | Administrator | +| **UC-072** | Validate Provider Accreditation | **P2** | Administrator, Policy Owner | +| **UC-080** | Scoped Interaction Credentials (Zero Trust) | **P1** | Administrator | +| **UC-081** | Session Management and Auth Failover | **P0** | Consumer, Administrator | +| **UC-090** | ITSM Integration (ServiceNow) | **P2** | Administrator, Operator | +| **UC-091** | Git-Based Request Ingress | **P1** | Consumer, Platform Engineer | +| **UC-100** | Deploy a Resource Pattern from Pattern Catalog | **P0** | Consumer, Platform Engineer | + +--- + +## Day 0 — Platform Setup + +### UC-001: Deploy DCM Control Plane + +An administrator deploys the DCM control plane to an OpenShift cluster. PostgreSQL is the only required infrastructure dependency. The deployment includes 9 control plane services (API Gateway, Catalog Manager, Request Processor, Policy Engine, Placement Engine, Request Orchestrator, Audit Service, Discovery Service, Provider Manager), the PostgreSQL database with 18 tables and RLS tenant isolation, and initial configuration (system admin account, default profile, built-in auth provider). After deployment, the administrator verifies all services are healthy and the API gateway is reachable. + +**Success criteria:** All 9 services healthy. API gateway responds to requests. Admin can authenticate and access the admin API. No external dependencies required beyond PostgreSQL. + +### UC-002: Register Service Providers + +An infrastructure operator registers a service provider (e.g., OpenStack Nova compute) with DCM. The registration includes: provider type (service, information, meta, auth, peer_dcm, or process), capability declaration (resource types supported, lifecycle operations), health endpoint URL, sovereignty zone declarations (which jurisdictions the provider operates in), accreditation declarations (compliance certifications), and callback authentication configuration. DCM validates the registration, probes the health endpoint, and activates the provider. The provider is now available for placement consideration. + +**Success criteria:** Provider registered and active. Health check passes. Resource types appear in the registry. Provider is eligible for placement queries. + +### UC-003: Populate Service Catalog + +A platform engineer defines resource types and catalog items. A resource type specification (e.g., `Compute.VirtualMachine v1.0.0`) declares the vendor-neutral field schema (cpu_count, memory_gb, os_family), constraints (min/max values, allowed enums), type-level dependencies (every VM requires a Network.IPAddress), and lifecycle rules. A provider catalog item (e.g., "EU-WEST OpenStack VM — Standard") ties a resource type spec to a specific provider with pricing, SLAs, and availability. The platform engineer also creates data layers: datacenter layers (location, network ranges), environment layers (production vs dev defaults), tenant layers (team-specific overrides), and compliance layers (EU data residency requirements). + +**Success criteria:** Consumers can browse the catalog filtered by RBAC. Resource type schemas are queryable. Dependency graphs are visible. Data layers resolve correctly during assembly. + +### UC-004: Configure Organizational Policies + +A policy and compliance owner authors the baseline policy set: a sovereignty GateKeeper policy that blocks resources from deploying outside designated zones (hard enforcement); a sizing validation policy that enforces CPU and memory limits per tenant tier; a naming transformation policy that auto-generates standardized hostnames; a monitoring transformation policy that injects the organization's monitoring agent into every production resource; and a cost GateKeeper policy that blocks requests exceeding budget thresholds. Each policy is first deployed in **shadow mode** — it evaluates against real traffic and logs results without blocking requests. After validation, the policy is promoted to active. Policies use the Gatekeeper ConstraintTemplate pattern: reusable Rego logic with parameterized instances. + +**Success criteria:** Policies evaluate correctly in shadow mode. No false positives on legitimate requests. Activation enforces the policy on all matching requests. Audit records produced for every evaluation. + +### UC-005: Configure Deployment Profiles + +An administrator configures deployment profiles that govern operational behavior per environment: + +| Profile | Audit Granularity | Override Timeout | Policy Minimums | Use | +|---------|-------------------|-----------------|----------------|-----| +| minimal | stage | 24h | none | Homelab, evaluation | +| dev | stage | 4h | naming, tagging | Development | +| standard | mutation | 8h | all core policies | Production | +| fsi | field | 48h | sovereignty + all core | Financial services | +| sovereign | field (synchronous) | 72h | all policies on all ops | Government, classified | + +**Success criteria:** Profile assignment per tenant governs audit depth, override windows, and policy enforcement minimums. Profile changes take effect on next request. + +--- + +## Day 1 — Consumer Operations + +### UC-010: Provision a Virtual Machine + +A consumer developer browses the service catalog, selects "Virtual Machine — Standard," and submits a request with 6 fields: cpu_count, memory_gb, storage_gb, os_family, environment, and name. The consumer does not specify a provider, datacenter, or network configuration. + +**DCM processes the request through the full pipeline:** +1. **Intent captured** — Consumer's raw declaration stored with entity_uuid assigned +2. **Layer assembly** — 5 data layers merge organizational context (datacenter, environment, compliance, tenant, provider defaults) into the consumer's 6 fields, producing 10+ fields with full provenance +3. **Dependency resolution** — Resource type spec declares VM requires Network.IPAddress; DCM creates an IP sub-request automatically +4. **IP policy evaluation** — Sovereignty, subnet isolation, and pool selection policies evaluate against the IP sub-request; IPAM provider selected and IP allocated +5. **VM policy evaluation** — GateKeeper (sizing, sovereignty, approved OS images), Validation (field constraints), and Transformation (monitoring injection) policies evaluate +6. **Placement** — Sovereignty pre-filter eliminates non-compliant providers; remaining providers scored by capacity and confidence; best provider selected +7. **Dispatch** — Request Orchestrator sends the assembled payload (including the dependency-injected IP address) to the selected provider; the provider naturalizes DCM's unified payload into its native API +8. **Realization** — Provider provisions the VM, denaturalizes the result back to DCM's format, and callbacks with realized state +9. **Audit** — 17 Merkle tree leaves recorded across the VM entity and IP dependency + +**Success criteria:** Consumer receives a running VM with the pre-allocated IP address. Full provenance chain traceable from every field value back to its origin. Audit trail is tamper-evident and verifiable. Consumer tracked progress through pipeline stages via status API. + +### UC-011: Provision a Three-Tier Application + +A consumer requests a "Web Application — Standard" catalog item. This is a composite service backed by a composite resource type specification that decomposes into four constituent resources: network port, database VM, application server VM, and load balancer. + +The composite resource type spec declares the dependency graph and binding fields: +- Network port has no dependencies — provisioned first +- Database depends on network port — IP address injected from port +- Application server depends on database — connection string and credentials injected +- Load balancer depends on application server — backend pool configured from app server IPs + +DCM processes each constituent through the full pipeline (policy evaluation, placement, dispatch) independently, respecting dependency order. Realized outputs from each constituent flow into dependent constituents via binding fields. If any required constituent fails, a compensation policy triggers reverse-order teardown of already-realized constituents. + +**Success criteria:** Single catalog request produces 4 running, interconnected resources. Runtime values (IPs, connection strings) flow correctly between constituents. Each constituent is independently managed in DCM (its own entity_uuid, audit trail, drift detection). Decommission reverses dependency order. + +### UC-012: Handle a Policy-Blocked Request + +A consumer requests a VM with `environment: production` in a zone that violates their tenant's EU data residency policy. The request enters `POLICY_BLOCKED` state. + +The consumer receives a structured response containing: the blocking policy name, type, and enforcement level; the specific field that violated (sovereignty_zone); compliant value suggestions (e.g., "zone must be one of: eu-west-1, eu-west-2"); and four resolution paths. + +| Resolution | What happens | +|-----------|-------------| +| **Modify request** | Consumer changes the zone to eu-west-1 and resubmits. Request proceeds. | +| **Request override** | Consumer provides justification. For hard policies, dual-approval required (two approvers, different roles). | +| **Cancel** | Consumer abandons the request. Audit trail preserved. | +| **Escalate** | Request routed to the role responsible for the blocking policy domain (e.g., sovereignty admin for data residency violations, security admin for compliance blocks, cost admin for budget overruns). The responsible role reviews and may register an Exception Grant for future similar requests. Routing is configurable per policy domain and profile. | + +**Success criteria:** Consumer receives actionable guidance, not just "denied." Override requests are auditable. Block timeout auto-cancels abandoned requests. Override frequency is tracked for policy review. + +### UC-013: View Cost Before Committing + +A consumer sees an estimated cost for their resource request before submitting. Cost data comes from provider catalog item metadata (declared at registration). The placement engine considers cost alongside sovereignty, tier, and capacity constraints — not as the sole factor. + +**Success criteria:** Cost estimate displayed before submission. Cost attribution tracks to tenant and business unit. + +### UC-014: Track Request Progress + +After submitting a request, the consumer monitors progress through pipeline stages: SUBMITTED → ASSEMBLING → POLICY_EVALUATION → PLACEMENT → DISPATCHED → REALIZING → OPERATIONAL. For composite services, each constituent's status is tracked independently. + +**Success criteria:** Real-time status updates via Server-Sent Events (SSE) or polling. Constituent-level tracking for composite service requests. Failed stages show clear error with remediation guidance. + +--- + +## Day 2 — Ongoing Management + +### UC-020: Update a Running Resource + +A consumer modifies a running VM's memory from 8 GB to 16 GB. DCM creates a new request with operation_type `update`. Only lifecycle-relevant policies fire: the sizing validation policy re-evaluates (lifecycle_scope includes `update`, changed_field_filter includes `memory_gb`). The sovereignty policy does not re-evaluate (no zone change). Placement does not re-run (no provider change). The provider receives a delta payload and applies the change. + +**Success criteria:** Only relevant policies fire. Unchanged fields are not re-evaluated. Audit records the specific mutation. Realized state updated to reflect new memory. + +### UC-021: Scale Resource Capacity + +A consumer scales application server replicas from 2 to 4. If the existing provider/zone has capacity, the scale occurs in place. If capacity is insufficient, placement re-evaluates to find a zone with capacity — subject to the same sovereignty and policy constraints as initial provisioning. + +**Success criteria:** Scale-in-place when capacity exists. Automatic placement re-evaluation when it doesn't. Sovereignty constraints honored throughout. + +### UC-022: Detect and Remediate Drift + +The Discovery service polls providers on a configurable interval (default: 5 minutes). It compares discovered state to realized state. If a VM was manually modified outside DCM (e.g., memory changed from 8 GB to 16 GB via the hypervisor console), drift is detected. The configured drift policy determines the response: notify (alert the consumer and operator), auto-remediate (revert to realized state), or log (record the drift for manual review). + +**Success criteria:** Drift detected within one polling interval. Notification sent to appropriate parties. Auto-remediation restores realized state when configured. Both VM entity and its IP dependency are independently discoverable. + +### UC-030: Rebuild from Stored State (Disaster Recovery) + +A datacenter failure renders an entire availability zone unavailable. The infrastructure operator initiates rehydration for all affected resources. DCM reads the original **Intent State** for each affected entity — the consumer's raw declaration, preserved immutably since submission. + +Each entity re-enters the full pipeline as a new request with operation_type `rehydration`: + +1. **Layer assembly re-runs** — layers may have changed since original provisioning (new compliance requirements, updated monitoring agents). The resource gets current organizational context, not stale data. +2. **All policies re-evaluate** — current sovereignty policies, sizing limits, and security requirements apply. A resource that was compliant when originally provisioned may now violate a newer policy. If so, it enters POLICY_BLOCKED and the operator must resolve before rehydration proceeds. +3. **Placement re-evaluates** — the original provider/zone is unavailable. The placement engine scores surviving providers, subject to the same sovereignty pre-filter. A resource originally in EU-WEST-Prod-1 may rehydrate to EU-WEST-Prod-2. +4. **Dependencies rehydrate in order** — for composite services (three-tier apps), DCM reads the dependency graph and rehydrates constituents in dependency order: database first, then backend (with new DB IP injected), then frontend (with new backend IP injected). Binding fields resolve against newly realized values, not cached originals. +5. **Entity UUID is preserved** — the resource keeps its original entity_uuid across rehydration. Audit trail links the original lifecycle to the rehydrated one. + +**What does NOT happen:** The realized state from the lost zone is not replayed. The intent is re-processed from scratch with current layers, current policies, and current provider availability. This is a design choice — rehydration produces resources that comply with today's rules, not yesterday's. + +**Success criteria:** All affected resources rebuilt on surviving infrastructure. Current policies enforced (not original-time policies). Dependencies resolve correctly with new runtime values. Entity UUIDs preserved. Complete audit trail links original and rehydrated lifecycles. Sovereignty constraints honored — resources cannot rehydrate into non-compliant zones. + +### UC-040: Decommission a Resource + +A consumer or TTL trigger initiates decommission. DCM checks for dependencies: if other resources depend on this one (e.g., a VM using an IP address), decommission is blocked with guidance. If no blockers, the request enters the pipeline with operation_type `decommission`. The provider tears down the resource. Credentials are revoked. The IP address is released back to its pool. Audit trail is preserved permanently. + +**Success criteria:** Dependency checks prevent premature teardown. Provider confirms teardown. Dependent resources (IPs, credentials) are cleaned up. Audit trail survives decommission. + +--- + +## Governance + +### UC-050: Enforce Sovereignty and Data Residency + +All resources handling restricted, PHI, or PCI data are placed exclusively in designated sovereignty zones. The sovereignty GateKeeper policy fires on **every lifecycle operation** (initial provisioning, update, scale, rehydration, ownership transfer) — not just initial provisioning. A resource in EU-WEST stays in EU-WEST for its entire lifecycle. + +Override requires dual-approval: two approvers from different roles, with written justification and compensating controls. Every override produces a Merkle audit leaf at field granularity. + +**Success criteria:** No resource is ever realized in a non-compliant zone without an audited, dual-approved override. Sovereignty is enforced end-to-end, not just at creation. + +### UC-052: Prove Pipeline Integrity to Auditor + +An external auditor requests proof that a specific provisioning request was processed correctly. DCM returns: +- **Inclusion proof:** Mathematical proof that the specific audit record exists in the Merkle tree +- **Consistency proof:** Mathematical proof that the tree has only grown since the last signed tree head (no deletions) +- **Request chain:** The complete hash chain from intent through realization with Ed25519 signatures from each service +- **Signed tree head:** The current root hash signed by DCM's identity + +The auditor can independently verify these proofs without trusting DCM. + +**Success criteria:** Auditor verifies integrity using only DCM's public key and the proof data. No trust relationship with DCM required. Verification completes in seconds regardless of tree size. + +--- + +## Platform Operations + +### UC-071: Onboard a New Tenant + +An administrator creates a new tenant with profile assignment (e.g., "standard" for a production team, "dev" for a sandbox). RLS isolation is enforced immediately — the new tenant cannot see other tenants' data. RBAC is configured: which roles exist, which actors belong to which roles. The catalog is filtered: the tenant sees only catalog items they're authorized to request. The administrator submits a test request to verify full pipeline isolation. + +**Success criteria:** Tenant data is isolated by RLS from first query. RBAC filters the catalog correctly. Test request flows through the complete pipeline with no cross-tenant data leakage. + +### UC-081: Session Management and Auth Failover + +An actor authenticates via the configured auth provider (built-in, Keycloak, LDAP, OIDC). Session tokens are issued with configurable TTL. Concurrent session limits are enforced. If the primary auth provider fails, existing sessions remain valid (cached) and new authentication routes to the failover chain. Session revocation propagates immediately. + +**Success criteria:** Authentication works through all configured providers. Failover is transparent. Session revocation takes effect immediately. Concurrent session limits enforced. + +### UC-091: Git-Based Request Ingress + +A developer submits an infrastructure request via Git Pull Request. The PR identity is resolved to the same DCM actor as API/UI login. The request enters the standard pipeline. Policy dry-run feedback is posted as PR comments. Approval via PR review maps to DCM approval. On merge, the request is dispatched. + +**Success criteria:** Git PR identity resolves to DCM actor. Policy feedback appears as PR comments. Merge triggers dispatch. Full audit trail links PR to DCM request. + +--- + +## Federation + +### UC-060: Cross-Instance Placement (Federation) + +An organization runs two DCM instances: DCM-EMEA (EU datacenters) and DCM-APAC (Asia-Pacific datacenters). A consumer on DCM-EMEA requests a resource that, due to latency requirements for an APAC-facing application, should be placed in an APAC zone. The consumer does not need to know which DCM instance will fulfill the request. + +**How federation works:** + +1. **Peer DCM registration** — DCM-APAC is registered as a `peer_dcm` provider on DCM-EMEA. The registration includes sovereignty declarations, federation eligibility scope (which resource types and operations are permitted), and an mTLS certificate for the federation tunnel. + +2. **Request enters normal pipeline on DCM-EMEA** — Layer assembly, policy evaluation, and placement all run locally. The consumer's sovereignty policy allows APAC placement for this resource type. + +3. **Placement considers remote providers** — The placement engine treats DCM-APAC's providers as candidates alongside local providers. Cross-DCM confidence scoring applies: `cross_dcm_confidence = source_confidence × (tunnel_trust_score / 100)`. Remote providers are scored lower by the trust factor, but may win if local providers lack capacity or sovereignty eligibility. + +4. **DCM-EMEA dispatches to DCM-APAC via federation tunnel** — The request payload is sent over the mTLS tunnel. DCM-APAC receives it, runs its own local policy evaluation (local policies govern — remote policies cannot override), and dispatches to its selected provider. + +5. **Dual audit** — Audit records are written in both DCM instances with a shared `correlation_id`. DCM-EMEA records the outbound federation dispatch. DCM-APAC records the local pipeline execution and realization. Either instance can produce a complete audit trail for its portion. + +6. **Realized state flows back** — DCM-APAC's provider callback flows back through the federation tunnel to DCM-EMEA. The consumer on DCM-EMEA sees the resource as OPERATIONAL with the APAC provider's realized fields. + +**Federation constraints:** +- Sovereignty is verified before tunnel establishment — a classified-zone provider cannot participate in federation +- Storage providers default to `federation_eligibility: none` — data sovereignty prohibits storage federation unless explicitly authorized +- Remote DCMs cannot decommission local resources through a tunnel — decommission is always local +- Certificate rotation uses a 30-day overlap period so peers can update trust stores without coordinated downtime + +**Success criteria:** Consumer requests a resource on DCM-EMEA and receives it from DCM-APAC without knowing the routing. Audit trail in both instances with shared correlation_id. Sovereignty enforced at both ends. Federation tunnel is mTLS-only. + +--- + +## Integration + +### UC-090: ITSM Integration (ServiceNow Change Management) + +An organization requires that all infrastructure provisioning creates change records in ServiceNow, updates are tracked through CMDB configuration items, and decommission retires the CI. DCM integrates with ServiceNow via a **process_provider** — a bidirectional integration that enriches DCM entities with ITSM metadata without making ServiceNow a required dependency. + +**Design principle:** DCM replaces the infrastructure ticket as the provisioning mechanism. ITSM integration is additive — it enriches, it does not gate (unless explicitly configured to do so). + +**Outbound flow (DCM → ServiceNow):** + +1. **ITSM Policy evaluates** — An ITSM Action policy fires on DCM lifecycle events. It is a side-effect policy: it triggers ITSM actions but does not block the pipeline by default. + +2. **On `request.dispatched`** — The ITSM integration creates a ServiceNow change request (CHG record) containing the request details, requesting actor, tenant, resource type, and placement decision. The CHG number is stored on the DCM entity as `business_data.itsm_references[].external_id`. + +3. **On `entity.realized`** — The ITSM integration updates the change request to "Implemented" and creates or updates a CMDB Configuration Item (CI) with the realized resource's details (IP address, provider, datacenter, ownership). + +4. **On `entity.updated`** — The ITSM integration updates the CMDB CI and creates a new change task linked to the parent CHG. + +5. **On `entity.decommissioned`** — The ITSM integration closes the change request, retires the CMDB CI, and updates the CI's lifecycle status. + +**Inbound flow (ServiceNow → DCM):** + +6. **Change Advisory Board (CAB) approval** — For organizations that require CAB approval before provisioning, the ITSM Policy can be configured as `block_until_approved`. The DCM pipeline pauses after the change request is created in ServiceNow. When the CAB approves, ServiceNow calls DCM's approval API (`POST /api/v1/admin/approvals/{uuid}:vote`) with the decision. The pipeline resumes. + +7. **ITSM-initiated requests** — A ServiceNow workflow can create a DCM request via the Admin API, enabling "request infrastructure from ServiceNow" patterns for organizations transitioning from ticket-based provisioning. + +**Safety guardrail:** `block_until_created` (wait for CHG creation before proceeding) requires a `block_timeout` — the pipeline never permanently stalls waiting for an ITSM system. If the timeout fires, the pipeline proceeds with a warning and the ITSM record is created asynchronously when the system recovers. + +**Supported ITSM systems:** ServiceNow, Jira Service Management, BMC Remedy/Helix, Freshservice, PagerDuty, Opsgenie, ManageEngine, Cherwell, TOPdesk, and a generic REST adapter for others. + +**Success criteria:** Every provisioning request has a corresponding ServiceNow change record. CMDB CIs are created on realization and retired on decommission. CAB approval gates work when configured. ITSM system unavailability does not block DCM operations (non-blocking default with configurable blocking mode). Bidirectional links between DCM entity_uuid and ServiceNow CHG/CI numbers. + +--- + +## Pattern Catalog + +### UC-100: Deploy a Resource Pattern from the Pattern Catalog + +A consumer browses the Pattern Catalog section of the service catalog and selects "Standard Web Application." The catalog shows the pattern's components (network segment, database, app servers, load balancer, DNS), the parameters the consumer needs to provide, the dependency graph, and estimated cost. + +The consumer fills in 6 parameters (app_name, environment, db_engine, db_storage_gb, app_replicas, expose_public) and submits. DCM decomposes the pattern into 6 constituent resources, resolves the dependency graph, and processes each constituent through the full pipeline — layer assembly, policy evaluation, placement, dispatch — independently. Each constituent may be placed with a different provider: the database with a managed database provider, the VMs with a compute provider, the load balancer with a network provider. + +**Dependency resolution and binding fields in action:** + +1. Network Segment provisioned first (no dependencies) → produces subnet_cidr, security_group_id +2. Database provisioned next → receives network values via binding fields → produces ip_address, port, credentials_ref +3. App Servers provisioned next → receive DB connection string and network config via binding fields → produce ip_addresses, port +4. Load Balancer provisioned next → receives app server IPs as backend pool via binding fields → produces vip_address +5. DNS records provisioned last → receive LB address via binding fields → public DNS conditional on consumer parameter + +The consumer monitors aggregate progress ("4 of 6 constituents realized") and can drill into individual constituent status. On completion, the consumer has a fully wired application environment. + +**Policy interaction:** No new policy types are needed. Each constituent is a standard resource type, and existing policies match naturally — sovereignty policies check every constituent, sizing policies check VMs, storage policies check the database, naming policies check DNS records. A policy block on any constituent blocks the entire pattern until the consumer resolves it. + +**Failure handling:** If any required constituent fails, the pattern's lifecycle policy determines the response: rollback all realized constituents (default), continue in degraded mode, or notify and hold for manual intervention. + +**Pattern authoring:** Platform engineers create patterns as compound Resource Type Specifications in the Resource Type Registry. Patterns define constituents, dependencies, binding fields, exposed parameters, and lifecycle policies. Consumers see them as catalog items indistinguishable from atomic offerings — the decomposition is invisible. + +**Success criteria:** Single request produces a complete, wired application environment. Runtime values flow correctly between constituents. Each constituent is independently managed (own entity_uuid, audit trail, drift detection, lifecycle). Decommission reverses dependency order. No new control plane services, policy types, or API endpoints required — patterns use existing DCM machinery. + +### Pattern Catalog — Architectural Overlay + +A deployment pattern is a reusable, provider-agnostic blueprint that defines a collection of resources, their dependencies, their runtime wiring, and their operational policies — together delivering a service that no single provider offers. + +**Example patterns:** + +| Pattern | Constituents | What it delivers | +|---------|-------------|-----------------| +| Standard Web Application | 2 app VMs, 1 DB, 1 LB, 1 network, 3 DNS | Production web app with HA, monitoring, DNS | +| Secure Data Pipeline | 1 Kafka cluster, 2 workers, 1 object store, 1 network policy, 1 key | Encrypted ingest pipeline with data residency | +| Developer Sandbox | 1 VM, 1 namespace, 1 ephemeral DB, 1 port forward | Disposable dev environment with TTL auto-cleanup | +| Regulated Database Service | 1 PostgreSQL (HA), 1 backup, 1 key, 1 audit sink, 2 DNS | Database meeting FSI data handling requirements | +| Edge Compute Node | 1 bare metal, 1 MicroShift, 1 VPN, 1 monitor agent, 1 cert | Self-contained edge node with central management | + +**The key property:** No single provider owns the pattern. The database might come from one provider, the VMs from another, the load balancer from a third. The pattern defines *what* is needed and *how the pieces connect* — DCM figures out *who* provides each piece. + +### How Patterns Layer on DCM + +``` +┌──────────────────────────────────────────────────────┐ +│ PATTERN CATALOG │ +│ Curated library of reusable deployment blueprints │ +│ Authored by: Platform Engineers │ +│ Consumed by: Consumer Developers │ +│ Stored in: Resource Type Registry (composite types) │ +├──────────────────────────────────────────────────────┤ +│ SERVICE CATALOG │ +│ Provider-specific offerings + pattern offerings │ +│ Populated by: Providers (atomic) + Patterns (composite)│ +├──────────────────────────────────────────────────────┤ +│ DCM CONTROL PLANE │ +│ Decompose → Policy → Placement → Dispatch → Audit │ +│ Each constituent → full pipeline independently │ +├──────────────────────────────────────────────────────┤ +│ SERVICE PROVIDERS │ +│ VM · Network · Database · DNS · Storage · Container │ +│ Fulfill individual constituents of a pattern │ +└──────────────────────────────────────────────────────┘ +``` + +**The Pattern Catalog is not a new architectural component.** It is a curated view of the Resource Type Registry filtered to composite resource types. DCM already has all the machinery — the composite service definition model, dependency graphs, binding fields, and constituent dispatch. The Pattern Catalog adds the curation and consumer experience layer. + +### How a Pattern Maps to DCM Constructs + +| Pattern concept | DCM construct | Where it lives | +|----------------|--------------|----------------| +| The pattern itself | Composite Resource Type Specification | Resource Type Registry | +| The constituents | Resource Type references with dependency declarations | `constituents[]` in the composite spec | +| How pieces connect | Binding fields — runtime values from one constituent injected into another | `binding_fields[]` on dependent constituents | +| What the consumer fills in | Parameterized fields exposed at the pattern level | `fields_from_parent[]` mapping pattern params to constituent fields | +| Who provides each piece | `provided_by: external` (DCM places) or `provided_by: self` (composite service definition handles) | Per-constituent declaration | +| What happens on failure | Lifecycle policy on the composite spec | `on_constituent_failure: rollback_all` or `continue_degraded` or `notify` | +| Operational policies | Standard DCM policies scoped to constituent resource types | Policy match on resource_type per constituent | + +### Pattern Definition Example — Standard Web Application + +```yaml +resource_type: ApplicationStack.WebApp +version: "1.0.0" +entity_type: composite_resource + +# Consumer-facing parameters +parameters: + app_name: { type: string, required: true } + environment: { type: string, required: true, constraint: { layer_reference: "environment" } } + db_engine: { type: string, default: postgresql, constraint: { enum: [postgresql, mysql] } } + db_storage_gb: { type: integer, default: 50, constraint: { min: 10, max: 1000 } } + app_replicas: { type: integer, default: 2, constraint: { min: 1, max: 10 } } + expose_public: { type: boolean, default: false } + +constituents: + - name: network_segment + resource_type: Network.Segment + provided_by: external + depends_on: [] + required_for_delivery: required + fields_from_parent: + - { source: "environment", target: "environment" } + - { source: "app_name", target: "segment_name_prefix" } + + - name: database + resource_type: Database.Managed + provided_by: external + depends_on: [network_segment] + required_for_delivery: required + binding_fields: + - { source: "network_segment.subnet_cidr", target: "database.network_cidr" } + - { source: "network_segment.security_group_id", target: "database.security_group_id" } + fields_from_parent: + - { source: "db_engine", target: "engine" } + - { source: "db_storage_gb", target: "storage_gb" } + + - name: app_server + resource_type: Compute.VirtualMachine + provided_by: external + depends_on: [database, network_segment] + required_for_delivery: required + binding_fields: + - { source: "database.ip_address", target: "app_server.config.db_host" } + - { source: "database.port", target: "app_server.config.db_port" } + - { source: "database.credentials_ref", target: "app_server.config.db_credentials_ref" } + - { source: "network_segment.subnet_cidr", target: "app_server.network_cidr" } + fields_from_parent: + - { source: "app_name", target: "hostname_prefix" } + - { source: "app_replicas", target: "replicas" } + + - name: load_balancer + resource_type: Network.LoadBalancer + provided_by: external + depends_on: [app_server] + required_for_delivery: required + binding_fields: + - { source: "app_server.ip_addresses", target: "load_balancer.backend_pool" } + - { source: "app_server.port", target: "load_balancer.backend_port" } + fields_from_parent: + - { source: "expose_public", target: "public_listener" } + + - name: dns_internal + resource_type: DNS.Record + provided_by: external + depends_on: [load_balancer] + required_for_delivery: partial + binding_fields: + - { source: "load_balancer.vip_address", target: "dns_internal.target_address" } + fields_from_parent: + - { source: "app_name", target: "hostname" } + + - name: dns_public + resource_type: DNS.Record + provided_by: external + depends_on: [load_balancer] + required_for_delivery: optional + condition: "parent.expose_public == true" + binding_fields: + - { source: "load_balancer.public_vip_address", target: "dns_public.target_address" } + fields_from_parent: + - { source: "app_name", target: "hostname" } + +lifecycle_policy: + on_constituent_failure: rollback_all + decommission_order: reverse_dependency +``` + +### What the Consumer Submits + +```json +POST /api/v1/requests +{ + "catalog_item_uuid": "webapp-standard-uuid", + "fields": { + "app_name": "pet-clinic", + "environment": "production", + "db_engine": "postgresql", + "db_storage_gb": 100, + "app_replicas": 3, + "expose_public": true + } +} +``` + +Six fields. The consumer has no idea this produces 6 resources across 4 different providers. + +### What DCM Executes + +``` +Round 1: Network Segment (no deps) + → Placed with EU-WEST network provider + → Realized: subnet_cidr=10.5.0.0/24, security_group_id=sg-abc123 + +Round 2: Database (binding: network values injected) + → config.network_cidr = 10.5.0.0/24, security_group_id = sg-abc123 + → Placed with EU-WEST database provider + → Realized: ip_address=10.5.0.50, port=5432, credentials_ref=vault:secret/pet-clinic-db + +Round 3: App Server (binding: DB + network values injected) + → config.db_host=10.5.0.50, db_port=5432, db_credentials_ref=vault:secret/pet-clinic-db + → Placed with EU-WEST compute provider, 3 replicas + → Realized: ip_addresses=[10.5.0.10, 10.5.0.11, 10.5.0.12], port=8080 + +Round 4: Load Balancer (binding: app server IPs injected) + → config.backend_pool=[10.5.0.10, 10.5.0.11, 10.5.0.12], backend_port=8080 + → Placed with EU-WEST network provider + → Realized: vip_address=10.5.0.100, public_vip_address=203.0.113.50 + +Round 5: DNS records (binding: LB addresses injected) + → Internal: pet-clinic.internal → 10.5.0.100 + → Public: pet-clinic.example.com → 203.0.113.50 (expose_public=true) + +All 6 constituents realized → composite status: OPERATIONAL +``` + +### Policy Interaction with Patterns + +No new policy types are needed. Each constituent is a standard resource type, and existing policies match naturally: + +| Policy | Fires on | What it does | +|--------|---------|-------------| +| Sovereignty GateKeeper | All 6 constituents | Ensures everything lands in EU-WEST | +| VM sizing limits | App Server | Validates replicas and VM size within tenant tier | +| DB storage limits | Database | Validates db_storage_gb within allowed range | +| Network naming | Network Segment, DNS | Enforces naming conventions | +| Monitoring injection | App Server, Database | Injects monitoring agent config | +| Backup policy injection | Database | Injects backup schedule based on environment | + +### Who Authors Patterns vs Who Consumes Them + +| Role | Responsibility | +|------|---------------| +| **Platform Engineer** | Authors pattern definitions as compound Resource Type Specs. Defines constituents, dependencies, binding fields, exposed parameters, lifecycle policies. Registers in Resource Type Registry. Creates service catalog items. | +| **Policy/Compliance Owner** | Writes policies that apply to pattern constituents. Does not need pattern-specific awareness — policies match on resource types, which patterns decompose into. May write pattern-level policies (e.g., "all ApplicationStack types require monitoring on every constituent"). | +| **Consumer Developer** | Browses catalog, selects a pattern, fills in parameters, submits. Sees aggregate status. Can drill into constituent detail. Does not need to understand the decomposition. | +| **Infrastructure Operator** | Provides the atomic services that patterns compose. Registers providers for Compute, Network, Database, DNS — not for the pattern itself. | + +### Pattern Interaction with Other DCM Features + +| Feature | How it works with patterns | +|---------|--------------------------| +| **Drift detection** | Each constituent independently discoverable. Drift attributed per-constituent. | +| **Decommission** | Reverse-dependency-order teardown. Individual constituents can also be removed independently. | +| **Rehydration** | All constituents rebuilt in dependency order with current policies. Binding fields resolve against newly realized values. | +| **Sovereignty** | Every constituent independently sovereignty-checked. A pattern cannot span zones unless every constituent passes. | +| **Cost estimation** | Pattern cost = sum of constituent costs from provider catalog items. | +| **Audit** | Each constituent has its own Merkle audit trail. Composite entity links all constituent entity_uuids. | +| **Override** | Block on any constituent blocks the pattern. Consumer resolves per-constituent. Escalation routes to the responsible policy domain owner. | +| **Federation** | Constituents can be placed across DCM instances. Database local, app servers federated — if sovereignty permits. | + +### Pattern Lifecycle + +Patterns follow the standard DCM artifact lifecycle: `developing → proposed → active → deprecated → retired`. Adding an optional constituent (e.g., a cache layer) is a minor version bump — existing deployments unaffected, new requests get the new constituent. Removing a required constituent is a major version bump. + +--- + +# Architecture Principles + +- **Management plane, not provisioning tool.** DCM orchestrates lifecycle and enforces governance. Provisioning is delegated to service providers that implement the provider contract. +- **Three abstractions.** Everything in DCM is Data, Provider, or Policy. No exceptions. If a new concept doesn't map to one of these three, the abstraction model needs revision. +- **Provider-agnostic.** Any infrastructure platform is consumable through the same interface via naturalization/denaturalization. Providers declare capabilities (realize_resources, serve_data, authenticate, federate, execute_workflows) rather than being assigned rigid types. Multi-capability providers register once. No lock-in to any platform. +- **Discoverable.** DCM advertises its capabilities via a machine-readable endpoint. External systems query what DCM offers and subscribe to data streams without reading documentation. Providers declare what they need from DCM at registration; DCM matches needs to capabilities automatically. +- **Policy-mandatory.** Every request is policy-evaluated. This is not optional. Governance is the value proposition, not a feature toggle. +- **Tamper-evident audit.** Every mutation is recorded in a Merkle tree with Ed25519 signatures. Auditors can verify integrity without trusting DCM. +- **Sovereignty first-class.** Data residency is enforced at request time on every lifecycle operation, not discovered after deployment. +- **Minimal infrastructure.** PostgreSQL is the only required dependency. Everything else (Kafka, Vault, Keycloak) is optional and follows the Internal/External delegation pattern. +- **Declarative and idempotent.** Consumers declare desired state. DCM converges toward it. Resubmitting the same request produces the same result. +- **API-first.** All capabilities are accessible via API. The Web UI (RHDH) is a consumer of the same API. AEP conventions (snake_case, JSON, flat REST). + +--- + +# Success Outcomes + +Each capability area maps to a measurable outcome: + +| **Outcome** | **Enabling Use Cases** | +|-----------|----------------------| +| Developers provision infrastructure without knowing which platform fulfills it | UC-010, UC-011, UC-014 | +| Compound applications deploy as a single catalog request with dependency resolution | UC-011, UC-100 | +| Policy violations are caught at request time with actionable guidance | UC-004, UC-012 | +| Sovereignty and data residency enforced on every lifecycle operation | UC-050 | +| Audit integrity is mathematically provable to external auditors | UC-052, UC-053 | +| Drift between intended and actual state is detected and remediated | UC-022 | +| New infrastructure platforms are addable without changing DCM core | UC-002 | +| Bootstrap requires only PostgreSQL — no middleware stack | UC-001 | +| Override governance provides flexibility without undermining compliance | UC-012, UC-053 | +| Full lifecycle management from request through decommission | UC-010, UC-020, UC-040 | +| Disaster recovery rebuilds from stored intent with current policy evaluation | UC-030 | +| Multi-region placement without consumer awareness of DCM instance topology | UC-060 | +| ITSM records created automatically without gating provisioning by default | UC-090 | +| Self-service reduces platform engineering ticket volume | UC-010, UC-011, UC-014 | + +--- + +# Priority Summary + +| **Priority** | **Count** | **Use Cases** | +|------------|---------|-------------| +| **P0 (Must Have)** | 16 | UC-001 through UC-004, UC-010 through UC-012, UC-014, UC-020, UC-022, UC-040, UC-050, UC-052, UC-071, UC-081, UC-100 | +| **P1 (Should Have)** | 9 | UC-005, UC-013, UC-021, UC-030, UC-051, UC-053, UC-070, UC-080, UC-091 | +| **P2 (Future)** | 6 | UC-023, UC-024, UC-031, UC-060, UC-072, UC-090 | + +--- + +# Open Design Questions + +| **ID** | **Question** | **Status** | +|--------|-------------|-----------| +| **DQ-1** | Application definition language — How should consumers define multi-resource applications? YAML manifests, API composition, external DSL, or catalog-only? | Open — See [ADR-016](adr/016-application-definition-language.md) | +| **DQ-2** | RHDH integration depth — Is RHDH the sole frontend, or should DCM expose its own lightweight UI for environments without RHDH? | Open | +| **DQ-3** | Spectral AEP linter — Should OpenAPI specs be linted in CI? Requires 3.1 compatibility verification. | Open — See [DISCUSSION-TOPICS item 7](DISCUSSION-TOPICS.md) | +| **DQ-4** | Kessel integration — What, if any, integration with Project Kessel for authorization? | Discussion only — See doc 44 | + +--- + +# Appendix: Key Terms + +- **Naturalization:** Translation of DCM's unified payload into a provider's native API format. +- **Denaturalization:** Translation of a provider's native response back into DCM's unified format. +- **Evaluation Context:** The complete payload, provenance chain, constraint accumulator, and governance scope passed to the policy engine for each evaluation. +- **Binding Fields:** Declarations in composite service definitions that connect realized outputs of one resource (e.g., an IP address) to inputs of a dependent resource (e.g., a VM's network config). +- **Merkle Tree:** A binary hash tree where modifying any leaf changes the root hash. Enables inclusion proofs (a record exists) and consistency proofs (the tree has only grown). RFC 9162. +- **RLS (Row-Level Security):** PostgreSQL feature that automatically scopes every query to the actor's tenant — application code cannot leak cross-tenant data. +- **Sovereignty Zone:** A geopolitical or regulatory boundary declared by providers and enforced by policy. Resources placed in a zone are governed by that zone's data residency rules. +- **Shadow Mode:** A policy lifecycle stage where the policy evaluates against real traffic and logs results without blocking requests. Used for safe validation before activation. +- **RHDH (Red Hat Developer Hub):** Backstage-based developer portal used as DCM's primary Web UI frontend. +- **AEP (API Enhancement Proposals):** Open-source API design guidelines adopted by DCM for consistent API conventions. From 2170c2c0030fa8d06e2eaaf4019f0221d57b1a0d Mon Sep 17 00:00:00 2001 From: croadfeldt Date: Tue, 7 Jul 2026 23:17:01 -0500 Subject: [PATCH 2/2] sync: update to current upstream (dcm main) Brings this PR's files up to the current DCM spec state (post UDLM/DCM separation, AEP alignment, taxonomy reconciliation, and review resolutions). Content-only update on existing paths. Signed-off-by: croadfeldt --- architecture/DCM-Capabilities-Matrix.md | 26 +++++++++++++++------ architecture/DISCUSSION-TOPICS.md | 24 +++++++++---------- architecture/dcm-pattern-catalog-overlay.md | 2 +- architecture/dcm-platform-requirements.md | 10 ++++---- 4 files changed, 37 insertions(+), 25 deletions(-) diff --git a/architecture/DCM-Capabilities-Matrix.md b/architecture/DCM-Capabilities-Matrix.md index ff3547e..76d791f 100644 --- a/architecture/DCM-Capabilities-Matrix.md +++ b/architecture/DCM-Capabilities-Matrix.md @@ -321,7 +321,7 @@ | ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| -| FCM-001 | Consumer Policy Authoring | Author and submit tenant-domain policies (GateKeeper, Transformation, Recovery, Lifecycle, Orchestration Flow, Governance Matrix rules) via API or Flow GUI; receive PR URL and shadow mode results | — | Configure consumer policy authoring permissions (policy_author role); manage review requirements per profile | POL-001, IAM-003, IAM-007 | +| FCM-001 | Consumer Policy Authoring | Author and submit tenant-domain policies (Gating Policy, Transformation, Recovery, Lifecycle, Orchestration Flow, Governance Matrix rules) via API or Flow GUI; receive PR URL and shadow mode results | — | Configure consumer policy authoring permissions (policy_author role); manage review requirements per profile | POL-001, IAM-003, IAM-007 | | FCM-002 | Provider Resource Type Publication | — | Publish Resource Type Specifications and Catalog Items for offered resource types via provider contribution API; receive registry PR for platform admin review | Manage provider contribution registry; configure review requirements for provider specs; manage Organization-tier registry | PRV-001, GOV-003 | | FCM-003 | Provider Service Layer Contribution | — | Contribute Service Layers for offered resource types; layers applied during request assembly for all consumers requesting that resource type | Review and activate provider-contributed layers; manage layer compatibility | PRV-001, LAY-002 | | FCM-004 | Consumer Resource Group and Definition Contribution | Author and manage resource groups, notification subscriptions, webhook registrations, and cross-tenant authorization records within own Tenant | — | Configure contribution permissions per role; manage Tenant-scoped artifact lifecycle | IAM-007, GOV-002 | @@ -336,7 +336,7 @@ | ID | Capability | Consumer | Service Provider | Platform/Admin | Depends On | |----|-----------|---------|---------|---------------|-----------| -| SMX-001 | Operational GateKeeper Scoring | Receive risk score and score_drivers with request acknowledgment; understand why score is at its level | Declare `enforcement_class: operational` and `scoring_weight` on contributed GateKeeper policies | Configure operational GateKeeper policies with appropriate weights; manage per-policy enforcement class | POL-001, REQ-004 | +| SMX-001 | Operational Gating Policy Scoring | Receive risk score and score_drivers with request acknowledgment; understand why score is at its level | Declare `enforcement_class: operational` and `scoring_weight` on contributed Gating policies | Configure operational Gating policies with appropriate weights; manage per-policy enforcement class | POL-001, REQ-004 | | SMX-002 | Advisory Validation and Completeness Score | Receive advisory_warnings list with request acknowledgment; understand what optional improvements exist | Declare `output_class: advisory` on advisory Validation policies | Configure advisory Validation policies; manage completeness score thresholds | POL-001, REQ-004 | | SMX-003 | Actor Risk History Tracking | View own risk history score and contributing events via Consumer API | — | Monitor actor risk history; reset scores for trusted automation accounts; configure decay parameters | AUD-001, IAM-001 | | SMX-004 | Quota Pressure Scoring | Receive quota_pressure as a score driver when approaching Tenant quota limits | — | Configure per-resource-type quota limits; manage free_threshold parameter | IAM-007, REQ-004 | @@ -344,7 +344,7 @@ | SMX-006 | Profile Scoring Threshold Management | — | — | Configure approval routing thresholds per profile (auto/reviewed/verified/authorized + custom tiers via named-tier list); manage signal weights; enforce SMX-008 (max auto_approve_below: 50) | POL-005, REQ-004 | | SMX-007 | Policy Enforcement Class Override | — | Contribute policies with declared enforcement_class; receive notification when profile overrides enforcement class | Declare per-profile enforcement class overrides; manage regulatory_mandate flag to protect compliance-class policies from demotion | POL-004, POL-005 | | SMX-008 | Score Audit Trail | Query risk score and routing decision for own requests; view score_drivers and advisory_warnings | — | Query full Score Record detail including signal breakdown and actor risk history; manage score audit retention | AUD-001, REQ-004 | -| SMX-009 | Scoring Weight Range Enforcement | — | Declare operational GateKeeper scoring_weight between 1 and 100 | Enforce weight range at policy activation; reject out-of-range weights | SMX-001 | +| SMX-009 | Scoring Weight Range Enforcement | — | Declare operational Gating Policy scoring_weight between 1 and 100 | Enforce weight range at policy activation; reject out-of-range weights | SMX-001 | | SMX-010 | Score Breakdown Audit Inclusion | View score breakdown in request audit record | — | Configure score breakdown storage in Audit Store for all scored requests | SMX-001, AUD-001 | --- @@ -477,7 +477,7 @@ |----|-----------|---------|---------|---------------|-----------| | SCH-001 | Request Scheduling | Submit requests with schedule.dispatch: at/window/recurring; SCHEDULED requests visible in GET /api/v1/requests; cancellable before dispatch; receive request.scheduled event | — | Manage Maintenance Windows; configure Request Scheduler; monitor scheduled queue depth | REQ-001 | | SCH-002 | Maintenance Windows | Reference maintenance windows in scheduled requests; view available windows at GET /api/v1/maintenance-windows | — | Create/manage/suspend maintenance windows; approve window schedules; configure platform-wide windows | SCH-001, GOV-001 | -| SCH-003 | Dual Policy Evaluation | — | — | Understand that scheduled requests run GateKeeper at declaration AND at dispatch; dispatch-time failure → FAILED with schedule_policy_rejection (SCH-003) | SCH-001, POL-001 | +| SCH-003 | Dual Policy Evaluation | — | — | Understand that scheduled requests run Gating Policy at declaration AND at dispatch; dispatch-time failure → FAILED with schedule_policy_rejection (SCH-003) | SCH-001, POL-001 | | SCH-004 | Deadline Enforcement | Set not_after on scheduled requests; receive request.failed(schedule_deadline_missed) if deadline passes without dispatch | — | Monitor deadline miss rates; configure alerting on deadline misses | SCH-001, EVT-001 | | SCH-005 | Not-After Expiry Failure | Receive FAILED status when scheduled request expires before dispatch | — | Configure not_after enforcement; manage SCHEDULE_EXPIRED recovery policy | SCH-001 | | SCH-006 | Maintenance Window Platform Authorization | — | — | Require platform_admin or tenant_admin authority to create/modify Maintenance Windows | SCH-004 | @@ -534,8 +534,8 @@ | GUI-005 | Admin Panel — Platform Dashboard | Control plane component health grid; provider health summary; pending approvals count; open drift records by severity; request throughput; all driven by GET /api/v1/admin/health | — | Platform Admins, SREs configure dashboard widgets; role-gated sections | HLT-003 | | GUI-006 | Admin Panel — Governance and Approvals | Approval queue (all tenants); approval detail with risk score breakdown; authority tier registry editor (drag-and-drop reordering, impact report visualization, degradation acceptance flow); scoring threshold editor (auto_approve_below ≤ 50 hard-stop) | — | Policy Owners and Platform Admins | ATM-004, SMX-001 | | GUI-007 | Admin Panel — Audit and Compliance | Platform-wide cross-tenant audit trail; pre-built compliance reports (SOC 2, FedRAMP, HIPAA); audit chain integrity status; correlation ID trace; session and security event feed | — | Auditors, Security team, Platform Admins | AUD-001, SES-003 | -| GUI-008 | Provider Management — Common Shell | Overview, configuration, health history, audit trail, and notification tabs for all 11 provider types; provider owner role gates access; Platform Admins see all providers | — | Provider owners manage own providers; Platform Admins manage all | PRV-001, IAM-001 | -| GUI-009 | Provider Management — Type Extensions | Service Provider: capacity, managed entities, naturalization mapping, realization history; secrets management: inventory, rotation, revocation, external CA config, algorithm compliance; Auth Provider: session stats, SCIM sync, connection status; external policy evaluation: trust level, contribution pipeline | — | Provider owners access type-specific tabs for their provider type | GUI-008, PRV-001 | +| GUI-008 | Provider Management — Common Shell | Overview, configuration, health history, audit trail, and notification tabs per provider, organized by declared CAPABILITY (not a fixed 11-type taxonomy — ADR-005); provider owner role gates access; Platform Admins see all providers | — | Provider owners manage own providers; Platform Admins manage all | PRV-001, IAM-001 | +| GUI-009 | Provider Management — Type Extensions | Service Provider: capacity, managed entities, naturalization mapping, realization history; secrets management: inventory, rotation, revocation, external CA config, algorithm compliance; authentication capability: session stats, SCIM sync, connection status (auth is a capability, not a standalone provider type — ADR-005); external policy evaluation: trust level, contribution pipeline | — | Provider owners access type-specific tabs for their provider type | GUI-008, PRV-001 | | GUI-011 | RHDH Plugin Suite | Use DCM capabilities within Red Hat Developer Hub (RHDH) or Backstage via Dynamic Plugins (@dcm/backstage-plugin-*); no RHDH rebuild required for updates | — | Configure RHDH app-config.yaml with DCM connection; configure Dynamic Plugin loading | | GUI-012 | Scaffolder Template Auto-Generation | DCM catalog items automatically generate Backstage Software Templates; new resource types appear as templates without UI code; field schema → JSON Schema → Scaffolder form | — | Configure @dcm/backstage-plugin-catalog-backend; template generation is automatic | | GUI-013 | DCM Entity Provider | DCMService (catalog items) and DCMResource (realized entities) appear in RHDH Software Catalog; entities sync every PT5M; search-indexed; tenancy enforced via namespace | — | Service account credential configuration; sync interval configuration | @@ -663,7 +663,7 @@ | Federated Contribution Model | 7 | | Scoring Model | 10 | | Composite Service Composition | 8 | -| secrets management Model | 12 | +| Credential Management (CPX) | 12 | | Authority Tier Model | 12 | | Event Catalog | 7 | | API Versioning | 8 | @@ -708,6 +708,18 @@ PRV-001 (Provider Registration) — parallel critical path IAM-001 → IAM-002 → IAM-003 → IAM-007 → CAT-001 → REQ-001 → REQ-002 → REQ-003 → REQ-004 → REQ-005 → REQ-006 → REQ-007 → PRV-001 → PRV-002 → PRV-003 → PRV-004 → PRV-005 → LCM-001 → DRF-001 → DRF-002 → AUD-001 +### Findings-review additions (2026-07-07) — capability domains flagged as absent + +These domains have ADR backing but were missing from the matrix (consistency review). Rows are +intentionally concise pointers pending the DCM team's detailed capability breakdown: + +| ID | Capability | Description | ADR | +|----|-----------|-------------|-----| +| PLP-001 | Placement Policy | Declarative affinity/anti-affinity/spread/co-locate/pin over abstract Topology kinds; the 8th typed policy; engine evaluates + enforces portability | ADR-019 | +| MIG-001 | Migration & Operational Gating | Migration permission (Governance-Matrix) + sequence (Orchestration-Flow) + freshness gating + rehearsal scheduling | ADR-020 | +| TRU-001 | Trust Model / Introduction Grant | Five trust planes; credential-API selection; the Introduction Grant primitive for provider onboarding | ADR-022 | +| ING-005 | Ingestion — claim/adoption + backport | Reverse placement, provider claim (Discovered→Realized), correlation-id entity resolution, Intent backport | ADR-017 | + **21 capabilities for a functional end-to-end demonstration.** **Note:** FCM-001 through FCM-007 (Federated Contribution Model) are not on the critical path — they extend DCM's multi-user capabilities but are not required for the initial end-to-end lifecycle demonstration. diff --git a/architecture/DISCUSSION-TOPICS.md b/architecture/DISCUSSION-TOPICS.md index 844e6f8..a2ffc7a 100644 --- a/architecture/DISCUSSION-TOPICS.md +++ b/architecture/DISCUSSION-TOPICS.md @@ -40,8 +40,8 @@ platform exists or a canned solution is desired, a packaged **dcm-observability** component serves as the authoritative telemetry/monitoring platform (provider-contract §7 / PRV-007). - First consumer and reference-implementation test bed: the roadfeldt - homelab observability stack (`roadfeldt-observability`). + First consumer and reference-implementation test bed: a homelab + observability stack. - **Brownfield inventory ingestion — adopt, don't recreate** *(added 2026-06-07)* — migrating existing configuration-management estates (Ansible inventory) into DCM. Patterns established from live homelab migrations: @@ -153,7 +153,7 @@ DCM needs a webhook integration model that allows external systems to be notifie - Notify an external ITSM system (ServiceNow, Jira) when a request is created, updated, or completed - Notify a monitoring system when Entity lifecycle state changes - Notify a FinOps platform when new Resource/Service Entities are realized or decommissioned -- Notify a compliance system when GateKeeper policies fire or sovereignty constraints are applied +- Notify a compliance system when Gating policies fire or sovereignty constraints are applied **Operational Notifications** - Notify SRE teams when provider capacity falls below threshold @@ -176,7 +176,7 @@ DCM needs a webhook integration model that allows external systems to be notifie 7. **Filtering** — can webhook registrations declare filters — only receive events of specific types, for specific Resource Types, for specific Tenants, or for specific Resource Groups? -8. **Policy Engine integration** — should GateKeeper and other policy types be able to trigger webhook notifications as a policy action? This would make webhooks a first-class policy response alongside ALERT, REVERT, etc. +8. **Policy Engine integration** — should Gating Policy and other policy types be able to trigger webhook notifications as a policy action? This would make webhooks a first-class policy response alongside ALERT, REVERT, etc. 9. **Provider webhook obligations** — should providers be required to support webhook endpoints as part of their Provider Contract? Or is webhook support optional for providers? @@ -234,25 +234,25 @@ The Intent State is captured when a consumer submits a request but the exact str --- -### TOPIC-003 — GateKeeper vs Validation Policy Distinction +### TOPIC-003 — Gating Policy vs Validation Policy Distinction **Area:** Policy Engine **Priority:** P2 **Status:** 🟢 Resolved **Resolved:** 2026-03 -**Resolution:** GateKeeper has enforcement_class (compliance=boolean deny, operational=risk score). Validation has output_class (structural=boolean fail, advisory=completeness score). GateKeeper can both block AND modify. See doc 29 (Scoring Model), AI Prompt Sections 17, 61. +**Resolution:** Gating Policy has enforcement_class (compliance=boolean deny, operational=risk score). Validation has output_class (structural=boolean fail, advisory=completeness score). Gating Policy can both block AND modify. See doc 29 (Scoring Model), AI Prompt Sections 17, 61. **Raised:** 2026-03 #### Description -The distinction between GateKeeper and Validation policy categories needs better examples and a clearer formal definition. Both involve checking data against rules, but GateKeeper has override authority while Validation is pass/fail only. The boundary between them needs to be unambiguous. +The distinction between Gating Policy and Validation policy categories needs better examples and a clearer formal definition. Both involve checking data against rules, but Gating Policy has override authority while Validation is pass/fail only. The boundary between them needs to be unambiguous. #### Questions to Resolve -1. What is the precise trigger condition that makes a policy a GateKeeper vs a Validation policy? -2. Can a GateKeeper policy both block AND modify in the same execution? -3. Are there cases where Validation and GateKeeper would produce different outcomes for the same rule? -4. Should GateKeeper policies require explicit authorization (e.g., only CISO-owned policies can be GateKeeper)? +1. What is the precise trigger condition that makes a policy a Gating Policy vs a Validation policy? +2. Can a Gating policy both block AND modify in the same execution? +3. Are there cases where Validation and Gating Policy would produce different outcomes for the same rule? +4. Should Gating policies require explicit authorization (e.g., only CISO-owned policies can be Gating Policy)? #### References - [Layering and Versioning](https://github.com/croadfeldt/udlm/blob/main/foundations/layering-and-versioning.md) — Policy Layer section @@ -351,7 +351,7 @@ The dependency model currently assumes dependencies are resolved within a single 2. What authorization is required for a cross-tenant dependency? Does the owning Tenant need to approve? 3. How does cost attribution work for cross-tenant service consumption? 4. How does drift detection work when a dependency is in another Tenant? -5. Can a GateKeeper policy block cross-tenant dependencies? +5. Can a Gating policy block cross-tenant dependencies? --- diff --git a/architecture/dcm-pattern-catalog-overlay.md b/architecture/dcm-pattern-catalog-overlay.md index 9ade886..83b1f59 100644 --- a/architecture/dcm-pattern-catalog-overlay.md +++ b/architecture/dcm-pattern-catalog-overlay.md @@ -298,7 +298,7 @@ Policies don't need special awareness of patterns. Each constituent is a standar | Policy | Fires on which constituent | What it does | |--------|---------------------------|-------------| -| Sovereignty GateKeeper | All 6 | Ensures all constituents land in EU-WEST | +| Sovereignty Gating Policy | All 6 | Ensures all constituents land in EU-WEST | | VM sizing limits | App Server only | Validates replica count and VM size within tenant tier | | DB storage limits | Database only | Validates db_storage_gb within allowed range | | Network naming | Network Segment, DNS | Enforces naming conventions | diff --git a/architecture/dcm-platform-requirements.md b/architecture/dcm-platform-requirements.md index 76a398d..7470db8 100644 --- a/architecture/dcm-platform-requirements.md +++ b/architecture/dcm-platform-requirements.md @@ -60,7 +60,7 @@ Manages the physical and virtual infrastructure that DCM's providers abstract. I ## Policy and Compliance Owner -Defines and manages the governance rules that DCM enforces. Authors GateKeeper policies (allow/deny), validation policies, sovereignty constraints, and override approval rules. Reviews audit trails and compliance reports. +Defines and manages the governance rules that DCM enforces. Authors Gating policies (allow/deny), validation policies, sovereignty constraints, and override approval rules. Reviews audit trails and compliance reports. - **Key activities:** Author and activate policies, configure policy profiles (minimal through sovereign), review override requests (dual-approval), verify audit integrity, manage compliance rescans @@ -134,7 +134,7 @@ A platform engineer defines resource types and catalog items. A resource type sp ### UC-004: Configure Organizational Policies -A policy and compliance owner authors the baseline policy set: a sovereignty GateKeeper policy that blocks resources from deploying outside designated zones (hard enforcement); a sizing validation policy that enforces CPU and memory limits per tenant tier; a naming transformation policy that auto-generates standardized hostnames; a monitoring transformation policy that injects the organization's monitoring agent into every production resource; and a cost GateKeeper policy that blocks requests exceeding budget thresholds. Each policy is first deployed in **shadow mode** — it evaluates against real traffic and logs results without blocking requests. After validation, the policy is promoted to active. Policies use the Gatekeeper ConstraintTemplate pattern: reusable Rego logic with parameterized instances. +A policy and compliance owner authors the baseline policy set: a sovereignty Gating policy that blocks resources from deploying outside designated zones (hard enforcement); a sizing validation policy that enforces CPU and memory limits per tenant tier; a naming transformation policy that auto-generates standardized hostnames; a monitoring transformation policy that injects the organization's monitoring agent into every production resource; and a cost Gating policy that blocks requests exceeding budget thresholds. Each policy is first deployed in **shadow mode** — it evaluates against real traffic and logs results without blocking requests. After validation, the policy is promoted to active. Policies use the Gatekeeper ConstraintTemplate pattern: reusable Rego logic with parameterized instances. **Success criteria:** Policies evaluate correctly in shadow mode. No false positives on legitimate requests. Activation enforces the policy on all matching requests. Audit records produced for every evaluation. @@ -165,7 +165,7 @@ A consumer developer browses the service catalog, selects "Virtual Machine — S 2. **Layer assembly** — 5 data layers merge organizational context (datacenter, environment, compliance, tenant, provider defaults) into the consumer's 6 fields, producing 10+ fields with full provenance 3. **Dependency resolution** — Resource type spec declares VM requires Network.IPAddress; DCM creates an IP sub-request automatically 4. **IP policy evaluation** — Sovereignty, subnet isolation, and pool selection policies evaluate against the IP sub-request; IPAM provider selected and IP allocated -5. **VM policy evaluation** — GateKeeper (sizing, sovereignty, approved OS images), Validation (field constraints), and Transformation (monitoring injection) policies evaluate +5. **VM policy evaluation** — Gating Policy (sizing, sovereignty, approved OS images), Validation (field constraints), and Transformation (monitoring injection) policies evaluate 6. **Placement** — Sovereignty pre-filter eliminates non-compliant providers; remaining providers scored by capacity and confidence; best provider selected 7. **Dispatch** — Request Orchestrator sends the assembled payload (including the dependency-injected IP address) to the selected provider; the provider naturalizes DCM's unified payload into its native API 8. **Realization** — Provider provisions the VM, denaturalizes the result back to DCM's format, and callbacks with realized state @@ -264,7 +264,7 @@ A consumer or TTL trigger initiates decommission. DCM checks for dependencies: i ### UC-050: Enforce Sovereignty and Data Residency -All resources handling restricted, PHI, or PCI data are placed exclusively in designated sovereignty zones. The sovereignty GateKeeper policy fires on **every lifecycle operation** (initial provisioning, update, scale, rehydration, ownership transfer) — not just initial provisioning. A resource in EU-WEST stays in EU-WEST for its entire lifecycle. +All resources handling restricted, PHI, or PCI data are placed exclusively in designated sovereignty zones. The sovereignty Gating policy fires on **every lifecycle operation** (initial provisioning, update, scale, rehydration, ownership transfer) — not just initial provisioning. A resource in EU-WEST stays in EU-WEST for its entire lifecycle. Override requires dual-approval: two approvers from different roles, with written justification and compensating controls. Every override produces a Merkle audit leaf at field granularity. @@ -593,7 +593,7 @@ No new policy types are needed. Each constituent is a standard resource type, an | Policy | Fires on | What it does | |--------|---------|-------------| -| Sovereignty GateKeeper | All 6 constituents | Ensures everything lands in EU-WEST | +| Sovereignty Gating Policy | All 6 constituents | Ensures everything lands in EU-WEST | | VM sizing limits | App Server | Validates replicas and VM size within tenant tier | | DB storage limits | Database | Validates db_storage_gb within allowed range | | Network naming | Network Segment, DNS | Enforces naming conventions |