Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/check-format.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Check formatting

on:
pull_request:
branches:
- main
paths:
- 'content/docs/**'
- 'content/blog/**'
- 'Makefile'
- '.prettierignore'

jobs:
check:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: '20'

- name: Install prettier
run: npm install -g prettier

- name: Check formatting
run: PRETTIER=prettier make check-format
6 changes: 6 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Hugo homepage and section stubs (shortcodes, inline HTML)
content/_index.md
content/about/_index.md

# RemoteMD wrappers; canonical source is in the enhancements repo
content/docs/enhancements/
12 changes: 11 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: build serve clean check-spell
.PHONY: build serve clean check-spell format check-format

# Use cspell when installed; otherwise npx. Set SPELLCHECK when invoking make to override.
ifeq ($(SPELLCHECK),)
Expand All @@ -8,7 +8,11 @@ ifeq ($(SPELLCHECK),)
SPELLCHECK := cspell
endif
endif
PRETTIER ?= npx prettier
FILE ?= **/*.md
# Prose docs only; see .prettierignore for paths under these trees to skip.
FORMAT_FILES ?= content/docs/**/*.md content/blog/**/*.md
PRETTIER_FLAGS ?= --prose-wrap always --print-width 80

build:
hugo --gc --minify
Expand All @@ -21,3 +25,9 @@ clean:

check-spell:
$(SPELLCHECK) "$(FILE)"

format:
$(PRETTIER) $(PRETTIER_FLAGS) --write $(FORMAT_FILES)

check-format:
$(PRETTIER) $(PRETTIER_FLAGS) --check $(FORMAT_FILES)
1 change: 0 additions & 1 deletion content/blog/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,3 @@ title: Blog

Stay updated with the latest news, releases, and technical articles about the
DCM project.

84 changes: 42 additions & 42 deletions content/blog/sovereignty-rehydrate-demo/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ images:

It is 2 a.m. when your on-call team gets an alert that a Region A datacenter is
down. The application holding customer data is unreachable, customers are
waiting, and someone in leadership is already asking for an ETA. The engineer
on duty opens the provider list and finds a healthy datacenter in Region B at
the top. If you are only looking at infrastructure health, that is where you
would send the workload next.
waiting, and someone in leadership is already asking for an ETA. The engineer on
duty opens the provider list and finds a healthy datacenter in Region B at the
top. If you are only looking at infrastructure health, that is where you would
send the workload next.

The workload was never supposed to leave Region A. Whether the constraint comes
from GDPR, a data residency requirement, a customer contract, or an internal
Expand All @@ -46,10 +46,10 @@ The application still stays in Region A.

We run the scenario on three datacenters split across two regions:

| Region | Datacenters | Notes |
| --- | --- | --- |
| **Region A** | `dc-demo-2`, `dc-demo-3` | Where the app is allowed to live |
| **Region B** | `dc-demo-1` | Healthy, but off limits for this workload |
| Region | Datacenters | Notes |
| ------------ | ------------------------ | ----------------------------------------- |
| **Region A** | `dc-demo-2`, `dc-demo-3` | Where the app is allowed to live |
| **Region B** | `dc-demo-1` | Healthy, but off limits for this workload |

All three start healthy. Here is the twist: **Region B sorts first.** If DCM
only picked the first available provider, every new deployment would land there
Expand All @@ -61,8 +61,9 @@ by default. That is exactly the trap the sovereignty policy is meant to prevent.

## One policy, enforced everywhere

The regional rule is a [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/)
policy you create once in the DCM UI under **Policies → Create**. Name it
The regional rule is a
[Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) policy you
create once in the DCM UI under **Policies → Create**. Name it
`Sovereignty Region Policy`, set type to `GLOBAL`, priority to `1`, and paste
the policy below.

Expand All @@ -84,18 +85,16 @@ Region A workload.

### First match wins (within the region)

Among the survivors, the policy picks the first provider alphabetically. That
is why the initial deploy lands on `dc-demo-2`, and why failover moves to
Among the survivors, the policy picks the first provider alphabetically. That is
why the initial deploy lands on `dc-demo-2`, and why failover moves to
`dc-demo-3` when `dc-demo-2` fails.

{{< callout type="warning" >}}
**What if an entire region goes down?**
{{< callout type="warning" >}} **What if an entire region goes down?**

If no healthy provider exists in the requested region, the request is rejected
with a clear error. DCM will not silently fail over across a regional boundary
to "help." That is the point. A hard failure is easier to explain to a
compliance officer than a quiet data residency violation.
{{< /callout >}}
compliance officer than a quiet data residency violation. {{< /callout >}}

{{% details title="View the full Rego policy" closed="true" %}}

Expand All @@ -104,7 +103,7 @@ package provider.sovereignty

import rego.v1

spm_url := "http://service-provider-manager:8080/api/v1alpha1/providers"
spm_url := "http://control-plane:8080/api/v1alpha1/providers"

main := {"rejected": true, "rejection_reason": "spec.service_type is required"} if {
not input.spec.service_type
Expand Down Expand Up @@ -156,18 +155,18 @@ _providers_result(providers, _, _) := {"rejected": false, "selected_provider": p

### 1. Deploy to Region A

Create a **Pet Clinic** instance from the **Instances** tab and choose
**Region A**. DCM evaluates the sovereignty policy, skips Region B, and places
the app on `dc-demo-2`.
Create a **Pet Clinic** instance from the **Instances** tab and choose **Region
A**. DCM evaluates the sovereignty policy, skips Region B, and places the app on
`dc-demo-2`.

Behind the scenes, DCM stores your original request. That stored intent is the
source of truth for everything that follows.

### 2. Simulate the outage

Take `dc-demo-2` offline (`podman stop` on the provider containers). Region B
is still healthy and still first on the list. In a typical setup, that is where
the story ends and the compliance review begins.
Take `dc-demo-2` offline (`podman stop` on the provider containers). Region B is
still healthy and still first on the list. In a typical setup, that is where the
story ends and the compliance review begins.

Here, the policy has already ruled Region B out.

Expand Down Expand Up @@ -198,14 +197,14 @@ Most platforms draw a hard line between **day-one placement** and **disaster
recovery**. Different workflows, different runbooks, different approval paths.
DCM treats them as the same operation:

| Scenario | What you do | What DCM does |
| --- | --- | --- |
| First deploy | Create a catalog instance | Evaluates policy, places the app |
| Datacenter failure | Rehydrate | Re-evaluates the same policy against current health |
| Planned migration | Rehydrate | Same code path, same policy engine |
| Scenario | What you do | What DCM does |
| ------------------ | ------------------------- | --------------------------------------------------- |
| First deploy | Create a catalog instance | Evaluates policy, places the app |
| Datacenter failure | Rehydrate | Re-evaluates the same policy against current health |
| Planned migration | Rehydrate | Same code path, same policy engine |

**Initial placement, disaster recovery, and migration share one mechanism.**
The catalog instance holds your intent. The policy engine applies your rules.
**Initial placement, disaster recovery, and migration share one mechanism.** The
catalog instance holds your intent. The policy engine applies your rules.
Rehydration is replay, not a separate rebuild workflow invented for emergencies.

That consistency is what makes regional sovereignty hold at 2 a.m. the same way
Expand All @@ -221,17 +220,18 @@ Click through the DCM UI at your own pace: explore the providers, create the
policy, deploy Pet Clinic, and watch where the app lands.

The walkthrough covers almost the entire scenario in the browser, including
rehydration from the **Instances** tab. The step that still needs a
terminal today is simulating the datacenter failure.
The guide flags that when you reach it.
rehydration from the **Instances** tab. The step that still needs a terminal
today is simulating the datacenter failure. The guide flags that when you reach
it.

Want to reproduce the full lab on your own hardware? You will need the
[DCM stack](/docs/getting-started/local-setup/), [three workload clusters with
region-tagged providers](https://github.com/dcm-project/api-gateway/blob/main/RUN.md#running-with-service-providers)
(see [`SP_REGION`](https://github.com/dcm-project/k8s-container-service-provider#provider-identity)
for region metadata), and the [Pet Clinic catalog
item](https://github.com/dcm-project/api-gateway/blob/main/docs/three-tier-app-kind.md)
configured with a [deployment region
field](/docs/user-guide/catalog-items/#fields-and-policy-evaluation). If you are
new to DCM, start with the walkthrough above and come back to the lab setup when
you are ready to go deeper.
[DCM stack](/docs/getting-started/local-setup/),
[three workload clusters with region-tagged providers](https://github.com/dcm-project/control-plane/blob/main/deploy/RUN.md#running-with-service-providers)
(see
[`SP_REGION`](https://github.com/dcm-project/k8s-container-service-provider#provider-identity)
for region metadata), and the
[Pet Clinic catalog item](https://github.com/dcm-project/control-plane/blob/main/deploy/docs/three-tier-app-kind.md)
configured with a
[deployment region field](/docs/user-guide/catalog-items/#fields-and-policy-evaluation).
If you are new to DCM, start with the walkthrough above and come back to the lab
setup when you are ready to go deeper.
9 changes: 6 additions & 3 deletions content/docs/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ Documentation for the DCM project.

## Documentation Sections

- **[Getting Started](getting-started/)** - Set up and run DCM on your local machine.
- **[User Guide](user-guide/)** - Manage DCM resources using the CLI or the web UI.
- **[Enhancements](enhancements/)** - Enhancement proposals documenting new features, architectural decisions, and significant changes to the DCM project.
- **[Getting Started](getting-started/)** - Set up and run DCM on your local
machine.
- **[User Guide](user-guide/)** - Manage DCM resources using the CLI or the web
UI.
- **[Enhancements](enhancements/)** - Enhancement proposals documenting new
features, architectural decisions, and significant changes to the DCM project.
21 changes: 14 additions & 7 deletions content/docs/getting-started/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,17 @@ weight: 1

Get up and running with DCM.

- **[Architecture Overview](architecture/)** — How DCM components fit together and the request flow.
- **[Local Setup](local-setup/)** — Install prerequisites, start DCM services, and build the CLI.
- **[Create Small VM Catalog Item](create-small-vm-catalog-item/)** — Define and create a catalog item for a small virtual machine.
- **[Create Placement Policy](create-placement-policy/)** — Define a placement policy to route instances to the KubeVirt service provider.
- **[Create Instance of Small VM Catalog Item](create-small-vm-instance/)** — Deploy a VM by creating an instance of the catalog item.
- **[Register Another Provider](register-another-provider/)** — Register a second service provider and create a random selection policy.
- **[Troubleshooting](troubleshooting/)** — Diagnose issues using container logs and status checks.
- **[Architecture Overview](architecture/)** — How DCM components fit together
and the request flow.
- **[Local Setup](local-setup/)** — Install prerequisites, start DCM services,
and build the CLI.
- **[Create Small VM Catalog Item](create-small-vm-catalog-item/)** — Define and
create a catalog item for a small virtual machine.
- **[Create Placement Policy](create-placement-policy/)** — Define a placement
policy to route instances to the KubeVirt service provider.
- **[Create Instance of Small VM Catalog Item](create-small-vm-instance/)** —
Deploy a VM by creating an instance of the catalog item.
- **[Register Another Provider](register-another-provider/)** — Register a
second service provider and create a random selection policy.
- **[Troubleshooting](troubleshooting/)** — Diagnose issues using container logs
and status checks.
55 changes: 25 additions & 30 deletions content/docs/getting-started/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,17 @@ type: docs
weight: 1
---

DCM (Data Center Management) is a control plane for managing infrastructure services across multiple providers. This page gives a high-level overview of how the components fit together.
DCM (Data Center Management) is a control plane for managing infrastructure
services across multiple providers. This page gives a high-level overview of how
the components fit together.

## Components

```mermaid
graph TD
User["User (CLI / API)"]
GW["API Gateway<br/>(Traefik :9080)"]

subgraph Managers
subgraph CP["Control Plane (:8080)"]
CM["Catalog Manager"]
PM["Policy Manager"]
PLM["Placement Manager"]
Expand All @@ -23,25 +24,20 @@ graph TD
subgraph Infrastructure
PG["PostgreSQL"]
NATS["NATS"]
OPA["OPA"]
end

subgraph Providers
SP1["KubeVirt SP 1"]
SP2["KubeVirt SP 2"]
end

User --> GW
GW --> CM
GW --> PM
GW --> SPM
User --> CP

CM --> PG
PM --> PG
PLM --> PG
SPM --> PG

PM --> OPA
PLM --> PM
PLM --> SPM
CM --> PLM
Expand All @@ -55,17 +51,16 @@ graph TD

## Component Responsibilities

| Component | Role |
|---|---|
| **API Gateway** | Single entry point for all API requests. Routes to the appropriate manager based on URL path. |
| **Catalog Manager** | Manages service types, catalog items, and catalog item instances. Triggers placement when an instance is created. |
| **Policy Manager** | Stores placement policies (Rego) and evaluates them via OPA. |
| **Placement Manager** | Selects a service provider for a new instance by evaluating policies against available providers. |
| **Service Provider Manager** | Tracks registered service providers and their health status. |
| **PostgreSQL** | Persistent storage for all managers. |
| **NATS** | Message bus for communication between the Service Provider Manager and service providers. |
| **OPA** | Evaluates Rego policies for placement decisions. |
| **Service Providers** | External systems (e.g., KubeVirt) that create and manage the actual resources (VMs, containers, etc.). |
| Component | Role |
| ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| **Control Plane** | Single process that exposes all DCM APIs on `:8080`. Hosts the catalog, policy, placement, and service provider managers in one binary. |
| **Catalog Manager** | Manages service types, catalog items, and catalog item instances. Triggers placement when an instance is created. |
| **Policy Manager** | Stores placement policies (Rego) and evaluates them via an embedded OPA engine. |
| **Placement Manager** | Selects a service provider for a new instance by evaluating policies against available providers. |
| **Service Provider Manager** | Tracks registered service providers and their health status. |
| **PostgreSQL** | Persistent storage for the control plane. |
| **NATS** | Message bus for communication between the Service Provider Manager and service providers. |
| **Service Providers** | External systems (e.g., KubeVirt) that create and manage the actual resources (VMs, containers, etc.). |

## Request Flow

Expand All @@ -74,26 +69,26 @@ When a user creates a catalog item instance, the following happens:
```mermaid
sequenceDiagram
participant User
participant GW as API Gateway
participant CP as Control Plane
participant CM as Catalog Manager
participant PLM as Placement Manager
participant PM as Policy Manager
participant OPA
participant SPM as SP Manager
participant SP as KubeVirt SP

User->>GW: Create catalog item instance
GW->>CM: POST /catalog-item-instances
User->>CP: Create catalog item instance
CP->>CM: POST /catalog-item-instances
CM->>PLM: Request placement
PLM->>PM: Evaluate policies
PM->>OPA: Evaluate Rego
OPA-->>PM: Placement decision
PLM->>PM: Evaluate policies (embedded OPA)
PM-->>PLM: Placement decision
PLM->>SPM: Create resource on provider
SPM->>SP: Create VM
SP-->>User: VM scheduling
```

1. The **Catalog Manager** receives the request and asks the **Placement Manager** to find a suitable provider.
2. The **Placement Manager** evaluates placement policies through the **Policy Manager** and **OPA**.
3. Once a provider is selected, the resource is created on that provider through the **Service Provider Manager**.
1. The **Catalog Manager** receives the request and asks the **Placement
Manager** to find a suitable provider.
2. The **Placement Manager** evaluates placement policies through the **Policy
Manager**, which uses an embedded OPA engine to evaluate Rego.
3. Once a provider is selected, the resource is created on that provider through
the **Service Provider Manager**.
7 changes: 6 additions & 1 deletion content/docs/getting-started/create-placement-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ type: docs
weight: 4
---

Placement policies control how DCM decides which service provider should handle a catalog item instance. Policies are written in [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/), the policy language used by Open Policy Agent (OPA). Each policy can accept or reject a placement request and optionally select a specific provider.
Placement policies control how DCM decides which service provider should handle
a catalog item instance. Policies are written in
[Rego](https://www.openpolicyagent.org/docs/latest/policy-language/), the policy
language used by Open Policy Agent (OPA). Each policy can accept or reject a
placement request and optionally select a specific provider.

Policies have a `priority` — lower values are evaluated first.

Expand All @@ -26,6 +30,7 @@ rego_code: |
```

This policy:

- Has the highest priority (`1`)
- Always selects `kubevirt-service-provider` as the target service provider
- Never rejects a request (`rejected: false`)
Expand Down
Loading
Loading