-
Notifications
You must be signed in to change notification settings - Fork 0
feat: implement statefulset primitive #25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
877105f
a20aaf4
dff7299
ccbe198
8054491
f458085
77f2693
490c612
90b402f
d6a38e3
1718512
25c31c6
cde2525
227f800
5657a88
65cd2c5
787cdf5
e851efd
e9b9d12
27f6a63
d7f0248
b0bc771
6578643
ec8729d
1026007
d1876e7
b57c7e9
17f93a4
b1b97a4
aed5593
2370329
aa5d005
0408c40
0283c0d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -122,6 +122,7 @@ build-examples: ## Build all example binaries. | |
| run-examples: ## Run all examples to verify they execute without error. | ||
| go run ./examples/deployment-primitive/. | ||
| go run ./examples/configmap-primitive/. | ||
| go run ./examples/statefulset-primitive/. | ||
| go run ./examples/replicaset-primitive/. | ||
| go run ./examples/rolebinding-primitive/. | ||
| go run ./examples/custom-resource-implementation/. | ||
|
Comment on lines
122
to
128
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,320 @@ | ||
| # StatefulSet Primitive | ||
|
|
||
| The `statefulset` primitive is the framework's built-in workload abstraction for managing Kubernetes `StatefulSet` | ||
| resources. It integrates fully with the component lifecycle and provides a rich mutation API for managing containers, | ||
| pod specs, metadata, and volume claim templates. | ||
|
sourcehawk marked this conversation as resolved.
|
||
|
|
||
|
sourcehawk marked this conversation as resolved.
|
||
| ## Capabilities | ||
|
|
||
| | Capability | Detail | | ||
| | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | **Health tracking** | Verifies `ObservedGeneration` matches `Generation` before evaluating `ReadyReplicas`; reports `Healthy`, `Creating`, `Updating`, or `Scaling`; grace handler can mark Down/Degraded | | ||
| | **Rollout health** | Surfaces stalled or failing rollouts by transitioning the resource to `Degraded` or `Down` (no grace-period timing) | | ||
| | **Suspension** | Scales to zero replicas; reports `Suspending` / `Suspended` | | ||
| | **Mutation pipeline** | Typed editors for metadata, statefulset spec, pod spec, containers, and volume claim templates | | ||
|
Comment on lines
+9
to
+14
|
||
|
|
||
| ## Building a StatefulSet Primitive | ||
|
|
||
| ```go | ||
| import "github.com/sourcehawk/operator-component-framework/pkg/primitives/statefulset" | ||
|
|
||
| base := &appsv1.StatefulSet{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Name: "db", | ||
| Namespace: owner.Namespace, | ||
| }, | ||
| Spec: appsv1.StatefulSetSpec{ | ||
| ServiceName: "db-headless", | ||
| Selector: &metav1.LabelSelector{ | ||
| MatchLabels: map[string]string{"app": "db"}, | ||
| }, | ||
| Template: corev1.PodTemplateSpec{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Labels: map[string]string{"app": "db"}, | ||
| }, | ||
| Spec: corev1.PodSpec{ | ||
| Containers: []corev1.Container{ | ||
| {Name: "db", Image: "postgres:15"}, | ||
| }, | ||
| }, | ||
| }, | ||
| }, | ||
| } | ||
|
|
||
| resource, err := statefulset.NewBuilder(base). | ||
| WithMutation(MyFeatureMutation(owner.Spec.Version)). | ||
| Build() | ||
| ``` | ||
|
|
||
| ## Mutations | ||
|
|
||
| Mutations are the primary mechanism for modifying a `StatefulSet` beyond its baseline. Each mutation is a named function | ||
| that receives a `*Mutator` and records edit intent through typed editors. | ||
|
|
||
| The `Feature` field controls when a mutation applies. Leaving it nil applies the mutation unconditionally. A feature | ||
| with no version constraints and no `When()` conditions is also always enabled: | ||
|
|
||
| ```go | ||
| func MyFeatureMutation(version string) statefulset.Mutation { | ||
| return statefulset.Mutation{ | ||
| Name: "my-feature", | ||
| Feature: feature.NewResourceFeature(version, nil), // always enabled | ||
| Mutate: func(m *statefulset.Mutator) error { | ||
| // record edits here | ||
| return nil | ||
| }, | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Mutations are applied in the order they are registered with the builder. If one mutation depends on a change made by | ||
| another, register the dependency first. | ||
|
|
||
| ### Boolean-gated mutations | ||
|
|
||
| Use `When(bool)` to gate a mutation on a runtime condition: | ||
|
|
||
| ```go | ||
| func TracingMutation(version string, enabled bool) statefulset.Mutation { | ||
| return statefulset.Mutation{ | ||
| Name: "tracing", | ||
| Feature: feature.NewResourceFeature(version, nil).When(enabled), | ||
| Mutate: func(m *statefulset.Mutator) error { | ||
| m.EnsureInitContainer(corev1.Container{ | ||
| Name: "init-config", | ||
| Image: "config-init:latest", | ||
| }) | ||
| return nil | ||
| }, | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Version-gated mutations | ||
|
|
||
| Pass a `[]feature.VersionConstraint` to gate on a semver range: | ||
|
|
||
| ```go | ||
| var legacyConstraint = mustSemverConstraint("< 2.0.0") | ||
|
|
||
| func LegacyStorageMutation(version string) statefulset.Mutation { | ||
| return statefulset.Mutation{ | ||
| Name: "legacy-storage", | ||
| Feature: feature.NewResourceFeature( | ||
| version, | ||
| []feature.VersionConstraint{legacyConstraint}, | ||
| ), | ||
| Mutate: func(m *statefulset.Mutator) error { | ||
|
sourcehawk marked this conversation as resolved.
|
||
| m.EditContainers(selectors.ContainerNamed("db"), func(e *editors.ContainerEditor) error { | ||
| e.EnsureEnvVar(corev1.EnvVar{Name: "STORAGE_BACKEND", Value: "legacy"}) | ||
| return nil | ||
| }) | ||
| return nil | ||
| }, | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| All version constraints and `When()` conditions must be satisfied for a mutation to apply. | ||
|
|
||
| ## Internal Mutation Ordering | ||
|
|
||
| Within a single mutation, edit operations are grouped into categories and applied in a fixed sequence regardless of the | ||
| order they are recorded. This ensures structural consistency across mutations. | ||
|
|
||
| | Step | Category | What it affects | | ||
| | ---- | -------------------------------- | ----------------------------------------------------------------------- | | ||
| | 1 | StatefulSet metadata edits | Labels and annotations on the `StatefulSet` object | | ||
| | 2 | StatefulSetSpec edits | Replicas, service name, update strategy, etc. | | ||
| | 3 | Pod template metadata edits | Labels and annotations on the pod template | | ||
| | 4 | Pod spec edits | Volumes, tolerations, node selectors, service account, security context | | ||
| | 5 | Regular container presence | Adding or removing containers from `spec.template.spec.containers` | | ||
| | 6 | Regular container edits | Env vars, args, resources (snapshot taken after step 5) | | ||
| | 7 | Init container presence | Adding or removing containers from `spec.template.spec.initContainers` | | ||
| | 8 | Init container edits | Env vars, args, resources (snapshot taken after step 7) | | ||
| | 9 | Volume claim template operations | Adding or removing entries from `spec.volumeClaimTemplates` | | ||
|
|
||
| Container edits (steps 6 and 8) are evaluated against a snapshot taken _after_ presence operations in the same mutation. | ||
| This means a single mutation can add a container and then configure it without selector resolution issues. | ||
|
|
||
| ## Editors | ||
|
|
||
| ### StatefulSetSpecEditor | ||
|
|
||
| Controls statefulset-level settings via `m.EditStatefulSetSpec`. | ||
|
|
||
| Available methods: `SetReplicas`, `SetServiceName`, `SetPodManagementPolicy`, `SetUpdateStrategy`, | ||
| `SetRevisionHistoryLimit`, `SetMinReadySeconds`, `SetPersistentVolumeClaimRetentionPolicy`, `Raw`. | ||
|
|
||
| ```go | ||
| m.EditStatefulSetSpec(func(e *editors.StatefulSetSpecEditor) error { | ||
| e.SetReplicas(3) | ||
| e.SetServiceName("db-headless") | ||
| e.SetPodManagementPolicy(appsv1.ParallelPodManagement) | ||
| return nil | ||
| }) | ||
| ``` | ||
|
|
||
| For fields not covered by the typed API, use `Raw()`: | ||
|
|
||
| ```go | ||
| m.EditStatefulSetSpec(func(e *editors.StatefulSetSpecEditor) error { | ||
| e.Raw().UpdateStrategy = appsv1.StatefulSetUpdateStrategy{ | ||
| Type: appsv1.OnDeleteStatefulSetStrategyType, | ||
| } | ||
| return nil | ||
| }) | ||
| ``` | ||
|
|
||
| ### PodSpecEditor | ||
|
|
||
| Manages pod-level configuration via `m.EditPodSpec`. | ||
|
|
||
| Available methods: `SetServiceAccountName`, `EnsureVolume`, `RemoveVolume`, `EnsureToleration`, `RemoveTolerations`, | ||
| `EnsureNodeSelector`, `RemoveNodeSelector`, `EnsureImagePullSecret`, `RemoveImagePullSecret`, `SetPriorityClassName`, | ||
| `SetHostNetwork`, `SetHostPID`, `SetHostIPC`, `SetSecurityContext`, `Raw`. | ||
|
|
||
| ```go | ||
| m.EditPodSpec(func(e *editors.PodSpecEditor) error { | ||
| e.SetServiceAccountName("db-sa") | ||
| e.EnsureVolume(corev1.Volume{ | ||
| Name: "config", | ||
| VolumeSource: corev1.VolumeSource{ | ||
| ConfigMap: &corev1.ConfigMapVolumeSource{ | ||
| LocalObjectReference: corev1.LocalObjectReference{Name: "db-config"}, | ||
| }, | ||
| }, | ||
| }) | ||
| return nil | ||
| }) | ||
| ``` | ||
|
|
||
| ### ContainerEditor | ||
|
|
||
| Modifies individual containers via `m.EditContainers` or `m.EditInitContainers`. Always used in combination with a | ||
| [selector](../primitives.md#container-selectors). | ||
|
|
||
| Available methods: `EnsureEnvVar`, `EnsureEnvVars`, `RemoveEnvVar`, `RemoveEnvVars`, `EnsureArg`, `EnsureArgs`, | ||
| `RemoveArg`, `RemoveArgs`, `SetResourceLimit`, `SetResourceRequest`, `SetResources`, `Raw`. | ||
|
|
||
| ```go | ||
| m.EditContainers(selectors.ContainerNamed("db"), func(e *editors.ContainerEditor) error { | ||
| e.EnsureEnvVar(corev1.EnvVar{Name: "PGDATA", Value: "/var/lib/postgresql/data"}) | ||
| e.SetResourceLimit(corev1.ResourceMemory, resource.MustParse("2Gi")) | ||
| return nil | ||
| }) | ||
| ``` | ||
|
|
||
| ### ObjectMetaEditor | ||
|
|
||
| Modifies labels and annotations. Use `m.EditObjectMetadata` to target the `StatefulSet` object itself, or | ||
| `m.EditPodTemplateMetadata` to target the pod template. | ||
|
|
||
| Available methods: `EnsureLabel`, `RemoveLabel`, `EnsureAnnotation`, `RemoveAnnotation`, `Raw`. | ||
|
|
||
| ```go | ||
| // On the StatefulSet itself | ||
| m.EditObjectMetadata(func(e *editors.ObjectMetaEditor) error { | ||
| e.EnsureLabel("app.kubernetes.io/version", version) | ||
| return nil | ||
| }) | ||
|
|
||
| // On the pod template | ||
| m.EditPodTemplateMetadata(func(e *editors.ObjectMetaEditor) error { | ||
| e.EnsureAnnotation("prometheus.io/scrape", "true") | ||
| return nil | ||
| }) | ||
| ``` | ||
|
|
||
| ## Volume Claim Templates | ||
|
|
||
| The mutator provides `EnsureVolumeClaimTemplate` and `RemoveVolumeClaimTemplate` for managing persistent storage: | ||
|
|
||
| ```go | ||
| m.EnsureVolumeClaimTemplate(corev1.PersistentVolumeClaim{ | ||
| ObjectMeta: metav1.ObjectMeta{Name: "data"}, | ||
| Spec: corev1.PersistentVolumeClaimSpec{ | ||
| AccessModes: []corev1.PersistentVolumeAccessMode{corev1.ReadWriteOnce}, | ||
| Resources: corev1.VolumeResourceRequirements{ | ||
| Requests: corev1.ResourceList{ | ||
| corev1.ResourceStorage: resource.MustParse("10Gi"), | ||
| }, | ||
| }, | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| **Important:** `spec.volumeClaimTemplates` is immutable after creation in Kubernetes. These mutation methods are | ||
| primarily useful for constructing the initial desired state or when recreating a StatefulSet. | ||
|
|
||
| ## Convenience Methods | ||
|
|
||
| The `Mutator` also exposes convenience wrappers: | ||
|
|
||
| | Method | Equivalent to | | ||
| | ----------------------------- | ------------------------------------------------------------- | | ||
| | `EnsureReplicas(n)` | `EditStatefulSetSpec` → `SetReplicas(n)` | | ||
| | `EnsureContainerEnvVar(ev)` | `EditContainers(AllContainers(), ...)` → `EnsureEnvVar(ev)` | | ||
| | `RemoveContainerEnvVar(name)` | `EditContainers(AllContainers(), ...)` → `RemoveEnvVar(name)` | | ||
| | `EnsureContainerArg(arg)` | `EditContainers(AllContainers(), ...)` → `EnsureArg(arg)` | | ||
| | `RemoveContainerArg(arg)` | `EditContainers(AllContainers(), ...)` → `RemoveArg(arg)` | | ||
|
|
||
| ## Full Example: Database StatefulSet with Storage | ||
|
|
||
| ```go | ||
| func DatabaseMutation(version string) statefulset.Mutation { | ||
| return statefulset.Mutation{ | ||
| Name: "database-storage", | ||
| Feature: feature.NewResourceFeature(version, nil), | ||
| Mutate: func(m *statefulset.Mutator) error { | ||
| // Configure the StatefulSet spec | ||
| m.EditStatefulSetSpec(func(e *editors.StatefulSetSpecEditor) error { | ||
| e.SetReplicas(3) | ||
| e.SetPodManagementPolicy(appsv1.OrderedReadyPodManagement) | ||
| return nil | ||
| }) | ||
|
|
||
| // Add a volume claim template for persistent data | ||
| m.EnsureVolumeClaimTemplate(corev1.PersistentVolumeClaim{ | ||
| ObjectMeta: metav1.ObjectMeta{Name: "data"}, | ||
| Spec: corev1.PersistentVolumeClaimSpec{ | ||
| AccessModes: []corev1.PersistentVolumeAccessMode{corev1.ReadWriteOnce}, | ||
| Resources: corev1.VolumeResourceRequirements{ | ||
| Requests: corev1.ResourceList{ | ||
| corev1.ResourceStorage: resource.MustParse("50Gi"), | ||
| }, | ||
| }, | ||
| }, | ||
| }) | ||
|
|
||
| // Mount the volume in the database container | ||
| m.EditContainers(selectors.ContainerNamed("db"), func(e *editors.ContainerEditor) error { | ||
| e.Raw().VolumeMounts = append(e.Raw().VolumeMounts, corev1.VolumeMount{ | ||
| Name: "data", | ||
| MountPath: "/var/lib/postgresql/data", | ||
| }) | ||
| return nil | ||
| }) | ||
|
|
||
| return nil | ||
| }, | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Guidance | ||
|
|
||
| **`Feature: nil` applies unconditionally.** Omit `Feature` (leave it nil) for mutations that should always run. Use | ||
| `feature.NewResourceFeature(version, constraints)` when version-based gating is needed, and chain `.When(bool)` for | ||
| boolean conditions. | ||
|
|
||
| **Register mutations in dependency order.** If mutation B relies on a container added by mutation A, register A first. | ||
| The internal ordering within each mutation handles intra-mutation dependencies automatically. | ||
|
|
||
| **Prefer `EnsureContainer` over direct slice manipulation.** The mutator tracks presence operations so that selectors in | ||
| the same mutation resolve correctly and reconciliation remains idempotent. | ||
|
|
||
| **VolumeClaimTemplates are immutable.** Plan your storage layout before the first creation. | ||
|
|
||
| **Use selectors for precision.** Targeting `AllContainers()` when you only mean to modify the primary container can | ||
| cause unexpected behavior if sidecar containers are present. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # StatefulSet Primitive Example | ||
|
|
||
| This example demonstrates the usage of the `statefulset` primitive within the operator component framework. It shows how | ||
| to manage a Kubernetes StatefulSet as a component of a larger application, utilizing features like: | ||
|
|
||
| - **Base Construction**: Initializing a StatefulSet with basic metadata, spec, and volume claim templates. | ||
| - **Feature Mutations**: Applying version-gated or conditional changes (sidecars, env vars, annotations) using the | ||
| `Mutator`. | ||
| - **Custom Status Handlers**: Overriding the default logic for determining readiness (via `ConvergingStatus` and the | ||
| `WithCustomConvergeStatus` builder hook) and health assessment during rollouts (`GraceStatus`). | ||
| - **Custom Suspension**: Extending the default suspension logic (scaling to 0) with additional mutations. | ||
| - **Data Extraction**: Harvesting information from the reconciled resource. | ||
|
|
||
| ## Directory Structure | ||
|
|
||
| - `app/`: Defines the mock `ExampleApp` CRD and the controller that uses the component framework. | ||
| - `features/`: Contains modular feature definitions: | ||
| - `mutations.go`: sidecar injection, env vars, and version-based image updates. | ||
| - `handlers.go`: custom status and suspension handlers. | ||
| - `resources/`: Contains the central `NewStatefulSetResource` factory that assembles all features using the | ||
| `statefulset.Builder`. | ||
| - `main.go`: A standalone entry point that demonstrates a single reconciliation loop using a fake client. | ||
|
|
||
| ## Running the Example | ||
|
|
||
| You can run this example directly using `go run`: | ||
|
|
||
| ```bash | ||
| go run examples/statefulset-primitive/main.go | ||
| ``` | ||
|
|
||
|
Comment on lines
+24
to
+31
|
||
| This will: | ||
|
|
||
| 1. Initialize a fake Kubernetes client. | ||
| 2. Create an `ExampleApp` owner object. | ||
| 3. Reconcile the `ExampleApp` components. | ||
| 4. Print the resulting status conditions. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PR description/checklist states that 'Shared file modifications are limited to documentation updates', but this PR also changes shared/non-doc files (e.g.,
Makefileandpkg/mutation/editors/statefulsetspec.go). Either update the PR description/checklist item to reflect the actual scope, or move these non-doc changes into a separate PR if that restriction is important for review/release processes.