Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 6 additions & 60 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A Kubernetes controller that schedules resource deletion and Pod template updates through time-based annotations. It supports namespaced and cluster-scoped resources that expose the required API operations.

Use it to schedule resource deletion or request application restarts without a separate CronJob or wrapper object. Pod replacement depends on the workload's update behavior. See [Restart mechanism](#restart-mechanism).
Use it to schedule resource deletion or request application restarts without a separate CronJob or wrapper object. Pod replacement depends on the workload's update behavior. See [Restart mechanism](docs/controller-behavior.md#restart-mechanism).

## Quick start

Expand Down Expand Up @@ -101,6 +101,7 @@ spec:
- The value must be an RFC3339 timestamp with explicit timezone offset (`Z` or `±hh:mm`).
- This can be applied directly to a `Namespace` to trigger its deletion. Kubernetes will handle the subsequent removal of all resources within that namespace.
- `lifecycle.cezary.dev/delete-after` - relative TTL (e.g., `5m`, `1h`, `3d`). The controller calculates a deletion time from the selected reference point, saves it as `lifecycle.cezary.dev/delete-at`, and removes `delete-after`.
- `lifecycle.cezary.dev/catch-up` - overrides the global policy for a missed deletion or restart. Accepts `never`, `always`, or a positive duration (e.g., `15m`). See [Missed actions](docs/controller-behavior.md#missed-actions).
- `lifecycle.cezary.dev/dry-run` - enables dry-run for a resource. Accepts standard boolean values such as `true`, `false`, `1`, and `0`. Invalid values create a warning Event and no action is taken.
- `lifecycle.cezary.dev/managed-by: "lifecycle-controller"` - added by the controller when it mutates a resource, such as converting `*-after` annotations to `*-at` annotations, maintaining restart schedule state, or triggering a restart.

Expand All @@ -115,70 +116,15 @@ spec:

### Relative duration format

`delete-after`, `restart-after`, and `restart-every` require exactly one positive integer and unit, without signs or spaces. Valid units are `s`, `m`, `h`, and `d`. Values such as `1d`, `25h`, and `90m` are valid.
`delete-after`, `restart-after`, `restart-every`, and duration values for `catch-up` require exactly one positive integer and unit, without signs or spaces. Valid units are `s`, `m`, `h`, and `d`. Values such as `1d`, `25h`, and `90m` are valid.

Zero, compound values (e.g., `1d2h`), arithmetic expressions (e.g., `1d-25h`), decimal values (e.g., `1.5h`), and values that exceed the Go duration limit are invalid. If the selected action has an invalid duration, the controller records an `InvalidAnnotation` warning Event and leaves the resource unchanged.
Zero, compound values (e.g., `1d2h`), arithmetic expressions (e.g., `1d-25h`), decimal values (e.g., `1.5h`), and values that exceed the Go duration limit are invalid. Invalid values record an `InvalidAnnotation` warning Event and leaves the resource unchanged.

## Controller behavior

### Relative timers
Actions can run after their scheduled time. Restarts update the Pod template. The workload's update strategy determines whether Pods are replaced. See [Controller behavior](docs/controller-behavior.md) for more information.

Relative timers (`delete-after`, `restart-after`) start from the selected reference point. By default, `applyTimestamp` means the time when the controller processes the annotation, not the time when a client applies the manifest. Cluster load or controller downtime can delay processing.

Re-applying a manifest that restores the `-after` annotation resets the derived `-at` deadline when the controller processes it. This is the default "keep-alive" behavior.

Use `lifecycle.cezary.dev/reference-point: "creationTimestamp"` to calculate the deadline from `metadata.creationTimestamp`. Re-applying `-after` does not reset an existing `-at` deadline with this reference point.

Use `delete-at` or `restart-at` for a fixed timestamp, or `restart-cron` for a recurring calendar schedule. Actions can run after their scheduled time. These annotations do not guarantee exact-time execution or completion.

### Precedence

If a resource mixes `restart-*` and `delete-*` action annotations, the controller records a warning Event and takes no action.

Within one action family, the controller processes a non-empty `delete-after` or `restart-after` before the corresponding `-at` annotation:

| Reference point | When both `-after` and `-at` are present |
| --- | --- |
| `applyTimestamp` (default) | Convert `-after` to a new `-at` value, replacing the existing deadline. |
| `creationTimestamp` | Keep the existing `-at` value and remove `-after`. |

After conversion, restart priority is `restart-at`, then `restart-cron`, then `restart-every`. For deletion, the controller acts on `delete-at`.

### Restart mechanism

Restarts are supported only for native Deployments, StatefulSets, and DaemonSets, while deletion remains available for all watched resources.

- Triggering a restart - the controller writes `lifecycle.cezary.dev/restartedAt: "<timestamp>"` into the resource's `spec.template.metadata.annotations`. The workload controller determines how to apply the changed template.
- The same pod template mutation also adds `lifecycle.cezary.dev/managed-by: "lifecycle-controller"` to `spec.template.metadata.annotations`.
- State tracking for recurring restarts - for `restart-every` and `restart-cron` schedules, the controller uses a top-level `lifecycle.cezary.dev/last-restart-timestamp: "<timestamp>"` annotation as the anchor for calculating the next restart.
- Initialization - if `last-restart-timestamp` is missing, the controller sets it to the current time.
- Reconciliation - the controller calculates the next occurrence from the schedule and timestamp. When due, it updates the Pod template and timestamp, then schedules the next occurrence. Missed occurrences produce only one template update.
- Cleanup - the restart patch also removes the one-time `restart-at` annotation or updates recurring schedule state. This acknowledges the template update, not rollout completion.

### Dry-run

Dry-run logs planned actions without changing resources. Enable it globally with the `--dry-run` flag or the `controllerManager.dryRun` Helm value, or per resource with the `lifecycle.cezary.dev/dry-run` annotation.

In dry-run, relative and recurring schedules are logged without saving state or immediately requeueing the same occurrence.

## Scope and permissions

### Configuration flags

By default, the controller discovers resources across all namespaces and watches those that support `get`, `list`, `watch`, `patch`, and `delete`. It skips subresources and APIs that lack these verbs. An explicit watch pattern that selects an unsupported API causes a discovery error unless an ignore rule excludes that API.

Resource filters select which API types to watch. The controller reads, deletes, and restarts resources only in the filtered namespaces. Namespace patterns limit deletions and restarts but require cluster-wide reads.

- `--watch-resource` (repeatable) - glob pattern for resources to watch.
- Format - `<resource>.<group>` for grouped APIs (e.g., `deployments.apps`) or `<resource>` for core APIs (e.g., `pods`).
- If not provided, all eligible resources are watched unless excluded by ignore rules. Literal `*.*` is not equivalent to this default: it does not match core keys such as `pods`.
- Broad patterns can select APIs that lack the required verbs. Prefer exact resource names.
- `--ignore-resource` (repeatable) - glob pattern for resources to strictly ignore. Takes precedence over watch rules.
- `--watch-namespace` (repeatable) - glob pattern for namespaces to watch (e.g. `default`, `dev-*`).
- If provided, the controller deletes or restarts only objects in matching namespaces. Namespace objects also require `--watch-resource=namespaces`. Other cluster-scoped resources are excluded.
- `--ignore-namespace` (repeatable) - glob pattern for namespaces to strictly ignore. Takes precedence over watch rules.

### RBAC
## RBAC

By default, the Helm chart watches standard workloads, ConfigMaps, Services, Ingresses, NetworkPolicies, PersistentVolumeClaims, HorizontalPodAutoscalers, PodDisruptionBudgets, Namespaces, and PersistentVolumes. It grants `get`, `list`, `watch`, `patch`, `update`, and `delete` for those resources, plus Event permissions and permissions for enabled leader-election and secure-metrics features.

Expand Down
13 changes: 13 additions & 0 deletions charts/lifecycle-controller/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,19 @@ spec:
- --enable-http2
{{- end }}
- --dry-run={{ .Values.controllerManager.dryRun }}
{{- $timing := .Values.controllerManager.actionTiming | default dict }}
{{- $restartTiming := $timing.restart | default dict }}
{{- $deleteTiming := $timing.delete | default dict }}
{{- $restartCatchUp := "always" }}
{{- $deleteCatchUp := "always" }}
{{- if hasKey $restartTiming "catchUp" }}
{{- $restartCatchUp = $restartTiming.catchUp }}
{{- end }}
{{- if hasKey $deleteTiming "catchUp" }}
{{- $deleteCatchUp = $deleteTiming.catchUp }}
{{- end }}
- {{ printf "--restart-catch-up=%v" $restartCatchUp | quote }}
- {{ printf "--delete-catch-up=%v" $deleteCatchUp | quote }}
- --zap-log-level={{ .Values.controllerManager.logLevel }}
{{- range (include "lifecycle-controller.watchResources" . | fromJsonArray) }}
- --watch-resource={{ . }}
Expand Down
80 changes: 80 additions & 0 deletions charts/lifecycle-controller/tests/action_timing_test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
suite: Action timing configuration
templates:
- templates/deployment.yaml

tests:
- it: preserves catch-up by default for both actions
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: --restart-catch-up=always
- contains:
path: spec.template.spec.containers[0].args
content: --delete-catch-up=always

- it: configures restart and deletion independently
set:
controllerManager.actionTiming.restart.catchUp: never
controllerManager.actionTiming.delete.catchUp: 15m
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: --restart-catch-up=never
- contains:
path: spec.template.spec.containers[0].args
content: --delete-catch-up=15m

- it: accepts a day duration and an explicit always policy
set:
controllerManager.actionTiming.restart.catchUp: 1d
controllerManager.actionTiming.delete.catchUp: always
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: --restart-catch-up=1d
- contains:
path: spec.template.spec.containers[0].args
content: --delete-catch-up=always

- it: defaults missing action timing to always for preserved release values
set:
controllerManager.actionTiming: null
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: --restart-catch-up=always
- contains:
path: spec.template.spec.containers[0].args
content: --delete-catch-up=always

- it: defaults missing action maps to always
set:
controllerManager.actionTiming.restart: null
controllerManager.actionTiming.delete: null
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: --restart-catch-up=always
- contains:
path: spec.template.spec.containers[0].args
content: --delete-catch-up=always

- it: defaults a missing policy to always
set:
controllerManager.actionTiming.restart.catchUp: null
controllerManager.actionTiming.delete.catchUp: null
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: --restart-catch-up=always
- contains:
path: spec.template.spec.containers[0].args
content: --delete-catch-up=always

- it: passes an empty policy to startup validation
set:
controllerManager.actionTiming.restart.catchUp: ""
asserts:
- contains:
path: spec.template.spec.containers[0].args
content: --restart-catch-up=
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ tests:
- --metrics-bind-address=:8443
- --metrics-secure=true
- --dry-run=false
- --restart-catch-up=always
- --delete-catch-up=always
- --zap-log-level=info
- --watch-resource=pods
- --watch-resource=deployments.apps
Expand Down
8 changes: 8 additions & 0 deletions charts/lifecycle-controller/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,14 @@ controllerManager:
# Enable global dry-run mode (logs actions without executing them)
dryRun: false

# Default policy for missed actions: always. Also accepts never or a positive duration.
# The lifecycle.cezary.dev/catch-up annotation overrides these defaults.
actionTiming:
restart:
catchUp: "always"
delete:
catchUp: "always"

# Log level for the controller (debug, info, error)
logLevel: info

Expand Down
17 changes: 17 additions & 0 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ func main() {
var watchResources, ignoreResources arrayFlags
var watchNamespaces, ignoreNamespaces arrayFlags
var globalDryRun bool
var deleteCatchUp, restartCatchUp string

flag.StringVar(&metricsAddr, "metrics-bind-address", "0", "The address the metrics endpoint binds to. "+
"Use :8443 for HTTPS or :8080 for HTTP, or leave as 0 to disable the metrics service.")
Expand Down Expand Up @@ -91,13 +92,27 @@ func main() {
flag.Var(&ignoreNamespaces, "ignore-namespace", "Glob pattern for namespaces to ignore. Can be repeated.")
flag.BoolVar(&globalDryRun, "dry-run", false,
"Enable dry-run mode for all resources. When set, actions are logged but not executed.")
flag.StringVar(&deleteCatchUp, "delete-catch-up", "always",
"Default deletion catch-up policy: always, never, or a positive duration (e.g. 15m).")
flag.StringVar(&restartCatchUp, "restart-catch-up", "always",
"Default restart catch-up policy: always, never, or a positive duration (e.g. 15m).")

opts := zap.Options{}

opts.BindFlags(flag.CommandLine)
flag.Parse()

ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts)))
deletePolicy, err := controller.ParseCatchUpPolicy(deleteCatchUp)
if err != nil {
setupLog.Error(err, "invalid --delete-catch-up")
os.Exit(1)
}
restartPolicy, err := controller.ParseCatchUpPolicy(restartCatchUp)
if err != nil {
setupLog.Error(err, "invalid --restart-catch-up")
os.Exit(1)
}

// if the enable-http2 flag is false (the default), http/2 should be disabled
// due to its vulnerabilities. More specifically, disabling http/2 will
Expand Down Expand Up @@ -210,12 +225,14 @@ func main() {
"ignoreNamespaces", ignoreNamespaces,
)
setupLog.Info("Global dry-run mode", "enabled", globalDryRun)
setupLog.Info("Action catch-up policies", "delete", deleteCatchUp, "restart", restartCatchUp)

reconciler := &controller.LifecycleReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
Config: scopeConfig,
GlobalDryRun: globalDryRun,
ActionTiming: controller.ActionTimingConfig{Delete: deletePolicy, Restart: restartPolicy},
}
if err := reconciler.SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "Lifecycle")
Expand Down
Loading
Loading