feat(instance): attach to an externally-managed LiteLLM deployment

[davidgibbons]

Closes #29.

Adds spec.workload.managed: false so a LiteLLMInstance can describe a LiteLLM proxy the operator did not deploy, making the entity CRDs usable against an existing installation without handing the operator the workload.

This implements the issue's proposal, plus the endpoint half of "worth considering alongside". readinessRef is deliberately not implemented — see below.

What changes

spec.workload.managed (*bool, defaults true). When false, reconcileResources and reconcileAutoRollback are skipped entirely. Nothing is created, nothing existing is adopted or mutated, and no Forbidden is logged on every loop — the instance reports its actual state instead of sitting Degraded about resources it doesn't manage.

spec.workload.endpoint (optional, unmanaged only, CEL-enforced). Without it the endpoint stays today's formula, which requires the CR to be named after a Service it doesn't own. With it you can attach to a Service under another name, in another namespace, or to a proxy outside the cluster.

Readiness for an unmanaged instance comes from the admin API answering at status.endpoint (CheckLiveness, which probeInstanceHealth already performs) rather than a name-matched Deployment. This is what makes endpoint coherent: with an explicit endpoint there may be no Deployment of that name to look at, and the proxy may be a StatefulSet or off-cluster. It also closes a hole in the RBAC workaround — a Deployment that merely shares the CR's name can no longer make an unreachable proxy report ready: true.

updateInstanceStatus computes the endpoint before deriving readiness, since readiness now depends on it.

apiVersion: litellm.palena.ai/v1alpha1
kind: LiteLLMInstance
metadata:
  name: my-gateway
spec:
  workload:
    managed: false
    endpoint: http://litellm.platform.svc:4000   # optional
  masterKey:
    secretRef:
      name: litellm-master-key
      key: LITELLM_MASTER_KEY
  database: {}

Status differences when unmanaged

	Managed (default)	Unmanaged
Workload + optional resources	Created and reconciled	Never touched
Auto-rollback	Active	Skipped
Database migration Job	Created when enabled	Never — the proxy owns its schema
status.ready	A Deployment replica is ready	Admin API answers at status.endpoint
status.replicas / readyReplicas	Deployment counts	0
status.version	spec.image.tag	Empty, unless the proxy discloses litellm_version
PodsHealthy condition	Set	Absent — the operator owns no pods
Ready reason	AllResourcesReady / DeploymentNotReady	ProxyReachable / ProxyNotReachable
Health probing, config sync, entity CRDs, finalizer cleanup	Active	Active

status.version is worth calling out. spec.image.tag describes nothing the operator deployed, so leaving it in place would print a fabricated latest in the VERSION print column. probeInstanceHealth already fetches /health/readiness and discarded the payload, so it now reads litellm_version off it — but that field is only present when the proxy's own general_settings sets allow_public_health_readiness_details: true. The endpoint is unauthenticated, so the master key does not unlock it, and the default payload is {"status", "db"}. In the common case status.version is therefore empty for an unmanaged instance. That is deliberate: empty means "the operator does not know", which is true, where latest would be a guess. Both payload shapes are covered by tests. Say the word if you would rather it fell back to the image tag.

Decisions worth a look

Managed is *bool, not bool. An envtest spec caught this: with a plain bool and no omitempty, a Go-typed client constructing WorkloadSpec{Endpoint: ...} marshals "managed": false explicitly, so the API-server default never fires and the caller silently gets an unmanaged instance. Nil now means managed, matching enableServiceLinks and friends upstream. YAML users see no difference — workload: {} still defaults to true.

No readinessRef. Probing the endpoint covers Deployment, StatefulSet and off-cluster uniformly, so kind-switching would add surface without adding capability. Happy to add it if you'd rather have the Kubernetes-object signal.

spec.database.migration is gated too, and ignored entirely when unmanaged. An externally-managed proxy owns its own schema — LiteLLM migrates on startup, and whatever deployed it ships its own migration hook, so the operator would be a second migrator racing the real one. Mechanically it is worse than that: BuildMigrationJob takes its image from spec.image.tag (falling back to latest), which for an unmanaged instance describes nothing the operator deployed. Leaving it open meant running prisma migrate deploy at an arbitrary schema version against a database the operator does not own. DatabaseReady reports WorkloadUnmanaged, and the message says the migration was ignored rather than skipping it silently when one was configured anyway.

Tests

internal/controller/litellminstance_unmanaged_test.go — no resources created; a name-colliding Helm-owned Deployment left byte-identical, un-mutated and un-adopted; readiness tracking the probe in both directions; a name-matched Deployment not faking readiness; version read from the proxy; workloadManaged and instanceEndpoint tables.

internal/controller/litellminstance_controller_test.go — four envtest specs for the CEL rule and the managed default, since only envtest runs API-server validation.

What was verified against upstream

/health/liveliness and /health/readiness are the paths the operator's shipped client already uses for every managed instance, so the readiness signal reuses a proven code path rather than a new one. /health/liveliness returns the bare string I'm alive!; CheckLiveness passes a nil result and so never unmarshals it, which is why it is safe as a reachability probe. The litellm_version gating described above was checked against litellm/proxy/health_endpoints/_health_endpoints.py on main, not assumed — the field had been declared in this repo's ReadinessResponse since the initial commit but never read by anything until now.

spec.workload.endpoint is trimmed of trailing slashes by the client (strings.TrimRight(endpoint, "/")), so http://host:4000/ and http://host:4000 behave identically. The CRD pattern is ^https?://[^\s/?#]+, which requires a host — the value becomes an outbound request URL, so a scheme alone should not pass.

Housekeeping

make manifests generate sync-helm-crds and make bundle are committed. make test, make lint, helm lint and helm template pass locally. Commit is signed off, README and docs/reference/litellminstance.md document the field, and there's a sample at config/samples/litellm_v1alpha1_litellminstance_unmanaged.yaml.

A small drive-by: the master-key-with-autogenerate-fallback block was copy-pasted in resolveInstance and probeInstanceHealth; it's now one masterKeyRef helper that the new readiness probe also uses.