Skip to content

feat: bump kube-prometheus-stack CRDs to 86.1.0 and add the opentelemetry-operator CRDs - #93

Open
venkatamutyala wants to merge 5 commits into
mainfrom
feat/otel-20260902
Open

venkatamutyala wants to merge 5 commits into
mainfrom
feat/otel-20260902

Conversation

@venkatamutyala

Copy link
Copy Markdown
Collaborator

Replaces #68 and #92 (both auto-closed by GitHub when their head branches were renamed). This is the original otel branch reconciled with main (#83, 0.1.5, #90); the pre-sync branch is kept as old-otel-20260902.

Why

The new monitoring stack (GlueOps/k8s-monitoring-helm, wired in by the platform chart's glueops-core-monitoring app-of-apps — see the companion feat/otel-20260902 PRs) runs kube-prometheus-stack 86.1.0 and the opentelemetry-operator chart 0.114.1 (appVersion v0.152.0). Both used to install their own CRDs from inside Argo CD Applications; this bundle is now their only installer.

What

  • kube-prometheus-stack 59.1.0 → 86.1.0 — same 10 CRDs, still single-version with no conversion webhooks; no served version removed (hack/version-guard.sh passes). This is the pin that the chart in k8s-monitoring-helm uses, so the bundle and the operator match exactly.
  • opentelemetry-operator v0.152.0 — instrumentations, opampbridges, opentelemetrycollectors, targetallocators, sourced from the operator repo's config/crd/bases/ (the Helm chart's conf/crds/ copies are Go templates carrying the webhook caBundle, not YAML kustomize can read). The chart-only, feature-gated ClusterObservability CRD is excluded in hack/verify.sh.
  • OpenTelemetryCollector serves only v1beta1. Upstream serves v1alpha1 too and relies on the operator's conversion webhook to translate between them. This bundle deliberately cannot carry a conversion webhook (an orphan CRD with one is not inert — hack/verify.sh check 5), so kustomization.yaml patches v1alpha1 to served: false (a JSON-patch test op guards the index). With a single served version no conversion is ever invoked, strategy: None is correct and status.storedVersions stays [v1beta1]. Everything the platform deploys is v1beta1 (GlueOps/otel-resources-helm); Instrumentation is single-version upstream. Documented in the README.
    • Alternative if you'd rather not diverge from upstream: keep the OTel CRDs out of the bundle and let the operator chart ship them ("ship it with its operator", the README's sanctioned exception). That means flipping crds.create back to true in k8s-monitoring-helm and dropping the 4 sources + patch here.
  • hack/pins.map, hack/pins.sh, renovate.json learn the new upstream so it is pinned (glueops.dev/pin.opentelemetry-operator) and bumped like every other source. The renovate rule notes it must move together with the chart appVersion in k8s-monitoring-helm.

Verification (local)

  • hack/render.sh → 87 base + 2 kubeadm CRDs
  • hack/verify.sh → every source complete and pinned, no forbidden groups, no conversion webhooks, package round-trips
  • hack/version-guard.sh origin/main → no served versions removed
  • Server-side apply on kind v1.34.8: 89 CRDs Established; a v1beta1 OpenTelemetryCollector is accepted, v1alpha1 is refused by the API server

Review follow-ups (second commit)

  • Every bundle CRD now carries argocd.argoproj.io/sync-options: Prune=false,Delete=false (stamped by hack/render.sh, asserted by hack/verify.sh check 5d). On a cluster where an Argo CD Application used to install CRDs (the pre-#347 k8s-monitoring-helm umbrella did, for kube-prometheus-stack and OpenTelemetry), those CRDs stay tracked by that Application, and Argo CD would prune them once they leave its desired state or cascade-delete them through its resources-finalizer — garbage-collecting every object of those kinds. Stripping the tracking (which captain_utils → crds does) is a race against self-heal; the annotation is honoured on the live object regardless of who tracks it. Verified on the k3d e2e cluster: a throwaway CRD tracked to glueops-core-otel was pruned within 20 s without the annotation and reported ignored (no prune) with it.
  • hack/verify.sh's stale-webhook recipe now nulls the webhook stanza too ({"spec":{"conversion":{"strategy":"None","webhook":null}}}); the API server rejects strategy None while spec.conversion.webhook is still set, and the OpenTelemetryCollector CRD is the first bundle CRD whose previous installer set one.
  • README: new section on taking over Application-installed CRDs.
  • Re-verified: hack/render.sh, hack/verify.sh, hack/version-guard.sh origin/main, and the re-rendered bundle applied to the e2e cluster (87 changed, all Established).

Merge order

  1. This PR (bundle release, e.g. v0.2.0 — the kps bump is a minor per the 0.x policy)
  2. GlueOps/k8s-monitoring-helm feat/otel-20260902 (stops installing CRDs)
  3. GlueOps/platform-helm-chart-platform feat/otel-20260902 (switches to the new stack)
  4. GlueOps/terraform-module-cloud-multy-prerequisites feat/otel-20260902 (pins the released chart + bundle)

Clusters must run captain_utils → crds with the released bundle before the platform chart that carries the new stack.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L9V4AwNYgrznbZVpcA2cw8

venkatamutyala and others added 5 commits September 2, 2026 12:18
…etry-operator CRDs

The new monitoring stack (GlueOps/k8s-monitoring-helm, wired in by the platform
chart's glueops-core-monitoring app-of-apps) runs kube-prometheus-stack 86.1.0 and
the opentelemetry-operator chart 0.114.1 (appVersion v0.152.0). Both used to
install their own CRDs from inside Argo CD Applications; this bundle is now their
only installer, so:

- kube-prometheus-stack: 59.1.0 -> 86.1.0 (same 10 CRDs, still single-version, no
  conversion webhooks; no served version removed, version-guard passes).
- opentelemetry-operator v0.152.0: instrumentations, opampbridges,
  opentelemetrycollectors, targetallocators, sourced from the operator repo's
  config/crd/bases (the Helm chart's conf/crds copies are Go templates). The
  chart-only ClusterObservability CRD is excluded in hack/verify.sh.
- OpenTelemetryCollector serves only v1beta1: upstream serves v1alpha1 too and
  relies on the operator's conversion webhook, which this bundle cannot carry
  (verify.sh check 5). A kustomize JSON patch (with a `test` op guarding the
  index) flips v1alpha1 to served: false, so no conversion is ever invoked and
  strategy None is correct. Everything the platform deploys is v1beta1.
- pins.map / pins.sh / renovate.json learn the new upstream so it is pinned and
  bumped like every other source.

Verified locally: hack/render.sh, hack/verify.sh (89 CRDs, every source complete
and pinned), hack/version-guard.sh origin/main, server-side apply on kind
v1.34.8 (89 Established), a v1beta1 OpenTelemetryCollector is accepted and
v1alpha1 is refused.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L9V4AwNYgrznbZVpcA2cw8
…ument taking over Application-installed CRDs

Review finding: on a cluster where an Argo CD Application used to install CRDs
(the pre-#347 k8s-monitoring-helm umbrella rendered the kube-prometheus-stack
CRDs from a git directory source and the OpenTelemetry CRDs from the operator
chart), those CRDs stay tracked by that Application. When the Application stops
declaring them Argo CD prunes them, and when the Application is deleted its
resources-finalizer cascade-deletes them — either way every object of that kind
is garbage-collected. Stripping the tracking (captain_utils crds does it) is a
race against the old Application's self-heal.

- hack/render.sh stamps argocd.argoproj.io/sync-options=Prune=false,Delete=false
  on every CRD; Argo CD honours it on the live object, and the bundle's field
  manager owns it. hack/verify.sh asserts it (check 5d).
  Verified on the k3d e2e cluster: a throwaway CRD tracked to glueops-core-otel
  was pruned within 20s without the annotation and reported "ignored (no prune)"
  with it.
- hack/verify.sh: the stale-webhook recipe must also null the webhook stanza —
  the API server rejects strategy None while spec.conversion.webhook is set.
- README: "Taking over CRDs an Argo CD Application used to install".

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L9V4AwNYgrznbZVpcA2cw8
… document both takeover paths and the otel audit

- hack/render.sh renders each profile into a temp dir and swaps it in only on
  success. kustomize's ~27s git fetch timeout on the argo-cd source hit three
  times during review and used to leave crds/ deleted, which CI reports as drift.
- README: a CRD an earlier installer created with client-side kubectl apply
  (Argo CD's default) is migrated wholesale by kubectl's CSA->SSA upgrade on the
  bundle's first apply (foreign labels, annotations and a conversion webhook
  stanza are removed); one created any other way keeps them. The
  glueops.dev/pin.* annotations are informational — nothing compares them yet.
- MIGRATIONS.md: audit/clear recipe for opentelemetrycollectors.opentelemetry.io
  on clusters where the operator chart installed it (webhook stanza survives the
  takeover; a stored v1alpha1 object with a dead webhook breaks reads and blocks
  CRD deletion — migrate while the operator still runs).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L9V4AwNYgrznbZVpcA2cw8
Brings in #83 (Kubernetes 1.35 CRD source bumps), release 0.1.5 and #90 (auto-render on every PR).
kustomization.yaml: main's source versions plus this branch's kube-prometheus-stack 86.1.0 and
opentelemetry-operator v0.152.0 sources. Generated files (crds/, Chart.yaml) take main's side here;
CI re-renders them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant