Skip to content

feat(cert-manager): pin v1.21.2, real gateway-shim, safe fallback - #36

Merged
surrealwolf merged 1 commit into
mainfrom
feat/cert-manager-v1.21.2
Oct 5, 2026
Merged

surrealwolf merged 1 commit into
mainfrom
feat/cert-manager-v1.21.2

Conversation

@surrealwolf

@surrealwolf surrealwolf commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Bring cert-manager on the three apps clusters (poc/nprd/prd) back under Terraform at the latest 1.21 patch, and fix two latent problems in the cert_manager module.

Findings that drove this

  • Live state: all four clusters (manager + apps) already run cert-manager v1.21.1, installed by a manual helm upgrade on 2026-07-31 with only installCRDs=true. Terraform state still records v1.19.2, which is the "6 to replace" drift.
  • Owner: the Helm release cert-manager/cert-manager is owned by this repo's modules/cert_manager (apps clusters). gitops-core Fleet bundles (cert-manager/overlays/<cluster>) only ship the ClusterIssuer, default-cert ConfigMap, rke2-ingress-nginx HelmChartConfig and (manager) the cert-sync CronJob — not the chart, so there is no double management. No HelmChart CRs for cert-manager exist. The manager's cert-manager is installed by rancher_cluster/deploy-rancher.sh (no version trigger, so out of scope here).
  • Version: v1.21.2 (2026-09-11) is the latest patch. It fixes webhook/controller panics plus ACME/Vault response-body disclosure, and bumps Go and dependencies for CVEs. cert-manager 1.21 supports and tests Kubernetes 1.33–1.36, so it's fine on RKE2 v1.36.2. 1.21 breaking changes (tokenrequest RBAC, metrics Helm values, cert-manager-edit ACME perms) don't affect us: we set none of those values and nothing uses serviceAccountRef against the controller SA.
  • gateway-shim never ran: since cert-manager 1.15, --controllers=*,gateway-shim registers the shim, but its Gateway informers only start when gatewayAPI.enabled is true, so it waited forever. The manual 1.21.1 upgrade also dropped that flag. The prd high-command-gateway-https-dataknife-ai Certificate has no owner (it was created by hand) and keeps renewing normally either way. Gateway API CRDs (v1.4.1) are present on all three apps clusters, which gatewayAPI.enabled requires (otherwise the controller crash-loops). The shim refuses to modify Certificates it doesn't own.

Changes

  • cert_manager_version default/example: v1.19.2 → v1.21.2 (root var, cert_manager module, rancher_cluster module).
  • Helm values: crds.enabled=true crds.keep=true (replaces deprecated installCRDs; CRDs keep helm.sh/resource-policy: keep) and config.gatewayAPI.enabled=true.
  • Safety: if the Helm release isn't found, the module used to delete the cert-manager CRDs and namespace, which deletes every Certificate/Issuer. A transient helm list failure was enough to trigger it. It now exits with an error unless cleanup_unmanaged_install = true. Release detection uses helm status.

Rollout (after merge)

Targeted -refresh=false applies, one cluster at a time: poc-apps → nprd-apps → prd-apps, with a state backup first and the full plan reviewed (it must contain only module.cert_manager_<cluster>.null_resource.{deploy,verify}_cert_manager). Each cluster gets verified before moving on. nprd/prd are gated on the in-flight CNPG Barman plugin work being idle.

Test plan

  • helm template v1.21.2 with the new values renders the controller ConfigMap with gatewayAPI.enabled: true and 6 CRDs with resource-policy: keep
  • terraform fmt on the module
  • Targeted plans show only the intended resources
  • Per-cluster verification (pods on v1.21.2, cmctl check api, ClusterIssuer/Certificates Ready, self-signed test cert, TLS endpoints)

Note

Medium Risk
Changes cert-manager Helm values and version on apps clusters where all cluster TLS depends on Certificate/Issuer CRDs; the new fail-safe reduces accidental CRD deletion risk during apply.

Overview
Re-aligns Terraform-managed cert-manager with live clusters by bumping the default chart pin from v1.19.2 to v1.21.2 (root variable, rancher_cluster, example tfvars, CHANGELOG).

The cert_manager module updates Helm install/upgrade to use crds.enabled / crds.keep instead of deprecated installCRDs, and config.gatewayAPI.enabled=true instead of --controllers=*,gateway-shim so the gateway-shim actually runs on cert-manager 1.15+. Post-install checks read gatewayAPI.enabled from the controller ConfigMap.

Safety fix: when cert-manager resources exist but Helm does not show a cert-manager release, the module errors by default instead of deleting CRDs/namespace (which would wipe all Certificates/Issuers). Destructive cleanup requires new cleanup_unmanaged_install = true. Release detection uses helm status rather than grepping helm list.

Reviewed by Cursor Bugbot for commit d780413. Bugbot is set up for automated code reviews on this repo. Configure here.

- Default/example cert_manager_version v1.19.2 -> v1.21.2 (latest 1.21
  patch; k8s 1.33-1.36). Live clusters already run v1.21.1 from a manual
  Helm upgrade on 2026-07-31.
- Helm values: crds.enabled/crds.keep instead of deprecated installCRDs;
  config.gatewayAPI.enabled=true instead of --controllers=*,gateway-shim,
  which registers the shim but never starts its informers (>=1.15).
- Refuse to delete CRDs/namespace when the Helm release is not found
  unless cleanup_unmanaged_install=true; detect release via helm status.

Co-authored-by: Cursor <cursoragent@cursor.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Cursor Bugbot completed successfully and reported no findings that need human review, and no approval policy requires additional review.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

@surrealwolf
surrealwolf merged commit df69fcf into main Oct 5, 2026
5 checks passed
@surrealwolf
surrealwolf deleted the feat/cert-manager-v1.21.2 branch October 5, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant