diff --git a/CHANGELOG.md b/CHANGELOG.md
index 11ed2d9..14766ce 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -21,8 +21,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - [docs/UPGRADE_PLAN.md](docs/UPGRADE_PLAN.md): prioritized next upgrade wave (reviewed 2026-10-04: live vs latest for Rancher, RKE2, Envoy Gateway, CNPG, OS, monitoring, apps, Terraform providers) and the cert-manager v1.21.2 rollout log; the completed 2.13→2.15 / 1.34→1.36 runbook is kept as history
 - **cert-manager** pin `v1.19.2` → **`v1.21.2`** (latest 1.21 patch; supports Kubernetes 1.33–1.36). Live clusters were already on `v1.21.1` via manual Helm (2026-07-31); Terraform now owns the version again
 - cert-manager module: Helm values `crds.enabled=true,crds.keep=true` (replaces deprecated `installCRDs`) and `config.gatewayAPI.enabled=true` for the gateway-shim (replaces `--controllers=*,gateway-shim`, which has not started the shim since cert-manager 1.15)
+- **Rancher** pin `v2.15.0` → **`v2.15.2`** (live since 2026-10-04)
+- **Envoy Gateway** `v1.6.1` → **`v1.9.2`** and **Gateway API** CRDs `v1.4.1` → **`v1.6.1`**. The module applies the CRDs server-side and waits for them before the controller, re-runs when the script changes, and verifies the controller by its real label ([#40](https://github.com/DataKnifeAI/rancher-deploy/pull/40), [#42](https://github.com/DataKnifeAI/rancher-deploy/pull/42))
+- **CloudNativePG** default `1.28.0` → **`1.30.1`** ([#41](https://github.com/DataKnifeAI/rancher-deploy/pull/41))
+- [docs/UPGRADE_PLAN.md](docs/UPGRADE_PLAN.md): execution log for items #1–#6 of the 2026-10-04 wave (Rancher, democratic-csi, Envoy Gateway, CNPG, Grafana, rolling OS reboots of all 31 nodes); [docs/RKE2_CERT_ROTATION.md](docs/RKE2_CERT_ROTATION.md): renewal results; [docs/OPS_NOTES.md](docs/OPS_NOTES.md): new known issues
 
 ### Fixed
+- CloudNativePG resources no longer have destroy provisioners: a version bump replaced the resource and deleted the operator manifest, CRDs included, and with them every Cluster/Backup ([#41](https://github.com/DataKnifeAI/rancher-deploy/pull/41))
+- Envoy Gateway module no longer deletes Gateway API CRDs on a version mismatch ([#40](https://github.com/DataKnifeAI/rancher-deploy/pull/40))
 - cert-manager module no longer deletes the cert-manager CRDs/namespace (and with them every Certificate/Issuer) when it cannot see the Helm release; it fails unless `cleanup_unmanaged_install = true`. Release detection uses `helm status` instead of grepping `helm list`
 - Broken doc links (`TERRAFORM_VARIABLES.md`) and an unterminated code fence in `RANCHER_DOWNSTREAM_MANAGEMENT.md`
 - Docs no longer claim `~/.kube/<cluster>.yaml` are RKE2 admin kubeconfigs (they were overwritten by Rancher-proxied ones); break-glass is `~/.kube/<cluster>-rke2.yaml`
@@ -34,7 +40,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Known issues
 - Terraform Rancher API token expired; registration modules read a hard-coded `/home/lee/git/rancher-deploy/config/.rancher-api-token`
-- RKE2 leaf certs on all control-plane nodes expire 2027-01-08 (poc-apps 2027-01-15) — rotate per [docs/RKE2_CERT_ROTATION.md](docs/RKE2_CERT_ROTATION.md)
+- RKE2 leaf certs were renewed on 2026-10-04 and now expire 2027-10-05 (poc-apps-1 2027-09-19); renew again from 2027-06-07 per [docs/RKE2_CERT_ROTATION.md](docs/RKE2_CERT_ROTATION.md)
+- Do **not** revoke the TrueNAS API key that was in `democratic_csi_api_key`: it is the key TrueNAS CSI uses ([docs/TRUENAS_CSI_MIGRATION.md](docs/TRUENAS_CSI_MIGRATION.md#truenas-manual))
 - TrueNAS CSI restarts; two prd CNPG replicas need re-cloning — see [docs/OPS_NOTES.md](docs/OPS_NOTES.md)
 
 ## [1.2.0-beta.1] - 2026-08-31
diff --git a/docs/CLUSTER_ACCESS_AND_SSO.md b/docs/CLUSTER_ACCESS_AND_SSO.md
index a3fb412..59dc3e2 100644
--- a/docs/CLUSTER_ACCESS_AND_SSO.md
+++ b/docs/CLUSTER_ACCESS_AND_SSO.md
@@ -2,7 +2,7 @@
 
 How people and tools log in to Rancher and the RKE2 clusters, and what to do when that breaks.
 
-**Last verified:** 2026-10-04 — Rancher `v2.15.0`, RKE2 `v1.36.2+rke2r1` on all clusters, Rancher CLI `v2.15.2`, Authentik `2026.8.3`.
+**Last verified:** 2026-10-04 — Rancher `v2.15.2`, RKE2 `v1.36.2+rke2r1` on all clusters, Rancher CLI `v2.15.2`, Authentik `2026.8.3`.
 
 Kubeconfigs no longer embed long-lived Rancher tokens. `kubectl` calls the Rancher CLI (`rancher token`) as an exec credential plugin, the CLI logs you in through Authentik (SAML) and caches a token. Break-glass paths do not depend on Authentik.
 
@@ -250,10 +250,10 @@ Use the first layer that works:
 
 | File | Cluster / user / context | Server | Client cert expires |
 |------|--------------------------|--------|---------------------|
-| `~/.kube/rancher-manager-rke2.yaml` | `rancher-manager-rke2` | `https://manager.dataknife.net:6443` | 2027-01-08 |
-| `~/.kube/nprd-apps-rke2.yaml` | `nprd-apps-rke2` | `https://nprd-apps.dataknife.net:6443` | 2027-01-08 |
-| `~/.kube/prd-apps-rke2.yaml` | `prd-apps-rke2` | `https://prd-apps.dataknife.net:6443` | 2027-01-08 |
-| `~/.kube/poc-apps-rke2.yaml` | `poc-apps-rke2` | `https://poc-apps.dataknife.net:6443` | 2027-09-19 (pulled from poc-apps-1, already renewed) |
+| `~/.kube/rancher-manager-rke2.yaml` | `rancher-manager-rke2` | `https://manager.dataknife.net:6443` | 2027-10-05 (re-fetched 2026-10-04 from rancher-manager-2) |
+| `~/.kube/nprd-apps-rke2.yaml` | `nprd-apps-rke2` | `https://nprd-apps.dataknife.net:6443` | 2027-10-05 (re-fetched 2026-10-04 from nprd-apps-2) |
+| `~/.kube/prd-apps-rke2.yaml` | `prd-apps-rke2` | `https://prd-apps.dataknife.net:6443` | 2027-10-05 (re-fetched 2026-10-04 from prd-apps-1) |
+| `~/.kube/poc-apps-rke2.yaml` | `poc-apps-rke2` | `https://poc-apps.dataknife.net:6443` | 2027-10-05 (re-fetched 2026-10-04 from poc-apps-2) |
 
 The `<cluster>.dataknife.net` names are round-robin DNS over the three control-plane nodes and are in every node's apiserver cert SANs (`tls-san`), so the files survive the loss of one server. There is no control-plane VIP: kube-vip runs with `cp_enable=false` (Service VIPs only). If DNS is down, point `server` at a control-plane IP (each node's own IP is in its SANs) — that is a single-node dependency.
 
@@ -329,8 +329,8 @@ The script logs in through the local provider, so it keeps working with SAML ena
 | `rancher.dataknife.net` (`cattle-system/tls-rancher-ingress`, cert-manager, LE) | 2026-11-15 | Auto-renews (~2026-10-16) | Rancher UI/API TLS errors; CLI login fails |
 | Authentik SAML signing cert (`authentik Self-signed Certificate`) | **2027-10-04** | Manual: create a new cert in Authentik, set it on the `Rancher` provider, re-download metadata, paste it into Rancher's SAML config (re-supply the SP key) and re-enable | **All SSO logins fail** |
 | Rancher SP cert (`~/.config/rancher-saml/sp.crt`) | 2036-10-01 | Regenerate (above) and re-upload | SSO logins fail |
-| RKE2 leaf certs (apiserver `:6443` serving, `client-admin`, `client-kube-apiserver`, `auth-proxy`, etcd, controller-manager, scheduler, supervisor) on control-plane nodes | **2027-01-08** (manager, nprd, prd); **2027-01-15** (poc-apps-2/-3; poc-apps-1 already renewed to 2027-09-19) | Not automatic unless rke2-server restarts within 120 days of expiry. Planned rolling renewal: [RKE2_CERT_ROTATION.md](RKE2_CERT_ROTATION.md) | Apiservers stop serving / components can't authenticate — cluster down |
-| RKE2 admin client certs (`system:admin`) in `~/.kube/<cluster>-rke2.yaml` and the gitops-core `cert-sync-kubeconfig` Secret | 2027-01-08 (poc: 2027-09-19) | Re-fetch `rke2.yaml` after each RKE2 rotation ([fetch](#fallback-and-break-glass)) and re-run `gitops-core/scripts/create-cert-sync-kubeconfig-secret.sh` | Break-glass access fails; the `cert-sync` Job fails (it now exits non-zero) and downstream wildcard copies go stale |
+| RKE2 leaf certs (apiserver `:6443` serving, `client-admin`, `client-kube-apiserver`, `auth-proxy`, etcd, controller-manager, scheduler, supervisor) on control-plane nodes; agent certs on all nodes | **2027-10-05** (renewed 2026-10-04 by the rolling reboots); poc-apps-1 server certs **2027-09-19** | Renewed automatically only when rke2 restarts within 120 days of expiry (from 2027-05-22 on poc-apps-1, 2027-06-07 elsewhere): plan a rolling reboot/RKE2 upgrade by 2027-09-01, or `rke2 certificate rotate` ([RKE2_CERT_ROTATION.md](RKE2_CERT_ROTATION.md)) | Apiservers stop serving / components can't authenticate — cluster down |
+| RKE2 admin client certs (`system:admin`) in `~/.kube/<cluster>-rke2.yaml` and the gitops-core `cert-sync-kubeconfig` Secret | 2027-10-05 (all four, re-fetched 2026-10-04) | Re-fetch `rke2.yaml` after each RKE2 rotation ([fetch](#fallback-and-break-glass)) and re-run `gitops-core/scripts/create-cert-sync-kubeconfig-secret.sh` | Break-glass access fails; the `cert-sync` Job fails (it now exits non-zero) and downstream wildcard copies go stale |
 | Rancher API / kubeconfig tokens | ≤ 90 days from issue | Re-login (humans) / recreate (automation, `config/.rancher-api-token`) | 401s; exec plugin asks for login |
 
 Spot checks:
diff --git a/docs/OPS_NOTES.md b/docs/OPS_NOTES.md
index 19ec987..c6e55d3 100644
--- a/docs/OPS_NOTES.md
+++ b/docs/OPS_NOTES.md
@@ -4,10 +4,10 @@ Short operational truths that do not belong in the README.
 
 ## Cluster access (SSO)
 
-- Rancher `v2.15.0` at `https://rancher.dataknife.net`; auth providers **local** + **Keycloak (SAML)** pointed at Authentik (`https://auth.dataknife.net`, on prd-apps). Full guide: [CLUSTER_ACCESS_AND_SSO.md](CLUSTER_ACCESS_AND_SSO.md).
+- Rancher `v2.15.2` at `https://rancher.dataknife.net`; auth providers **local** + **Keycloak (SAML)** pointed at Authentik (`https://auth.dataknife.net`, on prd-apps). Full guide: [CLUSTER_ACCESS_AND_SSO.md](CLUSTER_ACCESS_AND_SSO.md).
 - kubectl: `./scripts/setup-rancher-kubeconfig.sh --install-cli --merge --auth-provider keyCloakProvider --break-glass`, then `kubectl --context <local|nprd-apps|prd-apps|poc-apps>`. Kubeconfigs embed no tokens; `rancher token` caches one (≤ 90 days) in `~/.rancher/cli2.json`.
 - SAML site access is **`required`** by design: only principals in `allowedPrincipalIds` may log in via SSO (currently `keycloak_user://4` = akadmin). Allow people via an Authentik group, `keycloak_group://<exact group name>`; single users are `keycloak_user://<Authentik numeric pk>`, not the username. Local logins are unaffected. See [CLUSTER_ACCESS_AND_SSO.md § Site access](CLUSTER_ACCESS_AND_SSO.md#site-access-required).
-- Break-glass: `<cluster>-local` contexts (Rancher local user), UI **Log in with Local User**, RKE2 admin kubeconfigs `~/.kube/<cluster>-rke2.yaml` (independent of Rancher/Authentik; client certs expire 2027-01-08, poc-apps 2027-09-19 — re-fetch after RKE2 cert rotation per [RKE2_CERT_ROTATION.md](RKE2_CERT_ROTATION.md)), Authentik recovery key.
+- Break-glass: `<cluster>-local` contexts (Rancher local user), UI **Log in with Local User**, RKE2 admin kubeconfigs `~/.kube/<cluster>-rke2.yaml` (independent of Rancher/Authentik; client certs expire 2027-10-05 after the 2026-10-04 renewal — re-fetch after every RKE2 cert renewal per [RKE2_CERT_ROTATION.md](RKE2_CERT_ROTATION.md)), Authentik recovery key.
 - Global settings: `kubeconfig-generate-token=false`, `kubeconfig-default-token-ttl-minutes=129600`, `auth-token-max-ttl-minutes=129600` (default) — so **every** Rancher API token, including Terraform's `ttl: 0` one, expires in 90 days.
 - Unattended jobs: use a dedicated scoped API token or RKE2 admin kubeconfigs, never a personal SSO token cache.
 
@@ -20,16 +20,17 @@ Short operational truths that do not belong in the README.
 
 - **Official TrueNAS CSI** (`truenas_csi_*`, driver `csi.truenas.io`, class `truenas-csi-nfs`) is the only CSI driver on the app clusters. See [TRUENAS_CSI_MULTI_NODE.md](TRUENAS_CSI_MULTI_NODE.md).
 - **democratic-csi removed (2026-10-04).** No driver, StorageClass, PV, Helm release or namespace left on any cluster; the last orphaned `Released` PV on nprd (`pvc-25d4289c…`, old Loki ingester) was deleted (its dataset `/mnt/SAS/RKE2/pvc-25d4289c-38a0-4c20-96ef-35d2cd91356d` is still on TrueNAS). The `democratic_csi_*` variables, resources, output and scripts are gone from this repo. Migration history: [TRUENAS_CSI_MIGRATION.md](TRUENAS_CSI_MIGRATION.md).
+- **Don't revoke the TrueNAS API key that was in `democratic_csi_api_key`.** It is the same key as `truenas_csi_api_key` (TrueNAS key ID 4), live in `truenas-csi/truenas-api-credentials` on nprd, prd and poc; revoking it breaks TrueNAS CSI everywhere. Move TrueNAS CSI to a new key first if you want it gone. The older democratic-csi keys (IDs 1 and 2) are unused: revoke them.
 - Apps-cluster nodes get label `topology.truenas.io/pool=<truenas_csi_pool>` via RKE2 `node-label` at bootstrap, plus a post-kubeconfig `null_resource` that labels all nodes. Manager nodes are not labeled (CSI is apps-side).
 
 ## Version pins (RKE2 / Rancher / OS)
 
-Live clusters match these defaults (verified 2026-10-04: Rancher `v2.15.0`, all four clusters RKE2 `v1.36.2+rke2r1`, cert-manager `v1.21.1` after the 2026-08-01 promote).
+Live clusters match these defaults (verified 2026-10-04: Rancher `v2.15.2`, all four clusters RKE2 `v1.36.2+rke2r1`, cert-manager `v1.21.2` on the app clusters and `v1.21.1` on the manager, Ubuntu kernel `6.8.0-146` on all 31 nodes).
 
-| Pin | Variable | Default (as of 2026-07) | Notes |
+| Pin | Variable | Default (as of 2026-10) | Notes |
 |-----|----------|-------------------------|--------|
 | RKE2 | `rke2_version` | `v1.36.2+rke2r1` | New nodes only unless `enable_rke2_upgrade=true`. Needs Rancher 2.15+ (2.14.x max certified RKE2 is 1.35) |
-| Rancher | `rancher_version` | `v2.15.0` | Stable chart with K8s/RKE2 1.36 support; **minor upgrades must step** latest patch of current minor first (e.g. `v2.14.3` → `v2.14.4` → `v2.15.0`) |
+| Rancher | `rancher_version` | `v2.15.2` | Stable chart with K8s/RKE2 1.36 support; **minor upgrades must step** latest patch of current minor first (e.g. `v2.14.3` → `v2.14.4` → `v2.15.0`) |
 | Unattended upgrades | `enable_unattended_upgrades` | `true` | Security pocket only; `Automatic-Reboot false`. New nodes via `cloud-init-rke2.sh`; existing via `scripts/enable-unattended-upgrades.sh` |
 | OS patch | `enable_os_patch` | `false` | SSH `apt update/upgrade` via `scripts/patch-os-nodes.sh`; optional `os_patch_reboot` |
 
@@ -72,11 +73,19 @@ Example layout (mgmt `vmbr0`/`bond0`, storage jumbo `vmbr1`/`bond1`, aux `vmbr2`
 ## Known issues (2026-10-04)
 
 - **TrueNAS CSI pods restart often on all app clusters.** The TrueNAS API (middlewared, `192.168.9.10`) freezes for ~20–60 s about every 7 minutes; the driver's liveness probe treats the lost API connection as fatal. A driver fix is in progress in the `truenas-csi` fork. Restarts are noisy but mounts recover; don't chase them per pod.
-- **CNPG replicas stuck on timeline divergence (prd-apps) — fixed 2026-10-04.** `coder-postgres-1` and `high-command-postgres-1` were re-cloned with `kubectl cnpg destroy <cluster> <n> -n <ns>` (now `-6` and `-4`; both clusters 3/3). coder's dead slot had pinned about 115 GB of WAL by LSN (3.8G on disk thanks to ZFS compression). pg_wal is back to about 20M. Every CNPG cluster now sets `max_slot_wal_keep_size` (1–4GB, sized to the volume). Rebuild any future broken replica the same way. Use a `kubectl-cnpg` plugin that matches the operator version (1.28.x).
-- **CNPG backups (since 2026-10-04).** All four clusters (prd-apps: coder, high-command, authentik; nprd-apps: harbor) archive WAL and take daily base backups through the Barman Cloud plugin (gitops-core `cnpg-barman-cloud`) to `s3://rke2-backups/cnpg/<k8s-cluster>/<db>/`. They use the rustfs S3 API on port **30292**; 30293 is the console. Retention is 14 days. Restore and checks are documented in each repo's README. Adding or changing the plugin rolls each cluster with an in-place primary restart (`primaryUpdateMethod: restart`). Authentik SSO was down about 4 min (its server pods were restarted by the liveness probe); Coder and high-command lost the DB for under a minute.
+- **CNPG replicas stuck on timeline divergence (prd-apps) — fixed 2026-10-04.** `coder-postgres-1` and `high-command-postgres-1` were re-cloned with `kubectl cnpg destroy <cluster> <n> -n <ns>` (now `-6` and `-4`; both clusters 3/3). coder's dead slot had pinned about 115 GB of WAL by LSN (3.8G on disk thanks to ZFS compression). pg_wal is back to about 20M. Every CNPG cluster now sets `max_slot_wal_keep_size` (1–4GB, sized to the volume). Rebuild any future broken replica the same way. Use a `kubectl-cnpg` plugin that matches the operator version (1.30.x since 2026-10-04).
+- **CNPG backups (since 2026-10-04).** All four clusters (prd-apps: coder, high-command, authentik; nprd-apps: harbor) archive WAL and take daily base backups through the Barman Cloud plugin (gitops-core `cnpg-barman-cloud`) to `s3://rke2-backups/cnpg/<k8s-cluster>/<db>/`. They use the rustfs S3 API on port **30292**; 30293 is the console. Retention is 14 days. Restore and checks are documented in each repo's README. Adding the plugin rolled each cluster with an in-place primary restart (`primaryUpdateMethod: restart`): Authentik SSO was down about 4 min (its server pods were restarted by the liveness probe); Coder and high-command lost the DB for under a minute. Since 2026-10-04 all four clusters use `primaryUpdateMethod: switchover`. The CNPG 1.30.1 upgrade and the node drains cost authentik about 10–15 s per switchover and the others nothing measurable.
 - **Open: CNPG backups share the `rke2-backups` bucket and key with Rancher backups.** The existing rustfs key can't create buckets. Create a dedicated `cnpg-backups` bucket and a key scoped to it in the rustfs console, then update the `cnpg-backup-rustfs` Secrets (coder, high-command, authentik on prd-apps; harbor on nprd-apps) and each ObjectStore's `destinationPath`.
 - **high-command-postgres is now Fleet-managed** (gitops-mcp `k8s/high-command/`, adopted with `helm.takeOwnership`). Helm two-way-patches custom resources, so adoption only adds ownership metadata. Spec and Fleet labels were synced to git by hand once. Keep the manifest identical to live, including CNPG-defaulted fields such as `plugins[].enabled`, or Fleet's drift correction loops rollbacks.
-- **poc-apps wildcard cert expired (2026-04-18).** `cert-manager/cert-sync` (gitops-core, manager) fails for poc-apps: its kubeconfig entry uses the Rancher proxy URL with a deleted token, and the job image fails x509 verification. nprd/prd copies are current. Fix in gitops-core (RKE2 admin kubeconfig entry for poc-apps). The other entries' client certs expire 2027-01-08. See [CLUSTER_ACCESS_AND_SSO.md § Certificate calendar](CLUSTER_ACCESS_AND_SSO.md#certificate-and-expiry-calendar).
+- **poc-apps wildcard cert expired (2026-04-18) — RESOLVED 2026-10-04** via [gitops-core#6](https://github.com/DataKnifeAI/gitops-core/pull/6). `cert-manager/cert-sync` had been failing for poc-apps (Rancher proxy URL with a deleted token, x509 failure) while reporting success. All four entries of `cert-sync-kubeconfig` are now RKE2 admin client certs against `:6443`, and the job exits non-zero on any failure. The Secret was rebuilt after each cluster's RKE2 cert renewal on 2026-10-04, and each manual `cert-sync` run synced all four clusters. Its client certs now expire 2027-10-05. See [CLUSTER_ACCESS_AND_SSO.md § Certificate calendar](CLUSTER_ACCESS_AND_SSO.md#certificate-and-expiry-calendar).
+- **RKE2 leaf certs renewed 2026-10-04** by the rolling OS reboots (restart-triggered renewal inside the 120-day window). All server leaf certs and all agent certs now expire **2027-10-05**; poc-apps-1's server certs stay at 2027-09-19. Next renewal: any rke2 restart from 2027-05-22 (poc-apps-1) / 2027-06-07 (everything else), or `rke2 certificate rotate`. Details in [RKE2_CERT_ROTATION.md](RKE2_CERT_ROTATION.md#renewal-on-2026-10-04).
+- **LAN `:443` on Gateway VIPs answers with ingress-nginx (prd-apps).** `192.168.14.184:443` (high-command Gateway) and `.187:443` return the ingress-nginx default `*.dataknife.net` cert, not Envoy's `hc.dataknife.ai` cert. The rke2-ingress-nginx DaemonSet uses `hostPort: 443`, and the CNI hostPort DNAT matches **any** local address, including kube-vip VIPs bound on that node (`.184`/`.187` live on prd-apps-2). The high-command Envoy Service is also `externalTrafficPolicy: Local` with its only proxy pod on another node, so traffic would be dropped even without nginx. Public `hc.dataknife.ai` is unaffected: it goes through the Cloudflare tunnel. Fix options: an `EnvoyProxy` with `externalTrafficPolicy: Cluster` for high-command, plus either HTTPS listeners on a non-443 port or ingress-nginx off hostPort (the Traefik migration, [UPGRADE_PLAN.md](UPGRADE_PLAN.md) #8).
+- **Gateway API v1.6 follow-ups.** TCPRoute/UDPRoute storage moved to `v1`, but the game-server manifests in gitops still use `v1alpha2`, which is served but deprecated. Migrate the manifests to `v1`, rewrite stored objects (storage-version migration), then trim `status.storedVersions` on the CRDs. The orphaned `xlistenersets.gateway.networking.x-k8s.io` CRD (replaced by `listenersets` in v1.5) can be deleted once nothing lists it.
+- **Pre-existing on prd, not caused by the upgrades:** satisfactory routes show `ResolvedRefs=False` (no satisfactory server pod), and the windrose dashboard returns HTTP 000.
+- **Disk on prd-apps-worker-1 and -6 is 82–84%** after the 2026-10-04 reboots (worker-3 73%), close to the kubelet's default 85% image-GC threshold. Prune images or grow the disks.
+- **Game servers are pinned to single nodes:** palworld by a hostname nodeSelector (`prd-apps-worker-6`), windrose by `node-type: large` (only `prd-apps-large-worker-1`). Draining those nodes means downtime until the node is back (about 4–5 min for a reboot).
+- **Grafana chart is frozen.** `grafana/grafana` stops at `10.5.15`; nprd runs `12.4.12` through an `image.tag` override. Move to `grafana-community/grafana` before Grafana 13.
+- **system-upgrade-controller** is `v0.20.1` on every cluster after the Rancher `v2.15.2` upgrade (the release notes list `v0.20.2`). Check whether Rancher rolls it, or update the managed app.
 - **Terraform's Rancher API token is expired** (`config/.rancher-api-token` = `rancher_api_token` in tfvars, HTTP 401), and the registration modules read a hard-coded `/home/lee/git/rancher-deploy/config/.rancher-api-token`. Refresh before any `register_downstream_cluster` apply — [RANCHER_API_TOKEN_CREATION.md](RANCHER_API_TOKEN_CREATION.md).
 - **Break-glass kubeconfigs — resolved 2026-10-04.** Break-glass now lives in `~/.kube/<cluster>-rke2.yaml` (RKE2 admin, straight to `https://<cluster>.dataknife.net:6443`; `manager.dataknife.net` for rancher-manager). The old `~/.kube/<cluster>.yaml` files on the main workstation are Rancher-proxied token kubeconfigs (`rancher-manager.yaml`'s token is expired); don't use them for break-glass. Terraform's `get_kubeconfig` will overwrite them with RKE2 admin kubeconfigs on the next full apply. See [CLUSTER_ACCESS_AND_SSO.md § Fallback](CLUSTER_ACCESS_AND_SSO.md#fallback-and-break-glass).
 
@@ -90,3 +99,6 @@ Example layout (mgmt `vmbr0`/`bond0`, storage jumbo `vmbr1`/`bond1`, aux `vmbr2`
 | `~/.config/rancher-saml/` | Rancher SAML SP cert/key + Authentik IdP metadata (outside the repo) |
 | `~/.rancher/cli2.json` | Rancher CLI token cache (`rancher token delete all` to clear) |
 | `~/.kube/*.yaml`, `~/.kube/config*` | Kubeconfigs; RKE2 admin ones are cluster-admin |
+| `state/terraform.tfstate*` | Local Terraform state (backend path `../state/`); holds secrets in plaintext (gitignored) |
+
+On the main workstation `state/terraform.tfstate*` were mode `0777` on 2026-10-04 (and `terraform/terraform.tfstate.*.backup` `0644`). Fix with `chmod 600 state/terraform.tfstate* terraform/terraform.tfstate.*.backup`; keep `terraform.tfvars` at `0600`.
diff --git a/docs/RKE2_CERT_ROTATION.md b/docs/RKE2_CERT_ROTATION.md
index 8830a37..c44732f 100644
--- a/docs/RKE2_CERT_ROTATION.md
+++ b/docs/RKE2_CERT_ROTATION.md
@@ -1,12 +1,49 @@
 # RKE2 certificate rotation runbook
 
-RKE2 leaf certificates (client and server) are valid for 365 days. On the control-plane nodes of every cluster they expire in **January 2027**, and every server already emits `CertificateExpirationWarning` events. This runbook renews them one cluster at a time, poc-apps first. CA certificates (10 years, to 2036) are **not** touched.
+RKE2 leaf certificates (client and server) are valid for 365 days. On the control-plane nodes of every cluster they were due to expire in **January 2027**. This runbook renews them one cluster at a time, poc-apps first. CA certificates (10 years, to 2036) are **not** touched.
 
-**Status:** plan only — not executed yet. Recommended window [below](#recommended-window).
+**Status: done 2026-10-04.** The certs were renewed by restart during the rolling OS reboots ([UPGRADE_PLAN.md #4](UPGRADE_PLAN.md#executed-2026-10-04-items-16)), not by `rke2 certificate rotate`. Everything now expires **2027-10-05**; poc-apps-1's server certs stay at 2027-09-19. See [Renewal on 2026-10-04](#renewal-on-2026-10-04). The rest of this page is the runbook for next time.
 
-## Current expiries
+## Renewal on 2026-10-04
 
-Read on 2026-10-04 with `sudo rke2 certificate check --output table` on every server and one agent per cluster (all nodes run `v1.36.2+rke2r1`).
+Method: restart-triggered renewal. Every server was drained, patched and rebooted one at a time (etcd leader last), so `rke2-server` started inside the 120-day window and renewed every leaf cert on that node. The private keys were reused. Agents reissue their certs on every start, so the agent reboots refreshed those too. Pre-reboot backups: an etcd snapshot per cluster (`~/backups/etcd/pre-reboots-<cluster>-*`) and `/root/rke2-tls-pre-reboot-2026100{4,5}.tgz` on every server.
+
+| Cluster | Server | `:6443` cert notAfter (was) | Now |
+|---------|--------|-----------------------------|-----|
+| poc-apps | poc-apps-1 (.130) | 2027-09-19 05:26 UTC | unchanged (renewed 2026-09-19, outside the window) |
+| | poc-apps-3 (.132) | 2027-01-15 | **2027-10-05 01:39 UTC** |
+| | poc-apps-2 (.131, leader) | 2027-01-15 | **2027-10-05 01:44 UTC** |
+| nprd-apps | nprd-apps-1 (.110) | 2027-01-08 | **2027-10-05 02:16 UTC** |
+| | nprd-apps-3 (.112) | 2027-01-08 | **2027-10-05 02:21 UTC** |
+| | nprd-apps-2 (.111, leader) | 2027-01-08 | **2027-10-05 02:38 UTC** |
+| prd-apps | prd-apps-2 (.121) | 2027-01-08 | **2027-10-05 03:19 UTC** |
+| | prd-apps-3 (.122) | 2027-01-08 | **2027-10-05 03:24 UTC** |
+| | prd-apps-1 (.120, leader) | 2027-01-08 | **2027-10-05 03:29 UTC** |
+| rancher-manager | rancher-manager-1 (.100) | 2027-01-08 | **2027-10-05 04:11 UTC** |
+| | rancher-manager-3 (.102) | 2027-01-08 | **2027-10-05 04:16 UTC** |
+| | rancher-manager-2 (.101, leader) | 2027-01-08 | **2027-10-05 04:22 UTC** |
+
+`sudo rke2 certificate check` afterwards showed:
+- all 17 leaf certs on every server at 2027-10-05 (poc-apps-1: 13 at 2027-09-19 and 4 at 2027-10-05);
+- the agent certs on the sampled workers (one per cluster) at 2027-10-05;
+- no warnings.
+
+After each cluster's servers:
+- the break-glass `~/.kube/<cluster>-rke2.yaml` was re-fetched from a renewed server, with its client cert now expiring 2027-10-05 (old files kept as `~/.kube/<cluster>-rke2.yaml.bak-<ts>`);
+- `gitops-core/scripts/create-cert-sync-kubeconfig-secret.sh` was re-run;
+- a manual `cert-sync` job synced all four clusters.
+
+Old `system:admin` client certs (earlier `rke2.yaml` copies) remain valid until their own `notAfter` (2027-01-08, poc 2027-09-19); nothing was revoked.
+
+**Next renewal.** Any rke2-server restart inside the 120-day window renews again:
+- from **2027-05-22** on poc-apps-1;
+- from **2027-06-07** everywhere else.
+
+Plan a rolling reboot or RKE2 upgrade between 2027-06-07 and **2027-09-01**, or run `rke2 certificate rotate` at any time.
+
+## Expiries before the renewal
+
+Read on 2026-10-04 (before the renewal) with `sudo rke2 certificate check --output table` on every server and one agent per cluster (all nodes run `v1.36.2+rke2r1`).
 
 | Cluster | Node(s) | Server leaf certs¹ | kubelet / kube-proxy / rke2-controller | rke2 last started |
 |---------|---------|--------------------|----------------------------------------|-------------------|
@@ -49,6 +86,8 @@ Neither method revokes anything. Kubernetes has no CRL, so old `system:admin` cl
 
 ## Recommended window
 
+Superseded: the 2026-10-04 reboots did the renewal. The plan below was for the January 2027 expiry.
+
 - **Prerequisite:** the TrueNAS CSI v0.22 rollout and the Terraform state repair (in progress 2026-10-04) are finished, with at least one quiet week afterwards. No `terraform apply` runs during the window.
 - **poc-apps:** Saturday **2026-10-24** (only poc-apps-2 and -3 need it).
 - **nprd-apps:** Saturday **2026-10-31**.
@@ -121,7 +160,7 @@ echo | openssl s_client -connect $NODE:6443 2>/dev/null | openssl x509 -noout -e
 $SSH ubuntu@$NODE 'sudo rke2 certificate check --output table | grep -v CertSign'          # no WARNING rows
 ```
 
-**Agents:** no action required in this window (their certs run to 2027-08-01 / 2027-09-09 and are reissued on every agent start, e.g. the next RKE2 upgrade or OS-patch reboot). To refresh them anyway: `sudo systemctl restart rke2-agent` one node at a time, waiting for `Ready`. Put **2027-07-01** on the calendar as the latest date for a rolling agent restart if none has happened by then (prd-apps agents expire first, 2027-08-01).
+**Agents:** no action required in this window (their certs run to 2027-08-01 / 2027-09-09 and are reissued on every agent start, e.g. the next RKE2 upgrade or OS-patch reboot). To refresh them anyway: `sudo systemctl restart rke2-agent` one node at a time, waiting for `Ready`. Since the 2026-10-04 reboots every agent's certs run to 2027-10-05; put **2027-09-01** on the calendar as the latest date for a rolling agent restart if none has happened by then.
 
 ## After each cluster
 
diff --git a/docs/TRUENAS_CSI_MIGRATION.md b/docs/TRUENAS_CSI_MIGRATION.md
index e7ba5a7..189d6f1 100644
--- a/docs/TRUENAS_CSI_MIGRATION.md
+++ b/docs/TRUENAS_CSI_MIGRATION.md
@@ -122,9 +122,8 @@ democratic-csi used the `freenas-api-nfs` driver against `192.168.9.10`. TrueNAS
 | Dataset `SAS/RKE2` and children `SAS/RKE2/pvc-*` (e.g. `pvc-25d4289c-38a0-4c20-96ef-35d2cd91356d`, the old nprd Loki ingester) | Check nothing you want is left, then delete recursively |
 | Dataset `SAS/RKE2-snapshots` (detached snapshots parent), if present | Delete |
 | NFS shares with paths under `/mnt/SAS/RKE2/` | Delete (before the datasets) |
-| democratic-csi API key (was `democratic_csi_api_key`, user `rke2`) | Revoke. TrueNAS CSI uses its own key (`truenas_csi_api_key` → secret `truenas-api-credentials`) |
-
-Before revoking, confirm in **Credentials → API Keys** that the key's last-used time is not recent. If the same key value was ever pasted into `truenas_csi_api_key`, revoking it would break TrueNAS CSI on all app clusters.
+| Last democratic-csi API key (was `democratic_csi_api_key`, user `rke2`) | **Don't revoke.** It is the same key as `truenas_csi_api_key` (TrueNAS key ID 4), live in secret `truenas-csi/truenas-api-credentials` on nprd, prd and poc (checked 2026-10-04). Revoking it breaks TrueNAS CSI on every app cluster. To retire it, create a new key, roll it into `truenas_csi_api_key` and the three secrets, confirm the CSI pods are healthy, then revoke ID 4 |
+| Older democratic-csi API keys (TrueNAS key IDs 1 and 2) | Revoke; nothing uses them |
 
 ---
 
diff --git a/docs/UPGRADE_PLAN.md b/docs/UPGRADE_PLAN.md
index e0c3a9f..f2e6ff1 100644
--- a/docs/UPGRADE_PLAN.md
+++ b/docs/UPGRADE_PLAN.md
@@ -2,7 +2,7 @@
 
 Two parts:
 
-1. **[Next wave (reviewed 2026-10-04)](#next-wave-reviewed-2026-10-04)**: what's live now, what's behind, and a prioritized order. **Plan only**: nothing in it has been applied except the cert-manager work logged in [cert-manager v1.21.2 (done 2026-10-04)](#cert-manager-v1212-done-2026-10-04).
+1. **[Next wave (reviewed 2026-10-04)](#next-wave-reviewed-2026-10-04)**: what was live, what's behind, and a prioritized order. Items #1–#6 were applied on 2026-10-04 ([Executed 2026-10-04](#executed-2026-10-04-items-16)), along with the cert-manager work in [cert-manager v1.21.2 (done 2026-10-04)](#cert-manager-v1212-done-2026-10-04). Everything else is still plan only.
 2. **[History: 2026-07-31 → 2026-08-01 wave](#history-2026-07-31--2026-08-01-wave-rancher-213--215-rke2-134--136)**: the completed Rancher 2.13 → 2.15 / RKE2 1.34 → 1.36 runbook, kept as-is. Its gating, drain, and rollback procedures (§2, §5, §7) still apply to everything below.
 
 Ground rules carried over: back up Terraform state (`terraform state pull` → `~/terraform-state-backups/rancher-deploy/`, mode 600) before every apply; use `-refresh=false` and targeted plans, and apply only when the full plan contains exactly the intended resources; poc-apps first, then nprd-apps, then prd-apps, with the manager last unless noted; use the break-glass `~/.kube/<cluster>-rke2.yaml` for drains.
@@ -13,6 +13,8 @@ Ground rules carried over: back up Terraform state (`terraform state pull` → `
 
 ### Live snapshot
 
+Taken before the 2026-10-04 execution. What changed is in [Executed 2026-10-04](#executed-2026-10-04-items-16).
+
 | Area | Live (2026-10-04) |
 |------|-------------------|
 | Rancher (manager) | `v2.15.0`; rancher-webhook `0.11.0`, Fleet `0.16.0`, system-upgrade-controller `v0.20.1`, rancher-backup `9.0.1`, turtles `0.27.0` |
@@ -34,14 +36,14 @@ Priority: **P0** = security fix or EOL/unsupported on k8s 1.36, do next. **P1**
 
 | # | Pri | Component | Current → target (latest) | Why / risk notes | Owner | Effort |
 |---|-----|-----------|---------------------------|------------------|-------|--------|
-| 1 | P0 | **Rancher** | `v2.15.0` → **`v2.15.2`** | 2.15.1 fixes SAML assertion replay across HA replicas (CVE-2026-75034; we run 3 replicas behind Authentik SAML), plus token/RBAC CVEs. 2.15.2 fixes an unauthenticated settings XSS on the login page (CVE-2026-88804), Fleet CVEs, and Backups missing on k8s 1.36. Brings webhook `0.11.3`, Fleet `0.16.2`, SUC `v0.20.2`. No 2.16 GA yet | `rancher_version` tfvars; apply by Helm (`deploy-rancher.sh` has no version trigger): `--reset-then-reuse-values --set networkExposure.type=ingress` | S |
-| 2 | P0 | **Envoy Gateway** + **Gateway API CRDs** | EG `v1.6.1` → **`v1.9.2`**; CRDs `v1.4.1` → **`v1.6.1`** | EG 1.6 went EOL 2026-05-13 and was tested only up to k8s 1.33; 1.9 is the first line tested on 1.36 and is built against Gateway API v1.6.1. Step 1.7 → 1.8 → 1.9 and read each release's breaking changes. CRDs first (experimental channel). prd game-server and high-command Gateways ride on it. cert-manager 1.21's gateway-shim works with these CRDs | `envoy_gateway_version` / `gateway_api_version` tfvars → `modules/envoy_gateway` | M |
-| 3 | P0 | **CloudNativePG operator** | `1.28.0` → **`1.30.1`** | 1.28 gets no more patches (last was 1.28.4 on 2026-06-29; we never took it); 1.29/1.30 are the supported lines. Barman plugin `v0.15.1` (latest) needs CNPG ≥ 1.26, so it's fine. An operator upgrade restarts or in-place-updates every PG instance: wait until the Barman backups have a good base backup per cluster, and do one cluster at a time | `CNPG_MANIFEST_URL` in `terraform/main.tf` (3 copies; switch to a variable) | M |
-| 4 | P0 | **Node OS reboots** | 31/31 nodes reboot-required | Kernel/security updates are installed but not running. Rolling drain → reboot → uncordon (poc → nprd → prd → manager). Can share the drain with #7 (RKE2 patch) but keep OS and RKE2 as separate steps per node | `scripts/patch-os-nodes.sh` (per-IP) / manual | M |
-| 5 | P0 | **Grafana** (nprd) | `12.3.1` → `12.4.12`, then 13.x (`13.2.3`) | 12.3 is out of the security-patch window (fixes now ship for 12.4 / 13.0–13.2). The `grafana/grafana` chart is frozen at `10.5.15`; the chart moved to `grafana-community/grafana` (`13.2.7`). 13.x removes Angular plugins, so check dashboards | manual Helm + `gitops-tools` grafana overlay | M |
-| 6 | P0 | **Secret hygiene: democratic-csi leftovers** | n/a | `install_democratic_csi = false`, but `terraform.tfvars` still holds `democratic_csi_*` including a TrueNAS API key, and outputs still print the democratic config. Rotate or revoke that API key on TrueNAS, delete the vars/outputs, and delete the orphaned nprd PV | `terraform.tfvars` (local), `outputs.tf` | S |
-| 7 | P1 | **RKE2** | `v1.36.2+rke2r1` → **`v1.36.4+rke2r1`** | Highest 1.36 patch in Rancher 2.15's KDM (`v1.36.5` shipped 2026-09-30 but isn't in KDM yet, so take it once listed). etcd/grpc CVE bumps, containerd `2.3.4`, canal `3.32.2`, ingress-nginx `4.15.110` (prime build). **After #1.** Same drain + `scripts/upgrade-rke2-nodes.sh` per node. `v1.37.1` exists but needs Rancher 2.16 and #8 | `rke2_version` tfvars + script | M |
-| 8 | P1 | **ingress-nginx → Traefik** | rke2-ingress-nginx on all clusters | Upstream ingress-nginx was retired in March 2026. RKE2 1.36 defaults new clusters to Traefik and only patches "prime" nginx builds; **RKE2 1.37 removes the standalone ingress-nginx option**, so this blocks any 1.37 move. Affects every Ingress (Rancher, Harbor, Grafana, Authentik, Coder, MCP), the gitops-core `HelmChartConfig` default-cert setup (nginx-specific `--default-ssl-certificate`) and ingress annotations. `rke2-traefik-crd` also ships Gateway API CRDs, so pick a single owner for those CRDs (Envoy Gateway module vs RKE2) before enabling Traefik. Also investigate: the high-command Gateway VIP `192.168.14.184:443` answers with ingress-nginx's wildcard cert on the LAN (Envoy itself serves the right `hc.dataknife.ai` cert); likely host-network nginx on the node holding the VIP | RKE2 config templates (`ingress-controller`), `gitops-core` overlays, app Ingress manifests | L |
+| 1 | P0 | **Rancher** | `v2.15.0` → **`v2.15.2`** (**done 2026-10-04**) | 2.15.1 fixes SAML assertion replay across HA replicas (CVE-2026-75034; we run 3 replicas behind Authentik SAML), plus token/RBAC CVEs. 2.15.2 fixes an unauthenticated settings XSS on the login page (CVE-2026-88804), Fleet CVEs, and Backups missing on k8s 1.36. Brings webhook `0.11.3`, Fleet `0.16.2`, SUC `v0.20.2`. No 2.16 GA yet | `rancher_version` tfvars; apply by Helm (`deploy-rancher.sh` has no version trigger): `--reset-then-reuse-values --set networkExposure.type=ingress` | S |
+| 2 | P0 | **Envoy Gateway** + **Gateway API CRDs** | EG `v1.6.1` → **`v1.9.2`**; CRDs `v1.4.1` → **`v1.6.1`** (**done 2026-10-04**) | EG 1.6 went EOL 2026-05-13 and was tested only up to k8s 1.33; 1.9 is the first line tested on 1.36 and is built against Gateway API v1.6.1. Step 1.7 → 1.8 → 1.9 and read each release's breaking changes. CRDs first (experimental channel). prd game-server and high-command Gateways ride on it. cert-manager 1.21's gateway-shim works with these CRDs | `envoy_gateway_version` / `gateway_api_version` tfvars → `modules/envoy_gateway` | M |
+| 3 | P0 | **CloudNativePG operator** | `1.28.0` → **`1.30.1`** (**done 2026-10-04**) | 1.28 gets no more patches (last was 1.28.4 on 2026-06-29; we never took it); 1.29/1.30 are the supported lines. Barman plugin `v0.15.1` (latest) needs CNPG ≥ 1.26, so it's fine. An operator upgrade restarts or in-place-updates every PG instance: wait until the Barman backups have a good base backup per cluster, and do one cluster at a time | `CNPG_MANIFEST_URL` in `terraform/main.tf` (3 copies; switch to a variable) | M |
+| 4 | P0 | **Node OS reboots** | 31/31 nodes reboot-required (**done 2026-10-04**: all on `6.8.0-146`, RKE2 certs renewed) | Kernel/security updates are installed but not running. Rolling drain → reboot → uncordon (poc → nprd → prd → manager). Can share the drain with #7 (RKE2 patch) but keep OS and RKE2 as separate steps per node | `scripts/patch-os-nodes.sh` (per-IP) / manual | M |
+| 5 | P0 | **Grafana** (nprd) | `12.3.1` → `12.4.12` (**done 2026-10-04**), then 13.x (`13.2.3`) | 12.3 is out of the security-patch window (fixes now ship for 12.4 / 13.0–13.2). The `grafana/grafana` chart is frozen at `10.5.15`; the chart moved to `grafana-community/grafana` (`13.2.7`). 13.x removes Angular plugins, so check dashboards | manual Helm + `gitops-tools` grafana overlay | M |
+| 6 | P0 | **Secret hygiene: democratic-csi leftovers** | n/a (**done 2026-10-04**, except the TrueNAS key cleanup: [see below](#executed-2026-10-04-items-16)) | `install_democratic_csi = false`, but `terraform.tfvars` still holds `democratic_csi_*` including a TrueNAS API key, and outputs still print the democratic config. Rotate or revoke that API key on TrueNAS, delete the vars/outputs, and delete the orphaned nprd PV | `terraform.tfvars` (local), `outputs.tf` | S |
+| 7 | P1 | **RKE2** | `v1.36.2+rke2r1` → **`v1.36.4+rke2r1`** | Highest 1.36 patch in Rancher 2.15's KDM (`v1.36.5` shipped 2026-09-30 but isn't in KDM yet, so take it once listed). etcd/grpc CVE bumps, containerd `2.3.4`, canal `3.32.2`, ingress-nginx `4.15.110` (prime build). **After #1.** Same drain + `scripts/upgrade-rke2-nodes.sh` per node. `v1.37.1` exists but needs Rancher 2.16 and #8. The RKE2 leaf certs no longer depend on this: the #4 reboots renewed them to 2027-10-05 | `rke2_version` tfvars + script | M |
+| 8 | P1 | **ingress-nginx → Traefik** | rke2-ingress-nginx on all clusters | Upstream ingress-nginx was retired in March 2026. RKE2 1.36 defaults new clusters to Traefik and only patches "prime" nginx builds; **RKE2 1.37 removes the standalone ingress-nginx option**, so this blocks any 1.37 move. Affects every Ingress (Rancher, Harbor, Grafana, Authentik, Coder, MCP), the gitops-core `HelmChartConfig` default-cert setup (nginx-specific `--default-ssl-certificate`) and ingress annotations. `rke2-traefik-crd` also ships Gateway API CRDs, so pick a single owner for those CRDs (Envoy Gateway module vs RKE2) before enabling Traefik. The high-command Gateway VIP `192.168.14.184:443` answering with ingress-nginx's wildcard cert on the LAN was investigated on 2026-10-04: ingress-nginx's `hostPort: 443` also catches traffic to kube-vip VIPs bound on its node ([OPS_NOTES.md](OPS_NOTES.md#known-issues-2026-10-04)). Moving off ingress-nginx, or off its hostPort, removes it | RKE2 config templates (`ingress-controller`), `gitops-core` overlays, app Ingress manifests | L |
 | 9 | P1 | **Terraform providers / CLI** | rancher2 `3.4.0` → **`v15.x`** (`15.1.2`); proxmox `0.93.0` → `0.115.0`; kubernetes `2.38.0` → `3.3.0`; null `3.2.4` → `3.3.2`; local `2.6.1` → `2.9.1`; CLI `1.15.9` → `1.16.5` | rancher2 majors track the Rancher minor since v13 (v15 = Rancher 2.15); 3.x predates that. The proxmox provider jump is large: expect schema/plan diffs on all VMs. Fix the known refresh problems first (rancher-manager-1 refresh error, pve1/pve2 placement, Ubuntu image re-download), then upgrade one provider at a time and review `-refresh=false` plans. Commit `.terraform.lock.hcl`. Terraform's Rancher API token (`config/.rancher-api-token`) is expired and must be renewed before rancher2 resources can plan | `terraform/provider.tf`, lock file | M–L |
 | 10 | P1 | **kube-vip** | `v1.0.3` → `v1.2.4` | Two minors behind; every LoadBalancer VIP depends on it. poc first and watch for VIP flaps | `kube_vip_version` tfvars → `modules/kube-vip` | S–M |
 | 11 | P1 | **Monitoring stack** | kube-prometheus-stack `88.0.1` → `91.9.0`; Alloy image `v1.12.2` → `v1.20.1` (chart `1.13.0`); Loki `3.6.11` → `3.6.17` (or `3.7.8`, chart `7.3.0`) | kps is three majors behind: apply the prometheus-operator CRDs (`v0.94.x`) before the chart, one major at a time. The Alloy image is pinned well below its own chart's appVersion; drop the override. Loki patch first, 3.7 later | manual Helm + `gitops-tools` monitoring/grafana overlays | M |
@@ -75,6 +77,33 @@ Priority: **P0** = security fix or EOL/unsupported on k8s 1.36, do next. **P1**
 4. **Window D:** #3 CNPG 1.30 (nprd Harbor, then prd Coder → Authentik → high-command).
 5. **Then:** #5 Grafana, #11 monitoring, #10 kube-vip, #12–#14, #16–#21. Start #8 (Traefik) on poc in parallel as a longer project, and #9 once refresh issues are fixed.
 
+### Executed 2026-10-04 (items #1–#6)
+
+In this order: Rancher, democratic-csi, Envoy Gateway, CloudNativePG, Grafana, node reboots. Each step had its own backups and verification. Nothing regressed, and nothing was rolled back.
+
+| # | Item | What was done | PRs | Backups (workstation unless noted) |
+|---|------|---------------|-----|------------------------------------|
+| 1 | Rancher `v2.15.2` | Helm upgrade with unchanged values (revision 6). Rancher 3/3; rancher-webhook `v0.11.3`, Fleet `v0.16.2`; all four clusters Ready and Connected. system-upgrade-controller is still `v0.20.1` on every cluster (the release notes list `v0.20.2`). `rancher_version` default bumped in this PR | this PR (pin only) | rancher-backup `pre-rancher-2152-20261004-1817` (S3); etcd `~/backups/etcd/manager-pre-rancher-2152-*`; `~/backups/rancher/rancher-{values,manifest}-rev5-20261004.yaml` |
+| 2 | democratic-csi leftovers | Variables, resources, output and scripts removed; orphaned nprd PV deleted (its dataset is still on TrueNAS). **TrueNAS API key: don't revoke the key from `democratic_csi_api_key`.** It was the same key as `truenas_csi_api_key` (TrueNAS key ID 4), which is live in `truenas-csi/truenas-api-credentials` on nprd, prd and poc. Older democratic-csi keys (IDs 1 and 2) are unused and should be revoked | [#39](https://github.com/DataKnifeAI/rancher-deploy/pull/39), [#44](https://github.com/DataKnifeAI/rancher-deploy/pull/44) | PV manifest `~/backups/democratic-csi/` |
+| 3 | Envoy Gateway `v1.9.2`, Gateway API `v1.6.1` | Stepped `1.6.1` → `1.7.5` → `1.8.5` → `1.9.2` on poc, nprd, then prd, with the matching CRDs applied first (server-side, experimental channel). Proxies on `distroless-v1.39.1`. prd checked against a baseline after every step: Gateway IPs `.184`–`.187` Accepted/Programmed, routes, game TCP ports, `hc.dataknife.ai` 200 with its own cert (to 2026-12-29). Terraform reconciled with a targeted apply of the six envoy `null_resource`s | [#40](https://github.com/DataKnifeAI/rancher-deploy/pull/40), [#42](https://github.com/DataKnifeAI/rancher-deploy/pull/42) | `~/backups/envoy-gateway/<cluster>-pre-1.9.2-20261004-183212/` |
+| 4 | CloudNativePG `1.30.1` | Targeted Terraform applies (poc, nprd, prd) after removing the destroy provisioners that would have deleted the CNPG CRDs on a version bump. All four clusters were switched to `primaryUpdateMethod: switchover` first. Switchovers: harbor 2 → 4, authentik 1 → 2, coder 4 → 5, high-command 3 → 2. Verified instance manager `1.30.1`, primary Leases, WAL archiving, `cnpg_metrics_exporter`, post-upgrade Barman backups | [#41](https://github.com/DataKnifeAI/rancher-deploy/pull/41); gitops-tools [#51](https://github.com/DataKnifeAI/gitops-tools/pull/51), gitops-dev [#8](https://github.com/DataKnifeAI/gitops-dev/pull/8), gitops-mcp [#16](https://github.com/DataKnifeAI/gitops-mcp/pull/16) | On-demand Barman backups `*-pre-cnpg-1301-*` per cluster; `pg_dumpall` to `~/backups/cnpg/<cluster>-<ts>.sql.gz`; Terraform state under `~/terraform-state-backups/rancher-deploy/` |
+| 5 | Grafana `12.4.12` (nprd) | `image.tag` override on the frozen `grafana/grafana` chart `10.5.15`; rollout `maxSurge: 0, maxUnavailable: 1` (RWO volume). Health, admin login, Loki and Prometheus datasources OK. Unified-storage migration ran automatically. The admin password was rotated the same day | gitops-tools [#52](https://github.com/DataKnifeAI/gitops-tools/pull/52) | `~/backups/grafana/nprd-pre-12.4.12-20261004-183818/` (`grafana.db` + API exports) |
+| 6 | Node OS reboots, 31/31 | poc → nprd → prd → manager; servers one at a time with the etcd leader last, then agents. Per node: drain (PDBs respected, no `--force`), `apt-get upgrade` (~41 packages, kernel `6.8.0-146`), reboot, wait for the new boot ID + `Ready` (servers also `/readyz`, etcd 3/3 healthy and the four static pods), uncordon, wait for the node's pods to settle. CNPG moved each primary off a cordoned node by itself. After each cluster's servers: break-glass kubeconfig re-fetched, `cert-sync-kubeconfig` rebuilt, `cert-sync` job succeeded. RKE2 leaf certs renewed ([RKE2_CERT_ROTATION.md](RKE2_CERT_ROTATION.md#renewal-on-2026-10-04)) | — | etcd `~/backups/etcd/pre-reboots-<cluster>-*`; server TLS tarballs `/root/rke2-tls-pre-reboot-2026100{4,5}.tgz` on each server (root, 600) |
+
+**Impact.** Measured on prd with an HTTP/TCP probe every ~2.5 s:
+- CNPG switchovers: authentik had about 10–15 s of errors; coder, high-command and harbor had none.
+- Envoy: every Envoy Gateway step rolled the proxies (three short game and high-command interruptions).
+- Reboots, most services: about 10–15 s blips each time a single-replica pod moved (high-command, coder, authentik), and up to 25 s on game TCP listeners when an Envoy proxy moved.
+- Reboots, game servers: palworld was down about 4 min (pinned to `prd-apps-worker-6` by a hostname nodeSelector), and windrose about 5 min (needs `node-type: large`, and `prd-apps-large-worker-1` is the only such node). Both came back by themselves.
+- Grafana: one pod replacement.
+
+**Learnings for the next window.**
+- Run long rolling scripts detached (log to a file, poll it) and behind a lock. An agent's tool runner re-ran a long command, so nprd-apps-1 and -3 were rebooted twice. No impact, but only because the script waits for etcd 3/3 before moving on.
+- The reboot script now refuses to start if another node is NotReady or cordoned, or if etcd isn't 3/3.
+- Node status and pod status stay stale for about 40 s after a fast reboot. Gate on `.status.nodeInfo.bootID` changing, not on `Ready` alone.
+- Nodes may report NotReady for 10–20 s just after uncordon; they recover by themselves.
+- Pin floating tags before reboots: they re-pull on reschedule. Coder workspaces still run `archlinux:latest`.
+
 ### cert-manager v1.21.2 (done 2026-10-04)
 
 [PR #36](https://github.com/DataKnifeAI/rancher-deploy/pull/36).
diff --git a/terraform/environments/manager/terraform.tfvars.example b/terraform/environments/manager/terraform.tfvars.example
index 257528a..70b20a9 100644
--- a/terraform/environments/manager/terraform.tfvars.example
+++ b/terraform/environments/manager/terraform.tfvars.example
@@ -14,7 +14,7 @@ ssh_private_key = "~/.ssh/id_rsa"
 # Rancher Configuration
 rancher_hostname = "rancher.lab.local"
 rancher_password = "your-secure-password"
-rancher_version  = "v2.15.0"
+rancher_version  = "v2.15.2"
 
 # Network Configuration
 domain      = "lab.local"
diff --git a/terraform/terraform.tfvars.example b/terraform/terraform.tfvars.example
index 49202c4..f4646ad 100644
--- a/terraform/terraform.tfvars.example
+++ b/terraform/terraform.tfvars.example
@@ -177,7 +177,7 @@ cert_manager_version = "v1.21.2"
 # Set to false to deploy manager and apps clusters without Rancher
 install_rancher = true
 
-rancher_version = "v2.15.0"
+rancher_version = "v2.15.2"
 # Live upgrade path (Rancher minor stepping): latest patch of current minor → next minor.
 # Example from 2.14.3: first set v2.14.4 and apply Rancher only, then set v2.15.0.
 # Rancher 2.14.x only certifies RKE2 through 1.35; 1.36 needs Rancher 2.15+.
diff --git a/terraform/variables.tf b/terraform/variables.tf
index fa5ee59..0794269 100644
--- a/terraform/variables.tf
+++ b/terraform/variables.tf
@@ -104,7 +104,7 @@ variable "ubuntu_cloud_image_url" {
 variable "rancher_version" {
   description = "Rancher Helm chart / image version (stable). New installs use this pin. Live upgrades from older minors must step latest-patch-of-current-minor → next minor (e.g. v2.14.3 → v2.14.4 → v2.15.0). Rancher 2.15+ required for Kubernetes/RKE2 1.36."
   type        = string
-  default     = "v2.15.0"
+  default     = "v2.15.2"
 }
 
 variable "rke2_version" {