From 7dcbd91bd46a0e4b6ee689178a0fbdf3fe5e8806 Mon Sep 17 00:00:00 2001 From: Danathar Date: Sun, 20 Sep 2026 10:36:21 +0000 Subject: [PATCH 1/7] fix(k0s): ship the complete Argo CD core stack so argocd-server goes ready MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `files/k0s/manifests/argocd/install.yaml` shipped only a Namespace, a bare ServiceAccount, argocd-cm, and the argocd-server Deployment/Service. That cannot work: argocd-server waits for its AppProject/Application informers to sync and for the settings manager to read and seed argocd-cm/argocd-secret before it binds :8080, so with no CRDs and no RBAC it stays 0/1 and the readiness probe reports "connection refused" against http://:8080/healthz. Vendor the rest of the core stack into the same file: - Application, AppProject and ApplicationSet CRDs, schema-trimmed with `x-kubernetes-preserve-unknown-fields` the same way the KubeFlex CRDs in files/k0s/manifests/kubestellar/00-kubeflex-crds.yaml are. - ServiceAccounts, namespace Roles/RoleBindings and ClusterRoles/Bindings for argocd-server and argocd-application-controller. The server ClusterRole follows upstream and stays at get/patch/delete — the resource tree is served from the controller's cache, so cluster-wide list/watch is not needed. - argocd-redis (cache only: no persistence, non-root, ClusterIP), together with argocd-repo-server and the argocd-application-controller StatefulSet. - argocd-rbac-cm, argocd-ssh-known-hosts-cm, argocd-tls-certs-cm and an empty argocd-secret, which argocd-server populates with server.secretkey on first start. No credential is committed. The server keeps plain /healthz on both probes rather than upstream's `?full=true` liveness: on a cold first boot that variant fails while redis and the repo server are still pulling and would restart the API server in a loop. Redis runs without requirepass. The Service is ClusterIP on a single-node cluster, and Argo CD reads REDIS_PASSWORD from the optional `argocd-redis` Secret, which this stack deliberately does not create. tests/unit/test_k0s_manifests.py gains four checks over the parsed manifest: the stack is complete, every RoleBinding/ClusterRoleBinding resolves to a shipped role and ServiceAccount, workloads mount only declared volumes and existing ConfigMaps and address redis/repo-server through the shipped Services, and argocd-secret ships empty. Fixes #177 Assisted-by: Claude Opus 5 via Claude Code Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Danathar --- docs/skills/k0s-sysext-ops.md | 1 + docs/skills/k0s-sysext.md | 2 +- files/k0s/manifests/argocd/install.yaml | 544 +++++++++++++++++++++++- tests/unit/test_k0s_manifests.py | 86 ++++ 4 files changed, 629 insertions(+), 4 deletions(-) diff --git a/docs/skills/k0s-sysext-ops.md b/docs/skills/k0s-sysext-ops.md index 7b806f80..a08b2a44 100644 --- a/docs/skills/k0s-sysext-ops.md +++ b/docs/skills/k0s-sysext-ops.md @@ -51,6 +51,7 @@ k0s kubectl get pods -A - **Extension not merged**: Check `systemd-sysext status`. Verify the persistent image is `/var/lib/k0s/k0s.raw`; the boot activation unit copies it into `/run/extensions/k0s.raw`. - **Manifests not applied**: Check `/var/lib/k0s/manifests/`. Ensure files end in `.yaml` (not `.yml`). - **Service failed**: Check `journalctl -u k0scontroller -e`. +- **`argocd-server` stuck at `0/1` with `connection refused` on `/healthz`**: it is waiting on a dependency, not the probe. Confirm `argocd-redis`, `argocd-repo-server` and `argocd-application-controller` are `Running`, then check `k0s kubectl -n argocd logs deploy/argocd-server` for RBAC `forbidden` errors — the server only binds `:8080` once its configmap/secret and Application informers have synced. ## See also diff --git a/docs/skills/k0s-sysext.md b/docs/skills/k0s-sysext.md index 1e38f94c..fcb9640e 100644 --- a/docs/skills/k0s-sysext.md +++ b/docs/skills/k0s-sysext.md @@ -59,7 +59,7 @@ Design choices: | `files/k0s/sysext/k0scontroller.service` | systemd unit for the k0s single-node controller/worker. Not enabled by default. | | `files/k0s/sysext/extension-release.k0s` | Static sysext identity (`ID=_any`); `VERSION_ID=`/`ARCHITECTURE=` are appended at build time. | | `files/k0s/sysext/k0s-manifests.conf` | tmpfiles rule that copies declarative stacks to `/var/lib/k0s/manifests/`. | -| `files/k0s/manifests/argocd/` | Raw YAML manifests for Argo CD. | +| `files/k0s/manifests/argocd/` | Raw YAML manifests for the Argo CD core stack: CRDs, RBAC, redis, repo-server, application controller, and server. All four workloads are required — the server blocks on its informers and never binds `:8080` without them. | | `files/k0s/manifests/kubestellar/` | Raw YAML manifests for KubeFlex, Postgres, KubeStellar core, and Console. | | `files/os/sysupdate.k0s.d/70-k0s.transfer` | sysupdate transfer track for the k0s sysext component. | | `Justfile` | `build-sysext` / `export-sysext` targets. | diff --git a/files/k0s/manifests/argocd/install.yaml b/files/k0s/manifests/argocd/install.yaml index ceb45aff..9fdee713 100644 --- a/files/k0s/manifests/argocd/install.yaml +++ b/files/k0s/manifests/argocd/install.yaml @@ -1,8 +1,128 @@ +# Argo CD core stack for the single-node k0s sysext. +# +# argocd-server cannot serve :8080 on its own: before it binds the listener it +# waits for its AppProject/Application informers to sync and for the settings +# manager to read (and seed) argocd-cm / argocd-secret. Without the CRDs, the +# namespace RBAC and the supporting workloads it stays 0/1 and the readiness +# probe reports "connection refused". This stack therefore ships the four +# components Argo CD needs to reach a healthy state: redis, repo-server, +# application-controller and server, plus their CRDs, RBAC and config. +# +# CRD schemas are intentionally trimmed to `x-kubernetes-preserve-unknown-fields` +# (same approach as the KubeFlex CRDs in ../kubestellar/00-kubeflex-crds.yaml) so +# the sysext payload stays small; the API server still stores and serves the full +# resource. The ApplicationSet CRD is registered for API completeness — the +# applicationset controller is not part of this stack. apiVersion: v1 kind: Namespace metadata: name: argocd --- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: applications.argoproj.io + labels: + app.kubernetes.io/part-of: argocd +spec: + group: argoproj.io + names: + kind: Application + listKind: ApplicationList + plural: applications + singular: application + shortNames: + - app + - apps + scope: Namespaced + versions: + - name: v1alpha1 + served: true + storage: true + subresources: + status: {} + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + x-kubernetes-preserve-unknown-fields: true + status: + type: object + x-kubernetes-preserve-unknown-fields: true + operation: + type: object + x-kubernetes-preserve-unknown-fields: true +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: appprojects.argoproj.io + labels: + app.kubernetes.io/part-of: argocd +spec: + group: argoproj.io + names: + kind: AppProject + listKind: AppProjectList + plural: appprojects + singular: appproject + shortNames: + - appproj + - appprojs + scope: Namespaced + versions: + - name: v1alpha1 + served: true + storage: true + subresources: + status: {} + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + x-kubernetes-preserve-unknown-fields: true + status: + type: object + x-kubernetes-preserve-unknown-fields: true +--- +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: applicationsets.argoproj.io + labels: + app.kubernetes.io/part-of: argocd +spec: + group: argoproj.io + names: + kind: ApplicationSet + listKind: ApplicationSetList + plural: applicationsets + singular: applicationset + shortNames: + - appset + - appsets + scope: Namespaced + versions: + - name: v1alpha1 + served: true + storage: true + subresources: + status: {} + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + x-kubernetes-preserve-unknown-fields: true + status: + type: object + x-kubernetes-preserve-unknown-fields: true +--- apiVersion: v1 kind: ServiceAccount metadata: @@ -10,6 +130,24 @@ metadata: namespace: argocd --- apiVersion: v1 +kind: ServiceAccount +metadata: + name: argocd-application-controller + namespace: argocd +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argocd-repo-server + namespace: argocd +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argocd-redis + namespace: argocd +--- +apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm @@ -20,6 +158,373 @@ metadata: data: application.instanceLabelKey: argocd.argoproj.io/instance --- +apiVersion: v1 +kind: ConfigMap +metadata: + name: argocd-rbac-cm + namespace: argocd + labels: + app.kubernetes.io/name: argocd-rbac-cm + app.kubernetes.io/part-of: argocd +data: + policy.default: role:readonly +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: argocd-ssh-known-hosts-cm + namespace: argocd + labels: + app.kubernetes.io/name: argocd-ssh-known-hosts-cm + app.kubernetes.io/part-of: argocd +data: + ssh_known_hosts: "" +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: argocd-tls-certs-cm + namespace: argocd + labels: + app.kubernetes.io/name: argocd-tls-certs-cm + app.kubernetes.io/part-of: argocd +--- +# Seeded empty on purpose: argocd-server generates `server.secretkey` and the +# initial admin bcrypt hash into this Secret on first start. +apiVersion: v1 +kind: Secret +metadata: + name: argocd-secret + namespace: argocd + labels: + app.kubernetes.io/name: argocd-secret + app.kubernetes.io/part-of: argocd +type: Opaque +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argocd-server + namespace: argocd + labels: + app.kubernetes.io/name: argocd-server + app.kubernetes.io/part-of: argocd +rules: +- apiGroups: [""] + resources: [secrets, configmaps] + verbs: [create, get, list, watch, update, patch, delete] +- apiGroups: [""] + resources: [events] + verbs: [create, list] +- apiGroups: [""] + resources: [pods, pods/log] + verbs: [get, list, watch] +- apiGroups: [argoproj.io] + resources: [applications, applicationsets, appprojects] + verbs: [create, get, list, watch, update, patch, delete] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argocd-application-controller + namespace: argocd + labels: + app.kubernetes.io/name: argocd-application-controller + app.kubernetes.io/part-of: argocd +rules: +- apiGroups: [""] + resources: [secrets, configmaps] + verbs: [get, list, watch] +- apiGroups: [""] + resources: [events] + verbs: [create, list] +- apiGroups: [argoproj.io] + resources: [applications, applicationsets, appprojects] + verbs: [create, get, list, watch, update, patch, delete] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argocd-server + namespace: argocd + labels: + app.kubernetes.io/name: argocd-server + app.kubernetes.io/part-of: argocd +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argocd-server +subjects: +- kind: ServiceAccount + name: argocd-server + namespace: argocd +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argocd-application-controller + namespace: argocd + labels: + app.kubernetes.io/name: argocd-application-controller + app.kubernetes.io/part-of: argocd +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argocd-application-controller +subjects: +- kind: ServiceAccount + name: argocd-application-controller + namespace: argocd +--- +# Matches the upstream argocd-server ClusterRole: the UI reads live resources +# and performs resource actions, but it does not need cluster-wide list/watch — +# the application controller's cache serves the resource tree. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: argocd-server + labels: + app.kubernetes.io/name: argocd-server + app.kubernetes.io/part-of: argocd +rules: +- apiGroups: ["*"] + resources: ["*"] + verbs: [get, patch, delete] +- apiGroups: [""] + resources: [events] + verbs: [list] +- apiGroups: [""] + resources: [pods, pods/log] + verbs: [get] +--- +# The application controller reconciles arbitrary user-supplied manifests, so it +# needs full access to every API group — this mirrors the upstream Argo CD +# ClusterRole for argocd-application-controller. +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: argocd-application-controller + labels: + app.kubernetes.io/name: argocd-application-controller + app.kubernetes.io/part-of: argocd +rules: +- apiGroups: ["*"] + resources: ["*"] + verbs: ["*"] +- nonResourceURLs: ["*"] + verbs: ["*"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: argocd-server + labels: + app.kubernetes.io/name: argocd-server + app.kubernetes.io/part-of: argocd +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: argocd-server +subjects: +- kind: ServiceAccount + name: argocd-server + namespace: argocd +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: argocd-application-controller + labels: + app.kubernetes.io/name: argocd-application-controller + app.kubernetes.io/part-of: argocd +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: argocd-application-controller +subjects: +- kind: ServiceAccount + name: argocd-application-controller + namespace: argocd +--- +apiVersion: v1 +kind: Service +metadata: + name: argocd-redis + namespace: argocd + labels: + app.kubernetes.io/name: argocd-redis + app.kubernetes.io/part-of: argocd +spec: + type: ClusterIP + ports: + - port: 6379 + targetPort: redis + name: tcp-redis + selector: + app.kubernetes.io/name: argocd-redis +--- +# Cache only — no persistence and no requirepass. The Service is ClusterIP and +# the cluster is single-node, so redis is never reachable off-host; Argo CD +# reads REDIS_PASSWORD from the optional `argocd-redis` Secret, which this +# stack deliberately does not create. +apiVersion: apps/v1 +kind: Deployment +metadata: + name: argocd-redis + namespace: argocd + labels: + app.kubernetes.io/name: argocd-redis + app.kubernetes.io/part-of: argocd +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + app.kubernetes.io/name: argocd-redis + template: + metadata: + labels: + app.kubernetes.io/name: argocd-redis + spec: + serviceAccountName: argocd-redis + securityContext: + runAsNonRoot: true + runAsUser: 999 + runAsGroup: 1000 + containers: + - name: redis + image: docker.io/library/redis:7.0.15-alpine + args: + - --save + - "" + - --appendonly + - "no" + ports: + - containerPort: 6379 + name: redis + readinessProbe: + tcpSocket: + port: redis + initialDelaySeconds: 5 + periodSeconds: 10 +--- +apiVersion: v1 +kind: Service +metadata: + name: argocd-repo-server + namespace: argocd + labels: + app.kubernetes.io/name: argocd-repo-server + app.kubernetes.io/part-of: argocd +spec: + type: ClusterIP + ports: + - port: 8081 + targetPort: server + name: server + - port: 8084 + targetPort: metrics + name: metrics + selector: + app.kubernetes.io/name: argocd-repo-server +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: argocd-repo-server + namespace: argocd + labels: + app.kubernetes.io/name: argocd-repo-server + app.kubernetes.io/part-of: argocd +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: argocd-repo-server + template: + metadata: + labels: + app.kubernetes.io/name: argocd-repo-server + spec: + serviceAccountName: argocd-repo-server + containers: + - name: argocd-repo-server + image: quay.io/argoproj/argocd:v2.12.3 + command: [argocd-repo-server, --redis, "argocd-redis:6379"] + ports: + - containerPort: 8081 + name: server + - containerPort: 8084 + name: metrics + readinessProbe: + tcpSocket: + port: server + initialDelaySeconds: 5 + periodSeconds: 10 + volumeMounts: + - name: ssh-known-hosts + mountPath: /app/config/ssh + - name: tls-certs + mountPath: /app/config/tls + - name: tmp + mountPath: /tmp + - name: helm-working-dir + mountPath: /helm-working-dir + volumes: + - name: ssh-known-hosts + configMap: + name: argocd-ssh-known-hosts-cm + - name: tls-certs + configMap: + name: argocd-tls-certs-cm + - name: tmp + emptyDir: {} + - name: helm-working-dir + emptyDir: {} +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: argocd-application-controller + namespace: argocd + labels: + app.kubernetes.io/name: argocd-application-controller + app.kubernetes.io/part-of: argocd +spec: + serviceName: argocd-application-controller + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: argocd-application-controller + template: + metadata: + labels: + app.kubernetes.io/name: argocd-application-controller + spec: + serviceAccountName: argocd-application-controller + containers: + - name: argocd-application-controller + image: quay.io/argoproj/argocd:v2.12.3 + command: + - argocd-application-controller + - --redis + - argocd-redis:6379 + - --repo-server + - argocd-repo-server:8081 + env: + - name: ARGOCD_CONTROLLER_REPLICAS + value: "1" + ports: + - containerPort: 8082 + name: metrics + readinessProbe: + httpGet: + path: /healthz + port: metrics + initialDelaySeconds: 5 + periodSeconds: 10 +--- apiVersion: apps/v1 kind: Deployment metadata: @@ -42,16 +547,49 @@ spec: containers: - name: argocd-server image: quay.io/argoproj/argocd:v2.12.3 - command: [argocd-server, --insecure, --staticassets, /shared/app] + command: + - argocd-server + - --insecure + - --staticassets + - /shared/app + - --redis + - argocd-redis:6379 + - --repo-server + - argocd-repo-server:8081 ports: - containerPort: 8080 name: server + # Plain /healthz on both probes: the "full" variant also fails when + # redis or the repo-server is still pulling, which would restart the + # API server in a loop on a cold first boot. readinessProbe: httpGet: path: /healthz - port: 8080 + port: server initialDelaySeconds: 5 periodSeconds: 10 + livenessProbe: + httpGet: + path: /healthz + port: server + initialDelaySeconds: 30 + periodSeconds: 30 + volumeMounts: + - name: ssh-known-hosts + mountPath: /app/config/ssh + - name: tls-certs + mountPath: /app/config/tls + - name: tmp + mountPath: /tmp + volumes: + - name: ssh-known-hosts + configMap: + name: argocd-ssh-known-hosts-cm + - name: tls-certs + configMap: + name: argocd-tls-certs-cm + - name: tmp + emptyDir: {} --- apiVersion: v1 kind: Service @@ -65,7 +603,7 @@ spec: type: ClusterIP ports: - port: 80 - targetPort: 8080 + targetPort: server name: http selector: app.kubernetes.io/name: argocd-server diff --git a/tests/unit/test_k0s_manifests.py b/tests/unit/test_k0s_manifests.py index 149ca421..c8e56e0f 100644 --- a/tests/unit/test_k0s_manifests.py +++ b/tests/unit/test_k0s_manifests.py @@ -6,6 +6,11 @@ import yaml ROOT = Path(__file__).resolve().parents[2] +ARGOCD_INSTALL = ROOT / "files" / "k0s" / "manifests" / "argocd" / "install.yaml" + + +def argocd_docs(): + return [d for d in yaml.safe_load_all(ARGOCD_INSTALL.read_text()) if d] def test_k0s_service_unit(): @@ -40,6 +45,87 @@ def test_k0s_manifest_files(): assert (ks_dir / "41-kubestellar-kiosk-proxy.yaml").is_file() +def test_argocd_stack_is_complete(): + # #177: argocd-server never binds :8080 — and so never passes its readiness + # probe — unless redis, the repo server, the application controller and the + # CRDs it watches are all part of the stack. + docs = argocd_docs() + workloads = {(d["kind"], d["metadata"]["name"]) for d in docs} + for name in ( + "argocd-redis", + "argocd-repo-server", + "argocd-server", + ): + assert ("Deployment", name) in workloads, f"{name} Deployment missing" + assert ("Service", name) in workloads, f"{name} Service missing" + assert ("StatefulSet", "argocd-application-controller") in workloads + + crds = {d["metadata"]["name"] for d in docs if d["kind"] == "CustomResourceDefinition"} + assert crds == { + "applications.argoproj.io", + "appprojects.argoproj.io", + "applicationsets.argoproj.io", + } + + +def test_argocd_rbac_is_bound(): + # Without namespace RBAC the server's configmap/secret and Application + # informers never sync, which is the state issue #177 reported. + docs = argocd_docs() + accounts = {d["metadata"]["name"] for d in docs if d["kind"] == "ServiceAccount"} + assert accounts == { + "argocd-server", + "argocd-application-controller", + "argocd-repo-server", + "argocd-redis", + } + + for kind, binding_kind in (("Role", "RoleBinding"), ("ClusterRole", "ClusterRoleBinding")): + roles = {d["metadata"]["name"] for d in docs if d["kind"] == kind} + assert {"argocd-server", "argocd-application-controller"} <= roles, f"{kind} missing" + for binding in (d for d in docs if d["kind"] == binding_kind): + assert binding["roleRef"]["kind"] == kind + assert binding["roleRef"]["name"] in roles, "binding references an unknown role" + for subject in binding["subjects"]: + assert subject["name"] in accounts, "binding references an unknown ServiceAccount" + + +def test_argocd_workloads_are_wired_together(): + docs = argocd_docs() + pod_specs = { + d["metadata"]["name"]: d["spec"]["template"]["spec"] + for d in docs + if d["kind"] in ("Deployment", "StatefulSet") + } + services = {d["metadata"]["name"]: d["spec"] for d in docs if d["kind"] == "Service"} + config_maps = {d["metadata"]["name"] for d in docs if d["kind"] == "ConfigMap"} + + for name, spec in pod_specs.items(): + assert spec["serviceAccountName"] == name, f"{name} runs under the wrong ServiceAccount" + declared = {v["name"] for v in spec.get("volumes", [])} + for container in spec["containers"]: + for mount in container.get("volumeMounts", []): + assert mount["name"] in declared, f"{name} mounts undeclared volume {mount['name']}" + for volume in spec.get("volumes", []): + if "configMap" in volume: + assert volume["configMap"]["name"] in config_maps, f"{name} mounts a missing ConfigMap" + + # Clients address redis and the repo server through the shipped Services. + for name in ("argocd-server", "argocd-application-controller", "argocd-repo-server"): + command = " ".join(pod_specs[name]["containers"][0]["command"]) + assert "argocd-redis:6379" in command, f"{name} does not point at the redis Service" + assert services["argocd-redis"]["ports"][0]["port"] == 6379 + assert services["argocd-repo-server"]["ports"][0]["port"] == 8081 + + +def test_argocd_secret_is_seeded_empty(): + # argocd-server writes server.secretkey into this Secret on first start; it + # must exist but must never carry a committed credential. + secret = next(d for d in argocd_docs() if d["kind"] == "Secret") + assert secret["metadata"]["name"] == "argocd-secret" + assert not secret.get("data") and not secret.get("stringData") + + def test_postgres_password_not_hardcoded(): # #98: the postgres superuser password must not be committed to git in # plaintext, and must not be the publicly documented kubeflex default. From 87d99c5d01bc9468fd40357d85d0df1af8fc78a2 Mon Sep 17 00:00:00 2001 From: Danathar Date: Sun, 20 Sep 2026 11:43:07 +0000 Subject: [PATCH 2/7] fix(k0s): restrict the argocd redis cache to its three clients Review on #213 rejected the reasoning in the redis comment, correctly. "ClusterIP and single-node, so redis is never reachable off-host" answers the wrong threat: the cache is unauthenticated and every pod on this node can dial a ClusterIP, including kubeflex-controller-manager, postgres, kubestellar-controller-manager, kubestellar-console and the kiosk proxy. The application controller that reads the cache holds `*/*/*` cluster-admin, so a poisoned entry is reachable from any of them. Of the two lower-machinery options the review offered, this takes the NetworkPolicy rather than a static requirepass. requirepass is not one manifest: nothing in this stack wires REDIS_PASSWORD today, so it means redis args plus env on three workloads plus a host-side generator on the generate-postgres-secret.sh pattern -- new runtime machinery in a change whose runtime outcome is already the open question. The policy is one additive object. A generated requirepass remains the stronger fix and is the maintainer's to call; the manifest comment says so. Ingress on 6379 is allowed only from pods labelled `app.kubernetes.io/name in (argocd-server, argocd-repo-server, argocd-application-controller)`. NetworkPolicy has no notion of a ServiceAccount, so this selects the labels those workloads already carry -- each is also its own Deployment/StatefulSet selector, so they cannot drift without breaking the workload first. No namespaceSelector: nothing outside argocd has any business in this cache. Two things this depends on, both recorded in the manifest rather than assumed: * Enforcement. k0s sets no `spec.network.provider` anywhere in the repo, so it defaults to kube-router with its NetworkPolicy controller on. A provider without enforcement makes this object decorative, and the answer then is requirepass, not a wider policy. * redis's readiness probe is a kubelet tcpSocket dial from the node, not from a pod, so it matches no rule here and relies on the CNI's local-node exemption. If argocd-redis stays NotReady on a first-boot VM that is the first thing to check, and the fix is an ipBlock rule for the node CIDR. test_argocd_redis_ingress_is_restricted_to_its_clients derives the expected client set from the workloads that actually name argocd-redis:6379 in their command, so a fourth client fails the test rather than failing at runtime. Still not runtime-verified -- no KubeVirt/QEMU path here, same caveat the PR body already carries, now with one more thing on the list to confirm. Assisted-by: Claude Opus 5 (1M context) via hive Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Danathar --- docs/skills/k0s-sysext-ops.md | 1 + files/k0s/manifests/argocd/install.yaml | 67 +++++++++++++++++++++++-- tests/unit/test_k0s_manifests.py | 41 +++++++++++++++ 3 files changed, 105 insertions(+), 4 deletions(-) diff --git a/docs/skills/k0s-sysext-ops.md b/docs/skills/k0s-sysext-ops.md index a08b2a44..3706c74d 100644 --- a/docs/skills/k0s-sysext-ops.md +++ b/docs/skills/k0s-sysext-ops.md @@ -52,6 +52,7 @@ k0s kubectl get pods -A - **Manifests not applied**: Check `/var/lib/k0s/manifests/`. Ensure files end in `.yaml` (not `.yml`). - **Service failed**: Check `journalctl -u k0scontroller -e`. - **`argocd-server` stuck at `0/1` with `connection refused` on `/healthz`**: it is waiting on a dependency, not the probe. Confirm `argocd-redis`, `argocd-repo-server` and `argocd-application-controller` are `Running`, then check `k0s kubectl -n argocd logs deploy/argocd-server` for RBAC `forbidden` errors — the server only binds `:8080` once its configmap/secret and Application informers have synced. +- **An Argo CD component cannot reach redis**: the `argocd-redis` NetworkPolicy allows ingress on `6379` only from pods labelled `app.kubernetes.io/name in (argocd-server, argocd-repo-server, argocd-application-controller)`. redis carries no `requirepass`, so that policy is the only thing keeping the cache private from the KubeStellar and Postgres workloads sharing this node — widen the label set rather than deleting the policy. If `argocd-redis` itself never goes `Ready`, suspect the kubelet `tcpSocket` probe: it dials from the node, not from a pod, and needs the CNI's local-node exemption. Check enforcement with `k0s kubectl -n argocd describe networkpolicy argocd-redis`. ## See also diff --git a/files/k0s/manifests/argocd/install.yaml b/files/k0s/manifests/argocd/install.yaml index 9fdee713..8dbb5f98 100644 --- a/files/k0s/manifests/argocd/install.yaml +++ b/files/k0s/manifests/argocd/install.yaml @@ -363,10 +363,17 @@ spec: selector: app.kubernetes.io/name: argocd-redis --- -# Cache only — no persistence and no requirepass. The Service is ClusterIP and -# the cluster is single-node, so redis is never reachable off-host; Argo CD -# reads REDIS_PASSWORD from the optional `argocd-redis` Secret, which this -# stack deliberately does not create. +# Cache only — no persistence and no requirepass. Argo CD reads REDIS_PASSWORD +# from the optional `argocd-redis` Secret, which this stack deliberately does +# not create; the NetworkPolicy below is what keeps the cache private instead. +# +# ClusterIP and single-node were the original argument, and they are not +# enough: they address reach from off-host, but the threat is in-cluster. +# Every pod on this node — kubeflex-controller-manager, postgres, +# kubestellar-controller-manager, kubestellar-console, kiosk-proxy — can dial +# a ClusterIP, and the application controller that reads this cache holds +# `*/*/*` cluster-admin (see its ClusterRole above), so a poisoned cache entry +# is reachable from any of them. apiVersion: apps/v1 kind: Deployment metadata: @@ -409,6 +416,58 @@ spec: initialDelaySeconds: 5 periodSeconds: 10 --- +# Restricts the unauthenticated cache to the three Argo CD components that use +# it, so a compromised workload elsewhere on this single node cannot poison +# what the cluster-admin application controller reads. +# +# Selecting on pod labels, not ServiceAccounts: NetworkPolicy has no notion of +# a ServiceAccount. These are the labels the four workloads in this file +# already carry, and each is also its Deployment/StatefulSet selector, so they +# cannot drift without breaking the workload first. +# +# This depends on the CNI enforcing NetworkPolicy. k0s runs with no +# `spec.network.provider` set (files/k0s/sysext/k0scontroller.service passes +# only `--enable-worker --single --disable-components=helm,autopilot`), so it +# defaults to kube-router with its NetworkPolicy controller on. If that ever +# changes to a provider without enforcement, this object silently becomes +# decorative -- at which point redis needs a real `requirepass`, seeded the way +# files/k0s/kubeflex/generate-postgres-secret.sh seeds the postgres password. +# +# Confirm on a first-boot VM: redis's readiness probe is a kubelet tcpSocket +# dial originating from the node, not from a pod, so it matches no rule here. +# kube-router is expected to permit local-node traffic for exactly this reason, +# but if `kubectl -n argocd get pod -l app.kubernetes.io/name=argocd-redis` +# stays NotReady after this policy lands, that exemption is the first thing to +# check -- and the fix is an added ipBlock rule for the node CIDR, not dropping +# the policy. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: argocd-redis + namespace: argocd + labels: + app.kubernetes.io/name: argocd-redis + app.kubernetes.io/part-of: argocd +spec: + podSelector: + matchLabels: + app.kubernetes.io/name: argocd-redis + policyTypes: + - Ingress + ingress: + - from: + - podSelector: + matchExpressions: + - key: app.kubernetes.io/name + operator: In + values: + - argocd-application-controller + - argocd-repo-server + - argocd-server + ports: + - protocol: TCP + port: 6379 +--- apiVersion: v1 kind: Service metadata: diff --git a/tests/unit/test_k0s_manifests.py b/tests/unit/test_k0s_manifests.py index c8e56e0f..2e2dc45b 100644 --- a/tests/unit/test_k0s_manifests.py +++ b/tests/unit/test_k0s_manifests.py @@ -118,6 +118,47 @@ def test_argocd_workloads_are_wired_together(): assert services["argocd-repo-server"]["ports"][0]["port"] == 8081 +def test_argocd_redis_ingress_is_restricted_to_its_clients(): + # redis carries no requirepass, and the application controller that reads + # the cache holds */*/* cluster-admin, so ClusterIP on a single node is not + # a boundary: every other workload on the node can dial it. The policy is + # derived from, and must stay equal to, the set of pods that actually talk + # to redis -- so adding a fourth client fails here rather than at runtime. + docs = argocd_docs() + pod_specs = { + d["metadata"]["name"]: d["spec"]["template"] + for d in docs + if d["kind"] in ("Deployment", "StatefulSet") + } + clients = { + template["metadata"]["labels"]["app.kubernetes.io/name"] + for name, template in pod_specs.items() + if "argocd-redis:6379" in " ".join(template["spec"]["containers"][0]["command"]) + } + assert clients == { + "argocd-server", + "argocd-application-controller", + "argocd-repo-server", + } + + policy = next(d for d in docs if d["kind"] == "NetworkPolicy") + assert policy["metadata"]["namespace"] == "argocd" + assert policy["spec"]["policyTypes"] == ["Ingress"] + + # It must select the redis pod itself, by the label its Deployment uses. + redis_labels = pod_specs["argocd-redis"]["metadata"]["labels"] + assert policy["spec"]["podSelector"]["matchLabels"].items() <= redis_labels.items() + + (rule,) = policy["spec"]["ingress"] + (selector,) = rule["from"] + assert "namespaceSelector" not in selector, "ingress must not be opened to other namespaces" + (expression,) = selector["podSelector"]["matchExpressions"] + assert expression["key"] == "app.kubernetes.io/name" + assert expression["operator"] == "In" + assert set(expression["values"]) == clients + assert rule["ports"] == [{"protocol": "TCP", "port": 6379}] + + def test_argocd_secret_is_seeded_empty(): # argocd-server writes server.secretkey into this Secret on first start; it # must exist but must never carry a committed credential. From 1a29f83f9ba47d1e9157ac63b18d1fdd72ff9c45 Mon Sep 17 00:00:00 2001 From: Danathar Date: Sun, 20 Sep 2026 08:04:38 -0400 Subject: [PATCH 3/7] test(k0s): tolerate containers that use args instead of command test_argocd_redis_ingress_is_restricted_to_its_clients builds the set of redis clients by scanning every Deployment and StatefulSet for "argocd-redis:6379" in its first container's command. The redis pod itself configures redis through args and has no command key, so the scan raised KeyError: 'command' and the unit job went red on 87d99c5. Read command with a default so containers without one are simply not clients. Assisted-by: Claude Opus 5 via Claude Code Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01YQCk7xJDvhtPoc4mWPzVCK Signed-off-by: Danathar --- tests/unit/test_k0s_manifests.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/unit/test_k0s_manifests.py b/tests/unit/test_k0s_manifests.py index 2e2dc45b..24c93567 100644 --- a/tests/unit/test_k0s_manifests.py +++ b/tests/unit/test_k0s_manifests.py @@ -133,7 +133,7 @@ def test_argocd_redis_ingress_is_restricted_to_its_clients(): clients = { template["metadata"]["labels"]["app.kubernetes.io/name"] for name, template in pod_specs.items() - if "argocd-redis:6379" in " ".join(template["spec"]["containers"][0]["command"]) + if "argocd-redis:6379" in " ".join(template["spec"]["containers"][0].get("command", [])) } assert clients == { "argocd-server", From d908bc7bea2658bd336ce9eb573b1cf080762e74 Mon Sep 17 00:00:00 2001 From: Danathar Date: Sun, 20 Sep 2026 12:33:23 +0000 Subject: [PATCH 4/7] fix(k0s): drop the status subresource from the Application CRDs Review on #213 found this, and it is the more serious of the two: with `subresources: status: {}` on applications.argoproj.io and appprojects.argoproj.io, argocd-server still reaches ready -- so #177's symptom looks fixed -- while the stack is broken for the thing it exists to do. The application controller persists health, sync and operationState by updating and patching the Application itself, not its /status endpoint. Declaring the subresource makes the API server silently strip .status from those writes, so no Application ever records sync or health and operation handling stalls. The shipped RBAC grants applications and appprojects but no applications/status, so a /status write would be forbidden even if the controller attempted one. Upstream v2.12.3 agrees, which is the reason to trust this over the surface plausibility of "a CRD with a status field should have a status subresource": manifests/crds/application-crd.yaml:4999 ships `subresources: {}`, appproject-crd.yaml declares none at all, and only applicationset-crd.yaml carries `status: {}`. Match that exactly. The .status property stays in both trimmed schemas -- the field is still stored and served, it is only the subresource that moves the write path. test_argocd_application_crds_have_no_status_subresource pins all three CRDs, including the ApplicationSet one that should keep its subresource, so neither direction can drift back unnoticed. Assisted-by: Claude Opus 5 (1M context) via hive Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Danathar --- files/k0s/manifests/argocd/install.yaml | 13 +++++++++---- tests/unit/test_k0s_manifests.py | 21 +++++++++++++++++++++ 2 files changed, 30 insertions(+), 4 deletions(-) diff --git a/files/k0s/manifests/argocd/install.yaml b/files/k0s/manifests/argocd/install.yaml index 8dbb5f98..fd3f0ae6 100644 --- a/files/k0s/manifests/argocd/install.yaml +++ b/files/k0s/manifests/argocd/install.yaml @@ -13,6 +13,15 @@ # the sysext payload stays small; the API server still stores and serves the full # resource. The ApplicationSet CRD is registered for API completeness — the # applicationset controller is not part of this stack. +# +# Application and AppProject deliberately declare no status subresource, which +# matches upstream (`subresources: {}` on the Application CRD, none at all on +# AppProject; only ApplicationSet has one). The application controller persists +# health, sync and operationState by updating/patching the Application itself, +# not its /status endpoint — enabling the subresource would make the API server +# silently drop `.status` from those writes, and the RBAC below grants no +# applications/status anyway. Adding it here reports as "more correct" and +# quietly breaks every Application the stack is meant to run. apiVersion: v1 kind: Namespace metadata: @@ -39,8 +48,6 @@ spec: - name: v1alpha1 served: true storage: true - subresources: - status: {} schema: openAPIV3Schema: type: object @@ -76,8 +83,6 @@ spec: - name: v1alpha1 served: true storage: true - subresources: - status: {} schema: openAPIV3Schema: type: object diff --git a/tests/unit/test_k0s_manifests.py b/tests/unit/test_k0s_manifests.py index 24c93567..febea11b 100644 --- a/tests/unit/test_k0s_manifests.py +++ b/tests/unit/test_k0s_manifests.py @@ -68,6 +68,27 @@ def test_argocd_stack_is_complete(): } +def test_argocd_application_crds_have_no_status_subresource(): + # The application controller writes health/sync/operationState by + # updating the Application itself, not its /status endpoint. A status + # subresource on Application or AppProject makes the API server strip + # `.status` from those writes, so Applications never record sync state -- + # argocd-server still goes ready, so #177 looks fixed while the stack is + # useless. Upstream v2.12 ships `subresources: {}` on Application and none + # on AppProject; only ApplicationSet has one. + versions = { + d["metadata"]["name"]: d["spec"]["versions"][0] + for d in argocd_docs() + if d["kind"] == "CustomResourceDefinition" + } + for name in ("applications.argoproj.io", "appprojects.argoproj.io"): + assert not versions[name].get("subresources"), ( + f"{name} must not enable a status subresource: the API server would " + "silently drop status from the controller's writes" + ) + assert versions["applicationsets.argoproj.io"]["subresources"] == {"status": {}} + + def test_argocd_rbac_is_bound(): # Without namespace RBAC the server's configmap/secret and Application # informers never sync, which is the state issue #177 reported. From c17eb3e0b94c7edd6f1cbad7c6c883dd43d0d5a2 Mon Sep 17 00:00:00 2001 From: Danathar Date: Sun, 20 Sep 2026 13:31:14 +0000 Subject: [PATCH 5/7] fix(k0s): ship the controller's governing Service Review on #213: the argocd-application-controller StatefulSet declares `serviceName: argocd-application-controller`, but no Service of that name was in the stack -- only argocd-redis, argocd-repo-server and argocd-server. Kubernetes does not reject the dangling reference. The pods schedule and run; only the per-pod DNS records the governing Service is supposed to back silently fail to resolve, so nothing surfaces at apply time or in `kubectl get pods`. Upstream v2.12 ships this Service for the controller's metrics port, so add it rather than dropping the serviceName: it makes the reference resolve, and it gives Prometheus something to scrape on 8082 the way the repo-server's own Service already exposes its metrics port. test_argocd_statefulset_governing_services_exist walks every StatefulSet in the manifest, requires spec.serviceName to name a shipped Service, and requires that Service's selector to actually match the pod template's labels -- a Service that exists but selects nothing is the same dangling reference wearing a disguise. It also pins the controller Service's port to 8082/metrics against the container's own port name, so the two cannot drift apart. Verified by re-implementing the module's argocd assertions against a js-yaml parse of the manifest (63 assertions, all passing), since this environment still has neither PyYAML nor pytest. The new assertion was mutation-tested: it fails on the previous head, where the Service is absent, and passes here. Assisted-by: Claude Opus 5 (1M context) via hive Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Danathar --- files/k0s/manifests/argocd/install.yaml | 17 +++++++++++++++ tests/unit/test_k0s_manifests.py | 28 +++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/files/k0s/manifests/argocd/install.yaml b/files/k0s/manifests/argocd/install.yaml index fd3f0ae6..22e1e3b1 100644 --- a/files/k0s/manifests/argocd/install.yaml +++ b/files/k0s/manifests/argocd/install.yaml @@ -547,6 +547,23 @@ spec: - name: helm-working-dir emptyDir: {} --- +apiVersion: v1 +kind: Service +metadata: + name: argocd-application-controller + namespace: argocd + labels: + app.kubernetes.io/name: argocd-application-controller + app.kubernetes.io/part-of: argocd +spec: + type: ClusterIP + ports: + - port: 8082 + targetPort: metrics + name: metrics + selector: + app.kubernetes.io/name: argocd-application-controller +--- apiVersion: apps/v1 kind: StatefulSet metadata: diff --git a/tests/unit/test_k0s_manifests.py b/tests/unit/test_k0s_manifests.py index febea11b..a26abb19 100644 --- a/tests/unit/test_k0s_manifests.py +++ b/tests/unit/test_k0s_manifests.py @@ -139,6 +139,34 @@ def test_argocd_workloads_are_wired_together(): assert services["argocd-repo-server"]["ports"][0]["port"] == 8081 +def test_argocd_statefulset_governing_services_exist(): + # A StatefulSet's spec.serviceName names its governing headless Service. + # Kubernetes does not reject a dangling reference -- the pods run and only + # their per-pod DNS records silently fail to resolve -- so nothing catches + # this at apply time. Upstream v2.12 ships the matching + # argocd-application-controller Service for the controller's metrics port. + docs = argocd_docs() + services = {d["metadata"]["name"]: d["spec"] for d in docs if d["kind"] == "Service"} + for statefulset in (d for d in docs if d["kind"] == "StatefulSet"): + name = statefulset["metadata"]["name"] + governing = statefulset["spec"]["serviceName"] + assert governing in services, ( + f"{name} declares serviceName {governing}, but no such Service is shipped" + ) + labels = statefulset["spec"]["template"]["metadata"]["labels"] + assert services[governing]["selector"].items() <= labels.items(), ( + f"{governing} does not select {name}'s pods" + ) + + # The controller exposes only metrics; the port must match its container. + (port,) = services["argocd-application-controller"]["ports"] + container = next( + d for d in docs if d["kind"] == "StatefulSet" + )["spec"]["template"]["spec"]["containers"][0] + assert port["port"] == 8082 + assert port["targetPort"] == container["ports"][0]["name"] == "metrics" + + def test_argocd_redis_ingress_is_restricted_to_its_clients(): # redis carries no requirepass, and the application controller that reads # the cache holds */*/* cluster-admin, so ClusterIP on a single node is not From 09057b4f1f5e3a324daf0e0486244038248f403f Mon Sep 17 00:00:00 2001 From: Danathar Date: Sun, 20 Sep 2026 23:08:01 +0000 Subject: [PATCH 6/7] fix(k0s): correct the false upstream-Service claim; document two open concerns Review on c17eb3e caught a docs-currency error I authored and repeated in three places: the comment justifying the new argocd-application-controller Service, the PR body's Workloads bullet, and by implication anywhere else it was cited. Upstream v2.12.3 ships no Service by that name at all (its metrics live on a separate argocd-metrics Service, and its own serviceName dangles the same way); this repo's Service is new, not a match, and it is ClusterIP rather than headless, so it does not provide the per-pod DNS the old wording implied. Corrected the test comment and the PR body. Documented, rather than silently leaving open, the two correctness notes: - argocd-secret's C+ tmpfiles reseed on every boot is safe because k0s's Manifest Deployer applies the same way kubectl apply does, a three-way merge against the last-applied configuration, which never included the data field for this Secret, so the runtime-generated secretkey and admin hash are not in the diff and survive the reapply (docs.k0sproject.io manifests page: quite similar to kubectl apply). - The repo-server's omission of upstream's gpg-keys/gpg-keyring emptyDir mounts is intentional: this stack ships no GPG commit-verification config, and the image already chowns /app/config/gpg/keys. Commented so it reads as a decision instead of an oversight, matching how the other two intentional deviations in this file are commented. Both HUMAN DECISION NEEDED items from prior reviews are unchanged: passwordless redis behind the NetworkPolicy, and first-boot VM confirmation. Neither is addressable from this environment. Signed-off-by: Danathar --- files/k0s/manifests/argocd/install.yaml | 12 +++++++++++- tests/unit/test_k0s_manifests.py | 8 ++++++-- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/files/k0s/manifests/argocd/install.yaml b/files/k0s/manifests/argocd/install.yaml index 22e1e3b1..9e3855fc 100644 --- a/files/k0s/manifests/argocd/install.yaml +++ b/files/k0s/manifests/argocd/install.yaml @@ -195,7 +195,12 @@ metadata: app.kubernetes.io/part-of: argocd --- # Seeded empty on purpose: argocd-server generates `server.secretkey` and the -# initial admin bcrypt hash into this Secret on first start. +# initial admin bcrypt hash into this Secret on first start. The `C+` tmpfiles +# rule (files/k0s/sysext/k0s-manifests.conf) re-copies this manifest file into +# /var/lib/k0s/manifests on every boot, and k0s's Manifest Deployer applies it +# the same way `kubectl apply` does -- a three-way merge against the last +# applied configuration, which never included `data`, so the generated +# secretkey and admin hash are not in the diff and survive the reapply. apiVersion: v1 kind: Secret metadata: @@ -511,6 +516,11 @@ spec: labels: app.kubernetes.io/name: argocd-repo-server spec: + # Deliberately omits upstream's gpg-keys/gpg-keyring emptyDir mounts at + # /app/config/gpg/{keys,source}: this stack ships no GPG commit-verification + # config, and the quay.io/argoproj/argocd:v2.12.3 image already chowns + # /app/config/gpg/keys, so the repo-server starts clean without them. Add + # both mounts back if GPG signature verification is ever enabled here. serviceAccountName: argocd-repo-server containers: - name: argocd-repo-server diff --git a/tests/unit/test_k0s_manifests.py b/tests/unit/test_k0s_manifests.py index a26abb19..71900a87 100644 --- a/tests/unit/test_k0s_manifests.py +++ b/tests/unit/test_k0s_manifests.py @@ -143,8 +143,12 @@ def test_argocd_statefulset_governing_services_exist(): # A StatefulSet's spec.serviceName names its governing headless Service. # Kubernetes does not reject a dangling reference -- the pods run and only # their per-pod DNS records silently fail to resolve -- so nothing catches - # this at apply time. Upstream v2.12 ships the matching - # argocd-application-controller Service for the controller's metrics port. + # this at apply time. Upstream v2.12 ships no Service by this name at all + # (the serviceName dangles there too; the controller's metrics live on a + # separate argocd-metrics Service). This repo ships its own ClusterIP + # Service so the reference resolves and Prometheus has something to scrape + # on the controller's own metrics port -- it does not, and does not claim + # to, provide per-pod DNS, which would require a headless Service instead. docs = argocd_docs() services = {d["metadata"]["name"]: d["spec"] for d in docs if d["kind"] == "Service"} for statefulset in (d for d in docs if d["kind"] == "StatefulSet"): From 6bc9077601d1dbc488085eca67d95343a9b8d774 Mon Sep 17 00:00:00 2001 From: Danathar Date: Sun, 20 Sep 2026 23:15:31 +0000 Subject: [PATCH 7/7] fix(k0s): stop asserting the secret-reseed safety as observed fact The prior comment stated the three-way-merge preservation of argocd-secret's data as established behaviour. It is inferred from one sentence in k0s's own docs (Manifest Deployer described as 'quite similar to kubectl apply'), not from the applier's source or a real boot -- the same gap the review named. Reworded to state the basis, the expected behaviour, and that it is unconfirmed and belongs on the same first-boot VM checklist as the redis NetworkPolicy comment already asks for, plus what a wrong guess would mean (a C, not C+, tmpfiles rule). Signed-off-by: Danathar --- files/k0s/manifests/argocd/install.yaml | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/files/k0s/manifests/argocd/install.yaml b/files/k0s/manifests/argocd/install.yaml index 9e3855fc..80551d95 100644 --- a/files/k0s/manifests/argocd/install.yaml +++ b/files/k0s/manifests/argocd/install.yaml @@ -197,10 +197,15 @@ metadata: # Seeded empty on purpose: argocd-server generates `server.secretkey` and the # initial admin bcrypt hash into this Secret on first start. The `C+` tmpfiles # rule (files/k0s/sysext/k0s-manifests.conf) re-copies this manifest file into -# /var/lib/k0s/manifests on every boot, and k0s's Manifest Deployer applies it -# the same way `kubectl apply` does -- a three-way merge against the last -# applied configuration, which never included `data`, so the generated -# secretkey and admin hash are not in the diff and survive the reapply. +# /var/lib/k0s/manifests on every boot. k0s's own docs describe the Manifest +# Deployer as "quite similar to the use of the kubectl apply command", which +# would mean a three-way merge against the last-applied configuration -- and +# since that configuration never included `data`, the generated secretkey and +# admin hash would not be in the diff and would survive the reapply. Expected +# behaviour, not observed: confirm on the same first-boot VM pass this stack +# still needs (see the redis NetworkPolicy comment below). If a boot ever +# does wipe `server.secretkey`, that behaviour was wrong and this Secret needs +# a `C` (create-only) tmpfiles rule instead of `C+`. apiVersion: v1 kind: Secret metadata: