Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/skills/k0s-sysext-ops.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,27 @@ k0s automatically applies all `.yaml` files under `/var/lib/k0s/manifests/argocd
k0s kubectl get pods -A
```

## KubeStellar Kiosk TLS Management

The KubeStellar kiosk proxy TLS certificate (`cert.pem`) and private key (`key.pem`) are generated on first boot by `k0s-kiosk-tls.service`, which runs `/usr/share/k0s/kiosk-tls/generate-kiosk-tls.sh` (source: `files/k0s/kiosk-tls/generate-kiosk-tls.sh`) before `k0scontroller.service` starts. The private key is stored persistently in `/var/lib/k0s/kiosk/key.pem` with mode `0600` (the script runs under `umask 077`, so the key is never world-readable, even transiently), and the certificate is configured with the host's actual IP addresses in the Subject Alternative Names (SAN). `files/k0s/sysext/k0s-manifests.conf` seeds the kiosk static assets (`nginx.conf`, `kiosk-gate.js`, `kiosk-gate.css`) into this same directory on every boot, file-by-file rather than as a whole-directory copy, specifically so it never deletes `cert.pem`/`key.pem`.

**Compromised build-time material is replaced automatically.** Sysext releases prior to `k0s-kiosk-tls.service` generated the certificate and key at image build time and shipped them inside the publicly-published release image, so those private keys are public knowledge. Hosts upgraded from such a release still carry the compromised pair in `/var/lib/k0s/kiosk`. The generator detects it — the build-time certificate carries the fixed SAN `DNS:localhost,DNS:*.local,IP:127.0.0.1,IP:10.0.2.15` and lacks the `OU=host-generated` subject marker the script adds — and regenerates the pair on the next boot (or on `systemctl restart k0s-kiosk-tls.service`). Host-generated and operator-provided certificates are never touched.

The SAN is computed once, at generation time, from whatever IP addresses the host has at that moment (3650-day validity, no periodic refresh). On a DHCP host whose address later changes, the certificate will not include the new address — rotate manually (below) after an address change if browser cert warnings start appearing.

To rotate or regenerate the TLS certificate and private key:

```bash
# Remove the existing certificate and key from persistent storage
rm -f /var/lib/k0s/kiosk/key.pem /var/lib/k0s/kiosk/cert.pem

# Trigger regeneration via the oneshot unit
systemctl restart k0s-kiosk-tls.service

# Restart the controller service to reload kiosk assets
systemctl restart k0scontroller.service
```

## Troubleshooting

- **Extension not merged**: Check `systemd-sysext status`. Verify the persistent image is `/var/lib/k0s/k0s.raw`; the boot activation unit copies it into `/run/extensions/k0s.raw`.
Expand Down
2 changes: 2 additions & 0 deletions docs/skills/k0s-sysext.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ Design choices:
| `elements/k0s/k0s-bin.bst` | Pins the upstream `k0s` binary SHA256; the release URL is derived from `include/k0s.yml`. |
| `elements/oci/k0s-sysext.bst` | Builds the EROFS sysext image (`k0s-<k0s-version>.raw`). |
| `files/k0s/sysext/k0scontroller.service` | systemd unit for the k0s single-node controller/worker. Not enabled by default. |
| `files/k0s/sysext/k0s-kiosk-tls.service` | Oneshot systemd unit generating persistent kiosk TLS cert and key on first boot. |
| `files/k0s/kiosk-tls/generate-kiosk-tls.sh` | Generator run by `k0s-kiosk-tls.service`; also replaces the compromised build-time cert/key pair shipped by earlier releases. |
| `files/k0s/sysext/extension-release.k0s` | Static sysext identity (`ID=_any`); `VERSION_ID=`/`ARCHITECTURE=` are appended at build time. |
| `files/k0s/sysext/k0s-manifests.conf` | tmpfiles rule that copies declarative stacks to `/var/lib/k0s/manifests/`. |
| `files/k0s/manifests/argocd/` | Raw YAML manifests for Argo CD. |
Expand Down
3 changes: 3 additions & 0 deletions elements/bluefin-server/os-stack.bst
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ depends:
# gpg binary required by systemd-sysupdate to verify SHA256SUMS.gpg
- freedesktop-sdk.bst:components/gnupg.bst

# openssl CLI required for first-boot TLS certificate generation
- freedesktop-sdk.bst:components/openssl.bst

# First-Boot Credentials Provisioning
- bluefin-server/os-creds-prov.bst

Expand Down
21 changes: 10 additions & 11 deletions elements/oci/k0s-sysext.bst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ description: |
Produce a systemd-sysext extension image for k0s.

Output: k0s-<k0s-version>.raw, k0s-<k0s-version>.raw.zst, and SHA256SUMS.
Contains /usr/bin/k0s, k0scontroller.service, tmpfiles.d/k0s-manifests.conf,
and /usr/share/k0s/manifests/ (Argo CD and KubeStellar).
Contains /usr/bin/k0s, k0scontroller.service, k0s-kiosk-tls.service,
tmpfiles.d/k0s-manifests.conf, and /usr/share/k0s/manifests/ (Argo CD and KubeStellar).

(@):
- include/arch.yml
Expand All @@ -15,7 +15,6 @@ build-depends:
- freedesktop-sdk.bst:components/erofs-utils.bst
- freedesktop-sdk.bst:components/xz.bst
- freedesktop-sdk.bst:components/zstd.bst
- freedesktop-sdk.bst:components/openssl.bst
- filename: k0s/k0s-bin.bst
config:
location: /
Expand All @@ -33,6 +32,9 @@ sources:
- kind: local
path: files/k0s/kubeflex
directory: kubeflex-src
- kind: local
path: files/k0s/kiosk-tls
directory: kiosk-tls-src
- kind: local
path: files/os/issue.d/40-kubestellar.issue
directory: issue-src
Expand Down Expand Up @@ -67,6 +69,7 @@ config:

# Stage systemd service
cp -a sysext-src/k0scontroller.service sysext/usr/lib/systemd/system/
cp -a sysext-src/k0s-kiosk-tls.service sysext/usr/lib/systemd/system/

# Stage extension release metadata
cp -a sysext-src/extension-release.k0s sysext/usr/lib/extension-release.d/
Expand All @@ -90,14 +93,10 @@ config:
cp -a kubeflex-src/. sysext/usr/share/k0s/kubeflex/
chmod 0755 sysext/usr/share/k0s/kubeflex/generate-postgres-secret.sh

# Generate self-signed TLS certificate for local kiosk proxy
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout sysext/usr/share/k0s/kiosk/key.pem \
-out sysext/usr/share/k0s/kiosk/cert.pem \
-subj "/CN=KubeStellar Console" \
-addext "subjectAltName=DNS:localhost,DNS:*.local,IP:127.0.0.1,IP:10.0.2.15"
chmod 0600 sysext/usr/share/k0s/kiosk/key.pem
chmod 0644 sysext/usr/share/k0s/kiosk/cert.pem
# Stage kiosk first-boot TLS generator (run by k0s-kiosk-tls.service).
mkdir -p sysext/usr/share/k0s/kiosk-tls
cp -a kiosk-tls-src/. sysext/usr/share/k0s/kiosk-tls/
chmod 0755 sysext/usr/share/k0s/kiosk-tls/generate-kiosk-tls.sh

# Build EROFS image
mkfs.erofs -d0 "${OUT}/${FNAME}" sysext
Expand Down
64 changes: 64 additions & 0 deletions files/k0s/kiosk-tls/generate-kiosk-tls.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/bin/bash
# First-boot generator for the KubeStellar kiosk TLS certificate and key.
#
# Run by k0s-kiosk-tls.service before k0scontroller.service. Writes key.pem
# (0600) and cert.pem (0644) into /var/lib/k0s/kiosk with the host's DNS name
# and IP addresses in the SAN. Idempotent: host-generated material is kept
# across boots so the certificate browsers have trusted stays stable.
#
# Earlier sysext releases generated cert.pem/key.pem at build time and shipped
# them inside the public release image, so those private keys are public
# knowledge. Build-time certificates carry exactly the SAN below (10.0.2.15 is
# the QEMU user-net guest address that was hardcoded at build time) and lack
# the OU=host-generated subject marker this script adds. When one is found on
# disk (seeded by the old directory-level tmpfiles rule on upgraded hosts), it
# is treated as compromised and regenerated in place. The marker keeps a QEMU
# user-net host whose own SAN legitimately matches from regenerating on every
# boot.
set -euo pipefail
umask 077

kiosk_dir="${KIOSK_TLS_DIR:-/var/lib/k0s/kiosk}"
key="$kiosk_dir/key.pem"
cert="$kiosk_dir/cert.pem"

leaked_build_san="DNS:localhost,DNS:*.local,IPAddress:127.0.0.1,IPAddress:10.0.2.15"

if [ -s "$key" ] && [ -s "$cert" ]; then
subject="$(openssl x509 -in "$cert" -noout -subject 2>/dev/null || true)"
existing_san="$(openssl x509 -in "$cert" -noout -ext subjectAltName 2>/dev/null | sed 1d | tr -d '[:space:]' || true)"
if [[ "$subject" == *host-generated* ]] || [ "$existing_san" != "$leaked_build_san" ]; then
exit 0
fi
echo "k0s-kiosk-tls: replacing compromised build-time certificate (private key was published in the release image)" >&2
rm -f "$key" "$cert"
fi

mkdir -p "$kiosk_dir"
# tmpfiles.d owns this dir as 0755; keep that under our restrictive umask so
# the kiosk proxy can still read the static assets seeded next to the certs.
chmod 0755 "$kiosk_dir"

SAN="DNS:localhost,DNS:*.local,IP:127.0.0.1"
if command -v hostname >/dev/null 2>&1; then
HN="$(hostname 2>/dev/null || true)"
if [ -n "$HN" ] && [ "$HN" != "localhost" ]; then
SAN="${SAN},DNS:${HN}"
fi
else
echo "k0s-kiosk-tls: hostname command not found, SAN has no host DNS name" >&2
fi
if command -v ip >/dev/null 2>&1; then
for addr in $(ip -o addr show scope global 2>/dev/null | tr -s ' ' | cut -d ' ' -f4 | cut -d/ -f1); do
SAN="${SAN},IP:${addr}"
done
else
echo "k0s-kiosk-tls: ip command not found, SAN has no host IP address" >&2
fi

openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout "$key" -out "$cert" \
-subj "/CN=KubeStellar Console/OU=host-generated" \
-addext "subjectAltName=${SAN}"
chmod 0600 "$key"
chmod 0644 "$cert"
11 changes: 11 additions & 0 deletions files/k0s/sysext/k0s-kiosk-tls.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[Unit]
Description=Generate persistent TLS certificate and key for KubeStellar kiosk
RequiresMountsFor=/var/lib/k0s
After=network-online.target systemd-tmpfiles-setup.service
Before=k0scontroller.service

[Service]
Type=oneshot
RemainAfterExit=yes
StateDirectory=k0s
ExecStart=/usr/share/k0s/kiosk-tls/generate-kiosk-tls.sh
12 changes: 11 additions & 1 deletion files/k0s/sysext/k0s-manifests.conf
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,14 @@
d /var/lib/k0s/manifests 0755 root root - -
C+ /var/lib/k0s/manifests/argocd - - - - /usr/share/k0s/manifests/argocd
C+ /var/lib/k0s/manifests/kubestellar - - - - /usr/share/k0s/manifests/kubestellar
C+ /var/lib/k0s/kiosk - - - - /usr/share/k0s/kiosk
# Per-file, not whole-directory: k0s-kiosk-tls.service also writes
# cert.pem/key.pem into this directory, and they must survive every boot.
# A directory-level `C+` here would delete-then-recopy the whole tree from
# /usr/share/k0s/kiosk (which never contains cert.pem/key.pem), wiping the
# generated TLS material on every boot. Targeting each static asset file
# individually still propagates sysext upgrades to nginx.conf/kiosk-gate.*
# without touching anything else in the directory.
d /var/lib/k0s/kiosk 0755 root root - -
C+ /var/lib/k0s/kiosk/nginx.conf - - - - /usr/share/k0s/kiosk/nginx.conf
C+ /var/lib/k0s/kiosk/kiosk-gate.js - - - - /usr/share/k0s/kiosk/kiosk-gate.js
C+ /var/lib/k0s/kiosk/kiosk-gate.css - - - - /usr/share/k0s/kiosk/kiosk-gate.css
4 changes: 2 additions & 2 deletions files/k0s/sysext/k0scontroller.service
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
Description=k0s - Zero Friction Kubernetes
Documentation=https://docs.k0sproject.io
ConditionFileIsExecutable=/usr/bin/k0s
After=network-online.target
Wants=network-online.target
After=network-online.target k0s-kiosk-tls.service
Wants=network-online.target k0s-kiosk-tls.service

[Service]
Type=simple
Expand Down
Loading
Loading