Repository navigation
Conversation
hanthor
left a comment
There was a problem hiding this comment.
Good fix. Two things to confirm before it lands, and one much larger issue in the same file.
Deleting 01-kubestellar-console-github-oauth.yaml is clearly right — jwt-secret: default-jwt-secret-123456789012345678901234 was a fixed signing key auto-applied on every default install, and shipping it in the sysext meant every Bluefin Server signed tokens with the same value. Removing it is the correct call.
I checked the obvious way this could break and it does not: deleting that file does not orphan the namespace, because 40-kubestellar-console.yaml declares it itself.
$ git show pr90:files/k0s/manifests/kubestellar/40-kubestellar-console.yaml | head -4
apiVersion: v1
kind: Namespace
metadata:
name: kubestellar-console
So the Service and Deployment still have a namespace to land in. Good.
1. The central safety claim is not verifiable from this repo
The body states:
When unset, the console container automatically generates an ephemeral random 32-byte secret at startup (
console/start.sh).
console/start.sh is not in this repository:
$ git ls-tree -r --name-only pr90 | grep -i 'start.sh'
(no output)
It lives inside ghcr.io/kubestellar/console:v0.3.34. The whole security argument for optional: true rests on that behaviour. If the image instead falls back to an empty or compiled-in default signing key when JWT_SECRET is unset, this change makes forgery easier than the status quo, because the bad-but-known secret is replaced by a bad-and-silent one. Please link the upstream start.sh line, or pin the behaviour with a comment in the manifest, so a future image bump can't quietly invalidate it.
2. The console image is on a mutable tag
image: ghcr.io/kubestellar/console:v0.3.34No digest. This is the same class of issue #81 is fixing for argocd, and the kiosk proxy in the sibling file is already pinned (nginx@sha256:62223d644fa234c3a1cc785ee14242ec47a77364226f1c811d2f669f96dc2ac8). Since this PR is already editing this file for supply-chain reasons and the behaviour in item 1 is image-dependent, pinning by digest here would be cheap and would make the claim durable.
3. The bigger hole is three lines above the one you changed
Unchanged by this PR, but in the same container spec:
env:
- name: DEV_MODE
value: "true"
- name: ALLOW_DEV_MODE_IN_CLUSTER
value: "true"
- name: DEV_USER_LOGIN
value: "admin"To be explicit about baseline: this is pre-existing on main, not introduced here. But it substantially undercuts what this PR achieves. Hardening the JWT signing key matters only if authentication is enforced; with dev mode explicitly enabled in-cluster and a hardcoded admin login, anyone who reaches the console is already an administrator, regardless of how the JWT is signed. Worth a follow-up issue — and it's arguably the finding that should be at the top of #72's queue.
CI status
No checks have run on this PR at all:
mcp__github__pull_request_read(method="get_check_runs", pullNumber=90)
-> {"total_count":0,"check_runs":[]}
mergeable_state is unstable; it's a fork PR (Danathar/server) and looks like it needs maintainer approval for workflows to run. I ran the gates locally on main + this branch: 201 passed, 1 xfailed, 41/41 bats, all three .github/scripts checkers exit 0 — identical to the main baseline at 7f376fa. Nothing regressed. The two new assertions are cheap and correctly aimed.
Merge-order note
This and #91 both edit docs/skills/k0s-sysext-ops.md and tests/unit/test_kubestellar_kiosk.py. Each merges cleanly into main alone, but not into each other:
$ git merge pr90 # onto main+pr91
CONFLICT (content): Merge conflict in docs/skills/k0s-sysext-ops.md
Whichever goes second needs a rebase.
Marking this as a comment rather than a block — items 1 and 2 are worth resolving first, but the change itself is an improvement over what's on main.
Generated by Claude Code
hanthor's review on projectbluefin#90 confirmed the optional: true change is correct but flagged two things to resolve first: 1. The safety argument ("the console generates an ephemeral random secret when JWT_SECRET is unset") rests on upstream behaviour this repo can't see. Fetched kubestellar/console's start.sh at the exact pinned tag (v0.3.34) and confirmed it directly: when JWT_SECRET is unset, it generates `openssl rand -hex 32` (falling back to /dev/urandom, never an empty or compiled-in default), persists it to $INSTALL_DIR/data/.jwt_secret so it survives a plain container restart, and exports it. Since this Deployment mounts no volume for that path, it's lost on pod recreation -- so "ephemeral" is accurate at the pod-lifetime granularity, just not on every single restart. Added a comment citing the exact upstream lines so a future image bump that changes this can't invalidate the claim silently. 2. The image was on a mutable tag while the sibling kiosk-proxy nginx image in the same manifest set is already digest-pinned. Resolved ghcr.io/kubestellar/console:v0.3.34's current digest via skopeo (confirmed it's an OCI image index, same multi-arch-safe shape as a tag reference) and pinned it. Left item 3 (DEV_MODE/admin login already on main) alone, per the review's own framing -- pre-existing, out of scope, worth a follow-up issue rather than scope creep here. Verified: all 8 relevant unit tests pass, full suite 201/201 + 1 xfailed, YAML parses clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNgJ8VMdoqzGequbxsP2Kc
|
@hanthor — pushed in
Left item 3 alone per your own framing — pre-existing on Verified: all 8 relevant unit tests pass, full suite 201/201 + 1 xfailed. On the merge-order note with #91: pushed both independently; whichever lands second will need the small rebase on |
The branch could not be refreshed onto current |
…ed manifest Delete files/k0s/manifests/kubestellar/01-kubestellar-console-github-oauth.yaml so no hardcoded JWT secrets or dummy OAuth credentials are automatically seeded into /var/lib/k0s/manifests/ or applied by the k0s manifest deployer. Make JWT_SECRET secretKeyRef in 40-kubestellar-console.yaml optional so the console starts cleanly in local development mode without requiring a pre-seeded Secret manifest (in-container console startup generates an ephemeral secret when unset). Update unit test contracts in test_k0s_manifests.py and test_kubestellar_kiosk.py to assert the secret manifest is not present and all secret references are optional. Document operator-provided OAuth Secret configuration in docs/skills/k0s-sysext-ops.md. Closes projectbluefin#72 Signed-off-by: Danathar <Danathar@users.noreply.github.com>
hanthor's review on projectbluefin#90 confirmed the optional: true change is correct but flagged two things to resolve first: 1. The safety argument ("the console generates an ephemeral random secret when JWT_SECRET is unset") rests on upstream behaviour this repo can't see. Fetched kubestellar/console's start.sh at the exact pinned tag (v0.3.34) and confirmed it directly: when JWT_SECRET is unset, it generates `openssl rand -hex 32` (falling back to /dev/urandom, never an empty or compiled-in default), persists it to $INSTALL_DIR/data/.jwt_secret so it survives a plain container restart, and exports it. Since this Deployment mounts no volume for that path, it's lost on pod recreation -- so "ephemeral" is accurate at the pod-lifetime granularity, just not on every single restart. Added a comment citing the exact upstream lines so a future image bump that changes this can't invalidate the claim silently. 2. The image was on a mutable tag while the sibling kiosk-proxy nginx image in the same manifest set is already digest-pinned. Resolved ghcr.io/kubestellar/console:v0.3.34's current digest via skopeo (confirmed it's an OCI image index, same multi-arch-safe shape as a tag reference) and pinned it. Left item 3 (DEV_MODE/admin login already on main) alone, per the review's own framing -- pre-existing, out of scope, worth a follow-up issue rather than scope creep here. Verified: all 8 relevant unit tests pass, full suite 201/201 + 1 xfailed, YAML parses clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LNgJ8VMdoqzGequbxsP2Kc
2de9d42 to
63ded6d
Compare
Update: the second refresh onto |
castrojo
left a comment
There was a problem hiding this comment.
Approving. The secret is genuinely removed rather than relocated, and both of @hanthor's asks are closed.
First, the thing that was actually blocking this PR was not review — it was that CI had never run. This is a fork PR from a contributor with read access, and build.yml triggers on plain pull_request while checking out the PR head, so its runs sat at conclusion: action_required with zero jobs dispatched. gh pr checks showed literally nothing, which reads identically to "not yet pushed". I have approved the workflow runs; unit and docs now pass and build is in progress. Worth knowing for future fork PRs from the same contributor.
Verified: it removes the secret, it does not move it. A repo-wide search for default-jwt-secret returns exactly one hit, and it is files/k0s/manifests/kubestellar/01-kubestellar-console-github-oauth.yaml — the file this PR deletes. No copy of the value appears anywhere else, and the diff introduces none. The replacement documentation in docs/skills/k0s-sysext-ops.md uses only placeholders (<github-client-id>, <github-client-secret>, <random-32-byte-or-longer-secret>) and tells operators to create the Secret themselves at runtime, so nothing is carried over into docs. A search for references to the deleted filename returns zero, so nothing else in the tree points at the removed manifest, and deleting it does not orphan the namespace — 40-kubestellar-console.yaml declares kind: Namespace / name: kubestellar-console itself.
That matters because the file was auto-applied by the k0s manifest deployer on every default install, so this was a fixed, published signing key on every machine, not a sample.
Both review asks are addressed. The optional: true safety argument now carries an inline citation to the upstream source it rests on (console/start.sh at v0.3.34, showing openssl rand -hex 32 persisted to $INSTALL_DIR/data/.jwt_secret, never an empty or compiled-in default), and the image is pinned by digest rather than a mutable tag. The digest pin matches the existing nginx@sha256:... pattern in the sibling kiosk-proxy manifest, so it is consistent with how this repo already does it.
Two things for the record, neither blocking.
assert "optional: false" not in console in tests/unit/test_kubestellar_kiosk.py is a whole-file negative assertion. It passes today, but it will fire spuriously the first time any future secret reference in that manifest legitimately needs to be mandatory. Scoping it to the JWT_SECRET block would make it durable. Not worth holding the PR for.
More importantly, and explicitly not this PR's fault: three lines above the changed line, untouched here and still on main, sit DEV_MODE: "true", ALLOW_DEV_MODE_IN_CLUSTER: "true" and DEV_USER_LOGIN: "admin". Hardening the JWT signing key is worth doing, but its value is capped while anyone who reaches the console arrives as admin. You were right to decline to scope-creep into it here. I think that deserves its own issue and is arguably more urgent than the one this PR closes — I would rather that be tracked than quietly inherited.
Approving. Needs one more independent approval under the two-reviewer policy. Merge should wait for build to report green and for the /var regression on main (#173) so this lands on a base where installer-test means something.
The whole-file negative `assert "optional: false" not in console` forbids the string anywhere in 40-kubestellar-console.yaml, so any future mandatory secret or configmap reference would trip it spuriously. Parse the manifest with yaml.safe_load_all, select the Deployment, and assert the JWT_SECRET env entry's secretKeyRef.optional is True — narrower and stronger, since it checks the actual field on the actual reference. Assisted-by: Claude Opus 4.6 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
castrojo
left a comment
There was a problem hiding this comment.
Re-approving. My earlier approval was dismissed when I pushed a fix to this branch; this restores it and explains the change, since I am not the author.
What I changed (commit bec017a, 6d47b37..bec017a, one file, tests/unit/test_kubestellar_kiosk.py).
The new test carried a whole-file negative assertion:
assert "optional: false" not in console
where console is the entire text of 40-kubestellar-console.yaml — a three-document manifest (Namespace, Service, Deployment). It passed, but it forbade the substring optional: false anywhere in all three documents, which is much broader than the intent. The first time any future secret or configMap reference in that file legitimately needed to be mandatory, it would fire and read as a regression.
Replaced with a parsed assertion on the actual field:
deployment = next(doc for doc in yaml.safe_load_all(...) if doc and doc.get("kind") == "Deployment")
jwt_secret = next(env for env in deployment["spec"]["template"]["spec"]["containers"][0]["env"] if env["name"] == "JWT_SECRET")
assert jwt_secret["valueFrom"]["secretKeyRef"]["optional"] is True
That is both narrower and stronger — it checks the real field on the real reference rather than the absence of a string. Parsing matches the sibling tests/unit/test_k0s_manifests.py, which already uses yaml.safe_load_all on manifest text, so it follows existing convention rather than introducing a new one.
Proof the new assertion is not vacuous, both directions:
- Flipping the JWT_SECRET reference to
optional: falsefails it:assert False is True,tests/unit/test_kubestellar_kiosk.py:110: AssertionError. Manifest restored afterwards. - Adding an unrelated mandatory reference elsewhere in the same Deployment (
CONSOLE_BRANDINGwithconfigMapKeyRef ... optional: false) trips the OLD assertion spuriously but passes the new one. That is the exact failure this change prevents.
Verification. python3 -m pytest tests/unit/test_kubestellar_kiosk.py -q gives 5 passed; with test_k0s_manifests.py too, 12 passed. gh pr diff 90 --name-only confirms the manifest is not in the diff — only the test file changed.
The PR itself is unchanged from my earlier review and still right. The secret is genuinely removed rather than relocated: a repo-wide search for default-jwt-secret returns exactly one hit, the file this PR deletes, and the replacement docs use only placeholders. Both of @hanthor's asks are closed — the optional: true argument now cites upstream console/start.sh at v0.3.34, and the image is digest-pinned.
Still not this PR's problem but worth its own issue, as I noted before: DEV_MODE: "true", ALLOW_DEV_MODE_IN_CLUSTER: "true" and DEV_USER_LOGIN: "admin" remain three lines above, untouched on main. Hardening the JWT signing key is capped in value while anyone reaching the console arrives as admin.
CI: unit and docs pass; build is running. The only expected red is installer-test, which fails on main itself and is unrelated.
Needs one more independent approval under the two-reviewer policy.
There was a problem hiding this comment.
correctness
- low — the manifest comment and PR body cite the wrong fallback mechanism.
files/k0s/manifests/kubestellar/40-kubestellar-console.yaml:65-72(and the PR body) say the image'sstart.shgenerates a random JWT secret whenJWT_SECRETis unset, persisted to$INSTALL_DIR/data/.jwt_secret. But the container's entrypoint isentrypoint.sh(Dockerfile at v0.3.34,ENTRYPOINT ["./entrypoint.sh"]), which never runsstart.sh— that script is the local quick-start installer. The actual fallback is in the Go server:pkg/api/server.go:81-89generates a random secret only whenDEV_MODE=true(vialoadOrCreateDevSecret,pkg/api/startup.go:359-387, persisted underos.UserConfigDir(), notdata/.jwt_secret); in non-dev mode an unsetJWT_SECRETis a fatal exit. The runtime outcome here is as intended because this manifest setsDEV_MODE=true(40-kubestellar-console.yaml:44-45), and the failure mode without it is a safe crash rather than a weak secret — so this is non-blocking. But the comment should be corrected: an operator removingDEV_MODE=truewhile trusting the comment's "start.sh generates a secret" claim will get a crash-looping console, and the cited persistence path is wrong.
Verified clean: the deleted 01-...-github-oauth.yaml Namespace is redundantly created by 40-kubestellar-console.yaml:1-4, so removal orphans nothing; the pinned digest sha256:f27ada… resolves to the v0.3.34 tag on ghcr.io; pyyaml is already installed by .github/workflows/unit-tests.yml, so the new import yaml in tests/unit/test_kubestellar_kiosk.py:5 is fine.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
|
On |
…comment The comment said the image's start.sh generates a random secret when JWT_SECRET is unset and persists it under $INSTALL_DIR/data/.jwt_secret. At the pinned v0.3.34 the image entrypoint is entrypoint.sh, which never runs start.sh (that script is the local quick-start installer). The fallback that actually applies is in the Go server: with DEV_MODE=true it calls loadOrCreateDevSecret and persists .jwt-secret under os.UserConfigDir(); without DEV_MODE an unset JWT_SECRET is a fatal exit. Say so, so an operator who drops DEV_MODE knows to set jwt-secret in the Secret instead of expecting a generated one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VXwvRu29c2MkwftzkgAHcC
|
Pushed @castrojo sorry for the churn: this push will have dismissed your approval again. The runtime behaviour is unchanged from what you reviewed. |
There was a problem hiding this comment.
intent-alignment
- (info) The diff also switches the console image from tag
v0.3.34to a digest pin (files/k0s/manifests/kubestellar/40-kubestellar-console.yaml:43), which the PR body's list of changes doesn't mention. I verified the digest resolves tov0.3.34on ghcr.io (sha256:f27adaf…be32), so it's benign — just note it in the description.
Verified while reviewing: the deleted 01-…-github-oauth.yaml also created the kubestellar-console Namespace, but 40-kubestellar-console.yaml:1-4 creates it too, so nothing breaks; and upstream kubestellar/console@v0.3.34 pkg/api/server.go:81-89 confirms DEV_MODE generates a random JWT secret while non-DEV_MODE fatals on unset JWT_SECRET, matching the PR body.
No findings from: correctness, security, style, docs-currency.
— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.78
|
Closing as superseded by #210. `files/k0s/manifests/kubestellar/` is deleted, and #210 fixes #72 more thoroughly than a manifest edit could:
Thanks for filing this — the in-cluster generator pattern came straight out of your analysis. |
Security Fix
Addresses finding in #72:
files/k0s/manifests/kubestellar/01-kubestellar-console-github-oauth.yamlcommitted dummy GitHub OAuth credentials and a static JWT secret that was automatically seeded into/var/lib/k0s/manifests/and deployed by the k0s manifest deployer on default installations.Changes
files/k0s/manifests/kubestellar/01-kubestellar-console-github-oauth.yamlso no hardcoded secrets or dummy credentials are baked into the sysext or auto-applied by k0s.JWT_SECRETOptional in Console Deployment: Infiles/k0s/manifests/kubestellar/40-kubestellar-console.yaml, setoptional: trueon theJWT_SECRETSecret reference (matching the existingoptional: trueonclient-idandclient-secret). When unset, the console server generates a random secret at startup because the manifest setsDEV_MODE=true(pkg/api/server.go→loadOrCreateDevSecret, persisted underos.UserConfigDir()inside the container). WithoutDEV_MODEan unsetJWT_SECRETis a fatal exit rather than a weak default, so droppingDEV_MODErequires populatingjwt-secretin the Secret.tests/unit/test_k0s_manifests.pyto assert01-kubestellar-console-github-oauth.yamlis not present in manifests, and updatedtests/unit/test_kubestellar_kiosk.pyto verify all Secret references are optional.docs/skills/k0s-sysext-ops.md.Closes #72
— hive: backend=agy