Repository navigation
feat: keep secrets out of the repository and refuse weak ones in production - #63
Merged
Merged
Conversation
The local overlay's development secrets (a JWT key, the Postgres password) were committed in overlays/local/secrets.env. The repository is public: anyone could read that JWT key and, had it reached production, mint valid tokens for any player. The file is gone. The overlay now reads overlays/local/.env.local, which .gitignore already covers (.env.*), and make k8s-secrets (scripts/k8s/local-secrets.sh, run by make k8s-apply) writes each missing key with openssl rand -hex 32, mode 600, never printing a value; existing keys are kept. Without the file the render fails instead of using anything stale. The manifests CI job generates throwaway values before rendering. Measured: render without the file fails; the script writes 64-character values and a second run changes nothing; a fresh checkout renders both overlays (34/34, 25/25). On k3d the new Postgres password against the existing volume failed authentication (SQLSTATE 28P01) in the new pods while the old ones kept serving and the rollout stalled; documented, and after recreating the volume 19/19 pods are healthy, register 201, login returns a token. The old development values stay in git history; they were never used outside a local cluster and are now replaced. Plan: A9 Refs: Bulgu #39 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Redis had no password and the cluster has no NetworkPolicy, so any pod could read and change every session, the matchmaking queue and the leaderboards. From the gateway pod with no credentials: PING +PONG, DBSIZE answered. Redis now starts with --requirepass from gamemesh-secrets (REDIS_PASSWORD); the probes authenticate through REDISCLI_AUTH, and the services already send the password (REDIS_PASSWORD via envFrom). The local secrets script generates the key; the production Secret has to provide it. Measured on k3d: from the gateway pod, commands without credentials get "-NOAUTH Authentication required", a wrong password "-WRONGPASS"; register 201, the Redis session check (GET /players/me) 200, joining the Redis queue 200; 19/19 pods healthy with no restarts. The password is not visible in the process list: Redis rewrites its title to "redis-server *:6379" (checked without printing the value). Plan: A9 Refs: Bulgu #56 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The monitoring component set GF_SECURITY_ADMIN_PASSWORD to "admin" in plain text; on k3d admin/admin returned 200 from /api/user. The password now comes from gamemesh-secrets (GRAFANA_ADMIN_PASSWORD), which the local secrets script generates like the others. After the change on k3d: admin/admin 401, admin with the generated password 200 (checked without printing it); 19/19 pods healthy. Monitoring stays out of production; when it moves there, the production Secret needs this key as well. Plan: A9 Refs: Bulgu #67 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JWT_SECRET defaults to insecure-dev-secret-do-not-use-in-prod and
nothing checked it: a production deployment missing its Secret started
silently with a key that is public in this repository, and anyone
could mint a valid token for any player.
Config.Validate(needs...) checks, when APP_ENV is production, each
secret the service declares it uses (NeedJWT, NeedPostgres for the
password in POSTGRES_DSN, NeedRedis): unset, shorter than 32
characters, or one of the two public development defaults long enough
to pass that check (the code's JWT default and .env.example's). Every
main declares its own needs, so leaderboard is not failed for a DSN
default it never uses. Errors are joined and name the setting, never
the value; outside production the defaults are accepted
(docker-compose runs development).
Tests: defaults pass outside production; in production the JWT
default, an empty JWT (falls back to the default), a short JWT, an
unset Redis password and a short Postgres password are refused with
the setting's name and without the value; only declared needs are
checked; strong values pass; both DSN forms are parsed.
Measured with the gateway image: in production with the defaults it
exits with both problems ("JWT_SECRET is a known development default",
"REDIS_PASSWORD is not set"), with a short JWT it exits, with
openssl-generated values it serves. On k3d the local secrets are
generated, so all 19 pods start; a gateway deploy with a weak
JWT_SECRET left the new pod in CrashLoopBackOff, the two old pods
serving (/health 200) and the rollout stalled until rollout undo.
Plan: A9
Refs: Bulgu #40
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The production overlay and the "Production secrets" commands still listed JWT_SECRET and the Postgres keys only. Redis now requires REDIS_PASSWORD, and the services refuse to start in production without it, so a Secret made from the old list would have stalled the first rollout. Both now list JWT_SECRET, POSTGRES_PASSWORD and REDIS_PASSWORD (openssl rand -hex 32 each) plus POSTGRES_USER and POSTGRES_DB, and say that a missing or weak value stalls the rollout rather than running. git grep -i jwt_secret deployments/ now finds only that comment; no tracked file under deployments/ assigns a value to any secret. Plan: A9 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Records the A9 decisions and what each change measured on k3d and with the gateway image: local secrets generated outside git, Redis requiring a password, Grafana's password from the Secret, the production fail fast with a stalled rollout and an undo, and the production Secret's keys. Marks #39, #40, #56 and #67 solved. Plan: A9 Refs: Bulgu #39, Bulgu #40, Bulgu #56, Bulgu #67 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gitleaks failed #63 with one finding and no location: the job ran without --verbose. Run locally the same way with --verbose (values REDACTED): generic-api-key on pkg/config/config_test.go:103, strongSecret, a test fixture ("0123456789abcdef" four times) that the rule flags for its name and entropy. Not a secret. gitleaks scans every commit, so changing the file does not clear a finding in pushed history, and rewriting that history is not an option. The test now builds the value with strings.Repeat, so no literal is left, and .gitleaksignore accepts that one finding by its fingerprint (commit:file:rule:line), with the reason. The fingerprint holds as long as the commit keeps its SHA: merge main into this branch rather than rebase it. The CI job adds --verbose, so a red run names the file, line, rule and commit while the value stays redacted. Locally: the history scan (187 commits) finds no leaks; the current pkg/config has none either. actionlint clean. Plan: A9 Refs: Bulgu #41 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A9. Every change measured on k3d (k3s v1.35.5) and with the gateway image.
overlays/local/secrets.envis gone; the overlay reads the gitignored.env.local, written bymake k8s-secrets(scripts/k8s/local-secrets.sh: each missing key fromopenssl rand -hex 32, mode 600, values never printed, existing keys kept). Without the file the render fails; the CImanifestsjob generates throwaway values. On k3d a new Postgres password against the existing volume failed authentication (SQLSTATE 28P01) in the new pods while the old ones kept serving; documented.PING +PONG,DBSIZEanswered; after-NOAUTH, wrong password-WRONGPASS. Register, the Redis session check and the queue work. The password is not in the process list (Redis rewrites its title).admin/admin200 → 401.Config.Validate(NeedJWT, NeedPostgres, NeedRedis), each service declaring what it uses; withAPP_ENV=productiona secret that is unset, shorter than 32 characters or a known development default is refused by name, never by value. Gateway image: defaults → exits reporting both problems; strong values → serves. k3d: a gateway deploy with a weakJWT_SECRETleft the new pod in CrashLoopBackOff, the old pods serving (/health200) and the rollout stalled untilrollout undo.JWT_SECRET,POSTGRES_PASSWORD,REDIS_PASSWORD(openssl rand -hex 32each),POSTGRES_USER,POSTGRES_DB. Creating it in the cluster happens with the server (B/D); values never enter the repository or logs.Done:
git grep -i jwt_secret deployments/finds only the comment naming the key; no tracked file underdeployments/assigns a value to a secret.Plan: A9 · Refs: Bulgu #39, #40, #56, #67
🤖 Generated with Claude Code