Repository navigation
docs: decide that our container images stay public (ADR 0004) - #18
Conversation
AbdikhaniAbdirashid
left a comment
AbdikhaniAbdirashid
left a comment
There was a problem hiding this comment.
Approving. Thanks for turning this round so fast — and for correcting the premise rather than just writing down what I assumed. I'd taken "GHCR packages start private" as given too; good that you checked instead.
The rule that comes with the decision is the part that makes this worth having: an image may only contain what is already public in the repo. That's the thing that makes "public images" safe rather than merely convenient, and it's now something either of us can state and defend.
One line comment on the verification command — it's the part of an ADR most likely to be run a year from now by whoever inherits this, so I'd like it to be hard to get wrong.
One thing I checked: the package really is public. Anonymous token, no login:
relay-api:a8274c14… -> 200
So the decision matches reality today.
| TOKEN=$(curl -s "https://ghcr.io/token?scope=repository:devinder-dev/relay-api:pull&service=ghcr.io" | jq -r .token) | ||
| curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $TOKEN" \ | ||
| -H "Accept: application/vnd.oci.image.index.v1+json, application/vnd.oci.image.manifest.v1+json" \ | ||
| https://ghcr.io/v2/devinder-dev/relay-api/manifests/<commit sha> |
There was a problem hiding this comment.
This command is right, but it's fragile in two ways I ran into testing it.
It needs a tag that exists, and most commits don't have one. The push trigger is path-filtered to api/**, so #14, #15 and #17 built no image. Running this against today's main:
relay-api:7187e54… (current main) -> 404
relay-api:a8274c14… (#12's merge) -> 200
Someone following this ADR would naturally pick the newest commit, get 404, and conclude the package had gone private.
404 isn't in the legend. The line above says 200 = public, 401 or 403 = private. But I also tried a package that doesn't exist at all, and that's 404 too — so 404 can't tell "no such tag" from "no such package", and neither case is what we're actually asking about.
The tags endpoint answers the real question without having to guess a tag:
TOKEN=$(curl -s "https://ghcr.io/token?scope=repository:devinder-dev/relay-api:pull&service=ghcr.io" | jq -r .token)
curl -s -H "Authorization: Bearer $TOKEN" https://ghcr.io/v2/devinder-dev/relay-api/tags/listPublic answers with the tag list; private answers DENIED. I ran both halves anonymously just now and got:
{"name":"devinder-dev/relay-api","tags":["a8274c145d758708b59fba95011c8b1a1692a0ad"]}Small related thing: keep the two lines joined with &&. I handed the second line to someone on its own and they got invalid token, because $TOKEN wasn't set in their shell.
Step 1.8: every merge to main is deployed, then checked from outside. Rebased onto main now that #12, #14 and #15 are merged. **Merge only after the switch-over below.** ## What changes - **`api.yml` → `pipeline.yml`.** It now deploys too, and the dashboard's jobs will join it later (`needs:` only works inside one workflow file). - **No path filter on `main`.** Every merge is a release: a deploy needs an image tagged with exactly the merged commit, even when only `k8s/app/` or the docs changed. - **New job `deploy-script-test`** on every PR and merge. It runs `infra/server/test-relay-deploy.sh` (relay-deploy's gates and rollback, with fake git and kubectl) and `shellcheck` on the server scripts. The deploy needs it, so a broken deploy script never reaches the server. - **New job `deploy`**, after `api-image` and `deploy-script-test`, only on a merge to main: - **`environment: production`:** the only way to get `DEPLOY_SSH_KEY`, and only on main. - **`permissions: {}`:** no GitHub access at all. - **`timeout-minutes: 30`:** the worst case is the server's lock (15 min, #19) plus a rollout and a rollback. - **Step 1:** writes the key and our server's `known_hosts`, then `ssh … deploy@<vps> "$GITHUB_SHA"` (`StrictHostKeyChecking=yes`, `BatchMode=yes`). The exit code of `relay-deploy` becomes the step's. - **Step 2:** asks `https://api.myrelay.tech/version` until it answers this commit, for up to 36 × 5 s, because the first deploy also waits for a new certificate. - **`timeout-minutes: 10`** on `api-test` and `api-image` (from the #12 review). - **`k8s/hello/` is deleted**, together with its row in `k8s/README.md`. The API's Ingress takes over `api.myrelay.tech`. ## Tested - `actionlint` 1.7.7 (with shellcheck on the `run:` scripts): no findings. - The new job's two steps, run locally: `26 passed, 0 failed`, and `shellcheck` is clean. - The version check, run on its own with `bash -eo pipefail` against a local container: - the right SHA → passes; - an old SHA → keeps asking, then fails; - nothing answering → "no answer" on every try, then fails. - The rebase was rehearsed in a throwaway clone first: the 3 commits replayed onto main without conflicts. ## Already done before merging - #12, #14 and #15 merged. - `deployer.yaml` applied: the deployer can read events, and still can't read Secrets or delete. - `relay-deploy` installed, byte-identical to main. - The image is pullable without credentials (ADR 0004, #18), and the server has already pulled `relay-api:a8274c1`. ## Switch-over (right before merging) 1. Delete the hello app from `default`: the Ingress, Service and Deployment, plus its leftover TLS Secret `api-myrelay-tech-tls`. `api.myrelay.tech` is down for about 1–2 minutes, once. 2. Merge → the pipeline tests, builds, deploys → `/version` answers the commit. ## After the first deploy - **A real rollback:** deploy #15's merge commit `a78934c` by hand. It has no image, so the rollout fails and `relay-deploy` redeploys the last good commit, while a curl loop shows only 200s. - **Zero downtime:** a load loop during a normal deploy (merging #19) should show no failed requests. - Then make `api-test` and `deploy-script-test` required checks.
Abdikhani asked in the review of #12 for the public-image decision to be written down, because we both have to defend it.
docs/decisions/0004-public-container-images.md:.envfiles or keys, ever.imagePullSecret(a long-lived token to rotate and protect, which hides nothing the repo doesn't already show), and Docker Hub (rate limits).docs/CONTEXT.md: the Images row links the decision.Checked on 29 Oct: after #12 merged,
relay-api:<commit>was already pullable without any login (an anonymous pull answered 200, a missing package 403). Our earlier note "GHCR packages start private" turned out to be wrong for a package published from a public repo, so no visibility change was needed. The ADR includes the check command.