feat(deploy): stamp both dashboard images with their git revision (#3315) - #3386
Merged
Merged
Conversation
) Deploy verification is inference today: compare `docker images` creation time with the merge time, or grep a string out of the shipped binary. `apiary-backend:latest` had `Labels: null`, so every merge since the last rebuild looked identical to the one before it, and docs/ARCANE-GIT-SYNC.md already names the state this produces -- "green, healthy, running the old code". Both images now carry the revision three ways: - backend-service compiles it in (build.rs: GIT_SHA -> APIARY_GIT_SHA) and /healthz returns it as `revision`, so the question becomes a curl rather than a comparison of timestamps. build.rs also emits `cargo:rerun-if-env-changed=GIT_SHA`; without that, cargo reuses a cached build and the second build of an identical tree reports the first revision -- the exact failure this fixes, reproduced by the fix. - frontend-next bakes it into a static /build.json, generated in the build stage before `npm run build` so vite copies it into the output. A static file rather than a route because Nitro serving from public/ is already proven in this image (the container healthcheck reads /static/theme.css the same way). - both Dockerfiles declare ARG GIT_SHA per stage and set org.opencontainers.image.{revision,source,created}. Both tiers normalize to a bare lowercase hex object name or "unknown", and are held to one shared table (backend-service/src/revision-corpus.json) so two normalizers in two languages, in two CI lanes, cannot drift -- the failure mode of their disagreeing is a deploy mismatch that is not one. scripts/verify-deploy.sh compares the live /healthz revision and the image labels against an expected revision, defaulting to origin/main. Its exit codes are the point: 0 agrees, 1 is a finding, and 2 is "could not tell" -- deliberately distinct, so an unreadable body or an image that is not on the host can never read as a stale deploy. diagnostics.yml runs it in --warn-only mode, against $GITHUB_SHA, because the home runner has no clone to measure against (deploy.yml rsyncs into /opt/stacks/apiary with --exclude .git/). Unset stays the honest default: nothing on the homeserver writes GIT_SHA yet, so the first run reports "not stamped", which is true, and the docs say how to turn it on.
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Owner
Author
|
Holding this one per Niklas's instruction to not dispatch on the dashboard
The backend half of #3315 is sound and the same revision stamp is the right Not merged. #3315 stays open. |
Xore
enabled auto-merge (squash)
September 27, 2026 09:16
Both files conflicted because #3385 (liveness/readiness split) and #3315 (revision stamp) landed on the same two places. backend-service/src/main.rs — #3385 replaced the single `Health { ok, es }` struct with `Liveness`/`Readiness` and three handlers, so #3315's `revision` field had nowhere to land. Rather than resurrect the old struct, the field moves onto `Liveness`: /healthz stays an alias of /livez, so the deploy verifier and diagnostics both still read `revision` off the same body, and #3317's property (liveness never touches Elasticsearch) is preserved because git_revision() reads a compile-time env, not the cluster. `built` and `revision` are both kept -- they are not redundant: a timestamp can only be compared, an object name can be looked up, and an image rebuilt from an old commit passes the timestamp check while running month-old code. containers.yml — #3316 added `boot_smoke` to the same two matrix rows #3315 added `stamp` to, and both needed the `labels:` line. Resolution is the union: both rows carry both flags, `labels` keeps #3316's per-row build-row handle, and `build-args` is added alongside it. Local: cargo test 543 passed / 0 failed / 1 pre-existing ignored (544 total, unchanged from main); cargo clippy --all-targets -- -D warnings clean; cargo build --all-targets clean; also green with GIT_SHA set. test_3315_image_revision.py 40 passed; test_3316_image_boot_smoke.py 44 passed. zizmor --offline --min-severity medium reports the same single allowlisted dangerous-triggers advisory as main, and no new findings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
apiary-backend:latestreportsLabels: null,/healthzreturns{ok, es}, and"is the merge on the host?" is answered by comparing
docker imagescreation timewith the merge time.
docs/ARCANE-GIT-SYNC.mdalready names what that misses:green, healthy, running the old code.
This makes the answer a field both tiers report, plus a tool that compares it.
Where the revision is
/healthz{"ok":true,"es":true,"revision":"3dca4457…"}, compiled in bybuild.rs(GIT_SHA→APIARY_GIT_SHA) and also in the boot log line/build.json{"revision":…,"built":…,"source":…}, static, generated intopublic/beforenpm run buildso vite copies itorg.opencontainers.image.{revision,source,created}build.rsemitscargo:rerun-if-env-changed=GIT_SHA. That line is load-bearing:without it, cargo reuses a cached build and the second build of an identical tree
keeps the first revision — the exact failure this issue is about, reproduced by
the fix itself.
ARG GIT_SHAis declared in both stages of each Dockerfile, because Dockerscopes an ARG to its stage: the build stage needs it for
build.rs, the runtimestage again for the label. The frontend's stamp is written before the build that
copies
public/into the output, and is gitignored — a committedbuild.jsonwould be a revision claim about nobody's build.
Normalization
Both tiers reduce the value to a bare lowercase hex object name (optional
sha256:prefix stripped, 7–64 hex chars) orunknown. They are held to onetable,
backend-service/src/revision-corpus.json, read by a cargo test and byscripts/tests/test_3315_image_revision.py— two implementations in two languagesin two CI lanes drift silently, and the failure mode of their disagreeing is a
deploy mismatch that is not one.
Verification
Exit 0 agrees, 1 is a finding (missing / unstamped / malformed / stale
past
--behind-days), 2 is "could not tell". That third code is the point: anunreadable body, a rewired endpoint or an image that is not on this host must
never read as a stale deploy. A short
git rev-parse --shortrevision isaccepted and reported as an abbreviation rather than as a mismatch.
diagnostics.ymlruns it--warn-onlyagainst$GITHUB_SHA: a scheduleddiagnostics red X is the alert, and a stale deploy is not on #2222's list.
What an operator still has to do (deliberately not hidden)
Nothing on the homeserver writes
GIT_SHAyet, and it cannot be computed inplace —
/opt/stacks/apiaryis a working-tree rsync with.git/excluded, andArcane's directory sync has no
.giteither. So the first run reportsnot stamped, which is true. Turning it on is one line per stack
(
echo "GIT_SHA=$(git rev-parse HEAD)" >> …/.env) plus thePOST /projects/{id}/buildthe sync does not do. The docs say so in the sameplace they say why the image will not invent a value.
Gates
scripts/tests/test_3315_image_revision.py(40 tests) — the generator run forreal over the corpus, the Dockerfile/compose/CI wiring asserted as text, and
verify-deploy.shdriven against a throwaway git repo with stubcurl/dockeracross its pass / mismatch / unstamped / malformed / lag / warn-only / exit-2
paths. Plus the existing
cargo test(535), actionlint, hadolint, shellcheck,zizmor, and the doc/compose lint gates.
Note
scripts/tests/test_compose_drift_watch_sweep.pyhas two failures(
…privileged_fallback, both0 != 1) that reproduce on a clean checkout of thebase commit with this branch stashed — pre-existing, unrelated, and left alone
here rather than folded into this PR.