Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/guides/hetzner_deployment_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -918,7 +918,7 @@ here, because a second copy of them is how this section came to be wrong.
For when the last pipeline run happened, `tail` the refresh log:

```bash
sudo -u views-deploy tail -5 ~/views-datafactory/logs/refresh.log
sudo -u views-deploy bash -c 'tail -5 ~/views-datafactory/logs/refresh.log'
```

### Log rotation
Expand Down
22 changes: 14 additions & 8 deletions docs/guides/server_quickref.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,12 @@ sudo -u views-deploy bash -c 'source ~/.profile && cd ~/views-datafactory && git
# 3. Sync dependencies
sudo -u views-deploy bash -c 'source ~/.profile && cd ~/views-datafactory && uv sync'

# 4. Verify — all three must name the same version
sudo -u views-deploy cat ~/.views-deploy-tag
# 4. Verify — all three must name the same version.
# Every path goes inside bash -c: a bare ~ is expanded by YOUR shell
# before sudo runs, so it reads your home, not views-deploy's.
sudo -u views-deploy bash -c 'cat ~/.views-deploy-tag'
sudo -u views-deploy bash -c 'cd ~/views-datafactory && git describe --tags --exact-match 2>/dev/null || git log --oneline -1'
sudo -u views-deploy bash -c 'source ~/.profile && cd ~/views-datafactory && uv run python -c "import importlib.metadata as m; print(m.version(\"views-datafactory\"))"'
sudo -u views-deploy bash -c 'ls ~/views-datafactory/.venv/lib/python*/site-packages | grep views_datafactory'
```

A **rollback is a deploy**: same four blocks, previous tag.
Expand Down Expand Up @@ -88,18 +90,22 @@ sudo -u views-deploy bash -c 'source ~/.profile && cd ~/views-datafactory && uv
## Check status

```bash
# Every path goes inside bash -c — see the note in step 4 above.

# Version the server is CONFIGURED to run (intent, not evidence —
# use all three checks in §"Deploy a new version" step 4 to confirm)
sudo -u views-deploy cat ~/.views-deploy-tag
sudo -u views-deploy bash -c 'cat ~/.views-deploy-tag'

# Last run duration
sudo -u views-deploy cat ~/views-datafactory/logs/pipeline_duration.json
sudo -u views-deploy bash -c 'cat ~/views-datafactory/logs/pipeline_duration.json'

# Last failure
sudo -u views-deploy cat ~/views-datafactory/logs/pipeline_failure.json 2>/dev/null || echo "No recent failure"
# Last failure. The fallback is INSIDE bash -c on purpose: outside it, an
# unreadable path makes `cat` fail and the || branch print "No recent
# failure" — a permission error reported as good news.
sudo -u views-deploy bash -c 'cat ~/views-datafactory/logs/pipeline_failure.json 2>/dev/null || echo "No recent failure"'

# Raw data size
sudo -u views-deploy du -sh ~/views-datafactory/data/raw/*/
sudo -u views-deploy bash -c 'du -sh ~/views-datafactory/data/raw/*/'

# Zarr metadata (from laptop)
curl -s http://204.168.219.108/grid.zarr/.zattrs | python3 -m json.tool
Expand Down
21 changes: 21 additions & 0 deletions reports/technical_risk_register.md
Original file line number Diff line number Diff line change
Expand Up @@ -3253,6 +3253,27 @@ Bash buffers the script and does not re-read it; git writes in place (inode unch

All four corrected, each pointing at `server_quickref.md` as the single operational home with "if they disagree, the quickref wins", and `server_quickref.md` grew the verification block it was missing (step 4: tag file, checked-out tag, **installed distribution version**). ADR-022 carries a dated amendment in the ADR-027 style rather than a rewrite — the Decision it records is sound; what was wrong was the inference that the gate's own `git checkout` makes the tag file sufficient.

**Update 2026-10-16 — the corrected procedure was run, and the verification step did not work.** v1.14.0 deployed cleanly: `git checkout` moved HEAD from `10c4d24` (v1.13.0) to `2b22a64` (v1.14.0), and `uv sync` reported `- views-datafactory==1.13.0` / `+ views-datafactory==1.14.0` — direct confirmation, on the box, that step 3 is load-bearing and that the tag file alone would have left 1.13.0 installed.

Then step 4's first command failed:

```
$ sudo -u views-deploy cat ~/.views-deploy-tag
cat: /home/simmaa_prio/.views-deploy-tag: Permission denied
```

**`~` is expanded by the calling shell, before `sudo` runs.** Every such command reads the *operator's* home, not `views-deploy`'s. Six were live in the guides, two of them added by the 2026-10-13 correction above. All now wrapped in `bash -c '...'`, where the inner shell expands `~` as the target user.

One of the six failed green rather than loud, which is why this is recorded here rather than left as a typo:

```
sudo -u views-deploy cat ~/.../pipeline_failure.json 2>/dev/null || echo "No recent failure"
```

The `||` sat **outside** `bash -c`, so a permission error on the wrong home directory printed **"No recent failure"** — an unreadable path reported as good news, in the command an operator runs to ask whether the last pipeline run broke. Belongs to the **mechanisms that fail green** cluster alongside the rest of this entry. The fallback is now inside the quoted command.

**The shape worth noticing.** C-343 is now on its third round of correction, and each round fixed the written procedure while the *instrument that would have caught it* stayed unbuilt. A documented verification step that nobody has executed is not evidence, and this one had been in the quickref §"Check status" since before the entry existed.

**What this update changes about the entry's severity, honestly.** It does not raise the tier, but it does weaken the "corrected in the same PR as this entry" claim above: the correction was partial, and the stale copies outlived it by two months and one release. The pattern is #402's — *one procedure, one home* — and C-343 is an instance of it as much as it is a deployment risk. The entry stays **open**, and the still-unbuilt instrument below is now the only thing that would catch a sixth copy appearing.

**The instrument, not yet built.** A pre-flight check in `refresh_pipeline.sh` comparing `$DEPLOY_TAG` against the HEAD the script *started* from, and refusing to run when they disagree — fail-loud per ADR-011, rather than silently self-correcting one run late. Proposed for #424; deliberately not built here, because this PR is a register entry and a comment fix.
Expand Down
Loading